tau-i18n extension evaluates languages
The story so far
OpenAI has released tau-i18n, expanding tau-Voice to five languages: Spanish, Brazilian Portuguese, Hindi, Korean, and Mandarin, for cross-language performance evaluation in aviation, retail, and telecommunications tasks. The evaluation is based on task completion, interaction effectiveness, and generated quality, with results showing Korean performing weakest, GPT Live 1 highest in task completion and generated quality, Grok Voice Think Fast 2.0 best in interaction but low in generated quality. Users can build representative callers, review voices, and sample dialogues to define agent evaluation criteria. If a language or region is missing, OpenAI calls on contributors to help expand tau-i18n.
AI-written from the reports · Updated 27 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Sierra Blog𝜏-i18n: Evaluation across languages and beyond task completion
𝜏-i18n expands 𝜏-Voice to five languages: Spanish, Brazilian Portuguese, Hindi, Korean, and Mandarin, evaluating cross-language performance using aviation, retail, and telecommunications tasks. The evaluation includes three dimensions: task completion, interaction effectiveness, and generation quality. The results show that Korean performed the weakest, GPT Live 1 had the highest task completion and generation quality, and Grok Voice Think Fast 2.0 had the best interaction but low generation quality.