Skip to content
Live
Sierra Blogยท Soham Rayยทยท 24 days ago75

๐œ-i18n: Evaluation across languages and beyond task completion

๐œ-i18n: Evaluation across languages and beyond task completion

Brief overview

๐œ-i18n expands ๐œ-Voice to five languages: Spanish, Brazilian Portuguese, Hindi, Korean, and Mandarin, evaluating cross-language performance using aviation, retail, and telecommunications tasks. The evaluation includes three dimensions: task completion, interaction effectiveness, and generation quality. The results show that Korean performed the weakest, GPT Live 1 had the highest task completion and generation quality, and Grok Voice Think Fast 2.0 had the best interaction but low generation quality.

Source: Sierra Blog ยท sierra.ai