AWS fine-tunes GPT-OSS-20B to improve multilingual reasoning
The story so far
AWS has fine-tuned the open-source inference model GPT-OSS-20B on Amazon SageMaker HyperPod to enhance its reasoning capabilities in French, German, Spanish, and Italian. GPT-OSS-20B thinks in English by default, and only 22% meet the target language requirements in multi-language reasoning tests. Through supervised fine-tuning (SFT) and reinforcement learning (GRPO), the team has improved the model's performance. SFT is responsible for teaching main behaviors, and GRPO optimizes when there is room for improvement. AWS further uses multi-round reinforcement learning (MTRL) to train the model on Amazon SageMaker AI to perform sequential tasks, enhancing GPT-OSS-20B's Wordle game solving rate from 61% to 84% (default reasoning strength), with a cost of about 339 US dollars.
AI-written from the reports · Updated 26 days ago
Developments
2 developments
- 24 Oct, 01:48 · 1 reportsMulti-Turn Reinforcement Learning on Amazon SageMaker AIAWS Machine Learning Blog:Multi-Turn Reinforcement Learning on Amazon SageMaker AI: A hands-on guide
- 21 Oct, 01:48 · 1 reportsFine-tuning GPT-OSS-20B for multilingual reasoningAWS Machine Learning Blog:Fine-tuning GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod
Coverage timeline
Follow the coverage to see every side of the story.
- AWS Machine Learning BlogMulti-Turn Reinforcement Learning on Amazon SageMaker AI: A hands-on guide
Amazon SageMaker AI trains models to perform sequential tasks using Multi-Turn Reinforcement Learning (MTRL) without requiring correct trajectories, only defining result scoring. Taking the Wordle game as an example, it increases the solving rate of the GPT-OSS-20B model from 61% to 84% (default inference strength), with a cost of about 339 USD.
- AWS Machine Learning BlogFine-tuning GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod
GPT-OSS-20B is an open-source inference model that thinks in English by default, and only 22% met the target language requirements in multilingual inference tests. By using supervised fine-tuning (SFT) and reinforcement learning (GRPO) on Amazon SageMaker HyperPod, the team can improve the model's inference capabilities in French, German, Spanish, and Italian, where SFT is responsible for primary behavior teaching and GRPO optimizes when there is room for improvement.