New model release improves reasoning performance
The story so far
Order-Only On-Policy Distillation (O^{3}PD) is released, training student models to match the token ordering of the teacher model rather than probabilities, averaging 2.9 points better than OPD on HMMT25, AIME24, and AIME25, with a 5.6-point improvement on AIME25. This technology does not require temperature tuning and is not affected by the teacher model's temperature.
AI-written from the reports · Updated 15 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Hugging Face Daily PapersOrder-Only On-Policy Distillation
Order-Only On-Policy Distillation (O^{3}PD) trains student models to match the token ordering of the teacher model rather than probabilities, averaging 2.9 points better than OPD on HMMT25, AIME24, and AIME25, with a 5.6 point improvement on AIME25. This technique requires no temperature tuning and is unaffected by the teacher model's temperature.