OPSD has deviations
The story so far
Baseten Labs published a research report indicating that on-policy self-distillation (OPSD) has biases and limited effectiveness. This technique trains student models using outputs generated with context prompts by the student models themselves, but the study shows that this intensive supervision may introduce biases, especially for trained models, which perform worse.
AI-written from the reports · Updated 12 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Baseten Labs01The bias of on-policy self-distillationResearch·NOV 2026
Research indicates that on-policy self-distillation (OPSD) may still be valuable in specific scenarios, but more research resources should not be invested. OPSD trains through the outputs generated by the student model itself with context prompts, but research shows that this intensive supervision may introduce bias, especially for trained models, making it less effective.