Skip to content
Live
Trending storyPast story

OPSD has deviations

1 reports1 outlets13 days agoUpdate

The story so far

AI overview

Baseten Labs published a research report indicating that on-policy self-distillation (OPSD) has biases and limited effectiveness. This technique trains student models using outputs generated with context prompts by the student models themselves, but the study shows that this intensive supervision may introduce biases, especially for trained models, which perform worse.

AI-written from the reports · Updated 12 days ago

Coverage timeline

Follow the coverage to see every side of the story.

6 Nov
  1. Baseten Labs
    01The bias of on-policy self-distillationResearch·NOV 2026

    Research indicates that on-policy self-distillation (OPSD) may still be valuable in specific scenarios, but more research resources should not be invested. OPSD trains through the outputs generated by the student model itself with context prompts, but research shows that this intensive supervision may introduce bias, especially for trained models, making it less effective.