Research shows that on-policy self-distillation (OPSD) is biased and has limited effectiveness
Research indicates that on-policy self-distillation (OPSD) may still be valuable in specific scenarios, but more research resources should not be invested. OPSD trains through the outputs generated by the student model itself with context prompts, but research shows that this intensive supervision may introduce bias, especially for trained models, making it less effective.



















