Function-Space Temporal Self-Distillation for Policy Optimization
Function-Space Temporal Self-Distillation for Policy Optimization
The Function-Space Policy Optimization (FSPO) framework significantly improves the SDPO method by storing sparse historical next-word distributions as fixed time targets, combining privileged adaptation with function-space time integration, achieving time separation without maintaining a persistent teacher model.
The paper shows a significant improvement over the recent self-distillation policy methods, SDPO.
This work introduces Function-Space Policy Optimization (FSPO), a single-policy framework that combines current privileged adaptation with function-space temporal consolidation. FSPO stores sparse historical next-token distributions as fixed temporal targets, providing temporal separation without maintaining a persistent teacher model.
Please feel free to discuss the paper if you find it interesting.
Source: Hugging Face Daily Papers · huggingface.co