Denser not equal to Better: Limits of On-Policy Self-Distillation for Continual Post-Training
The paper investigates whether on-policy self-distillation (SDPO) can stabilize continual learning by using teacher signals to guide model updates. While SDPO can speed up in-domain specialization when teacher signals are stable, it struggles with out-of-distribution data and exhibits stronger forgetting than sequence-level methods like GRPO. The key takeaway is that dense supervision alone does not guarantee better results.