Privileged Context as Drift in On-Policy Self-Distillation
This study investigates how different types of privileged context—such as demonstrations, feedback, or rephrases—affect policy drift during on-policy self-distillation (OPSD). By varying both content and source generation methods, the researchers found that content choices cause a wider range of policy drift than source choices. This suggests privileged context design is crucial for stabilizing continual learning processes.