Privileged Information Does Not Guarantee Privileged Guidance: Auditing On-Policy Self-Distillation for Reasoning Models
cs.CL
Submitted: 2026-11-11
Updated: 2026-11-11
Terminology
Sources
- On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
- OpenThoughts: Data Recipes for Reasoning Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Privileged, but Biased: How PI-Conditioned Teachers Break Self-Distillation
- Learn from Whoever Is Right: Answer-Verified Multi-Teacher Distillation for Multi-Domain LLMs
- Large Language Models Cannot Self-Correct Reasoning Yet
- Reinforcement Learning via Self-Distillation
- Privileged Solutions or Context-Induced Teacher Behavior? Dissecting On-Policy Self-Distillation
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs
- Rethinking On-Policy Self-Distillation for Thinking Models
- Hindsight Compacts but Does Not Repair: Rethinking On-Policy Self-Distillation in Reasoning Models
- Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?
- On-Policy Self-Distillation without Any Supervision
- PHF: Privileged Hidden Flow for On-Policy Self-Distillation
- Unifying distillation and privileged information
- Privileged Likelihood Is Not Automatically Value: Three Checks for Token Credit in On-Policy Self-Distillation
- Privileged Information Distillation for Language Models
- Diagnosing and Mitigating Thinking Collapse in On-Policy Self-Distillation
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data