Train Ahead, Distill Back: Bootstrapping On-Policy Self-Distillation for Large Language Models
cs.AI
Submitted: 2026-09-29
Updated: 2026-09-29
Terminology
Sources
- Reducing the Safety Tax in LLM Safety Alignment with On-Policy Self-Distillation
- Consensus as Privileged Context for Label-Free Self-Distillation
- Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation
- UniSD: Towards a Unified Self-Distillation Framework for Large Language Models
- Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing
- RISE: Recursive Improvement via Self-Extrapolating Policy Distillation
- On-Policy Self-Distillation without Any Supervision
- Distilling LLM Feedback for Lean Theorem Proving
- Future Policy Approximation for Offline Reinforcement Learning Improves Mathematical Reasoning
- Privileged Information Distillation for Language Models
- Learning from the Near Future: Temporal Self-Distillation for RLVR
- Purified OPSD: On-Policy Self-Distillation Without Losing How to Think
- GATES: Self-Distillation under Privileged Context with Consensus Gating
- PBSD: Privileged Bayesian Self-Distillation for Long-Horizon Credit Assignment
- CriPO: Enhancing Rubric-based RL via Self-Distillation
- Qwen3 Technical Report
- Self-Distilled RLVR
- Why Does Feedback-Augmented Self-Distillation Fail to Improve Retrieval-Interleaved Search Agents?
- Adaptive Supervised Anchoring for On-Policy Self-Distillation
- OGLS-SD: On-Policy Self-Distillation with Outcome-Guided Logit Steering for LLM Reasoning
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems