Bootstrapped On-Policy Distillation
cs.LG, cs.AI
Submitted: 2026-11-10
Updated: 2026-11-10
Code: https://github.com/YueWu0301/BOPD
Terminology
Sources
- DualOPSD: Closing the Feedback Loop in On-Policy Self-Distillation
- OpenThoughts: Data Recipes for Reasoning Models
- Measuring Massive Multitask Language Understanding
- LoRA: Low-Rank Adaptation of Large Language Models
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- Rethinking On-Policy Self-Distillation for Thinking Models
- TACO: Topics in Algorithmic COde generation dataset
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation
- ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection
- One Student, Many Teachers: Multi-Task On-Policy Distillation via Soft-Prompt Privileged Context
- Self-Distillation Enables Continual Learning
- PriFT: Guiding Supervised Fine-Tuning with Pretrained Priors
- On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification
- $\beta$-OPSD: Deriving with Policy Optimization, Training with Self-Distillation
- Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Sampling
- Qwen3 Technical Report
- DOPD: Dual On-policy Distillation
- Latent On-Policy Self-Distillation
- Towards On-Policy SFT: Distribution Discriminant Theory and its Applications in LLM Training
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis