Not Every Token Is Worth Distilling: Selective Supervision for Direct-OPD
cs.LG, cs.AI
Submitted: 2026-09-24
Updated: 2026-09-26
Comments: 19 pages. Yibo Zhao and Zixuan Yang are equal contributors and may list their names in either order on their CVs
Code: https://github.com/THUDM/slime
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Weak-to-Strong Generalization via Direct On-Policy Distillation
- Proxy OPD: On-Policy Distillation with Transferable Relative Proxy Update
- Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
- Rethinking On-Policy Distillation of Large Language Models II: One Training Example
- GLM-5: from Vibe Coding to Agentic Engineering
- JustRL: Scaling a 1.5B LLM with a Simple RL Recipe
- Skywork Open Reasoner 1 Technical Report
- On-Policy Delta Distillation
- Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
- Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level
- Scaling Laws for Neural Language Models
- Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
- MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
- RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillation
- Rethinking Selective Knowledge Distillation
- Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillation
- LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Training
- Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks