Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy Distillation
cs.LG
Submitted: 2026-06-11
Updated: 2026-11-05
Code: https://github.com/SydCS/OPD-Param-Analysis
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Qwen2.5-VL Technical Report
- On Predictability of Reinforcement Learning Dynamics for Large Language Models
- Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
- OpenThoughts: Data Recipes for Reasoning Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- JustRL: Scaling a 1.5B LLM with a Simple RL Recipe
- Distilling the Knowledge in a Neural Network
- LoRA: Low-Rank Adaptation of Large Language Models
- Reinforcement Learning via Self-Distillation
- Editing Models with Task Arithmetic
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- Muon is Scalable for LLM Training
- MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- RL's Razor: Why Online Reinforcement Learning Forgets Less
- A Survey of On-Policy Distillation for Large Language Models
- MiMo-V2-Flash Technical Report
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis