Proxy OPD: On-Policy Distillation with Transferable Relative Proxy Update for Large Language Models
cs.LG, cs.AI
Submitted: 2026-07-13
Updated: 2026-11-16
Code: https://github.com/KnowledgeXLab/P-OPD
Terminology
Sources
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- A Survey of On-Policy Distillation for Large Language Models
- MiMo-V2-Flash Technical Report
- Qwen3 Technical Report
- Fine-Tuning Language Models from Human Preferences
- Nemotron-4 340B Technical Report
- HybridFlow: A Flexible and Efficient RLHF Framework
- GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning
- Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling
- Constitutional AI: Harmlessness from AI Feedback
- Distilling the Knowledge in a Neural Network
- Distillation Scaling Laws
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
- Black-Box On-Policy Distillation of Large Language Models
- DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning
- Process Reinforcement through Implicit Rewards
- Skywork Open Reasoner 1 Technical Report
- Evaluating Large Language Models Trained on Code
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis