On the Generalization Limitation of On-Policy Distillation for Post-RL Consolidation
cs.LG
Submitted: 2026-11-03
Updated: 2026-11-03
Project page: https://yichenzw.com/opd-generalization.github.io
Terminology
Sources
- On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
- Git Re-Basin: Merging Models modulo Permutation Symmetries
- Exploring Length Generalization in Large Language Models
- Program Synthesis with Large Language Models
- LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks
- Evaluating Large Language Models Trained on Code
- Training Verifiers to Solve Math Word Problems
- MiMo-V2-Flash Technical Report
- CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVR
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Models
- Reinforcement Learning for Compositional Generalization with Outcome-Level Optimization
- GLM-5: from Vibe Coding to Agentic Engineering
- Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks
- Distilling the Knowledge in a Neural Network
- Breaking Barriers: Do Reinforcement Post Training Gains Transfer To Unseen Domains?
- Editing Models with Task Arithmetic
- Trajectory-Refined Distillation
- Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in LLM Reasoning
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis