Domain-Routed Delta Optimization for Multi-Teacher On-Policy Distillation
cs.LG, cs.CL
Submitted: 2026-11-09
Updated: 2026-11-12
Terminology
Sources
- Poly-OPD: Heterogeneous Multi-Teacher On-Policy Distillation for Capability-Selectable Flow Models
- Distilling the Knowledge in a Neural Network
- Editing Models with Task Arithmetic
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- No Pain, More Gain: Iterative Merging for Effective Multi-Teacher On-Policy Distillation
- Decoupled Weight Decay Regularization
- MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training
- Generalizing Verifiable Instruction Following
- Proximal Policy Optimization Algorithms
- Composing What Each Teacher Learned: Delta Distillation for Multi-Teacher On-Policy Learning
- AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning
- Gradient Surgery for Multi-Task Learning
- Instruction-Following Evaluation for Large Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis