Reward-Aligned Distillation
cs.LG, cs.CL
Submitted: 2026-10-10
Updated: 2026-10-10
Code: https://github.com/huggingface/Math-Verify
Terminology
Sources
- Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
- Process Reinforcement through Implicit Rewards
- Adapting Auxiliary Losses Using Gradient Similarity
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning
- Measuring Coding Challenge Competence With APPS
- On-Policy Delta Distillation
- Distilling the Knowledge in a Neural Network
- Reinforcement Learning via Self-Distillation
- OpenAI o1 System Card
- R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents
- TACO: Topics in Algorithmic COde generation dataset
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Self-Distillation Enables Continual Learning
- Finding the Evidence: Discovering Decision-Supporting Tokens for On-Policy Reasoning Distillation
- KDRL: Post-Training Reasoning LLMs via Unified Knowledge Distillation and Reinforcement Learning
- TIP: Token Importance in On-Policy Distillation
- Qwen3 Technical Report
- Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks