Validity-Calibrated Reasoning Distillation
cs.LG, cs.AI
Submitted: 2026-04-14
Updated: 2026-10-29
Code: https://github.com/QwenLM/Qwen2.5-Math
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- GPT-4 Technical Report
- Program Synthesis with Large Language Models
- Evaluating Large Language Models Trained on Code
- Training Verifiers to Solve Math Word Problems
- The Llama 3 Herd of Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Distilling the Knowledge in a Neural Network
- Qwen2.5-Coder Technical Report
- Explanations from Large Language Models Make Small Reasoners Better
- DeepSeek-V3 Technical Report
- Orca-Math: Unlocking the potential of SLMs in Grade School Math
- Orca: Progressive Learning from Complex Explanation Traces of GPT-4
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- CMATH: Can Your Language Model Pass Chinese Elementary School Math Test?
- Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
- ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates
- Evaluating the Performance of Large Language Models on GAOKAO Benchmark
Related papers
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis
- (Mis)generalization of Helpful-Only Fine-tuning