Reward Budgeting Reduces Premature Convergence in Reinforcement Learning for LLM Reasoning
cs.LG
Submitted: 2026-11-04
Updated: 2026-11-06
Code: https://github.com/Qwen-Applications/RewardBudget
Terminology
Sources
- The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
- On-Policy RL with Optimal Reward Baseline
- Skywork Open Reasoner 1 Technical Report
- Rethinking Entropy Regularization in Large Reasoning Models
- Revisiting LLM Reasoning via Information Bottleneck
- CURE: Critical-Token-Guided Re-Concatenation for Entropy-Collapse Prevention
- MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
- Clip-Low Increases Entropy and Clip-High Decreases Entropy in Reinforcement Learning of Large Language Models
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- SCOPE-RL: Stable and Quantitative Control of Policy Entropy in RL Post-Training
- EPO: Entropy-regularized Policy Optimization for LLM Agents Reinforcement Learning
- Qwen3 Technical Report
- EntroPIC: Towards Stable Long-Term Training of LLMs via Entropy Stabilization with Proportional-Integral Control
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- No Free Lunch: Rethinking Internal Feedback for LLM Reasoning
- Group Sequence Policy Optimization
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis