Is the Importance Ratio Necessary for Stable Reinforcement Learning in LLMs?
cs.LG, cs.AI, cs.CL
Submitted: 2026-11-11
Updated: 2026-11-11
Terminology
Sources
- MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
- The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
- Soft Adaptive Policy Optimization
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- On-Policy RL with Optimal Reward Baseline
- Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning
- OpenAI o1 System Card
- CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Models
- ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
- Understanding R1-Zero-Like Training: A Critical Perspective
- Rethinking the Trust Region in LLM Reinforcement Learning
- Qwen2.5 Technical Report
- LLMs Can Learn to Reason Via Off-Policy RL
- Tapered Off-Policy REINFORCE: Stable and efficient reinforcement learning for LLMs
- High-Dimensional Continuous Control Using Generalized Advantage Estimation
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning
- Rethinking Groups in Critic-Free RLVR
- BNPO: Beta Normalization Policy Optimization
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis