The Illusion of Diversity: Aligning LLM Exploration via Effective Entropy
cs.LG
Submitted: 2026-11-11
Updated: 2026-11-11
Code: https://github.com/huggingface/math-verify
Terminology
Sources
- MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
- Reasoning with Exploration: An Entropy Perspective
- The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
- Maximum Entropy RL (Provably) Solves Some Robust RL Problems
- How to Allocate, How to Learn? Dynamic Rollout Allocation and Advantage Modulation for Policy Optimization
- From $\log \pi$ to $\pi$: Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weight
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
- Skywork Open Reasoner 1 Technical Report
- Measuring Mathematical Problem Solving With the MATH Dataset
- Let's Verify Step by Step
- Heterogeneous Adaptive Policy Optimization: Tailoring Optimization to Every Token's Nature
- Modern hierarchical, agglomerative clustering algorithms
- Combining policy gradient and Q-learning
- Supervised Fine Tuning on Curated Data is Reinforcement Learning (and can be improved)
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving Clipping Policy Optimization
- CE-GPPO: Coordinating Entropy via Gradient-Preserving Clipping Policy Optimization in Reinforcement Learning
- GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropy
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis