What Broke & Why: Practical Lessons from Reinforcement Learning Post-Training
cs.AI
Submitted: 2026-11-07
Updated: 2026-11-07
Terminology
Sources
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Proximal Policy Optimization Algorithms
- Understanding R1-Zero-Like Training: A Critical Perspective
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs
- Group Sequence Policy Optimization
- GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization
- S-GRPO: Early Exit via Reinforcement Learning in Reasoning Models
- MC-GRPO: Median-Centered Group Relative Policy Optimization for Small-Rollout Reinforcement Learning
- DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning
- Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
- The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
- Process Reinforcement through Implicit Rewards
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- Training language models to follow instructions with human feedback
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
- On a few pitfalls in KL divergence gradient estimation for RL
- HybridFlow: A Flexible and Efficient RLHF Framework
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- SGLang: Efficient Execution of Structured Language Model Programs
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents