What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents?
cs.LG, cs.CL
Submitted: 2026-11-04
Updated: 2026-11-04
Project page: https://rethinking-agentic-rl.github.io
Terminology
Sources
- SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
- The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
- ReTool: Reinforcement Learning for Strategic Tool Use in LLMs
- Group-in-Group Policy Optimization for LLM Agent Training
- Rethinking On-Policy Distillation of Large Language Models II: One Training Example
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Why Multi-Step Tool-Use Reinforcement Learning Collapses and How Supervisory Signals Fix It
- JustRL: Scaling a 1.5B LLM with a Simple RL Recipe
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
- Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application
- Network transport with nonlinear dynamics at the nodes
- Evaluating Test-Time Scaling of General LLM Agents
- ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
- The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation
- WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning
- Proximal Policy Optimization Algorithms
- HybridFlow: A Flexible and Efficient RLHF Framework
- ZeroSearch: Incentivize the Search Capability of LLMs without Searching
- Hindsight Credit Assignment for Long-Horizon LLM Agents
- TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis