TSR: Trajectory-Search Rollouts for Multi-Turn RL of LLM Agents
cs.AI, cs.CL, cs.LG
Submitted: 2026-02-12
Updated: 2026-09-27
Journal ref: ICLR 2026 Workshop on Lifelong Agents: Learning, Aligning, Evolving
Code: https://github.com/THUDM/slime
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Tasks
- RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning
- Agent Lightning: Train ANY AI Agents with Reinforcement Learning
- AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning
- SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agent
- Process Reinforcement through Implicit Rewards
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning
- RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment
- Thinking Fast and Slow with Deep Learning and Tree Search
- Qwen2.5 Technical Report
- Imagination-Augmented Agents for Deep Reinforcement Learning
- OpenAI Gym
- High-Dimensional Continuous Control Using Generalized Advantage Estimation
- Revisiting Group Relative Policy Optimization: Insights into On-Policy and Off-Policy Training
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models
- ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
- AgentGym: Evolving Large Language Model-based Agents across Diverse Environments
- Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems