VETTA: Coordinating Turn- and Token-Level Credit Assignment for Multi-Turn LLM Agents

arXiv:2610.08402 · cs.LG · Submitted 2026-10-06 · Read on arXiv

cs.LG

Submitted: 2026-10-06

Updated: 2026-10-06

Code: https://github.com/Jiaju-Chen/VETTA-official

Project page: https://wouterkool.github.io/publication/buy-4-samples-free-baseline

Terminology

• Credit assignment — 1× in this paper · explained in 3 episodes

Sources

• Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learning
• Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Group-in-Group Policy Optimization for LLM Agent Training
• Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• Turn-PPO: Turn-Level Advantage Estimation with PPO for Improved Multi-Turn RL in Agentic LLMs
• What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents
• Asymmetric Proximal Policy Optimization: mini-critics boost LLM reasoning
• How Much, Then Where: Credit-Conserving Action-to-Token Allocation for Multi-Turn Agent Reinforcement Learning
• HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents
• Qwen2.5 Technical Report
• Toolformer: Language Models Can Teach Themselves to Use Tools
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Proximal Policy Optimization Algorithms
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• Reflexion: Language Agents with Verbal Reinforcement Learning
• AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents
• Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning
• Gated-BEPO: Confidence-Gated Bellman Credit Assignment for Large Language Model Agents

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks