MAS-ProVe: Understanding the Process Verification of Multi-Agent Systems
cs.AI, cs.CL, cs.MA
Submitted: 2026-02-03
Updated: 2026-08-31
Comments: ICML 2026 Regular
Code: https://github.com/Wang-ML-Lab/MAS-ProVe
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Multi-Agent Systems (MAS) built on Large Language Models (LLMs) often exhibit high variance in their reasoning trajectories.
Terminology
Abstract
Multi-Agent Systems (MAS) built on Large Language Models (LLMs) often exhibit high variance in their reasoning trajectories. Process verification, which evaluates intermediate steps in trajectories, has shown promise in general reasoning settings, and has been suggested as a potential tool for guiding coordination of MAS; however, its actual effectiveness in MAS remains unclear. To fill this gap, we present MAS-ProVe, a systematic empirical study of process verification for multi-agent systems (MAS). Our study spans three verification paradigms (LLM-as-a-Judge, reward models, and process reward models), evaluated across two levels of verification granularity (agent-level and iteration-level). We further examine five representative verifiers and four context management strategies, and conduct experiments over six diverse MAS frameworks on multiple reasoning benchmarks. We find that process-level verification does not consistently improve performance and frequently exhibits high variance, highlighting the difficulty of reliably evaluating partial multi-agent trajectories. Among the methods studied, LLM-as-a-Judge generally outperforms reward-based approaches, with trained judges surpassing general-purpose LLMs. We further observe a small performance gap between LLMs acting as judges and as single agents, and identify a context-length-performance trade-off in verification. Overall, our results suggest that effective and robust process verification for MAS remains an open challenge, requiring further advances beyond current paradigms. Code is available at https://github.com/Wang-ML-Lab/MAS-ProVe.
Sources
- Two Heads are Better Than One: Test-time Scaling of Multi-agent Collaborative Reasoning
- A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems
- gpt-oss-120b & gpt-oss-20b Model Card
- Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
- No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding
- Why Do Multi-Agent LLM Systems Fail?
- AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors
- Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
- Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy
- Do LLM Evaluators Prefer Themselves for a Reason?
- Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling
- Improving Factuality and Reasoning in Language Models through Multiagent Debate
- A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration
- Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies
- LLM Evaluators Recognize and Favor Their Own Generations
- Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators
- DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning
- Reflexion: Language Agents with Verbal Reinforcement Learning
- A Survey on Large Language Models for Mathematical Reasoning
- Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domains
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems