Error-Propagation Modeling for Failure Attribution in LLM-Based Multi-Agent Systems
cs.AI
Submitted: 2026-10-08
Updated: 2026-10-08
License: http://creativecommons.org/licenses/by/4.0/
The gist: LLM-based multi-agent systems (MASs) are increasingly used to solve complex tasks through coordinated reasoning, tool use, and interaction with external resources.
Terminology
Abstract
LLM-based multi-agent systems (MASs) are increasingly used to solve complex tasks through coordinated reasoning, tool use, and interaction with external resources. However, attributing failures in such systems remains challenging because the observed outcome often does not directly reveal the error responsible for the failed execution. In this work, the attribution target is the decisive error, defined as the agent--step pair whose correction would recover the failed execution. Existing approaches largely identify suspicious steps without explicitly modeling how errors propagate across interactions or persist in unresolved loops, making decisive errors difficult to distinguish from downstream failure symptoms. We propose Error-Propagation Modeling for Failure Attribution (EMFA). EMFA constructs a structured representation of the failed trajectory, models both cascading propagation and persistent interaction loops, and uses propagation-aware candidate screening followed by counterfactual verification to identify the decisive agent--step pair. On the Who&When benchmark, EMFA achieves state-of-the-art step-level attribution accuracy and remains competitive at the agent level. It improves the previous best step-level results by 3.45 and 4.40 percentage points on the Hand-Crafted and Algorithm-Generated subsets, respectively.
Sources
- Where Did It All Go Wrong? A Hierarchical Look into Multi-Agent Error Attribution
- Why Do Multi-Agent LLM Systems Fail?
- Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks
- GAIA: a benchmark for General AI Assistants
- HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scale
- ChatDev: Communicative Agents for Software Development
- Adaptive In-conversation Team Building for Language Model Agents
- From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systems
- AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?
- AFlow: Automating Agentic Workflow Generation
- Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems