Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?

arXiv:2607.09996 · cs.AI, cs.MA · Submitted 2026-07-10 · Read on arXiv

cs.AI, cs.MA

Submitted: 2026-07-10

Updated: 2026-10-09

Code: https://github.com/meta-llama/llama-models

Project page: https://whowhenpro.github.io

Terminology

• Agentic System — 2× in this paper · explained in 2 episodes
• Benchmark — 2× in this paper · explained in 2 episodes
• AI Agents — 1× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 4 episodes

Sources

• Self-Improving LLM Agents at Test-Time
• Fara-7B: An Efficient Agentic Model for Computer Use
• Where Did It All Go Wrong? A Hierarchical Look into Multi-Agent Error Attribution
• AgentRx: Diagnosing AI Agent Failures from Execution Trajectories
• ChemCrow: Augmenting large-language models with chemistry tools
• Evaluating Large Language Models Trained on Code
• Multi-Agent Evolve: LLM Self-Improve through Co-evolution
• A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems
• Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks
• A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence
• Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling
• Who is Introducing the Failure? Automatically Attributing Failures of Multi-Agent Systems via Spectrum Analysis
• GLM-5: from Vibe Coding to Agentic Engineering
• Embodied LLM Agents Learn to Cooperate in Organized Teams
• DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agents
• SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge
• On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents
• EvoChart: A Benchmark and a Self-Training Approach Towards Real-World Chart Understanding
• LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
• MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Engines

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems