POIROT: Agents as their own auditors for failure attribution in large language model multi-agent systems
cs.AI
Submitted: 2026-06-01
Updated: 2026-11-12
Code: https://github.com/11inaki11/BLAME
Project page: https://www.poirot-framework.com
Terminology
Sources
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
- DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing
- LLM Multi-Agent Systems: Challenges and Open Problems
- Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents
- Rethinking the Illusion of Thinking
- Why Language Models Hallucinate
- Failure Modes in LLM Systems: A System-Level Taxonomy for Reliable AI Applications
- A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
- Holistic Evaluation of Language Models
- A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations
- Constitutional AI: Harmlessness from AI Feedback
- Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges
- HaluMem: Evaluating Hallucinations in Memory Systems of Agents
- DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems
- Self-Refine: Iterative Refinement with Self-Feedback
- Chain-of-Verification Reduces Hallucination in Large Language Models
- Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems
- TradingAgents: Multi-Agents LLM Financial Trading Framework
- The relationship between reasoning and performance in large language models--o3 (mini) thinks harder, not longer
- Token-Budget-Aware LLM Reasoning
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents