Attributing Agent Failures to Their History Needs an Executed Counterfactual: Toward a Reporting Standard
cs.AI
Submitted: 2026-11-03
Updated: 2026-11-03
Code: https://github.com/nokia-applied-research/Trace
Terminology
Sources
- Non-Determinism of "Deterministic" LLM Settings
- $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
- Thought Anchors: Which LLM Reasoning Steps Matter?
- CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures
- Why Do Multi-Agent LLM Systems Fail?
- The Self-Correction Illusion: Role Relabeling Gates Explicit Error Flagging in Large Language Models
- Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems
- DeltaBox: Millisecond-Scale Sandbox Checkpoint and Rollback for Stateful AI Agents
- How to Interpret Agent Behavior
- Causal Abstractions of Neural Networks
- ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenance
- TalkToAgent: A Human-centric Explanation of Reinforcement Learning Agents with Large Language Models
- Aegis: Automated Error Generation and Attribution for Multi-Agent Systems
- REFLECT: Intervention-Supported Error Attribution for Silent Failures in LLM Agent Traces
- Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?
- The Memory Curse: How Expanded Recall Erodes Cooperative Intent in LLM Agents
- DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems
- Towards Reliable Context Compression for Long-Horizon Agents: An Empirical Study of Execution Instability
- Engineering Record And Replay For Deployability: Extended Technical Report
- Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents