Knowing When to Intervene: Causal Evaluation of AI Agents
cs.AI, cs.LG, econ.EM, stat.ME
Submitted: 2026-11-06
Updated: 2026-11-10
Code: https://github.com/sawarniayush/agent-evaluation-sequential-randomization
Terminology
Sources
- $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
- interwhen: A Generalizable Framework for Steering Reasoning Models with Test-time Verification
- CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures
- Why Do Multi-Agent LLM Systems Fail?
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- Double/Debiased Machine Learning for Dynamic Treatment Effects via g-Estimation
- AgentBench: Evaluating LLMs as Agents
- Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures
- Progent: Securing AI Agents with Privilege Control
- Structural Nested Models and G-estimation: The Partially Realized Promise
- When Harnesses Lose the Signal: Causal Evaluation of Recovery in LLM Agents
- $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
- ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents