Disentangling Task Difficulty from Run-Level Failure in Agent Failure Prediction
cs.LG, cs.SE
Submitted: 2026-10-04
Updated: 2026-10-04
Sources
- How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks
- AgentRx: Diagnosing AI Agent Failures from Execution Trajectories
- $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
- On Randomness in Agentic Evals
- It's the Problem, Not the Path: Budget and Difficulty Confounds in LLM Reasoning Trajectories
- Why Do Multi-Agent LLM Systems Fail?
- Automata from Agent Traces: Failure and Next-Step Prediction
- Real-Time Detection and Repair of LLM Agent Failures
- Agent psychometrics: Task-level performance prediction in agentic coding benchmarks
- From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future
- Predicting Task Difficulty Without Rollouts
- Capable but Unreliable: Canonical Path Deviation as a Causal Mechanism of Agent Failure in Long-Horizon Tasks
- LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks
- Beyond Resolution Rates: Behavioral Drivers of Coding Agent Success and Failure
- Proper Scoring Rules for Agentic Uncertainty Quantification
- Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade
- EarlyEval: Cheaper Agent Evaluation via Early Outcome Prediction
- Latent Programming Horizons in Coding Agents
- Evaluating Language Models on Cross-Language Code Functional Equivalence
- Accurate Failure Prediction in Agents Does Not Imply Effective Failure Prevention
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks