What Structurally Breaks an Agent? Auditing Difficulty in Stateful Tool-Use Workflows
cs.LG
Submitted: 2026-11-02
Updated: 2026-11-02
Code: https://github.com/Shereen20112003/what-structurally-breaksan-agent
Project page: https://ala-workshop.github.io/papers/ALA2025_paper_45.pdf
Terminology
Sources
- DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
- AgentBench: Evaluating LLMs as Agents
- TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Use
- Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use
- Scaling Laws for Tool-Use Agents via Controllable Environment Synthesis
- MemCalib: Benchmarking and Optimizing Memory Use in LLM Agents
- Synthetic POMDPs to Challenge Memory-Augmented RL: Memory Demand Structure Modeling
- On the Importance of Task Complexity in Evaluating LLM-Based Multi-Agent Systems
- Behaviour Suite for Reinforcement Learning
- BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning
- $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
- Measuring AI Ability to Complete Long Software Tasks
- ARIES: Autonomous Reasoning with LLMs on Interactive Thought Graph Environments
- Qwen2.5 Technical Report
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis