Complex Agents, Shallow Tests: Demystifying and Enhancing Test Adequacy of Agent Harness in the Wild
cs.SE, cs.AI
Submitted: 2026-10-04
Updated: 2026-10-04
Code: https://github.com/Aider-AI/aider
Project page: https://openclaw.ai
Terminology
Sources
- How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks
- Understanding Agent-Reactive Bugs at the Model-Harness Boundary: An Empirical Study of LLM Agent Issue Reports
- Automated structural testing of LLM-based agents: methods, framework, and case studies
- Meta-Harness: End-to-End Optimization of Model Harnesses
- AutoHarness: improving LLM agents by automatically synthesizing a code harness
- The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
- Testing the Untestable? An Empirical Study on the Testing Process of LLM-Powered Software Systems
- Understanding Code Agent Behaviour: An Empirical Study of Success and Failure Trajectories
- Code as Agent Harness
- AlphaEvolve: A coding agent for scientific and algorithmic discovery
- Measuring Agents in Production
- Tangent: An Empirical Study of Testing Practices for LLM-Based Agent Applications
- Agents' Last Exam
- Large Language Models for Robotics: Opportunities, Challenges, and Perspectives
- A Systematic Security Evaluation of OpenClaw and Its Variants
- Lacuna: A Research Map for Machine Learning
- R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Science
- Understanding Bugs in Modern Agentic Frameworks: A Study of Symptoms, Root Causes, and Triggering Conditions
- Stop Comparing LLM Agents Without Disclosing the Harness
- Agent-SafetyBench: Evaluating the Safety of LLM Agents
Related papers
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
- PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
- IntentCoding: Amplifying User Intent in Code Generation
- Incentives and Outcomes in Bug Bounties