Can Agents Design Libraries for Agents?
cs.AI, cs.CL, cs.SE
Submitted: 2026-09-29
Updated: 2026-09-29
Comments: 26 pages, 6 figures, 11 tables. Code and data: https://github.com/SprocketLab/librarydesignbench
Code: https://github.com/SprocketLab/librarydesignbench
License: http://creativecommons.org/licenses/by/4.0/
The gist: Agents increasingly build on code written by other agents, and they reimplement rather than reuse, growing the codebases later agents must work in.
Terminology
Abstract
Agents increasingly build on code written by other agents, and they reimplement rather than reuse, growing the codebases later agents must work in. To measure how well agents design libraries for other agents, we introduce LibraryDesignBench, a two-phase benchmark in which an agent implements a full-featured library from a specification that defines required capabilities and potential use cases without prescribing the design. We evaluate the library through the correctness and simplicity of programs written by three user agents from different model families. The benchmark spans 242 expert-validated programming problems across 15 library-design tasks in four languages. On eleven of the fifteen tasks, agent designers reproduce the abstractions of the human-written production library. Downstream agents adopt agent- and human-written libraries alike but underuse them, reimplementing capabilities the library already provides. Our failure analysis finds that downstream agents write extra code mainly because agent-written libraries are rigid or hard to use, not because capabilities are missing. We also experiment with giving designers more prescriptive, agent-first guidance and having them test their library with subagents; this improves downstream scores and yields simpler programs. LibraryDesignBench provides both a testbed for evaluating library-design practices for agent users and an initial design baseline that improves downstream reuse.
Sources
- Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments
- Code for Machines, Not Just Humans: Quantifying AI-Friendliness with Code Health Metrics
- Large Language Models as Tool Makers
- When LLMs Meet API Documentation: Can Retrieval Augmentation Aid Code Generation Just as It Helps Developers?
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
- GLM-5: from Vibe Coding to Agentic Engineering
- LILO: Learning Interpretable Libraries by Compressing and Documenting Code
- Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarios
- ADK Arena: Evaluating Agent Development Kits via LLM-as-a-Developer
- On Mitigating Code LLM Hallucinations with API Documentation
- Beyond Task Completion: A Verification-vs.-Conformance Gap in Tool-Evolving Agents
- Kimi K3: Open Frontier Intelligence
- Refactoring Codebases through Library Design
- DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation
- ProjectEval: A Benchmark for Programming Agents Automated Evaluation on Project-Level Code Generation
- Building to the Test: Coding Agents Deliver What You Check, Not What You Requested
- A Survey on Large Language Model Impact on Software Evolvability and Maintainability: the Good, the Bad, the Ugly, and the Remedy
- Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems