ForgeSystem: An Architecture for Demand-Specific Artefact Production with AI Agents
cs.SE
Submitted: 2026-11-09
Updated: 2026-11-09
Comments: 17 pages, 3 figures, 2 tables
Code: https://github.com/droodman/inference-cost
License: http://creativecommons.org/licenses/by/4.0/
The gist: General-purpose software serves many different demands, but this flexibility costs performance and efficiency on any specific demand.
Terminology
Abstract
General-purpose software serves many different demands, but this flexibility costs performance and efficiency on any specific demand. Software tailored to a specific demand avoids these losses, yet developing it requires substantial time and effort from human experts. AI agents offer an alternative: they can design, implement and test artefacts tailored to individual demands. We call this production paradigm Forge. Forging for a demand is economically justified when the cumulative benefit of specialisation exceeds the total cost of construction, verification and maintenance. To make Forge a repeatable production process, we propose ForgeSystem, which consists of a control layer, a forge loop and a recursive self-improvement (RSI) loop. The control layer selects which artefact serves each demand and decides which demands to forge, when to forge them and from which starting point; the forge loop iterates between production and verification until a candidate artefact passes verification; and the RSI loop draws on accumulated production records to supply starting points, refine the harness and train models. We implement ForgeSystem for training frameworks, decoding megakernels and individual computational kernels. Forged training engines reach 1.047-1.332x the model-FLOPs utilisation of their reference frameworks, forged megakernels achieve a geometric-mean decoding speedup of 1.21x over SGLang across thirteen configurations, and forged kernels match or exceed expert libraries in eighteen of twenty-two configurations, while satisfying task-specific quality requirements. Controlled comparisons show that explicit quality requirements steer agents towards optimisations that preserve training quality. We also discuss a gradual transition to Forge through service routing, and how demand-specific production may shift value accumulation and trust towards production systems.
Sources
- VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?
- Bespoke OLAP: Synthesizing Workload-Specific One-size-fits-one Database Engines
- Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation
- CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution
- K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model
- ForgeKernel: Self-Evolving Kernels Catching and Surpassing Experts and Agents
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- DeepSeek-V3 Technical Report
- Scaling Laws for Neural Language Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
- Qwen3 Technical Report
- MiniCPM4: Ultra-Efficient LLMs on End Devices
- MPK: A Compiler and Runtime for Mega-Kernelizing Tensor Programs
- FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling
- cuDNN: Efficient Primitives for Deep Learning
Related papers
- ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
- TestPrism: Rethinking Test Evaluation Beyond a Single Reference
- Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- For Coding Agents, the Repository Matters