Agent Stack for ML Training Lifecycle: Generation, Validation, and Optimization
cs.SE, cs.AI
Submitted: 2026-10-30
Updated: 2026-10-30
Code: https://github.com/karpathy/autoresearch
License: http://creativecommons.org/licenses/by/4.0/
The gist: Production model development requires repeated architecture design, platform integration, debugging, training, and evaluation.
Terminology
Abstract
Production model development requires repeated architecture design, platform integration, debugging, training, and evaluation. We present an agent stack deployed at LinkedIn with two independently usable agent loops: AutoTorch for model generation and migration, and AutoResearch for model optimization. AutoTorch generates PyTorch models and training pipelines for TensorFlow migration, XGBoost replacement, and new-model development. Executable validation hooks check platform API compliance, trainability, and conversion fidelity on sampled data, guiding iterative repair before engineer review and full-scale distributed training. AutoResearch explores architecture and training-configuration changes through expert-steered or autonomous experimentation. An external orchestrator manages parallel trials, experiment history, candidate selection, and wall-clock budgets across agent sessions. A shared runtime preserves execution state independently of agent reasoning and allocates GPUs separately from agent execution. Across ranking, content understanding, and recommendation workloads, AutoTorch completed generation and full-scale training in 92 of 115 workflow runs (80%). Of these, 77 (83.7%) produced models meeting post-training quality requirements. In a separate cohort, AutoResearch improved the primary task metric on held-out data for 28 of 100 production pipelines. Engineering teams estimate that the stack reduced elapsed end-to-end migration time from approximately eight weeks to two weeks, a 4x speedup.
Sources
- AgentScope: A Flexible yet Robust Multi-Agent Platform
- AIDE: AI-Driven Exploration in the Space of Code
- Auto-RecSys: Harnessing Autonomous Research Agents for Industry-Scale Recommender System
- Agentic Framework for Deep Learning workload migration via In-Context Learning
- The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
- MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement
- A Multi-agent AI System for Deep Learning Model Migration from TensorFlow to JAX
- AlphaEvolve: A coding agent for scientific and algorithmic discovery
- R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Science
Related papers
- ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
- TestPrism: Rethinking Test Evaluation Beyond a Single Reference
- Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- For Coding Agents, the Repository Matters