Agent Stack for ML Training Lifecycle: Generation, Validation, and Optimization

arXiv:2611.00384 · cs.SE, cs.AI · Submitted 2026-10-30 · Read on arXiv

cs.SE, cs.AI

Submitted: 2026-10-30

Updated: 2026-10-30

Code: https://github.com/karpathy/autoresearch

License: http://creativecommons.org/licenses/by/4.0/

The gist: Production model development requires repeated architecture design, platform integration, debugging, training, and evaluation.

Terminology

Abstract

Production model development requires repeated architecture design, platform integration, debugging, training, and evaluation. We present an agent stack deployed at LinkedIn with two independently usable agent loops: AutoTorch for model generation and migration, and AutoResearch for model optimization. AutoTorch generates PyTorch models and training pipelines for TensorFlow migration, XGBoost replacement, and new-model development. Executable validation hooks check platform API compliance, trainability, and conversion fidelity on sampled data, guiding iterative repair before engineer review and full-scale distributed training. AutoResearch explores architecture and training-configuration changes through expert-steered or autonomous experimentation. An external orchestrator manages parallel trials, experiment history, candidate selection, and wall-clock budgets across agent sessions. A shared runtime preserves execution state independently of agent reasoning and allocates GPUs separately from agent execution. Across ranking, content understanding, and recommendation workloads, AutoTorch completed generation and full-scale training in 92 of 115 workflow runs (80%). Of these, 77 (83.7%) produced models meeting post-training quality requirements. In a separate cohort, AutoResearch improved the primary task metric on held-out data for 28 of 100 production pipelines. Engineering teams estimate that the stack reduced elapsed end-to-end migration time from approximately eight weeks to two weeks, a 4x speedup.

Sources

Related papers