Validation-Gated ML Pipeline Synthesis with Expert-Grounded Self-Evolving Skills
cs.AI, cs.SE
Submitted: 2026-11-03
Updated: 2026-11-03
Comments: 49th International Conference on Software Engineering
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent advances in large language models have enabled agentic systems that automate machine learning pipeline construction from minimal user input.
Abstract
Recent advances in large language models have enabled agentic systems that automate machine learning pipeline construction from minimal user input. However, existing agentic AutoML methods either depend on rigid templates that limit adaptability, target narrow task domains, or rely on free-form code generation that lacks reliability guarantees, leaving intermediate errors to propagate silently and degrade the final model. We present EvoML, an agentic framework for end-to-end ML pipeline synthesis built on expert-grounded skills that evolve across runs, executed through a validation-gated harness. The framework decomposes pipeline construction into four phases coordinated by three agents: an Orchestrator that enforces validation gates, advancing each phase only when its outputs pass structural, execution, and semantic checks, which precludes error propagation and keeps every artifact auditable; an Executor that runs expert-grounded skills, grounded in the nnU-Net workflow and pairing reusable templates with online adaptation; and a Curator that performs post-run evolution, consolidating the scripts and guidance recovered during a run into the skill library so it improves across datasets. We evaluate EvoML on eight datasets spanning six sophisticated, real-world ML task types, against agentic AutoML systems and command-line coding tools. EvoML achieves full automation and the best final model quality on all eight, spanning segmentation (up to 0.94 Dice), classification (0.99 accuracy), reconstruction (0.95 SSIM), and denoising (0.79 normalized PSNR), while transferring to broader ML tasks.
Sources
- Automated Machine Learning: From Principles to Practices
- TrainerAgent: Customizable and Efficient Model Training through LLM-Powered Multi-Agent System
- Program Synthesis with Large Language Models
- AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities
- Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI
- MaCTG: Multi-Agent Collaborative Thought Graph for Automatic Programming
- AutoML-GPT: Automatic Machine Learning with GPT
- SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learning
- fastMRI: An Open Dataset and Benchmarks for Accelerated MRI
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents