DataFoundry: Evolving Data Preparators via Recursive Self-Improvement

arXiv:2608.29966 · cs.CL · Submitted 2026-08-30 · Read on arXiv

cs.CL

Submitted: 2026-08-30

Updated: 2026-08-30

License: http://creativecommons.org/licenses/by/4.0/

The gist: Domain adaptation of large language models increasingly depends on constructing high-quality training data, yet existing data-preparation pipelines typically address quality only after generation

Terminology

Abstract

Domain adaptation of large language models increasingly depends on constructing high-quality training data, yet existing data-preparation pipelines typically address quality only after generation through post-hoc filtering. This creates a fundamental mismatch: data-quality issues often originate from the construction process itself, while quality control is applied only to its outputs. We introduce DataFoundry, a framework for evolving data preparators through recursive self-improvement before large-scale data production. DataFoundry represents a data preparator as an evolvable runtime specification and instantiates its evolution with a Skills-as-Modules architecture, in which a central Controller orchestrates modular skills to compile executable runtimes, diagnose deficiencies on small pilot sets using domain-appropriate criteria, and translate diagnostic feedback into adapters that revise individual preparation components while preserving stable interfaces. We evaluate DataFoundry on DataPrep-Bench across mathematics, finance, law, and medicine, and find that recursively evolved preparators produce training data with higher downstream utility than baselines. Experiments across different backbones further demonstrate that these improvements are not tied to a particular model, while analyses and case studies further reveal the framework's optimization dynamics and illustrate how its evolution unfolds in practice.

Sources

Related papers