Adaptive Graph-of-Islands Evolution for Automatic Feature Engineering with LLMs
cs.AI, cs.LG
Submitted: 2026-07-25
Updated: 2026-09-02
Code: https://github.com/codelion/openevolve
License: http://creativecommons.org/publicdomain/zero/1.0/
The gist: Automatic feature engineering (AutoFE) for tabular data requires discovering informative transformations from a large program space.
Terminology
Abstract
Automatic feature engineering (AutoFE) for tabular data requires discovering informative transformations from a large program space. Existing approaches suffer from three limitations: classical methods rely on fixed operator libraries with limited expressivity, LLM-based methods generate proposals from static prompts without retaining search experience, and evolutionary methods use fixed migration policies that ignore task-specific cross-family transfer utility. We introduce TOPOFE, a framework that formulates AutoFE as graph-structured multi-island evolutionary program search. The transformation space is partitioned into semantically coherent families, each explored by an island through LLM-guided mutation and crossover. Each island maintains a Prompt Adaptation Memory that accumulates accept/reject feedback to steer proposals toward productive regions without parameter updates. To coordinate global exploration, TOPOFE dynamically learns a directed topology graph whose edge weights encode transfer utility between transformation families. Cross-island transfer is triggered by adaptive saturation detection and performed through LLM-mediated hybrid synthesis, enabling discovery of compositional feature programs that cannot emerge from isolated local search. Experiments on 29 tabular datasets show that TOPOFE consistently outperforms most state-of-the-art AutoFE methods on classification and regression tasks. Beyond predictive performance, TOPOFE produces feature sets with lower redundancy and higher representational coverage, while the learned topology graph acquires meaningful task-specific transfer structure correlated with downstream gains. The discovered feature programs transfer reliably across diverse predictors and LLM backbones, demonstrating that improvements arise from TOPOFE's structured search and adaptive coordination rather than backbone-specific generation capability.
Sources
- LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizers
- Qwen3 Technical Report
- Qwen2.5-Coder Technical Report
- GPT-4o System Card
- FELA: A Multi-Agent Evolutionary System for Feature Engineering of Industrial Event Log Data
- FAMOSE: A ReAct Approach to Automated Feature Discovery
- Embedding Domain-Specific Knowledge from LLMs into the Feature Engineering Pipeline
- CodeEvolve: an open source evolutionary coding agent for algorithmic discovery and optimization
- AlphaEvolve: A coding agent for scientific and algorithmic discovery
- ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
- HeurAgenix: Leveraging LLMs for Solving Complex Combinatorial Optimization Challenges
- MolEvolve: LLM-Guided Evolutionary Search for Interpretable Molecular Optimization
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems