ELMER: Evolutionary Language Model that Explores and Refines

ELMER: Evolutionary Language Model that Explores and Refines introduces an Evolutionary Language Model that searches over natural-language policy descriptions and compiles typed programs for execution. A fully fine-tuned Qwen3-8B model learns three task-conditioned operations: conditional semantic mutation, natural language to domain-specific language (GPTL) compilation, and GPTL to natural language translation. The model is fine-tuned with conditional input on the mutation strength (low, medium, high) using Direct Preference Optimization (oDPO). Across 252 fixed-budget evolutionary searches, oDPO improves both behavioral calibration and finite-budget search efficiency. Natural-language attains the highest observed held-out fitness. Our analysis shows that the condition input (mutation strength) systematically changes semantic edit composition and that language mutations preserve more parent fitness at matched small-to-moderate behavioral displacement. These results show that language can serve as a steerable, execution-grounded search representation over executable program space.

The paper addresses the problem that program evolution can measure whether a mutation improves fitness, but it often cannot control how far that mutation moves in behavior space. Continuous optimizers expose an explicit update scale in that program search typically relies on syntactic proxies such as token, tree, or subtree edits. These proxies can be poorly aligned with execution in that a small comparator change may alter nearly every action a policy takes, while a larger rewrite may preserve the same action sequence. The missing object is a mutation scale whose requested magnitude predicts realized behavior.

The system separates the representation used for variation from the representation used for evaluation. Evolution mutates standardized natural-language (NL) descriptions of trading policies, while a learned compiler maps each child into typed Genetic Programming Trading Language (GPTL) code for deterministic execution. Parent and child programs are run on the same historical market data, and disagreement between their action sequences defines realized behavioral displacement. A low, medium, or high request targets an ordered, overlapping behavioral regime.

A single fine-tuned Qwen3-8B large language model (LLM) learns three task-conditioned operations: NL-to-GPTL compilation, GPTL-to-NL translation, and conditional NL mutation. Mutation supervision is constructed from grammar-valid GPTL degradation chains whose endpoints can be executed and assigned deterministic behavior distances, then those transitions are translated into language. Behavior-grounded supervised fine-tuning (SFT) learns partial regime control. Common-parent offset Direct Preference Optimization (oDPO) sharpens that control by preferring sibling children whose executed displacement better matches the requested regime. While fitness remains the outer-loop selection signal, behavioral displacement trains the variation operator.

Financial trading is used as a testbed for stochastic, state-dependent programmatic control. Fixed historical trajectories permit deterministic, highly parallel replay and strict temporal holdouts. Across 252 fixed-budget searches, oDPO significantly improves mutation-strength calibration and validation-trajectory area under the curve (AUC) relative to behavior-grounded SFT, native abstract syntax tree (AST) mutation, and a matched code-output oDPO model. The matched model holds the backbone, training transitions, preference data, conditioning, and objective fixed while changing the mutation representation from language to GPTL code. Natural-language oDPO also discovers the highest observed held-out policy, while held-out mean and upper-tail results are treated descriptively. Mechanistic analyses show that requested strength changes semantic edit composition and that language mutations preserve more parent fitness at matched small-to-moderate behavioral displacement.

The contributions are: an execution-grounded definition of mutation displacement and a conditional variation operator over ordered behavioral regimes; a multitask language model trained for semantic mutation, compilation, and translation using behaviorally calibrated executable transitions; controlled ablations that isolate domain fine-tuning, behavior-grounded conditioning, oDPO, and the natural-language representation; and mechanistic analyses of calibration, semantic edit composition, syntax–behavior locality, and mutation quality at matched displacement.

Behavioral displacement is defined as follows: Let G be the set of valid executable programs, RNL the set of language descriptions, S the market-state space, and B = 0, 1 4 the raw signal space for long entry, short entry, long exit, and short exit. A policy π: S → B maps a market state to a four-signal vector. For a shared evaluation trajectory s1:N = (s1,..., sN) ∈ S N, behavioral displacement is equivalent to the raw-signal disagreement: dbeh (πa, πb) = (1/N) Σ I[πa (st) ̸= πb (st)].

A learned compiler C: RNL → G maps child description xc to a program inducing πc. Sampling xc ∼ pθ (· xp, s) and compiling induces qθ (πc πp, s) for s ∈ low, medium, high. Let µs be the empirical center of regime s. An idealized target is minθ Eπp,s Eπc ∼qθ (·πp,s) [dbeh (πp, πc) − µs]. Because execution is nondifferentiable, common-parent oDPO instead ranks children by proximity to µs, subject to Dlow ≺ Dmedium ≺ Dhigh. These categories are ordered, overlapping regimes rather than exact numerical radii.

The model is fine-tuned on three task-conditioned operations. Compilation maps a natural-language policy description to a typed GPTL program that can be checked, executed, and evaluated through backtesting. Program-to-language translation maps a GPTL program into the standardized natural-language representation consumed by the semantic mutator. Mutation receives a parent description, its backtest feedback, and a requested strength s ∈ low, medium, high, then produces a child description whose executed behavioral displacement should fall within the corresponding regime. Together with deterministic backtesting and fitness-based selection, these operations form a closed evolutionary loop. Policies are mutated in language, compiled into GPTL code, executed to obtain fitness, and retained or rejected by the outer optimization algorithm. Translation allows a hand-designed GPTL seed, a directly mutated program, or another code-space discovery to enter the language representation used by the mutator. During ordinary natural-language evolution, the child description is inherited directly and its compiled GPTL program is stored as the evaluated artifact, so translation is not repeatedly applied along the lineage.

The same loop provides a controlled way to construct behavior-grounded mutation supervision. Starting from high-fitness GPTL programs, defined as training Sharpe greater than 0.5, grammar-valid AST edits are applied and edges that directionally reduce fitness are retained, producing degradation chains. Reversing one-, two-, and four-link segments yields directionally improving parent-child transitions at several mutation scales. Because both endpoints are executable programs, they are run on the same training-only market windows and their behavioral displacement is computed. GLM 5.2, employed as an annotation model with reasoning off and temperature equal to 0.1, renders both endpoints in the standardized natural-language format. The resulting examples pair a natural-language semantic edit with a displacement measured deterministically from the original executable programs. The model therefore learns to mutate in language while developing an implicit association between semantic edit patterns and their realized behavioral regimes.

The final supervised corpus contains 45,639 examples: 13,551 conditional mutations, 16,044 compilations, and 16,044 program-to-language translations. These collections are split into 44,727 training and 912 validation examples. The mutation data are derived from 2,811 degradation chains and 44,920 graded reverse transitions, of which 44,917 execute successfully. Strength labels are assigned using equal-frequency behavioral-distance bins with cut points 0.165 and 0.466 as determined by the data distribution. Null transitions below 0.005 and transitions near the category boundaries are removed, then the retained examples are balanced across strength and asset. Distances are measured on six training-only windows of 6,000 hourly bars. Mutation prompts provide the available domain primitives, the parent policy (both its GPTL and NL representations), its fitness, and seven backtest statistics. Conditioned prompts add the requested mutation strength.

For common-parent preferences and oDPO, for each eligible parent, one representative child per strength category is selected. Given request s, the child nearest µs is designated as preferred y + and a child outside the requested category as rejected y −. Pairs whose behavioral separation falls below 0.15 times the training interquartile range (IQR) are filtered out to ensure a distinct preference gap. Each retained pair receives m = min(1, d+ − d− / Q0.95 (d+ − d−)), where Q0.95 is the 95th percentile of training separations. Clipping at one prevents extreme outliers from dominating the oDPO offset and gradient. The procedure yields 14,735 training and 775 validation pairs from 6,369 parents. The loss is LoDPO = −E[log σ(β∆rθ − λm)], with ∆rθ = rθ (y + x) − rθ (y − x), β = 0.3, and λ = 0.2. SFT uses one epoch at 2 × 10−5; oDPO uses one epoch at 2 × 10−6. Checkpoints must exceed 95% on compilation validity, translation signal preservation, and output format, and pass a held-out monotonicity gate with Spearman ρ ≥ 0.4.

The ablation ladder separates four effects. Stock Qwen3-8B measures zero-shot mutation; no-label SFT isolates domain and mutation-task fine-tuning; prompt-only labels test whether strength words alone induce control; behavior-grounded SFT tests supervised conditioning on measured displacement; and oDPO isolates the additional effect of common-parent preference optimization after conditional SFT. Two code controls isolate representation and operator effects. The native baseline compares learned language mutation with a hand-designed weighted AST operator spanning parameter, indicator, subtree, comparator, logical, clause, crossover, and signal-leg edits. This same operator system was used to generate the degradation chains that ultimately produced the multi-task training dataset. The matched code-output model holds the backbone, training transitions, preference pairs, strength labels, and oDPO objective fixed while changing the output substrate from natural language to GPTL. It therefore provides the cleanest test of natural language as a search representation.

Experiments use hourly continuous futures from January 2008 through October 2025: the E-mini S&P 500 contract (ES; 106,685 bars), Silver (SI; 107,085), and U.S. Treasury Bond (US; 105,124). The backtester starts with 10,000, applies 0.005% commission and 0.01% slippage, and uses unit long, short, or flat positions. Each asset has three non-overlapping temporal folds, split chronologically into 70% training, 15% validation, and 15% test, with a ten-day embargo. During search, the mutation model receives training-fold feedback; validation scores drive evolutionary selection, and the best-validation policy is evaluated once on the corresponding held-out test fold after the 1,000-evaluation budget ends. The learned-operator matrix contains six operators, three algorithms, three assets, and four seeds (216 searches). A separate 36-search AST matrix supplies the native baseline, and the 36 NL oDPO runs are shared across analyses. Validation Sharpe, held-out test Sharpe, top-quartile mean test Sharpe, maximum test Sharpe, validity, and area under the best-so-far validation curve on the common 1,000-evaluation grid (AUC) are reported.

Lineage analysis measures normalized AST token distance, behavioral displacement, edit taxonomy, entropy, and fitness change in shared behavioral-distance bins. A mutation is neutral when dbeh = 0 and catastrophic when an execution-valid child’s validation Sharpe falls by more than one; failures are tracked separately through validity. Runs are paired by algorithm, asset, and seed, yielding 36 matched observations per contrast. Six confirmatory contrasts use two-sided Wilcoxon signed-rank tests with Holm correction, 10,000-resample paired-bootstrap intervals, and rank-biserial effects. Held-out test fitness is the endpoint for the three capability and conditioning contrasts; AUC is the endpoint for the three oDPO and representation contrasts. Representation-level mean, top-quartile, and maximum test fitness are descriptive. Calibration uses run-level Spearman correlations followed by paired Wilcoxon tests with Holm correction; pooled low–high Cliff’s δ is descriptive.

Results show that in the run-level analysis used for inference, mean Spearman correlation between requested strength and realized displacement is 0.824 for oDPO, 0.310 for behavior-grounded SFT, and 0.018 for prompt-only conditioning. Every one of the 36 paired runs favors oDPO in both comparisons (rrb = 1.00, pHolm < 0.0001). Descriptive pooled low–high Cliff’s deltas are 0.98, 0.36, and 0.02. oDPO therefore turns partial supervised ordering into consistent regime control; the broad medium distribution and high-strength saturation still preclude exact numerical calibration.

The untrained model yields negative validation, test, and AUC. No-label SFT becomes functional. Prompt-only labels lower every average metric relative to no-label SFT, showing that grounding gives the conditioning interface its operational meaning. Domain SFT creates a functional mutator, prompt-only labels significantly degrade performance, and behavior-grounded SFT restores a useful conditional interface. All three oDPO AUC contrasts remain significant after Holm correction, establishing its primary gain in finite-budget search efficiency. Behavior-grounded SFT has the highest mean and top-quartile test fitness, while NL oDPO attains the highest observed test fitness (2.3445). The label-only bootstrap interval targets the paired mean, whereas its Wilcoxon test targets a paired rank shift, so the interval can narrowly cross zero while the corrected rank test remains significant.

Comparing NL oDPO with native AST mutation and the matched code-output control, both AUC contrasts remain significant after Holm correction. Because the matched model holds the backbone, transitions, preferences, labels, and objective fixed, its contrast isolates the representation used to express variation. The code-output effect is modest and near the corrected threshold, but the two comparisons support a finite-budget search-efficiency advantage for language. NL oDPO has higher mean test fitness than native AST (0.4478 versus 0.3526) and matched code output (0.4478 versus 0.3024), as well as the highest top-quartile mean (1.5063) and observed maximum (2.3445). These values indicate a stronger observed upper tail but are not confirmatory representation endpoints. All systems include negative outcomes, and the outer-loop pattern is heterogeneous: MAP-Elites and ProFiT favor NL on mean test, whereas (µ, λ)-ES favors matched code output. Archive memory may help preserve NL’s structured exploration, but the experiment does not isolate this interaction.

Mechanistic analyses show that parameter changes constitute 86.4% of low-strength edits, 35.5% at medium, and 0.46% at high. High strength reallocates probability toward indicator substitutions (39.5%), comparator swaps (19.7%), structural rewrites (12.4%), directional flips (11.4%), and clause operations. Entropy rises from 0.82 to 2.23 to 2.40 bits, while prompt-only remains near 1.86 bits. Requested strength therefore selects distinct semantic mutation regimes rather than merely changing surface wording.

Compiled NL mutations have a stronger syntax–behavior relationship (ρ = 0.88, r = 0.66) than native AST mutation (ρ = 0.44, r = 0.31) and a lower catastrophi…

Based on this paper, here are the specific improvements you can make to AI systems and what the improved systems can do:

1. Execution-Grounded Mutation Control for Program Synthesis

• Improvement: Train a language model to mutate program descriptions with a requested behavioral displacement (low/medium/high) that is calibrated against actual executed behavior, not syntactic similarity.

• Improved system capability: An AI that can generate program variants (e.g., trading strategies, control policies) with predictable, controllable changes in behavior—e.g., make a slightly different version or make a radically different version—while preserving fitness where appropriate.

2. Multitask Language–Code Translation for Evolutionary Search

• Improvement: Fine-tune a single LLM to perform three operations: (a) natural-language → executable typed code, (b) code → natural-language description, (c) conditional semantic mutation in language, all sharing one backbone.

• Improved system capability: A self-contained evolutionary loop where the AI can explore program space in human-readable language, compile proposals into executable artifacts, and translate any discovered code back into language for further variation—enabling human-in-the-loop or fully automated search over complex, stochastic programs.

3. Offset Direct Preference Optimization (oDPO) for Ordinal Control

• Improvement: Use common-parent preference pairs where the preferred child is the one whose executed behavioral displacement is closest to the requested regime, with a margin-based offset (λm) to sharpen separation.

• Improved system capability: An AI whose variation operator becomes reliably ordinal—e.g., consistently producing small, medium, or large behavioral changes on demand—rather than only learning a rough correlation. This is critical for applications where the magnitude of change must be controlled (e.g., safe exploration, staged policy refinement).

4. Behavior-Grounded Supervision from Degradation Chains

• Improvement: Construct training data by applying grammar-valid edits that reduce fitness, then reversing those edits to create directionally improving parent–child pairs with measured behavioral distances.

• Improved system capability: An AI that learns to mutate in a way that is execution-aware—it understands that a tiny syntactic change (e.g., flipping a comparator) can be behaviorally catastrophic, while a large rewrite might be neutral. This reduces wasted evaluations and catastrophic failures during search.

5. Natural-Language as a Search Representation Over Code

• Improvement: Replace direct code-space mutation (e.g., AST edits) with mutation in standardized natural-language descriptions, then compile to code for evaluation.

• Improved system capability: An AI that achieves better syntax–behavior locality (ρ=0.88 vs 0.44 for AST) and lower catastrophic mutation rate (9% vs 14%), meaning its search moves are more coherent and less damaging at matched behavioral displacement. This is especially useful for domains where code is brittle and small edits have outsized effects.

6. Conditional Mutation with Semantic Edit Composition Control

• Improvement: Condition the mutation model on requested strength, and verify that this conditioning changes the type of edits (e.g., low strength → parameter tweaks; high strength → structural rewrites, indicator substitutions).

• Improved system capability: An AI that can perform stratified exploration—e.g., fine-tune parameters locally for exploitation, or perform structural rewrites for global exploration—by simply changing a prompt token. This enables adaptive search strategies (e.g., simulated annealing over mutation strength).

7. Matched Code-Output Ablation for Representation Isolation

• Improvement: When evaluating a new representation (language vs. code), hold everything else fixed—backbone, training transitions, preference data, conditioning, objective—and only change the output substrate.

• Improved system capability: A rigorous methodology for AI developers to isolate whether a representation choice (e.g., language vs. raw code) truly improves search efficiency, rather than confounding with other training differences. This leads to more trustworthy claims about which design choices matter.

8. Validation-Trajectory AUC as a Finite-Budget Search Metric

• Improvement: Use area under the best-so-far validation curve (AUC) as a primary metric for comparing evolutionary operators under a fixed evaluation budget.

• Improved system capability: An AI search system that is optimized for sample efficiency—finding good solutions quickly—rather than only final fitness. This is critical for real-world applications where each evaluation is expensive (e.g., robotics, drug design, financial backtesting).

9. Mechanistic Analysis of Edit Taxonomy and Entropy

• Improvement: Track edit-type distributions and entropy as a function of requested strength to verify that conditioning is not just surface-level wording but changes the actual semantic operations.

• Improved system capability: An AI whose behavior is interpretable—you can inspect what kind of changes it makes at each strength level, enabling debugging, safety audits, and user trust. For example, you can confirm that high strength actually performs structural rewrites, not just parameter jitter.

10. Handling Stochastic, State-Dependent Environments via Deterministic Replay

• Improvement: Use fixed historical trajectories for evaluation, enabling deterministic, parallel replay of any candidate program.

• Improved system capability: An AI that can be trained and evaluated in stochastic environments (e.g., financial markets, traffic control) without variance in fitness estimates, allowing reliable comparison of mutations and stable gradient signals for preference optimization.

What the improved AI system can do overall:

• Controllable program evolution: Generate variants of a policy with predictable behavioral impact (small/medium/large changes) in a single forward pass.

• Sample-efficient search: Find high-fitness programs within a fixed budget by making more coherent, less catastrophic mutations.

• Human-interpretable search: Explore program space via natural-language descriptions, while still producing executable, validated code.

• Safe exploration: Avoid destructive mutations (e.g., flipping a comparator that breaks all behavior) by learning from behavior-grounded supervision.

• Adaptive search strategies: Dynamically adjust mutation strength based on progress (e.g., start with high strength for exploration, then fine-tune with low strength).

• Transferable across domains: The same architecture can be applied to any domain where programs can be compiled from language and executed deterministically (e.g., trading, robotics control, data-processing pipelines).

Program evolution can measure whether a mutation helped, but it rarely controls how far the mutation moves in behavior space. Syntactic edit size is an unreliable proxy: a small code change can alter nearly every action, while a larger rewrite can preserve the same execution trace. We introduce an Evolutionary Language Model that searches over natural-language policy descriptions and compiles typed programs for execution. A fully fine-tuned Qwen3-8B model learns three task-conditioned operations: conditional semantic mutation, natural language to domain-specific language (GPTL) compilation, and GPTL to natural language translation. The model is fine-tuned with conditional input on the mutation strength (low, medium, high) using Direct Preference Optimization (oDPO). Across 252 fixed-budget evolutionary searches, oDPO improves both behavioral calibration and finite-budget search efficiency. Natural-language attains the highest observed held-out fitness. Our analysis shows that the condition input (mutation strength) systematically changes semantic edit composition and that language mutations preserve more parent fitness at matched small-to-moderate behavioral displacement. These results show that language can serve as a steerable, execution-grounded search representation over executable program space.