ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs

Detailed Research Summary of ANTE (Anticipate, Then Execute) Framework

As a diligent researcher, I have meticulously synthesized the provided excerpts to construct a comprehensive and detailed understanding of the ANTE (Anticipate, Then Execute) framework. This framework represents a pioneering agentic Reinforcement Learning (RL) approach specifically designed for training coding agents to develop robust internal world models capable of predicting action outcomes and learning effectively from execution feedback.

Core Concept: The Agentic Loop and Internal Belief State

At its heart, ANTE formalizes the coding agent as a Partially Observable Markov Decision Process (POMDP). A critical innovation is the introduction of an explicit belief state (beta i,t) that the policy articulates before every action. This belief state is structured to encapsulate three essential components:

1. State Summary (z i,t): A concise, one-sentence description summarizing what has been established and what remains broken within the agent's understanding of the task context.

2. Outcome Prediction (i,t): A forecast of the likely outcome in a defined outcome space (e.g., whether a specific check field will pass or fail). This prediction is bifurcated into two components: test st i (prediction for test results) and exe st i (prediction for execution results).

3. Decision (d i,t): A decision regarding certainty, labeled as either trust or abstain.

This belief state (beta i,t = (z i,t, i,t, d i,t)) serves as the agent's internal representation of its knowledge—what has happened (past), what is currently true (present), and what the next action is expected to achieve (future).

The Three-Stage Framework

ANTE operates through a structured three-stage pipeline:

Stage I: ANTE-SFT (Supervised Fine-Tuning)

This initial stage focuses on teaching the policy how to generate these structured belief states. The agent is fine-tuned using BeliefTraj to learn the correspondence between its reasoning, its actions, and the required belief state format.

Stage II: ANTE-RL (Belief-Anchored Policy Optimization - BAPO)

This is the core learning phase where the policy learns from experience in the execution environment. The key mechanism here is Belief-Anchored Policy Optimization (BAPO). BAPO achieves credit assignment by:

• Grouping Steps: Comparing steps across multiple rollouts that share a predicted outcome.

• Anchoring Credit: Assigning per-step credit based on the policy's verified forecasts and its trust/abstain decisions. This mechanism turns settled predictions into tangible, per-step advantages, effectively anchoring the learning signal to the agent's own predictive capabilities.

Stage III: ANTE-TTO (Test-Time Optimization)

At inference time, ANTE utilizes ANTE-TTO to select actions dynamically without relying on an external scorer. Instead of executing a single predicted action, it samples candidate actions and ranks them based solely on the belief state they carry. The scoring function s beta(k) is defined as:

s beta(k) = phi(test st i(k)) + I(exe st i(k))

where phi maps outcomes (pass=3, no-answer=1, fail=0), and I is an indicator function. The agent executes only the highest-ranked action from this self-generated ranking.

Key Contributions and Theoretical Formalization

The paper asserts three major contributions:

1. Pioneering Verification: ANTE is presented as the first agentic RL framework in the code domain that verifies explicit policy forecasts against real execution throughout the entire RL process, providing crucial step-level supervision for internal world model refinement.

2. Per-Step Credit via BAPO: BAPO is proposed to transform these verified predictions into per-step advantages by anchoring credit directly onto the policy's own forecasts and trust decisions across correlated rollouts.

3. Self-Scoring Inference (ANTE-TTO): ANTE-TTO allows for intelligent, self-guided action selection at inference time by ranking candidates based on their internal beliefs, eliminating the need for an external scorer.

The framework formally models the agent as a POMDP and provides a concrete training objective:

L(theta) = -E tau 1:N [1 over Z sum i=1 X T i,t=1 X k in K i,t i, t, k(theta)] + eta D KL(pi theta | pi ref) + lambda L pred(theta)

where L pred is the prediction loss that supervises the forecasts using execution-derived labels.

Empirical Results and Impact

The empirical validation demonstrates significant performance gains:

• Performance Boost: ANTE raised Qwen3-32B from a 25.0% to 57.2% pass@1 on SWE-bench Verified, achieving a 6.2% improvement over a matched GRPO baseline trained without the belief state.

• World Model Accuracy: The world model's accuracy during RL improved substantially, climbing from 44% to 69%.

• Generalization: The learned foresight capability showed successful transfer across different harnesses and out-of-distribution domains.

Conclusion: Foresight as Actionable Intelligence

In summation, ANTE aims to construct a code world model that possesses comprehensive foresight—it must know what has happened (past), what holds now (present), and what the next action will achieve (future). The framework proves that this internal belief structure is not merely predictive but is also actionable, enabling agents to anticipate consequences, trust their predictions appropriately, and optimize their execution strategy in real-time. Case studies confirm this utility, showing how ANTE-TTO can correct erroneous abstentions and how the world model sharpens as RL progresses.

1. Confidence-driven action selection via ANTE-TTO: The system can select among candidate actions without an external scorer by using the learned forecasts to choose among candidate actions without an external scorer at inference time, which improves task success rate by approximately 31% additional inference tokens over a single greedy attempt on the 7B scale.

2. Per-step credit assignment via Belief-Anchored Policy Optimization (BAPO): This mechanism allows the system to turn verified forecasts into per-step credit for its actions, which is achieved by comparing steps across rollouts that predict the same outcome and assigns per-step credit based on their verified forecasts and trust decisions.

3. Robust error correction via internal belief state: The agent can achieve better reasoning by maintaining an internal belief stating what has been done, what the code does now, and what the next command will change, which allows it to anticipate failures before acting, as shown by how ANTE anticipates that the proposed edit would break other tests.

4. Improved generalization across domains: The learned foresight capability transfers across harnesses and out-of-distribution domains, enabling the model to improve on eleven out-of-distribution benchmarks by an average of 6.2 percentage points over a matched RL baseline.

Rapid advances in large language models have made software engineering agents capable of tackling complex coding tasks. Within a containerized environment, a coding agent receives tool responses or execution results, and determines its later actions by those earlier results. However, most existing coding agents never look ahead: their policies do not anticipate the consequences of actions, and the terminal reward barely attributes a failure to certain earlier steps, so an ill-judged edit can corrupt the repository state irreversibly before any signal arrives. Recent code world models aim to address these challenges, but they are either trained and then frozen before reinforcement learning (RL), or reduced to an auxiliary next-token loss on the environment's outputs. We present ANTE (Anticipate, Then Execute), an agentic RL framework that verifies the policy's pre-action predictions by real execution and turns them into per-step credit for its actions and trust-or-abstain decisions. With an internal belief stated before each action, Belief-Anchored Policy Optimization (BAPO) turns the settled predictions into per-step advantages by anchoring credit on the policy's own forecasts. To turn the trained forecasts into better actions at inference, thereby facilitating effective test-time scaling, ANTE-TTO (test-time optimization) selects among candidate actions without an external scorer. ANTE raises Qwen3-32B from 25.0% to 57.2% pass@1 on SWE-bench Verified, 6.2% above a matched GRPO baseline trained without the belief state; its world model's accuracy climbs from 44% to 69% during RL, and the learned foresight capability transfers across harnesses and out-of-distribution domains. In summary, we aim to build a code world model that knows what has happened, what holds now, and what the next action will achieve.