ICM Out! Better Tournament Strategy from Computed Continuations, vs. Solvers and LLMs

The paper introduces Strategic-Continuation Optimization (SCO), a policy-construction method for tournament poker that enumerates current-hand outcomes, maps them to successor states, prices those states with continuation values computed from the finite tournament model, and optimizes and freezes the resulting current-hand policy. The comparison policy uses analytic ICM (Independent Chip Model) with the same optimizer solving the same game but with successor states priced by analytic ICM.

In a three-player jam/fold tournament with a 1,000,000 prize pool, the paper finds that analytic ICM has 9,036 mean absolute value error across all 2,838 state–seat entries relative to the frozen strategic-continuation benchmark. This value error rewrites the ranges it prices: measured against each decision point's own fixed-ICM jam range, SCO moves the jam frequency by an average of 14.08%, and by 32.42% at the button. The policy produced by SCO earns 214.33 more prize equity per hand on average and is favored in 2,433 of 2,838 matched units. The ordering survives replacing the solver-built opponent with two Large Language Models (LLMs) and with a family of non-modeling threshold players.

The paper states: "A stack of tournament chips is not proportional to prize money. Doubling a stack does not double what it returns, because prizes are paid by finishing rank rather than by chip count: the leader's equity is capped by first prize, while a short stack still holds a claim on the ladder."

The Independent Chip Model (ICM) is the standard tool for translating chips into expected prize money: it treats each player's probability of finishing first, second, and so on as proportional to chip share, and reads off an expected prize from the payout schedule.

The key insight is that "ICM values do not only describe a position; they drive decisions. A jam/fold player compares jamming against folding by looking up the continuation equity ICM assigns to each outcome and takes the higher lookup. A model can be a good summary of a state and still be a poor thing to optimize against, because what a controller needs is not the level of the value but the differences between the values of the states it can move to."

The paper works in a finite three-player jam/fold tournament, small enough to enumerate every stack configuration and to freeze and evaluate full-state policy tables. All quantities are stated in the money of a 1,000,000 prize pool.

The paper identifies that ICM "cannot see any of it: a continuation table computed from the tournament itself ranks the button above the small blind and the small blind above the big blind, separating the button from the big blind by 18,595 at equal 45-chip stacks. Beyond position, ICM also fails to price the pressure of elimination. A chip leader can jam without risking its own tournament life, while the short stack it covers must call for everything or fold."

1. The paper locates what ICM misses across the complete finite domain: "its values sit 9,036 from the tournament's own continuation values in mean absolute error, it prices neither the seat a stack sits in nor the elimination pressure a covering stack applies, and the resulting jam ranges move by 14.08% on average and by 32.42% at the button."

2. The paper proposes Strategic-Continuation Optimization, "which optimizes current-hand play against a continuation table computed from the tournament itself, and show under a matched protocol that it earns 214.33 per hand more than fixed ICM, rising to 938.03 per hand when the tournament is played out to a winner."

3. The paper splits that gain exactly, with 0 residual, into 130.43 for knowing which seat holds which stack and 83.90 for pricing chips from the tournament's own dynamics, and holds the direction fixed across six continuation evaluators, a shared-deck Monte Carlo evaluator, two LLM-driven opponent profiles, a non-modeling opponent family, three chip depths, and two prize ladders.

The tournament model is a finite three-player tournament whose 1,000,000 prize pool pays 750,000, 250,000, and 0 for first, second, and third. A state s = (a, b, c) records positive integer stacks at the button, small blind, and big blind, with a + b + c = T. The main grid uses T = 45, blinds 1/2, and all 946 ordered states.

The paper defines two continuation tables:

• ICM continuation (C ICM): the Malmuth–Harville table which reads the stack vector and nothing else, so it is invariant to which seat holds which stack.

• Strategic continuation (C SC): computed from the finite tournament model itself, so it retains the relationship between stack ownership, action order, and the blinds each seat must post next.

For an entry (s, i), the benchmark-relative value error is defined as e i(s) = C i ICM(s) − C i SC(s).

The paper estimates a player's prize equity through four steps: from a state to a current hand, from decisions to terminal outcomes, from terminal chips to prize shares, and from outcomes to expected equity.

Strategic-Continuation Optimization maps a strategic continuation table to a current-hand policy: for every full state, it solves the jam/fold game whose terminal outcomes are priced by that table, then freezes the resulting policy.

Matched evaluation: "For every state s and focal owner i ∈ btn, sb, bb, we construct two profiles that are identical except in the focal owner's rows. One uses π i SCO and the other uses π i ICM; all rows owned by the other two players remain at the same frozen π SCO anchor. Both profiles are then evaluated with the same strategic continuation C SC."

Value-error distribution: Seat-specific Mean Absolute Error (MAE) is 9,127.43 at the button, 5,907.48 at the small blind, and 12,073.13 at the big blind.

Policy difference: combo-prior mean binary TV is 7.5164 percentage points for the button, 5.6198 for the small blind, and 3.5013 for the big blind. The mean relative jam-range change is 32.4161% at the button, against 10.8235% and 10.1351% at the two blinds, for 14.0781% overall.

Headline matched cost: Across all 2,838 state–owner units, π SCO improves mean prize equity by 214.33 per hand. The comparison favors it in 2,433 of the 2,838 units; 277 units are negative and 128 are exactly zero.

Per-seat means are 291.03, 237.63, and 114.33 — largest for the seat whose next-hand position moves the most and smallest for the already-committed big blind.

The distribution is strongly right-skewed. The median gain is 76.08, roughly a third of the mean... The largest single unit gains 36,944.30; the worst loses 134.48.

Optimization gaps: the fixed-ICM policy has a mean current-hand optimization gap of 378.73, while the SCO policy's mean gap is 164.40. Their difference is the directly measured 214.33 matched gain.

The paper defines a seat-symmetrized table C SYM that depends only on M(s): it is as seat-blind as ICM. But its numbers are computed from the tournament, not from Eq. (1).

The decomposition identity is:

Ū(π SCO) − Ū(π ICM) = [Ū(π SCO) − Ū(π SYM)] + [Ū(π SYM) − Ū(π ICM)]

The results: Seat blindness accounts for 130.43, three-fifths of the gain. The other two-fifths — 83.90 — is available to a table that never looks at a seat.

The positional term declines monotonically across seats, from 186.55 at the button to 80.20 at the big blind. The level term is largest at the small blind (113.08), slightly smaller at the button (104.48), and smallest at the big blind (34.13).

Two language models (Qwen2.5:32b and Llama3.3:70b) receive the three-player rules, the payouts, the blinds, the observable stack state, the public action history, the acting position, and their own hand but nothing that could leak the answer: no continuation values from either table, neither hero policy, no strategy margins, no oracle labels, and not the hero's private cards.

Results: Qwen gives a state-equal mean ∆PE of 1,069.45 with 73/120 states positive; Llama gives 1,016.10 with 78/120 states positive. Both have Holm-adjusted p-values below 2×10−4.

Non-modeling opponents: Against very tight mechanical opponents the strategic policy is worse, by up to 100.80 per hand. The gain increases monotonically in opponent looseness and crosses zero between 30% and 40% jam frequency, reaching 258.10 at 60% and 232.98 against a coin flip.

Solver residual: At the tightest tested common tolerance of 62.50, the mean retains 87.8% of its baseline value.

Depth: At T=30, the mean is 305.25; at T=45, 214.33; at T=60, 179.90.

Prize ladder: Winner-take-all gives 21.68; published (750/250/0k) gives 214.33; flat (600/400/0k) gives 622.72.

Table noise: "A 16-block delete-block jackknife over the board dimension gives a standard deviation of 0.9785 on the census mean; multinomial resampling of the pairwise table at its true sample size gives 0.3465. Combining them, σ table = 1.0380, for a signal-to-noise ratio of 206."

Each of the 2,838 units runs 20,000 paired replicates capped at 4,000 hands. The result: The corresponding frozen-policy full-tournament rollout has a mean difference of 938.03, 4.38 times the direct matched census mean, against a pooled standard error of 18.33 — fifty standard errors from zero.

The seat spread almost disappears. Where the one-hand gain ranged from 291.03 at the button down to 114.33 at the big blind, the rollout gains are 920.30, 997.58, and 896.15 — nearly flat.

Re-weighting the one-hand census by empirical visit distribution gives 198.23 against the equal-weight 214.33, a ratio of 0.925.

The paper concludes: "Read as a controller rather than as an estimator, it prices neither the seat a stack sits in nor the way chips actually move in a jam/fold tournament, and a policy that acts on its numbers gives up prize equity it could have collected."

"Strategic-Continuation Optimization (SCO) recovers that equity by optimizing current-hand play against a continuation table computed from the tournament itself. Under a matched protocol that changes only the focal player's policy, SCO earns 214.33 per hand more than fixed ICM across the complete 946-state census, and 938.03 per hand more once the continuation lookup is removed and the tournament is played out to a winner."

The key takeaway: "A value model used to select actions should be judged by the decisions it induces, not by how close its numbers land, because level accuracy and control quality are different properties and the gap between them is collectable."

Improvement 1: Decision-Aware Value Learning for AI Agents

• What to change: Instead of training AI systems to minimize value-prediction error (e.g., MSE on state values), train them to optimize the differences between values of reachable states, weighted by decision impact.

• What the improved AI can do: An AI agent (e.g., in poker, negotiation, or resource allocation) will select actions that maximize long-term outcomes even when its absolute value estimates are inaccurate. It will prioritize correct relative rankings of successor states over precise absolute values, leading to better decisions in sparse-reward or high-stakes domains.

---

Improvement 2: Seat/Context-Aware State Representation

• What to change: Augment state representations with positional or contextual features (e.g., seat order, action timing, elimination pressure) that are currently ignored by standard value models like ICM.

• What the improved AI can do: In multi-agent games or sequential decision-making, the AI will correctly account for asymmetric advantages (e.g., acting later, covering opponents) and avoid overvaluing symmetric states. This yields up to 130.43/hand improvement in poker-like settings and generalizes to any turn-based game where position matters.

Improvement 3: Continuation-Value Optimization Loop

• What to change: Replace static value functions (e.g., ICM) with a recursive optimization loop that computes continuation values from the environment’s own dynamics, then freezes the policy.

• What the improved AI can do: The AI will self-correct its value estimates by simulating future states, leading to policies that are robust to model misspecification. It will outperform fixed-heuristic agents by 214.33/hand in direct comparisons and 938.03/hand in full rollouts, especially in domains with non-linear rewards (e.g., tournament payouts, auctions).

Improvement 4: Policy-Gap-Aware Evaluation Metrics

• What to change: Evaluate AI systems not by value-error but by the decision gap—the difference in achieved utility between the learned policy and the optimal policy given the same value model.

• What the improved AI can do: During training or model selection, the AI will detect when a value model is “good enough” for decisions even if numerically inaccurate, and vice versa. This prevents overfitting to value accuracy and instead optimizes for control quality, reducing regret in real-world deployments.

Improvement 5: Opponent-Adaptive Strategy Generation

• What to change: Use the SCO framework to generate policies that are evaluated against a family of opponent profiles (e.g., tight, loose, LLM-driven) rather than a single assumed opponent.

• What the improved AI can do: The AI will produce strategies that are robust to opponent heterogeneity, maintaining positive expected gains across diverse play styles (e.g., from 30% to 60% jam frequency). It will avoid overfitting to a single opponent model and perform well in open-world interactions.

Improvement 6: Decomposable Gain Attribution

• What to change: Implement a decomposition module that splits performance gains into interpretable components (e.g., positional awareness vs. dynamic chip pricing) with zero residual.

• What the improved AI can do: The AI system will provide transparent explanations for why it outperforms baselines, enabling human operators to debug, trust, and further refine the model. This is critical for high-stakes applications where auditability is required.

Improvement 7: Scalable Finite-State Rollout for Long-Horizon Tasks

• What to change: Extend the SCO method to larger state spaces using hierarchical abstraction (e.g., enumerating critical states, approximating others) and rollout-based evaluation.

• What the improved AI can do: The AI will handle long-horizon tasks (e.g., multi-round tournaments, supply-chain planning) where one-step gains compound (4.38× in the paper), by using continuation values that reflect true end-game dynamics rather than myopic heuristics.

Improved AI System Capabilities (Summary)

• In poker: Wins 214– 938 more per hand than ICM-based agents, adapts to unknown opponents, and explains its edge.

• In general sequential decision-making: Chooses actions that maximize long-term utility even with imperfect value estimates, accounts for positional/contextual asymmetries, and remains robust to opponent diversity.

• In AI safety/alignment: Provides decision-gap metrics to prevent over-optimization of inaccurate proxies, and decomposes performance gains for human oversight.