Solver-Guided Reasoning for Mixed-Equilibrium Strategies
Han Wang, Philippe Beardsell, Boning Li, Aaron Sasmita, Shuai Li, Hongyuan Zha, Baoxiang Wang
cs.LG, cs.GT
Submitted: 2026-08-07
Updated: 2026-08-10
License: http://creativecommons.org/licenses/by/4.0/
The gist: Reasoning in large language models (LLMs) is often grounded in human text, human demonstrations, and human-generated rationales.
Terminology
Abstract
Reasoning in large language models (LLMs) is often grounded in human text, human demonstrations, and human-generated rationales. For equilibrium reasoning in complex games, however, relying on human data can be suboptimal. In fact, human play is often guided by intuition and heuristics and can deviate substantially from game equilibrium. This discrepancy is amplified in games with mixed-strategy equilibria, where human data is heavily biased toward pure strategies. Consequently, conditioning LLMs on this data yields weak game strategies. To grant LLMs the reasoning capacity in games, in this work, we study how to elicit equilibrium play using solver output. We propose Mixed-Strategy Decision Tree (MDT), which articulates the silent optimality of the equilibrium into sparse strategic rules that both humans and LLMs could understand. Using solver output rather than human annotation allows us to extend the input to arbitrarily new states and continuations. We instantiate this study on No-Limit Texas Hold'em by querying a solver oracle for over 250 million mixed-strategy decisions; MDT together with other techniques reduces the 1 distance to the equilibrium by 52.6% across 8 different LLM configurations. A Route-only ablation tests the incremental contribution of the shadow-based contrast, while complete River-endgame and Liar's Dice experiments evaluate strategic fidelity and portability beyond the original NLH communication setting.
Sources
- GPT-4 Technical Report
- Distilling a Neural Network Into a Soft Decision Tree
- Textbooks Are All You Need
- Are ChatGPT and GPT-4 Good Poker Players? -- A Pre-Flop Analysis
- Distilling the Knowledge in a Neural Network
- PokerGPT: An End-to-End Lightweight Solver for Multi-Player Texas Hold'em via Large Language Model
- GTO Wizard Benchmark
- Policy Distillation
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks