Belief-Guided Decision Making with Uncertainty Gating in the Game of Go
cs.AI
Submitted: 2026-07-29
Updated: 2026-09-28
Comments: 8 pages, 7 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent advancements in Computer Go, driven by AlphaZero and MuZero, rely heavily on Monte Carlo Tree Search (MCTS) to correct the errors of the neural network policy.
Terminology
Abstract
Recent advancements in Computer Go, driven by AlphaZero and MuZero, rely heavily on Monte Carlo Tree Search (MCTS) to correct the errors of the neural network policy. While effective on massive computational clusters, this dependence creates a critical bottleneck on consumer-grade hardware, where the computational cost of tree management severely limits inference rates. Furthermore, without deep search, these models suffer from hallucination, proposing moves with high confidence that are strategically fatal. This paper introduces a novel Belief-Guided architecture that disentangles the Policy head from a distinct Belief head. Unlike traditional value functions, the Belief head acts as an internal simulator and independent critic, modeling epistemic uncertainty and strategic stability. By integrating memory mechanisms (Transformer/GRU) to handle long-term dependencies and the Ko rule, and utilizing a gating mechanism to filter overconfident policy errors, our model shifts the burden of intelligence from runtime search to parametric "intuition." Experimental results demonstrate that this approach significantly improves search-free win rates and reduces hallucination, enabling professional-level play on limited hardware where massive MCTS is infeasible.
Sources
- Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
- Accelerating Self-Play Learning in Go
- Mind the Uncertainty: Risk-Aware and Actively Exploring Model-Based Reinforcement Learning
- Mitigating Value Hallucination in Dyna Planning via Multistep Predecessor Models
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Evaluation Beyond Task Performance: Analyzing Concepts in AlphaZero in Hex
- Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM
- Mastering Atari Games with Limited Data
- Transformers represent belief state geometry in their residual stream
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems