Where It Breaks: Diagnosing Perception, Memory, and Decision Failures of a Vision-Language Agent in Pac-Man
cs.LG
Submitted: 2026-11-04
Updated: 2026-11-16
Comments: 27 pages, 7 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: Spatial agents must interpret observations, maintain consistent spatial memory and make decisions over long interactions.
Terminology
Abstract
Spatial agents must interpret observations, maintain consistent spatial memory and make decisions over long interactions. We use Pac-Man as a testbed to study the training challenges behind these capabilities in a Qwen3.5-9B-based agent. Our research trajectory progresses from primitive-action RL on text-based mazes to visual inputs with an internal-state-assisted harness, then toward observation-driven memory and joint policy--harness improvement. The main analysis focuses on text-based maze RL; supplementary sections report visual-adaptation and harness experiments and discuss subsequent directions. Text-based maze RL requires curriculum and training-recipe design to overcome cold starts; normal-start clear rates on eight held-out mazes rise from 0% to a peak of 81.25%. In the second step, model adaptation and harness improvements raise the clear rate from 26.6% to 96.9% on one fixed map with three extra lives. These results highlight three development lessons: use curricula to address cold starts, establish visual grounding before language adaptation, and treat harness design as a determinant of agent performance. They provide practical guidance for subsequent VLM training and spatial-agent development.
Sources
- Uncertainty-Informed Active Perception for Open Vocabulary Object Goal Navigation
- The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
- Beyond the Remembered World: Predictive 4D Belief for Persistent Navigation in Evolving Worlds
- lmgame-Bench: How Good are LLMs at Playing Games?
- BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
- Vision language models are blind: Failing to translate detailed visual features into words
- Backplay: "Man muss immer umkehren"
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- NavHarness: Adaptive Goals for Agentic Vision-Language Navigation
- When Memory Lies: An Empirical Study of Spatial Memory Staleness in VLM Agents
- Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems?
- VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis