Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wild
cs.AI
Submitted: 2026-10-08
Updated: 2026-10-08
Code: https://github.com/hkust-nlp/MineOdyssey
Project page: https://mine-odyssey.github.io
License: http://creativecommons.org/licenses/by/4.0/
The gist: Advances in foundation models are driving efforts to introduce agents to assist people in the physical world.
Terminology
Abstract
Advances in foundation models are driving efforts to introduce agents to assist people in the physical world. Such agents require agentic spatial intelligence: exploring unfamiliar environments, updating spatial understanding through interaction, and adapting actions based on feedback to sustain progress toward a sequence of goals. Existing benchmarks cover only a limited range of spatial layouts, scales, and traversal requirements. We introduce Mine Odyssey, a benchmark for evaluating agentic spatial intelligence using Minecraft reconstructions of real-world locations. It comprises 180 tasks covering 30 such locations across 20 countries and regions on five continents, including 20 outdoor and 10 indoor settings. These settings span diverse spatial scales, layouts, terrains, and connectivity patterns, from Midtown Manhattan and rural Entrup to Santa Lucía Hill and Buckingham Palace. We select meaningful waypoints, such as landmarks, buildings, and rooms, and manually verify their accessibility. Each task provides a natural-language instruction specifying which waypoints to visit and in what order. Completing these tasks requires agents to find accessible routes and entrances, open doors, and move between levels using stairs and ladders, while monitoring their progress and recovering from navigation errors. Across eight evaluated state-of-the-art models, GPT-6 Astra achieves the highest success rate of 85.6%. However, the second-best model, Claude Opus 5.5, completes 73.9% of tasks, while the strongest evaluated open-weight model, DeepSeek-V4.1-Flash, reaches 23.9%, highlighting substantial room for improvement in the agentic spatial intelligence of current models. Comprehensive analyses and ablation studies on Mine Odyssey reveal current models' limitations and provide insights for advancing agentic spatial intelligence.
Sources
- ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects
- SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoning
- SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks
- Gemini Robotics: Bringing AI into the Physical World
- MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft
- ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop
- MineExplorer: Evaluating Open-World Exploration of MLLM Agents in Minecraft
- UrbanGround: From Local Perception to Spatial Agency in a Real-Scale City
- SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes
- The StreetLearn Environment and Dataset
- GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents
- SAP-Nav: Spatial Semantic Representation Meets Active Perception for Hierarchical Open-Vocabulary Object Navigation
- Blueprint-Bench: Comparing spatial intelligence of LLMs, agents and image models
- Communicating about Space: Language-Mediated Spatial Integration Across Partial Views
- OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks
- VideoGameBench: Can Vision-Language Models complete popular video games?
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems