Seeing, Saying, but Not Using: From Reportable Spatial Facts to Usable States in Multimodal Large Language Models
cs.CV
Submitted: 2026-10-02
Updated: 2026-10-02
Code: https://github.com/changcv2021/Seein
Project page: https://anonymous
License: http://creativecommons.org/licenses/by/4.0/
The gist: A multimodal large language model that correctly reports a spatial fact does not necessarily use that fact in subsequent reasoning.
Abstract
A multimodal large language model that correctly reports a spatial fact does not necessarily use that fact in subsequent reasoning. To study this distinction, we introduce SpaceConflict, a benchmark of 23, 196 inputs for the construction and use of spatial state. Under a unified Supported/Contradictory/Unknown judgment interface, it covers local fact binding (L1), relational composition (L2), cross-observation consistency (L3), and state judgment under transformation (L4). Posing a direct-state query, a full-transformation query, and an explicit-initial-state query on the same world reveals an availability--utilization gap: models recover the initial state from visual evidence yet fail when that state must drive a transformation. For Qwen3.5-9B, 50 of 100 sequences with a correctly recovered initial state fail the full transformation, and supplying the state explicitly repairs all 50; the gap narrows with scale but does not close. We therefore propose Operational State Supervision (OSS), which supervises task-relevant spatial states and their transformation trajectories and aligns shared facts across contexts. OSS improves paired accuracy on matched judgments most on L3 and L4, the levels that depend on organizing and using state. Evaluating multimodal spatial reasoning thus requires asking not only whether a model can see and state a spatial fact, but whether that fact becomes a usable state in subsequent computation.
Sources
- PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding
- MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs
- OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models
- MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency
- Cubify Anything: Scaling Indoor 3D Object Detection
- Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model
- STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?
- Hypo3D: Exploring Hypothetical Reasoning in 3D
- ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models
- EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI
- Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces
- MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence
- From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D
- DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Models
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models