Explore, Map, Remember, Decide: Are Embodied VLMs Ready for Safety-Critical Scenarios?

arXiv:2608.08077 · cs.AI, cs.MA, cs.RO · Submitted 2026-08-08 · Read on arXiv

Gabriele La Malfa, Nitay Alon, Emanuele La Malfa, Reuth Mirsky, Stefan Sarkadi

cs.AI, cs.MA, cs.RO

Submitted: 2026-08-08

Updated: 2026-08-11

Code: https://github.com/gabrielelamalfakcl/emrd

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Steps After Leaving Field of View — 21× in this paper
• Grounding — 20× in this paper · explained in 3 episodes
• Accuracy — 18× in this paper · explained in 1 episode
• Positional Accuracy — 13× in this paper
• Fidelity — 11× in this paper · explained in 2 episodes
• Vision-language models — 11× in this paper · explained in 2 episodes
• Benchmark — 7× in this paper · explained in 2 episodes
• Serial Position Effect — 7× in this paper
• Theory of Space — 7× in this paper
• Focal-Point Spatial Grounding — 6× in this paper
• Absolute Environment Coverage — 5× in this paper
• Coverage-Anchored Spatial Identity Resolution — 5× in this paper
• Hallucination — 5× in this paper · explained in 4 episodes
• Robustness — 5× in this paper · explained in 6 episodes
• Active Exploration — 4× in this paper · explained in 2 episodes
• Cognitive Load — 4× in this paper · explained in 2 episodes
• Sparsity — 3× in this paper · explained in 5 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Embodied AI — 2× in this paper · explained in 2 episodes
• Foundation Model — 2× in this paper · explained in 3 episodes
• Foundation Models — 2× in this paper · explained in 8 episodes
• Hyperparameters — 2× in this paper · explained in 2 episodes
• Inflation — 2× in this paper · explained in 2 episodes
• Maximum Likelihood Estimation — 2× in this paper
• POMDP — 2× in this paper · explained in 2 episodes
• Partially Observable Markov Decision Process — 2× in this paper
• Transformer — 2× in this paper · explained in 2 episodes
• Transformers — 2× in this paper · explained in 2 episodes
• AI Agents — 1× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 4 episodes
• Catastrophic Forgetting — 1× in this paper · explained in 9 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Decoupling — 1× in this paper · explained in 2 episodes
• Human-in-the-loop — 1× in this paper · explained in 4 episodes
• Latent Space — 1× in this paper · explained in 7 episodes
• Localization — 1× in this paper · explained in 2 episodes
• Memory Decay — 1× in this paper · explained in 3 episodes
• Multi-agent Coordination — 1× in this paper · explained in 2 episodes
• Pairwise Comparisons — 1× in this paper · explained in 2 episodes
• Partially Observable Markov Decision Process (POMDP) — 1× in this paper · explained in 3 episodes
• Temporal Consistency — 1× in this paper · explained in 3 episodes
• Trace — 1× in this paper · explained in 2 episodes
• Vision-Language Models (VLMs) — 1× in this paper · explained in 7 episodes

Sources

• Pixtral 12B
• Multi-Object Hallucination in Vision-Language Models
• EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents
• ProcTHOR: Large-Scale Embodied AI Using Procedural Generation
• InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Models
• What do navigation agents learn about their environment?
• Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenes
• What's "up" with vision-language models? Investigating their struggle with spatial reasoning
• Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making
• Ovis: Structural Embedding Alignment for Multimodal Large Language Model
• Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models
• eMEM: A Hybrid Spatio-Temporal Memory System For Embodied Agents
• VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought
• Embodied4C: Measuring What Matters for Embodied Vision-Language Navigation
• RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics
• BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments
• Do VLMs Perceive or Recall? Probing Visual Perception vs. Memory with Classic Visual Illusions
• Qwen3 Technical Report
• DivScene: Towards Open-Vocabulary Object Navigation with Large Vision Language Models in Diverse Scenes
• Visual Room Rearrangement

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems