RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysis

arXiv:2606.28385 · cs.RO, cs.AI · Submitted 2026-06-22 · Read on arXiv

cs.RO, cs.AI

Submitted: 2026-06-22

Updated: 2026-10-10

Comments: 34 pages, including appendix. Project page: https://robogazeproject.github.io/RoboGazeProject/

Project page: https://robogazeproject.github.io/RoboGazeProject

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• World Model — 3× in this paper · explained in 7 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Benchmark — 1× in this paper · explained in 2 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• Temporal consistency — 1× in this paper · explained in 2 episodes
• Verification — 1× in this paper · explained in 2 episodes
• Verifier — 1× in this paper · explained in 2 episodes
• Vision-Language Model — 1× in this paper
• Vision-Language Model (VLM) — 1× in this paper · explained in 3 episodes

Sources

• World Simulation with Video Foundation Models for Physical AI
• GAIA-1: A Generative World Model for Autonomous Driving
• Cosmos 3: Omnimodal World Models for Physical AI
• Physion-Eval: Evaluating Physical Realism in Generated Video via Human Reasoning
• VideoScore2: Think before You Score in Generative Video Evaluation
• WorldJen: An End-to-End Multi-Dimensional Benchmark for Generative Video Models
• VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness
• Evaluating Newtonian Mechanics in Video Generative Models with Real Physical Systems
• RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulation
• VideoGen-Eval: Agent-based System for Video Generation Evaluation
• PAI-Bench: A Comprehensive Benchmark For Physical AI
• Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Grounding
• WorldSimBench: Towards Video Generation Models as World Simulators
• Rethinking Video Generation Model for the Embodied World
• DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos
• Clutter-Robust Vision-Language-Action Models through Object-Centric and Geometry Grounding
• Self-Improving VLA Policies: Selected Diffusion Noise for Spurious-Robust Action Smoothing
• Ctrl-World: A Controllable Generative World Model for Robot Manipulation
• World-VLA-Loop: Closed-Loop Learning of Video World Model and VLA Policy
• Towards Accurate Generative Models of Video: A New Metric & Challenges

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• ARC: A Reasoning Recipe for Robot Foundation Models
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies