Does Local Video Understanding Transfer Across Encounters? The EgoGears Benchmark
cs.CV, cs.AI, cs.RO
Submitted: 2026-09-29
Updated: 2026-09-29
Code: https://github.com/lei-qi-233/EgoGears
License: http://creativecommons.org/licenses/by/4.0/
The gist: Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination.
Terminology
Abstract
Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination. Yet aggregate cross-video accuracy conflates failures of local perception with failures to preserve observation identity, establish correspondence, and compose evidence, obscuring whether local video understanding actually transfers. We introduce EgoGears, a complementary single- and multi-video benchmark designed to diagnose this transition. It contains 567 single-video and 1,487 multi-video questions derived from 126 human-collected egocentric recordings covering 39 outdoor routes. Repeated traversals across movement speeds and lighting conditions ground comparisons in shared physical environments; 531 questions require alignment across independent recordings. Single-video questions measure the local visual, spatial, and motion evidence available to a model, while multi-video questions test whether evidence remains bound to the correct observation and can be composed into consistent route relationships. We report 29 single-video and 31 multi-video MLLM configurations across six model families in the main leaderboard. Among the 20 configurations evaluated comparably on both splits, every model performs worse on multi-video questions, with a mean decrease of 22.5 percentage points, and the gap persists when answer format and scoring are held fixed. The gap is not explained simply by additional videos or recording boundaries. The central bottlenecks are observation--evidence binding and ordered route-state tracking. The code and benchmark are publicly available at https://github.com/lei-qi-233/EgoGears.
Sources
- Qwen3-VL Technical Report
- Qwen2.5-VL Technical Report
- EgoMonth: A Month-Level Egocentric Video Benchmark for Long-Term Spatiotemporal Memory
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Gemma 3 Technical Report
- Gemma 4 Technical Report
- GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
- MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
- Kimi-VL Technical Report
- MMSI-Video-Bench: A Holistic Benchmark for Video-Based Spatial Intelligence
- EgoExoMem: Cross-View Memory Reasoning over Synchronized Egocentric and Exocentric Videos
- Seeing Together: Multi-Robot Cooperative Egocentric Spatial Reasoning with Multimodal Large Language Models
- Slow-Fast Architecture for Video Multi-Modal Large Language Models
- InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
- PROVIA: Procedure State Tracking for Online Mistake Detection in Egocentric Videos
- LanP: Rethinking the Impact of Language Priors in Large Vision-Language Models
- Qwen3-Omni Technical Report
- SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition
- InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models