Scene-R1: Video-Grounded Large Language Models for 3D Scene Reasoning with Lightweight Annotations
Scene-R1 is a video-grounded framework that enables large language models to reason about 3D scenes without needing dense 3D supervision. It uses reinforcement learning and a two-stage grounding pipeline: temporal grounding from video segments followed by image grounding using SAM2 for pixel masks, projecting them into 3D. This allows for transparent reasoning and accurate 3D understanding using only lightweight annotations.