4DVLT: Dynamic Scene Understanding with Worldline-Centered Vision-Language Tracking
The paper introduces 4DVLT, a method for understanding dynamic scenes by grounding natural language instructions to a persistent worldline across time and multiple camera views. It creates Instruct-4D, a large benchmark, and proposes 4DTrack, an offline tracking framework that uses an object-centric state graph to estimate this worldline. Results show 4DTrack significantly outperforms existing methods in both target identification and trajectory quality.