Track and Caption Any Motion: Open-Vocabulary Spatiotemporal Captioning via Trajectory-Conditioned Generation
TCAM is a generative framework that creates open-vocabulary video descriptions without any text prompts or region hints. It achieves this by combining dense point tracking with language modeling through a Caption-Aware Resampler, which compresses motion trajectories into a context for the decoder. This allows the model to describe sequential events and concurrent subjects while pointing to the exact motion paths involved.