TrajTok: Learning Trajectory Tokens enables better Video Understanding
TrajTok is an end-to-end video tokenizer that learns to create compact trajectory tokens by integrating a unified segmenter and a trajectory encoder. It dynamically adapts token granularity based on semantic complexity, decoupling token count from video duration. This method results in efficient models like TrajViT2, achieving state-of-the-art accuracy at scale and showing strong versatility for various downstream tasks.