The TIME Machine: On The Power of Motion for Efficient Perception
TIME uses motion, specifically point-tracks from synthetic video data, as a representation for video understanding. This method trains a model to reconstruct missing motion by learning key displacement features like local and global movement. It achieves state-of-the-art temporal reasoning performance using significantly less training data than traditional methods.