Grounded Human-Attributed Description and Activity Recognition in Videos
The authors introduce Grounded Human-Attributed Description and Activity Recognition (GHADAR), a task requiring describing every person's characteristics and activity from a video's location. They propose Constrained Attention Masking-based Pretraining (CAMP) to improve performance on this task using Vision Language Models. The work includes a new dataset, AVA-Captions, and a novel evaluation schema.