How Well Do Vision-Language Models Understand Movies?

Name: How Well Do Vision-Language Models Understand Movies?
Uploaded: 2026-10-20T20:20:59.530911+00:00
Duration: 351 s
Description: MF2 is a new benchmark testing if vision-language models can truly understand and recall key narrative information from full-length movies (50-170 minutes). It uses manually created pairs of true and false claims about character motivations and events. Results show current models struggle compared to humans, but using subtitles significantly improves performance.

MF2 is a new benchmark testing if vision-language models can truly understand and recall key narrative information from full-length movies (50-170 minutes). It uses manually created pairs of true and false claims about character motivations and events. Results show current models struggle compared to humans, but using subtitles significantly improves performance.

Open the full episode