SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding

Name: SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding
Uploaded: 2026-10-03T01:08:39.601293+00:00
Duration: 778 s
Description: SONIC-O1 is a human-verified benchmark testing Multimodal Large Language Models (MLLMs) by requiring them to jointly process audio, video, and text for real-world reasoning. It reveals performance gaps, especially in temporal localization, showing that models often hallucinate timing when asked to pinpoint specific events within long conversational videos.

SONIC-O1 is a human-verified benchmark testing Multimodal Large Language Models (MLLMs) by requiring them to jointly process audio, video, and text for real-world reasoning. It reveals performance gaps, especially in temporal localization, showing that models often hallucinate timing when asked to pinpoint specific events within long conversational videos.

Open the full episode