FLARE: Full-Modality Long-Video Audiovisual Retrieval Benchmark with User-Simulated Queries
summary
The gist
FLARE introduces a full-modality long-video audiovisual retrieval benchmark with user-simulated queries to better reflect realistic multimedia search scenarios The first full-modality long-video…
In short
FLARE creates a new benchmark for searching long videos across multiple senses (vision, audio, and unified video). It uses user-simulated queries to test how well AI systems can find relevant content in realistic multimedia search scenarios. Findings show that audio retrieval is a major challenge and that combining modalities improves performance.
Key concepts
- FLARE
- A full-modality long-video retrieval benchmark designed to simulate real user searches. It combines long videos, detailed clip annotations, and user-style queries across vision, audio, and unified audiovisual formats to rigorously test multimedia search models.
- Hard Bimodal Constraint
- A specific rule applied to cross-modal queries in FLARE. These queries must fail when searched using only one modality (vision or audio) but succeed only when both modalities are used together, forcing models to truly understand the combined information.
- Modality Bottleneck
- The consistent finding that audio retrieval is the weakest link in unified audiovisual models. Audio-only media and queries often perform worse than visual counterparts, suggesting that aligning or fusing audio information effectively remains a significant technical hurdle for current AI systems.
Terminology used across episodes
This episode discusses
- FLARE: Full-Modality Long-Video Audiovisual Retrieval Benchmark with User-Simulated Queries · Paper Radio
- MusicLM: Generating Music From Text
- LoVR: A Benchmark for Long Video Retrieval in Multimodal Contexts
- JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation
- Meta CLIP 2: A Worldwide Scaling Recipe
- Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking
- Learning Transferable Visual Models From Natural Language Supervision
- Qwen3-ASR Technical Report
- Qwen3 Technical Report
- Pushing the Frontier of Audiovisual Perception with Large-Scale Multimodal Correspondence Learning
- VideoCLIP-XL: Advancing Long Description Understanding for Video CLIP Models
- Scaling Audio-Text Retrieval with Multimodal Large Language Models
- Qwen3-Omni Technical Report
The paper
FLARE: Full-Modality Long-Video Audiovisual Retrieval Benchmark with User-Simulated Queries · Read on arXiv
Qijie You, Hao Liang, Mingrui Chen, Bohan Zeng, Meiyi Qiang, Zhenhao Wong
University of Science and Technology Beijing · Peking University · Institute of Automation, Chinese Academy of Sciences
As video becomes increasingly central to information dissemination and multimodal large language models (MLLMs) continue to advance, evaluating video retrieval has become increasingly important. In realistic search scenarios, this requires matching short user queries to long-form content using both visual and auditory evidence. Yet existing retrieval benchmarks are still dominated by short clips, single modalities, and caption-based evaluation. We introduce FLARE, a full-modality long-video audiovisual retrieval benchmark with user-simulated queries. Built from 399 carefully screened Video-MME videos (10--60 min, 225.4 h) to ensure source quality and diversity, FLARE contains 87,697 clips annotated with vision, audio, and unified audiovisual captions, together with 274,933 user-style queries. Cross-modal queries are further filtered by a hard bimodal constraint, requiring retrieval to fail under either modality alone but succeed when both are combined. FLARE evaluates models under two regimes, caption-based and query-based retrieval, across vision, audio, and unified audiovisual settings. Experiments with 15 representative retrievers show that user-style queries substantially change model behavior, strong caption-based performance does not always transfer to query-based retrieval, and audio--language alignment remains a key bottleneck for unified audiovisual retrieval. Our code and data are released at https://flarebench.github.io/
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "FLARE: Full-Modality Long-Video Audiovisual Retrieval Benchmark with User-Simulated Queries".
Paper summary: Tom: So we're wrapping up this look at FLARE: Full-Modality Long-Video Audiovisual Retrieval Benchmark with User-Simulated Queries one.
Jane: It’s a new way to test how well AI can find long videos when the search query is realistic, not just a short clip or a single piece of text.
Tom: Right, so they built this benchmark using three hundred ninety-nine long videos and user-style queries to see if unified models can actually reason across vision and audio in real search scenarios one.
Lu: The authors are trying to find where the current AI systems break when they have to combine different types of data from long media.
Meng: It really shows us where the failure points are right now when we're shipping models to production because it’s not just about accuracy on a test set.
Lalam: FLARE is designed for research and testing, so it gives us the evidence needed to fix these core integration problems one.
Tom: And the authors of FLARE have built this framework to show that deep fusion is where the real challenge lies in long video retrieval one.
Conclusion: Tom: So we're wrapping up this look at FLARE: Full-Modality Long-Video Audiovisual Retrieval Benchmark with User-Simulated Queries one.
Jane: It’s a new way to test how well AI can find long videos when the search query is realistic, not just a short clip or a single piece of text.
Tom: Right, so they built this benchmark using three hundred ninety-nine long videos and user-style queries to see if unified models can actually reason across vision and audio in real search scenarios one.
Lu: The authors are trying to find where the current AI systems break when they have to combine different types of data from long media.
Meng: It really shows us where the failure points are right now when we're shipping models to production because it’s not just about accuracy on a test set.
Lalam: FLARE is designed for research and testing, so it gives us the evidence needed to fix these core integration problems one.
Jane: It really highlights that real-world search scenarios are much harder than what we've been testing before, especially when you have long videos involved one.
Tom: And the authors of FLARE have built this framework to show that deep fusion is where the real challenge lies in long video retrieval one.
Jane: So next up, we're going to look at those specific results and see how much audio actually drags down the overall performance when things get complicated.
More episodes
- 2505.16254-Human-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMs
- 2506.10913-Choreographic Quick Changes: First-Class Location (Set) Polymorphism
- 2507.14812-A Black-Box Approach for Exogenous Replenishment in Online Resource Allocation
- 2507.15448-Frames over finite fields with respect to the Galois inner product and their connections to self-orthogonal and LCD codes
- 2508.21572-NewsReX: An Open-Source Multi-Framework for Neural News Recommendation
- 2509.17028-Impact of Packetization on Network Calculus Analysis
- 2510.17342-AoA Services in 5G Networks: A Framework for Real-World Implementation and Systematic Testing
- 2511.04088-Efficient and rate-optimal list-decoding in the presence of minimal feedback
- 2511.17264-Notes on Stack Machines and Quantum Stack Machines
- 2601.15318-On the closest balanced game