Conditional Multi-Event Temporal Grounding in Long-Form Video
cs.CV
Submitted: 2026-06-13
Updated: 2026-10-10
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Multimodal large language models have made rapid progress in video temporal grounding, yet real-world applications routinely require localizing every event that satisfies compositional temporal and
Terminology
Abstract
Multimodal large language models have made rapid progress in video temporal grounding, yet real-world applications routinely require localizing every event that satisfies compositional temporal and spatial conditions. Existing benchmarks fall short: they localize only a single moment per query, count without temporal conditions, or treat grounding and counting as disjoint tasks. We introduce CoMET-Bench for Conditional Multi-Event Temporal Grounding in long-form video, comprising 2789 queries over 600 videos averaging 33.8 minutes across five real-world domains, with each query composed from 4 temporal conditions, 3 spatial conditions, and a dedicated negative-query subset. We further propose a unified evaluation protocol jointly measuring counting, grounding, and negative-query recognition, including a new Rejection-F1 metric that prevents trivial gaming by lazy "always-empty" models. Benchmarking a broad suite of MLLMs, agent-based, and grounding-specialized methods reveals that existing approaches remain far from solving this task. Building on these findings, we propose CoMET-Agent, a training-free agentic framework that reformulates the task as structured search-and-aggregate, improving F1@0.5 by 6.1% over GPT-5 purely through structural reasoning. Failure analysis further surfaces three open directions: fine-grained entity tracking, position-uniform retrieval, and causal event pairing.
Sources
- LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training
- Qwen3-VL Technical Report
- TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models
- Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Models
- CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understanding
- TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability
- Scaling RL to Long Videos
- LongVILA: Scaling Long-Context Visual Language Models for Long Videos
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- OVR: A Dataset for Open Vocabulary Temporal Repetition Counting in Videos
- TRACE: Temporal Grounding Video LLM via Causal Event Modeling
- TransRAC: Encoding Multi-scale Temporal Correlation with Transformers for Repetitive Action Counting
- How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigms
- Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning
- SAM 2: Segment Anything in Images and Videos
- TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models
- LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding
- Vgent: Graph-based Retrieval-Reasoning-Augmented Generation For Long Video Understanding
- DINOv3
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models