Beyond Answering: Learning to Speak First and Act in Streaming Video
cs.CV, cs.AI
Submitted: 2026-11-12
Updated: 2026-11-15
Terminology
Sources
- AURA: Always-On Understanding and Real-Time Assistance via Video Streams
- Harnessing Streaming Video in the Wild
- MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learning
- Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding
- LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streams
- OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction
- PhoStream: Benchmarking Real-World Streaming for Omnimodal Assistants in Mobile Scenarios
- ProactiveVideoQA: A Comprehensive Benchmark Evaluating Proactive Interactions in Video Large Language Models
- IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams
- StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding
- Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
- Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams
- StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant
- Proact-VL: A Proactive VideoLLM for Real-Time AI Companions
- JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligence
- MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction
- Qwen3.8-Omni: Towards Native Omni-Modal Agents
- Qwen2.5-Omni Technical Report
- Qwen3-Omni Technical Report
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs