From Senses to Decisions: The Information Flow of Auditory and Visual Perception in Multimodal LLMs
cs.AI, cs.CL, cs.CV, cs.SD
Submitted: 2026-06-08
Updated: 2026-11-05
Terminology
Sources
- LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training
- Qwen3-VL Technical Report
- VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs
- Qwen2-Audio Technical Report
- More Images, More Problems? A Controlled Analysis of VLM Failure Modes
- OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Models
- VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction
- Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models
- EchoingPixels: Aliasing-Resistant Joint Token Reduction for Audio-Visual LLMs
- AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?
- Listen, Think, and Understand
- When Attention Sink Emerges in Language Models: An Empirical View
- WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMs
- GPT-4o System Card
- See What You Are Told: Visual Attention Sink in Large Multimodal Models
- Map the Flow: Revealing Hidden Pathways of Information in VideoLLMs
- Enhancing Multi-Image Understanding through Delimiter Token Scaling
- DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression
- LLaVA-OneVision: Easy Visual Task Transfer
- OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents