Hear the World in Stereo: Learning Dynamic Spatial Correspondence for Immersive Joint Video-Audio Generation

arXiv:2609.38748 · cs.CV · Submitted 2026-09-30 · Read on arXiv

cs.CV

Submitted: 2026-09-30

Updated: 2026-10-12

Project page: https://vivocameraresearch.github.io/Stereo-Bind-Project

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Consistency — 21× in this paper · explained in 3 episodes
• Transformer — 14× in this paper · explained in 2 episodes
• Grounding — 11× in this paper · explained in 3 episodes
• Alignment — 9× in this paper · explained in 1 episode
• Computer Vision and Pattern Recognition — 9× in this paper
• Spatial Track Encoder — 8× in this paper
• Verification — 8× in this paper · explained in 2 episodes
• Fidelity — 7× in this paper · explained in 2 episodes
• Benchmark — 6× in this paper · explained in 2 episodes
• Ablation — 4× in this paper · explained in 1 episode
• Diffusion Model — 4× in this paper · explained in 8 episodes
• Diffusion Models — 4× in this paper · explained in 6 episodes
• Inter-channel Differential Spectrogram — 4× in this paper
• Visual Motion Binding — 4× in this paper
• Benchmarking — 3× in this paper · explained in 3 episodes
• Calibration — 3× in this paper · explained in 7 episodes
• Core — 3× in this paper · explained in 2 episodes
• Scene Distribution Bank — 3× in this paper
• Sound Event Localization and Detection — 3× in this paper
• Spatial Data Synthesis — 3× in this paper
• Ablation study — 2× in this paper · explained in 1 episode
• Accuracy — 2× in this paper · explained in 1 episode
• Aggregation — 2× in this paper · explained in 1 episode
• Foundation Model — 2× in this paper · explained in 3 episodes
• Short-Time Fourier Transform — 2× in this paper
• Wasserstein Distance — 2× in this paper · explained in 4 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Dual-Stream Architecture — 1× in this paper · explained in 3 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• Generative Models — 1× in this paper · explained in 2 episodes
• International Conference on Computer Vision — 1× in this paper
• Latent Diffusion Model — 1× in this paper · explained in 2 episodes
• Latent Diffusion Models — 1× in this paper · explained in 2 episodes
• Production Quality — 1× in this paper
• Segmentation Masks — 1× in this paper · explained in 2 episodes
• Signed Interchannel Level Difference — 1× in this paper
• Unified Framework — 1× in this paper · explained in 5 episodes
• Verifier — 1× in this paper · explained in 2 episodes
• World Model — 1× in this paper · explained in 7 episodes

Sources

• AVControl: Efficient Framework for Training Audio-Visual Controls
• DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generation
• LTX-2: Efficient Joint Audio-Visual Foundation Model
• SonoWorld: From One Image to a 3D Audio-Visual Scene
• Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation
• FoleyDesigner: Immersive Stereo Foley Generation with Precise Spatio-Temporal Alignment for Film Clips
• Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation
• Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
• Seedance 2.0: Advancing Video Generation for World Complexity
• MOVA: Towards Scalable and Synchronized Video-Audio Generation
• Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Sound
• AV-DiT: Efficient Audio-Visual Diffusion Transformer for Joint Audio and Video Generation
• Qwen3-Omni Technical Report
• Qwen3 Technical Report
• BAT: Learning to Reason about Spatial Sounds with Large Language Models

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models