Supervising Sound Localization by In-the-wild Egomotion

arXiv:2610.01388 · cs.CV, cs.AI, cs.MM, cs.SD · Submitted 2026-10-01 · Read on arXiv

cs.CV, cs.AI, cs.MM, cs.SD

Submitted: 2026-10-01

Updated: 2026-10-01

Comments: CVPR 2025 Highlight (IEEE/CVF Conference on Computer Vision and Pattern Recognition)

Journal ref: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Accuracy — 11× in this paper · explained in 1 episode
• European Conference on Computer Vision — 5× in this paper
• Interaural Intensity Difference — 5× in this paper
• Alignment — 4× in this paper · explained in 1 episode
• Conference on Computer Vision and Pattern Recognition — 4× in this paper
• Benchmark — 2× in this paper · explained in 2 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Loss Function — 2× in this paper · explained in 2 episodes
• Ablation — 1× in this paper · explained in 1 episode
• AdamW Optimizer — 1× in this paper · explained in 1 episode
• Adaptability — 1× in this paper · explained in 1 episode
• Calibration — 1× in this paper · explained in 7 episodes
• Data Augmentation — 1× in this paper · explained in 7 episodes
• Direction of Arrival — 1× in this paper
• Embodied AI — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Real-World Audio-Visual Scene — 1× in this paper
• Representation Learning — 1× in this paper · explained in 2 episodes
• Self-Supervised Learning — 1× in this paper · explained in 8 episodes

Sources

• A multi-room reverberant dataset for sound event localization and detection
• Audio-Visual Synchronisation in the wild
• Structure from Silence: Learning Scene Structure from Ambient Sound
• BatVision with GCC-PHAT Features for Better Sound to Vision Predictions
• FMA: A Dataset For Music Analysis
• Self-Supervised Video Forensics by Audio-Visual Anomaly Detection
• Synchformer: Efficient Synchronization from Sparse Cues
• Self-Supervised Generation of Spatial Audio for 360 Video
• A Dataset of Reverberant Spatial Sound Scenes with Moving Sources for Sound Event Localization and Detection
• A Dataset of Dynamic Reverberant Sound Scenes with Directional Interferers for Sound Event Localization and Detection
• STARSS22: A dataset of spatial recordings of real scenes with spatiotemporal annotations of sound events
• Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI
• BAT: Learning to Reason about Spatial Sounds with Large Language Models

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models