Hear, Localize, and Reason: Spatially Aware Scene Understanding for Audio-Visual LLMs
The work introduces HLR-AVSceneQA, a comprehensive benchmark for spatial audio-visual understanding organized by a hear-localize-reason hierarchy. It also presents HLR-LLM, an augmented foundation model featuring an explicit binaural spatial audio branch and chain-of-thought training. This combination significantly improves models' ability to localize sound sources and reason about their relationship to the visual scene without losing semantic understanding.