SciPhi-Chat: Training a Spatial Audio Agent for Question Answering, Dialogue, and Tool Use

arXiv:2611.04114 · cs.SD · Submitted 2026-11-03 · Read on arXiv

cs.SD

Submitted: 2026-11-03

Updated: 2026-11-03

Code: https://github.com/microsoft/sciphi-chat

License: http://creativecommons.org/licenses/by/4.0/

The gist: Spatial audio question answering requires associating sound identities with locations and reasoning about their relations.

Abstract

Spatial audio question answering requires associating sound identities with locations and reasoning about their relations. This paper introduces SciPhi-Chat, a spatial audio agent that answers questions, conducts multi-turn dialogue, and calls tools on first-order ambisonic recordings. The accompanying MASU benchmark contains 12,000 samples across 42 task categories, the widest task coverage of any spatial model, with real recordings and simulated audio scenes. SciPhi-Chat achieves a score of 44.9%, 7.4 percentage points above the previous state-of-the-art. On the external ASA-Bench, SciPhi-Chat achieves 74.5% azimuth and 60.0% elevation accuracy, the highest among locally evaluated models. Removing reasoning supervision reduces ASA-Bench accuracy by 17.4 percentage points. All materials will be released via https://github.com/microsoft/sciphi-chat

Sources

Related papers