MuSTMM: Learning Sound-Triggered Mobile Manipulation across Multiple Scenes
MuSTMM introduces a dataset with 31 diverse scenes combining spatial audio and physical interaction for sound-triggered mobile manipulation. It augments navigation policies with explicit spatial memory, fusing depth observations into a Bird's-Eye View map to guide robots to targets based on both sound and scene structure. This method boosts success rates by leveraging visual geometry alongside acoustic cues in unseen environments.