Multimodal Duplex Interaction Agent

arXiv:2609.08977 · eess.AS, cs.AI, cs.LG, cs.MM, cs.SD · Submitted 2026-09-08 · Read on arXiv

eess.AS, cs.AI, cs.LG, cs.MM, cs.SD

Submitted: 2026-09-08

Updated: 2026-09-12

Comments: Project Page: https://Omni-Interaction-Gander.github.io/Omni-Interaction-Agent

Project page: https://omni-interaction-gander.github.io/Omni-Interaction-Agent

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Agentic Task — 1× in this paper · explained in 1 episode
• Agentic Tasks — 1× in this paper · explained in 1 episode
• Full-duplex interaction — 1× in this paper · explained in 2 episodes
• Robustness — 1× in this paper · explained in 5 episodes

Related papers

• Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs
• X-VC: Zero-shot Streaming Voice Conversion in Codec Space
• Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers
• VM-ARRAYDPS: Virtual Microphone Augmented Diffusion Posterior Sampling for Unsupervised Blind Speech Separation
• Conditional Flow Matching for Visually-Guided Acoustic Highlighting
• Anonymization, Not Elimination: Utility-Preserved Speech Anonymization