Virtual-Flow: Virtual Microphone-based Speech Enhancement via Semi-supervised Flow Matching
eess.AS
Submitted: 2026-11-08
Updated: 2026-11-08
Comments: Accepted to NeurIPS 2026
License: http://creativecommons.org/licenses/by/4.0/
The gist: Neural network-based virtual microphone estimation (neural-VME) synthesizes microphone signals at unobserved spatial locations from a limited set of real microphone (RM) physical recordings.
Terminology
Abstract
Neural network-based virtual microphone estimation (neural-VME) synthesizes microphone signals at unobserved spatial locations from a limited set of real microphone (RM) physical recordings. This enables high-resolution arrays that improve spatial processing for downstream tasks such as speech enhancement. However, existing methods rely on supervised training with ground-truth signals captured at virtual locations, requiring datasets that scale with the number of virtual microphones (VM) and leaving open the question of how to design optimal large-array configurations. We propose Virtual-Flow, a flow-matching framework that implicitly generates VM signals conditioned on real recordings, without requiring ground-truth VM signals. We introduce a semi-supervised training strategy based on pseudo-VM targets constructed from time-delayed superpositions of real recordings. Virtual-Flow shows improved magnitude estimation in high-frequency bands, yielding richer spatial cues for beamforming. Experiments demonstrate that the proposed Virtual-Flow, combined with reflow distillation, outperforms existing neural-VME baselines in VM-based beamforming and achieves competitive end-to-end speech enhancement performance with low computational cost.
Sources
- EasyCom: An Augmented Reality Dataset to Support Algorithms for Easy Communication in Noisy Environments
- Spatial-Magnifier: Spatial upsampling for multichannel speech enhancement
Related papers
- WavCube: Unifying Speech Representation for Understanding and Generation via Semantic-Acoustic Joint Modeling
- From perception to production: how acoustic invariance facilitates articulatory learning in a self-supervised vocal imitation model
- Read What You Hear: Reference-Free Hypotheses Evaluation with Acoustic Discrepancy
- pyFDN: A Python Toolbox for Feedback Delay Network Design, Analysis, and Optimization
- GigaSpeechBench: A Real-World Multilingual Speech-to-Text Benchmark
- DM-ASR: Diarization-aware Multi-speaker ASR with Unified Offline and Online Decoding with Large Language Models