Adapting Personalized Speech Enhancement for Low-Latency Audio-Visual Target-Speaker Extraction

arXiv:2609.30631 · eess.AS, cs.CV, cs.SD · Submitted 2026-09-24 · Read on arXiv

eess.AS, cs.CV, cs.SD

Submitted: 2026-09-24

Updated: 2026-09-24

Code: https://github.com/resemble-ai/Resemblyzer

Terminology

• Fine-Tuning — 12× in this paper · explained in 5 episodes
• Accuracy — 6× in this paper · explained in 1 episode
• Adam — 2× in this paper · explained in 2 episodes
• Benchmark — 2× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Aggregation — 1× in this paper · explained in 1 episode
• Calibration — 1× in this paper · explained in 7 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Decoupling — 1× in this paper · explained in 2 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes

Sources

• ICASSP 2023 Deep Noise Suppression Challenge
• LRS3-TED: a large-scale dataset for visual speech recognition
• Plug-and-Steer: Decoupling Separation and Selection in Audio-Visual Target Speaker Extraction
• Robust Speech Recognition via Large-Scale Weak Supervision

Related papers

• Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs
• X-VC: Zero-shot Streaming Voice Conversion in Codec Space
• Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers
• LiveAnon: Improving Privacy and Utility in Streaming Speaker Anonymization with Codec Language Models
• EarLog: Opportunistic Ear Physiology Sensing toward Personalized Hearing Risk Awareness
• VM-ARRAYDPS: Virtual Microphone Augmented Diffusion Posterior Sampling for Unsupervised Blind Speech Separation