Speech Generation Speaker Poisoning: Capability Erasure in Zero-Shot Text-to-Speech

arXiv:2603.07551 · cs.SD, cs.AI · Submitted 2026-03-08 · Read on arXiv

cs.SD, cs.AI

Submitted: 2026-03-08

Updated: 2026-09-14

Comments: SLT2026

Code: https://github.com/openai/whisper

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• AUC — 15× in this paper · explained in 2 episodes
• Machine Unlearning — 7× in this paper · explained in 2 episodes
• Cosine Similarity — 6× in this paper · explained in 3 episodes
• Embeddings — 5× in this paper · explained in 2 episodes
• Distillation — 4× in this paper · explained in 2 episodes
• Encoder-Guided Poisoning — 4× in this paper
• Generative AI — 4× in this paper · explained in 2 episodes
• Speech Generation Speaker Poisoning — 4× in this paper
• Zero-shot Text-to-Speech — 4× in this paper
• Consistency — 3× in this paper · explained in 3 episodes
• Contrastive Learning — 3× in this paper · explained in 5 episodes
• Generalization — 3× in this paper · explained in 13 episodes
• Knowledge Distillation — 3× in this paper · explained in 11 episodes
• Teacher-Guided Poisoning — 3× in this paper
• Verification — 3× in this paper · explained in 2 episodes
• Adversarial Attacks — 2× in this paper · explained in 1 episode
• Area Under the Curve — 2× in this paper
• Diffusion Model — 2× in this paper · explained in 8 episodes
• Fidelity — 2× in this paper · explained in 2 episodes
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• Generative Model — 2× in this paper · explained in 2 episodes
• Generative Models — 2× in this paper · explained in 2 episodes
• IEEE Symposium on Security and Privacy — 2× in this paper
• Latent Space — 2× in this paper · explained in 7 episodes
• Zero-Shot Generalization — 2× in this paper · explained in 5 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Actor — 1× in this paper · explained in 3 episodes
• AdamW Optimizer — 1× in this paper · explained in 1 episode
• Adversarial Training — 1× in this paper · explained in 1 episode
• Area Under the Curve (AUC) — 1× in this paper · explained in 1 episode
• Benchmark — 1× in this paper · explained in 2 episodes
• Clustering — 1× in this paper · explained in 2 episodes
• Loss Function — 1× in this paper · explained in 2 episodes
• Mean Opinion Score — 1× in this paper
• Office of the Director of National Intelligence — 1× in this paper
• Word Error Rate — 1× in this paper

Related papers

• Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement
• Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems
• Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stems
• Harmonic Representations in Symbolic Music Transformers are Localized and Controllable
• AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
• SoundWeaver: Compositional Warm-Starting for Text-to-Audio Diffusion Serving