Preference Optimization for Non-Verbal Vocalization Synthesis

arXiv:2608.24163 · eess.AS, cs.AI, cs.LG · Submitted 2026-08-25 · Read on arXiv

eess.AS, cs.AI, cs.LG

Submitted: 2026-08-25

Updated: 2026-08-25

Code: https://github.com/resemble-ai/Resemblyzer

Project page: https://nvvspeech-challenge.github.io

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Accuracy — 9× in this paper · explained in 1 episode
• Benchmark — 4× in this paper · explained in 2 episodes
• Direct Preference Optimization — 3× in this paper
• Fidelity — 3× in this paper · explained in 2 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Affective Computing — 2× in this paper · explained in 1 episode
• Alignment — 2× in this paper
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Supervised Fine-Tuning (SFT) — 2× in this paper · explained in 8 episodes
• Automatic Speech Recognition (ASR) — 1× in this paper · explained in 2 episodes
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Diffusion Model — 1× in this paper · explained in 10 episodes
• Diffusion Models — 1× in this paper · explained in 9 episodes
• Direct Preference Optimization (DPO) — 1× in this paper · explained in 10 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Flow Matching — 1× in this paper · explained in 16 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Latent Diffusion Model — 1× in this paper · explained in 2 episodes
• Latent Diffusion Models — 1× in this paper · explained in 2 episodes

Sources

• Qwen3-TTS Technical Report
• NonverbalTTS: A Public English Corpus of Text-Aligned Nonverbal Vocalizations with Emotion Annotations for Text-to-Speech
• A Scalable Pipeline for Enabling Non-Verbal Speech Generation and Understanding
• NV-Bench: Benchmark of Nonverbal Vocalization Synthesis for Expressive Text-to-Speech Generation
• NVV-SuperBench: Beyond Words, Beyond Quality-Benchmarking Nonverbal Vocalizations in Speech Generation
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• Improving Code-Switching ASR with Code-Mixing Guided Synthetic Speech
• Fish Audio S2 Technical Report
• CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models
• CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training
• FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs

Related papers

• Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs
• X-VC: Zero-shot Streaming Voice Conversion in Codec Space
• Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers
• LiveAnon: Improving Privacy and Utility in Streaming Speaker Anonymization with Codec Language Models
• EarLog: Opportunistic Ear Physiology Sensing toward Personalized Hearing Risk Awareness
• VM-ARRAYDPS: Virtual Microphone Augmented Diffusion Posterior Sampling for Unsupervised Blind Speech Separation