Distributional Metrics for Evaluating Spoken Conversational Systems

arXiv:2609.31719 · eess.AS, cs.CL, cs.SD · Submitted 2026-09-21 · Read on arXiv

eess.AS, cs.CL, cs.SD

Submitted: 2026-09-21

Updated: 2026-09-21

Comments: 5 pages, 3 figures, 1 table. Submitted to IEEE ICASSP 2027

Code: https://github.com/snakers4/silero-vad

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 6× in this paper · explained in 2 episodes
• Actor — 2× in this paper · explained in 3 episodes
• Conversational Distribution Score — 2× in this paper
• Foundation Model — 1× in this paper · explained in 3 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Multimodal Dataset — 1× in this paper · explained in 2 episodes
• Pairwise Comparisons — 1× in this paper · explained in 2 episodes
• Perplexity — 1× in this paper · explained in 3 episodes
• Silero Voice Activity Detection — 1× in this paper

Sources

• A Survey on Neural Speech Synthesis
• M3-DuplexBench: A Multi-Turn, Multilingual, Multidomain Benchmark for Full-Duplex Spoken Dialogue Models
• SPEARBench: A Benchmark for Naturalness Evaluation in Streaming Speech-to-Speech Language Models
• Moshi: a speech-text foundation model for real-time dialogue
• Reference-Based Prosody and Rhythm Evaluation for Spoken Dialogue Systems

Related papers

• Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs
• X-VC: Zero-shot Streaming Voice Conversion in Codec Space
• Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers
• VM-ARRAYDPS: Virtual Microphone Augmented Diffusion Posterior Sampling for Unsupervised Blind Speech Separation
• Conditional Flow Matching for Visually-Guided Acoustic Highlighting
• Anonymization, Not Elimination: Utility-Preserved Speech Anonymization