Beyond Utterance-Level Speaker Similarity: Perception-Aligned Segmental Evaluation of Accent in Speech Generation
eess.AS, cs.CL, cs.SD
Submitted: 2026-10-27
Updated: 2026-10-27
License: http://creativecommons.org/licenses/by/4.0/
The gist: Utterance-level similarity ratings do not reveal where perceived accent differences occur in generated speech.
Abstract
Utterance-level similarity ratings do not reveal where perceived accent differences occur in generated speech. We introduce a segmental evaluation protocol combining speaker and accent similarity ratings with subword annotations of perceived accent differences provided by listeners from target accent regions. Using newly collected recordings, 47 retained listeners evaluate 200 content-matched reference--candidate pairs spanning four Scottish and Australian voices, two zero-shot TTS systems, and an articulation-based voice conversion system. Subword annotations locate salient pronunciation differences and sometimes yield different system rankings from similarity ratings. Speaker and accent ratings remain strongly associated (ρ= 0.71), highlighting the value of examining local perceptual evidence beyond these scores. Evaluation of seven objective distances reveals weak correspondence with subword selections, with correlations reaching at most ρ= 0.22. We release code, data, and an interactive demo to support the development of perceptually grounded segmental accent evaluation.
Related papers
- WavCube: Unifying Speech Representation for Understanding and Generation via Semantic-Acoustic Joint Modeling
- From perception to production: how acoustic invariance facilitates articulatory learning in a self-supervised vocal imitation model
- Read What You Hear: Reference-Free Hypotheses Evaluation with Acoustic Discrepancy
- pyFDN: A Python Toolbox for Feedback Delay Network Design, Analysis, and Optimization
- GigaSpeechBench: A Real-World Multilingual Speech-to-Text Benchmark
- DM-ASR: Diarization-aware Multi-speaker ASR with Unified Offline and Online Decoding with Large Language Models