Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech: A Case Study
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech".
Jane: Here's the gist of the paper. In this study, the paper compares human and automatic recognition of Dutch continuous speech to develop personalized dysarthric speech recognition (DSR) models, which is important because personalized DSR models substantially outperform human listeners on dysarthric speech Study Goal and Motivation In our goal to develop personalised dysarthric speech recognition (DSR) models, this study compared the recognition performances of human listeners and those of three state-of-the-art, off-the-shelf ASR systems (Whisper-large-V3, Google Chirp 3, and Omnilingual) on the recognition of Dutch continuous read and spontaneous speech from a single speaker with severe dysarthria The overarching goal of our work is to develop personalised DSR technology for dysarthric speakers that can be used in interpersonal communication, i.e., communication between a dysarthric speaker and naive listeners Experimental Setup The stimuli for the listening and ASR experiments were taken from the DysOne dataset for personalised DSR model development The DysOne dataset is a bilingual (native Dutch and non-native English) and bimodal (speech and video) dataset containing recordings of a 35-year-old male Dutch speaker with severe dysarthria resulting from acquired brain injury due to high-impact trauma The stimuli consisted of 188 Dutch utterances (with 686 unique words) selected as stimuli for our experiments, consisting of generic read speech, customized read speech, and spontaneous speech Performance Comparison Results showed that both humans listeners and the three off-the-shelf ASR systems exhibit word error rates (WER) exceeding 70% on average, indicating that DSR is highly challenging for both humans and ASR systems Fine-tuning on the dysarthric speech significantly reduced WER Although overall WERs are still quite high (>23%), the personalised DSR models outperformed the human listeners, and performance is getting closer to being useful for supporting day-to-day communication of dysarthric speakers Key Findings on Model Performance 1.
Tom: Both the human listeners and the off-the-shelf ASR models had substantial difficulty in recognizing the dysarthric speech, particularly for customized read speech, as shown by the high WERs 2.
Jane: The Whisper model significantly outperformed the Google Chirp (95% CI,
+8.197%, +19.215%: , p <.001) and the OmniASR (95% CI,
+11.099%, +20.558%: , p <.001) models for generic read speech 3.
Tom: After fine-tuning on the target speaker’s speech, both personalised DSR models (FT-Whisper and FT-OmniASR) outperformed the human listeners by a large margin with an average absolute improvement of WER by more than 34% Areas for Future Research Future research should focus on improving personalized DSR on spontaneous speech and longer utterances in the case of read speech, with a specific focus on particular phonemes The phoneme error analysis showed that future research should focus on improving the DSR recognition performance on a particular set of Dutch phonemes, including /Z/, /S/, /E:/, /f/, /y/, /N/, and /j/ The paper concludes that the recognition of both read and spontaneous speech of a single Dutch speaker with severe dysarthria is highly challenging for both native human listeners and SotA off-the-shelf ASR models Methodological Details The procedure involved dividing the 188 utterances into four lists of 47 utterances each, with each list assigned randomly to a listener and listened to by five different listeners For automatic speech recognisers, Whisper-large-V3 was selected because it has been reported to achieve the best performance on dysarthric speech in comparison to eleven typical ASR models The Omnilingual ASR model was included because it has not yet been tested on dysarthric speech The evaluation involved statistical significance tests using a paired nonparametric bootstrap test with 10,000 utterance-based samples and 95% confidence intervals Error Analysis Insights In the error analyses, we focus on the human listeners and the best off-the-shelf and fine-tuned models: Whisper and FT-Whisper For particularly the /S/ and /ø:/, / / /a / / / e/i/o/u/y/, the phoneme error rates (PERs) of FT-Whisper are substantially lower The results suggest that when human listeners do not understand an utterance, they tend not to write anything down, whereas ASR systems typically output something whenever speech is detected The analysis of listening times suggested that for most listeners, the play times were higher for the more difficult utterances In conclusion, the recognition of both read and spontaneous speech of a single Dutch speaker with severe dysarthria is highly challenging for both native human listeners and SotA off-the-shelf ASR models Personalised DSR models, however, substantially outperform human listeners Though the performance remains far below SotA performance on typical speech, it is getting closer to being helpful for supporting the day-to-day communication of dysarthric speakers In addition, personalised DSR systems can be particularly helpful for naive human listeners in understanding dysarthric speech The paper is 2606.30237.
Title and authors: Jane: Let's start by looking at the title again, Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech: A Case Study. It sounds very specific, which is good for understanding exactly what this research is about.
Tom: Yeah, it sets the stage clearly. The paper focuses on comparing human performance against automated recognition systems for Dutch speech that's spoken by someone with severe dysarthria.
Lu: The authors are Yuanyuan Zhang, Dimme de Groot, Jorge Martinez Castaneda, and Odette Scharenborg from the Multimedia Computing Group at Delft University of Technology in the Netherlands. They bring a strong background in multimedia computing and speech recognition.
Meng: I'm curious what they are really trying to achieve with this specific comparison. Is it just showing that AI is bad at this, or are they proposing a path forward?
Lalam: The overarching goal stated in the paper is to develop personalized dysarthric speech recognition models for use in interpersonal communication, specifically between a dysarthric speaker and naive listeners.
Tom: So the implication there isn't just about improving accuracy; it’s fundamentally about enabling practical communication where it was previously very difficult. It moves beyond just a technical benchmark.
Jane: Exactly. They aren't just measuring error rates; they are looking at how this technology can be applied to help people who are struggling with speaking clearly because of their disability.
Lu: The paper builds on previous work, specifically mentioning comparisons made by Mengistu et al in two thousand eleven and Green et al in two thousand twenty-one regarding personalized models versus human performance on English read speech <ref:2606.30237#pg2>.
Meng: So they're connecting this Dutch study to a broader trend where personalized AI is starting to show promise, even if the results are still challenging.
Lalam: That connection is important because it suggests that moving away from just reading isolated words toward spontaneous speech is the next logical step for this kind of work.
The paper's summary: Tom: So to summarize what we've covered, the paper compared human listeners and three ASR systems—Whisper large V three, Google Chirp three, and Omnilingual—on Dutch continuous read and spontaneous speech from one speaker with severe dysarthria.
Jane: The key takeaway here is that both humans and the off-the-shelf ASR systems had word error rates over seventy percent on average initially, which shows how difficult this speech is for everyone involved.
Lu: But the summary also emphasizes that fine-tuning these models significantly reduced those word error rates, and although they still were high above twenty three percent, the personalized dysarthric speech recognition models outperformed the human listeners and were getting closer to being useful for supporting day to day communication.
Meng: So, personalization is key here. It's not just a general model; it has to be tailored to that specific speaker's speech characteristics for the performance boost they see.
Lalam: And another important summary point is that after fine-tuning, both personalized dysarthric speech recognition models significantly outperformed the human listeners by a large margin, with an average absolute improvement in word error rates of over thirty four percent.
Tom: That thirty four percent improvement is what really drives the interest here. It's not just marginal gains; it’s a substantial jump that makes a difference in how usable the technology becomes for people.
Jane: And they also noted that personalized DSR systems can be particularly helpful for naive human listeners who want to understand dysarthric speech, as they significantly outperformed the Omnilingual ASR model on all three speech types when compared to human listeners.
The paper's improvements: Lu: The paper suggests a few clear directions for future research. First, they should focus on improving personalized DSR recognition performance on spontaneous speech and longer utterances in the case of read speech.
Meng: They also identified that there is a specific focus needed on certain Dutch phonemes for future work, like /Z/, /S/, /E:/, /f/, /y/, /N/, and the letter j. This is where they saw the most significant improvement in error rates after fine-tuning compared to off-the-shelf models.
Tom: So it’s not just about getting better at general recognition; it’s about targeting specific phonetic weaknesses to really sharpen those personalized models. That makes a lot of sense from a machine learning standpoint.
Jane: It also suggests that the error analysis showed that when human listeners don't understand an utterance, they tend not to write anything down, whereas ASR systems typically output something whenever speech is detected.
Lalam: That observation about how humans react—not writing anything down if they don't understand—is a really insightful piece of information for understanding the communication gap.
Tom: And that leads into the idea of what these personalized models can actually do in practice, which is supporting day to day communication. The paper suggests that this technology can be particularly helpful for naive human listeners in understanding dysarthric speech.
Conclusion: Jane: So wrapping things up, the main message from Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech: A Case Study is that while both human listeners and state-of-the-art off-the-shelf ASR systems struggle with this speech, personalized DSR models substantially outperform human listeners.
Tom: That’s the core finding we need to remember: personalization makes a real difference, even if the final word error rates are still high compared to typical speech benchmarks. The performance is getting closer to being helpful for supporting day to day communication of dysarthric speakers.
Lu: From my side, I think the focus on specific phonemes like /Z/ and /S/ is crucial because that’s where the fine-tuning actually delivered substantial gains over the general models they compared against.
Meng: For deployment, it suggests that we need to keep pushing personalization efforts because speaker-dependent models are consistently better for speakers with severe dysarthria, which means they're probably more reliable than what we've seen in general benchmarks.
Lalam: And I just want to reiterate that personalized DSR systems can be particularly helpful for naive human listeners in understanding dysarthric speech, which is a really practical application we should focus on.
Tom: So, in the end, the paper on Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech: A Case Study shows that personalization substantially outperforms human listeners in this challenging scenario. It’s a step toward making AI tools genuinely helpful for people with severe dysarthria.
Yuanyuan Zhang, Dimme de Groot, Jorge Martinez, Odette Scharenborg
Delft University of Technology
cs.CL
Submitted: 2026-06-29
Updated: 2026-11-12
Code: https://github.com/karkirowle/relative
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Importance score: 77/100
The gist: In this study, the paper compares human and automatic recognition of Dutch continuous speech to develop personalized dysarthric speech recognition (DSR) models…
Key concepts
- Dysarthric Speech Recognition (DSR)
- This is the technology being developed to understand speech produced by people with dysarthria, a speech disorder caused by brain injury. The goal is to create personalized models that can accurately transcribe this difficult speech for communication purposes.
- Word Error Rate (WER)
- WER measures how many words an automatic system gets wrong when transcribing spoken audio. High WERs, often above 70% in this study, indicate that the speech is very challenging for both humans and current ASR technology.
- Personalized DSR Models
- These are ASR models that have been specifically 'fine-tuned' using recordings from the target speaker with dysarthria. This customization allows the model to learn the unique acoustic characteristics of that individual's speech, leading to much better recognition than general models.
Terminology
Summary
In this study, the paper compares human and automatic recognition of Dutch continuous speech to develop personalized dysarthric speech recognition (DSR) models, which is important because personalized DSR models substantially outperform human listeners on dysarthric speech
Study Goal and Motivation
In our goal to develop personalised dysarthric speech recognition (DSR) models, this study compared the recognition performances of human listeners and those of three state-of-the-art, off-the-shelf ASR systems (Whisper-large-V3, Google Chirp 3, and Omnilingual) on the recognition of Dutch continuous read and spontaneous speech from a single speaker with severe dysarthria The overarching goal of our work is to develop personalised DSR technology for dysarthric speakers that can be used in interpersonal communication, i.e., communication between a dysarthric speaker and naive listeners
Experimental Setup
The stimuli for the listening and ASR experiments were taken from the DysOne dataset for personalised DSR model development The DysOne dataset is a bilingual (native Dutch and non-native English) and bimodal (speech and video) dataset containing recordings of a 35-year-old male Dutch speaker with severe dysarthria resulting from acquired brain injury due to high-impact trauma The stimuli consisted of 188 Dutch utterances (with 686 unique words) selected as stimuli for our experiments, consisting of generic read speech, customized read speech, and spontaneous speech
Performance Comparison
Results showed that both humans listeners and the three off-the-shelf ASR systems exhibit word error rates (WER) exceeding 70% on average, indicating that DSR is highly challenging for both humans and ASR systems Fine-tuning on the dysarthric speech significantly reduced WER Although overall WERs are still quite high (>23%), the personalised DSR models outperformed the human listeners, and performance is getting closer to being useful for supporting day-to-day communication of dysarthric speakers
Key Findings on Model Performance
-
Both the human listeners and the off-the-shelf ASR models had substantial difficulty in recognizing the dysarthric speech, particularly for customized read speech, as shown by the high WERs
-
The Whisper model significantly outperformed the Google Chirp (95% CI, [+8. p. 3
197%, +19.215%], p <. p. 3Where the paper says it
These results indicate substantial variability in recognition performance across naive listeners. Comparison of the off-the-shelf ASR models showed that the Whisper model significantly outperformed the Google Chirp (95% CI, [+8.
001) and the OmniASR (95% CI, [+11. p. 3Where the paper says it
Comparison of the off-the-shelf ASR models showed that the Whisper model significantly outperformed the Google Chirp (95% CI, [+8. 215%], p < .
099%, +20.558%], p <. p. 3Where the paper says it
215%], p < . 001) and the OmniASR (95% CI, [+11.
001) models for generic read speechWhere the paper says it
001) and the OmniASR (95% CI, [+11. 558%], p < .
-
After fine-tuning on the target speaker’s speech, both personalised DSR models (FT-Whisper and FT-OmniASR) outperformed the human listeners by a large margin with an average absolute improvement of WER by more than 34%
Areas for Future Research
Future research should focus on improving personalized DSR on spontaneous speech and longer utterances in the case of read speech, with a specific focus on particular phonemes The phoneme error analysis showed that future research should focus on improving the DSR recognition performance on a particular set of Dutch phonemes, including /Z/, /S/, /E:/, /f/, /y/, /N/, and /j/ The paper concludes that the recognition of both read and spontaneous speech of a single Dutch speaker with severe dysarthria is highly challenging for both native human listeners and SotA off-the-shelf ASR models
Methodological Details
The procedure involved dividing the 188 utterances into four lists of 47 utterances each, with each list assigned randomly to a listener and listened to by five different listeners For automatic speech recognisers, Whisper-large-V3 was selected because it has been reported to achieve the best performance on dysarthric speech in comparison to eleven typical ASR models The Omnilingual ASR model was included because it has not yet been tested on dysarthric speech The evaluation involved statistical significance tests using a paired nonparametric bootstrap test with 10,000 utterance-based samples and 95% confidence intervals
Error Analysis Insights
In the error analyses, we focus on the human listeners and the best off-the-shelf and fine-tuned models: Whisper and FT-Whisper For particularly the /S/ and /ø:/, / / /a / / / e/i/o/u/y/, the phoneme error rates (PERs) of FT-Whisper are substantially lower The results suggest that when human listeners do not understand an utterance, they tend not to write anything down, whereas ASR systems typically output something whenever speech is detected The analysis of listening times suggested that for most listeners, the play times were higher for the more difficult utterances
Conclusion
In conclusion, the recognition of both read and spontaneous speech of a single Dutch speaker with severe dysarthria is highly challenging for both native human listeners and SotA off-the-shelf ASR models Personalised DSR models, however, substantially outperform human listeners Though the performance remains far below SotA performance on typical speech, it is getting closer to being helpful for supporting the day-to-day communication of dysarthric speakers In addition, personalised DSR systems can be particularly helpful for naive human listeners in understanding dysarthric speech The paper is 2606.30237.
--- Page 1 ---
Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech: A Case Study This paper is 2606.30237. The speech recognition performance of human listeners has long been considered the upper bound for automatic speech recognition (ASR) performance Their performance has been compared by researchers on a variety of tasks, including recognizing phonemes [1], isolated words [2], read speech [2], [3], and spontaneous speech [2], [4], and in quiet and in background noise [2], [3] With the rapid development of deep learning techniques [6]–[8], ASR systems have improved significantly over the past decade and are now widely used in diverse applications, including voice assistants [9]–[11], search engines [12], and health-related applications [13], [14] For typical speech, defined by Zhang et al. p. 0 With the rapid development of deep learning techniques [6]–
[8], ASR systems have improved significantly over the past
decade and are now widely used in diverse applications,
including voice assistants [9]–[11], search engines [12], and
health-related applications [13], [14]. For typical speech, defined by Zhang et al. as “adult, typically highly-educated, firstlanguage speakers of a standardized language variety, without
a speech disability” [15], state-of-the-art (SotA) ASR systems
have achieved performance on par with or even better than that
of humans [4], [16], [17]. For example, recent ASR systems
have reached word error rates (WERs) of below 5% on conversational speech benchmarks such as Switchboard [18], [19]
and below 3% on read speech [16], [20]. This is particularly the case for dysarthric speakers. In 2019,
a comparison of three ASR cloud platforms [23] found that
the best model on English dysarthric speech, the Google cloud
model, achieved a WER of 59. 8% on the TORGO dataset [23],
[24]. Results from a 2025 study [25] on TORGO showed
that Whisper-large-v3 [16] achieved the best performance
among eight leading commercial ASR services, including
AssemblyAI, GPT, and Gemini, still with a WER as high as
49.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 2 ---
the difficulty that human listeners have with recognising dysarthric read speech. Nevertheless, a comparison of human and personalised DSR model performance on dysarthric speech by Mengistu et al. p. 1 the difficulty that human listeners have with recognising
dysarthric read speech. Nevertheless, a comparison of human and personalised DSR model performance on dysarthric
speech by Mengistu et al. (2011) found that naive human
listeners outperformed personalised DSR models on English
read isolated words [28]. More recently, in 2021, Green et
al. A similar finding was
reported for Mandarin read sentences [30], where personalized
DSR models outperformed experienced listeners for five out
of seven dysarthric speakers on read speech. The overarching goal of our work is to develop personalised
DSR technology for dysarthric speakers that can be used in
interpersonal communication, i. In this paper,
we benchmark our DSR models against human performance,
as such, this paper builds on and extends previous research
comparing human and automatic recognition of dysarthric
speech. In our work, we investigate recognition performance
of the latest SotA off-the-shelf ASR models and personalised DSR models on the dysarthric speech of a single
Dutch speaker with severe dysarthria in comparison to naive
human listeners on the exact same stimuli.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 3 ---
All listening experiments were conducted in a soundattenuated booth using the same headphones (Sennheiser HD 200 Pro) and laptop Prior to the experiment, listeners received instructions, signed an informed consent form, and filled in a questionnaire for meta information, including gender, age, and familiarity with dysarthric speech The 188 utterances were divided into four lists of 47 utterances each Each list contained ten generic read utterances with (1,5] words, ten with (5,10] words, and ten or eleven with (10,15] words, six or seven customized read utterances, and ten spontaneous utterances
--- Page 4 ---
To explore the effect of speech type and utterance length (word count) on the recognition performance of the human listeners and ASR models, a multi-factor linear model was applied using the estimated marginal means package (emmeans) in R [56] Moreover, we conducted word-level and phoneme-level error analyses and investigated the effect of the number of replays of the stimuli on recognition performance
--- Page 5 ---
To ensure consistency between human transcriptions and those of the ASR systems, all transcriptions were converted to lowercase, punctuation was removed (except for the apostrophe), extra spaces were removed, digits or numbers were written out in full, and all non-linguistic symbols, filler words, and other transcriptions of non-lexical sounds were removed Speech recognition performance is measured in WER For the recognition performance comparisons of the human listeners and ASR models, we conducted statistical significance tests using a paired nonparametric bootstrap test with 10,000 utterance-based samples and 95% confidence intervals (CIs) [53], [54] To explore the effect of speech type and utterance length (word count) on the recognition performance of the human listeners and ASR models, a multi-factor linear model was applied using the estimated marginal means package (emmeans) in R [56] Moreover, we conducted word-level and phoneme-level error analyses and investigated the effect of the number of replays of the stimuli on recognition performance
--- Page 6 ---
In our work, we investigate recognition performance of the latest SotA off-the-shelf ASR models and personalised DSR models on the dysarthric speech of a single Dutch speaker with severe dysarthria in comparison to naive human listeners on the exact same stimuli The best-performing model was the fine-tuned Whisper model (FT-Whisper) FT-Whisper significantly outperformed the fine-tuned OmniASR model (FT-OmniASR) on generic read speech (95% CI, [+8.984%, +18.919%], p <. p. 3 FT-Whisper significantly outperformed
the fine-tuned OmniASR model (FT-OmniASR) on generic
read speech (95% CI, [+8. 919%], p < . FT-Whisper significantly outperformed
the fine-tuned OmniASR model (FT-OmniASR) on generic
read speech (95% CI, [+8. 919%], p < . 858%], p =
. Interestingly, both models significantly outperformed
the human listeners and the Whisper model for all speech
types (p < . 858%], p =
. Interestingly, both models significantly outperformed
the human listeners and the Whisper model for all speech
types (p < .Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 7 ---
In conclusion, the recognition of both read and spontaneous speech of a single Dutch speaker with severe dysarthria is highly challenging for both native human listeners and SotA off-the-shelf ASR models Personalised DSR models, however, substantially outperform human listeners Though the performance remains far below SotA performance on typical speech, it is getting closer to being helpful for supporting the day-to-day communication of dysarthric speakers In addition, personalised DSR systems can be particularly helpful for naive human listeners in understanding dysarthric speech
--- Page 8 ---
The best-performing model was the fine-tuned Whisper model (FT-Whisper) FT-Whisper significantly outperformed the fine-tuned OmniASR model (FT-OmniASR) on generic read speech (95% CI, [+8.984%, +18.919%], p <.001) Interestingly, both models significantly outperformed the human listeners and the Whisper model for all speech types (p <.001)
--- Page 9 ---
In conclusion, the recognition of both read and spontaneous speech of a single Dutch speaker with severe dysarthria is highly challenging for both native human listeners and SotA off-the-shelf ASR models Personalised DSR models, however, substantially outperform human listeners Though the performance remains far below SotA performance on typical speech, it is getting closer to being helpful for supporting the day-to-day communication of dysarthric speakers In addition, personalised DSR systems can be particularly helpful for naive human listeners in understanding dysarthric speech
--- Page 10 ---
The authors thank Wenyi Chu for her help and support with finding human listener participants During the preparation of this work the authors used Generative AI (ChatGPT) to improve language and readability and to debug Python and R scripts for statistical significance tests After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication The paper is 2606.30237.
--- Page 11 ---
The authors thank Wenyi Chu for her help and support with finding human listener participants During the preparation of this work the authors used Generative AI (ChatGPT) to improve language and readability and to debug Python and R scripts for statistical significance tests After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication The paper is 2606.30237.
--- Page 12 ---
The authors thank Wenyi Chu for her help and support with finding human listener participants During the preparation of this work the authors used Generative AI (ChatGPT) to improve language and readability and to debug Python and R scripts for statistical significance tests After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication The paper is 2606.30237.
--- Page 13 ---
The authors thank Wenyi Chu for her help and support with finding human listener participants During the preparation of this work the…
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Personalized DSR models can substantially outperform human listeners by achieving an average absolute improvement of WER by more than 34% compared to naive human listeners after fine-tuning on target speaker speech. This allows for better supporting day-to-day communication of dysarthric speakers.
-
Fine-tuning models on the dysarthric speech, such as FT-Whisper, can remove the greater challenge of customized read speech for ASR systems, suggesting that personalized models may improve dysarthric speakers’ ability to communicate about their favorite topics in personalized communication support systems.
-
Future DSR improvements should focus on improving recognition performance on a particular set of Dutch phonemes including /Z/, /S/, /E:/, /f/, /y/, /N/, and/or/j/. This targets the phonemes where the best fine-tuned model showed substantially lower phoneme error rates compared to off-the-shelf models.
-
Personalized DSR systems can be particularly helpful for naive human listeners in understanding dysarthric speech, as they are shown to significantly outperform the Omnilingual ASR model on all three speech types when compared to human listeners.
Abstract
In our goal to develop personalised dysarthric speech recognition (DSR) models, this study compared the recognition performances of human listeners and those of three state-of-the-art, off-the-shelf ASR systems (Whisper-large-V3, Google Chirp 3, and Omnilingual) on the recognition of Dutch continuous read and spontaneous speech from a single speaker with severe dysarthria. Results showed that both humans listeners and the three off-the-shelf ASR systems exhibit word error rates (WER) exceeding 70% on average, indicating that DSR is highly challenging for both humans and ASR systems. Fine-tuning on the dysarthric speech significantly reduced WER. Although overall WERs are still quite high (>23%), the personalised DSR models outperformed the human listeners, and performance is getting closer to being useful for supporting day-to-day communication of dysarthric speakers. Future research should focus on improving personalized DSR on spontaneous speech and longer utterances in the case of read speech, with a specific focus on particular phonemes.
Sources
- Achieving Human Parity in Conversational Speech Recognition
- Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
- Zero-Shot Recognition of Dysarthric Speech Using Commercial Automatic Speech Recognition and Multimodal Large Language Models
- CDSD: Chinese Dysarthria Speech Database
- Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languages
- Granary: Speech Recognition and Translation Dataset in 25 European Languages
- Good practices for evaluation of machine learning systems
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data