Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech: A Case Study

arXiv:2606.30237 · cs.CL · Submitted 2026-06-29 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech".

Jane: Here's the gist of the paper. In this study, the paper compares human and automatic recognition of Dutch continuous speech to develop personalized dysarthric speech recognition (DSR) models, which is important because personalized DSR models substantially outperform human listeners on dysarthric speech Study Goal and Motivation In our goal to develop personalised dysarthric speech recognition (DSR) models, this study compared the recognition performances of human listeners and those of three state-of-the-art, off-the-shelf ASR systems (Whisper-large-V3, Google Chirp 3, and Omnilingual) on the recognition of Dutch continuous read and spontaneous speech from a single speaker with severe dysarthria The overarching goal of our work is to develop personalised DSR technology for dysarthric speakers that can be used in interpersonal communication, i.e., communication between a dysarthric speaker and naive listeners Experimental Setup The stimuli for the listening and ASR experiments were taken from the DysOne dataset for personalised DSR model development The DysOne dataset is a bilingual (native Dutch and non-native English) and bimodal (speech and video) dataset containing recordings of a 35-year-old male Dutch speaker with severe dysarthria resulting from acquired brain injury due to high-impact trauma The stimuli consisted of 188 Dutch utterances (with 686 unique words) selected as stimuli for our experiments, consisting of generic read speech, customized read speech, and spontaneous speech Performance Comparison Results showed that both humans listeners and the three off-the-shelf ASR systems exhibit word error rates (WER) exceeding 70% on average, indicating that DSR is highly challenging for both humans and ASR systems Fine-tuning on the dysarthric speech significantly reduced WER Although overall WERs are still quite high (>23%), the personalised DSR models outperformed the human listeners, and performance is getting closer to being useful for supporting day-to-day communication of dysarthric speakers Key Findings on Model Performance 1.

Tom: Both the human listeners and the off-the-shelf ASR models had substantial difficulty in recognizing the dysarthric speech, particularly for customized read speech, as shown by the high WERs 2.

Jane: The Whisper model significantly outperformed the Google Chirp (95% CI,

+8.197%, +19.215%: , p <.001) and the OmniASR (95% CI,

+11.099%, +20.558%: , p <.001) models for generic read speech 3.

Tom: After fine-tuning on the target speaker’s speech, both personalised DSR models (FT-Whisper and FT-OmniASR) outperformed the human listeners by a large margin with an average absolute improvement of WER by more than 34% Areas for Future Research Future research should focus on improving personalized DSR on spontaneous speech and longer utterances in the case of read speech, with a specific focus on particular phonemes The phoneme error analysis showed that future research should focus on improving the DSR recognition performance on a particular set of Dutch phonemes, including /Z/, /S/, /E:/, /f/, /y/, /N/, and /j/ The paper concludes that the recognition of both read and spontaneous speech of a single Dutch speaker with severe dysarthria is highly challenging for both native human listeners and SotA off-the-shelf ASR models Methodological Details The procedure involved dividing the 188 utterances into four lists of 47 utterances each, with each list assigned randomly to a listener and listened to by five different listeners For automatic speech recognisers, Whisper-large-V3 was selected because it has been reported to achieve the best performance on dysarthric speech in comparison to eleven typical ASR models The Omnilingual ASR model was included because it has not yet been tested on dysarthric speech The evaluation involved statistical significance tests using a paired nonparametric bootstrap test with 10,000 utterance-based samples and 95% confidence intervals Error Analysis Insights In the error analyses, we focus on the human listeners and the best off-the-shelf and fine-tuned models: Whisper and FT-Whisper For particularly the /S/ and /ø:/, / / /a / / / e/i/o/u/y/, the phoneme error rates (PERs) of FT-Whisper are substantially lower The results suggest that when human listeners do not understand an utterance, they tend not to write anything down, whereas ASR systems typically output something whenever speech is detected The analysis of listening times suggested that for most listeners, the play times were higher for the more difficult utterances In conclusion, the recognition of both read and spontaneous speech of a single Dutch speaker with severe dysarthria is highly challenging for both native human listeners and SotA off-the-shelf ASR models Personalised DSR models, however, substantially outperform human listeners Though the performance remains far below SotA performance on typical speech, it is getting closer to being helpful for supporting the day-to-day communication of dysarthric speakers In addition, personalised DSR systems can be particularly helpful for naive human listeners in understanding dysarthric speech The paper is 2606.30237.

Title and authors: Jane: Let's start by looking at the title again, Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech: A Case Study. It sounds very specific, which is good for understanding exactly what this research is about.

Tom: Yeah, it sets the stage clearly. The paper focuses on comparing human performance against automated recognition systems for Dutch speech that's spoken by someone with severe dysarthria.

Lu: The authors are Yuanyuan Zhang, Dimme de Groot, Jorge Martinez Castaneda, and Odette Scharenborg from the Multimedia Computing Group at Delft University of Technology in the Netherlands. They bring a strong background in multimedia computing and speech recognition.

Meng: I'm curious what they are really trying to achieve with this specific comparison. Is it just showing that AI is bad at this, or are they proposing a path forward?

Lalam: The overarching goal stated in the paper is to develop personalized dysarthric speech recognition models for use in interpersonal communication, specifically between a dysarthric speaker and naive listeners.

Tom: So the implication there isn't just about improving accuracy; it’s fundamentally about enabling practical communication where it was previously very difficult. It moves beyond just a technical benchmark.

Jane: Exactly. They aren't just measuring error rates; they are looking at how this technology can be applied to help people who are struggling with speaking clearly because of their disability.

Lu: The paper builds on previous work, specifically mentioning comparisons made by Mengistu et al in two thousand eleven and Green et al in two thousand twenty-one regarding personalized models versus human performance on English read speech <ref:2606.30237#pg2>.

Meng: So they're connecting this Dutch study to a broader trend where personalized AI is starting to show promise, even if the results are still challenging.

Lalam: That connection is important because it suggests that moving away from just reading isolated words toward spontaneous speech is the next logical step for this kind of work.

The paper's summary: Tom: So to summarize what we've covered, the paper compared human listeners and three ASR systems—Whisper large V three, Google Chirp three, and Omnilingual—on Dutch continuous read and spontaneous speech from one speaker with severe dysarthria.

Jane: The key takeaway here is that both humans and the off-the-shelf ASR systems had word error rates over seventy percent on average initially, which shows how difficult this speech is for everyone involved.

Lu: But the summary also emphasizes that fine-tuning these models significantly reduced those word error rates, and although they still were high above twenty three percent, the personalized dysarthric speech recognition models outperformed the human listeners and were getting closer to being useful for supporting day to day communication.

Meng: So, personalization is key here. It's not just a general model; it has to be tailored to that specific speaker's speech characteristics for the performance boost they see.

Lalam: And another important summary point is that after fine-tuning, both personalized dysarthric speech recognition models significantly outperformed the human listeners by a large margin, with an average absolute improvement in word error rates of over thirty four percent.

Tom: That thirty four percent improvement is what really drives the interest here. It's not just marginal gains; it’s a substantial jump that makes a difference in how usable the technology becomes for people.

Jane: And they also noted that personalized DSR systems can be particularly helpful for naive human listeners who want to understand dysarthric speech, as they significantly outperformed the Omnilingual ASR model on all three speech types when compared to human listeners.

The paper's improvements: Lu: The paper suggests a few clear directions for future research. First, they should focus on improving personalized DSR recognition performance on spontaneous speech and longer utterances in the case of read speech.

Meng: They also identified that there is a specific focus needed on certain Dutch phonemes for future work, like /Z/, /S/, /E:/, /f/, /y/, /N/, and the letter j. This is where they saw the most significant improvement in error rates after fine-tuning compared to off-the-shelf models.

Tom: So it’s not just about getting better at general recognition; it’s about targeting specific phonetic weaknesses to really sharpen those personalized models. That makes a lot of sense from a machine learning standpoint.

Jane: It also suggests that the error analysis showed that when human listeners don't understand an utterance, they tend not to write anything down, whereas ASR systems typically output something whenever speech is detected.

Lalam: That observation about how humans react—not writing anything down if they don't understand—is a really insightful piece of information for understanding the communication gap.

Tom: And that leads into the idea of what these personalized models can actually do in practice, which is supporting day to day communication. The paper suggests that this technology can be particularly helpful for naive human listeners in understanding dysarthric speech.

Conclusion: Jane: So wrapping things up, the main message from Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech: A Case Study is that while both human listeners and state-of-the-art off-the-shelf ASR systems struggle with this speech, personalized DSR models substantially outperform human listeners.

Tom: That’s the core finding we need to remember: personalization makes a real difference, even if the final word error rates are still high compared to typical speech benchmarks. The performance is getting closer to being helpful for supporting day to day communication of dysarthric speakers.

Lu: From my side, I think the focus on specific phonemes like /Z/ and /S/ is crucial because that’s where the fine-tuning actually delivered substantial gains over the general models they compared against.

Meng: For deployment, it suggests that we need to keep pushing personalization efforts because speaker-dependent models are consistently better for speakers with severe dysarthria, which means they're probably more reliable than what we've seen in general benchmarks.

Lalam: And I just want to reiterate that personalized DSR systems can be particularly helpful for naive human listeners in understanding dysarthric speech, which is a really practical application we should focus on.

Tom: So, in the end, the paper on Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech: A Case Study shows that personalization substantially outperforms human listeners in this challenging scenario. It’s a step toward making AI tools genuinely helpful for people with severe dysarthria.

Yuanyuan Zhang, Dimme de Groot, Jorge Martinez, Odette Scharenborg

Delft University of Technology

cs.CL

Submitted: 2026-06-29

Updated: 2026-11-12

Code: https://github.com/karkirowle/relative

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

Importance score: 77/100

The gist: In this study, the paper compares human and automatic recognition of Dutch continuous speech to develop personalized dysarthric speech recognition (DSR) models…

Key concepts

Dysarthric Speech Recognition (DSR)
This is the technology being developed to understand speech produced by people with dysarthria, a speech disorder caused by brain injury. The goal is to create personalized models that can accurately transcribe this difficult speech for communication purposes.
Word Error Rate (WER)
WER measures how many words an automatic system gets wrong when transcribing spoken audio. High WERs, often above 70% in this study, indicate that the speech is very challenging for both humans and current ASR technology.
Personalized DSR Models
These are ASR models that have been specifically 'fine-tuned' using recordings from the target speaker with dysarthria. This customization allows the model to learn the unique acoustic characteristics of that individual's speech, leading to much better recognition than general models.

Terminology

Summary

In this study, the paper compares human and automatic recognition of Dutch continuous speech to develop personalized dysarthric speech recognition (DSR) models, which is important because personalized DSR models substantially outperform human listeners on dysarthric speech

Study Goal and Motivation

In our goal to develop personalised dysarthric speech recognition (DSR) models, this study compared the recognition performances of human listeners and those of three state-of-the-art, off-the-shelf ASR systems (Whisper-large-V3, Google Chirp 3, and Omnilingual) on the recognition of Dutch continuous read and spontaneous speech from a single speaker with severe dysarthria The overarching goal of our work is to develop personalised DSR technology for dysarthric speakers that can be used in interpersonal communication, i.e., communication between a dysarthric speaker and naive listeners

Experimental Setup

The stimuli for the listening and ASR experiments were taken from the DysOne dataset for personalised DSR model development The DysOne dataset is a bilingual (native Dutch and non-native English) and bimodal (speech and video) dataset containing recordings of a 35-year-old male Dutch speaker with severe dysarthria resulting from acquired brain injury due to high-impact trauma The stimuli consisted of 188 Dutch utterances (with 686 unique words) selected as stimuli for our experiments, consisting of generic read speech, customized read speech, and spontaneous speech

Performance Comparison

Results showed that both humans listeners and the three off-the-shelf ASR systems exhibit word error rates (WER) exceeding 70% on average, indicating that DSR is highly challenging for both humans and ASR systems Fine-tuning on the dysarthric speech significantly reduced WER Although overall WERs are still quite high (>23%), the personalised DSR models outperformed the human listeners, and performance is getting closer to being useful for supporting day-to-day communication of dysarthric speakers

Key Findings on Model Performance

  1. Both the human listeners and the off-the-shelf ASR models had substantial difficulty in recognizing the dysarthric speech, particularly for customized read speech, as shown by the high WERs

  2. The Whisper model significantly outperformed the Google Chirp (95% CI, [+8. p. 3

    Where the paper says it

    These results indicate substantial variability in recognition performance across naive listeners. Comparison of the off-the-shelf ASR models showed that the Whisper model significantly outperformed the Google Chirp (95% CI, [+8.

    197%, +19.215%], p <. p. 3
    Where the paper says it

    Comparison of the off-the-shelf ASR models showed that the Whisper model significantly outperformed the Google Chirp (95% CI, [+8. 215%], p < .

    001) and the OmniASR (95% CI, [+11. p. 3
    Where the paper says it

    215%], p < . 001) and the OmniASR (95% CI, [+11.

    099%, +20.558%], p <. p. 3
    Where the paper says it

    001) and the OmniASR (95% CI, [+11. 558%], p < .

    001) models for generic read speech

  3. After fine-tuning on the target speaker’s speech, both personalised DSR models (FT-Whisper and FT-OmniASR) outperformed the human listeners by a large margin with an average absolute improvement of WER by more than 34%

Areas for Future Research

Future research should focus on improving personalized DSR on spontaneous speech and longer utterances in the case of read speech, with a specific focus on particular phonemes The phoneme error analysis showed that future research should focus on improving the DSR recognition performance on a particular set of Dutch phonemes, including /Z/, /S/, /E:/, /f/, /y/, /N/, and /j/ The paper concludes that the recognition of both read and spontaneous speech of a single Dutch speaker with severe dysarthria is highly challenging for both native human listeners and SotA off-the-shelf ASR models

Methodological Details

The procedure involved dividing the 188 utterances into four lists of 47 utterances each, with each list assigned randomly to a listener and listened to by five different listeners For automatic speech recognisers, Whisper-large-V3 was selected because it has been reported to achieve the best performance on dysarthric speech in comparison to eleven typical ASR models The Omnilingual ASR model was included because it has not yet been tested on dysarthric speech The evaluation involved statistical significance tests using a paired nonparametric bootstrap test with 10,000 utterance-based samples and 95% confidence intervals

Error Analysis Insights

In the error analyses, we focus on the human listeners and the best off-the-shelf and fine-tuned models: Whisper and FT-Whisper For particularly the /S/ and /ø:/, / / /a / / / e/i/o/u/y/, the phoneme error rates (PERs) of FT-Whisper are substantially lower The results suggest that when human listeners do not understand an utterance, they tend not to write anything down, whereas ASR systems typically output something whenever speech is detected The analysis of listening times suggested that for most listeners, the play times were higher for the more difficult utterances

Conclusion

In conclusion, the recognition of both read and spontaneous speech of a single Dutch speaker with severe dysarthria is highly challenging for both native human listeners and SotA off-the-shelf ASR models Personalised DSR models, however, substantially outperform human listeners Though the performance remains far below SotA performance on typical speech, it is getting closer to being helpful for supporting the day-to-day communication of dysarthric speakers In addition, personalised DSR systems can be particularly helpful for naive human listeners in understanding dysarthric speech The paper is 2606.30237.

--- Page 1 ---

Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech: A Case Study This paper is 2606.30237. The speech recognition performance of human listeners has long been considered the upper bound for automatic speech recognition (ASR) performance Their performance has been compared by researchers on a variety of tasks, including recognizing phonemes [1], isolated words [2], read speech [2], [3], and spontaneous speech [2], [4], and in quiet and in background noise [2], [3] With the rapid development of deep learning techniques [6]–[8], ASR systems have improved significantly over the past decade and are now widely used in diverse applications, including voice assistants [9]–[11], search engines [12], and health-related applications [13], [14] For typical speech, defined by Zhang et al. p. 0

Where the paper says it

With the rapid development of deep learning techniques [6]– [8], ASR systems have improved significantly over the past decade and are now widely used in diverse applications, including voice assistants [9]–[11], search engines [12], and health-related applications [13], [14]. For typical speech, defined by Zhang et al.

as “adult, typically highly-educated, firstlanguage speakers of a standardized language variety, without a speech disability” [15], state-of-the-art (SotA) ASR systems have achieved performance on par with or even better than that of humans [4], [16], [17] For example, recent ASR systems have reached word error rates (WERs) of below 5% on conversational speech benchmarks such as Switchboard [18], [19] and below 3% on read speech [16], [20] As reported in [21], [22], the WERs of professional transcribers on Switchboard range from 5. p. 0
Where the paper says it

as “adult, typically highly-educated, firstlanguage speakers of a standardized language variety, without a speech disability” [15], state-of-the-art (SotA) ASR systems have achieved performance on par with or even better than that of humans [4], [16], [17]. For example, recent ASR systems have reached word error rates (WERs) of below 5% on conversational speech benchmarks such as Switchboard [18], [19] and below 3% on read speech [16], [20].

1%–5.9% Despite the remarkable performance of SotA ASRs on typical speech, not all speakers have benefited from this large recognition performance improvement of SotA ASR systems This is particularly the case for dysarthric speakers In 2019, a comparison of three ASR cloud platforms [23] found that the best model on English dysarthric speech, the Google cloud model, achieved a WER of 59. p. 0
Where the paper says it

This is particularly the case for dysarthric speakers. In 2019, a comparison of three ASR cloud platforms [23] found that the best model on English dysarthric speech, the Google cloud model, achieved a WER of 59.

8% on the TORGO dataset [23], [24] Results from a 2025 study [25] on TORGO showed that Whisper-large-v3 [16] achieved the best performance among eight leading commercial ASR services, including AssemblyAI, GPT, and Gemini, still with a WER as high as 49. p. 0
Where the paper says it

8% on the TORGO dataset [23], [24]. Results from a 2025 study [25] on TORGO showed that Whisper-large-v3 [16] achieved the best performance among eight leading commercial ASR services, including AssemblyAI, GPT, and Gemini, still with a WER as high as 49.

4% Although advancements have been made in dysarthric speech recognition (DSR) over the past years, these results show that dysarthric speech remains challenging for SotA ASR systems It is important to close this gap, since dysarthric speakers are among those who could benefit the most from speech technology: their everyday communication is often difficult or impaired due to their interlocutors having problems understanding their speech One of the challenges for DSR is that the acoustic characteristics of dysarthric speech vary substantially between individuals, influenced by differences in underlying etiologies, severity, and stage of disease progression [26], [31]–[33] For example, even among speakers with the same etiology (cerebral palsy), the same severity (very low intelligibility), and the same gender (male), the same DSR model performed differently across individuals, with a word recognition accuracy difference between the best (65%) and worst (42%) recognized speaker of 23% [34] Likewise, there is ample evidence that personalised (or speaker-dependent) DSR models consistently outperform speaker-independent models [30], [35], [36], especially for speakers with severe dysarthria [31], [37], [38] Given the current limitations of SotA ASR systems in generalizing across the highly variable dysarthric speech [23], [25], [21], developing personalised DSR systems aligns with the goal of developing inclusive speech recognition: developing ASR for everyone [39]

--- Page 2 ---

the difficulty that human listeners have with recognising dysarthric read speech. Nevertheless, a comparison of human and personalised DSR model performance on dysarthric speech by Mengistu et al. p. 1

Where the paper says it

the difficulty that human listeners have with recognising dysarthric read speech. Nevertheless, a comparison of human and personalised DSR model performance on dysarthric speech by Mengistu et al.

(2011) found that naive human listeners outperformed personalised DSR models on English read isolated words [28] More recently, in 2021, Green et al. p. 1
Where the paper says it

(2011) found that naive human listeners outperformed personalised DSR models on English read isolated words [28]. More recently, in 2021, Green et al.

showed, for the first time, that personalised DSR models outperformed three speech-language pathologists (SLPs) on English dysarthric read phrases [31] Nevertheless, these expert listeners still outperformed the off-the-shelf Google Cloud model [31] on the read speech A similar finding was reported for Mandarin read sentences [30], where personalized DSR models outperformed experienced listeners for five out of seven dysarthric speakers on read speech The overarching goal of our work is to develop personalised DSR technology for dysarthric speakers that can be used in interpersonal communication, i. p. 1
Where the paper says it

A similar finding was reported for Mandarin read sentences [30], where personalized DSR models outperformed experienced listeners for five out of seven dysarthric speakers on read speech. The overarching goal of our work is to develop personalised DSR technology for dysarthric speakers that can be used in interpersonal communication, i.

e., communication between a dysarthric speaker and naive listeners For this, we need to move beyond read speech to spontaneous speech In this paper, we benchmark our DSR models against human performance, as such, this paper builds on and extends previous research comparing human and automatic recognition of dysarthric speech In our work, we investigate recognition performance of the latest SotA off-the-shelf ASR models and personalised DSR models on the dysarthric speech of a single Dutch speaker with severe dysarthria in comparison to naive human listeners on the exact same stimuli We used three speech types: generic read, customized read, and, for the first time, spontaneous speech Specifically, we compare human listeners’ performance with that of three off-the-shelf SotA ASR systems, i. p. 1
Where the paper says it

In this paper, we benchmark our DSR models against human performance, as such, this paper builds on and extends previous research comparing human and automatic recognition of dysarthric speech. In our work, we investigate recognition performance of the latest SotA off-the-shelf ASR models and personalised DSR models on the dysarthric speech of a single Dutch speaker with severe dysarthria in comparison to naive human listeners on the exact same stimuli.

e., Whisper-large-V3 [16], Google Chirp 3 [8], and the Omnilingual ASR model [42]

--- Page 3 ---

All listening experiments were conducted in a soundattenuated booth using the same headphones (Sennheiser HD 200 Pro) and laptop Prior to the experiment, listeners received instructions, signed an informed consent form, and filled in a questionnaire for meta information, including gender, age, and familiarity with dysarthric speech The 188 utterances were divided into four lists of 47 utterances each Each list contained ten generic read utterances with (1,5] words, ten with (5,10] words, and ten or eleven with (10,15] words, six or seven customized read utterances, and ten spontaneous utterances

--- Page 4 ---

To explore the effect of speech type and utterance length (word count) on the recognition performance of the human listeners and ASR models, a multi-factor linear model was applied using the estimated marginal means package (emmeans) in R [56] Moreover, we conducted word-level and phoneme-level error analyses and investigated the effect of the number of replays of the stimuli on recognition performance

--- Page 5 ---

To ensure consistency between human transcriptions and those of the ASR systems, all transcriptions were converted to lowercase, punctuation was removed (except for the apostrophe), extra spaces were removed, digits or numbers were written out in full, and all non-linguistic symbols, filler words, and other transcriptions of non-lexical sounds were removed Speech recognition performance is measured in WER For the recognition performance comparisons of the human listeners and ASR models, we conducted statistical significance tests using a paired nonparametric bootstrap test with 10,000 utterance-based samples and 95% confidence intervals (CIs) [53], [54] To explore the effect of speech type and utterance length (word count) on the recognition performance of the human listeners and ASR models, a multi-factor linear model was applied using the estimated marginal means package (emmeans) in R [56] Moreover, we conducted word-level and phoneme-level error analyses and investigated the effect of the number of replays of the stimuli on recognition performance

--- Page 6 ---

In our work, we investigate recognition performance of the latest SotA off-the-shelf ASR models and personalised DSR models on the dysarthric speech of a single Dutch speaker with severe dysarthria in comparison to naive human listeners on the exact same stimuli The best-performing model was the fine-tuned Whisper model (FT-Whisper) FT-Whisper significantly outperformed the fine-tuned OmniASR model (FT-OmniASR) on generic read speech (95% CI, [+8.984%, +18.919%], p <. p. 3

Where the paper says it

FT-Whisper significantly outperformed the fine-tuned OmniASR model (FT-OmniASR) on generic read speech (95% CI, [+8. 919%], p < .

p. 3
Where the paper says it

FT-Whisper significantly outperformed the fine-tuned OmniASR model (FT-OmniASR) on generic read speech (95% CI, [+8. 919%], p < .

001) Interestingly, both models significantly outperformed the human listeners and the Whisper model for all speech types (p <. p. 3
Where the paper says it

858%], p = . Interestingly, both models significantly outperformed the human listeners and the Whisper model for all speech types (p < .

p. 3
Where the paper says it

858%], p = . Interestingly, both models significantly outperformed the human listeners and the Whisper model for all speech types (p < .

001)

--- Page 7 ---

In conclusion, the recognition of both read and spontaneous speech of a single Dutch speaker with severe dysarthria is highly challenging for both native human listeners and SotA off-the-shelf ASR models Personalised DSR models, however, substantially outperform human listeners Though the performance remains far below SotA performance on typical speech, it is getting closer to being helpful for supporting the day-to-day communication of dysarthric speakers In addition, personalised DSR systems can be particularly helpful for naive human listeners in understanding dysarthric speech

--- Page 8 ---

The best-performing model was the fine-tuned Whisper model (FT-Whisper) FT-Whisper significantly outperformed the fine-tuned OmniASR model (FT-OmniASR) on generic read speech (95% CI, [+8.984%, +18.919%], p <.001) Interestingly, both models significantly outperformed the human listeners and the Whisper model for all speech types (p <.001)

--- Page 9 ---

In conclusion, the recognition of both read and spontaneous speech of a single Dutch speaker with severe dysarthria is highly challenging for both native human listeners and SotA off-the-shelf ASR models Personalised DSR models, however, substantially outperform human listeners Though the performance remains far below SotA performance on typical speech, it is getting closer to being helpful for supporting the day-to-day communication of dysarthric speakers In addition, personalised DSR systems can be particularly helpful for naive human listeners in understanding dysarthric speech

--- Page 10 ---

The authors thank Wenyi Chu for her help and support with finding human listener participants During the preparation of this work the authors used Generative AI (ChatGPT) to improve language and readability and to debug Python and R scripts for statistical significance tests After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication The paper is 2606.30237.

--- Page 11 ---

The authors thank Wenyi Chu for her help and support with finding human listener participants During the preparation of this work the authors used Generative AI (ChatGPT) to improve language and readability and to debug Python and R scripts for statistical significance tests After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication The paper is 2606.30237.

--- Page 12 ---

The authors thank Wenyi Chu for her help and support with finding human listener participants During the preparation of this work the authors used Generative AI (ChatGPT) to improve language and readability and to debug Python and R scripts for statistical significance tests After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication The paper is 2606.30237.

--- Page 13 ---

The authors thank Wenyi Chu for her help and support with finding human listener participants During the preparation of this work the…

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Personalized DSR models can substantially outperform human listeners by achieving an average absolute improvement of WER by more than 34% compared to naive human listeners after fine-tuning on target speaker speech. This allows for better supporting day-to-day communication of dysarthric speakers.

  2. Fine-tuning models on the dysarthric speech, such as FT-Whisper, can remove the greater challenge of customized read speech for ASR systems, suggesting that personalized models may improve dysarthric speakers’ ability to communicate about their favorite topics in personalized communication support systems.

  3. Future DSR improvements should focus on improving recognition performance on a particular set of Dutch phonemes including /Z/, /S/, /E:/, /f/, /y/, /N/, and/or/j/. This targets the phonemes where the best fine-tuned model showed substantially lower phoneme error rates compared to off-the-shelf models.

  4. Personalized DSR systems can be particularly helpful for naive human listeners in understanding dysarthric speech, as they are shown to significantly outperform the Omnilingual ASR model on all three speech types when compared to human listeners.

Abstract

In our goal to develop personalised dysarthric speech recognition (DSR) models, this study compared the recognition performances of human listeners and those of three state-of-the-art, off-the-shelf ASR systems (Whisper-large-V3, Google Chirp 3, and Omnilingual) on the recognition of Dutch continuous read and spontaneous speech from a single speaker with severe dysarthria. Results showed that both humans listeners and the three off-the-shelf ASR systems exhibit word error rates (WER) exceeding 70% on average, indicating that DSR is highly challenging for both humans and ASR systems. Fine-tuning on the dysarthric speech significantly reduced WER. Although overall WERs are still quite high (>23%), the personalised DSR models outperformed the human listeners, and performance is getting closer to being useful for supporting day-to-day communication of dysarthric speakers. Future research should focus on improving personalized DSR on spontaneous speech and longer utterances in the case of read speech, with a specific focus on particular phonemes.

Sources

Related papers