Fine-Tuning a Transducer for Impaired Speech: Where the Gains Appear and What They Mean for Evaluation
cs.CL
Submitted: 2026-11-05
Updated: 2026-11-05
Comments: 6 pages, 4 tables. Accepted at the SAPC2 Competition Workshop at NeurIPS 2026
Code: https://github.com/SYSTRAN/faster-whisper
Project page: https://xiuwenz2.github.io/SAPC2-website
License: http://creativecommons.org/licenses/by/4.0/
The gist: Speech recognition still works poorly for many people whose speech is affected by conditions such as ALS, Parkinson's disease, cerebral palsy, Down syndrome or stroke.
Terminology
Abstract
Speech recognition still works poorly for many people whose speech is affected by conditions such as ALS, Parkinson's disease, cerebral palsy, Down syndrome or stroke. We took part in the Speech Accessibility Project Challenge 2 (SAPC2) with a public speech recognizer that we fine-tuned on the challenge data. Our best entry scored 7.81% character error rate on Test1, the first of two hidden test sets (official baseline 15.26%). Our live entries ran on CPUs with a combined median latency of about 0.9 seconds at 7.96-8.07% error. On new speakers, fine-tuning reduced errors most for the speakers that an off-the-shelf recognizer finds hardest to understand. Sentences already seen in training had far lower error rates than new ones (2.58% vs 6.42%), a larger difference than being a familiar speaker made. We close with lessons for evaluating such systems.
Sources
- NeMo: a toolkit for building AI applications using Neural Modules
- The Interspeech 2025 Speech Accessibility Project Challenge
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data