A Semi-spontaneous Dutch Speech Dataset for Speech Enhancement and Speech Recognition

summary

The gist

We present DRES: a 1.5-hour Dutch realistic elicited (semi-spontaneous) speech dataset from 80 speakers recorded in noisy…

In short

Researchers created DRES, a realistic 1.5-hour Dutch speech dataset from 80 speakers recorded in noisy public indoor settings. They tested six single-channel speech enhancement algorithms and eight state-of-the-art ASR models before and after enhancement. The study found that no single-channel SE improved ASR performance, highlighting the need to test these systems on realistic conditions.

Key concepts

DRES Corpus
This is a 1.5-hour dataset of spontaneous Dutch speech recorded from eight speakers in four different public buildings (Ahoy, Pulse, IDE, Architecture). It was designed to capture diverse speech and noise conditions for rigorous testing of ASR and enhancement techniques.
DNSMOS P.835
This is an objective measure used to assess the quality of enhanced speech. It gives a score from 1 (very poor) to 5 (excellent) based on how natural the sound is, allowing researchers to quantitatively compare how much each enhancement algorithm improves speech clarity.
Single-channel SE
These are classical and modern algorithms used to clean up audio by processing it through a single microphone channel. The study tested six such methods, including simple techniques like spectral subtraction and complex deep learning models, to see their impact on recognition.
WER (Word Error Rate)
This metric measures the accuracy of an Automatic Speech Recognition (ASR) system by calculating the percentage of words incorrectly transcribed. A lower WER indicates better performance in recognizing spoken Dutch speech.

Terminology used across episodes

This episode discusses

The paper

A Semi-spontaneous Dutch Speech Dataset for Speech Enhancement and Speech Recognition · Read on arXiv

Dimme de Groot, Yuanyuan Zhang, Jorge Martinez, Odette Scharenborg

Delft University of Technology

We present DRES: a 1.5-hour Dutch realistic elicited (semi-spontaneous) speech dataset from 80 speakers recorded in noisy, public indoor environments. DRES was designed as a test set for the evaluation of state-of-the-art (SotA) automatic speech recognition (ASR) and speech enhancement (SE) models in a real-world scenario: a person speaking in a public indoor space with background talkers and noise. The speech was recorded with a four-channel linear microphone array. In this work we evaluate the speech quality of five well-known single-channel SE algorithms and the recognition performance of eight SotA off-the-shelf ASR models before and after applying SE on the speech of DRES. We found that five out of the eight ASR models have WERs lower than 22% on DRES, despite the challenging conditions. In contrast to recent work, we did not find a positive effect of modern single-channel SE on ASR performance, emphasizing the importance of evaluating in realistic conditions.

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: We're looking at the paper: "A Semi-spontaneous Dutch Speech Dataset for Speech Enhancement and Speech Recognition".

Dev: Here's the gist of the paper. We present DRES: a 1.5-hour Dutch realistic elicited (semi-spontaneous) speech dataset from 80 speakers recorded in noisy, public indoor environments This work evaluates the speech quality of six well-known single-channel SE algorithms and the recognition performance of eight SotA off-the-shelf ASR models before and after applying SE on the speech of DRES We found that five out of the eight ASR models have word error rates (WERs) lower than 22% on DRES, despite the challenging conditions In contrast to recent work, we did not find a positive effect of modern single-channel SE on ASR performance, emphasizing the importance of evaluating in realistic conditions Dataset Construction and Elicitation Methods The DRES corpus was designed to elicit spontaneous speech and ensure both lexical and phonetic diversity through three tasks: Free speech, Picture card, and Prompt card The prompt cards contained an instruction of the following form (translated from Dutch): “Tell something about

topic 1: or

topic 2: ” The speech was recorded at the main halls of four different public buildings: Ahoy, Pulse, IDE, and Architecture (Arch) These locations were chosen to recruit participants from a wide and diverse population, thus increasing speaker diversity and reducing the risk of participant re-identification All recordings were made using a linear microphone array consisting of four AKG C147 PP lavalier microphones with an inter-microphone spacing of 5 cm connected to an RME Fireface UFX III audio-interface recording at a sampling rate of 48 kHz Speech Quality Assessment The quality of the speech data was assessed using DNSMOS P.835, which is a no-reference objective speech quality measure used for estimating the MOS of enhanced speech and is commonly used in recent literature Figure 3 shows the distribution of the mean-opinion-score (MOS) for the recordings at the four recording locations, estimated using DNSMOS The speech quality is particularly low for the recordings made at location Ahoy, highlighting the difficult acoustic conditions A Kruskal-Wallis test indicated that the DNSMOS differed over recording locations (X 2(3) = 16.63, p <.001) ASR Performance Analysis Table II presents the average ASR results over the four locations on both the original and enhanced speech signals after applying the six SE algorithms, for each of the eight ASR models For example, Google Chirp 3 performed significantly better than the second-best model Whisper-large-V3 (Whisper), with an absolute WER difference of 4.60% (95% CI,

+3.95%, +5.25%: , p <.001) Notably, Chirp was trained with far less data than Telephony and Azure Impact of Speech Enhancement Algorithms The results indicated that none of the SE algorithms yielded improved recognition performance on DRES, irrespective of the ASR model Instead, the SE algorithms generally increase the deletion and substitution errors for most ASR models For instance, SNG increased deletion errors by more than 100% for five out of the eight models (Chirp, Telephony, Azure, Nemo-nl, Conformer) This suggests that the effects of SE are somewhat ASR model dependent Our results showed that our tested classical and SotA single-channel SE methods do not improve ASR performance on Dutch speech in realistic scenarios We hypothesize that the lack of improvement in recognition performance may be attributed to language differences and to different noise scenarios Our findings extend prior work, where single-channel SE failed to improve HMMGMM and hybrid DNN-HMM ASR performance

17: ,

18: ,

37: –

39: In future work, we plan to extend our work by including statistical SE algorithms, left out in this work, and to investigate the effect of multi-channel SE approaches The results demonstrated the importance of assessing ASR and SE performance across diverse and realistic scenarios Pruning redundant reasoning tokens The authors evaluated eight SotA ASR models on speech from DRES and evaluated the effect of several classical and modern single-channel speech enhancement (SE) algorithms The evaluation involved applying six single-channel SE algorithms, including two simple but still widely used algorithms (spectral subtraction and spectral noise gating)

45: ,

46: and four modern deep-learning-based algorithms (MetricGAN-OKD, two variants of SGMSE+, and SEMamba)

40: ,

43: ,

47: Speech signals might be pre-processed using SE algorithms before inputting them into an ASR, with the aim of improving ASR performance

35: ,

36: How the training signal is built The DRES corpus consists of 1.5 hours of raw audio After removing silences, 1.4 hours of speech was retained The raw audio (prior to segmentation) was used for the SE experiments, while the segmented version was used for the ASR experiments Where the gains come from The largest improvements in speech quality were achieved for SGV and SEM Specifically, SEM and both versions of SGMSE+ improve the speech quality for each utterance in the dataset However, other SE algorithms, in particular SNG and SS, reduce the speech quality for some of the utterances in the dataset The observation that SEM reduced deletion errors for Whisper and insertion errors for Turbo, though substitution errors increased for both These results indicate that the effects of SE are somewhat ASR model dependent Pruning redundant reasoning tokens The authors selected seven popular models which perform well on Dutch speech developed by Google, Microsoft, Meta, OpenAI, and NVIDIA, and one E2E model trained on a publicly available Dutch speech corpus, i.e., CGN

20: The eight ASR models that we tested on DRES are the following Google Chirp 3 is employed since it is the most advanced Google general purpose ASR model

62: and the latest version of the Chirp model

29: (January 2026) Whisper-large-V3-Turbo achieves comparable performance with Whisper-large-V3 on Dutch speech from the FLEURS dataset

65: but is optimized for faster inference

66: Pruning redundant reasoning tokens The authors selected six single-channel SE algorithms based on ease-of-use, computational complexity, SE performance, and popularity As baseline signal (Base) we use the recordings from channel 2, which is at the center of the microphone-array (see Fig.

Rosa: 2) Spectral subtraction (SS) is a traditional and wellestablished low-complexity signal-processing-based SE technique which uses the difference between an estimate of the noise spectrum and the noisy-speech spectrum to compute the enhanced speech

45: ,

51: ,

52: Pruning redundant reasoning tokens The authors selected eight ASR models that we tested on DRES are the following For Telephony and Whisper, SS had no significant effect SEM had no significant effect on Whisper, but significantly degraded performance for Telephony (p <.05) The remaining SE methods significantly degraded ASR performance for both models (Telephony: p <.001; Whisper: p <.05) Pruning redundant reasoning tokens The authors concluded that none of the SE algorithms yielded improved recognition performance on DRES, irrespective of the ASR model The results demonstrated the importance of assessing ASR and SE performance across diverse and realistic scenarios In summary, DRES has clear potential to benefit both the ASR and SE communities Our corpus of realistic elicited Dutch speech in diverse natural background noise enables evaluation of SotA ASR models on realistic speech and allows for assessment of SE algorithms on natural speech in noise The authors plan to extend their work by including statistical SE algorithms, left out in this work, and to investigate the effect of multi-channel SE approaches Pruning redundant reasoning tokens The paper is available under end-user license agreement (EULA) from

48: The authors thank Ilse Huisman and Ansen Weng for their support with constructing the ground-truth transcriptions Generative AI (ChatGPT) was used to improve language and readability, and debug Python scripts for bootstrapping The authors' contact information is listed as dimme de groot, yuanyuan zhang, jorge martinez castaneda, odette scharenborg@tudelft.nl Pruning redundant reasoning tokens The paper is available under end-user license agreement (EULA) from

48: The authors thank Ilse Huisman and Ansen Weng for their support with constructing the ground-truth transcriptions Generative AI (ChatGPT) was used to improve language and readability, and debug Python scripts for bootstrapping The authors' contact information is listed as dimme de groot, yuanyuan zhang, jorge martinez castaneda, o.e.

Dev: scharenborg@tudelft.nl Pruning redundant reasoning tokens The authors thank Ilse Huisman and Ansen Weng for their support with constructing the ground-truth transcriptions Generative AI (ChatGPT) was used to improve language and readability, and debug Python scripts for bootstrapping The authors' contact information is listed as dimme de groot, yuanyuan zhang, jorge martinez castaneda, o.e.

Rosa: scharenborg@tudelft.nl Pruning redundant reasoning tokens The authors thank Ilse Huisman and Ansen Weng for their support with constructing the ground-truth transcriptions Generative AI (ChatGPT) was used to improve language and readability, and debug Python scripts for bootstrapping The authors' contact information is listed as dimme de groot, yuanyuan zhang, jorge martinez castaneda, o.e.

Title and authors: Rosa: Okay, so we're diving into the title and authors of this paper, "A Semi-spontaneous Dutch Speech Dataset for Speech Enhancement and Speech Recognition." It sounds pretty technical, but it's telling us exactly what they built.

Dev: That title points to two main things: a dataset that is semi-spontaneous—meaning it’s not perfectly scripted—and the dual focus on speech enhancement and speech recognition.

Rosa: Right. The authors are Dimme de Groot, Yuanyuan Zhang, Jorge Martinez, and Odette Scharenborg from Delft University of Technology in the Netherlands. They're coming from a Multimedia Computing Group there.

Taro: I’m curious about their background since they’re tackling this specific language and acoustic environment for these tasks.

Dev: Well, their work is focused on how computing can handle real-world audio signals, which is exactly what this paper is doing—testing enhancement and recognition systems under realistic conditions.

Rosa: It seems like the main takeaway from the title itself is that they’ve created a dataset specifically designed to test these two areas together in a messy, public setting.

Dev: Exactly. It’s not just about getting clean audio; it's about seeing if cleaning that audio actually makes the speech recognition models better in a way that matters for people using it.

Taro: So, what does this mean for someone who is building an ASR system? They need to consider the specific noise and acoustic challenges of public indoor spaces, not just clean studio recordings.

Rosa: That's right. This dataset forces you to confront how your system will handle background talkers and the general clutter you hear on a busy street or in a public hall.

Dev: It also sets up an evaluation framework where you can compare different enhancement tools against different ASR models on the same challenging data.

Taro: So, if I'm testing a new noise reduction technique, I need to know how it affects the final recognition accuracy under these specific conditions.

Rosa: That’s what they are giving us—a structured way to look at the trade-offs between quality improvement and recognition performance in this context.

Dev: It’s a dataset that forces you to think about the entire pipeline, from recording noise to final text output accuracy.

The paper's summary: Rosa: We’ve already touched on what the paper is about, but let's get into the specifics of their summary. They present DRES as a test set for state-of-the-art automatic speech recognition and speech enhancement models in a real-world setting.

Dev: They record this one point five hours of Dutch spontaneous speech from eighty speakers across four locations, using a four channel linear microphone array to capture the noise and signal structure accurately <ref:2603.09725#pg1,a four channel linear microphone array>.

Taro: So they are focusing on capturing not just the words, but also the acoustic context—the background talkers and general indoor noise that makes real speech recognition difficult.

Rosa: That's right. They’re explicitly setting up a scenario where people are speaking in public spaces, which is far more challenging than what you get from typical lab recordings.

Dev: The core summary is that they evaluate six single-channel speech enhancement algorithms against eight state of the art off-the-shelf ASR models before and after applying the enhancement to DRES speech.

Taro: And they report objective speech quality metrics using DNSMOS P point eight three five to quantify how much the audio improved after each step.

Rosa: That’s right. They found that while some algorithms, like SGMSE plus and SEMamba, improved speech quality for each utterance in the dataset, others like SNG and SS actually reduced the speech quality for some parts of it.

Dev: And they also noted that this effect is model dependent; what helps one ASR model might hurt another one.

Taro: So the summary boils down to a structured comparison: test these tools on real data, measure the quality change, and see how that translates to recognition performance across different models.

Rosa: Right. It’s a comprehensive evaluation of the enhancement pipeline in action on a complex language like Dutch speech in real noise conditions.

The paper's improvements: Dev: Moving on to what they suggest are the improvements this approach offers, they highlight that DRES gives us a chance to test how single-channel enhancement influences state of the art ASR systems for Dutch real world noisy speech.

Rosa: They’re pointing out that their work addresses the open question about whether single-channel enhancement works for Dutch real-world noisy speech, contrasting it with previous work on English.

Taro: So, by doing this study in Dutch, they are trying to give us a more specific answer about language generalization.

Dev: Right. They tested six different methods—two simple ones like spectral subtraction and spectral noise gating and four deep learning based ones like MetricGAN OKD, two variants of SGMSE plus, and SEMamba.

Rosa: And they found that even though five out of the eight ASR models had word error rates lower than twenty-two percent on DRES, their main finding is that modern single-channel SE doesn't show a clear positive effect on ASR performance in this specific real-world context.

Dev: They also showed that most enhancement algorithms generally increase the deletion and substitution errors for most ASR models, suggesting some effects are dependent on the specific model architecture you are using.

Taro: That’s a crucial detail for us; it means we can't just assume a good enhancement tool will automatically fix every problem in every ASR system.

Rosa: So the paper suggests that the next steps involve exploring more complex enhancement and ASR methods, like statistical speech enhancement algorithms, which they didn't test here.

Dev: And investigating multi-channel speech enhancement approaches to capture more spatial information, which is a direction they feel is really important for future work.

Taro: I think focusing on that multi-channel aspect makes sense because it could give us the information needed to understand why single-channel methods fall short.

Conclusion: Rosa: So wrapping up this discussion on the paper, the main implication is that these tested classical and state of the art single-channel SE methods don't actually improve ASR performance on Dutch speech when tested in these realistic settings.

Dev: They’re confirming that DRES is a valuable resource because it clearly shows us where current enhancement techniques stop working for real-world speech recognition tasks.

Taro: The data shows that we have to be cautious about applying any single tool without seeing how it behaves across different ASR architectures.

Rosa: Absolutely. The paper’s limitations are that they didn't test statistical SE algorithms, and they didn't look at multi-channel approaches as deeply as they could.

Dev: Exactly. So the big takeaway is that DRES is important for showing us the performance of these systems in a realistic scenario, even if it doesn't yield immediate recognition gains from single-channel enhancement alone.

Taro: It’s a good setup for future work, pushing toward those more complex methods we discussed earlier to really tackle these real-world challenges.

Rosa: So that’s our take on the paper "A Semi-spontaneous Dutch Speech Dataset for Speech Enhancement and Speech Recognition." We've seen how realistic data helps us evaluate the limits of current single-channel SE and ASR systems.

More episodes

← Home