Paediatric-HGNN: A Hybrid Heterogeneous Graph Neural Network for Detecting Disfluency in Children's Speech via Multiscale Acoustic Fusion

arXiv:2606.08210 · eess.AS, cs.CL, cs.SD · Submitted 2026-06-06 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: We're looking at the paper: "Paediatric-HGNN: A Hybrid Heterogeneous Graph Neural Network for Detecting Disfluency in Children's Speech via Multiscale Acoustic Fusion".

Dev: Here's the gist of the paper. Automated stuttering detection (ASD) systems struggle with paediatric speech due to high acoustic variability in developing voices and the subtle distinction between pathological stuttering and typical developmental disfluencies Paediatric-HGNN achieves 82.4% weighted accuracy and a Typical Disfluency F1-score of 0.386 Modelling hierarchical lexical-acoustic interactions captures developmental “searching” behaviour, offering a more robust and interpretable tool for early clinical intervention The Core Problem Addressed Stuttering is a neuro-developmental communication disorder characterised by disruptions in the forward flow of speech, which affects approximately 5% to 8% of children during their preschool years Early diagnosis is critical, yet current clinical assessments largely rely on subjective manual observations by Speech-Language Pathologists (SLPs), which can lead to high inter-rater variability and significant time costs Most automated stuttering detection (ASD) models are optimised for adult speech through datasets like SEP-28k These models often fail to generalise to the unique acoustic and linguistic variability of children, as they have not been extensively trained or evaluated on the higher fundamental frequencies and the phonological characteristics of paediatric populations Paediatric speech presents a unique diagnostic challenge because early childhood language development involves a natural phase of typical disfluency with significant acoustic and linguistic overlap between core stuttering (e.g., pathological blocks and part-word repetitions) and typical disfluencies (e.g., developmental whole-word repetitions and revisions) How the training signal is built The model builds a heterogeneous graph capturing hierarchical relationships between lexical units (word nodes) and fine-grained acoustic segments (frame nodes) Nodes include word nodes, initialized with the 945D hybrid vector, and frame nodes, representing acoustic windows initialized with Wav2Vec2 embeddings Hierarchical Edges (Eh) connect each frame node to its parent word node based on temporal alignment Sequential Edges (Es) connect adjacent words to model the forward flow of speech Contextual Edges (Ec) provide a context window of ±2 words for nodes to aggregate information from their local linguistic neighbourhood The 945-dimensional hybrid feature vector for each word node integrates deep self-supervised contextual embeddings from a Wav2Vec2-base-960h1 model, mel-spectrogram textures (via global mean and max pooling) to capture the visual signatures of disfluencies, and traditional handcrafted acoustic features Where the gains come from The Paediatric-HGNN architecture uses a hierarchical cross-modal attention mechanism where word nodes (Queries) selectively attend to relevant acoustic artifacts in their constituent frames (Keys/Values) This is followed by two stages of relational graph convolutions (GATv2Conv) across sequential and contextual (±2 words) edges to capture local and neighbourhood disfluency patterns A learnable gated residual fusion dynamically weights spatial GNN features against temporal RNN features: H = g·HRNN+(1−g)·HGNN The model is optimised using AdamW

18: and a Focal Loss function

19: (γ = 2) to mitigate the impact of class imbalance Class-specific weights α =

1, 3, 3: force the model to prioritise the minority (pathological) classes over the majority (fluent) class Performance and Clinical Utility Paediatric-HGNN achieved a stable overall accuracy of 82.4% ± 2.7% The model demonstrated high reliability in identifying Fluent speech (F1 0.904 ± 0.02) The Typical Disfluency class reached a peak F1-score of 0.43 in specific folds (e.g., Fold 1) A key finding is the significant performance degradation when utilising adult-trained models for paediatric diagnostics, where Typical Disfluency detection collapsed to an F1 score of 0.08 The “white-box” nature of our CaPIN framework allows clinicians to inspect the hierarchical attention weights For Core Stutters, the model attends heavily to micro-level energy spikes in the frame nodes (Figure 3A) For Typical Disfluencies, attention is distributed across the word nodes and their ±2 word context Comparative Analysis The model's performance on the consolidated 3-class clinical taxonomy showed a Mean Core F1 of 0.280 ± 0.06 This performance confirms that the integration of lexical word nodes provides the necessary context to categorise developmental revisions that are often mislabelled in purely acoustic models The ablation study showed that removing contextual edges resulted in a drop in Typical Disfluency F1 from 0.386 to 0.

Title and authors: Rosa: So, let’s talk about the summary of "Paediatric-HGNN: A Hybrid Heterogeneous Graph Neural Network for Detecting Disfluency in Children's Speech via Multiscale Acoustic Fusion." Essentially, the core idea is that instead of treating speech as one long continuous signal, they are modeling it like a network where words and sound frames interact hierarchically.

Dev: By separating the lexical units—the word nodes—from the acoustic segments—the frame nodes—they can ground any acoustic disruption within its specific linguistic context. This helps them understand the part-whole interactions that are crucial for accurate paediatric diagnosis, which is what they call a Context-aware Part-whole Interaction Network or CaPIN.

Taro: But how does this graph structure actually help distinguish between the two classes they mentioned earlier, pathological stuttering and typical disfluencies? Does it give them a clearer signal?

Rosa: It helps because the model is trained on data specifically from children, using corpora like UCLASS and FluencyBank-CWS. This means it learns features that are relevant to those developmental speech patterns rather than just adult speech issues, which is a big step.

Dev: And they report some solid results from this setup, showing an eighty-two point four weighted accuracy and a Typical Disfluency F1 score of zero point three eight six. That score on the typical disfluency class is pretty telling for how well it's handling those developmental revisions.

The paper's summary: Rosa: Now, let’s look at what the authors suggest they did to improve this approach. One major thing they focus on is using a transfer learning baseline, where they take an adult-pretrained version of this Paediatric-HGNN and then fine-tune it specifically on their paediatric data set.

Dev: That transfer learning step is important because it lets them empirically measure the difference in how things change between general speech issues seen in adults and the specific developmental disfluencies we see in children. It’s testing if the paediatric modeling adds real value over just taking an adult model and tweaking it slightly.

Taro: I wonder if that transfer learning approach actually helps with detecting those core, pathological stutters, or is it more about improving how well the model handles the typical ones? Because they mentioned a specific mechanism in their architecture that seems to target different things.

Rosa: For core stutters, their hierarchical attention mechanism allows the model to focus on micro-level energy spikes within those frame nodes, which means it’s concentrating on those small acoustic disruptions while ignoring the surrounding fluent context.

Dev: That sounds like a good way to isolate the stuttering pattern. But they also have this learnable gated residual fusion that dynamically weights the spatial graph features against temporal recurrent neural network features, which helps prioritize local acoustic spikes for repetitions or long-range rhythmic patterns before passing that information to the final classifier.

The paper's improvements: Rosa: So, wrapping up the Paediatric-HGNN paper, what we see is a framework that moves away from simple one-dimensional signal models by building a rich heterogeneous graph structure for speech analysis. It’s explicitly designed to model how words and their acoustic frames interact across different scales.

Dev: The main implication here is that for early clinical intervention, this model offers a more interpretable tool because the hierarchical attention weights show exactly where the system is focusing its attention—either on local acoustic spikes or broader word context.

Taro: For someone listening who isn't a data scientist, what does it actually mean in terms of diagnosis? It means that instead of just getting a single score, you get a model that can point to specific parts of the speech structure that are causing the issue.

Rosa: Right. And while they achieved an eighty-two point four weighted accuracy and a typical disfluency F1 score of zero point three eight six, they also highlighted their limitations, which is important. They noted that in their evaluation protocol, they used a five fold cross-validation strategy to ensure clinical robustness, but they also acknowledge the inherent difficulty in paediatric data scarcity.

Dev: And one limitation mentioned is that the performance on the consolidated three class clinical taxonomy showed a mean core F1 of zero point two eight zero with a standard deviation of zero point zero six, which suggests there’s still room for improvement in accurately categorizing all types of disfluency perfectly <ref:2606.08210#pg1>.

Taro: So, to sum up, the Paediatric-HGNN paper shows a promising way to use complex graph models to tackle the unique acoustic challenges of children's speech by incorporating both lexical and acoustic context.

Rosa: It’s definitely an interesting piece of work that shows how you can tailor deep learning architectures for very specific clinical domains like paediatrics, and it sets a benchmark for using these kinds of networked models in speech analysis.

Conclusion: Rosa: So we just talked about Paediatric-HGNN, which is this hybrid heterogeneous graph neural network they built to detect disfluency in children's speech by mixing acoustic and lexical information across different scales.

Dev: Right, it’s basically using a graph structure where word nodes talk to acoustic frame nodes hierarchically to figure out those developmental patterns.

Taro: I liked hearing how they focused the attention mechanism on micro-level energy spikes for core stutters, ignoring the surrounding fluent context.

Rosa: That’s exactly what they did, and it shows a real ability to isolate those specific acoustic events within the speech structure itself. It gives clinicians a clearer picture of what’s going on at the smallest level.

Dev: And they didn't just stop there; they used that learnable gated residual fusion to dynamically weight how much importance the spatial graph features get compared to those temporal recurrent neural network features.

Taro: That sounds like a solid control for when things get messy in real-time processing, making sure it prioritizes the local acoustic spikes or rhythmic patterns before making a final call.

Rosa: It’s definitely an architecture designed to be robust, even when the input data is as variable as paediatric speech is. The transfer learning step they mentioned shows they were thinking about how this could actually move from a lab setting into real clinical trials outside of the controlled environment.

Dev: And that domain shift measurement is key; it proves whether you can take an adult model and actually make it useful when the fundamental speech characteristics are completely different, which is a big hurdle in any speech recognition system.

Taro: It makes sense because if you can’t handle the difference between adult motor-speech issues and typical developmental disfluencies, the tool isn't really general purpose enough for what they are trying to do.

Rosa: So to wrap up, Paediatric-HGNN is showing that by explicitly modeling those hierarchical lexical-acoustic interactions, we can build tools that are sensitive enough to distinguish between pathology and normal development in children’s speech.

Dev: It gives clinicians a method for looking beyond just the sound and seeing the relationship between the word structure and the acoustic delivery.

Taro: It’s a good demonstration of how graph theory can handle these complex, multi-layered speech problems when you give it the right nodes and edges to follow.

Rashini Liyanarachchi, Rachael Mackay, Alison Short, Aditya Joshi, Erik Meijering

University of New South Wales (UNSW) · Western Sydney University (WSU) · Resourced Music Therapy

eess.AS, cs.CL, cs.SD

Submitted: 2026-06-06

Updated: 2026-10-31

Comments: Proceedings of Interspeech 2026

DOI: 10.21437/Interspeech.2026-1131

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 77/100

The gist: Automated stuttering detection (ASD) systems struggle with paediatric speech due to high acoustic variability in developing voices and the subtle distinction between pathological stuttering and…

Key concepts

Hierarchical Lexical-Acoustic Interactions
This concept describes how the model connects individual words (lexical units) to the specific acoustic sounds (frames) that make up those words. It captures the way children search for correct speech by linking word nodes to their constituent acoustic segments, providing a multi-level view of speech structure.
Heterogeneous Graph Neural Network (HGNN)
An HGNN is a type of neural network that models data as a graph with different types of nodes and edges. In this paper, it uses word nodes and frame nodes connected by different types of edges (sequential, contextual), allowing the model to understand complex relationships between linguistic units and acoustic features.
Contextual Edges
These are special connections in the graph that link a word node to its neighboring words (within ±2 words). This feature is crucial because it helps the model recognize typical developmental disfluencies, as removing these edges significantly reduced performance on identifying these common revisions.

Terminology

Summary

Automated stuttering detection (ASD) systems struggle with paediatric speech due to high acoustic variability in developing voices and the subtle distinction between pathological stuttering and typical developmental disfluencies Paediatric-HGNN achieves 82. p. 0

Where the paper says it

au Abstract Automated stuttering detection (ASD) systems struggle with paediatric speech due to high acoustic variability in developing voices and the subtle distinction between pathological stuttering and typical developmental disfluencies. We introduce Paediatric-HGNN, a framework using a Context-aware Part-whole Interaction Network (CaPIN) tailored for paediatric data.

4% weighted accuracy and a Typical Disfluency F1-score of 0. p. 0
Where the paper says it

Trained on curated paediatric corpora (UCLASS and FluencyBank), Paediatric-HGNN achieves 82. 4% weighted accuracy and a Typical Disfluency F1- score of 0.

386 Modelling hierarchical lexical-acoustic interactions captures developmental “searching” behaviour, offering a more robust and interpretable tool for early clinical intervention

The Core Problem Addressed

Stuttering is a neuro-developmental communication disorder characterised by disruptions in the forward flow of speech, which affects approximately 5% to 8% of children during their preschool years Early diagnosis is critical, yet current clinical assessments largely rely on subjective manual observations by Speech-Language Pathologists (SLPs), which can lead to high inter-rater variability and significant time costs Most automated stuttering detection (ASD) models are optimised for adult speech through datasets like SEP-28k These models often fail to generalise to the unique acoustic and linguistic variability of children, as they have not been extensively trained or evaluated on the higher fundamental frequencies and the phonological characteristics of paediatric populations Paediatric speech presents a unique diagnostic challenge because early childhood language development involves a natural phase of typical disfluency with significant acoustic and linguistic overlap between core stuttering (e.g., pathological blocks and part-word repetitions) and typical disfluencies (e. p. 0

Where the paper says it

Paediatric speech presents a unique diagnostic challenge: unlike adult speech, early childhood language development involves a natural phase of typical disfluency with significant acoustic and linguistic overlap between core stuttering (e. , pathological blocks and part-word repetitions) and typical disfluencies (e.

g., developmental whole-word repetitions and revisions)

How the training signal is built

The model builds a heterogeneous graph capturing hierarchical relationships between lexical units (word nodes) and fine-grained acoustic segments (frame nodes) Nodes include word nodes, initialized with the 945D hybrid vector, and frame nodes, representing acoustic windows initialized with Wav2Vec2 embeddings Hierarchical Edges (Eh) connect each frame node to its parent word node based on temporal alignment Sequential Edges (Es) connect adjacent words to model the forward flow of speech Contextual Edges (Ec) provide a context window of ±2 words for nodes to aggregate information from their local linguistic neighbourhood The 945-dimensional hybrid feature vector for each word node integrates deep self-supervised contextual embeddings from a Wav2Vec2-base-960h1 model, mel-spectrogram textures (via global mean and max pooling) to capture the visual signatures of disfluencies, and traditional handcrafted acoustic features

Where the gains come from

The Paediatric-HGNN architecture uses a hierarchical cross-modal attention mechanism where word nodes (Queries) selectively attend to relevant acoustic artifacts in their constituent frames (Keys/Values) This is followed by two stages of relational graph convolutions (GATv2Conv) across sequential and contextual (±2 words) edges to capture local and neighbourhood disfluency patterns A learnable gated residual fusion dynamically weights spatial GNN features against temporal RNN features: H = g·HRNN+(1−g)·HGNN The model is optimised using AdamW [18] and a Focal Loss function [19] (γ = 2) to mitigate the impact of class imbalance Class-specific weights α = [1, 3, 3] force the model to prioritise the minority (pathological) classes over the majority (fluent) class

Performance and Clinical Utility

Paediatric-HGNN achieved a stable overall accuracy of 82.4% ± 2.7% The model demonstrated high reliability in identifying Fluent speech (F1 0. p. 2

Where the paper says it

Results and Discussion Paediatric-HGNN, trained from scratch on paediatric data, achieved a stable overall accuracy of 82. Notably (Table 1), the model demonstrated high reliability in identifying Fluent speech (F1 0.

904 ± 0.02) The Typical Disfluency class reached a peak F1-score of 0. p. 2
Where the paper says it

Notably (Table 1), the model demonstrated high reliability in identifying Fluent speech (F1 0. The Typical Disfluency class reached a peak F1-score of 0.

43 in specific folds (e. p. 2
Where the paper says it

To avoid information leakage, the data was partitioned such that all recordings from a specific speaker were confined to a single fold. As performance metrics we used precision, recall, F1 score, and overall accuracy, reported as the weighted averages across all 5 folds.

g., Fold 1) A key finding is the significant performance degradation when utilising adult-trained models for paediatric diagnostics, where Typical Disfluency detection collapsed to an F1 score of 0. p. 2
Where the paper says it

This confirms that the integration of lexical word nodes provides the necessary context to categorise developmental revisions that are often mislabelled in purely acoustic models. Failure of Adult SOTA Models for Paediatric Speech: A key finding of this study is the significant performance degradation when utilising adult-trained models for paediatric diagnostics.

08 The “white-box” nature of our CaPIN framework allows clinicians to inspect the hierarchical attention weights For Core Stutters, the model attends heavily to micro-level energy spikes in the frame nodes (Figure 3A) For Typical Disfluencies, attention is distributed across the word nodes and their ±2 word context

Comparative Analysis

The model's performance on the consolidated 3-class clinical taxonomy showed a Mean Core F1 of 0.280 ± 0.06 This performance confirms that the integration of lexical word nodes provides the necessary context to categorise developmental revisions that are often mislabelled in purely acoustic models The ablation study showed that removing contextual edges resulted in a drop in Typical Disfluency F1 from 0. p. 2

Where the paper says it

This confirms that the integration of lexical word nodes provides the necessary context to categorise developmental revisions that are often mislabelled in purely acoustic models. Failure of Adult SOTA Models for Paediatric Speech: A key finding of this study is the significant performance degradation when utilising adult-trained models for paediatric diagnostics.

386 to 0.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improve generalization across speech domains by implementing a transfer learning pipeline where an adult-pretrained backbone is fine-tuned on paediatric data to empirically measure the domain shift between adult motor-speech pathologies and paediatric developmental disfluencies.

  2. Enhance core stuttering detection by utilizing the hierarchical attention mechanism, which allows the model to focus on micro-level energy spikes in the frame nodes (Figure 3A) for Core Stutters, effectively ignoring the surrounding fluent context.

  3. Increase sensitivity to typical developmental revisions by ensuring contextual awareness is maintained by incorporating contextual edges (Contextual Edges (Ec) provide a context window of ±2 words for nodes to aggregate information from their local linguistic neighbourhood), as the ablation study showed that removing these edges caused the Typical Disfluency F1 score to drop significantly.

  4. Improve model stability and performance by using a learnable gated residual fusion, which allows prioritising local acoustic “spikes” for repetitions or long-range rhythmic patterns for hesitations before passing the representation to a 3-class multilayer perceptron.

Abstract

Automated stuttering detection (ASD) systems struggle with paediatric speech due to high acoustic variability in developing voices and the subtle distinction between pathological stuttering and typical developmental disfluencies. We introduce Paediatric-HGNN, a framework using a Context-aware Part-whole Interaction Network (CaPIN) tailored for paediatric data. Instead of conventional 1D signal modelling, our approach builds a heterogeneous graph capturing hierarchical relationships between lexical units (word nodes) and fine-grained acoustic segments (frame nodes). Trained on curated paediatric corpora (UCLASS and FluencyBank), Paediatric-HGNN achieves 82.4% weighted accuracy and a Typical Disfluency F1-score of 0.386. Modelling hierarchical lexical-acoustic interactions captures developmental "searching" behaviour, offering a more robust and interpretable tool for early clinical intervention.

Sources

Related papers