From Black-Box to Clinical Insight: A Multi-Stage Explainable Framework for Speech-Based Cognitive Impairment Detection

summary

Video file (mp4)

The gist

The gist The framework proposes a multi-stage explainability system that translates black-box transformer predictions into clinically grounded narratives by integrating SHAP attribution, linguistic

In short

The framework translates complex speech recognition model predictions into understandable clinical narratives for cognitive impairment detection. It combines SHAP attribution, linguistic features, and a four-stage LLM pipeline to map model outputs onto cognitive dimensions like lexical richness. This bridges the gap between black-box AI and actionable physician insights.

Key concepts

SHAP Adaptation
This technique adjusts SHAP values for transformer models that use subword tokenization. It processes the input text through the tokenizer, calculates embeddings, and then aggregates the importance scores from individual subwords back to meaningful word-level explanations.
Linguistic Features
These are quantifiable measures of speech patterns relevant to cognition. The framework extracts features like lexical richness (word variety), syntactic complexity (sentence structure), disfluencies, and semantic coherence to provide concrete evidence alongside AI predictions.
LLM Reasoning Pipeline
A four-stage process using the LLaMA-3.1-70B model to synthesize information. It first interprets token importance, then linguistic features, compares these two sources, and finally generates a structured summary report for clinicians.

Terminology used across episodes

This episode discusses

The paper

From Black-Box to Clinical Insight: A Multi-Stage Explainable Framework for Speech-Based Cognitive Impairment Detection · Read on arXiv

Yasaman Haghbin, Sina Rashidi, Ali Zolnour, Fatemeh Taherinezhad, Ali Fartoot, Hossein Azadmaleki, James M Noble, Maryam Dadkhah, Maryam Zolnoori

Columbia University · Chalmers University of Technology

DOI: 10.21437/Interspeech.2026-1252

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "From Black-Box to Clinical Insight".

Jane: The gist The framework proposes a multi-stage explainability system that translates black-box transformer predictions into clinically grounded narratives by integrating SHAP attribution, linguistic features, and an LLM reasoning pipeline.

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, what it claims is that this framework takes the transformer predictions and translates them into clinically grounded narratives by using SHAP based token attribution, theory informed linguistic features, and a four stage LLM reasoning pipeline using LLaMA-three point one 70B Instruct > <ref:2606.27973#pg1,SHAP based token attribution, theory informed linguistic features, and a four stage>

Jane: It maps those model outputs to four cognitive linguistic dimensions like lexical richness syntactic complexity semantic coherence and also impaired executive function difficulty with spatial reasoning >

Tom: And they built this on the SpeechCARE Adaptive Gating Network multimodal screening model which already has an AUC of eighty-six point eighty-three percent and an F1 score of seventy-two point eleven percent on the PREPARE challenge test set >

Lu: The method uses SHAP adapted for transformer architectures with hierarchical aggregation to map subword level attributions back to word level explanations >

Meng: And they use linguistic features across four domains lexical richness syntactic complexity disfluencies and repetition and semantic coherence >

Lalam: Then they chain four specialized LLMs using LLaMA-three point one 70B Instruct, where the first stage interprets SHAP scores, the second interprets those linguistic features, the third compares them to integrate evidence, and the fourth generates a structured report > <ref:2606.27973#pg1,using LLaMA-3.1 70B Instruct>

Tom: That sounds like a lot of sequential reasoning just to get from a prediction to an explanation >

Jane: It is quite complex because they are trying to link what the model sees with what we actually call cognitive function through those structured steps >

Conclusion: Tom: So, looking at this paper "From Black-Box to Clinical Insight: A Multi-Stage Explainable Framework for Speech-Based Cognitive Impairment Detection" and its authors Yasaman Haghbin Sina Rashidi Ali Zolnour Fatemeh Taherinezhad Ali Fartoot Hossein Azadmaleki James M Noble Maryam Dadkhah Maryam Zolnoori >

Jane: The title itself tells you the core idea is moving away from just a black box model to getting actual clinical insight through explanation. It's about bridging that gap between the prediction and what it means for a patient.

Lu: What this does is show that we can sequentially combine SHAP attributions, theory informed linguistic features, and LLM based reasoning to bridge the gap between black box transformer predictions and clinically meaningful narratives >

Meng: It's important because speech analysis gives us a functional complement to things like plasma assays which reflect pathology but don't capture the functional impact on communication >

Lalam: The framework maps model outputs to four cognitive linguistic dimensions including lexical richness syntactic complexity and semantic coherence, and physician evaluation showed strong alignment with patient level cognitive profiles >

Tom: And the system usability scale score of eighty-two out of one hundred suggests this has high potential for getting integrated into actual clinical workflows > <ref:2606.27973#pg1>

Jane: It positions sequential LLM orchestration as a promising way forward for explainable AI in cognitive screening, showing that we can actually get meaningful narratives from these complex models >

More episodes

← Home