LSEAD: A Privacy-Preserving LLM-Based Speech Analysis Framework for Early Alzheimer's Disease Screening

This paper proposes LSEAD (LLM-based Speech-assisted Early AD detection), a privacy-preserving framework for early Alzheimer's disease (AD) screening that leverages locally deployed, open-source large language models (LLMs) to analyze spontaneous patient speech. The authors state: "In this study, we propose an LLM-based Speech-assisted Early AD (LSEAD) detection framework. It leverages the linguistic and clinically relevant knowledge encoded in pretrained LLMs to capture cognitive-linguistic AD markers without relying on hand-crafted features." The framework is designed as an end-to-end, deployable system that supports non-invasive and privacy-preserving cognitive assessment in real-world clinical environments.

The paper highlights that Alzheimer's disease is the leading cause of dementia among older adults and is characterized by the progressive deterioration of memory, language, and cognitive function and accounts for approximately 60–70% of all dementia cases worldwide. Conventional diagnostic approaches—including clinical evaluations, neuropsychological testing, MRI, and PET—are often resource-intensive and costly, require specialized equipment and trained clinical personnel, and are typically confined to hospital or specialized clinical settings.

The authors argue that speech-based analysis is among the most promising non-invasive alternatives: Speech is a natural, low-burden signal that can be collected unobtrusively without specialized equipment, making it particularly well suited for scalable, remote, and repeated screening. Speech data are typically elicited using picture description tasks (specifically the Cookie Theft picture from the Boston Diagnostic Aphasia Examination), which simultaneously engage semantic memory, lexical retrieval, syntactic planning, and discourse organization—cognitive processes known to be affected early in AD.

A central motivation is the privacy and deployment limitation of commercial LLMs: "Many high-performing commercial LLMs, such as ChatGPT, are closed-source and rely on cloud-based infrastructures that are not fully compliant with the Health Insurance Portability and Accountability Act (HIPAA), raising significant concerns regarding the handling of sensitive patient data." The authors therefore adopt Zephyr-7B-β, a chat-optimized, open-source model, noting that unlike commercial LLMs, open-source models such as Zephyr-7B-β can be deployed entirely within secure, on-premises clinical environments. They emphasize that since Zephyr is used exclusively for embedding extraction rather than text generation, its beta-stage limitations are not critical: the primary requirement is the model's ability to encode stable and discriminative linguistic representations from patient speech, rather than to generate precise or up-to-date textual outputs.

The framework is evaluated on two benchmark datasets:

• ADReSS20: contains 54 AD and 54 cognitively normal (CN) subjects in the training cohort, and 24 AD and 24 CN subjects in the test cohort (78 AD and 78 CN total).

• ADReSSo2021: includes 87 AD and 79 CN subjects in the training cohort, along with 35 AD and 36 CN subjects in the test set (122 AD and 215 CN total).

Both datasets consist of spontaneous speech from the Cookie Theft picture description task, with clinical information such as MMSE scores. To ensure consistency, the authors retained only the full enhanced audio from both datasets for all experiments.

The framework follows a sequential processing pipeline (Figure 1):

1. Speech Signal Preprocessing: Raw audio is resampled to a fixed sampling rate, amplitude-normalized, processed with voice activity detection to remove silence, and framed.

2. Automatic Speech Recognition (ASR): Preprocessed speech is converted to text transcripts. The authors note this intentionally prioritizes linguistic content over acoustic features, motivated by prior studies demonstrating that textual information captures the most salient cognitive and semantic impairments associated with AD. Two ASR systems are evaluated: Wav2Vec 2.0 and Whisper.

3. Text Embedding Extraction: Transcripts are passed through the pretrained LLM (Zephyr-7B-β). The authors extract high-level feature representations from the penultimate layer of the LLM (specifically, hidden states from the final transformer layer). An utterance-level embedding is computed using attention-mask–aware mean pooling to produce a fixed-dimensional utterance-level embedding... while preventing padding artifacts and ensuring stable representations across transcripts of varying lengths.

4. Dimensionality Reduction via PCA: "To improve computational efficiency and reduce redundancy in high-dimensional embeddings, PCA is employed to project the feature space into a lower-dimensional representation while preserving the most informative variance." The explained variance threshold is selected via grid search from [0.9, 0.95, 0.97, 0.99, 0.999].

5. Binary Classification: PCA-reduced features are fed to a supervised classifier (logistic regression, SVC, XGBoost, or neural networks) to distinguish AD from CN with a decision threshold of τ = 0.5.

The combined dataset setting (training on 274 samples, testing on 119 samples) yielded the following results (Table 2):

• Logistic Regression (LR) achieved the best overall performance: 90.0% accuracy, 91.2% precision, 88.1% recall, and 89.7% F1 score.

• SVC and XGBoost both achieved 82.4% accuracy, while neural networks achieved 78.2%.

The authors explain that the cross-validation results indicate that LR and SVC achieve the most stable and competitive performance and that the consistent superiority of LR further supports the effectiveness of the PCA-reduced LLM embeddings, as linear classifiers are well-suited to capturing discriminative patterns in low-dimensional feature spaces. They emphasize that PCA is critical: removing PCA and training classifiers directly on the high-dimensional LLM embeddings consistently degrades performance across all models, because the raw embedding space contains redundant and noisy dimensions that obscure class-discriminative structure, which is especially detrimental in small-sample settings. The PCA visualization (Figure 2) reveals a clear linear separation trend between CN and AD samples, with CN samples predominantly in the upper-right region and AD samples clustered toward the lower-left.

A key clinical objective is early-stage AD detection. The authors analyzed classification outcomes against MMSE scores (Figure 3), finding that correctly identified AD cases span a broad MMSE range, with a mean score of 19.4 ± 7.3, covering moderate and mild cognitive impairment and extending toward the early-stage region near the clinical cutoff. Notably, a substantial proportion of true positives fall within the mild impairment range (MMSE 19–23), where cognitive deficits are often subtle and more difficult to detect using conventional screening tools. This indicates the framework is sensitive to early linguistic alterations associated with AD, even when global cognitive impairment remains relatively mild. False negatives were predominantly concentrated in the higher MMSE range, with a mean score of 23.8 ± 4.6, corresponding largely to the transition between mild impairment and no cognitive impairment, reflecting the inherent difficulty of detecting very early or preclinical AD.

The proposed LSEAD framework was compared against representative state-of-the-art methods (Table 3):

• Mortensen et al.: 84.9% accuracy

• Bang et al.: 83.1% accuracy

• Agbavor et al.: 80.3% accuracy

• Luz et al. (ADReSSo baseline): 78.9% accuracy

• LSEAD (Ours): 90.0% accuracy, 91.2% precision, 88.1% recall, 89.7% F1

The authors emphasize that our approach consistently achieves the strongest results, demonstrating clear advantages over existing techniques, and that the proposed method attains an accuracy of 90.0%, outperforming Mortensen et al. by 5.1%, Bang et al. by 6.9%, Agbavor et al. by 9.7%, and the ADReSSo challenge baseline reported by Luz et al. by 11.1%. They also note the high precision is critical for clinical use: high precision reduces the risk of incorrectly flagging cognitively normal individuals. Compared to GPT-based approaches, the authors note that Kheirkhahzadeh et al. reported an accuracy of only 62% using GPT-3.5 with XGBoost, highlighting the limitations of direct API-based commercial LLM integration.

The authors compared Zephyr-7B-β against two alternative open-source LLMs within the same pipeline:

• Llama 2-7B achieved a maximum accuracy of 83.2% with LR, consistently underperforming the proposed Zephyr-7B-β framework and exhibiting comparatively lower recall and F1 scores, suggesting reduced sensitivity to subtle linguistic markers associated with early-stage cognitive decline.

• Qwen3-30B, despite being substantially larger, achieved a best of 87.4% accuracy and 87.0% F1 with LR. The authors conclude: increased model scale alone does not guarantee superior performance for speech-based AD detection, and highlights the importance of instruction tuning and task-aligned representation learning over sheer model size.

ROC analysis showed The proposed Zephyr model achieves a high area under the ROC curve (AUC) of 0.95±0.034, compared to the Qwen3 baseline achieves an AUC of 0.92 ± 0.035. The Zephyr-based framework consistently achieves higher true-positive rates than the baseline model, particularly in the low-false-positive region, which is clinically important for minimizing unnecessary follow-up assessments and reducing patient burden in screening scenarios.

To assess robustness to transcription quality, two ASR systems were evaluated while keeping all other pipeline components fixed:

• Wav2Vec 2.0: LR achieved 88.2% accuracy and 87.7% F1.

• Whisper: LR achieved the best results overall—90.0% accuracy and 89.7% F1—while SVC reached 87.4% accuracy.

The authors attribute Whisper's superior performance to its robustness to speaker variability, background noise, and heterogeneous recording conditions, which are common in spontaneous clinical speech datasets. They conclude that the proposed framework is not tightly coupled to a specific ASR system, demonstrating stable performance across different ASR systems is thus essential for ensuring clinical reliability.

Cross-dataset experiments assessed robustness to domain shifts:

• Trained on ADReSS20, tested on ADReSSo21 (Table 7): LR achieved 87.4% accuracy, 90.7% precision, 83.1% recall, 86.7% F1; SVC achieved 84.9% accuracy with 91.8% precision.

• Trained on ADReSSo21, tested on ADReSS20 (Table 8): LR achieved 87.3% accuracy, 90.6% precision, 82.9% recall, 86.6% F1; SVC achieved 83.1% accuracy.

The authors conclude: "Across both cross-dataset experiments, the proposed method exhibits stable and comparable performance regardless of the training–testing direction, highlighting its strong generalization ability... These results demonstrate that the proposed framework is robust to dataset shifts and can reliably generalize across independent clinical speech datasets for AD detection."

The paper concludes that LSEAD demonstrates strong performance, robustness, and generalization capability, highlighting its potential as a practical tool for non-invasive cognitive assessment in real-world clinical environments. Key findings include:

• By leveraging pretrained LLMs, LSEAD captures subtle cognitive-linguistic patterns associated with AD that are difficult to represent using traditional hand-crafted or shallow features.

• "The incorporation of PCA plays a critical role in improving system stability by reducing feature redundancy, suppressing noise, and producing compact representations that are well suited for small-sample clinical datasets."

• "Across all evaluated classifiers, LR consistently achieves the best performance in both cross-validation and independent testing scenarios... carefully designed low-complexity models can provide reliable and robust performance."

• The framework demonstrates strong cross-dataset generalization, with only modest performance degradation observed when training and testing across ADReSS20 and ADReSSo2021 datasets.

The authors acknowledge three main limitations: the study relies on relatively small benchmark datasets, which may not fully capture the diversity of real-world patient populations, including variations in language, dialect, and comorbid conditions; the current framework focuses exclusively on textual representations derived from speech, without incorporating complementary acoustic features; and the proposed framework currently lacks interpretability, as the decision-making process of the LLM-derived embeddings and downstream classifiers is not explicitly transparent, which may limit clinical trust and adoption.

Future directions include: extending the framework to larger and more diverse datasets, including multilingual and spontaneous conversational speech; multimodal approaches that integrate acoustic and linguistic features will be investigated to enhance diagnostic performance; and "improving interpretability remains a key priority for clinical adoption; future research will focus on explainable AI techniques to identify the linguistic markers driving model predictions, thereby enhancing transparency, supporting clinician trust, and facilitating integration into clinical decision-making workflows."

The code is stated to be available at https://github.com/kelci2017/AD Text LLMs.

Based on this paper, I can improve AI systems for speech-based Alzheimer's disease screening (and similar clinical applications) by incorporating the following specific enhancements:

1. Use locally deployed, open-source LLMs for feature extraction instead of commercial APIs.

The improved system can process sensitive patient speech entirely on-premises, ensuring HIPAA compliance and protecting patient privacy while still leveraging LLM-scale linguistic knowledge.

2. Extract embeddings from the penultimate layer of an instruction-tuned chat LLM using attention-mask-aware mean pooling.

This yields fixed-dimensional, noise-robust utterance representations that capture subtle cognitive-linguistic markers of AD without hand-crafted features, and works reliably across transcripts of varying lengths.

3. Apply PCA to the LLM embeddings before classification.

This reduces feature redundancy, suppresses noise, and creates compact representations that dramatically improve classification stability and accuracy in small-sample clinical datasets—producing a system that performs well even with only a few hundred training examples.

4. Use a simple linear classifier (logistic regression) on the PCA-reduced embeddings.

The improved system achieves 90% accuracy, 91.2% precision, and 89.7% F1, with low computational overhead and less risk of overfitting compared to more complex models like neural networks or XGBoost.

5. Integrate Whisper ASR for transcription.

The improved system maintains high diagnostic performance (90% accuracy) despite background noise, speaker variability, and heterogeneous recording conditions common in real clinical speech data, and remains robust to ASR choice.

6. Choose task-aligned, instruction-tuned models over larger raw-scale models.

The improved system can use a 7B chat-optimized LLM (Zephyr) to outperform both Llama 2-7B and Qwen3-30B, demonstrating that instruction tuning is more important than model scale—saving substantial compute while improving accuracy and AUC (0.95 ± 0.034).

7. Enable effective early-stage AD detection.

The system can identify patients with mild cognitive impairment (MMSE 19–23) and early-stage AD, detecting subtle linguistic alterations that conventional screening tools often miss.

8. Achieve high precision to minimize false positives.

With 91.2% precision, the system reduces the risk of incorrectly flagging cognitively normal individuals, lowering unnecessary follow-up procedures and patient burden in screening settings.

9. Generalize across independent datasets and domain shifts.

The improved system maintains 87% accuracy when trained on one dataset and tested on another (ADReSS20 ↔ ADReSSo21), enabling reliable deployment across different clinical populations and recording conditions without retraining.

10. Provide an end-to-end, deployable pipeline from raw audio to screening output.

The system integrates speech preprocessing, ASR, LLM embedding, PCA, and classification into a single automated workflow, enabling non-invasive, low-burden, repeatable cognitive assessment in real-world clinical environments.

Early diagnosis of Alzheimer's disease (AD) is critical for enabling timely interventions that may slow disease progression and improve patient outcomes. There is a growing need for AD detection methods that are non-invasive and cost-effective, especially in real-world clinical settings with diverse patient populations and recording conditions. Speech-based screening addresses these needs by using natural speech collected without specialized equipment. Recent advances in large language models (LLMs) have improved speech analysis by providing rich linguistic representations and strong generalization. In this study, we propose LSEAD, a speech-based AD detection framework using pretrained open-source LLMs. Speech recordings are automatically transcribed, and text embeddings are extracted using locally deployed LLMs. Principal component analysis (PCA) is applied to reduce dimensionality before classification. Because the framework relies only on speech transcripts and locally deployed models, it supports privacy-preserving AD risk assessment without external data exchange. We evaluate LSEAD on the ADReSS20 and ADReSSo2021 benchmark datasets. Experimental results show that LLM-based embeddings generalize well across datasets and improve AD classification accuracy by up to 5 percent over existing methods, especially for early-stage detection. These results demonstrate that LSEAD provides a practical, secure, and scalable approach for early AD screening.