Evidence for Phenotype-Driven Disparities in Freezing of Gait Detection and Approaches to Bias Mitigation

summary

Video file (mp4)

The gist

The gist The systematic evaluation of bias and fairness in state-of-the-art human activity recognition models for freezing of gait detection revealed that conventional mitigation techniques were

In short

Systematic evaluation of human activity recognition models for freezing of gait detection found conventional bias mitigation ineffective. Transfer learning from multi-site datasets showed the most promise in reducing disparities across FOG phenotypes and demographics. The study concludes that proactive data collection, including stratified sampling by phenotype and age/sex, is necessary to achieve equitable detection.

Key concepts

Freezing of Gait (FOG) Phenotypes
These are different types of freezing episodes associated with Parkinson's disease, such as tremulous or akinetic FOG. The paper found that certain phenotypes are significantly underrepresented in existing datasets, leading to performance disparities when models are trained on them.
Group Fairness Metrics
These metrics quantify fairness by measuring model performance across different groups defined by protected attributes like phenotype, age, and sex. Key measures include Demographic Parity (equal positive predictions) and Equalized Odds (equal true positive/false positive rates).
Transfer Learning
This involves taking a model trained on one set of data and adapting it to a new task or dataset. The study found that using transfer learning from multi-site datasets significantly improved fairness metrics, suggesting exposure to diverse data helps create more robust and equitable models.
Intentional Sampling Strategies
Instead of just fixing errors after training, this means actively collecting more data points specifically from underrepresented groups, such as rare FOG phenotypes or specific age/sex combinations. This proactive step is crucial because the performance gaps often stem from genuine differences in how different phenotypes move.

Terminology used across episodes

This episode discusses

The paper

Evidence for Phenotype-Driven Disparities in Freezing of Gait Detection and Approaches to Bias Mitigation · Read on arXiv

Timothy Odonga, Christine D. Esper, Stewart A. Factor, J Lucas McKay, Hyeokhyen Kwon

Emory University · Georgia Institute of Technology

Freezing of gait (FOG) is a debilitating feature of Parkinson's disease (PD) and a common cause of injurious falls. Recent advances in wearable-based human activity recognition (HAR) have enabled FOG detection, but bias and fairness in these models remain understudied. Bias is defined as a systematic error leading to unequal outcomes, while fairness refers to consistent performance across subject groups. Biased models could systematically underserve patients with specific FOG phenotypes or demographics, widening care disparities. We evaluated the bias and fairness of state-of-the-art HAR models for FOG detection across FOG phenotypes and patient demographics using 4 public wearable datasets (145 subjects and 59.8 hours of recording) with multiple on-body sensor locations and temporal annotations on FOG episodes across various behavioral test protocols. Four mitigation approaches were assessed: conventional methods (threshold optimization and adversarial debiasing) and two transfer learning-based approaches (multi-site transfer and fine-tuning pre-trained models). Fairness was primarily quantified using the demographic parity ratio (DPR) and the equalized odds ratio (EOR). HAR models exhibited substantial bias (DPR & EOR < 0.8) across age, sex, disease duration, and critically, FOG phenotype. This phenotype-specific bias is particularly concerning as tremulous and akinetic FOG may require different clinical management. Conventional methods failed: threshold optimization (DPR=-0.126, EOR=+0.063) and adversarial debiasing (DPR=-0.008, EOR=-0.001). In contrast, multi-site transfer learning improved fairness (DPR=+0.037, p<0.01; EOR=+0.045, p<0.01) and performance (F1-score=+0.020, p<0.05), though neither metric reached 0.8. Transfer learning across diverse datasets may partially mitigate these disparities, helping wearable-based FOG monitoring benefit all individuals with PD.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Evidence for Phenotype-Driven Disparities in Freezing of Gait Detection and Approaches to Bias Mitigation".

Jane: Here's the gist of the paper. The gist The systematic evaluation of bias and fairness in state-of-the-art human activity recognition models for freezing of gait detection revealed that conventional mitigation techniques were often ineffective, while transfer learning approaches showed the most promise for reducing disparities across FOG phenotypes and demographics.

Tom: Freezing of gait (FOG) is a debilitating symptom of Parkinson’s disease (PD) that is challenging for clinicians to capture and evaluate during time-limited clinical assessments.

Jane: This phenotype-specific bias is particularly concerning as tremulous and akinetic FOG may require different clinical management.

Tom: Few of these models have been comprehensively evaluated for performance across phenotypes, which could potentially introduce systematic error.

Title and authors: Tom: Now they test four different ways to try and fix this problem. These include conventional methods like threshold optimization we talked about, adversarial debiasing which tries to make the features attribute-agnostic >

Lu: And then they introduce two transfer learning based approaches. One is multi-site transfer from datasets that have more diverse examples, and the other is fine-tuning large pretrained foundation models >

Meng: I’m curious about the cost here. Transfer learning sounds promising because it exposes the model to more data, but what does that actually do for us when we're running this on a live system >

Jane: Well, they show that transfer learning from multi-site datasets was the most effective approach overall in terms of both fairness and performance improvements across those models >

Tom: They found multi-site transfer increased the F1-score by zero point zero two four for Model A2 and plus zero point zero one seven for Model A3, and it improved demographic parity by zero point zero three seven >

Lu: And that same transfer learning also improved equalized odds by zero point zero four five for both architectures, which is a solid improvement over the baseline > <ref:2502.09626#pg2>

Meng: So, it’s not just about getting a higher score; it's about seeing these specific metrics move in the right direction across those protected groups >

Lalam: They did show one experiment where transfer learning from Model B2 on one dataset actually managed to reduce bias down to meet that four-fifths rule threshold, but they noted it came with a substantial performance cost >

Jane: So, the message here is that transferring knowledge across more diverse sites is a much better path than just trying different local tuning methods >

Tom: It really shows that the way we train these models matters more than just picking the right hyperparameter for a fixed architecture >

The paper's summary: Tom: The main improvement they suggest is shifting focus from just accuracy to systematically assessing differential performance across different FOG manifestations >

Lu: They argue that best practices for evaluating this differential performance haven't really been established, and they point out a gap where prior studies have shown higher performance for tremulous FOG compared to akinetic FOG with an intraclass correlation coefficient of roughly zero point eight six versus zero point seven eight > <ref:2502.09626#pg3>

Meng: So, they are calling out that the existing benchmarks aren't looking at this phenotype stratification properly, which is a huge issue if we want reliable clinical tools >

Jane: They suggest that since tremulous and akinetic episodes co-occur in over eighty percent of patients, systematically evaluating performance differences across those two manifestations is really important > <ref:2502.09626#pg3>

Lalam: The paper concludes that achieving equitable FOG detection requires proactive considerations during data collection, specifically stratified sampling by phenotype and demographics, rather than just relying on downstream bias mitigation >

Tom: That’s a strong stance. They aren't just saying "mitigate the bias after training," they are saying we need to change how we collect the data in the first place >

Lu: They suggest that future efforts might require intentional sampling strategies that explicitly enrich underrepresented FOG phenotypes and demographic subgroups during data collection >

Meng: I worry about implementing that. Stratified sampling by phenotype sounds incredibly labor-intensive, and it adds complexity to any production pipeline we have to manage >

Jane: But they argue that the performance differences arise from genuine pathophysiological differences in motor mechanisms between those phenotypes, so you can't just ignore them >

The paper's improvements: Tom: So, to wrap up the Evidence for Phenotype-Driven Disparities in Freezing of Gait Detection and Approaches to Bias Mitigation paper: the key finding is that transfer learning from multi-site datasets was substantially better than conventional mitigation methods at reducing disparities >

Lu: And they emphasize that for developing equitable HAR models that reliably detect FOG across all patient phenotypes, transfer learning across diverse datasets is essential >

Meng: From a practical side, it means we need to look beyond just the model weights and think about the data pipeline itself if we want something that actually works reliably in the real world >

Jane: It seems like they’re pushing for a proactive approach where you sample specifically for those underrepresented groups instead of just patching problems later >

Lalam: They really put it on data collection. The paper is about moving toward intentional sampling strategies to enrich those FOG phenotypes and demographic subgroups during data collection >

Tom: So, we're done with this one. It’s clear that for FOG detection, the path forward involves looking at how we collect the data rather than just tweaking the model after it’s been built >

Jane: We’ll see what other papers come out next that build on these ideas >

Conclusion: Tom: So to wrap up, the paper "Evidence for Phenotype-Driven Disparities in Freezing of Gait Detection and Approaches to Bias Mitigation" shows that standard models are hitting a wall when it comes to fairness across different FOG phenotypes and demographics >

Jane: Right. The main thing is that the performance improvements we see in these state-of-the-art human activity recognition models don't actually translate into better fairness metrics like demographic parity or equalized odds >

Lu: Exactly. The authors systematically evaluated five different AI models and found that increasing architectural complexity didn't fix the bias; it actually made things worse for fairness metrics >

Meng: From a deployment standpoint, if you’re looking at those baseline performance gains without any fairness fixes, you’re just baking systematic errors into your production system >

Lalam: The paper strongly points to transfer learning as the most promising mitigation strategy because multi-site transfer increased F1-scores by zero point zero two four for Model A2 and improved demographic parity by zero point zero three seven >

Tom: So it’s not just about finding a better threshold; it's about moving the model to see more diverse examples before you even start tuning parameters >

Jane: And they found that transfer learning from multi-site datasets was the most effective approach overall in terms of both fairness and performance improvements across those models >

Lu: They showed that this transfer learning actually helped reduce disparities, and one experiment with Model B2 on one dataset managed to hit the four-fifths rule threshold, but it came at a substantial cost to its accuracy >

Meng: That performance cost is something I’d have to worry about before I deploy anything, especially when dealing with patient data where every missed detection matters >

Lalam: They also highlighted that FOG phenotype itself is a major source of bias because akinetic episodes are way underrepresented in the datasets and that’s what causes the performance disparities >

Tom: So they’re calling for intentional sampling strategies during data collection, specifically enriching those underrepresented FOG phenotypes and demographic subgroups >

Jane: That means we need to stop just patching the model after training and start thinking about how we collect our data from the beginning >

Lu: They suggest that a future direction might be developing phenotype-specific models where you train separate models for distinct FOG manifestations rather than one general system >

Meng: Stratified sampling sounds incredibly labor-intensive, but if it yields a truly robust and safe model, then maybe we’ll have to consider it for high-stakes applications >

Lalam: So that’s the gist of "Evidence for Phenotype-Driven Disparities in Freezing of Gait Detection and Approaches to Bias Mitigation" – the takeaway is proactive data collection over reactive mitigation >

Tom: Yeah, it highlights that just achieving high accuracy isn't enough if you don't control for those underlying clinical differences >

Jane: It really makes you think about how we build these systems for people who actually have these conditions and need reliable detection across all of them >

Lu: We’ll look at how other papers are tackling domain gaps in the next segment.

More episodes

← Home