FedHUG: Federated Heterogeneous Unsupervised Generalization for Remote Physiological Measurements

arXiv:2510.12132 · cs.CV · Submitted 2025-10-14 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "FedHUG: Federated Heterogeneous Unsupervised Generalization for Remote Physiological Measurements".

Jane: The gist: The FedHUG framework proposes a novel Federated Heterogeneous Unsupervised Generalization protocol to enhance model generalization during unsupervised federated learning for remote physiological measurements.

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we're looking at this paper called "FedHUG: Federated Heterogeneous Unsupervised Generalization for Remote Physiological Measurements." It tackles a big problem where we need to update models with user data that doesn't have labels, which is a huge thing since collecting sensitive info is tricky.

Jane: Exactly. The core idea here is they propose this new protocol called Federated Unsupervised Domain Generalization, or FUDG, which then builds the FedHUG framework on top of it to make models more general when learning from remote physiological measurements.

Lu: It sounds like they are trying to solve the issue where data comes from different users and different environments, leading to big domain shifts in the data distribution.

Meng: So how do they actually handle that shift without having all those labels? That’s the million-dollar question for any unsupervised learning approach.

Tom: Well, FedHUG does this by introducing a few key parts: first, they have Minimal Bias Aggregation or MBA to deal with biases from different data collection devices and user differences in feature distributions.

Jane: And then they have the Global Distribution-aware Learning Controller, or GDLC, which is designed to regulate the training process on the client side by modeling the label distribution implicitly on the server without needing any privacy concerns for information transmission.

Lu: The MBA part seems really interesting because it evaluates how far away a client's learned semantics are from an ideal semantic-invariant point, and then it assigns higher weights to clients with smaller biases.

Meng: So if a client has a small bias, they get more say in the global update, which makes sense for reducing those device and user differences.

Tom: Right. And the GDLC part tackles the problem of label distribution skew and long-tail phenomena in physiological labels by estimating parameters of each batch training sample distribution from the unit high-dimensional feature vector v of each sample.

Jane: That estimation lets them dynamically manipulate client-specific training strategies, which helps mitigate that skew during model aggregation.

Title and authors: Lu: They also calculate a weighting coefficient VKL based on the KL divergence between local and global distributions to enhance sensitivity to those tail samples in training batches.

Meng: So if there's a big difference in how the local data is distributed compared to the global data, they use that KL divergence to figure out how important those outliers are.

Tom: And they use that coefficient VKL with other things, like processing each original input through spatio-temporal augmentation to check for semantic consistency and see if dissimilar semantics indicate a larger bias.

Jane: So in short, FedHUG is this whole system working together to guide the global model toward better generalization across those different scenarios and users.

Lu: The math shows that the objective is to minimize the expected loss of unseen distributions, but they do this by letting each client calculate an expected loss that approximates its own data distribution, which is what equation (two) describes <ref:2510.12132#pg1>.

Meng: And then the server updates the global model using a weighted aggregation formula where pi is typically proportion to local sample size, which is equation (three) <ref:2510.12132#pg3,typically proportion to local sample size>.

Tom: The results they show are quite compelling too. They found significant improvements compared to state-of-the-art methods like FedGaLA, with MAE decreasing by nine point six percent and eleven point six percent on PURE and BUAA datasets specifically.

Jane: And that improvement is quite substantial when you look at the ablation tests; without the unsupervised generalization pretraining, they saw a reduction in average MAE by twenty-six point nine six percent.

Lu: The visualization results are pretty telling too, showing that without MBA, the bias distribution had more low-similarity values, but FedHUG shows a notable improvement where most instances have a high s value.

Meng: And when they look at the feature distribution on the unit sphere, FedHUG results in a distinct clustering of tail data compared to FedAvg.

Tom: That clustering of tail data is important because it suggests that when things are not perfectly generalizable, this method is more sensitive to implicit semantic information in those tail samples.

Title and authors: Jane: It also showed opposing trends between high HR and low HR tail data distributions, which points to the sensitivity of FedHUG to implicit semantic information in those areas.

Lu: The paper does have a limitation they mention; it's that they are not doing individual-wise FUDG yet, which is something they plan for future work.

Meng: That means they are focusing on the global model first, and personalizing it later might be the next step in their research path.

Tom: So to wrap up this FedHUG paper, it introduces a system that encourages global models to focus more on less biased domains during aggregation while implicitly modeling the semantic distribution during training to address long-tail issues.

Jane: It really shows how combining these ideas, like MBA and GDLC, can lead to better performance in real-world remote physiological measurement scenarios.

Lu: This approach is quite practical because it helps the global model adapt more effectively as it sees different scenarios during the federated learning process.

Meng: From an engineering standpoint, being able to tune these parameters like sigma and gamma for GDLC seems important, as performance dips when sigma is too high or low because GDLC struggles to detect anomalous batches.

Tom: So that's where we are with FedHUG, showing strong results on challenging datasets like PURE and BUAA, with clear improvements over existing methods.

Jane: It sets a good benchmark for how unsupervised joint learning can be used effectively when you have heterogeneous data sources in federated settings.

Lu: This work suggests that the way we aggregate models needs to become smarter about both the bias and the distribution shifts happening across different domains.

Meng: The paper confirms that this technique can also enhance performance when combining different modalities, like RGB video and mmWave radar, showing superior performance across state-of-the-art techniques in estimation with either RGB video or mmWave radar.

Tom: It’s a solid piece of research that shows how to move past just supervised learning dependencies for these kinds of applications.

Jane: So the FedHUG framework gives us a concrete way to enhance model generalization when we're dealing with remote physiological measurements using unsupervised federated learning.

The paper's summary: Tom: So, to recap, FedHUG is this new way to make models better when they learn from remote physiological measurements without having all those perfect labels you need.

Jane: Right, it’s about taking data from different users and different devices and making sure the model learns something general that works across all of them.

Tom: Exactly. The authors call it the Federated Unsupervised Domain Generalization protocol, or FUDG, which then feeds into this whole FedHUG framework to boost model generalization during federated learning.

Jane: What’s interesting is how they tackle the problem of bias from different collection devices and user differences in data distribution.

Tom: They introduce Minimal Bias Aggregation, or MBA, which essentially assigns more importance to clients whose data distributions are less biased towards a specific collection method or user type.

Jane: And then they have this Global Distribution Aware Learning Controller, GDLC, which is designed to handle the skew in those labels and the long-tail problem where some label patterns are much rarer than others.

Tom: GDLC estimates how the training batches are distributed on each client, and it uses that information to dynamically adjust the global feature distribution without ever sending any private data across networks.

Jane: It’s smart because it allows the global model to become more sensitive to those rare tail samples, which usually get ignored in standard learning processes.

Tom: And the results they show are pretty striking too. They found that FedHUG significantly improves performance on challenging datasets like PURE and BUAA, with a drop in mean absolute error of about nine point six percent and eleven point six percent compared to the best existing federated method.

Jane: That’s a solid win for unsupervised joint learning, especially when you look at how much better it is than just using standard averaging techniques.

Tom: And the ablation tests really back that up, showing that without this unsupervised generalization pretraining, the model's mean absolute error actually goes up by almost twenty-seven point.

Jane: It sounds like this work moves us away from needing massive labeled datasets just to get a good model for these kinds of real-world physiological tasks.

Tom: And since they tested it on both RGB video and mmWave radar data, it suggests this approach could actually be useful across different measurement principles, not just one type of sensor.

Jane: So it really shows how combining bias mitigation and distribution awareness can make federated learning much more robust in these messy, real-world settings.

Tom: This paper gives us a solid blueprint for making models that adapt better as they see new scenarios during the learning process.

The paper's improvements: Tom: So we’ve seen how they build this system, and now we look at what it actually does for performance improvements.

Jane: Right, this paper isn't just about setting up a process; it’s about showing tangible gains when you use these new techniques on real data.

Tom: They show that the combination of MBA and GDLC really pushes the model toward a much better generalization point than what you get from standard federated averaging.

Jane: It means that instead of just getting a decent model, you’re getting one that handles data coming from totally different sources or users much more reliably.

Tom: They demonstrate this by showing a twenty-six point reduction in the average mean absolute error when they use this unsupervised generalization pretraining compared to starting with no pretraining at all.

Jane: That’s huge because it cuts down on the amount of labeled data you need just to get a decent model working across different environments.

Tom: And they show that even if you only use MBA and GDLC, applying those specific techniques on top of a basic averaging method still gives you about five point one six percent improvement across four different datasets.

Jane: That tells us the methodology itself is robust enough to handle real-world noise without needing every single parameter tuned perfectly.

Tom: The visualization results were pretty compelling too; they show that without MBA, the bias distribution had way too many low-similarity values, but FedHUG shows a much better spread of those instances.

Jane: And when they look at the unit sphere visualization, it clearly shows that FedHUG creates a distinct clustering for those tail samples you’d normally struggle with.

Tom: That clustering means the model is actually learning something about the data structure, not just memorizing what it sees in one specific scenario.

Jane: It also pointed out an interesting opposing trend between high and low heart rate tail data, which suggests the method is picking up on subtle semantic cues that other methods miss.

Tom: So what does this mean for us on the ground? It means we can deploy these kinds of systems in remote monitoring where we don't have a perfect label for every possible user or situation.

Jane: It opens up possibilities for health monitoring tools that can be more adaptable and less prone to failing when they encounter unexpected data patterns.

Tom: Lalam thinks this is important because if the underlying AI culture gets built on these models, it’s going to be much more resilient and less brittle when it interacts with diverse users.

Jane: It moves the goal from just building an accurate model for one thing to building a model that understands the underlying variability of human data itself.

Conclusion: Tom: So we’re wrapping up FedHUG, which is this whole framework for Federated Heterogeneous Unsupervised Generalization for Remote Physiological Measurements.

Jane: It’s a pretty neat paper because it tackles real-world problems—like different users and devices—by using unsupervised learning to make models smarter.

Tom: Exactly. The big implication here is that we don't have to wait for perfect labels from every single user before we can get a useful, generalized model.

Jane: It shows how you can guide the global model to focus on the less biased data sources during aggregation, which is a smart way to handle the messy reality of remote measurements.

Tom: And they proved that this works better than just using standard averaging methods when those distribution shifts are present in PURE and BUAA datasets.

Jane: It’s practical because it means we can improve the reliability of health monitoring tools without needing a massive, perfectly labeled dataset upfront for every new deployment.

Tom: Lalam thinks this is important because if the underlying AI culture gets built on these models, it’s going to be much more resilient and less brittle when it interacts with diverse users.

Jane: And Meng agrees that from an engineering standpoint, this offers a way to build systems that are inherently more adaptable as they collect data in the field.

Tom: Lu is really excited about the possibility of individual-wise FUDG down the road, because that would take this concept and apply it on a per-user basis, which is where the real personalization happens.

Jane: It’s clear that FedHUG provides a solid foundation for making remote AI more dependable.

Tom: So yeah, this paper on FedHUG shows us how to build better models in federated settings when you don't have all that perfect supervision ready to go.

Jane: Next up, we’ve got a look at some work on how AI can handle risk evaluation and testing environments.

Hong Kong University of Science and Technology

cs.CV

Submitted: 2025-10-14

Updated: 2026-10-09

Comments: This work has been submitted to the IEEE for possible publication

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

Importance score: 79/100

The gist: The gist: The FedHUG framework proposes a novel Federated Heterogeneous Unsupervised Generalization protocol to enhance model generalization during unsupervised federated learning for remote

Key concepts

Federated Unsupervised Domain Generalization (FUDG): A protocol designed to fine-tune a pre-trained model using unlabeled data collected from diverse environments during federated learning. It aims to make the global model robust against domain shifts caused by different collection devices and user differences, without needing labels.
Minimal Bias Aggregation (MBA): A method used during aggregation that adjusts the influence of each client's model based on its bias. It assigns higher weights to clients whose learned features are closer to an ideal, unbiased semantic point, effectively mitigating biases introduced by different data collection devices or user characteristics.
Global Distribution-aware Learning Controller (GDLC): A controller that models the distribution of training samples on the server without transmitting private information. It estimates batch distributions and calculates a weighting coefficient based on KL divergence between local and global distributions, helping to identify and address long-tail issues in label distribution.
Long-tail Issue: The phenomenon where the distribution of physiological labels (phys-labels) is not uniform across all clients. Some clients have many samples, while others have very few, making it hard for the global model to learn from all data points effectively.

Terminology

Aggregation — 2× in this paper · explained in 1 episode
Generalization — 2× in this paper · explained in 13 episodes

Summary

The gist: The FedHUG framework proposes a novel Federated Heterogeneous Unsupervised Generalization protocol to enhance model generalization during unsupervised federated learning for remote physiological measurements.

Federated Unsupervised Domain Generalization (FUDG) Protocol

The paper instantiates a new protocol called Federated Unsupervised Domain Generalization (FUDG) in this work to resolve challenges in RPM where existing methods still require labeled data and domain shifts occur across different scenarios and users The goal is to fine-tune a pre-trained model in an unsupervised manner during the FL process to enhance generalization As shown in Fig. 1, a pre-trained network is deployed on the global server and shared with clients collecting unlabeled inputs from diverse environments Through unsupervised learning, client model parameters are uploaded to the server for aggregation, enabling the global model to adapt to changing scenarios. This protocol addresses two main challenges: (1) Biases from data collection devices and user differences make domain skew in feature distribution, complicating generalization during FL, and (2) The long-tail phenomenon in global physiological labels (phys-labels) distribution exists and shows distributional heterogeneity across clients.

Minimal Bias Aggregation (MBA)

To address the biases from data collection devices and user differences, a novel Minimal Bias Aggregation (MBA) method is introduced for the FUDG task. This method assigns higher weights to clients with smaller biases by evaluating the distance between the learned semantics from different clients and the ideal semantic-invariant point. The process involves processing each original input through spatio-temporal augmentation to check for semantic consistency, where dissimilar semantics indicate a larger bias. The MBA updates the global model using a weighted aggregation formula that incorporates these bias metrics

Global Distribution Aware Learning Controller (GDLC)

The Global Distribution aware Learning Controller (GDLC) is designed to resolve the semantic distributional shift between domains and agnostic global distribution by precisely regulating the training process on the client side through implicit semantic distribution modeling on the server without any privacy concern information transmission. GDLC estimates parameters of the batch training sample distribution from unit high-dimensional feature vector v of each sample and dynamically adjusts the global feature distribution by uploading these parameters. Furthermore, when comparing differences between local and global dataset parameters, it infers the distribution of the sample in the global set to identify outliers (i.e., tail samples) in training batches. To enhance sensitivity to these tail samples, a weighting coefficient VKL is calculated based on KL divergence between local and global distributions.

Parameter Update in FedHUG

The overall algorithm, shown in Algorithm 1, involves several steps for the FedHUG framework Before starting FL, a high-quality labeled dataset is used to pretrain an initial global model on the server. After completing one round of training in the source domains, s and global distribution parameters are calculated from client models. The server then updates the global model using a formula that incorporates the bias scores (s) from each client, where clients with smaller bias contribute more to the updates Subsequently, with KL divergence calculation on the client side, the next training round begins.

Quantitative Results and Ablation

Extensive experiments demonstrate significant improvements and effectiveness in our method compared to SOTA approaches. FedHUG exhibited outstanding performance compared to both other methods, with notable improvements in generalization through unsupervised joint learning Specifically, in PURE and BUAA, there was the largest performance improvement (MAE decreased by 9.6% and 11.6%) compared to the best existing FL method (i.e., FedGaLA). Ablation tests show that compared to the model without unsupervised generalization pretraining, FedHUG reduced the average MAE by 26.96%. Using the FL baseline model without MBA and GDLC (i.e., FedAvg), applying MBA to the baseline reduced MAE by 5.16% across four datasets, with the largest improvements on BUAA and PURE (10.94% and 9.25%, respectively).

Visualization Results

To verify whether the model effectively moves towards semantic invariance points, visualization shows that without the MBA, the s distribution has a larger proportion at low-similarity values, indicating poor generalization ability on target datasets and significant impact by bias. By contrast, FedHUG shows a notable improvement, and most instances present a high s in the distribution. The visualization of feature distribution on unit sphere clearly shows that compared to FedAvg, FedHUG results in a distinct clustering of tail data. Additionally, the distribution of high HR tail data and low HR tail data exhibit opposing trends, suggesting that FedHUG is more sensitive to implicit semantic information in the tail data.

Conclusion

In summary, we introduce the FUDG protocol in RPM and propose the novel FedHUG framework to unsupervisely enhance the model’s performance on unknown domains. The proposal encourages global models to focus more on less biased domains during the aggregation process; simultaneously, it implicitly models the semantic distribution during training to alleviate the long-tail issue. Our FedHUG outperforms existing methods, showing superior generalization and applicability in real-world deployment. Future work should further explore individual-wise FUDG.

Impact of Unsupervised Methods

In general unsupervised methods, we consider the data x ni and its spatio-temporal augmented sample a ni as positive samples, while pairs of x ni with other samples that are either temporally distant or collected from different individuals are regarded as negative samples. Among the general approaches, SimCLR demonstrates outstanding performance in the field of federated domain generalization, consistent with previous findings. Furthermore, among all unsupervised methods, SiNC exhibits the best overall performance. Testing on Source Domains showed that FedHUG outperforms all other methods on all domains, highlighting that it can generalize to unknown domains while still maintaining good performance on each client.

HR Estimation in mmWave

Following the road conditions described in the PhysDrive, our method still shows better performance compared to existing FL methods It suggests that our approach is also applicable to RPM with other modalities and measurement principles. When incorporating GDLC, the model performance further improved, with an average MAE decrease of 4.29%. Notably, after applying GDLC, the p-value showed a more substantial increase, with an average improvement of 5.91%, indicating that GDLC effectively mitigates overfitting to head samples and enhances the overall model performance.

Communication Round

FedGaLA recorded the changes in MAE values of the target dataset under the FUDG framework across different communication rounds in Fig. 6(c). In this experiment, we observed that FedHUG exhibited a faster MAE decline, with better model performance.

Impacts of the Hyperparameter

We tested our model’s performance on the MMPD domain for different σ and γ. As shown in Fig. 6(d), the model performs best with (σ = 50, γ = 0.5). Performance declines when σ is too high or low, as GDLC struggles to detect anomalous training batches. Additionally, γ affects the effectiveness of GDLC once activated.

Visualization of Feature Distribution

To demonstrate how FedHUG addresses the long-tail distribution, we simplified the model implicit semantic features to three dimensions using PCA and modeled them on the surface of a unit sphere. The visualization clearly shows that, compared to FedAvg, FedHUG results in a distinct clustering of tail data. Additionally, the distribution of high HR tail data and low HR tail data exhibit opposing trends.

Testing on Source Domains

Although FUDG prioritizes generalizability in unknown domains, performance in the source domain is equally important for deployment. Thus, we cancel the target domain evaluation but test on each source domain after FUDG training. As shown in Tab. II, FedHUG outperforms all other methods on all domains, highlighting that it can generalize to unknown domains while still maintaining good performance on each client.

HR Estimation in mmWave

Following the road conditions described in the PhysDrive, we divide the data into three groups: Flat & Unobstructed, Flat & Congested, and Bumpy & Congested. One group is designated as Dtarget, while the remaining two serve as Dsource, following the FUDG protocol. The results are shown in Tab. III, compared to existing FL methods, our method still shows better performance. It suggests that our approach is also applicable to RPM with other modalities and measurement principles.

Conclusion

Improvements for AI systems

No preamble
Mitigate domain shift through Minimal Bias Aggregation (MBA) by assigning higher weights to clients with smaller biases, as described by it assigns higher weights to clients with smaller biases. This improves generalization by guiding the global model toward an optimization point with less bias compared to the previous relatively optimal point.
Resolve label distribution skew and long-tail issues using the Global Distribution Aware Learning Controller (GDLC), which "directly estimates the parameters of the batch training sample distribution from the unit high-dimensional feature vector v of each sample and dynamically adjusts the global feature distribution by uploading these parameters. This allows for a more sensitive" global model to learn from tail samples, as indicated by increasing the impact of batches with higher KL divergence in gradient updates.
Enhance client learning sensitivity to tail samples by using a weighting coefficient VKL derived from KL divergence, where A higher KL divergence indicates the presence of samples in the local region that deviate from the global mean direction, or a small number of more scattered outlier samples. This ensures that clients’ parameters with smaller bias will contribute more to the updates on the server by multiplying their original loss by this coefficient.
Improve semantic invariance learning during aggregation by transmitting statistics like s from each client to the server, where this is used in Eq. (6) to update global parameters, ensuring that the clients’ parameters with smaller bias will contribute more to the updates on the server.
Perform robust physiological signal estimation across diverse modalities (RGB video and mmWave radar) by leveraging a multi-source FUDG protocol that can enhance model generalization during the unsupervised federated learning and show superior performance across state-of-the-art techniques in estimation with either RGB video or mmWave radar.
Achieve superior performance on challenging datasets like PURE and BUAA by demonstrating that FedHUG exhibits notable improvements in generalization through unsupervised joint learning, with specific results showing MAE decreased by 9.6% and 11.6% compared to the best existing FL method (FedGaLA).

Abstract

Remote physiological measurement gained wide attention, while it requires collecting users' privacy-sensitive information, and existing contactless measurements still rely on labeled client data. This presents challenges when we want to further update real-world deployed models with numerous user data lacking labels. To resolve these challenges, we instantiate a new protocol called Federated Unsupervised Domain Generalization (FUDG) in this work. Subsequently, the Federated Heterogeneous Unsupervised Generalization (FedHUG) framework is proposed and consists of: (1) Minimal Bias Aggregation module dynamically adjusts aggregation weights based on prior-driven bias evaluation to cope with heterogeneous non-IID features from multiple domains. (2) The Global Distribution-aware Learning Controller parameterizes the label distribution and dynamically manipulates client-specific training strategies, thereby mitigating the server-client label distribution skew and long-tail issue. The proposal shows superior performance across state-of-the-art techniques in estimation with either RGB video or mmWave radar. The code will be released.