Rethinking Model Aggregation for Federated Learning with Domain Skew: A Knowledge-Preserving Perspective

arXiv:2611.01893 · cs.DC, cs.AI · Submitted 2026-11-01 · Read on arXiv

cs.DC, cs.AI

Submitted: 2026-11-01

Updated: 2026-11-01

License: http://creativecommons.org/licenses/by/4.0/

The gist: Domain skew challenges federated learning by requiring a shared model to accommodate distinct visual distributions.

Terminology

Abstract

Domain skew challenges federated learning by requiring a shared model to accommodate distinct visual distributions. Domain-specific experts can capture these distributions separately, creating an opportunity to integrate their representations in a unified predictor. We propose Federated Knowledge-Preserving Learning (FKPL), a two-stage framework that combines domain-oriented expert training with prototype-based classifier calibration. First, a one-shot subspace-based clustering procedure groups clients for intra-cluster training. The resulting expert encoders are concatenated and frozen, preserving each learned feature mapping during global integration. Second, clients interpolate their concatenated features with class-matched domain prototypes and upload the resulting virtual features. The server trains a global linear classifier on these features, concentrating classifier optimization at the server after a single feature upload. FKPL achieves average accuracies of 73.01%, 67.00%, and 64.51% on Office-Caltech-10, PACS, and a four-domain subset of DomainNet, respectively. On DomainNet, its cross-domain standard deviation is 7.50 percentage points, the lowest among the evaluated methods. With the same concatenated extractor, prototype-based calibration improves average accuracy over federated classifier training by 9.99 and 14.94 percentage points on Office-Caltech-10 and PACS. These results identify classifier calibration as an effective way to combine preserved domain representations into a global predictor.

Related papers