SHEFL: Sparse Heterogeneous Ensemble Federated Learning with Group-Balanced Aggregation

arXiv:2508.08552 · cs.LG, cs.DC · Submitted 2025-08-12 · Read on arXiv

cs.LG, cs.DC

Submitted: 2025-08-12

Updated: 2026-09-25

Comments: 5 pages, 3 figures

License: http://creativecommons.org/licenses/by-sa/4.0/

The gist: System heterogeneity in federated learning (FL) requires training workloads and communication demands to be adapted to client resource capacities.

Terminology

Abstract

System heterogeneity in federated learning (FL) requires training workloads and communication demands to be adapted to client resource capacities. Federated ensemble learning distributes multiple predictors across clients, but assigning one ensemble member to each client reduces the number of contributors to each member as the ensemble grows. We propose SHEFL, a global ensemble-based FL framework that uses the additional computation of high-power clients (HPCs) to train all ensemble members while low-power clients (LPCs) train one assigned member. SHEFL addresses increased HPC uplink demand and biased aggregation toward HPC updates through resource-aware sparse update allocation and group-balanced aggregation. Across four vision datasets, SHEFL improves over Fed-Ensemble (FedEns) in all data heterogeneity settings. Under a matched total uplink budget, SHEFL reaches the target accuracy in fewer rounds than FedEns and improves final accuracy by up to 14.65 points. Ablations further show that group-balanced aggregation improves accuracy and convergence under different resource allocations.

Related papers