An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data
FedSub is an efficient federated learning algorithm tackling client drift and high costs in heterogeneous data settings. It uses low-dimensional subspace projections to drastically reduce communication, computation, and memory requirements. Dual variables are added to correct model drift caused by non-IID data distributions, leading to better convergence than standard methods.