Federated Mixture-of-Experts Alignment on Mobile Edge Networks under Data Heterogeneity

arXiv:2603.21276 · cs.LG, cs.AI · Submitted 2026-03-22 · Read on arXiv

cs.LG, cs.AI

Submitted: 2026-03-22

Updated: 2026-10-06

Comments: 15 pages, 17 figures

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Aggregation — 3× in this paper · explained in 1 episode
• Alignment — 3× in this paper · explained in 1 episode
• Consistency — 3× in this paper · explained in 3 episodes
• Fine-Tuning — 3× in this paper · explained in 5 episodes
• Data Heterogeneity — 2× in this paper · explained in 3 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Benchmark — 1× in this paper · explained in 2 episodes
• Federated Learning — 1× in this paper · explained in 17 episodes
• Federated Learning (FL) — 1× in this paper · explained in 7 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Mixture-of-Experts (MoE) — 1× in this paper · explained in 7 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Semantic Alignment — 1× in this paper · explained in 3 episodes

Sources

• Redundancy Meets Synergy: Dependency-aware Expert Selection for MoE via Submodular Optimization
• Resource-Aware Parameter-Efficient Model Adaptation for Onboard High-Dimensional Data
• HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
• HFedMoE: Resource-aware Heterogeneous Federated Learning with Mixture-of-Experts
• GPT-4 Technical Report
• LLaMA: Open and Efficient Foundation Language Models
• DeepSeek-V3 Technical Report
• OLMoE: Open Mixture-of-Experts Language Models
• DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
• Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
• FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts
• Specialized federated learning using a mixture of experts
• FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregation
• FFT-MoE: Efficient Federated Fine-Tuning for Foundation Models via Large-scale Sparse MoE under Heterogeneous Edge
• Measuring Massive Multitask Language Understanding
• Adaptive Expert Models for Personalization in Federated Learning

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks