RoboFL: Federated Expert Assembly for World Action Models
cs.RO, cs.AI
Submitted: 2026-09-28
Updated: 2026-09-28
License: http://creativecommons.org/licenses/by/4.0/
The gist: Vision-language-action and world-action models are increasingly popular, yet remain bottlenecked by physical interaction data that is scarce, institutionally siloed, and task-heterogeneous.
Terminology
Abstract
Vision-language-action and world-action models are increasingly popular, yet remain bottlenecked by physical interaction data that is scarce, institutionally siloed, and task-heterogeneous. A natural federated solution is to let each client adapt a shared foundation model through parameter-efficient fine-tuning, avoiding the exchange of full-model updates. However, federating these adapters is nontrivial, as naive aggregation can entangle incompatible updates, while incorporating MoE-style routing into federated aggregation may dilute specialization and destabilize expert selection. We present RoboFL, which instantiates MoSAIC (Mixture of Slotted Adapters) for federated world-action learning. MoSAIC directly installs locally trained LoRA adapters as the expert branches of a server MoE. Server-side routers learn token assignments over these prior-informed branches while jointly refining routing and expert parameters. Foresight-to-Action Routing Distillation (FARD) aligns routing across the model's three paths, while Path-Consensus Expert Aggregation (PCEA) converts complete expert updates into a compact global adapter for personalized redistribution. Experiments on RoboTwin 2.0, RLBench, and a real-world Franka robot arm show the superiority of RoboFL with structured expert assembly, as it outperforms centralized PEFT InternVLA-A1 by 12.23% on the Franka arm, while reducing per-round client communication by up to 86.81% relative to MoE-based federated VLA baselines.
Sources
- Cosmos World Foundation Model Platform for Physical AI
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulation
- Distilling the Knowledge in a Neural Network
- $\omega$-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation
- Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models
- Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain
- Three Approaches for Personalization with Applications to Federated Learning
- FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts
- FedVLA: Federated Vision-Language-Action Learning with Dual Gating Mixture-of-Experts for Robotic Manipulation
- RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)
- Expertise need not monopolize: Action-Specialized Mixture of Experts for Vision-Language-Action Learning
- GigaWorld-Policy: An Efficient Action-Centered World--Action Model
- World Action Models are Zero-shot Policies
- ForgeVLA: Federated Vision-Language-Action Learning without Language Annotations
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies