Sonar-MoE: In-Backward Expert Updates for Memory-Efficient Mixture-of-Experts Training

arXiv:2611.03577 · cs.LG, cs.AI, cs.CL, cs.PF · Submitted 2026-11-02 · Read on arXiv

cs.LG, cs.AI, cs.CL, cs.PF

Submitted: 2026-11-02

Updated: 2026-11-05

Code: https://github.com/ibm-granite/granite-3.0-language-models

Terminology

Sources

Related papers