MoE Pretraining:Parallelism Layout Changes the Model, Not Quality

arXiv:2611.15271 · cs.DC · Submitted 2026-11-11 · Read on arXiv

cs.DC

Submitted: 2026-11-11

Updated: 2026-11-11

Code: https://github.com/ethxin0011/moe_parallelism

Terminology

Sources

Related papers