Tune the Baseline First: Most of Muon's Scaled Weight Decay Gain Is a Tuning Gap, but Its Small Real Advantage Grows With Model Size

arXiv:2611.02529 · cs.CL, cs.LG · Submitted 2026-11-02 · Read on arXiv

cs.CL, cs.LG

Submitted: 2026-11-02

Updated: 2026-11-02

Code: https://github.com/ethxin0011/weight_decay_ladder

Terminology

Sources

Related papers