From Geometry to Generalization: Why Row Normalization Can Beat Adam and Muon
This research compares different optimization methods for high-dimensional classification, finding that row normalization methods (RowGD/RowGDM) outperform Adam and Muon. The core finding is that row normalization preserves class-mean directions in the data geometry, leading to better generalization accuracy. This advantage holds even under complex conditions like anisotropic noise.