Generalization in Nonlinear Least Squares via Learned Feature Geometry
This research investigates how overparameterized nonlinear models generalize by focusing on data-dependent geometry rather than worst-case complexity. It introduces an 'effective dimension' derived from the trained Jacobian features, showing that generalization gaps are controlled by this learned geometry, which can be small if the features are compressible.