Gauss-Newton Accuracy and Indefinite Hessians: Uniform Coexistence in Low-Cost Sets
cs.LG, stat.ML
Submitted: 2026-10-07
Updated: 2026-10-07
Comments: 28 pages, 1 table
License: http://creativecommons.org/licenses/by/4.0/
The gist: We study the accuracy of Gauss-Newton curvature in ridge-regularized nonlinear least squares.
Terminology
Abstract
We study the accuracy of Gauss-Newton curvature in ridge-regularized nonlinear least squares. Under local regularity and persistence of level-set curvature magnitude along an exact-fit section, we prove uniform coexistence of two curvature regimes. Global minimizers exist, and every global minimizer has relative Hessian error below (1+ sqrt2)/8, while the same low-cost set contains a point with an indefinite Hessian and relative error at least 15/8. One positive ridge cap works for all independent center and label perturbations in fixed neighborhoods and every positive ridge weight up to the cap. These neighborhoods do not shrink as the ridge weight tends to zero. A pointwise certificate based on the current prediction level set controls the normal, mixed, and tangent parts of the Hessian correction. We prove a sharp relative-error bound over the stated pointwise class when the prediction map and ridge vary. Analytic examples describe the roles of output alignment, curvature orientation, and persistence. A separate structural result gives full Jacobian row rank throughout low-cost sets and exact interpolation near a rank-deficient reference.
Sources
- Virtual Morse-Bott index, moduli spaces of pairs, and applications to topology of smooth four-manifolds
- Select-then-differentiate: Solving Bilevel Optimization with Manifold Lower-level Solution Sets
- Existence, Uniqueness and Regularity of the Projection onto Differentiable Manifolds
- Generalization in Nonlinear Least Squares via Learned Feature Geometry
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks