Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples
Xinliang Liu, Tong Mao, Jinchao Xu
math.NA, cs.LG, cs.NA
Submitted: 2026-08-07
Updated: 2026-08-10
License: http://creativecommons.org/licenses/by/4.0/
The gist: We develop a rigorous theory of discrete residual least-squares approximation for elliptic spectral equations L beta u=f using linearized ReLU k neural networks on the sphere, where L beta is a
Terminology
Abstract
We develop a rigorous theory of discrete residual least-squares approximation for elliptic spectral equations L beta u=f using linearized ReLU k neural networks on the sphere, where L beta is a positive elliptic spectral multiplier of order beta. Given a parameter set n= theta j* j=1 n S d, we approximate u in the linearized network space L n k(n) by the discrete residual on the collocation points eta i* i=1 m u n,m in v n in L n k(n) 1m sum i=1 m (f(eta i*)- L beta v n(eta i*)) squared. With k> d-1 over 2+ beta, for antipodally quasi-uniform network parameter sets and any quasi-uniform collocation points with m n, we prove that u-u n,m H beta(S d) f- L beta u n,m L 2(S d) n-r over d f W r,p(S d),& d over p<r at most d over 2, p>2, f H r(S d),&r> d over 2. We also establish a high-probability residual estimate, up to a logarithmic factor and an arbitrarily small smoothness loss, for i.i.d. uniformly distributed collocation points. The key analytical ingredient is a Bernstein inequality for linearized ReLU k network spaces. If h denotes the antipodal separation distance of the network parameters, then v n H r(S d) h-(r-s)v n H s(S d), 0 at most s<r<k+ 12.
Sources
- Out-of-time-order correlator as a detector of baryonic phase structure in holographic QCD with instanton
- Integral Representations of Sobolev Spaces via ReLU$^k$ Activation Function and Optimal Error Estimates for Linearized Networks
- Adaptive quadratures for nonlinear approximation of low-dimensional PDEs using smooth neural networks
- Approximation Rates for Shallow ReLU$^k$ Neural Networks on Sobolev Spaces via the Radon Transform
- Tractability of approximation by general shallow networks
- Impact of QCD sum rules coupling constants on neutron stars structure
- Coarse-to-Fine Concept Bottleneck Models
Related papers
- Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm
- A Neural-preconditioned Poisson Solver for Mixed Dirichlet and Neumann Boundary Conditions
- Advanced Linear Algebra with Applications - Part I (Numerical linear algebra for PDEs, machine learning, and data assimilation)
- Operator learning meets inverse problems: A probabilistic perspective
- Second-order consistency for learning chaotic dynamics via randomized Jacobian matching
- Windowed thinning and query complexity for the bouncy particle and Zigzag samplers