Parameter Bounds and Inverse Approximation for Linearized ReLU k Networks on the Sphere
math.NA, cs.NA
Submitted: 2026-10-31
Updated: 2026-10-31
License: http://creativecommons.org/licenses/by/4.0/
The gist: We study direct and inverse approximation for linearized shallow ReLU k networks on S d with quasi-uniform hidden parameters.
Abstract
We study direct and inverse approximation for linearized shallow ReLU k networks on S d with quasi-uniform hidden parameters. For 0<α< d+2k+1 over 2d, under the coefficient growth condition (n sum from j=1 to na j,n 2) 1 over 2 at most n αM, we prove that approximation at the order n-d+2k+1 over 2d+α is equivalent to membership in the parity-compatible Besov space B 2, infinity,* d+2k+1 over 2-αd(S d). Mere convergence under the same coefficient budget does not imply this regularity. For unconstrained approximation along a nested parameter sequence, a single ridge function gives a counterexample when α< 1 over 2, whereas for α> 1 over 2 the endpoint Besov inverse conclusion follows from a Bernstein inequality under antipodal quasi-uniformity. The inverse conclusion is also recovered when the prescribed rate holds for two quasi-uniform parameter sequences whose union is antipodally separated.
Sources
- Condition Numbers and Eigenvalue Spectra of Shallow Networks on Spheres
- Integral Representations of Sobolev Spaces via ReLU$^k$ Activation Function and Optimal Error Estimates for Linearized Networks
- Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples
- Configuration-Dependent Lower Bounds for Approximation by Shallow ReLU k Networks on the Sphere
- Tractability of approximation by general shallow networks
Related papers
- Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm
- A Neural-preconditioned Poisson Solver for Mixed Dirichlet and Neumann Boundary Conditions
- Advanced Linear Algebra with Applications - Part I (Numerical linear algebra for PDEs, machine learning, and data assimilation)
- Operator learning meets inverse problems: A probabilistic perspective
- Second-order consistency for learning chaotic dynamics via randomized Jacobian matching
- Windowed thinning and query complexity for the bouncy particle and Zigzag samplers