Local large deviations for linear-region growth in random piecewise-linear networks
Recep Özkan, Christian Hirsch
math.PR, stat.ML
Submitted: 2026-07-08
Comments: 23 pages, 2 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: We study a random compositional model for the growth of affine regions in deep piecewise-linear networks.
Terminology
Abstract
We study a random compositional model for the growth of affine regions in deep piecewise-linear networks. The model is generated by i.i.d. perturbations of the symmetric height-one tent map, and the main observable is the number N n of affine pieces after n layers. We prove the existence of a submultiplicative pressure for N n, yielding exponential upper bounds for both tails of n-1 N n. The same argument applies to abstract submultiplicative complexity observables and gives higher-dimensional extensions for convex-polytopal affine-cover counts and worst-line affine-piece counts. Since the true branch count has no matching supermultiplicative inequality, lower bounds require a separate certified construction. We introduce a finite-state defect process that records branches whose future splitting can be guaranteed, and use bridge words to obtain constructive upper-tail lower bounds. In a uniformly favorable small-noise regime, this process is governed by a companion matrix whose Perron root tends to 2, implying eventual exclusion of lower tails below 2-xi.
Sources
- Functional Large Deviations for Wide Deep Neural Networks with Gaussian Initialization and Lipschitz Activations
- Mathematical theory of deep learning
- Representation Benefits of Deep Feedforward Networks
Related papers
- Sharp Deviations Bounds for Dirichlet Weighted Sums with Application to analysis of Bayesian algorithms
- Subspace Uncertainty and Sharp Sampling Thresholds on the Boolean Cube
- Local Anticoncentration for Gaussian Boson Sampling via Conditional Wishart Geometry
- Beyond the Semicircle: Free Diffusion Models with Prescribed Equilibria
- Explicit Steady-State Approximations for Parallel Server Systems with Heterogeneous Servers
- A New Bound on the Cumulant Generating Function of Dirichlet Processes