Local large deviations for linear-region growth in random piecewise-linear networks

arXiv:2607.07014 · math.PR, stat.ML · Submitted 2026-07-08 · Read on arXiv

Recep Özkan, Christian Hirsch

math.PR, stat.ML

Submitted: 2026-07-08

Comments: 23 pages, 2 figures

License: http://creativecommons.org/licenses/by/4.0/

The gist: We study a random compositional model for the growth of affine regions in deep piecewise-linear networks.

Terminology

Abstract

We study a random compositional model for the growth of affine regions in deep piecewise-linear networks. The model is generated by i.i.d. perturbations of the symmetric height-one tent map, and the main observable is the number N n of affine pieces after n layers. We prove the existence of a submultiplicative pressure for N n, yielding exponential upper bounds for both tails of n-1 N n. The same argument applies to abstract submultiplicative complexity observables and gives higher-dimensional extensions for convex-polytopal affine-cover counts and worst-line affine-piece counts. Since the true branch count has no matching supermultiplicative inequality, lower bounds require a separate certified construction. We introduce a finite-state defect process that records branches whose future splitting can be guaranteed, and use bridge words to obtain constructive upper-tail lower bounds. In a uniformly favorable small-noise regime, this process is governed by a companion matrix whose Perron root tends to 2, implying eventual exclusion of lower tails below 2-xi.

Sources

Related papers