Finite-width shift of the edge of chaos in deep neural networks
cond-mat.dis-nn, cs.LG, stat.ML
Submitted: 2026-11-04
Updated: 2026-11-04
Comments: 9 pages, 5 figures, 1 table; Supplemental Material (28 pages, 9 figures, 12 tables) included. Code: https://github.com/Aureliuxs/Finite-Width-Edge-of-Chaos
Code: https://github.com/Aureliuxs/Finite-Width-Edge-of-Chaos
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
- Exponential expressivity in deep neural networks through transient chaos
- Deep Information Propagation
- Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice
- On the Impact of the Activation Function on Deep Neural Networks Training
- Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping
- Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers
- Random Walk Initialization for Training Very Deep Feedforward Networks
- How to Start Training: The Effect of Initialization and Architecture
- Fractional moment-preserving initialization schemes for training deep neural networks
- Optimal Initialization in Depth: Lyapunov Initialization and Limit Theorems for Deep Leaky ReLU Networks
- Critical dynamics governs deep learning
- Toward a Physics of Deep Learning and Brains
- Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity
- Order and Chaos: NTK views on DNN Normalization, Checkerboard and Boundary Artifacts
- Dropout Universality: Scaling Laws and Optimal Scheduling at the Edge-of-Chaos
- Competing nonlinearities, criticality, and order-to-chaos transition in deep networks
- Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networks
- Correlation flow governs learning at criticality
- Random Fully Connected Neural Networks as Perturbatively Solvable Hierarchies
Related papers
- Topological metal-insulator transitions in one-dimensional non-Hermitian quasicrystals: beyond PT-symmetry
- Microscopic and collective signatures of feature learning in neural networks
- Synchronous and asynchronous Monte Carlo dynamics of mean-field spin glasses with rotationally invariant couplings: what they share and where they differ
- Reentrant Localization Length in a Quasiperiodic Thue-Morse Chain
- Beyond Ground States: Learning Many-Body Hamiltonians for p-Bit Invertible Arithmetic
- Linear equivalence of nonlinear recurrent neural networks