Cascading Through the Hierarchy: Regularizer-Induced Feature Detection as Phase Transitions in Deep Linear Neural Networks
Björn Ladewig, Ibrahim Talha Ersoy, Karoline Wiesner
cond-mat.stat-mech, cond-mat.dis-nn, cs.LG
Submitted: 2026-08-06
Updated: 2026-08-10
Comments: 39 pages, 9 figures
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity
- Parameter Symmetry Potentially Unifies Deep Learning Theory
- Phase transitions reveal hierarchical structure in deep neural networks
- Applications of Statistical Field Theory in Deep Learning
- There Will Be a Scientific Theory of Deep Learning
- Regularization Implies balancedness in the deep linear network
- Entropic Regularization in the Deep Linear Network
- An entropy formula for the Deep Linear Network
- Understanding Learning Invariance in Deep Linear Networks
- Cautious Weight Decay
- Low-rank bias, weight decay, and model merging in neural networks
Related papers
- Extreme First-Passage Time of Many Interacting Particles
- Decoherence-controlled collective criticality in a two-dimensional quantum Stag Hunt
- Measurement-induced phase transitions in disordered fermions
- Hierarchical Reduction of Invariant Subspaces: From Fragmentation to Many-Body Scars
- Quantum Thermalization beyond Non-Integrability and Quantum Scars in a Multispecies Bose-Josephson Junction
- Markov length can diverge in systems whose universal physics is spatially Markovian