Toward a Physics of Deep Learning and Brains
The paper applies physics from non-equilibrium statistical mechanics to understand deep learning and brain activity by modeling neuronal avalanches as cascades in neural networks. It finds that deep networks learn best when operating near a 'quasi-critical' state between absorbing and active phases. Maximal susceptibility is a better predictor of learning than being exactly at the critical point, offering a blueprint for engineering better network performance.