Phase transitions reveal hierarchical structure in deep neural networks
The paper investigates how phase transitions in deep learning are governed by saddle points in loss landscapes, revealing a hierarchical structure of accuracy basins analogous to phases in statistical physics. Using a new algorithm, Pathfinder, researchers confirmed mode connectivity—smooth paths between optimal models—and showed that L2 regularization acts as a thermodynamic parameter tuning the system between these distinct learning phases.