Deep Learning Theory through the Lens of Statistical Physics: A Topical Review
cs.LG, cond-mat.stat-mech
Submitted: 2026-11-06
Updated: 2026-11-06
Terminology
Sources
- Empirical Analysis of the Hessian of Over-Parametrized Neural Networks
- Score-Based Generative Modeling through Stochastic Differential Equations
- Scaling Laws for Neural Language Models
- Training Compute-Optimal Large Language Models
- Depth Creates No Bad Local Minima
- The Full Spectrum of Deepnet Hessians at Scale: Dynamics with SGD Training and Sample Size
- Gradient Descent Happens in a Tiny Subspace
- Spectral alignment of stochastic gradient descent for high-dimensional classification tasks
- Topology and Geometry of Half-Rectified Network Optimization
- Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data
- A Minimalist Example of Edge-of-Stability and Progressive Sharpening
- Does SGD Seek Flatness or Sharpness? An Exactly Solvable Model
- Symmetry Induces Structure and Constraint of Learning
- The loss landscape of overparameterized neural networks
- Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning
- Towards Deepening Graph Neural Networks: A GNTK-based Optimization Perspective
- Deep Information Propagation
- Mean field theory for deep dropout networks: digging up gradient backpropagation deeply
- Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs
- Tensor Programs II: Neural Tangent Kernel for Any Architecture
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis