Optimal scaling laws in learning hierarchical multi-index models
stat.ML, cs.LG
Submitted: 2026-02-05
Updated: 2026-11-16
Terminology
Sources
- A Theory for Emergence of Complex Skills in Language Models
- Scaling and renormalization in high-dimensional regression
- Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models
- Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws
- A Dynamical Model of Neural Scaling Laws
- Fast Escape, Slow Convergence: Learning Dynamics of Phase Retrieval under Power-Law Data
- Computational-Statistical Gaps in Gaussian Single-Index Models
- Optimal Spectral Transitions in High-Dimensional Multi-Index Models
- Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regime
- The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networks
- Scaling Laws for Neural Language Models
- Spectral Estimators for Multi-Index Models: Precise Asymptotics and Optimal Weak Recovery
- Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networks
- Scaling Laws for Gradient Descent and Sign Descent for Linear Bigram Models under Zipf's Law
- A Solvable Model of Neural Scaling Laws
- High-Dimensional Analysis of Gradient Flow for Extensive-Width Quadratic Neural Networks
- Emergence and scaling laws in SGD learning of shallow neural networks
- Emergent Abilities of Large Language Models
- Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra
- Kernel ridge regression under power-law data: spectrum and generalization
Related papers
- Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks
- Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors
- Quantifying Error Propagation and Model Collapse in Diffusion Models
- Behavior of prediction performance metrics with rare events
- Statistical Inference and Learning for Shapley Additive Explanations (SHAP)
- Control Variate Design for Score-Function Gradient Estimators in Importance-Weighted Variational Inference