Beyond the Log-Linear Frontier: Pre-Asymptotic Analysis of Finite-Budget Neural Scaling Laws
cs.LG
Submitted: 2026-11-10
Updated: 2026-11-10
Terminology
Sources
- A Continuous-Time View of Early Stopping for Least Squares
- Chinchilla Scaling: A replication attempt
- DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
- Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
- Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks
- A Dynamical Model of Neural Scaling Laws
- Mat'ern Gaussian processes on Riemannian manifolds
- Language Models are Few-Shot Learners
- Broken Neural Scaling Laws
- Explaining Data Mixing Scaling Laws
- Sobolev Norm Learning Rates for Regularized Least-Squares Algorithm
- The Pile: An 800GB Dataset of Diverse Text for Language Modeling
- BiMix: A Bivariate Data Mixing Law for Language Model Pretraining
- Scaling Laws for Autoregressive Generative Modeling
- Scaling Laws for Transfer
- Scaling Laws and Interpretability of Learning from Repeated Data
- Deep Learning Scaling is Predictable, Empirically
- Training Compute-Optimal Large Language Models
- Learning Curve Theory
- Implicit Regularization of Random Feature Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis