A Spectral Renormalization-Group View of Learning
cs.LG, cond-mat.dis-nn, cs.AI
Submitted: 2026-10-30
Updated: 2026-11-05
Code: https://github.com/CalculatedContent/WeightWatcher
Terminology
Sources
- On the Opportunities and Risks of Foundation Models
- Qwen2.5 Technical Report
- gpt-oss-120b & gpt-oss-20b Model Card
- GPT-4 Technical Report
- Traditional and Heavy-Tailed Self Regularization in Neural Network Models
- SETOL: A Semi-Empirical Theory of (Deep) Learning
- Detecting overfitting in Neural Networks during long-horizon grokking using Random Matrix Theory
- Late-Stage Generalization Collapse in Grokking: Detecting anti-grokking with Weightwatcher
- On Orthogonal and Symplectic Matrix Ensembles
- Rational Decisions, Random Matrices and Spin Glasses
- Law of addition in random matrix theory
- Decoupled Weight Decay Regularization
- Muon is Scalable for LLM Training
- Modular Duality in Deep Learning
- XGBoost: A Scalable Tree Boosting System
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis