Fast Gauss-Newton for Multiclass Cross-Entropy
cs.LG
Submitted: 2026-05-07
Updated: 2026-11-09
Terminology
Sources
- On the Promise of the Stochastic Generalized Gauss-Newton Method for Training DNNs
- Adam: A Method for Stochastic Optimization
- Training Neural Networks with Stochastic Hessian-Free Optimization
- Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
- The Full Spectrum of Deepnet Hessians at Scale: Dynamics with SGD Training and Sample Size
- Efficient Subsampled Gauss-Newton and Natural Gradient Methods for Training Neural Networks
- Empirical Analysis of the Hessian of Over-Parametrized Neural Networks
- On the Power-Law Hessian Spectrums in Deep Learning
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis