Can Entry-Wise Clipping Give Spectral Control of Stochastic Gradients?
cs.LG
Submitted: 2026-05-26
Updated: 2026-10-22
Terminology
Sources
- The Llama 3 Herd of Models
- Kimi K2: Open Agentic Intelligence
- Old Optimizer, New Norm: An Anthology
- Enhancing LLM Training via Spectral Clipping
- Training Transformers with Enforced Lipschitz Constants
- AdaMuon: Adaptive Muon Optimizer
- ROOT: Robust Orthogonalized Optimizer for Neural Network Training
- Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
- From Gradient Clipping to Normalization for Heavy Tailed SGD
- On the Convergence Analysis of Muon
- Adam: A Method for Stochastic Optimization
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks