Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation
cs.LG, stat.ML
Submitted: 2026-05-12
Updated: 2026-11-08
Comments: NeurIPS 2026 (33 pages, 20 figures, project page: https://spherelab.ai/pion/, v2: added msign-based Pion and additional experiments)
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- gpt-oss-120b & gpt-oss-20b Model Card
- The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm
- Layer Normalization
- Riemannian Adaptive Optimization Methods
- Modular Duality in Deep Learning
- Old Optimizer, New Norm: An Anthology
- LoRA Learns Less and Forgets Less
- Muon Optimizes Under Spectral Norm Constraints
- Evaluating Large Language Models Trained on Code
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Training Verifiers to Solve Math Word Problems
- The Llama 3 Herd of Models
- Mano: Restriking Manifold Optimization for LLM Training
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning
- Training Compute-Optimal Large Language Models
- Adam: A Method for Stochastic Optimization
- Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform
- NorMuon: Making Muon more efficient and scalable
- DeepSeek-V3 Technical Report
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis