JPmHC Dynamical Isometry via Orthogonal Hyper-Connections
cs.LG, cs.AI
Submitted: 2026-02-20
Updated: 2026-11-08
Terminology
Sources
- Operator-valued free multiplicative convolution: analytic subordination theory and applications to random matrix theory
- On the Measure of Intelligence
- ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
- On the S-transform over a Banach algebra
- A Unified Framework for Implicit Sinkhorn Differentiation
- Less is More: Recursive Reasoning with Tiny Networks
- Adam: A Method for Stochastic Optimization
- Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be
- Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform
- Hutch++: Optimal Stochastic Trace Estimation
- Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
- Hierarchical Reasoning Model
- DeepNet: Scaling Transformers to 1,000 Layers
- mHC: Manifold-Constrained Hyper-Connections
- Tensor Programs III: Neural Matrix Laws
- KromHC: Manifold-Constrained Hyper-Connections with Kronecker-Product Residual Matrices
- Hyper-Connections
Related papers
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis
- (Mis)generalization of Helpful-Only Fine-tuning