The Birkhoff Geometry of Manifold-Constrained Hyper-Connections: Two Channels, Vertex Viscosity, and Sinkhorn as a Retraction
cs.LG, math.OC, stat.ML
Submitted: 2026-10-05
Updated: 2026-10-05
Sources
- Ranking via Sinkhorn Propagation
- Analyzing Stream Collapse in Hyper-Connections: From Diagnosis to Mitigation
- Interior interpretability with attention rollout: contraction and propagation profiles in Transformers
- go-$m$HC: Direct Parameterization of Manifold-Constrained Hyper-Connections via Generalized Orthostochastic Matrices
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- TEMPER: Tensorized Efficient Manifold-constrained Parameterization for Expressive Residual Routing
- oHC: Orthogonal Hyper-Connections on SO(4) via Quaternions
- A review of matrix scaling and Sinkhorn's normal form for matrices and positive maps
- Softmax Policy Gradient Methods Can Take Exponential Time to Converge
- Avoiding Premature Collapse: Adaptive Annealing for Entropy-Regularized Structural Inference
- Spectral-Sphere-Constrained Hyper-Connections
- TBP-mHC: full expressivity for manifold-constrained hyper connections through transportation polytopes
- Residual Matrix Transformers: Scaling the Size of the Residual Stream
- Ablate and Rescue: A Causal Analysis of Residual Stream Hyper-Connections
- JPmHC Dynamical Isometry via Orthogonal Hyper-Connections
- Accelerating Birkhoff Projection for Manifold-Constrained Hyper-Connections
- MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections
- mHC: Manifold-Constrained Hyper-Connections
- mHC-lite: You Don't Need 20 Sinkhorn-Knopp Iterations
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks