SO(3)-RoPE for Spherical Transformers
cs.LG, cs.AI
Submitted: 2026-10-05
Updated: 2026-10-05
Comments: Accepted at the Neurips Workshop 2026: Representations for the Physical Sciences
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Joint 2D-3D-Semantic Data for Indoor Scene Understanding
- Attention on the Sphere
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
- Spherical CNNs
- Gauge Equivariant Convolutional Networks and the Icosahedral CNN
- Steerable CNNs
- DeepSphere: a graph-based spherical CNN
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Platonic Transformers: A Solid Choice For Equivariance
- Adam: A Method for Stochastic Optimization
- Rethinking RoPE: A Mathematical Blueprint for N-dimensional Positional Embedding
- Scalable and Equivariant Spherical CNNs by Discrete-Continuous (DISCO) Convolutions
- (Sparse) Attention to the Details: Preserving Spectral Fidelity in ML-based Weather Forecasting Models
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks