Gauge-Equivariant Attention for Rotation-Stable 360 Scene Understanding

arXiv:2609.31755 · cs.CV, cs.GR · Submitted 2026-09-23 · Read on arXiv

cs.CV, cs.GR

Submitted: 2026-09-23

Updated: 2026-09-23

Comments: 23 pages. Accepted to ACM Transactions on Graphics (SIGGRAPH Asia 2026)

DOI: 10.1145/3842525

Code: https://github.com/Jaywalk18/equissl-release

License: http://creativecommons.org/licenses/by/4.0/

The gist: Panoramic 360 scene understanding increasingly relies on icosphere transformers, but a state-of-the-art spherical model loses more than half of its segmentation accuracy when the camera rotates by

Terminology

Abstract

Panoramic 360 scene understanding increasingly relies on icosphere transformers, but a state-of-the-art spherical model loses more than half of its segmentation accuracy when the camera rotates by 90, and controlled ablations identify gauge dependence in its relative-position bias as a major contributor. We propose gauge-equivariant relative position encoding (GE-RPE): a parameter-free Reynolds average of the bias over a finite cyclic subgroup C n! ! SO(2) of gauge rotations. Plugged into a SphereUFormer backbone the change is invisible at deployment---zero added parameters and 1.5 -- 4.4% forward latency---and the matched three-seed GE-RPE model records a 1.3% drop; the published SphereUFormer checkpoint records 53% under the same stress protocol but a different training recipe. Once the gauge defect is removed and a teacher-token permutation π R aligns the SSL views to the rotated student frame, iBOT + MAE pretraining stops being a liability and becomes a clean low-label lever: the full framework EquiSSL (GE-RPE + π R + iBOT + MAE) tightens the drop to 0.8% at 68.30% val mIoU and lifts 1% -label fine-tuning by +2.39 mIoU on the N = 373 test split (and by +4.10 on the smaller N = 40 val split); the same fix carries over to monocular depth and to zero-shot Structured3D segmentation. The construction is provably C n-invariant and O(n-2) -close to the continuous SO(2) average, making the resulting model a usable 360 visual-computing primitive across panoramic relighting, immersive video, and cross-dataset transfer. Code is available at https://github.com/Jaywalk18/equissl-release.

Sources

Related papers