HodgePhaseNet: Discrete Hodge Decomposition of Semantic Flows in Transformer Attention
cs.LG, cs.CL
Submitted: 2026-10-10
Updated: 2026-10-10
License: http://creativecommons.org/licenses/by/4.0/
The gist: WavePhaseNet introduced frequency-resolved semantic representations, QuantumPhaseNet introduced gauge-covariant phase connections, and ModularPhaseNet made a finite cyclic phase channel and its cycle
Terminology
Abstract
WavePhaseNet introduced frequency-resolved semantic representations, QuantumPhaseNet introduced gauge-covariant phase connections, and ModularPhaseNet made a finite cyclic phase channel and its cycle holonomy computable in an ordinary Transformer. These constructions are complementary, yet they do not by themselves separate an observed collection of pairwise relations into a globally consistent direction, circulation around local compositions, and circulation that persists around a larger unfilled cycle. HodgePhaseNet addresses this separation by representing semantic transitions as an antisymmetric real-valued 1-cochain on a finite oriented simplicial complex. With vertex-edge incidence matrix B 1 and edge-face incidence matrix B 2, every such field has a unique orthogonal decomposition Ω= B 1 TΦ+ B 2Ψ+ H, where B 1H=0 and B 2 TH=0. The three component fields are unique even when their potentials are not. We give explicit projectors, a sparse implementation, integrability and gauge results, a verified numerical example, and a redesigned attention interface. The attention bias is defined on allowed ordered query-key pairs, remains consistent under arbitrary choices of edge orientation, and preserves a decoder's causal information boundary by constructing its complex from the visible prefix. Gauge invariance of the attention output is conditional on using invariant or appropriately compensated features. A simple difference of vertex embeddings has no coexact or harmonic component; a genuine relation encoder is therefore indispensable. Local circulation is not automatically contradiction, and harmonic flow is not automatically missing knowledge. These interpretations and any reduction in factual error require controlled empirical evaluation. This paper presents a mathematical and architectural proposal, not external natural-language performance results.
Sources
- WavePhaseNet: A DFT-Based Method for Constructing Semantic Conceptual Hierarchy Structures (SCHS)
- ModularPhaseNet: Finite-Cyclic Phase Geometry for Computable Semantic Hierarchy, Direction, and Context Consistency in Standard Transformers
- Attention Is All You Need
- FNet: Mixing Tokens with Fourier Transforms
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
- Statistical ranking and combinatorial Hodge theory
- Random Walks on Simplicial Complexes and the normalized Hodge 1-Laplacian
- Topological Signal Processing over Simplicial Complexes
- Simplicial Neural Networks
- Hodgelets: Localized Spectral Representations of Flows on Simplicial Complexes
- Hodge-Compositional Edge Gaussian Processes
- Poincar'e Embeddings for Learning Hierarchical Representations
- Order-Embeddings of Images and Language
- HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment
- FEVER: a large-scale dataset for Fact Extraction and VERification
- Adversarial NLI: A New Benchmark for Natural Language Understanding
- TruthfulQA: Measuring How Models Mimic Human Falsehoods
- FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
- On Calibration of Modern Neural Networks
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks