THGFM: Dual-Branch Temporal Heterogeneous Graph Fusion Model
cs.LG, cs.CL, stat.ML
Submitted: 2026-07-29
Updated: 2026-09-28
License: http://creativecommons.org/licenses/by/4.0/
The gist: Temporal heterogeneous graphs offer a natural abstraction for dynamic relational systems in which diverse node and relation types co-exist and evolve over time.
Terminology
Abstract
Temporal heterogeneous graphs offer a natural abstraction for dynamic relational systems in which diverse node and relation types co-exist and evolve over time. Learning on such graphs requires jointly modeling cross-type structural heterogeneity and the temporal dynamics of interactions, yet existing methods still struggle to reconcile parameter-efficient cross-type transfer with relation-aware specialization, and typically inject time only as additive features outside the attention kernel. We propose THGFM, a web-scale temporal heterogeneous graph fusion model that addresses both limitations within a unified dual-path architecture. THGFM couples a Shared-Space Temporal Attention branch for parameter-efficient cross-type transfer with a Relational Type-Partitioned Temporal Attention branch for relation-aware specialization, and integrates them through Dual-Path Relational--Shared Fusion, instantiated with Type-Conditioned Non-Competitive Gated Sum Fusion: a adaptive mechanism that assigns independent, type-conditioned feature-wise gates to the shared and specialized branches, allowing both to be amplified or suppressed without zero-sum competition. To directly incorporate relative time into the attention score, THGFM further introduces Rotary Temporal Attention, which rotates queries and keys by half-phases of relative time before matching. THGFM consistently outperforms baseline graph transformer models on academic graphs benchmarks, delivering a +3.25% six-task mean gain, with peak relative gains of +12.37% on OAG-CS PV, +4.87% on PF- L 2, and +1.18% on PF- L 1, and +4.24%, +3.73%, and +4.61% on OGBN-MAG, HTAG-ArXiv, and HTAG-DBLP, respectively.
Sources
- Billion-Scale Graph Foundation Models
- Heterogeneous Temporal Graph Neural Network
- A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges
- Open Graph Benchmark: Datasets for Machine Learning on Graphs
- Multi-Scale Heterogeneous Text-Attributed Graph Datasets From Diverse Domains
- Temporal Graph Networks for Deep Learning on Dynamic Graphs
- RoFormer: Enhanced Transformer with Rotary Position Embedding
- Heta: Distributed Training of Heterogeneous Graph Neural Networks
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks