Riemannian Graph Tokenizer for Structural Knowledge Transfer

arXiv:2610.15427 · cs.LG · Submitted 2026-10-11 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Riemannian Graph Tokenizer for Structural Knowledge Transfer".

Jane: The gist The proposed Riemannian Graph Tokenizer (RGT) bridges structural knowledge and quantized representations using Riemannian geometry to support cross-domain structural knowledge transfer.

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, wrapping up this paper on "Riemannian Graph Tokenizer for Structural Knowledge Transfer," it’s about using Riemannian geometry to create a tokenizer that connects graph structure directly to quantized representations across different domains.

Jane: The authors, Zhou, Liu, Wang, Sun and Shi, proposed this RGT system which uses geometric vocabulary and a Riemannian quantizer with an RSTE to handle the tokenization process.

Lu: The main contribution is formalizing the geometric vocabulary for structures like trees and cycles and proving the two theorems about encoding consistency and quantization optimality that underpin the method.

Meng: For us in production, it means we have a new way to pretrain models on graph data from one domain and adapt them with minimal updates to a new domain, relying on those structural representations.

Lalam: Essentially, they show that these structurally similar geometric vocabularies can be consistently mapped to the same code in the corresponding codebook, which enables cross-domain structural alignment.

Tom: So, this moves graph foundation models beyond just learning patterns in one specific dataset and towards models that are inherently more adaptable across different graph structures.

Jane: It’s a shift from relying purely on raw data features to leveraging the underlying geometry of the graphs themselves for better generalization and transferability.

Conclusion: Segment: Conclusion**

Tom: So we’ve been deep in the mechanics of this Riemannian Graph Tokenizer paper, and now we’re at the conclusion where we gotta think about what this whole thing actually means for graph learning.

Jane: Right, so basically, they took these abstract mathematical ideas from Riemannian geometry—things like hyperbolic space and spherical space—and turned them into a way to "tokenize" graphs. It’s not just about looking at the nodes and edges anymore; it's about understanding the underlying shape of the graph itself.

Lu: Yeah, that geometric vocabulary they built, defining things like trees in hyperbolic space and sequences in Euclidean space, that’s really what lets you transfer knowledge between totally different types of data. It bridges that structural gap.

Meng: I’m thinking about deployment here; if this works well across domains, it means we don't have to train a whole new model from scratch every time we want to predict something new on a graph type we haven't seen before. That sounds like a massive win for scaling things up.

Lalam: From my side, I see the proof in their theorems—Theorem one and Theorem two—which show that when you do this quantization based on geodesic distance, the representations stay consistent across those different spaces they’re mapping to. It confirms that structurally similar vocabularies can share a code.

Tom: So it’s not just about getting a better accuracy number; it’s about getting a more unified understanding of what makes different graph structures similar at a fundamental level.

Jane: Exactly. It moves us toward models that don't just memorize patterns, but actually grasp the underlying spatial relationships, which is pretty cool for generalization.

Lu: The paper shows strong performance on link prediction and few-shot learning across web-scale data, and they even showed that using this geometric vocabulary—trees in hyperbolic space for instance—gives them the best results among all their configurations.

Meng: If we can reliably transfer structural knowledge like that without needing massive retraining for every new task, it drastically cuts down on the compute cost of building those foundation models.

Lalam: It’s a solid contribution to how we represent complex systems in AI, showing how geometry can be a powerful tool for representation learning in graph AI.

Tom: It really boils down to this: by using Riemannian geometry, we are giving the AI a better language for understanding structure itself. But what does that mean when you think about the next step?

Qimin Zhou, Haibo Liu, Yujie Wang, Li Sun, Chuan Shi

Beijing University of Posts and Telecommunications

cs.LG

Submitted: 2026-10-11

Updated: 2026-10-11

Comments: Accepted at WWW26

Journal ref: Zhou, Qimin, et al. "Riemannian Graph Tokenizer for Structural Knowledge Transfer." Proceedings of the ACM Web Conference 2026. 2026

DOI: 10.1145/3774904.3792120

Code: https://github.com/Miyla77/RGT

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

The gist: The gist The proposed Riemannian Graph Tokenizer (RGT) bridges structural knowledge and quantized representations using Riemannian geometry to support cross-domain structural knowledge transfer.

Key concepts

Geometric Vocabulary: This is the set of fundamental structural patterns used to model graphs. The paper defines specific shapes—like trees for hyperbolic space, cycles for spherical space, and sequences for Euclidean space—to capture the intrinsic geometry of different graph types. This vocabulary allows the model to understand underlying structural semantics across various domains.
Riemannian Graph Tokenizer (RGT): RGT is a system that converts complex graph structures into discrete tokens using constant curvature spaces. It uses a Riemannian Encoder and a quantizer based on geodesic distance to select codes from a codebook. This process preserves the intrinsic geometry of non-Euclidean manifolds during quantization, which is crucial for maintaining structural integrity.
Riemannian Straight-Through Estimator (RSTE): The RSTE is a technique used during training to ensure that gradients flow correctly within the tangent space of the underlying manifold. By formulating it as described in Equation 8, it guarantees that the optimization process respects the specific geometry of the space, preventing structural information from being lost during learning.
Geometry-Aligned Decoder: This decoder reconstructs original structural features by performing operations like Riemannian parallel transport. It aligns tokens from different geometric spaces (e.g., hyperbolic vs. Euclidean) into a unified tangent space at a specific point, ensuring that the final decoded representations are consistent regardless of the original manifold's curvature.

Summary

The gist The proposed Riemannian Graph Tokenizer (RGT) bridges structural knowledge and quantized representations using Riemannian geometry to support cross-domain structural knowledge transfer.

How it works

Geometric Vocabulary: The geometric vocabulary serves as fundamental structural patterns that facilitate structure-level knowledge transfer, where trees are modeled in hyperbolic space, cycles in spherical space and sequences in Euclidean space. This vocabulary is defined by the condition of Structural Completeness and Geometric Representability.
Riemannian Graph Tokenizer (RGT): RGT quantizes graph structures across multiple domains within constant curvature spaces, where the Riemannian quantizer preserves the intrinsic geometry of nonEuclidean manifolds and is supported by theoretical analysis. The process involves encoding structural features via a Riemannian Encoder (Fréchet mean method), followed by quantization using a mechanism that selects the nearest code from the codebook based on geodesic distance, as formalized in Eq.(6).
Riemannian Straight-Through Estimator (RSTE): To maintain intrinsic structure during training, RGT proposes the Riemannian Straight-Through Estimator (RSTE), formulated as Eq.(8), which ensures that gradients are propagated within the tangent space T zM, which is consistent with the underlying geometry of the space.
Geometry-Aligned Decoder: The decoder reconstructs original structural features by utilizing a geometry-aligned decoder that performs operations like Riemannian parallel transport to align eM i from the point Q M i(M in H, S) to the tangent space T oK M at the north pole oK. This ensures unified decoding of embeddings across geometric spaces with varying curvatures, ensuring consistency in representation.

Key Contributions and Theoretical Foundation

The main contributions of this work are summarized as follows:

- Geometric Vocabulary:

The geometric vocabulary (trees, cycles and sequences) serves as fundamental structural patterns that facilitate structure-level knowledge transfer. This modeling paradigm effectively bridges topological heterogeneity.

- Riemannian Graph Tokenizer:

We propose RGT that quantizes graph structures across multiple domains within constant curvature spaces, where the Riemannian quantizer preserves the intrinsic geometry of nonEuclidean manifolds and is supported by theoretical analysis.

- Extensive Evaluation:

We conduct thorough theoretical analysis and experiments on multiple real-world graph datasets, showing that RGT achieves competitive performance on graph tasks and exhibits significant cross-domain generalization.

The paper provides a theoretical foundation through two theorems:

- Theorem 1 (Cross-domain Encoding Consistency):

This theorem establishes that RQ preserves local consistency, we now show that RQ also enjoys a global optimality on Riemannian manifolds. It proves that if the geodesic distance between two points is small, the distance between their quantized representations is bounded by three times that small distance.

- Theorem 2 (Optimality of Riemannian Quantization):

This theorem shows that selecting each codeword as the global centroid of a set of geometric vocabularies minimizes the total expected geodesic distortion. This confirms that structurally similar geometric vocabularies (such as trees, cycles, and sequences) can be consistently mapped to the same code in the corresponding codebook, thereby enabling cross-domain structural alignment.

Experimental Validation and Results

Extensive experiments across diverse datasets demonstrate that RGT significantly enhances structural knowledge transferability across graph domains.

- Cross-Domain Transfer Learning Performance (RQ1):

RGT demonstrates significant advantages in link prediction, yielding superior results in link prediction compared to existing GFMs, particularly when handling non-attributed graphs.

- Few-Shot Learning Performance (RQ2):

RGT shows strong transferability in this lowresource setting, performing competitively even without exposure to the target domain, and improving significantly as more structural information becomes available in five-shot settings.

- Importance of Geometric Vocabulary (RQ3):

The assignment of trees to hyperbolic space, cycles to spherical space, and sequences to Euclidean space yields the best performance among all configurations. This confirms the effectiveness of the geometry vocabulary in this work.

Scalability and Ablation Studies

The model is evaluated on web-scale graph datasets like ogbn-products, where RGT exhibits strong scalability and competitive performance. Ablation studies indicate that applying the Riemannian quantizer leads to better performance compared to the non-quantized variant, and the Riemannian quantizer plays a more critical role in enhancing the model’s transferability across domains. Furthermore, pretraining significantly enhances RGT’s capability in transferring structural knowledge across domains.

The overall training process is summarized in Algorithm 1, with a computational complexity of O (V 2 + E). The empirical results confirm that RGT maintains consistently strong performance across domains, further reinforcing its robustness and adaptability in diverse graph learning scenarios. This work primarily contributes a novel Riemannian graph tokenizer, designed to quantise and transfer structural knowledge across graph domains.

The paper is licensed under a Creative Commons Attribution 4.0 International License. The ACM ISBN is 979-8-4007-2307-0/2026/04. https://doi.org/10.1145/3774904.3792120. The ACM Reference Format is Qimin Zhou, Haibo Liu, Yujie Wang, Li Sun, and Chuan Shi. 2026. Riemannian Graph Tokenizer for Structural Knowledge Transfer. In Proceedings of the ACM Web Conference 2026 (WWW ’26), April 13–17, 2026, Dubai, United Arab Emirates. ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3774904.3792120. The work is supported in part by the National Natural Science Foundation of China(No.62550138,62192784,62572064,62472329,62202164). The references are listed on page 9. The paper is available at https://github.com/Miyla77/RGT.git. The paper is published in WWW ’26, Dubai, United Arab Emirates.

--- Page 1 ---

Riemannian Graph Tokenizer for Structural Knowledge Transfer Qimin Zhou Haibo Liu Yujie Wang Li Sun Chuan Shi Abstract Foundation models are at the forefront of artificial intelligence. A tokenizer, converting the raw input into discrete representations that the model can understand, plays an important role to the success of foundation models. Unlike the text tokenizer that is well studied in large language models, graph tokenizer is still at its early stage, facing the challenges of tackling the non-Euclidean structures and capturing the structural semantics. How to design a graph tokenizer for structural knowledge transfer? To this end, we propose a Riemannian Graph Tokenizer (RGT) that bridges the structural knowledge and quantized representations to support cross-domain structural knowledge transfer. The connection is established by Riemannian geometry. Specifically, we first define the geometric vocabulary (trees, cycles and sequences), which captures fundamental structural patterns and reflects the intrinsic geometry of graph. Second, we construct a Riemannian quantizer with Riemannian Straight-Through Estimator to tokenise graph structures across multiple domains into discrete tokens. To ensure consistency and transferability across diverse geometric spaces, RGT further incorporates a geometry-aligned decoder that projects manifoldspecific tokens into a unified tangent space. The theoretical analysis and geometric interpretations are provided to support the effectiveness of our proposed method. Extensive experiments across diverse datasets demonstrate that RGT significantly enhances structural knowledge transferability across graph domains. CCS Concepts Computing methodologies Knowledge representation and reasoning; Theory of computation Computational geometry Keywords Graph Tokenizer Riemannian Geometry Structural Knowledge Transfer

--- Page 2 ---

Graph is a universal language for describing and modeling complex systems [51, 61]. Graphs data are ubiquitous in the real world, such as molecular graph [9], social graph [50] and citation networks [18]. Graph machine learning aims to harness machine learning techniques to uncover latent information in graphs and enhance the performance of downstream tasks. Among various graph machine learning approaches, Graph Neural Networks (GNNs) have received increasing attention due to their broad applicability to various Web-related problems [27] and strong empirical performance. However, most existing GNNs methods are typically tailored to specific datasets or tasks, lacking generalization across domains and tasks. Consequently, Graph Foundation Models (GFMs) have emerging as an interesting research topic in the graph domain. GFMs refer to pretrained models across diverse graph datasets that support multiple downstream tasks.

Improvements for AI systems

Bold header: Cross-domain structural knowledge transfer via Riemannian Graph Tokenizer (RGT)

This system can perform cross-domain structural knowledge transfer by quantizing graph structures into discrete tokens embedded in constant curvature spaces, specifically using hyperbolic space for trees, spherical space for cycles, and Euclidean space for sequences.

Bold header: Geometry-aware token generation with Riemannian Straight-Through Estimator (RSTE)

The RSTE formulation prevents distortion by retaining the quantization error in the tangent space, which prevents the distortion typically introduced by Euclidean additive operations during gradient propagation, ensuring geometric consistency between structural tokens and discrete codes.

Bold header: Unified geometric fusion for cross-space representation

The geometry-aligned decoder utilizes Riemannian parallel transport to align embeddings from different manifolds into a unified tangent space, resulting in a final representation where features from all spaces are fused into a single unified representation denoted as xˆi.

Bold header: Optimized structural discretization via Riemannian Quantizer (RQ)

The RQ minimizes expected geodesic distortion by selecting codewords as the global centroid of geometric vocabulary subsets, ensuring that structurally similar geometric vocabularies (such as trees, cycles, and sequences) can be consistently mapped to the same code in the corresponding codebook.

Bold header: Robust generalization across non-attributed graph domains

The RGT model demonstrates a key strength in handling non-attributed graphs by structural features, which outperforms RiemannGFM and shows stable performance across diverse pretraining datasets when compared against models like OpenGraph.

Bold header: Improved few-shot learning capabilities in low-resource regimes

By leveraging the geometric vocabulary, RGT exhibits strong transferability in this low-resource setting, enabling competitive performance on tasks like node classification even with limited supervision, as seen in the one-shot and five-shot results.