More Domains, No Gains? FLO: Fast-Slow Reciprocal Learning for Multi-Domain Graph Pre-training

arXiv:2610.14748 · cs.LG · Submitted 2026-10-10 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "More Domains, No Gains? FLO".

Jane: The gist The FLO approach proposes a Fast–sLow reciprocal learning method for multi-domain graph pre-training,

Tom: First, who's behind it and why it matters.

Paper summary: Jane: To wrap up, we're talking about the paper "More Domains, No Gains? FLO: Fast-Slow Reciprocal Learning for Multi-Domain Graph Pre-training". The authors are showing that existing methods struggle when you try to increase domain diversity because of how they handle message passing and domain alignment.

Tom: They propose FLO, which uses a fast slow process and a propagation-free attribute branch to refine relations using node attributes, leading to the best average performance on twelve datasets for cross-domain and cross-task transfer <ref:2610.14748#pg3>.

Lu: The main implication is that we should focus on refining those relations with attribute feedback rather than just trying to align feature spaces or metadata across domains.

Meng: For deployment, this suggests a framework where you can decouple relation refinement from domain adaptation, which could make training more robust when moving between different graph structures.

Lalam: I'm confident in the direction this is going because the experiments confirm that FLO achieves superior performance and benefits from increasing domain diversity.

Jane: So what does this mean for someone who just listens to the show? It means that if you are building a system that needs to work across different types of data structures, focusing on how you refine those underlying relationships with attribute information might be more effective than trying to force everything into one neat feature space.

Conclusion: Tom: So we’ve been deep in FLO, and now we’re at the end of this paper. The authors are asking a pretty blunt question right there in the title: "More Domains, No Gains?"

Jane: It's really direct. They're suggesting that just throwing more data domains at a graph pre-training model isn't automatically going to make it better.

Tom: Exactly. They’re arguing that existing methods often fail because they’re stuck in this single message-passing loop, which means they can’t actually learn anything beyond the immediate neighbors.

Jane: So, what FLO does differently is it introduces this fast slow process and a propagation-free attribute branch to refine those relations.

Tom: Yeah, it's about learning the actual connections between nodes using node attributes without relying solely on how information passes between nodes. It’s a reciprocal learning thing where the graph and the attributes talk back to each other.

Jane: The results are pretty compelling though. They showed that across twelve different datasets, FLO actually gets the best average performance for cross-domain and cross-task transfer compared to other methods.

Tom: That’s a big number because it means that as you add more domains, the accuracy actually goes up steadily with FLO. It's not just getting stuck; it's learning effectively.

Jane: And there’s this theoretical part too, where they prove that this fast slow knowledge acquisition really filters out all the cross-domain interference you might see when switching between sources.

Tom: That filtering effect is what makes me interested—it suggests that domain alignment isn't just a prerequisite anymore; it seems like a necessary step to actually acquire the right knowledge.

Jane: It puts things into perspective for anyone building something that needs to generalize across different types of data structure. We’ve seen how this approach handles those complex relationships in unseen graphs.

Tom: Because the paper suggests we should probably stop thinking about just aligning features and start focusing on these refined relations guided by attributes. That’s what we need to think about next if we want to make sure our AI models actually work where they are deployed.

Yupeng Tong, Lianze Shan, Jitao Zhao, Di Jin, Dongxiao He

School of Computer Science and Technology, Tianjin University

cs.LG

Submitted: 2026-10-10

Updated: 2026-10-10

Comments: 28pages,7 figures

Code: https://github.com/PetarV-/DGI

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

The gist: The gist The FLO approach proposes a Fast–sLow reciprocal learning method for multi-domain graph pre-training, which achieves state-of-the-art performance in cross-domain and cross-task transfer by

Key concepts

Propagation–Attribute Relation Refinement (PARR): This mechanism uses a two-layer GCN for the graph encoder and an MLP for the attribute branch. It predicts relations without message passing, refining the model's understanding by using node attributes alone as feedback. This allows the model to learn knowledge beyond simple neighbor propagation.
Fast–Slow Knowledge Acquisition (FSKA): FSKA separates parameters into a fast process that adapts quickly to the current domain and a slow process that maintains a copy of parameters via Exponential Moving Average (EMA). The slow process integrates knowledge across domains, filtering out interference from rapid changes in source domains.
Propagation-free Attribute Branch: This component predicts graph relations directly using node attributes instead of relying on message passing between neighboring nodes. This constraint ensures the model learns relation predictions that are grounded in attributes, enabling it to acquire knowledge that goes beyond what is learned through standard propagation.
Reciprocal Relation Learning: Instead of just one direction of learning, FLO uses a two-way process where the graph branch refines relations based on attribute feedback from the slow process, and vice versa. This reciprocal refinement stabilizes knowledge acquisition and leads to better transfer performance.

Summary

The gist The FLO approach proposes a Fast–sLow reciprocal learning method for multi-domain graph pre-training, which achieves state-of-the-art performance in cross-domain and cross-task transfer by refining node relations beyond message passing and stabilizing knowledge acquisition through a fast/slow process.

Limitations of Existing Methods

Existing methods struggle to benefit from increasing domain diversity because pre-training in most Graph Foundation Models is confined to a single message-passing paradigm, which flattens relations among neighboring nodes and prevents the model from perceiving knowledge beyond propagation. Furthermore, domain alignment strategies often rely on randomly initialized parameters that are jointly updated with the graph model using immediate loss feedback, leaving early knowledge acquisition vulnerable to cross-domain interference.

FLO's Core Mechanism

FLO addresses these issues by proposing a propagation-free attribute branch which predicts the graph model’s relations without message passing and returns these predictions to refine the model, thereby acquiring knowledge beyond propagation. The approach further stabilizes knowledge acquisition by pairing a fast process that adapts to the current domain with a slow process that integrates knowledge across domains. Specifically, it uses Propagation–Attribute Relation Refinement (PARR) to refine relations with feedback grounded in node attributes alone and takes this feedback from a slow process that integrates knowledge across domains.

Fast–Slow Knowledge Acquisition (FSKA)

The fast process adapts to the current domain, while the slow process integrates knowledge across domains by maintaining a slow copy of the online parameters updated via an Exponential Moving Average (EMA). This separation addresses the sharp fluctuations in pre-training loss observed when source domains alternate, as the slow process filters cross-domain interference from the feedback. The slow copy supplies the relation target qa in Eq. (6), which is used to guide reciprocal learning.

Theoretical Guarantees

The theoretical analysis proves that PARR reduces the propagation-induced part R of the relations and keeps P (knowledge beyond propagation). The slow process filters cross-domain interference by showing that its period-S oscillation is attenuated by a factor ρ, which can be as small as 0.009 with specific parameters.

Experimental Results

On 12 datasets, FLO benefits from increasing domain diversity and achieves the best average performance in cross-domain and cross-task transfer. Specifically, FLO achieves the best average accuracy in crossdomain and cross-task transfer on 12 datasets. The results demonstrate that the accuracy of FLO increases steadily as source domains are added.

Ablation Study Insights

The ablation study confirms the distinct roles of the two objectives: graph-to-attribute transfer identifies the propagation-free conditional target, while attribute-to-graph transfer returns this target to the transferable graph encoder. Removing both relation learning objectives lowers accuracy most on CiteSeer and Cora, the two graphs on which propagation changes the relations most.

Conclusion

FLO achieves superior performance by refining node relations with attribute-grounded feedback and stabilizing knowledge through a fast/slow reciprocal learning scheme. This suggests that domain alignment is a prerequisite rather than a guarantee of knowledge acquisition.

How it works

The proposed method involves several key components and steps:

** Propagation–Attribute Relation Refinement (PARR): The graph branch uses a two-layer GCN as the graph encoder fG, while the attribute branch fM uses an identity matrix in place of Ab and is implemented as a two-layer MLP. This constraint makes its optimal relation prediction the attribute-predictable component P (Definition 3.1), turning reciprocal relation learning into a conditional prediction-and-refinement process rather than ordinary agreement between peer encoders. The loss terms are defined as L G rec + L M rec + λ reL G kd + L M kd. 2**

** Fast–Slow Knowledge Acquisition (FSKA): This mechanism separates parameters that adapt to the current source graph (fast process) from those that supply the relation targets (slow process). The slow copy receives no gradients and forms the slow process, which integrates knowledge across domains. The online parameters are optimized on the current source graph and form the fast process, which acquires knowledge from the current domain. 3**

** Training Objective: The overall objective combines reconstruction with the two directions of relation learning: L = L G rec + L M rec + λ reL G kd + L M kd, where re increases its weight linearly over the first Tw epochs. 4**

** Inference: After pre-training, only the frozen graph encoder fG is applied to unseen graphs without input masks or domain-specific encoder parameters. 5**

Key Findings

The experiments confirm that FLO achieves the best average performance in crossdomain and cross-task transfer and benefits from increasing domain diversity. The results show that the relation change caused by propagation correlates with the accuracy drop when relation learning is removed for CiteSeer and Cora. FLO successfully learns generalizable knowledge that transfers to unseen tasks on unseen graphs. The model's computational cost is competitive, with its forward pass being faster than several other multi-domain graph pre-training methods.

Future Work

Future work will extend FLO to graphs with scarce node features. The paper concludes that the proposed method provides a robust framework for multi-domain graph pre-training by decoupling relation refinement from domain adaptation.

Improvements for AI systems

Bold header: Propagation-Free Relation Refinement (PARR) for Deep Knowledge Acquisition

This mechanism addresses how existing methods struggle to benefit from increasing domain diversity by using a propagation-free attribute branch that predicts relations without message passing, allowing the model to learn the knowledge beyond propagation.

Bold header: Fast–Slow Knowledge Acquisition (FSKA) for Cross-Domain Stability

The system implements FSKA, which pairs a fast process adapting to the current domain with a slow process integrating knowledge across domains, ensuring that early knowledge acquisition [is] protected from cross-domain interference by having the slow process filter cross-domain interference from the feedback.

Bold header: Reciprocal Relation Learning for Knowledge Correction

The approach utilizes two directions of learning: the attribute branch learns relations via L M kd and returns targets via L G kd, which corrects the graph encoder by making it move toward knowledge beyond propagation without domain-specific parameters.

Bold header: Robustness to Domain Diversity

FLO achieves superior performance because it demonstrates that domain alignment is a prerequisite rather than a guarantee of knowledge acquisition, as shown by the finding that accuracy of FLO increases steadily as source domains are added while baselines like BRIDGE and SAMGPT experience significant loss fluctuations.

Bold header: Transferable Knowledge for Unseen Tasks

By pre-training on node classification datasets and applying the frozen graph encoder to unseen link prediction or graph classification tasks, FLO is shown to learn generalizable knowledge that transfers to unseen tasks on unseen graphs, achieving the best average performance across all downstream evaluation protocols.