Training Graph Foundation Models on The Web Graph
cs.AI, cs.LG
Submitted: 2026-09-25
Updated: 2026-09-25
License: http://creativecommons.org/licenses/by/4.0/
The gist: We introduce Acacia, a graph foundation model, trained on the web graph.
Terminology
Abstract
We introduce Acacia, a graph foundation model, trained on the web graph. Acacia (i) supports arbitrary feature dimensionalities and semantics without additional training, (ii) supports a wide range of tasks, including node classification, link prediction, node clustering, and graph generation, without additional training, (iii) has in-context learning capabilities, and (iv) does not rely on pretrained LLMs. In particular, existing graph foundation models often require training additional classification heads or feature projectors to accommodate new graphs or new labels, whereas Acacia does not. Moreover, existing graph foundation models often gain their capabilities by being stitched together with pretrained LLMs, whereas Acacia is trained from scratch using only the Common Crawl web graph. This is also an important result because it provides evidence that graph models can acquire emergent capabilities from scratch like LLMs.
Sources
- SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
- Billion-Scale Graph Foundation Models
- Flatten Graphs as Sequences: Transformers are Scalable Graph Generators
- Graph Generative Pre-trained Transformer
- Can TabPFN Compete with GNNs for Node Classification via Graph Tabularization?
- Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors
- Turning Tabular Foundation Models into Graph Foundation Models
- GraphPFN: A Prior-Data Fitted Graph Foundation Model
- Advancing Graph Few-Shot Learning via In-Context Learning
- Bringing Graphs to the Table: Zero-shot Node Classification via Tabular Foundation Models
- Generalizing Graph Transformers Across Diverse Graphs and Tasks via Pre-training
- UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs
- UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphs
- Open Graph Benchmark: Datasets for Machine Learning on Graphs
- PRODIGY: Enabling In-context Learning Over Graphs
- Why Does Graph Learning Fail to Fully Benefit from a Text Teacher?
- GOFA: A Generative One-For-All Model for Joint Graph Language Modeling
- GraphFM: A generalist graph transformer that learns transferable representations across diverse domains
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Are Large Language Models In-Context Graph Learners?
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems