World-as-Graph: Relational World Modeling Through Latent Space Graphs
cs.LG, cs.CV
Submitted: 2026-09-30
Updated: 2026-09-30
Comments: under review
Code: https://github.com/Scarlett-Yyq/World-as-Graph
License: http://creativecommons.org/licenses/by/4.0/
The gist: World models aim to learn representations of real-world environments and predict their future evolution.
Terminology
Abstract
World models aim to learn representations of real-world environments and predict their future evolution. Recent object-centric world models have made expressive progress by representing visual scenes as sets of object-level latent states, but object-object relations are often captured only implicitly, which limits explicit relational and temporal structure modeling and object-centric dynamic memory modeling. To address such challenges, we propose World-As-Graph (WAG), a graph-based object-centric world model that introduces relational inductive bias into JEPA-style predictive representation learning. The proposed WAG contains two main modules: (1) Relation-aware structure induction, which constructs time-varying latent graphs from object-centric slots and designs relation-aware object masking policies to guide relational object representation learning in latent space; (2) Object-centric memory transition, which maintains and updates object-level dynamic states by combining relational information from neighboring objects with historical memory, enabling effective autoregressive future prediction. Extensive experiments on both visual reasoning and robotic manipulation tasks could demonstrate the superior performance of our proposed WAG.
Sources
- Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control
- LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics
- Relational inductive biases, deep learning, and graph networks
- Do We Really Need Complicated Model Architectures For Temporal Networks?
- World Models
- Conditional Object-Centric Learning from Video
- When Does LeJEPA Learn a World Model?
- Graph World Models: Concepts, Taxonomy, and Future Directions
- SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
- DINOv2: Learning Robust Visual Features without Supervision
- Sora as a World Model? A Complete Survey on Text-to-Video Generation
- Temporal Graph Networks for Deep Learning on Dynamic Graphs
- Understanding Rollout Error in Graph World Models
- Audio-JEPA: Joint-Embedding Predictive Architecture for Audio Representation Learning
- A Structural Dynamics Graph World Model: Unified Modeling, Constrained Rollout, and Interpretable Calibration
- SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models
- Inductive Representation Learning on Temporal Graphs
- CLEVRER: CoLlision Events for Video REpresentation and Reasoning
- DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks