GHOST: Geometry-Constrained Hybrid Diffusion for Open-Vocabulary 3D Scene Graph Generation

arXiv:2610.16455 · cs.CV, cs.MM · Submitted 2026-10-12 · Read on arXiv

cs.CV, cs.MM

Submitted: 2026-10-12

Updated: 2026-10-12

Comments: 10 pages. Accepted at ACM MM 2026 (34th ACM International Conference on Multimedia)

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

The gist: 3D scene graph generation requires understanding both object semantics and metric spatial relations.

Abstract

3D scene graph generation requires understanding both object semantics and metric spatial relations. However, existing open-vocabulary methods often struggle with spatial predicates because language similarity reflects semantic co-occurrence rather than explicit geometry. We propose GHOST, a Geometry-constrained Hybrid diffusion framework for Open-vocabulary 3D Scene graph generaTion. GHOST jointly models continuous node geometry and discrete edge predicates, following a geometry-to-constraint-to-predicate denoising process to ensure geometrically consistent relation prediction. It introduces language-grounded constraint induction, which maps predicate embeddings to differentiable geometric constraints that guide both layout refinement and predicate reweighting. A Geometry-Topology Collaborative Denoiser further couples node and edge reasoning through noise-adaptive interaction and orientation-aware modeling. Experiments on 3DSSG demonstrate state-of-the-art performance in both closed-set and open-vocabulary settings, including over 17-point gains over RelationField on spatial predicates.

Related papers