PART: Learning 3D Part Assembly and Retrieval with Transformers

arXiv:2609.19872 · cs.CV, cs.GR · Submitted 2026-09-17 · Read on arXiv

cs.CV, cs.GR

Submitted: 2026-09-17

Updated: 2026-09-17

Comments: Accepted to SIGGRAPH Asia 2026 Conference Papers. 11 pages, 12 figures. Project page: https://iambrc.github.io/PART-project-page/

Project page: https://iambrc.github.io/PART-project-page

License: http://creativecommons.org/licenses/by/4.0/

The gist: 3D assembly is fundamental to modern manufacturing and digital content creation.

Terminology

Abstract

3D assembly is fundamental to modern manufacturing and digital content creation. In this paper, we present PART, a unified transformer-based framework for 3D part retrieval and assembly: given a target shape and a part library, PART automatically selects the appropriate parts and predicts their 6-DoF poses to reconstruct the target. While prior work has achieved impressive progress on assembling a pre-defined set of parts, this more practical retrieval-based setting remains largely unexplored. The task faces three key challenges: (i) a combinatorially explosive search space that grows exponentially with library size; (ii) variable-length outputs, as different targets require different numbers of parts; and (iii) continuous 6-DoF pose estimation for part assembly. To address these, we formulate retrieval and assembly as a set prediction problem and design a novel transformer-based framework that retrieves parts and regresses their poses with variable-length output. Additionally, we exploit the duality between part pose estimation and target segmentation through joint training and a novel segmentation-enhanced optimization module. Finally, We curate a large-scale dataset of 80K+ shapes, and the results show that PART generalizes to scene layouts, image targets, and real-world scans. Project Page: https://iambrc.github.io/PART-project-page/.

Sources

Related papers