End-to-End Autonomous Generation of Human Assembly Plans
cs.RO, cs.GR
Submitted: 2026-10-07
Updated: 2026-10-07
Comments: 22 pages, 10 figures
Code: https://github.com/IDEALLab/AssembleX
License: http://creativecommons.org/licenses/by/4.0/
The gist: Turning a CAD design into an assembly plan is still largely done by hand, requiring engineers to reason about geometric feasibility, tool access, stability, and the ergonomics of human assembly.
Terminology
Abstract
Turning a CAD design into an assembly plan is still largely done by hand, requiring engineers to reason about geometric feasibility, tool access, stability, and the ergonomics of human assembly. In this work, we encode long-established design for assembly (DfA) principles into a contained, end-to-end approach for generating assembly plans. Our approach takes only a mesh assembly and produces either a step-by-step assembly manual or a structured failure report, requiring no joint metadata, fastener annotations, or additional information. Four major components of a manufacturing plan are addressed autonomously: an assembly tool list, the assembly sequence and subassemblies, an assembly manual, and design feedback for improving assemblability. For determining the sequence plan, we systematically disassemble the object in a physics simulator and apply a cost function that encodes DfA principles. Manual generation, tool labelling, and assembly feedback rely primarily on multimodal large language models. Compared with a baseline that always removes the outermost part first from Tian et al., DfA-aware sequence planning reduces simulated assembly time, measured with a robot-arm assembly-time proxy, by 35% on 136 assemblies of 5 to 30 parts. The correct tool is selected for 88.6% of assembly steps. A vision-language model judge compares the generated manuals against ablated variants, identifying which page elements carry the information a reader needs. The presented approach and open-source code are available for use by engineers or AI agents looking to rapidly accelerate the creation of manufacturing plans for a given product design.
Sources
- Physics-in-the-Loop: A Hybrid Agentic Architecture for Validated CAD Engineering Design
- MUSE: Benchmarking Manufacturable, Functional, and Assemblable Text-to-CAD Generation
- A Survey on LLM-as-a-Judge
- Judge's Verdict: A Comprehensive Analysis of LLM Judge Capability Through Human Agreement
- A Unified Framework for Automated Assembly Sequence and Production Line Planning using Graph-based Optimization
- OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models
- CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation
- ArtiCAD: Articulated CAD Assembly Design via Multi-Agent Code Generation
- IKEA Object State Dataset: A 6DoF object pose estimation dataset and benchmark for multi-state assembly objects
- Fabrica: Dual-Arm Assembly of General Multi-Part Objects via Integrated Planning and Learning
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies