SMART: Zero-Shot Sim-to-Real Articulated Object Manipulation via Large-Scale Synthetic Pretraining
cs.RO, cs.AI
Submitted: 2026-10-06
Updated: 2026-10-06
Comments: Technical Report, 31 pages
Project page: https://teamillusion-smart.github.io
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: The ability to interact with articulated objects is essential for embodied intelligent systems, but collecting large-scale real-world demonstrations for these interactions remains challenging due to
Abstract
The ability to interact with articulated objects is essential for embodied intelligent systems, but collecting large-scale real-world demonstrations for these interactions remains challenging due to the precise contact and constraint-following motions involved. Although simulation provides a promising alternative, existing synthetic data efforts cover limited articulated-object categories, while general-purpose synthesis pipelines lack explicit designs for part-level semantics and articulation constraints, hindering agentic task generation and scalable synthesis of high-quality articulated-manipulation demonstrations. To bridge this gap, we introduce SMART, a scalable system leveraging large-scale Synthesized Manipulation demonstrations for ARTiculated-object manipulation. At its core, we develop SMART-Sim, a simulation platform with articulation-aware design that enables effective task generation and efficient demonstration collection. Building on SMART-Sim, we apply agentic task generation and design a scalable distributed synthesis system, using them to synthesize SMART-Data, comprising over 1M demonstrations across 44 atomic task types, 5 robot setups, and 2,507 articulated objects. The vision-language-action (VLA) model pretrained on SMART-Data shows competitive performance on simulation benchmarks and achieves zero-shot sim-to-real transfer and scalable performance in real-world articulated-object manipulation tasks. This highlights the potential of synthetic demonstrations in providing effective and scalable supervision for improving VLA model performance in contact-rich articulated-object manipulation.
Sources
- ArtiBench and ArtiBrain: Benchmarking Generalizable Vision-Language Articulated Object Manipulation
- Articulated Object Manipulation using Online Axis Estimation with SAM2-Based Tracking
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
- A Pragmatic VLA Foundation Model
- RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation
- AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems
- EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
- Hy-Embodied-0.5-VLA: From Vision-Language-Action Models to a Real-World Robot Learning Stack
- EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos
- EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data
- RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robots
- InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policy
- MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation
- Procedural Generation of Articulated Simulation-Ready Assets
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations
- Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning
- robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
- SceneFoundry: Generating Interactive Infinite 3D Worlds
- SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies