MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: I'm Rosa.
Dev: And I'm Dev, here with Taro.
Rosa: We're looking at the paper: "MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation".
Dev: Here's the gist of the paper. MolmoBot introduces MolmoBot-Engine to generate large-scale procedural data and demonstrates that zero-shot transfer to real-world mobile manipulation is effective without any real-world fine-tuning.
Rosa: This capability is significant because it challenges the assumption that simulation alone is sufficient for robust manipulation policies.
Dev: The work provides an open-source pipeline and a dataset of 1.7 million expert trajectories to enable this transfer across diverse tasks and robots.
Rosa: Policy Architectures Three policy classes are trained on the MolmoBot-Data, including MolmoBot, which is a Molmo2-based multi-frame vision-language model with a flow-matching action head.
Title and authors: Rosa: So we're looking at this paper today, "MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation." It sounds like they're tackling that big problem where training robots in the real world is super expensive and slow.
Dev: Yeah, they’ve built a system called MolmoBot to generate a massive amount of procedural data in simulations, and the main claim is that you can use this data to get these policies into the real world without doing any extra, dedicated fine-tuning.
Rosa: It's about using simulation to create a really big foundation model for manipulation agents, which is something that’s become really important lately because building those models usually takes huge resources.
Taro: I'm thinking about the authors—they’ve got a lot of people listed on the title page, which shows how much effort went into making this data generation pipeline work across all those different robots and tasks.
Dev: Exactly, and it suggests they are trying to provide a more complete picture of what it takes to build these generalist agents, since we often don't see all the details behind the strongest models.
The paper's summary: Rosa: Okay, so what’s actually in this paper? They’re talking about using MolmoBot-Engine, which is this whole open-source pipeline for generating procedural data across different robots and environments in their simulation space called MolmoSpaces.
Dev: That engine uses a huge set of 232k environments with forty-eight thousand manipulable objects spread across eight different task types, and they do extensive domain randomization, which means they randomize the environment, the actions taken, and even how the camera sees things.
Rosa: It’s not just generating data; it’s about creating this dataset of one point seven million expert trajectories that allows these policies to generalize really well to things they haven't seen before in reality.
Taro: What I find interesting is that they aren't just training one thing; they trained three different policy classes: MolmoBot, which is a vision-language model with a flow-matching action head, MolmoBot-Pi0, which replicates the old pi zero architecture for comparison, and MolmoBot-SPOC.
Dev: That separation into classes is key because it lets them see if the performance gains come from the specific architecture of the policy or just from having this massive amount of diverse training data.
The paper's improvements: Rosa: They show some really solid results, particularly when they test these policies on two different robots: the Franka FR3 for tabletop tasks and the Rainbow Robotics RB-Y1 mobile manipulator.
Dev: On the tabletop pick-and-place tasks, their best MolmoBot policy gets a success rate of seventy nine point two percent in real world evaluations across four different settings, which is significantly better than pi zero point five at thirty nine point two percent.
Taro: That comparison between the best MolmoBot and pi zero point five is telling because it shows that their model, which uses this large-scale synthetic data, can perform better than a policy based on a much smaller dataset trained from scratch.
Rosa: And they also demonstrated zero-shot transfer. They showed that these policies can move directly from simulation to unseen real-world objects and environments without needing any extra real-world fine-tuning at all.
Dev: That zero-shot capability is what makes this work so significant, because it challenges the idea that you absolutely have to collect a lot of real world data or do task-specific fine tuning to get a robust manipulation policy.
Conclusion: Rosa: So, to wrap up on "MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation," the main point is that procedural environment generation combined with diverse assets allows policies to generalize broadly from simulation to the real world.
Dev: They conclude that zero-shot transfer works effectively for both static and mobile manipulation, and they open up the MolmoBot-Engine pipeline so other groups can extend this approach to new robots and tasks.
Taro: I just want to add that they did show some limitations; they didn't mention what happens when the real world misbehaves in a way their simulation didn't predict, which is always something we have to consider when moving these models out of the lab.
Rosa: That’s fair, and it makes sense because this is all about pushing the boundaries of what simulation alone can do for robotics.
Dev: Yeah, so we have this open-source engine and a dataset of one point seven million expert trajectories that shows that massive scale synthetic data can really help build generalist foundation models for manipulation agents.
Taro: Alright, yeah, that’s the picture for this paper. We'll take a quick break and then I think we’ll look at some papers on real-time execution next.
Abhay Deshpande, Maya Guru, Rose Hendrix, Snehal Jauhri, Ainaz Eftekhar, Rohun Tripathi MaxArgus, Jordi Salvador Haoquan Fang Matthew Wallingford Wilbert Pumacay Yejin Kim Quinn Pfeifer Ying-Chun Lee Piper Wolters Omar Rayyan Mingtong Zhang Jiafei Duan Karen Farley Winson Han Eli Vanderbilt Dieter Fox Ali Farhadi Georgia Chalvatzaki Dhruv Shah Ranjay Krishna
Allen Institute for AI · University of Washington · University of California, Los Angeles · Technische Universität Darmstadt · Princeton University
cs.RO
Submitted: 2026-03-17
Updated: 2026-11-08
Code: https://github.com/Physical-Intelligence/openpi
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 100/100
The gist: MolmoBot introduces MolmoBot-Engine to generate large-scale procedural data and demonstrates that zero-shot transfer to real-world mobile manipulation is effective without any real-world fine-tuning.
Key concepts
- MolmoBot
- This is a vision-language model based on Molmo2, designed to handle multi-frame vision and language inputs. It uses a flow-matching action head to generate control signals, making it capable of learning complex manipulation skills directly from simulated visual data.
- MolmoBot-Pi0
- This policy class replicates the standard $\pi_0$ architecture used in reinforcement learning for comparison. It serves as a baseline to isolate whether performance gains come from the specific MolmoBot design or simply from the quality and scale of the training data.
- MolmoBot-SPOC
- This is a lightweight policy designed specifically for deployment on edge devices. It is suitable for mobile robots and can be fine-tuned using reinforcement learning, offering a balance between performance derived from simulation and computational efficiency in real-world settings.
Terminology
Summary
MolmoBot introduces MolmoBot-Engine to generate large-scale procedural data and demonstrates that zero-shot transfer to real-world mobile manipulation is effective without any real-world fine-tuning. p. 0 This capability is significant because it challenges the assumption that simulation alone is sufficient for robust manipulation policies. p. 1 The work provides an open-source pipeline and a dataset of 1. p. 37 million expert trajectories to enable this transfer across diverse tasks and robots. p. 1
Policy Architectures
Three policy classes are trained on the MolmoBot-Data, including MolmoBot, which is a Molmo2-based multi-frame vision-language model with a flow-matching action head. Additionally, MolmoBot-Pi0 replicates the π0 architecture for direct comparison, and MolmoBot-SPOC is a lightweight policy suitable for edge deployment and amenable to RL fine-tuning. p. 0
Data Generation Pipeline
MolmoBot-Engine is the fully open-source pipeline for procedural data generation across robots, tasks, and diverse simulated environments in MolmoSpaces. This engine leverages MolmoSpaces [6], an ecosystem of 232k environments with 48k manipulable objects across 8 types of tasks. p. 1 The data generation involves extensive domain randomization across three axes: environment randomization, action randomization, and camera perturbation. p. 4
Performance Benchmarks
The policies are evaluated on two robotic platforms: the Franka FR3 for tabletop manipulation tasks and the Rainbow Robotics RB-Y1 mobile manipulator. On tabletop pick-and-place, MolmoBot achieves a success rate of 79. p. 02% in real world evaluations across 4 settings, outperforming π0. p. 05 at 39.2%.
Zero-Shot Transfer Results
The results demonstrate that procedural environment generation combined with diverse articulated assets can produce robust manipulation policies that generalize broadly to the real world. MolmoBot policies transfer zero-shot from simulation to unseen real-world objects and environments. p. 2
Model Comparison and Ablations
MolmoBot-Pi0 achieves a success rate of 46.7% in real world evaluations, improving upon π0. p. 25 at 39.2% when using the same architecture and training with MolmoBot-Data from scratch. p. 2 The study also shows that data scale and diversity are important, as performance improves monotonically as the amount of training data increases. p. 2
Conclusion
The work concludes by stating that zero-shot transfer to the real world is not only possible, but effective for both static and mobile manipulation. The authors release MolmoBot-Engine to enable the community to extend this approach to new robots, tasks, and object categories. p. 0
Author Contributions
Rose Hendrix led the project, wrote large parts of the core data engine and scaling infrastructure, and advised on all components. MayaGuru led data generation for RB-Y1 object manipulation + drawer/cabinet articulation, MolmoBot-SPOC architecture + training for mobile manipulation, and real evaluation for RB-Y1. p. 0 Snehal Jauhri led data generation for RB-Y1 door opening and MolmoBot training for mobile manipulation. p. 0
Acknowledgment
This work would not be possible without the support of our colleagues at Ai2. The authors thank Christopher Clark and Tanmay Gupta for helpful research discussions and sharing of relevant findings across related projects. p. 3
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improve zero-shot sim to real transfer by training policies exclusively on large-scale synthetic data, as
Without any real-world fine-tuning, our policies achieve zero-shot transfer to unseen objects and environments.
This allows for the creation of robust manipulation policies without requiring expensive real-world data collection or explicit domain adaptation. -
Enhance policy generalization across architectures by utilizing MolmoBot (VLM-based) which
builds on Molmo2 [7], a vision-language model pretrained on large-scale image-text data
and comparing it against MolmoBotPi0, whichexactly replicates the π0 architecture for controlled comparison.
This enables researchers to determine if performance gains are due to data diversity or architectural differences. -
Improve adaptability for edge deployment by training MolmoBot-SPOC, a
lightweight policy suitable for edge deployment and amenable to RL fine-tuning,
which is compared against the more complex MolmoBot and MolmoBot-Pi0 classes. This provides a smaller, deployable model while still leveraging the benefits of simulation data. -
Improve task instruction following by implementing language instructions that sample
referring expressions
based on CLIP similarity scores with other scene objects to producediverse yet unambiguous expressions.
This makes policies more robust to different phrasing and object references during training and inference. -
Enhance spatial grounding for point conditioning by using a learned coordinate position embedding added to point tokens in MolmoBot-SPOC, allowing the model to condition actions on 2D pixel coordinates specifying target objects or locations.
Sources
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- Gemini Robotics: Bringing AI into the Physical World
- Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding
- RT-1: Robotics Transformer for Real-World Control at Scale
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model
- LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfer
- GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data
- InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policy
- PartInstruct: Part-level Instruction Following for Fine-grained Robot Manipulation
- Procedural Generation of Articulated Simulation-Ready Assets
- MoTo: A Zero-shot Plug-in Interaction-aware Navigation for General Mobile Manipulation
- Adaptive Mobile Manipulation for Articulated Objects In the Open World
- Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer
- AI2-THOR: An Interactive 3D Environment for Visual AI
- AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems
- Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
- SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
- PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies
- Sigmoid Loss for Language Image Pre-Training
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation