RoboWits: Unexpected Challenges for Robotic Creative Problem Solving
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: I'm Rosa.
Dev: And I'm Dev, here with Taro.
Rosa: We're looking at the paper: "RoboWits: Unexpected Challenges for Robotic Creative Problem Solving".
Dev: Here's the gist of the paper. The ability to reason, adapt, and creatively solve problems under unexpected challenges is essential for robots operating in real-world environments.
Rosa: RoboWits introduces a bi-manual robotic benchmark designed to systematically evaluate cognitive reasoning, creative tool use, and robustness to unexpected conditions.
Dev: Pruning redundant reasoning tokens RoboWits is a new bi-manual robotic benchmark that explicitly targets evaluating cognitive reasoning, creative tool use, and robustness to unexpected challenges.
Rosa: The benchmark consists of manipulation tasks that require geometric, material, and assembly-based reasoning.
Title and authors: Rosa: So we're looking at this paper called "RoboWits: Unexpected Challenges for Robotic Creative Problem Solving" and it introduces a new way to test how good robots actually are when things go wrong in the real world.
Dev: Right, it’s not just about doing a task right, but about how the robot thinks when it hits something unexpected, which is where this benchmark comes in.
Taro: What I find interesting is that they aren't just giving us simple tasks; they are building these complex scenarios where the robot has to reason and adapt its strategy on the fly.
Rosa: Exactly, and what really stands out is their automated pipeline for creating these tasks, which means they can generate a lot of different reasoning-centric manipulation tasks at scale.
Dev: That pipeline involves several agents working together—one agent seeds the task, another mutates it to make it harder in different ways, and then others verify the task’s feasibility and what kind of evaluation code you need.
Taro: I see how that mutation strategy works because they have three main types: Pivot, Trap, and Add mutations which are specifically designed to block the original solution or introduce noise.
Rosa: The pivot strategy is interesting because it changes the object list by removing things or adding new ones that force the robot to find an entirely different way to solve it.
Dev: And then you have the trap strategy where they inject objects into the scene that look useful but are actually useless because of some subtle constraint, which tests if a robot can distinguish between what looks helpful and what is just noise.
Taro: And there's the add mutation which just increases scene complexity by adding useless distractors, which tests how robust the robot is to perceptual noise.
Rosa: So, this whole setup lets them curate thirty seed tasks that get expanded into two hundred eight tasks with graded difficulty levels from one to five <ref:2605.30326#pg1>.
Dev: And those graded levels let researchers see exactly how robotic models scale when the reasoning required gets more and more complex across geometry, materials, and assembly.
Taro: What I'm seeing is that pre-trained vision-language agents struggle a lot on these mutated tasks, suggesting they are brittle when things get unexpected or constrained in manipulation.
Rosa: That’s a key finding because it shows that even if a model works well on the initial seed task after some fine-tuning, it can fail when the environment throws an unexpected challenge at it.
Dev: The paper also tested VLM Planners and found they perform reasonably well on the seed tasks but fail to generalize effectively to those mutated variants.
Taro: That supports what I’m seeing in the results where models show no strategy transfer for pivot mutations, meaning they keep trying the same failed actions even when objects are displaced.
Rosa: And for trap and add mutations, the success rates drop as more objects are introduced, showing that physical errors there are mostly driven by distraction or freezing due to out-of-distribution visual data.
Dev: The paper also showed that training the VLM Controller with fifty demonstrations actually improves seed performance from seventy percent up to ninety-four percent on the Dominos task, but this gain doesn't translate well when facing mutation tasks.
Taro: That really confirms their conclusion that the primary bottleneck isn't necessarily control limitations, but poor reasoning and strategy adaptation under these unexpected challenges.
Rosa: So, overall, RoboWits gives us a rigorous framework to quantify the gap between basic manipulation skill and higher-level cognitive adaptation in robotics.
Dev: It’s a lot of work they did with that multi-agent pipeline for task generation, which is helpful because it makes the benchmark scalable and reproducible for future research.
Taro: I think this work really sets a good standard for what we need to test next—systems that can handle true uncertainty and creative problem solving in complex physical settings.
Rosa: We'll keep an eye on how researchers use RoboWits to develop more robust systems, because it clearly highlights where current models are falling short when the world misbehaves.
Dev: Next time we talk about paper, we'll look at how other papers are tackling real-time execution and the latency issues that come with those kinds of policies.
The paper's summary: Rosa: So, to wrap up what we just talked about, this paper is basically introducing RoboWits as a new benchmark for testing how robots handle unexpected challenges in creative problem solving.
Dev: Right, it sets up these bi-manual tasks that really push the limits of cognitive reasoning and tool use.
Rosa: What's huge here is their automated pipeline for building these tasks, which means they can generate hundreds of different reasoning-heavy scenarios at scale with actual evaluation code attached to them.
Dev: And that pipeline uses several agents working together—a seed generator, a mutator to make things harder in different ways, and a verifier to check if the task is even possible to run.
Rosa: The key finding they want us to see is that pre-trained vision language agents struggle significantly when faced with these mutated tasks, which points toward them being too brittle for real-world manipulation where you have to adapt your strategy on the fly.
Dev: They found that while the models can do pretty well on the initial seed tasks after a little fine tuning, they really fail to generalize to those mutated variants.
Rosa: And that confirms what we're seeing in practice: real-time feedback is critical for adapting strategies when things go wrong, and it shows a big gap between just controlling the motors and actually having good reasoning skills.
Dev: Specifically, they looked at different types of changes—pivot, trap, and add—and saw that models show no strategy transfer for pivot tasks because they keep trying the original failed actions.
Rosa: And for trap and add mutations, the success rates just drop as more objects get added to the scene because it becomes too much perceptual noise or distraction for them.
Dev: So, what this means for us in controls is that we can't just focus on making a policy that works in a perfect world; we have to engineer systems that are robust enough to handle the kind of messy, constrained environments they create here.
Rosa: It’s not just about fine motor control anymore; it’s about the robot having the higher-level cognitive ability to figure out how to get around things when they don't cooperate.
Dev: And this benchmark is designed to be a scalable testbed, which means researchers can use these tasks repeatedly without manually designing every single hard scenario from scratch.
Rosa: Exactly, so instead of just looking at whether a robot can lift an object, we’re now testing its ability to reason through fluid dynamics or assembly constraints under pressure.
Dev: It shifts the focus toward developing those advanced reasoning-driven VLA models that can truly adapt their plans in unpredictable situations.
Rosa: Next up, we're going to look at how this benchmark compares to other testing methods and see what these results tell us about the current state of embodied AI.
The paper's improvements: Taro: So, moving on from just showing us where the models fail, what are these authors proposing to fix it?
Rosa: They’re really focusing on how to make the task generation pipeline more robust and scalable so we can generate even more varied and challenging reasoning tasks easily.
Dev: Right, they're suggesting refining that multi-agent system—the seed generator, the mutator, the verifier—to ensure it produces high-quality data consistently.
Taro: I think they’re emphasizing that the verification agent needs to be really smart about checking for feasibility and necessity of reasoning before it spits out a task.
Rosa: Exactly, they want to make sure we aren't just getting random junk, but tasks where the robot actually *needs* to use complex reasoning skills like assembly or material interaction.
Dev: And they’re pushing for more sophisticated mutation strategies, building on pivot and trap ideas, but perhaps integrating them in a way that forces strategy adaptation instead of just adding noise.
Taro: They're hinting that the next step is moving beyond just making things harder to actively designing the difficulty curve based on where the model is currently struggling.
Rosa: It suggests a future where we don't just test if a model can pass one difficult task, but if it can maintain performance across a whole spectrum of reasoning complexity.
Dev: And that directly impacts how we build safety cases for real-world applications; if the benchmark is better at simulating unexpected failures, our safety guarantees become stronger.
Taro: It’s about moving from just reactive recovery to proactive strategy revision before things get too messy in the physical world.
Rosa: So, the implication here is that future work needs to focus on designing these agents so they can create scenarios that are truly representative of unpredictable real-world interactions.
Dev: And if we can get a better benchmark like this, it gives us a solid foundation for training AI models that aren't just memorizing solutions but actually learning how to solve problems creatively.
Taro: It sets a high bar for what success looks like in embodied AI—systems that can handle the unknown gracefully.
Rosa: So, we’re moving from simple manipulation tests to testing true cognitive adaptation, and that’s a big step for robotics in the real world.
Conclusion: Tom: So, to wrap up our talk on "RoboWits: Unexpected Challenges for Robotic Creative Problem Solving," we’ve seen how this benchmark systematically tests reasoning and tool use under tough conditions.
Rosa: We looked at how they build these two hundred eight mutated tasks from the initial seed set, and it really shows that a lot of the difficulty comes from subtle changes in the scene or object list.
Dev: Right, I'm thinking about that mutation pipeline; it’s pretty clever how they automate generating those difficult scenarios so we don't have to manually design every failure mode.
Taro: From an autonomy side, I see this as a way to stress test the system beyond simple success or failure and into genuine strategy adaptation when the world throws curveballs.
Rosa: It definitely changes how we view testing; instead of just checking if a robot can follow instructions, we're checking if it can think its way out of a broken situation.
Dev: For control engineers, the safety case gets stronger because this benchmark forces us to consider failure modes that aren't in the standard training set.
Taro: I think the biggest shift is realizing that models need to be trained not just for performance on known tasks, but for robustness when things are deceptive or constrained.
Rosa: It points toward a future where we prioritize reasoning-driven systems over just high-speed control loops in these complex physical environments.
Dev: So, this benchmark gives us a rigorous way to measure that gap between basic manipulation and true cognitive flexibility in robotic platforms.
Taro: It's a great tool for figuring out what’s actually missing from current VLA models when they try to handle things outside the training data.
Rosa: That’s where we’ll head next, because now that we know the testing ground, we need to look at how other papers are trying to tackle these reasoning gaps with different approaches.
Chunru Lin, Hongxin Zhang, Fenghao Yu, Zhehuan Chen, Thomas L. Griffiths, Yejin Choi, David Held, Chuang Gan
University of Massachusetts Amherst University of Princeton Stanford University Carnegie Mellon University
cs.RO, cs.AI
Submitted: 2026-05-28
Updated: 2026-11-09
Comments: The first two authors contributed equally
Project page: https://umass-embodied-agi.github.io/RoboWits/Preprint
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 100/100
The gist: The ability to reason, adapt, and creatively solve problems under unexpected challenges is essential for robots operating in real-world environments.
Key concepts
- RoboWits
- A new robotic benchmark featuring bi-manual manipulation tasks that require geometric, material, and assembly reasoning. It systematically evaluates how well robotic models can reason creatively and adapt to unexpected challenges.
- Automated Task Generation Pipeline
- A multi-agent framework where specialized AI agents collaboratively design tasks. This pipeline creates 30 seed tasks and expands them into hundreds of diverse, graded mutations using strategies like Pivot, Trap, and Add to ensure high-quality evaluation data.
- VLA Brittleness
- The finding that pre-trained Vision-Language Agents (VLAs) perform well on simple 'seed' tasks but fail significantly when facing mutated versions. This indicates they lack the necessary reasoning and strategy adaptation skills needed for complex, unexpected manipulation scenarios.
Terminology
Summary
The ability to reason, adapt, and creatively solve problems under unexpected challenges is essential for robots operating in real-world environments. p. 0 RoboWits introduces a bi-manual robotic benchmark designed to systematically evaluate cognitive reasoning, creative tool use, and robustness to unexpected conditions. p. 0
Pruning redundant reasoning tokens
RoboWits is a new bi-manual robotic benchmark that explicitly targets evaluating cognitive reasoning, creative tool use, and robustness to unexpected challenges. The benchmark consists of manipulation tasks that require geometric, material, and assembly-based reasoning. p. 1 These tasks are organized across multiple difficulty levels, enabling fine-grained evaluation of how robotic models scale with increasing reasoning complexity. p. 1 To produce diverse, high-quality tasks at scale, the authors propose an automated task generation pipeline formulated as a multi-agent cooperative framework. p. 1 This pipeline decomposes task construction into specialized agents with different roles: a seed task generator that proposes cognitively challenging task specifications, a task mutation generator that expands each seed into diverse task mutations where original solution are blocked by small scene changes, a task metric generator that produces executable evaluation criteria, a task verifier that ensures feasibility, simulatability, and necessity of reasoning, and a scene generation agent. p. 1
How the training signal is built
The automated pipeline generates 30 seed tasks spanning geometry, material, and assembly-based reasoning. These seed tasks are expanded into 208 tasks with graded difficulty and full evaluation code. p. 1 The mutation strategies include Pivot, Trap, and Add mutations, which are designed to increase task diversity and difficulty. p. 5 The Pivot strategy blocks the current solution by removing or modifying key objects in the object list and introducing new objects that enable alternative valid solutions. p. 5 The Trap strategy increases both reasoning and execution difficulty by injecting trap objects
into the scene, which are designed to appear useful but are functionally ineffective due to subtle constraints. The Add strategy increases scene complexity by adding useless objects as distractors, which introduce clutter and perceptual noise. p. 5
Where the gains come from
The benchmark reveals a significant performance gap where pre-trained VLAs struggle to perform on mutated tasks, implying their brittleness in manipulation tasks requiring reasoning, strategy adaptation, and robustness to deceptive or constrained environments. While pre-trained VLAs exhibit preliminary success on seed tasks after single-task fine-tuning, they struggle to perform on mutated tasks. p. 0 VLM Planner achieves reasonable performance on seed tasks but fails to generalize effectively to mutated task variants. p. 1 The results confirm that current VLAs cannot perform robustly when faced with unexpected reasoning-level challenges. p. 7
Failure Analysis across Mutation Types
Analysis of success rates across mutation types shows that for Pivot, models show no strategy transfer, physically failing by stubbornly attempting to manipulate immovable objects or continuing original trajectories even when objects are accidentally displaced. For Trap and Add mutations, success rates drop as more objects are introduced, with Add(1/2) SR > Add(3) SR. p. 5 The physical errors for Trap and Add are primarily driven by distraction, where models may physically freeze due to out-of-distribution visual data or incorrectly reach distractor objects. p. 5
Demonstration Scaling
Training the VLM Controller with 50 demonstrations improves Seed performance from 70% to 94% on the Dominos task, but performance on Mutation tasks remains low. This confirms that current VLAs cannot perform robustly when faced with unexpected reasoning-level challenges. p. 7 The primary bottleneck is indeed poor reasoning rather than control limitations. p. 2 The paper concludes that while fine-tuning improves basic manipulation skills, substantial gaps remain in reasoning, strategy adaptation, and robustness under deceptive or constrained conditions. p. 8
Limitations
As a simulation-based benchmark, this work is limited by the sim-to-real gap. Additionally, the reliance on highly realistic physical interactions yields a computational trade-off: non-linear dynamics of soft-body simulations are computationally expensive. The paper also notes that careful deployment and human oversight are necessary to mitigate risks associated with foundation models producing incorrect or biased outputs.
Potential Social Impacts
Positive impacts include advancing the study of reasoning-driven robotic manipulation by introducing RoboWits, which may contribute to safer and more reliable robotic systems for real-world applications. The automated task generation pipeline also reduces the cost of manual benchmark design and supports scalable, reproducible evaluation. p. 8 Negative impacts include potential labor displacement in domains relying on repetitive manual work and the misuse of stronger robotic tool-use abilities in adversarial settings. p. 2 Careful deployment and continuous auditing are necessary to mitigate risks from foundation models producing incorrect or biased outputs. p. 5
Conclusion
RoboWits offers a rigorous framework for quantifying the gap between low-level manipulation proficiency and high-level cognitive adaptation. The authors hope RoboWits will serve as a scalable testbed for developing and diagnosing next-generation reasoning-driven VLA models and robust robotic problem-solving systems. p. 8 The paper presents RoboWits, a new bi-manual manipulation benchmark designed to evaluate creative problem solving, tool use, and robustness to unexpected challenges. p. 8 The authors propose an automated multi-agent pipeline that jointly performs seed task creation, task diversification, verification, scene instantiation, and metric synthesis. p. 8 This enables diverse reasoning-centric tasks with ground-truth evaluation. p. 3 The paper is arXiv:2605. p. 030326v1 [cs. p. 0RO] 28 May 2026
References
The paper references a wide range of related works in robotic manipulation and creative problem solving, including RLBench [19], LIBERO [28], RoboTwin 2.0 [6], RoboEval [45], VLABench [53], MacGyver [38]. p. 2 It also cites foundational work on physics engines like Genesis [2] and various VLA models such as π0, π0. p. 75, ACT, and OpenVLA. p. 9 The authors build upon prior research in task diversification through domain randomization [19], scene generation via procedural generation with hand-crafted rules [11], image-driven pipelines that leverage 2D priors to construct 3D layouts [16], and LLM-based methods for symbolic constraints. p. 2 The work is situated within the context of creative problem solving, which has been studied in NLP and cognitive science for multi-step planning and strategy revision. p. 2 The benchmark is positioned to target capabilities that existing benchmarks do not systematically isolate, focusing on reasoning-centric bi-manual tasks. p. 2 The authors introduce a unified multi-agent pipeline for task creation, diversification, verification, scene instantiation, and metric synthesis. p. 3 The benchmark is instantiated in Genesis [2], a unified physics simulator that supports diverse materials and robot control. p. 6 The authors collect and release human teleoperation demonstrations for 10 of the seed tasks to facilitate benchmarking. p. 6 The paper's contribution is summarized by three main points: introducing RoboWits, proposing an automated multi-agent task generation pipeline, and benchmarking existing robotic policy methods. The authors also detail the specific constraints and prompt templates used for the agents, such as for the Seed Task Generator and Task Mutator. p. 3 p. 3 The benchmark task schema standardizes task specification and evaluation-relevant fields while allowing substantial variation in task content, scene composition, and difficulty. p. 4 p. 4 The authors provide detailed examples of mutation trees for the retrieve cube seed task to illustrate how different mutation strategies generate diverse task variants. p. 5 p. 5 The VLM Controller prompt template details the available primitives and coordinate system for robot action planning. p. 7 p. 7 The Metric Generator prompt template specifies the required output format for generating executable evaluation code. p. 6 p. 6 The Scene Generator prompt template outlines the workflow for scene instantiation using Genesis Physics Engine and asset library tools. p. 8 p. 8 The final results show that pre-trained VLAs leverage prior knowledge to outperform from-scratch models in low-data settings, yet still exhibit substantial difficulty with tasks requiring complex material interactions and assembly-level reasoning. p. 1 p. 1 The paper's empirical results show that VLM Planner achieves reasonable performance on seed tasks and demonstrates superior robustness to mutations when using closed-loop planning. The authors emphasize that real-time feedback is critical for adapting strategies to the unexpected challenges inherent in their benchmark. p. 8 p. 8 The paper's contribution is summarized by three main points: introducing RoboWits, proposing an automated multi-agent task generation pipeline, and benchmarking existing robotic policy methods. The authors also detail the specific constraints and prompt templates used for the agents, such as for the Seed Task Generator and Task Mutator. The benchmark task schema standardizes task specification and evaluation-relevant fields while allowing substantial variation in task content, scene composition, and difficulty. The authors provide detailed examples of mutation trees for the retrieve cube seed task to illustrate how different mutation strategies generate diverse task variants. The VLM Controller prompt template details the available primitives and coordinate system for robot action planning. The Metric Generator prompt template specifies the required output format for generating executable evaluation code. The Scene Generator prompt template outlines the workflow for scene instantiation using Genesis Physics Engine and asset library tools. The final results show that pre-trained VLAs leverage prior knowledge to outperform from-scratch models in low-data settings, yet still exhibit substantial difficulty with tasks requiring complex material interactions and assembly-level reasoning. The paper's empirical results show that VLM Planner achieves reasonable performance on seed tasks and demonstrates superior robustness to mutations when using closed-loop planning. The authors emphasize that real-time feedback is critical for adapting strategies to the unexpected challenges inherent in their benchmark.
--- Page 1 ---
RoboWits: Unexpected Challenges for Robotic Creative Problem Solving
Chunru Lin1∗, Hongxin Zhang1∗, Fenghao Yu1, Zhehuan Chen1 Thomas L. Griffiths2, Yejin Choi3 David Held4 Chuang Gan1 1 University of Massachusetts Amherst 2 Princeton University 3 Stanford University 4 Carnegie Mellon University. p. 0
--- Page 2 ---
Instruction: Retrieve the cube from the container and place it on the green target area. Pick the cube… Pick the cube… Pick the cube… Narrow open, can’t pick… Grasp the cup and pour out the cube… Pick the cube… Pick the cube… Failed to grasp, the cup may be fixed… Pour water to flush the cube up… Pick the cube…? p. 1 Narrow open, can’t pick…, Grasp the cup and pour out. p. 1
--- Page 3 ---
Figure 1: Creative problem solving under unexpected challenges. The figure contrasts the reasoning and execution processes of a standard robot (left) and an ideal robot (right) performing tasks under the same instruction but escalating difficulty. p. 1
--- Page 4 ---
Table 1: Benchmark Comparison. RoboWits features an automatic generation pipeline and provides systematic support for tasks involving diverse materials (fluids, soft bodies), creative tool use, and strategy adaptation. p. 2
--- Page 5 ---
Figure 2: Overview of our automated task generation pipeline, where foundation model-powered agents (depicted by role-based avatars) collaborate to design, validate and instantiate tasks.
--- Page 6 ---
Figure 4: Illustration of mutation on the retrieve cube seed task. Starting from the initial seed task, the mutator generates a family of tasks with distinct challenges and graded difficulty. p. 5
--- Page 7 ---
Table 3: Results on 10 Seed Tasks and their Mutations on RoboWits. Rows are split by Method and Task Type (Seed/Mutation). p. 7
--- Page 8 ---
Figure 7: Comparison of VLM planner∗∗ SRs under closed-loop and open-loop setting.
--- Page 9 ---
Figure 8: Ego-centric view of the 30 seed tasks in RoboWits.
--- Page 10 ---
Table 4: Coverage statistics of our benchmark tasks.
--- Page 11 ---
Table 5: Comparison of VLM Planner performance using GPT-4o and Gemini 3 Pro.
--- Page 12 ---
Figure 7: Comparison of VLM planner∗∗ SRs under closed-loop and open-loop setting.
--- Page 13 ---
Figure 8: Ego-centric view of the 30 seed tasks in RoboWits.
--- Page 14 ---
Table 7: Effect of increasing the number of demonstrations on two representative tasks.
--- Page 15 ---
Figure 9: Task mutation trees for retrieve cube, gap retrieve, and roll up box.
--- Page 16 ---
Table 7: Effect of increasing the number of demonstrations on two representative tasks.
--- Page 17 ---
Figure 9: Task mutation trees for retrieve cube, gap retrieve, and roll up box.
--- Page 18 ---
Table 7: Effect of increasing the number of demonstrations on two representative tasks.
--- Page 19 ---
Table 7: Effect of increasing the number of demonstrations on two representative tasks.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Automated task generation pipeline for scalable construction of reasoning-centric manipulation tasks enables scalable generation of diverse reasoning-centric bi-manual tasks with diverse solution spaces, varying cognitive difficulty, and ground-truth evaluation.
-
Benchmarking popular robot policies, pre-trained VLAs, and oracle-state planners on RoboWits reveals a significant performance gap where pre-trained VLAs struggle to perform on mutated tasks, implying their brittleness in manipulation tasks requiring reasoning and strategy adaptation.
-
VLM-based modular planners achieve reasonable performance on seed tasks but fail to generalize effectively to mutated task variants, confirming that
real-time feedback is critical for adapting strategies
as demonstrated by the superior performance of closed-loop planning over open-loop baselines on mutations. -
The benchmark allows for fine-grained evaluation of how robotic models scale with increasing reasoning complexity across geometry, material, and assembly-based reasoning through graded difficulty levels (scores 1 to 5).
Sources
- RT-1: Robotics Transformer for Real-World Control at Scale
- RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
- Benchmarking Generalizable Bimanual Manipulation: RoboTwin Dual-Arm Collaboration Challenge at CVPR 2025 MEIS Workshop
- BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark
- Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks
- Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation
- RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning
- ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills
- MesaTask: Towards Task-Driven Tabletop Scene Generation via 3D Spatial Reasoning
- 3D Diffuser Actor: Policy Diffusion with 3D Scene Representations
- OpenVLA: An Open-Source Vision-Language-Action Model
- MolmoAct: Action Reasoning Models that can Reason in Space
- CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation
- Evaluating Real-World Robot Manipulation Policies in Simulation
- UBSoft: A Simulation Platform for Robotic Skill Learning in Unbounded Soft Environments
- RobotSmith: Generative Robotic Tool Design for Acquisition of Complex Manipulation Skills
- RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation
- Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning
- RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots
- THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation