AutoDex: An Automated Real-World System for Dexterous Grasping Data Collection

summary

The gist

Learning robust dexterous grasping requires real-world data that records the physical outcomes of grasp attempts <ref:2606.23689#pg2> Such data are difficult to obtain at scale <ref:2606.23689#pg2> A…

This episode discusses

The paper

AutoDex: An Automated Real-World System for Dexterous Grasping Data Collection · Read on arXiv

Mingi Choi, Gunhee Kim, Jisoo Kim, Taeksoo Kim, Taeyun Ha, Jongbin Lim, Hanbyul Joo

Seoul National University

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "AutoDex: An Automated Real-World System for Dexterous Grasping Data Collection".

Dev: Here's the gist of the paper. Learning robust dexterous grasping requires real-world data that records the physical outcomes of grasp attempts <ref:2606.23689#pg2> Such data are difficult to obtain at scale <ref:2606.23689#pg2> A natural solution is to generate candidate grasps and verify them on real hardware, but this scales only if the entire collection loop—perception, execution, labeling, and reset—runs without human intervention <ref:2606.23689#pg2> We present AutoDex, an automated real-world data-collection system that performs this entire validation cycle without human intervention <ref:2606.23689#pg2> Given generated candidate grasps, it estimates the object’s initial 6D pose with dense 20-camera perception <ref:2606.23689#pg2>, executes selected candidates with collision-monitored robot motions <ref:2606.23689#pg2>, tracks the object during execution despite severe hand–object occlusion <ref:2606.23689#pg2>, labels lift-and-hold success or failure <ref:2606.23689#pg2>, and actively resets the object between trials to continue testing candidates across stable poses <ref:2606.23689#pg2> The result is a reusable database of physically labeled grasp trials that downstream systems can retrieve and filter for feasibility <ref:2606.23689#pg2> Pruning redundant reasoning tokens The paper introduces AutoDex, an end-to-end automated system designed to turn generated grasp candidates into physically labeled robot trials <ref:2606.23689#pg2> This system automates the entire validation cycle, which includes obtaining candidate grasps from a modular generator, executing them on physical hardware, and recording success or failure labels without human intervention <ref:2606.23689#pg2> The system is realized through four components: dense 20-camera perception for initial object pose estimation and execution-time tracking under hand–object occlusion <ref:2606.23689#pg2>, collision-monitored motion execution for safe unattended grasp attempts <ref:2606.23689#pg2>, physical success/failure labeling from lift-and-hold trials <ref:2606.23689#pg2>, and active object reset for continued testing across stable poses <ref:2606.23689#pg2> How the training signal is built The process begins with Grasp Candidate Generation, where AutoDex obtains candidate grasps from a modular candidate generator which takes an object model and a scene specification as input to return a set of kinematically and geometrically feasible candidates <ref:2606.23689#pg2> The generation uses BODex

8: to propose candidates based on scene primitives like wall, shelf, and box constraints <ref:2606.23689#pg2> Before real-robot execution, a simulation sanity pre-filter is applied where candidates are screened in MuJoCo

37: by applying a weak external force to reject those that translate more than 5 cm under perturbation <ref:2606.23689#pg2> The system selects an unattempted feasible candidate for the current stable pose and plans a collision-free arm trajectory to approach the candidate wrist pose <ref:2606.23689#pg2> Where the gains come from The primary throughput gain is demonstrated by comparing AutoDex with manual teleoperation, which requires 49.4 h for a matched 500-trajectory collection study compared to AutoDex's 10.3 h <ref:2606.23689#pg2> This yields a 4.8× throughput improvement <ref:2606.23689#pg2> Furthermore, grasps retrieved from the AutoDex-validated database achieve 76% real-world success, significantly higher than the 34% success rate for grasps selected without real-world validation <ref:2606.23689#pg2> This improvement is consistent across material, scene, and weight categories <ref:2606.23689#pg2> Impact of physical validation on dataset quality Physical validation substantially improves database quality by filtering out grasps that satisfy geometric constraints but fail under real-world contact effects such as slip, compliance, and multi-finger instability <ref:2606.23689#pg2> The AutoDex validated database achieves 76% grasp success when retrieved in deployment settings, compared to 34% for model-screened databases <ref:2606.23689#pg2> This indicates that kinematic and collision feasibility alone are insufficient for constructing a reliable dexterous-grasp database <ref:2606.23689#pg2> Reset Strategy Evaluation The system employs an explicit reset module to handle pose transitions between trials, which is critical because inter-trial reset cannot rely on passive settling alone <ref:2606.23689#pg2> The Stable Reorient Placement strategy maintains high success across the full difficulty range, including transitions with a passive transition probability of approximately 0 <ref:2606.23689#pg2> This method avoids the failure modes incompatible with unattended operation that occur in other reset strategies <ref:2606.23689#pg2> Impact of Camera Density on Pose Reliability Dense multi-view perception improves pose reliability for unattended collection because it provides redundant observations for robust initial pose estimation and helps recover object motion when the hand occludes much of the object <ref:2606.23689#pg2> The mean ADD-S between the full 20-camera reference pose and a k-camera subset poses decreases as camera count increases, showing that dense multi-view redundancy improves pose reliability for unattended collection <ref:2606.23689#pg2> This result indicates that dense multi-view redundancy improves pose reliability for unattended collection <ref:2606.23689#pg2> Discussion and Limitations AutoDex currently targets stable grasps for a fixed arm-hand setup, and the system also inherits the blind spots of its upstream generator which may miss grasps requiring dynamic finger motion during contact <ref:2606.23689#pg2> While dense multi-view perception improves pose reliability, the highest throughput setting still relies on a multi-camera workcell <ref:2606.23689#pg2> Bimanual coordination, mobile manipulation, finger-rolling regrasps, and functional grasps remain out of scope for the current system <ref:2606.23689#pg2> References

1: Y.

Rosa: Liu, Y.

Dev: Yang, Y.

Rosa: Wang, X.

Paper summary: Rosa: So, to recap, AutoDex tackles the problem of needing real physical outcome data for grasping by creating an automated system that does the whole validation process without a human in the loop.

Dev: The thesis is that you can generate many grasp candidates and test them on hardware to get reliable labels, provided you automate the entire collection loop—perception, execution, labeling, and reset.

Rosa: They show how this works by taking generated candidates from a modular generator and running them through a whole pipeline.

Dev: This pipeline involves using dense twenty camera perception for the initial six dimensional pose estimation, executing those selected candidates with collision-monitored robot motions, and then labeling the lift-and-hold success or failure.

Taro: I’m interested in how they manage the part where a candidate might be geometrically fine but still fail because of real physics like slip or compliance.

Rosa: The paper points out that kinematic and collision feasibility checks only confirm if a grasp can be executed, but they don't certify whether the contact will stably support the object under real contact effects.

Dev: That’s where the physical validation becomes crucial; it filters out grasps that look good on paper but fail because of things like friction or multi-finger instability.

Rosa: They demonstrate this by comparing their validated database to a baseline set of grasps selected without real-world validation, showing a success rate jump from thirty-four percent up to seventy-six percent when retrieved in deployment settings.

Dev: That improvement is consistent across different materials, scene types, and object weights, which suggests the physical validation really improves the quality of the resulting dataset.

Conclusion: Rosa: So what does this whole AutoDex system actually mean for us beyond just having a better dataset?

Dev: It means we can scale up data collection far past what manual teleoperation allows, which is only about ten point three hours for five hundred trials.

Rosa: That throughput gain is significant; they compared AutoDex’s ten point three hours to forty-nine point four hours for the same study with human teleoperation.

Dev: But more importantly, it means that the data you get from this system is much more reliable than what you get from just running candidates through simulation and planning tools.

Taro: The authors acknowledge that while dense multi-view perception helps with pose reliability, there are still limitations, especially when the object is textureless or thin.

Rosa: They admit that kinematic and collision feasibility alone aren't enough to build a reliable database; you need that real-world contact physics validation.

Dev: It also notes that the system currently targets stable grasps for a fixed arm hand setup, and it doesn't cover more complex things like bimanual coordination or finger-rolling regrasps right now.

Rosa: The implication is that this system is a powerful tool for building a foundation of physically verified data, even if it stops short of covering every kind of advanced manipulation we could ever imagine.

More episodes

← Home