Dex-X: Learning Visual-Tactile Dexterous Manipulation From Human Videos with Simulated Interaction
DEX-X learns robot manipulation skills from human video demonstrations by using simulation and physical contact dynamics as a tactile feedback engine. It reconstructs human interactions into a simulator, trains an expert policy in simulation using simulated tactile forces, and distills this expert into a deployable visual-tactile controller. This framework enables zero-shot sim-to-real transfer for various grasping and tool-use tasks.