Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: I'm Rosa.
Dev: And I'm Dev, here with Taro.
Rosa: We're looking at the paper: "Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration".
Dev: Here's the gist of the paper. Dexterous robot manipulation can benefit from the abundance of human demonstrations, but transferring such demonstrations to robot policies remains challenging.
Rosa: The key idea is object-centric contact wrench space guidance: we represent human and robot motions by the forces and torques they can induce on the object, enabling similarity to be measured by the induced instantaneous motions.
Dev: This guidance makes reinforcement learning more scalable for contact-rich dexterous manipulation.
Rosa: Framework and Benchmark CHORD is a framework for long-horizon manipulation of rigid and articulated objects with reinforcement learning.
Paper summary: Rosa: So we're looking at this paper called "Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration". The main thing is that you can use a lot of human demonstrations to teach a robot how to manipulate things dexterously, but transferring those demonstrations to the robot's own policy is really hard.
Dev: Yeah, it’s about making sure the robot learns something transferable instead of just copying the exact movements.
Taro: It seems like the core idea they are pushing is using contact wrench space guidance. Instead of just looking at where a human hand is in space, they represent both human and robot motions by all the forces and torques they can actually apply to the object.
Rosa: Exactly. The paper claims that this approach lets them measure similarity not by comparing hand positions directly, but by looking at how the induced instantaneous motions are similar between the human and the robot. This is supposed to make reinforcement learning much more scalable for complex manipulation tasks where contact is involved.
Dev: It sounds like they are trying to move away from brittle assumptions about how a demonstration should be transferred, which is a big problem in existing methods
seven–eleven: <ref:2607.00033#pg2>. They want a way to learn transferable knowledge without needing perfectly aligned human-robot data for every single task and object.
Taro: And the framework they introduced is called CHORD. It’s designed for long-horizon manipulation of both rigid and articulated objects, and it does this without needing specific robot demonstrations for every task.
Rosa: They built a large benchmark of four thousand seven hundred thirty-nine bimanual dexterous manipulation tasks from motion capture data that they reconstructed into videos themselves. They claim across those one thousand eight hundred thirty-one tasks, CHORD gets an average success rate of eighty-two point twelve percent.
Dev: That success rate sounds pretty solid for such a large set of complex interactions. But how do they actually structure the learning signal to achieve that? It’s not just one reward type, right?
Taro: No, it's a combination of three rewards: task-tracking, motion-imitation, and contact-guidance. The task reward pushes the robot to follow the reference object motion and the inter-part relations.
Rosa: And then there’s the imitation reward which regularizes the robot toward that retargeted human motion they see in a demonstration. But that's where the contact wrench guidance comes in, which compares their support function against the human reference function with some relative tolerance to encourage physically aligned contacts.
Paper summary: Dev: So, they are using this contact wrench reward to enforce physical alignment—making sure the robot applies forces and torques in a way that matches what a human would apply to achieve the desired outcome. But I always wonder if just comparing wrenches is enough for general manipulation.
Taro: They address that by showing how they generate task-relevant disturbances by sampling wrenches from the human contact wrench matrix, which creates disturbances aligned with demonstrated contact mechanics. Plus, when their contact estimates are noisy, they pivot from direct matching to a reduced objective that rewards the robot for generating positive support along each wrench space basis direction.
Rosa: That pivot sounds like a smart way to handle uncertainty in the real world. It shows robustness when things aren't perfectly clean or perfectly measured. How does this all translate to using it outside of the simulation?
Dev: They extend CHORD naturally to whole-body manipulation, allowing for either hand-only or whole-body references. For hand-only references, they use an inpainting module to predict the full body motion from just the end effector trajectories.
Taro: And for whole-body references, they switch to a reduced force closure objective while training the reinforcement learning policy. This allows them to generalize these learned policies out into real-world settings, whether it’s open-loop or closed-loop execution.
Rosa: The evaluation shows that CHORD consistently outperforms prior methods across all metrics when compared against state-of-the-art dexterous manipulation techniques. They also report a strong correlation between the contact wrench reward and task success, with a Pearson r of about zero point eighty across all datasets.
Dev: A zero point eight zero correlation with task success is pretty high, especially considering how complex these tasks are—long horizons and dense contact events. It suggests that this way of looking at contact space really captures the physics needed for success.
Taro: They also maintain high tracking accuracy even over very long interaction horizons, keeping near saturated tracking accuracy even on the longest ones. And they’ve shown success in real-world deployment on a Dexmate robot that has two Sharpa dexterous hands.
Rosa: So, to wrap up this paper "Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration", it’s about using contact wrench guidance to bridge the gap between human demonstrations and robot policies. The authors Xinghao Zhu and colleagues show that by comparing forces and torques instead of just positions, you can create a more scalable learning signal for dexterous manipulation.
Paper summary: Dev: It really matters because it suggests that we don't need perfect hand-to-hand data for every single thing a robot needs to do; we just need a way to understand the underlying physical interactions. This moves away from brittle assumptions about how demonstrations should be transferred
seven–eleven: <ref:2607.00033#pg2>.
Taro: The implication is that we can build systems that are more robust because they learn to handle noisy contacts and varying geometries by focusing on the physics of those contacts, not just the trajectory. It shows a way to extend manipulation from curated lab settings to actual real-world deployment.
Rosa: What this means for us outside of this paper is that for field robotics, we need methods that don't break when the world misbehaves. This approach seems to give robots a better physical intuition about what it means to interact with an object through contact wrench space, which should improve their performance in unpredictable environments.
Dev: The limitation they mention is that the method still relies on state-based observations for real-world deployment, and they also noted that the human demonstrations need to be relatively clean. So, we aren't seeing a fully autonomous system yet that can handle completely messy, low-quality data out of the box.
Taro: And they admit that the evaluation metric for object pose error isn't perfect either. So while it’s strong in simulation and on benchmarks, translating that high success rate to a truly robust real-world scenario still has hurdles to overcome.
Rosa: So, "Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration" offers a solid framework using physical interaction metrics to make learning more scalable. It shows how we can leverage human expertise without needing perfectly matched robot bodies or perfect data for every single task.
Dev: It’s a step in the right direction by focusing on the physical space of contact, which seems like a necessary shift from just matching visual coordinates to understanding actual physical interaction capabilities. That's where the real engineering challenge lies going forward.
Taro: And for me, it means we can start designing policies that are inherently more resilient because they are explicitly guided by wrench physics rather than just following a sequence of coordinates twelvetwenty-fivethirty-onethirty-two <ref:2607.00033#pg3>.
Rosa: So if you’re driving or cooking, this kind of thinking is what allows the robot to handle the messy reality of physical interaction without needing a perfect map beforehand. It’s about understanding the forces involved.
Conclusion: Rosa: So we're talking about this paper, "Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration."
Dev: Yeah, it’s about how you can use human demonstrations to teach a robot to manipulate things better without needing specific robot training data for every single object and task.
Rosa: Basically, the authors are showing that instead of just copying the exact movements a human makes, which is really hard when things get complicated, you can focus on what forces are actually being used during contact.
Dev: They propose this idea of object-centric contact wrench space guidance, where both the human and robot motions are represented by all the forces and torques they can apply to the object.
Rosa: That means measuring similarity not by comparing hand positions in space, but by how similar the resulting physical interactions are—the induced instantaneous motions.
Dev: The big takeaway is that this method makes reinforcement learning much more scalable for those contact-rich tasks we deal with in the real world.
Rosa: They built a large simulation benchmark of nearly five thousand bimanual manipulation tasks, and they claim their framework, CHORD, hits about eighty-two point twelve percent success on that set.
Dev: That number sounds strong for something involving so much contact and long interaction horizons. But it’s not just one reward type driving the learning signal here.
Rosa: Right, they use a mix of task tracking, motion imitation, and this crucial contact guidance reward to keep the robot physically aligned with what a human would do.
Dev: That physical alignment is key; they are rewarding the robot for applying support forces in a way that matches the human's demonstrated mechanics.
Rosa: They also have these clever tricks during training, like resetting the simulator to arbitrary points along the reference path and then using wrenches sampled from that human contact matrix to create relevant disturbances.
Dev: And when their contact estimates get noisy, they actually pivot away from direct matching and reward generating positive support in each direction of the wrench space basis.
Rosa: So what this means for us is that we can start designing policies that are inherently more resilient because they are guided by physics rather than just following a sequence of coordinates.
Dev: It suggests we don't need perfect hand-to-hand data for every single thing a robot needs to do; we just need a way to understand the underlying physical interactions.
Rosa: And they’ve even shown it works in the real world on a two-handed Dexmate robot, which is pretty cool for getting those results outside of pure simulation.
Dev: It's a step in the right direction by focusing on the physical space of contact, which seems like a necessary shift from just matching visual coordinates to understanding actual physical interaction capabilities.
Rosa: They still admit they need relatively clean human demonstrations and that object pose error isn't the perfect way to measure success, which is a limitation we have to keep in mind.
Dev: Exactly; so while the simulation results are solid, translating that high success rate into a totally messy real-world scenario still has hurdles ahead.
NVIDIA
cs.RO, cs.AI, cs.CV
Submitted: 2026-06-22
Updated: 2026-11-10
Project page: https://nvidia-isaac.github.io/video_to_data/chord
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 90/100
The gist: Dexterous robot manipulation can benefit from the abundance of human demonstrations, but transferring such demonstrations to robot policies remains challenging.
Key concepts
- Object-centric Contact Wrench Space Guidance
- This is a key idea where human and robot movements are described not by joint angles, but by the forces and torques they exert on an object. By focusing on this shared space of forces (wrench space), the system can measure similarity between demonstrations based on how these induced motions compare, regardless of the specific robot or hand configuration.
- CHORD Framework
- CHORD is a reinforcement learning framework designed for long-horizon manipulation of rigid and articulated objects. It allows robots to learn complex tasks without needing task-specific robot demonstrations, achieving success across diverse manipulation settings through a scalable training approach.
- Contact Wrench Space Reward
- This reward component in the training signal compares the robot's support function against the human reference function within their respective wrench spaces. This encourages the robot to generate physically aligned contacts by rewarding it for producing positive support along each basis direction of that space.
Terminology
Summary
Dexterous robot manipulation can benefit from the abundance of human demonstrations, but transferring such demonstrations to robot policies remains challenging. p. 0 Background shows CHORD on our large-scale benchmark of
bimanual manipulation tasks. Abstract: Dexterous robot manipulation can benefit from the abundance of human demonstrations, but
transferring such demonstrations to robot policies remains challenging. We present Contact Wrench Guidance
from Human Demonstration in Robotic Dexterous Manipulation (CHORD), a framework for long-horizon
manipulation of rigid and articulated objects with reinforcement learning. The key idea is object-centric
contact wrench space guidance: we represent human and robot motions by the forces and torques they can
induce on the object, enabling similarity to be measured by the induced instantaneous motions. The key idea is object-centric
contact wrench space guidance: we represent human and robot motions by the forces and torques they can
induce on the object, enabling similarity to be measured by the induced instantaneous motions. This guidance
makes reinforcement learning more scalable for contact-rich dexterous manipulation.Where the paper says it
Where the paper says it
Where the paper says it
Framework and Benchmark
CHORD is a framework for long-horizon manipulation of rigid and articulated objects with reinforcement learning. The method enables long-horizon RL across diverse rigid and articulated manipulation tasks without task-specific robot demonstrations. p. 1 Given a human reference, CHORD trains a robot policy to match the demonstrated object-state
evolution, using contact wrench space rewards to encourage physically aligned contacts. Our method enables
long-horizon RL across diverse rigid and articulated manipulation tasks without task-specific robot demonstrations. This guidance
makes reinforcement learning more scalable for contact-rich dexterous manipulation. We further introduce
a large-scale simulation benchmark with 4,739 bimanual dexterous manipulation tasks, constructed from
motion-capture datasets and reconstructed in-house videos. We further introduce a large-scale simulation benchmark of 4,739 bimanual dexterous manipulation
tasks, built from open-source motion-capture datasets [13–16] and reconstructed in-house videos. Across
1,831 tasks, CHORD achieves an average success rate of 82. Evaluated on 1,831 benchmark tasks, CHORD
achieves an average success rate of 82. 12%, demonstrating strong scalability.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
How the training signal is built
The reinforcement learning rewards are formulated as a combination of task-tracking, motion-imitation, and contact-guidance rewards: r = rtask + rimit + rcontact. The task reward encourages the rollout to follow the reference object motion, including both object-part poses and inter-part relations. p. 3 RL with Wrench Space Contact Guidance
We train the policy with task-tracking, motion-imitation, and contact-guidance rewards, 𝑟 = 𝑟task + 𝑟imit +
𝑟contact (Figure 2) with the VOC [1]. The task reward encourages the rollout to follow the reference object
motion, including both object-part poses and inter-part relations: 𝑟task = exp (︂
−
∑︀𝐾
𝑘=1
‖𝑥
object,𝑘
𝑡 ⊖𝑠
object,𝑘
𝑡
‖
2
2
varobj )︂
+
𝑟relative, where ⊖ denotes the pose difference in SE(3). We therefore add a relative reward, 𝑟relative = 𝑚(𝑡) exp (︁
−
𝑒object
varrel )︁
, where 𝑚(𝑡)
activates the term only during object-object interaction phases, and 𝑒object measures the inter-object pose error. The imitation reward regularizes the robot toward the retargeted human motion, 𝑟imit = exp (︁
−
‖𝑥
robot
𝑡 −𝑠
robot
𝑡 ‖
2
2
varimit )︁
. We compute the
robot support function 𝜎r,𝑘 in the same way. We then define the contact wrench-space reward by comparing the robot support function 𝜎r,𝑘 against the
human reference function 𝜎h,𝑘 with a relative tolerance 𝛽.Where the paper says it
Where the paper says it
Where the paper says it
Where the gains come from
CHORD's success is attributed to several mechanisms that improve training efficiency and robustness. These include:
-
Resetting the simulator to arbitrary states along the reference trajectory and keeping the VOC fully active for a short stabilization window. p. 4
Where the paper says it
1, we incorporate several mechanisms to improve training efficiency and robustness. First, we reset the simulator to arbitrary states along the reference trajectory and keep the VOC fully active for a short stabilization window, allowing the robot to recover contact before assistance is annealed.
-
Perturbing object parts with wrenches sampled from the human contact wrench matrix to produce task-relevant disturbances aligned with demonstrated contact mechanics. p. 4
Where the paper says it
First, we reset the simulator to arbitrary states along the reference trajectory and keep the VOC fully active for a short stabilization window, allowing the robot to recover contact before assistance is annealed. Second, we perturb object parts with wrenches sampled from the human contact wrench matrix 𝒲h,𝑘, producing task-relevant disturbances aligned with the demonstrated contact mechanics.
-
Pivoting from direct matching to a reduced contact wrench guidance objective when contact estimates are noisy, which rewards the robot for generating positive support along each wrench-space basis direction. p. 4
Where the paper says it
When contact estimates are noisy due to hand-object misalignment, as in demonstrations reconstructed from RGB videos, directly matching human contact wrenches can be unreliable. In such scenarios we pivot from direct matching to a reduced contact wrench guidance objective that rewards the robot for generating positive support along each wrench-space basis direction: 𝑟 𝑘 fc = 1 𝐵 ∑︀𝐵 𝑏=1 1[𝜎𝑟,𝑘,𝑏 > 𝜖], where 𝜎𝑟,𝑘,𝑏 is the robot support value for object part 𝑘 along basis direction 𝑏, and 𝜖 is a small threshold.
Generalizability and Real-World Transfer
CHORD naturally extends to whole-body manipulation with either hand-only or whole-body references. For hand-only references, an inpainting module is trained to predict full-body motion from end-effector trajectories. p. 4 CHORD naturally extends to whole-body manipulation with either hand-only or whole-body references. For hand-only references, such as egocentric reconstructions, we train an inpainting module [37] to predict
full-body motion from end-effector trajectories, then apply the same RL formulation with contact wrench-space
rewards. (a–j) Use hand-only references, where full-body motion is completed
by an inpainting module, and RL training. (k–n) Use whole-body references, where the reduced force-closure
objective is used during RL training. CHORD also generalizes to
whole-body manipulation from hand-only and third-person demonstrations, achieving a 90. 77% success rate,
and the learned policies transfer to the real world in both open-loop and closed-loop settings.Where the paper says it
Where the paper says it
Where the paper says it
Benchmark Evaluation
The dexterity manipulation benchmark spans diverse, large-scale settings with long horizons and dense contact events, requiring dexterous contact strategies and coordinated manipulation. CHORD consistently outperforms prior methods across all metrics when compared against state-of-the-art dexterous manipulation methods. p. 6 We compare CHORD against state-of-the-art dexterous manipulation methods
that learn from human demonstrations, including ManipTrans [2], DexMachina [1], SPIDER [3], and Human2Sim2Robot [43]. We adopted standardized metrics (AUC from Dexmachina, SP-SR from SPIDER, MP-SR
from ManipTrans). Figure 6: Correlation between contact wrench reward and task success rate. Across all runs, we observe a strong positive association between the normalized CWS reward and task
8 Our method enables
long-horizon RL across diverse rigid and articulated manipulation tasks without task-specific robot demonstrations. We further introduce a large-scale simulation benchmark of 4,739 bimanual dexterous manipulation
tasks, built from open-source motion-capture datasets [13–16] and reconstructed in-house videos. For most sequences, it maintains near-saturated tracking accuracy
(ADD-AUC ≈ 0. 98) even on the longest horizons (approximately 40−48 seconds), whereas baseline
methods exhibit substantial degradation as the interaction horizon increases. We reason this is due to contact position rewards
being tightly coupled with the demonstrated embodiment, whereas wrench space alignment guides learning
independently of embodiment constraints. Evaluation on Real World
To evaluate real-world transfer, we deploy the learned policies on a Dexmate robot with two Sharpa dexterous
hands.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Limitations
Several limitations remain, including the reliance on state-based observations for real-world deployment, the requirement for relatively clean human demonstrations, and the imperfect nature of object pose error as an evaluation metric. Future work should address vision-based deployment, robustness to noisier demonstrations, and more task-aware success metrics. p. 1 77%
success in whole-body manipulation tasks, with successful deployment on real hardware. Related Work
Learning from human demonstration is an increasingly prominent approach for enabling robots to acquire skills
from existing experts [17], demonstrating success in learning whole-body control [18–20] and representation
learning [7–9, 21].Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Contact Wrench Guidance enables learning dexterous manipulation through object-centric contact wrench space guidance, which makes reinforcement learning more scalable for contact-rich dexterous manipulation. This allows policies to learn from human demonstrations without relying on task-specific robot demonstrations.
-
The system can perform long-horizon manipulation of rigid and articulated objects and generalize to whole-body embodiments from hand-only or third-person demonstrations. It achieves a 90.77% success rate in whole-body manipulation tasks, showing it
generalizes to whole-body manipulation from hand-only and third-person demonstrations.
-
The policy learns to match the demonstrated object state evolution by using contact wrench space rewards, which encourages
physically aligned contacts
rather than just matching 3D human contact locations. This is achieved by comparing robot support functions against human reference functions in wrench space. -
The system can handle noisy demonstrations, showing robustness to moderate hand-pose noise and demonstrating that the
wrench-space guidance remains robust to realistic levels of state estimation and tracking error.
-
The policy learns effective contact behaviors despite morphology differences between the human hand and the robot hand when generalizing to whole-body manipulation, as
wrench space alignment guides learning independently of embodiment constraints
compared to contact position rewards.
Abstract
Dexterous robot manipulation can benefit from the abundance of human demonstrations, but transferring such demonstrations to robot policies remains challenging. We present Contact Wrench Guidance from Human Demonstration in Robotic Dexterous Manipulation (CHORD), a framework for long-horizon manipulation of rigid and articulated objects with reinforcement learning. The key idea is object-centric contact wrench space guidance: we represent human and robot motions by the forces and torques they can induce on the object, enabling similarity to be measured by the induced instantaneous motions. This guidance makes reinforcement learning more scalable for contact-rich dexterous manipulation. We further introduce a large-scale simulation benchmark with 4,739 bimanual dexterous manipulation tasks, constructed from motion-capture datasets and reconstructed in-house videos. Evaluated on 1,831 benchmark tasks, CHORD achieves an average success rate of 82.12%, demonstrating strong scalability. CHORD also generalizes to whole-body manipulation from hand-only and third-person demonstrations, achieving a 90.77% success rate, and the learned policies transfer to the real world in both open-loop and closed-loop settings.
Sources
- SPIDER: Scalable Physics-Informed Dexterous Retargeting
- TWIST: Teleoperated Whole-Body Imitation System
- SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control
- R3M: A Universal Visual Representation for Robot Manipulation
- Object-Centric Dexterous Manipulation from Human Motion Data
- Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning
- Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstration
- SAM 3D: 3Dfy Anything in Images
- ViPE: Video Pose Engine for 3D Geometric Perception
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation