Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos

arXiv:2605.23762 · cs.RO · Submitted 2026-05-22 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos".

Title and authors: Rosa: So we're looking at this paper, "Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos." It’s tackling that big problem of teaching complex skills to humanoid robots by using human video demonstrations.

Dev: Right. We know translating human motion to a robot is tough because there are these massive morphological mismatches between a person and a machine. This paper proposes something called Direct Dynamic Retargeting, or DDR, which aims to cut out some of the messy steps in the old methods.

Taro: I'm curious how it cuts things out. In previous work, they found that standard geometric retargeting and indirect dynamic retargeting introduce a geometric bias that really restricts where you can even look for a solution.

Rosa: Exactly. They suggest that these intermediate kinematic projections in those old methods create this bias, which limits the search space and ends up biasing the final feasible solution they get. Direct Dynamic Retargeting is supposed to bypass that entirely by generating trajectories directly from the expert videos using a specific Model Predictive Control framework called CEM-based MPC.

Dev: So instead of going through those intermediate projections, they are formulating the whole problem directly in task space and using a sampling-based solver inside a physics simulator. This means they're optimizing for physical consistency right from the start, which should help with demonstration tracking accuracy.

Taro: I'm thinking about what that means when we talk about the math here. They define the optimization as finding a trajectory Q in the feasible set Fq0 that minimizes the distance to the reference demonstration, where this distance function includes terms for both spatial tracking and relative shape matching.

Rosa: That's a key detail because it shows they aren't just optimizing for how close the robot is to a point; they are optimizing over something that accounts for both where things are in space and how well the body shapes match up. They call this augmented distance, composed of terms Ep and El.

Dev: And they show that this approach outperforms geometric retargeting and indirect dynamic retargeting baselines on diverse agile motions, specifically showing better demonstration tracking accuracy and physical consistency compared to those methods.

Taro: That leads us to the real-world tests. They validated this framework using the Unitree H1-two humanoid, and it successfully executed target motions without needing any task-specific manual tuning from the operators <ref:2605.23762#pg1>.

Rosa: That zero-shot sim-to-real transfer result is pretty significant because it shows that DDR can work directly on physical hardware without extensive fine-tuning on the robot itself. It’s a big validation for how robust these references are.

Dev: And what I find interesting from the experimental setup is that DDR maintains near perfect feasibility across most tasks, significantly outperforming GR and IDR in terms of both feasibility and contact sequence accuracy. That superior reference quality translates directly into higher reward for the downstream reinforcement learning agents they train next.

Taro: So when you look at it from a failure mode perspective, the paper points out that this enhanced reference quality actually makes the subsequent imitation learning policies converge faster and behave more robustly when dealing with diverse balancing tasks compared to policies trained on those other methods.

Rosa: It sounds like they're not just getting better references; they're setting up a much smoother training environment for the robots, which is crucial for complex skills. So, we’ve seen how this Direct Dynamic Retargeting framework works and where it stands against the existing techniques.

Dev: And we need to keep an eye on that limitation they mention: because the objective is defined purely in task space using that Laplacian graph distance, there’s a risk of getting ill-conditioned solutions or slower convergence if the initial setup isn't perfect.

Taro: That makes sense for a complex system; if the mathematical formulation itself can lead to slow convergence under certain conditions, we need to know what those conditions are before we deploy this broadly.

Rosa: So, to wrap up on this paper, Direct Dynamic Retargeting is presented as a novel framework that generates high-quality, dynamically feasible reference trajectories directly from expert videos by formulating the problem in task space using a Laplacian graph distance.

Dev: It achieves its gains by bypassing the intermediate geometric bias inherent in previous methods, leading to superior tracking accuracy and physical consistency across various motions.

Taro: The success on the Unitree H1-two with zero-shot sim-to-real transfer shows that this isn't just a theoretical improvement but something that translates well to real physical deployment <ref:2605.23762#pg1>.

Rosa: Ultimately, these higher fidelity references directly lead to downstream gains, where reinforcement learning agents trained on DDR trajectories converge faster and show more robust behavior across diverse balancing tasks.

Dev: That said, the authors flag that because the objective is defined purely in task space, it can sometimes lead to ill-conditioned solutions or slower convergence in those specific scenarios.

Taro: So for future work, they plan to scale this framework up to handle much larger and more diverse sets of motion variables coming from real-world web videos.

Rosa: That’s the direction they’re heading—moving from controlled expert data to handling the vast, messy reality of in-the-wild video demonstrations. We'll keep an eye on how that scaling goes.

The paper's summary: Rosa: So, to recap this paper, they’ve developed Direct Dynamic Retargeting or DDR, which is this new way to get high-quality movement references for teaching humanoid robots using just videos of people doing the task.

Dev: Right. The core idea is skipping those middle steps in older methods where you had to project the human motion into a robot shape, which was causing a kind of geometric bias that messed with what kind of movement the system could even search for.

Taro: It’s about formulating the whole problem right in the task space using something called a Laplacian graph distance, which means they’re optimizing based on how close you are to the goal in terms of task performance, not just how well your robot joints look compared to a person.

Rosa: Exactly. And they use this formulation inside a sampling-based solver that runs inside a physics simulator to make sure the resulting motion is actually physically possible, which is what Dev was worried about regarding stability and control loops.

Dev: It’s about ensuring the trajectory has physical viability by optimizing over that set of dynamically feasible motions while simultaneously minimizing the distance to that expert demonstration reference.

Taro: The way they build their optimization function includes terms for both tracking where things are in space and matching the relative shape between the human and the robot, which is what makes it more than just a simple spatial match.

Rosa: And when you look at what this means for real-world deployment, they’ve shown that DDR completely bypasses those geometric limitations we talked about earlier.

Dev: The result is that their references are much more physically consistent and accurate in tracking the actual demonstrations, which directly feeds into better learning outcomes for the robot.

Taro: For a researcher like me, what this means is that we can train downstream imitation learning agents on these references and they’ll converge faster because the training data itself is higher fidelity than what we could get from standard retargeting.

Rosa: That's a big deal for scaling up, because if we can generate better reference data this way, it makes teaching complex skills to robots much more efficient overall.

Dev: But there’s a caveat they mentioned, so keep an ear out: since the objective is defined purely in task space with that distance metric, there’s a risk that sometimes the mathematical setup can lead to solutions that are ill-conditioned or just take longer to converge than they should.

Taro: It means if you're trying to deploy this for something really high-stakes, you have to be careful because of those potential slowdowns under certain setups.

Rosa: So, it’s a powerful way to generate references by focusing on the task and physics simultaneously, but we still have that convergence speed issue to watch out for.

Dev: Exactly. But the real win is that they’ve proven this works outside the lab too with hardware like the Unitree H1-two without needing manual tuning, which shows it has practical utility.

Taro: That zero-shot sim to real transfer on physical hardware is a strong validation because it means we aren't just seeing theoretical improvements in simulation; they actually translate to doing things on the actual robot.

Rosa: And that’s what I find most interesting—when you combine that high-fidelity reference generation with the improved learning speed, it sets a much higher bar for what we expect from imitation learning setups.

Dev: So, the implication is that if we want robots to learn complex, dynamic tasks reliably from video demonstrations, focusing on this direct task-space formulation over geometric projections is a very promising path forward.

The paper's improvements: Taro: So, we’ve covered how they got their results by showing that Direct Dynamic Retargeting or DDR beats geometric retargeting and indirect dynamic retargeting on demonstration tracking accuracy and physical consistency.

Rosa: Right, but the authors are also pointing out what they can do next to make this even better.

Dev: They suggest that by formulating the problem directly in task space using that Laplacian graph distance, we can mitigate a lot of noise from the human data itself which is often messy in videos.

Taro: So they’re saying that if we use this task-space formulation, the resulting reference trajectories will be more stable and less sensitive to those noisy or drifting human motion data points extracted from a video.

Rosa: That means when an AI agent tries to learn from that reference, it’s going to have a cleaner signal to work with rather than having to fight through inconsistencies in the demonstration.

Dev: And another thing they touch on is that by using a sampling-based solver right inside the physics simulator, they are inherently optimizing over complex contact sequences while guaranteeing physical viability for every single motion generated.

Taro: That part is key because it means we don't have to worry as much about the resulting motion being physically impossible or unstable when we take that trajectory out of simulation and try to run it on actual hardware.

Rosa: It’s about building a system where the reference itself is both accurate to the human intent and guaranteed to be safe for the robot.

Dev: So, in terms of improvement, they aren't just changing how you retarget; they are changing the entire optimization process to prioritize physical feasibility alongside task tracking accuracy.

Taro: This moves away from just getting a geometrically close shape and toward generating a dynamically sound movement that respects the physics of the robot.

Rosa: It’s about making sure that the learning signal we give to our AI is actually high-quality, which should lead to those faster convergence speeds we talked about earlier.

Dev: That leads us into what they plan for next, since they admit there are still some limitations on how well this task-space definition works in certain edge cases where the setup might become ill-conditioned.

Conclusion: Rosa: So to wrap up, we’ve talked about how Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos is changing how we get high-quality movement references by skipping those intermediate geometric steps and optimizing directly in task space using physics simulation.

Dev: It really comes down to bypassing that geometric bias that limits the search space in older methods, which means the resulting learning agents are trained on much more consistent and physically viable data.

Taro: From my side, this makes it easier for autonomy researchers because if the reference motion is physically sound and accurate, we get better downstream policies without having to spend a lot of time fixing weird physical glitches during training.

Rosa: And the practical validation was strong too, with zero-shot sim to real transfer on hardware like the Unitree H1-two showing it works outside the lab right away.

Dev: That’s the safety case we need for a controls engineer—if it transfers well to physical robots without constant manual tuning, that significantly lowers the barrier to deployment.

Taro: I still see a limitation though, which is that since they define everything in task space using that Laplacian graph distance, there are still some edge cases where the solution can become ill-conditioned or convergence can get slow.

Rosa: That’s fair; it’s a risk because we don't want our robots getting stuck in those tricky situations when the world misbehaves.

Dev: So, the main implication is that for complex skills, having high-fidelity references generated this way leads to faster learning and more robust behavior across different balancing tasks.

Taro: It’s about moving from just getting a shape that looks right to generating actual dynamic motions that respect the physics of walking or balancing.

Rosa: Exactly, so we’ve seen how this Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos framework sets a new standard for generating reference data.

Dev: We gotta keep an eye on those convergence slowdowns when it comes to those specific task-space definitions, though that’s the main caveat.

Taro: Next up, we need to see how they scale this up when you move from just expert videos in a controlled setting to handling massive datasets from in-the-wild web videos.

LAAS-CNRS, University of Toulouse, CNRS, Toulouse, France · IRT Saint-Exupéry, Toulouse, France · Artificial and Natural Intelligence Toulouse Institute (ANITI), Toulouse, France

cs.RO

Submitted: 2026-05-22

Updated: 2026-11-17

Comments: https://gepetto.github.io/robot-motion/ddr/

Code: https://github.com/stephane-caron/pink

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 87/100

The gist: Imitation Learning from monocular video demonstrations provides a scalable approach for teaching complex skills to humanoid robots <ref:2605.23762#pg2> However…

Key concepts

Geometric Retargeting (GR)
This is a standard method that attempts to map human motion onto a robot's body shape using geometric transformations. The issue is that the intermediate steps introduce a bias, which restricts the possible movement paths and often leads to less realistic or suboptimal dynamic behaviors for the robot.
Direct Dynamic Retargeting (DDR)
This novel framework generates trajectories directly from expert videos by formulating the problem in task space. It uses a sampling-based Model Predictive Control solver within a physics simulator, ensuring the resulting motions are both high-fidelity and physically feasible, thus avoiding geometric bias.
Task Space Formulation
Instead of relying solely on body geometry, DDR defines the motion problem in 'task space.' This approach allows the system to optimize for keypoints distance and shape matching metrics directly within a physics simulator, which helps mitigate input drift from noisy video data.

Terminology

Summary

Imitation Learning from monocular video demonstrations provides a scalable approach for teaching complex skills to humanoid robots <ref:2605. p. 0

Where the paper says it

Imitation policies obtained from these trajectories can then be ported on a real full-size humanoid robot. Abstract— Imitation Learning from monocular video demonstrations provides a scalable approach for teaching complex skills to humanoid robots.

23762#pg2> However, translating human motion to humanoids requires overcoming significant morphological mismatches <ref:2605. p. 0
Where the paper says it

Abstract— Imitation Learning from monocular video demonstrations provides a scalable approach for teaching complex skills to humanoid robots. However, translating human motion to humanoids requires overcoming significant morphological mismatches.

23762#pg2>. Standard approaches rely on Geometric Retargeting or Indirect Dynamic Retargeting pipelines <ref:2605. p. 0
Where the paper says it

However, translating human motion to humanoids requires overcoming significant morphological mismatches. Standard approaches rely on Geometric Retargeting or Indirect Dynamic Retargeting pipelines.

23762#pg2>. In this paper, we propose Direct Dynamic Retargeting (DDR), a novel single-stage framework that generates high-fidelity, dynamically feasible trajectories directly from expert videos <ref:2605. p. 0
Where the paper says it

We identify that these intermediate kinematic projections introduce a geometric bias, restricting the search space and yielding suboptimal dynamic behaviors. In this paper, we propose Direct Dynamic Retargeting (DDR), a novel single-stage framework that generates high-fidelity, dynamically feasible trajectories directly from expert videos.

23762#pg2>.

Pruning redundant reasoning tokens

The core problem addressed is the translation of human motion to humanoid robots, which involves overcoming significant morphological mismatches <ref:2605.23762#pg2>. Standard methods like Geometric Retargeting (GR) introduce a geometric bias that restricts the search space and yields suboptimal dynamic behaviors <ref:2605. p. 0

Where the paper says it

Standard approaches rely on Geometric Retargeting or Indirect Dynamic Retargeting pipelines. We identify that these intermediate kinematic projections introduce a geometric bias, restricting the search space and yielding suboptimal dynamic behaviors.

23762#pg2>. The authors identify that intermediate kinematic projections in GR and Indirect Dynamic Retargeting (IDR) introduce this bias, which limits the search space and biases the final dynamically feasible solution <ref:2605. p. 0
Where the paper says it

We identify that these intermediate kinematic projections introduce a geometric bias, restricting the search space and yielding suboptimal dynamic behaviors. In this paper, we propose Direct Dynamic Retargeting (DDR), a novel single-stage framework that generates high-fidelity, dynamically feasible trajectories directly from expert videos.

23762#pg2>. Direct Dynamic Retargeting (DDR) is proposed as a novel single-stage framework that generates high-fidelity, dynamically feasible trajectories directly from expert videos <ref:2605. p. 0
Where the paper says it

We identify that these intermediate kinematic projections introduce a geometric bias, restricting the search space and yielding suboptimal dynamic behaviors. In this paper, we propose Direct Dynamic Retargeting (DDR), a novel single-stage framework that generates high-fidelity, dynamically feasible trajectories directly from expert videos.

23762#pg2>.

How the training signal is built

The DDR method formulates the problem in the task space and leverages a sampling-based Model Predictive Control solver within a physics simulator <ref:2605.23762#pg2>. The optimization is defined as finding a trajectory Q in the feasible set Fq0 such that it minimizes the keypoints distance d to a reference demonstration xref <ref:2605. p. 2

Where the paper says it

The Geometric Retargeting approach searches for a pair (QGR, xGR) ∈ Q × Q˜ that minimizes the distance to the expert keypoints d(xGR, xref ) while minimizing the distance to the feasible set Fq0 . The Indirect Dynamic Retargeting starts from the GR result and refines the trajectory to find a feasible pair (QIDR, xIDR) ∈ Fq0 × Feq0 minimizing the distance to the GR keypoints d(xIDR, xGR) in a physically consistent environment, such as a simulator.

23762#pg2>. This leads to the minimization of reference discrepancy over the dynamically feasible set Fq0, defined by min U∈U d(FK(Sq0(U)), xref) = min Q∈Fq0 d(FK(Q), xref) <ref:2605. p. 2
Where the paper says it

The alternative method proposed in this work, called Direct Dynamic Retargeting, directly searches for a pair (QDDR, xDDR) ∈ Fq0 × Feq0 that minimizes the keypoints distance d(xDDR, xref ) without relying on a geometric intermediate. According to the definition above, the DDR method comes to find a minimum of the reference discrepancy over the dynamically feasible set Fq0 : min U∈U d(FK(Sq0 (U)), xref ) = min Q∈Fq0 d(FK(Q), xref) (2) = d(xDDR, xref ) (3) As QIDR ∈ Fq0 , it follows that: d(xDDR, xref ) ≤ d(xIDR, xref ), (4) where xDDR = FK(QDDR) ∈ Feq0 and xIDR = FK(QIDR) ∈ Feq0 .

23762#pg2>. The augmented distance used accounts for both spatial tracking and relative shape-matching metric, composed of terms Ep and El <ref:2605. p. 2
Where the paper says it

Dynamically Feasible Trajectory IDR and DDR methods introduced in Section III-A rely on the minimization of the distance within the task space R 3m×T . In order to account for the the human-robot embodiment gap and avoid unnatural motion, we consider the augmented distance introduced in [2], composed of two terms, spatial tracking and relative shape-matching metric: Ep(FK(Q), xref ) = X T i=1 ∥f k(qi) − x˜i∥ 2 (5) El(FK(Q), xref ) = X T i=1 1 m ∥L(f k(qi) − x˜i)∥ 2 (6) where f k(·) is the classical forward kinematics, x˜ represents either xref in DDR or xGR in IDR, Ep is the Euclidean distance, and El is a relative metric that uses the Laplacian

23762#pg2>.

Where the gains come from

The primary gain is achieved by bypassing the intermediate geometric bias inherent in previous methods <ref:2605.23762#pg2>. The authors show that DDR outperforms GR and IDR baselines in terms of demonstration tracking accuracy, physical consistency, and learning efficiency <ref:2605. p. 1

Where the paper says it

of physical consistency, demonstration tracking accuracy, and learning efficiency. The main contributions are as follows: • Introduction of Direct Dynamic Retargeting, a novel CEM-based MPC framework that generates dynamically feasible humanoid trajectories directly from demonstrations.

23762#pg2>. Specifically, DDR consistently yields the closest match to the reference videos <ref:2605. p. 4
Where the paper says it

Table III summarizes the contact error rates across all motions. DDR consistently yields the closest match to the reference videos.

23762#pg2>. Furthermore, control policies trained on these DDR trajectories converge faster and exhibit more robust behavior across diverse balancing tasks compared to those trained on GR or IDR references <ref:2605. p. 6
Where the paper says it

Our evaluations demonstrate that DDR consistently outperforms state-of-the-art retargeting baselines, yielding superior tracking accuracy by eliminating the geometric bias inherent to previous methods. These higher-fidelity references translate directly into downstream gains: Reinforcement Learning agents trained on DDR trajectories converge faster and exhibit more robust behavior across diverse balancing tasks.

23762#pg2>.

Real World experiments

The framework has been validated through real-world experiments on the Unitree H1-2 humanoid <ref:2605.23762#pg2>. The robot successfully executes the target motions without any task-specific manual tuning, demonstrating zero-shot sim-to-real transfer <ref:2605. p. 6

Where the paper says it

As shown in Fig. 8 and the supplementary video, the robot successfully executes the target motions without any task-specific manual tuning.

23762#pg2>. The evaluation across various motions shows that DDR maintains near-perfect feasibility across most tasks, significantly outperforming GR and IDR in terms of feasibility and contact sequence accuracy <ref:2605. p. 4
Where the paper says it

We observe that IDR and DDR significantly outperform GR in terms of feasibility. These results indicate that despite incorporating feasibility costs, GR fails to adequately account for contact dynamics, whereas our proposed DDR maintains near-perfect feasibility across most tasks.

23762#pg2>. This superior reference quality translates directly into higher final reward and faster convergence for downstream Reinforcement Learning agents <ref:2605. p. 6
Where the paper says it

Our evaluations demonstrate that DDR consistently outperforms state-of-the-art retargeting baselines, yielding superior tracking accuracy by eliminating the geometric bias inherent to previous methods. These higher-fidelity references translate directly into downstream gains: Reinforcement Learning agents trained on DDR trajectories converge faster and exhibit more robust behavior across diverse balancing tasks.

23762#pg2>.

Conclusion

In this work, we introduce Direct Dynamic Retargeting (DDR), a novel framework for generating high-quality, dynamically feasible reference trajectories for imitation learning <ref:2605.23762#pg2>. By formulating the reference generation problem directly in the task space using a Laplacian graph distance, our approach effectively mitigates the impact of noisy and drifting human data extracted from videos <ref:2605. p. 6

Where the paper says it

8: The Unitree robot H1-2 performing (a) a stable squat, (b) a static kung fu pose, (c) an extended one-foot balance, (d) a dynamic pistol squat, and (e) a balancing stick pose. By formulating the reference generation problem directly in the task space using a Laplacian graph distance, our approach effectively mitigates the impact of noisy and drifting human data extracted from videos.

23762#pg2>. By employing a sampling-based solver directly within a physics simulator, DDR inherently optimizes over complex contact sequences while guaranteeing the physical viability of the resulting motions <ref:2605. p. 6
Where the paper says it

By formulating the reference generation problem directly in the task space using a Laplacian graph distance, our approach effectively mitigates the impact of noisy and drifting human data extracted from videos. Furthermore, by employing a sampling-based solver directly within a physics simulator, DDR inherently optimizes over complex contact sequences while guaranteeing the physical viability of the resulting motions.

23762#pg2>. Our evaluations demonstrate that DDR consistently outperforms state-of-the-art retargeting baselines, yielding superior tracking accuracy by eliminating the geometric bias inherent to previous methods <ref:2605. p. 6
Where the paper says it

Furthermore, by employing a sampling-based solver directly within a physics simulator, DDR inherently optimizes over complex contact sequences while guaranteeing the physical viability of the resulting motions. Our evaluations demonstrate that DDR consistently outperforms state-of-the-art retargeting baselines, yielding superior tracking accuracy by eliminating the geometric bias inherent to previous methods.

23762#pg2>. These higher-fidelity references translate directly into downstream gains: Reinforcement Learning agents trained on DDR trajectories converge faster and exhibit more robust behavior across diverse balancing tasks <ref:2605. p. 6
Where the paper says it

Our evaluations demonstrate that DDR consistently outperforms state-of-the-art retargeting baselines, yielding superior tracking accuracy by eliminating the geometric bias inherent to previous methods. These higher-fidelity references translate directly into downstream gains: Reinforcement Learning agents trained on DDR trajectories converge faster and exhibit more robust behavior across diverse balancing tasks.

23762#pg2>. However, DDR has limitations, as the objective is defined purely in task space, which can lead to ill-conditioned solutions or slower convergence <ref:2605. p. 6
Where the paper says it

However, DDR has limitations. The objective is defined purely in task space, which can lead to ill-conditioned solutions or slower convergence.

23762#pg2>. Future work will scale this framework to handle large, diverse sets of motion variables from in-the-wild web videos <ref:2605. p. 6
Where the paper says it

Finally, performance remains sensitive to task-space weighting and initialization, which currently require manual tuning. Future work will scale this framework to handle large, diverse sets of motion variables from in-the-wild web videos, enabling the creation of a comprehensive dataset of physically viable humanoid reference motions for the community.

23762#pg2>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Direct Dynamic Retargeting (DDR) framework allows for generating high-fidelity, dynamically feasible trajectories directly from expert videos, which bypasses intermediate geometric bias that restricts search space in previous methods.

  2. The DDR method enables the generation of physically consistent references by formulating the problem in task space and leveraging a sampling-based Model Predictive Control solver within a physics simulator, thus mitigating input drift.

  3. Downstream Imitation Learning agents trained on DDR trajectories converge faster and exhibit more robust behavior across diverse balancing tasks, leading to improved final reward compared to kinematic approaches.

  4. The system can achieve zero-shot sim-to-real transfer on physical hardware like the Unitree H1-2 humanoid, validating the practical utility of the framework for real-world deployment.

Abstract

Imitation Learning from monocular video demonstrations provides a scalable approach for teaching complex skills to humanoid robots. However, translating human motion to humanoids requires overcoming significant morphological mismatches. So far, dynamic retargeting methods obtain physically feasible references by tracking an intermediate geometric retargeting in simulation, usually through its joint angles. We show that this intermediate step biases the solution through two factors: the space in which the reference is tracked, joint angles rather than task-space keypoints, and the tracked reference itself, the keypoints of the geometric retargeting rather than those of the human demonstration. We propose Direct Dynamic Retargeting (DDR), which bypasses both: it tracks the human keypoints exclusively in task space with a sampling-based Model Predictive Control solver inside a physics simulator, without contact annotations. DDR is evaluated in simulation against joint-space dynamic retargeting under controlled conditions (identical solver, budget, and warm start), on three humanoids and on motions from monocular videos and motion capture. The tracking space explains most of the gain in end-effector accuracy and robustness, and tracking the human demonstration directly prevents the transmission of errors from the intermediate geometric reference. This advantage carries over to Reinforcement Learning imitation policies, which reproduce the human demonstration more faithfully than policies trained on joint-space dynamic references. These policies are deployed zero-shot on two physical humanoids of very different sizes, H1-2 and R1, including contact-rich motions. We release the source code and a dataset of dynamically feasible references covering all the flat-ground motions of AMASS on the three humanoids.

Sources

Related papers