Morphology-Aware Human Motion Retargeting for Wheeled-Humanoid Loco-Manipulation

arXiv:2609.11357 · cs.RO · Submitted 2026-09-10 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "Morphology-Aware Human Motion Retargeting for Wheeled-Humanoid Loco-Manipulation".

Paper summary: Rosa: So to summarize this paper called Morphology-Aware Human Motion Retargeting for Wheeled-Humanoid Loco-Manipulation, the authors introduce a reproducible pipeline that takes multi-dataset SMPLX motion and converts it into executable loco-manipulation behavior for the Galaxea R1 Pro wheeled humanoid.

Dev: The thesis is focused on solving the problem where human lower-body motion has to be redistributed across base motion and torso posture without sacrificing manipulation relevant arm geometry on this specific robot.

Rosa: They achieve this by combining canonical body-shape preprocessing, planar-base normalization, morphology-aware differential inverse kinematics, shoulder-rooted hierarchical arm retargeting, and continuous torso substitution for bending and squatting.

Dev: The key is that they separate the manipulation relevant upper body motion from the mobile base execution to make that separation explicit in both their retargeting and decoder contracts.

Rosa: They present a complete data and control pipeline where the first stage standardizes SMPL-X inputs, solves morphology-aware IK, and produces versioned planar-base motion files.

Dev: The second stage converts that planar base reference into three-wheel commands using a low dimensional base decoder that learns a physical tracking policy in Isaac Lab.

Rosa: They show how this works by demonstrating that the standard GMR formulation is an important starting point, but its direct leg to robot mapping is poorly matched to the R1 Pro morphology.

Dev: The paper emphasizes that direct leg to torso GMR mapping cannot represent this locomotion posture split, which motivates their morphology-aware redistribution of lower-body motion.

Rosa: Ultimately, the authors show that they modify the retargeting and planning stages specifically to handle this split between locomotion and posture in a way that is relevant to wheeled systems.

Conclusion: Dev: So looking at the title Morphology-Aware Human Motion Retargeting for Wheeled-Humanoid Loco-Manipulation and the authors, what does that actually mean for us on the ground?

Rosa: It means this work provides a concrete way to bridge human motion data to physically trackable wheeled humanoid locomotion and manipulation, which is something few wheeled systems currently support well.

Dev: It’s about taking general human motion and making it work for a specific robot with constraints like having only a planar three-wheel base and no leg joints.

Rosa: The implication is that we can start using SMPLX data to drive the control of these mobile manipulators, rather than just visualizing what the motion looks like on screen.

Dev: It moves it from visualization-only retargeting to providing actual executable loco-manipulation behavior, which is a significant step for real-world applications.

Rosa: They are showing that careful planning and decoding of planar references into bounded commands can give you stable physical execution even when you have to approximate the human lower body folding.

Dev: It's a practical approach to handling the non-legged morphology, showing how separating upper body motion from base execution is essential for this kind of control.

Rosa: This paper shows that for complex mobile systems like wheeled humanoids, you need specific adaptation beyond standard general motion retargeting to get stable results.

Harbin Institute of Technology

cs.RO

Submitted: 2026-09-10

Updated: 2026-11-18

Comments: 8 pages, 11 figures, 9 tables. Version 2 extends the motion-retargeting method to account for wheeled-base stability and adds corresponding ablation studies

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 81/100

The gist: Human-to-humanoid retargeting has largely been studied on legged platforms, while comparatively few wheeled-humanoid systems support coupled locomotion and manipulation from general human motion The…

Key concepts

Morphology-Aware Retargeting
This method adapts human body shapes to the robot's structure by treating the four-joint torso as a limited resource. It uses shoulder-rooted hierarchical arm scaling and task-partitioned differential inverse kinematics to correctly map non-human pelvis movements onto the robot's joints, preserving manipulation geometry.
Continuous Torso Substitution
This mechanism maps human lower-body folding motions onto the R1 Pro's torso using a knee-height feature. It employs sigmoid gating, temporal smoothing, and arm re-IK to smoothly translate complex human posture changes into physically achievable torso positions for squatting and bending.
Planner/Decoder Interface
This is the control contract that translates planar base motion (x, y) into continuous three-wheel steering and rolling commands. It decodes the motion into a reference twist, which is then combined with feedback to generate bounded pose commands for physical execution by the robot's policy.
Feasibility Filtering
This crucial step ensures that only stable or approximately stable human motions are used for training and execution. Since the R1 Pro cannot self-right, this filtering is essential for preventing falls during motion tracking.

Terminology

Summary

Human-to-humanoid retargeting has largely been studied on legged platforms, while comparatively few wheeled-humanoid systems support coupled locomotion and manipulation from general human motion The resulting system provides a complete bridge from human motion data to physically trackable wheeled-humanoid loco-manipulation rather than a visualization-only retargeter

Pipeline Overview

The authors present a reproducible pipeline that converts multi-dataset SMPLX motion into executable loco-manipulation behavior for the Galaxea R1 Pro wheeled humanoid This pipeline combines canonical body-shape preprocessing, planar-base normalization, morphology-aware differential inverse kinematics, shoulder-rooted hierarchical arm retargeting, and continuous torso substitution for bending and squatting The process involves a sequence of stages: standardizing SMPL-X inputs (Step 1), solving morphology-aware IK (Step 3), applying a feasibility filter (Step 4), planning the planar base motion (Step 5), decoding it into three-wheel commands via a continuous decoder (Steps 6 and 7)

Morphology and Kinematic Adaptation

The design addresses the challenge that human lower-body motion must be redistributed across base motion and torso posture without sacrificing manipulation-relevant arm geometry The authors follow five structural principles, treating the four-joint torso as a limited resource where upright motion prioritizes the spine, and squat motion reallocates joints to the missing lower-body fold Specifically, they employ a morphology-aware retargeting method that uses shoulder-rooted hierarchical arm scaling and task-partitioned differential IK to handle the non-human pelvis-to-shoulder topology Furthermore, a continuous torso substitution mechanism maps human lower-body folding to the R1 Pro torso using a knee-height feature, sigmoid gating, temporal smoothing, and arm re-IK

Planning and Control Layer

A reference-twist-driven planning layer decodes planar base motion into continuous three-wheel steering and rolling commands subject to hysteresis, kinematic continuity, acceleration, and actuator-rate limits This is achieved by transforming the planar base pose into a reference twist (v ref x, t, v ref y, t, ω ref t) and then combining this feedforward twist with bounded pose feedback to generate the desired twist ξ⋆t The final physical execution is handled by a 21-dimensional BaseDecode policy trained in Isaac Lab using directional joint-limit scaling, focused upper-body tracking, and a staged wheel-contact reward

Evaluation and Stability

The system incorporates rigorous feasibility filtering to ensure that only stable or approximately stable motions are used for training the policy The authors conduct qualitative PPO rollouts, showing that 15 of 16 selected references complete without a fall, demonstrating the reliability of the PPO–planner–PD chain Furthermore, ablation studies confirm that the combined hysteresis, equivalent-steering selection, dynamic limits, and staged wheel-contact layer are necessary for maintaining stable three-wheel contact during large translations and turns The paper concludes that the planner/decoder interface is the appropriate action contract because directly learning six coupled wheel and steering variables instead of planar (x, y, ψ) leads to mutually conflicting commands and a fall

Limitations

The authors acknowledge limitations, noting that the torso substitution is a task-level approximation of human lower-body folding rather than a biomechanically equivalent leg model Additionally, the wheel decoder assumes a planar three-wheel geometry and does not guarantee no slip under arbitrary dynamics The R1 Pro is not a self-righting system, making data screening a central prerequisite rather than just preprocessing The current evidence is based entirely on Isaac Sim simulation and generated reference motion, lacking hardware validation

Appendix Details

The appendix provides detailed configuration information, such as the two-stage differential-IK weights and morphology-aware arm scales used for R1 Pro retargeting It also lists the fixed evaluation motions used for the GMR baseline and details the B1–B5 motions selected for the canonical-shape normalization diagnostic The table detailing PPO configuration, including parallel environments and iteration counts, is also provided The fixed A–U motions used for the GMR baseline are enumerated in Table 6 The B evaluation motion index table lists the specific motions used for comparison The appendix also contains a table detailing IK weights and arm scaling configuration for R1 Pro retargeting The final section includes references to related work, including General Motion Retargeting (GMR) and other whole-body control systems

References

[1] Joao Pedro Araujo, Yanjie Ze, Pei Xu, Jiajun Wu, and C. Karen Liu. Retargeting matters: General motion retargeting for humanoid motion tracking. p. 1

Where the paper says it

Humanoid motion retargeting and tracking. General Motion Retargeting (GMR) provides a configurable MuJoCo/Mink differential-IK interface for multiple humanoid morphologies [1].

p. 1
Where the paper says it

Humanoid motion retargeting and tracking. General Motion Retargeting (GMR) provides a configurable MuJoCo/Mink differential-IK interface for multiple humanoid morphologies [1].

arXiv preprint arXiv:2510. p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

02252, 2025

[3] Donghoon Baek, Youngwoo Sim, Amartya Purushottam, Saurabh Gupta, and Joao Ramos. Toward control of wheeled humanoid robots with unknown payloads: Equilibrium point estimation via real-tosim adaptation. p. 1

Where the paper says it

Wheeled humanoids and whole-body control. Recent work studies wheeled humanoid teleoperation, trajectory transfer, and dynamics adaptation [8, 2, 3].

p. 1
Where the paper says it

Wheeled humanoids and whole-body control. Recent work studies wheeled humanoid teleoperation, trajectory transfer, and dynamics adaptation [8, 2, 3].

arXiv preprint arXiv:2403. p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

10948, 2024

[5] Tairan He, Zhengyi Luo, Xialin He, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning. p. 1

Where the paper says it

General Motion Retargeting (GMR) provides a configurable MuJoCo/Mink differential-IK interface for multiple humanoid morphologies [1]. TWIST, H2O, and OmniH2O demonstrate that retargeted or estimated human motion can serve as the reference layer for learned whole-body controllers [13, 6, 5].

p. 1
Where the paper says it

General Motion Retargeting (GMR) provides a configurable MuJoCo/Mink differential-IK interface for multiple humanoid morphologies [1]. TWIST, H2O, and OmniH2O demonstrate that retargeted or estimated human motion can serve as the reference layer for learned whole-body controllers [13, 6, 5].

In Conference on Robot Learning, 2024

[6] Tairan He, Zhengyi Luo, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Learning human-to-humanoid real-time whole-body teleoperation. p. 1

Where the paper says it

Recent work studies wheeled humanoid teleoperation, trajectory transfer, and dynamics adaptation [8, 2, 3]. Most existing wheeled-humanoid whole-body systems focus on online teleoperation from a human operator.

p. 1
Where the paper says it

Recent work studies wheeled humanoid teleoperation, trajectory transfer, and dynamics adaptation [8, 2, 3]. Most existing wheeled-humanoid whole-body systems focus on online teleoperation from a human operator.

arXiv preprint arXiv:2403.04436, 2024

[7] Mazeyu Ji, Xuanbin Peng, Fangchen Liu, Jialong Li, Ge Yang, Xuxin Cheng, and Xiaolong Wang. Exbody2: Advanced expressive humanoid whole-body control. p. 1

Where the paper says it

Our focus is complementary: the target is a wheeled humanoid whose torso must substitute for missing leg joints. Wheeled humanoids and whole-body control.

p. 1
Where the paper says it

Our focus is complementary: the target is a wheeled humanoid whose torso must substitute for missing leg joints. Wheeled humanoids and whole-body control.

arXiv preprint arXiv:2412. p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

13196, 2024

[8] Jiacheng Liu, Pengxiang Ding, Qihang Zhou, Yuxuan Wu, Da Huang, Zimian Peng, Xiao Wei, Weinan Zhang, Xin Yang, Cewu Lu, and Donglin Wang. Trajbooster: Boosting humanoid whole-body manipulation via trajectory-centric learning. p. 1

Where the paper says it

Wheeled humanoids and whole-body control. Recent work studies wheeled humanoid teleoperation, trajectory transfer, and dynamics adaptation [8, 2, 3].

p. 1
Where the paper says it

Wheeled humanoids and whole-body control. Recent work studies wheeled humanoid teleoperation, trajectory transfer, and dynamics adaptation [8, 2, 3].

arXiv preprint arXiv:2509. p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

11839, 2025

[9] Chenhao Lu, Xuxin Cheng, Jialong Li, Shiqi Yang, Mazeyu Ji, Chengjing Yuan, Ge Yang, Sha Yi, and Xiaolong Wang. Mobile-television: Predictive motion priors for humanoid whole-body control. p. 1

Where the paper says it

TWIST, H2O, and OmniH2O demonstrate that retargeted or estimated human motion can serve as the reference layer for learned whole-body controllers [13, 6, 5]. ExBody2 and Mobile-TeleVision emphasize physical filtering and upper/lower-body decomposition for robust tracking [7, 9].

p. 1
Where the paper says it

TWIST, H2O, and OmniH2O demonstrate that retargeted or estimated human motion can serve as the reference layer for learned whole-body controllers [13, 6, 5]. ExBody2 and Mobile-TeleVision emphasize physical filtering and upper/lower-body decomposition for robust tracking [7, 9].

arXiv preprint arXiv:2412.07773, 2024

[10] Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, and Michael J. Black. Amass: Archive of motion capture as surface shapes. p. 1

Where the paper says it

AMASS unifies motion capture datasets in a common representation and provides a large source of motion diversity [10]. Our input stage uses the SMPL-X body model for forward kinematics, but replaces dataset-dependent body shapes with a canonical shape before robot retargeting.

p. 1
Where the paper says it

AMASS unifies motion capture datasets in a common representation and provides a large source of motion diversity [10]. Our input stage uses the SMPL-X body model for forward kinematics, but replaces dataset-dependent body shapes with a canonical shape before robot retargeting.

In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5442–5451, 2019

[11] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3d hands, face, and body from a single image. p. 1

Where the paper says it

SMPL-X represents a human body with articulated pose, body shape, hands, and facial parameters [11]. AMASS unifies motion capture datasets in a common representation and provides a large source of motion diversity [10].

p. 1
Where the paper says it

SMPL-X represents a human body with articulated pose, body shape, hands, and facial parameters [11]. AMASS unifies motion capture datasets in a common representation and provides a large source of motion diversity [10].

IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(10):3365–3380, 2019

[12] Luis Sentis and Oussama Khatib. Synthesis of whole-body behaviors through hierarchical control of behavioral primitives. p. 2

Where the paper says it

IK and reinforcement learning. Our IK solver follows task-based differential IK with damping, limits, and iterative integration, in the tradition of hierarchical whole-body control [12, 4].

p. 2
Where the paper says it

IK and reinforcement learning. Our IK solver follows task-based differential IK with damping, limits, and iterative integration, in the tradition of hierarchical whole-body control [12, 4].

International Journal of Humanoid Robotics, 2(4):505–518, 2005

[13] Yanjie Ze, Zixuan Chen, Joao Pedro Araujo, Zi-ang Cao, Xue Bin Peng, Jiajun Wu, and C. Karen Liu. Twist: Teleoperated whole-body imitation system. p. 1

Where the paper says it

General Motion Retargeting (GMR) provides a configurable MuJoCo/Mink differential-IK interface for multiple humanoid morphologies [1]. TWIST, H2O, and OmniH2O demonstrate that retargeted or estimated human motion can serve as the reference layer for learned whole-body controllers [13, 6, 5].

p. 1
Where the paper says it

General Motion Retargeting (GMR) provides a configurable MuJoCo/Mink differential-IK interface for multiple humanoid morphologies [1]. TWIST, H2O, and OmniH2O demonstrate that retargeted or estimated human motion can serve as the reference layer for learned whole-body controllers [13, 6, 5].

arXiv preprint arXiv:2505. p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

p. 0
Where the paper says it

(iii) Shoulder and distal arm links retain direct correspondences because manipulation quality depends on their relative geometry. (iv) The squat reallocation is driven by a symmetric knee-height feature and a filtered continuous gate, so changing correspondence 1 arXiv:2609.

02833, 2025

A IK Weights and Arm Scaling Configuration

This appendix records the values used by the two-stage differential-IK configuration in our retargeting implementation Each pair is written as (wp, wR) for position and orientation weight, respectively The final column gives the local scale used when constructing the target: a scalar is a segment scale, whereas a triple is the independent (x, y, z) shoulder scale

B Evaluation Motion Index

This appendix identifies the fixed evaluation motions by their public dataset, subject, action label, and dataset-relative path The letter identifiers are used in the A–U baseline table and do not imply a ranking by performance

C Shape-Stressed Ablation Motions

The canonical-shape normalization ablation uses five randomly sampled, high-beta-distance motions from the feasibility-screened pool before evaluating either variant These are denoted B1–B5 only within this ablation; they are not additional members of the A–U main evaluation set

D Limitations and Reproducibility

The torso substitution is a task-level approximation of human lower-body folding, not a biomechanically equivalent leg model The wheel decoder assumes a planar three-wheel geometry and relies on contact to move the free base, not guaranteeing no slip under arbitrary dynamics The R1 Pro is not a self-righting system like a legged humanoid that can recover from a fall, making data screening central The current evidence is based entirely on Isaac Sim simulation and generated reference motion, with no hardware experiment included

Conclusion

We presented a reproducible modification of GMR for the non-legged morphology and mobile-manipulation demands of the Galaxea R1 Pro Canonical SMPL-X preprocessing, planar centering, shoulder-rooted arm IK, and continuous torso substitution replace the unstable direct leg mapping with a base-and-torso representation Feasibility screening is essential because the R1 Pro cannot recover from a fall The planner and three-wheel decoder convert planar references into bounded commands, while PPO and PD tracking provide physical execution in Isaac Sim Three qualitative studies support the design, including the PPO rollouts completing 15 of 16 selected references without a fall The plannerdecoder interface is therefore the appropriate action contract because directly learning six wheel/steering variables instead of planar (x, y, ψ) causes mutually conflicting wheel commands and a fall

[1] Joao Pedro Araujo, Yanjie Ze, Pei Xu, Jiajun Wu, and C. Karen Liu. Retargeting matters: General motion retargeting for humanoid motion tracking. arXiv preprint arXiv:2510.02252, 2025

[3] Donghoon Baek, Youngwoo Sim, Amartya Purushottam, Saurabh Gupta, and Joao Ramos. Toward control of wheeled humanoid robots with unknown payloads: Equilibrium point estimation via real-tosim adaptation. arXiv preprint arXiv:2403.10948, 2024

[5] Tairan He, Zhengyi Luo, Xialin He, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning. In Conference on Robot Learning, 2024

[6] Tairan He, Zhengyi Luo, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Learning human-to-humanoid real-time whole-body teleoperation. arXiv preprint arXiv:2403.04436, 2024

[7] Mazeyu Ji, Xuanbin Peng, Fangchen Liu, Jialong Li, Ge Yang, Xuxin Cheng, and Xiaolong Wang. Exbody2: Advanced expressive humanoid whole-body control. arXiv preprint arXiv:2412.13196, 2024

[8] Jiacheng Liu, Pengxiang Ding, Qihang Zhou, Yuxuan Wu, Da Huang, Zimian Peng, Xiao Wei, Weinan Zhang, Xin Yang, Cewu Lu, and Donglin Wang. Trajbooster: Boosting humanoid whole-body manipulation via trajectory-centric learning. arXiv preprint arXiv:2509.11839, 2025

[9] Chenhao Lu, Xuxin Cheng, Jialong Li, Shiqi Yang, Mazeyu Ji, Chengjing Yuan, Ge Yang, Sha Yi, and Xiaolong Wang. Mobile-television: Predictive motion priors for humanoid whole-body control. arXiv preprint arXiv:2412.07773, 2024

[10] Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, and Michael J. Black. Amass: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5442–5451, 2019

[11] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3d hands, face, and body from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(10):3365–3380, 2019

[12] Luis Sentis and Oussama Khatib. Synthesis of whole-body behaviors through hierarchical control of behavioral primitives. International Journal of Humanoid Robotics, 2(4):505–518, 2005

[13] Yanjie Ze, Zixuan Chen, Joao Pedro Araujo, Zi-ang Cao, Xue Bin Peng, Jiajun Wu, and C. Karen Liu. Twist: Teleoperated whole-body imitation system. arXiv preprint arXiv:2505.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Canonical SMPL-X preprocessing removes body-shape variation by recomputing body-model FK with a fixed canonical shape beta can, conditioning the input for robot retargeting to separate morphology from motion data. This ensures that retargeting is not dependent on dataset-specific body shapes, improving generalization across different human motions and robot morphologies.

  2. Morphology aware retargeting uses shoulder rooted hierarchical arm scaling and task partitioned differential IK to handle the nonhuman pelvis to shoulder topology, addressing the issue where a global pelvis rooted scale can produce a physically meaningful arm orientation but an invalid arm position. This allows the robot to reproduce arm swing while driving and maintain manipulation quality despite kinematic differences.

  3. A continuous torso substitution mechanism maps human lower body folding using a knee height feature, sigmoid gating, temporal smoothing, and arm re-IK, enabling the system to handle squatting mode as a more human-like squat by prioritizing center height change without requiring leg joints. This provides a continuous mapping transition that avoids hard switches between walking and squat modes.

  4. A reference twist driven planning layer decodes planar base motion into continuous three wheel steering and rolling commands subject to hysteresis, kinematic continuity, acceleration, and actuator-rate limits. This planner/decoder interface is preferred over direct wheel-steer tracking because it avoids mutually conflicting wheel commands that lead to a fall when human references do not satisfy physical constraints.

  5. A BaseDecode PPO formulation combines directional joint limit scaling and a staged wheel contact reward, using a two stage curriculum wc(k) following the schedule (0, 0 <= k < 750, 0.2, 750 <= k ≤ 1500). This training setup allows the policy to learn stable tracking by rewarding manipulation-focused tracking rewards and ensuring contact stability in stage two.

Abstract

We propose a whole-body teleoperation method for topology-mismatched humanoids, with a focus on motion retargeting and whole-body control for wheeled humanoids. First, we design a Topology-Adaptive Motion Retargeting (TAMR) method with a robust switching logic that selects mapping joints according to human posture. Second, we train a Teacher-Student DAgger method that supports real-time teleoperation with inertial motion capture and virtual-reality devices. We further develop a Temporal Feasibility-Aware Command Governor (T-FeaCG) for the wheeled base to reduce the mismatch between teleoperation policy intent and the feasible execution margin of a three-steer-wheel base. We will release the complete teleoperation code and dataset, and demonstrate whole-body humanoid skills learned from teleoperation. More videos are available at: https://xiachenbuo.github.io/Morphology-Aware-Human-Motion-Retargeting-for-Wheeled-Humanoid-Loco-Manipulation/.

Sources

Related papers