DiffMorphGrasp: A Diffusion Framework Conditioned on Hand Morphology for Cross-Embodiment Grasp Generation

arXiv:2602.00915 · cs.RO · Submitted 2026-01-31 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "DiffMorphGrasp: A Diffusion Framework Conditioned on Hand Morphology for Cross-Embodiment Grasp Generation".

Dev: The gist: UniMorphGrasp is a diffusion-based framework that incorporates hand morphological information into the grasp generation process for unified cross-embodiment grasp synthesis.

Rosa: First, who's behind it and why it matters.

Paper summary: Rosa: So we're looking at this paper called DiffMorphGrasp: A Diffusion Framework Conditioned on Hand Morphology for Cross-Embodiment Grasp Generation, and it’s about how to get robot hands to actually grab things when they look completely different.

Dev: Right, so the core idea here is that existing grasping policies usually only work for one specific hand design, and they don't transfer well when you switch to a different robot arm or gripper.

Rosa: Exactly. This paper proposes this new diffusion-based framework where you incorporate information about the hand’s physical shape directly into the process of generating a grasp pose.

Dev: It claims they can map grasps from all these different robotic hands into one single, unified human-like pose representation, which acts like a common space for the learning to happen.

Rosa: And they condition the grasp generation on structured representations of hand kinematics, which are encoded as graphs derived from how those hand configurations are set up.

Taro: From my side, I’m interested in how this handles when things go wrong in the real world; does this system actually manage to adapt when the physical environment misbehaves?

Dev: Well, they introduce a loss function that specifically targets errors at the joint level over the hand's kinematic structure to keep those generated grasps constrained during training.

Rosa: It’s not just about generating a grasp; it’s about making sure the resulting grasp actually respects how the joints are physically connected.

Taro: If we look at what this means for autonomy, this framework suggests that an autonomous system could potentially use one learned grasping policy and deploy it across a whole fleet of robot hands without needing to relearn everything for every new hand type.

Dev: The numbers they show in their experiments are pretty compelling; they demonstrate strong zero-shot generalization when testing on hand morphologies that the model hasn't seen before during training.

Rosa: They achieve an overall success rate of ninety-two point four percent across all tested metrics, which is quite high compared to other methods evaluated on existing benchmarks.

Taro: That level of performance across different embodiments sounds like it could make deploying robots in diverse settings much more scalable than what we're seeing now.

Dev: It’s about making the system robust enough so that even if the hand structure changes, the grasp generation process stays stable and reliable.

Rosa: So, to summarize this DiffMorphGrasp paper, it’s a diffusion model that uses hand morphology to unify cross-embodiment grasp synthesis by mapping poses into a canonical space and conditioning generation on graph-encoded kinematics.

Dev: And what they really mean for the world is that we might finally move past training grasping solutions for one specific robot hand and having to start from scratch every time we get a new piece of hardware.

Paper summary: Rosa: That’s the big implication, right? It suggests a path toward much more flexible and adaptable robotic manipulation systems.

Taro: I'm curious if this approach is scalable beyond just the hands themselves; can this framework handle grasping objects that have completely different physical properties or requires entirely new kinds of interaction?

Dev: The current setup is focused on mapping grasps for object point clouds, but the architecture, using a Point Transformer for object features and Graphormer for hand morphology, hints at a structure that could be extended.

Rosa: The authors themselves pointed out that they introduced this approach specifically to address the limitation where existing methods fail to generalize to unseen hand morphologies outside their training distribution.

Taro: And they did show how incorporating this morphology encoding mechanism improves performance; for example, using the Graphormer encoder instead of a basic Transformer resulted in a seven point three percent improvement in success rate on that specific dataset.

Dev: It seems like the structure of the hand itself is actually a major factor that needs to be explicitly modeled for successful grasping synthesis.

Rosa: So, if we look at how this paper positions itself, it’s building on previous work by introducing a principled way to describe diverse hand morphologies in a unified and structured form.

Taro: That unification through the morphological representation sounds like the key piece that unlocks this cross-embodiment learning capability.

Dev: And the training process involves conditioning on both that morphology representation and object features extracted via a Point Transformer, which is how it gets all those necessary inputs together for the denoising model.

Rosa: It’s a lot of moving parts—the mapping, the graph encoding, the point cloud features—but they tie it all together into one coherent diffusion process.

Taro: I just wonder about the practical deployment time; how fast does this whole diffusion process actually run in real-time on actual robotic hardware?

Dev: That’s a valid question because diffusion models can sometimes be slow, but the research focuses on making sure the loop rate and latency are managed correctly during the training phase.

Rosa: So, to wrap up this part of the talk about DiffMorphGrasp: it’s a diffusion framework that uses hand morphology to unify grasp generation across different robot hands by mapping them to a common space and conditioning on those structural representations.

Dev: And its implication is that we can have much more adaptable robotic systems where grasping policies aren't locked down to just one specific hand design anymore.

Taro: It moves the problem from "how do I grasp this object with this specific hand?" to "how do I generate a grasp that works across a family of hands?"

Rosa: Exactly, and that’s what makes this work genuinely interesting for the field of autonomous robotics right now.

Conclusion: Rosa: So we're wrapping up this look at DiffMorphGrasp, which is about using diffusion models to handle grasping across different robot hands.

Dev: Yeah, so if I'm getting the main idea right, it’s this framework that lets you take a hand shape and use it to generate grasps for any other hand shape.

Taro: It means we can finally have a grasp policy that works even if the robot arm changes its gripper halfway through a task.

Rosa: Exactly. The authors are showing how they map those different hand structures into one common space, which makes learning much simpler.

Dev: And they condition the generation on the actual physical structure of the hand using these graphs, which is a pretty neat way to keep the physics in check during training.

Taro: What this really changes for autonomy is that deployment becomes scalable across a whole fleet of robots without needing custom software for every single model.

Rosa: Right. It moves us away from training one specific gripper and having to start over every time we get new hardware.

Dev: The results they got, showing strong zero-shot performance on unseen hand morphologies, really backs up the idea that this works outside of just the lab setup.

Taro: I'm still thinking about how robust this is when things get messy in the real world, like unexpected object shapes or slippage.

Rosa: That’s what we’ll be talking about next—testing its limits on those kinds of tough scenarios and figuring out how long it actually holds up in practice.

King’s College London Department of Engineering Department of Imperial College London

cs.RO

Submitted: 2026-01-31

Updated: 2026-10-08

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 89/100

The gist: The gist: UniMorphGrasp is a diffusion-based framework that incorporates hand morphological information into the grasp generation process for unified cross-embodiment grasp synthesis.

Key concepts

Canonical Hand Mapping
Different robotic hands have varying numbers of joints (DoFs). This concept maps all these different hand structures into a single, standardized pose format. This allows the diffusion model to learn a common space for generating grasps, regardless of the specific hand's physical design.
Morphology Encoder
This component uses Graphormer to analyze the structure of a specific hand. It extracts detailed geometric data about each joint, such as its size and limits. This information is converted into a set of 'morph tokens' that capture the unique characteristics of that hand's physical structure.
Morphology-Aware Denoising Model
This is the core diffusion model modified to use hand morphology information during grasp generation. It conditions its iterative refinement process on the hand's morph representation and the object's features, enabling it to adapt its grasp strategy based on what kind of hand it is trying to simulate.

Terminology

Summary

The gist: UniMorphGrasp is a diffusion-based framework that incorporates hand morphological information into the grasp generation process for unified cross-embodiment grasp synthesis.

Problem Formulation

Given an object point cloud O ⊂ R cubed and the morphology M of a target hand embodiment, extracted from its URDF specification (detailed in Sec. 3), we aim to sample a batch of dexterous grasp poses H from a conditional distribution P(HO,M) <ref:2602.00915#pg6>. Each pose is parametrized as (t, R, θ) ∈ R 9+N, consisting of a global translation t ∈ R cubed, a global rotation R ∈ R 6, and joint angles θ ∈ R N, where N denotes the number of joint degrees of freedom (DoFs) (e.g., 24 for Shadow hand) <ref:2602.00915#pg2>. The conditional distribution P(HO,M) is modeled using a diffusion model ϵϕ(Ht, O,M, t), which iteratively transforms an isotropic Gaussian distribution N (0, I) into the desired data distribution: P(H0O,M) = P(HT) <ref:2602.00915#pg4>.

Canonical Hand Mapping

To unify the varying DoFs across different dexterous hands, we follow (Zhang et al., 2024b) and adopt a canonical hand pose format Hc, to map all different hand embodiments <ref:2602.00915#pg2>. We reformulate the problem as sampling Hc from P(HcO,M), where each canonical pose is parameterized as (t, R, θc, δ) ∈ R 9+2Nc, with θc ∈ R Nc denoting the canonical joint angles and δ ∈ R Nc a binary mask indicating active joints <ref:2602.00915#pg4>. In practice we use Nc = 24 DoF of the Shadow hand <ref:2602.00915#pg4>. For example, for 3-fingered Barrett hand, the ring and pinky fingers, the wrist-palm joint, and the excess distal joints are masked, with more details in Appendix A <ref:2602.00915#pg2>.

Morphology Encoder

Inspired by (Patel & Song, 2025; Zhang et al., 2025), we employ Graphormer (Ying et al., 2021) as our morphology encoder to encode the hand’s morphological information as graph-structured features <ref:2602.00915#pg4>. We extract each joint’s child link collision geometry, i.e., length, width, and height of the bounding box (3 dim), along with the joint’s maximum and minimum limits (2 dim), origin (3 dim), and axis (3 dim) to form the joint morphology J = 11 <ref:2602.00915#pg5>. The morphs of each joint are embedded to obtain a set of morph tokens X = Nc×D, where D denotes the feature dimension <ref:2602.00915#pg5>. The Graphormer encoder processes these tokens through self-attention layers with learned graph-based attention biases derived from the kinematic structure, including spatial adjacency and parent-child relationships, to obtain morph representations M = Nc×D <ref:2602.00915#pg5>. Specifically, the attention scores Aij between joints i and j are computed as: Aij = (xiwq)(xjwk)T√D + bgraph(i, j) + bmask(δi, δj) <ref:2602.00915#pg5>.

Morphology-Aware Denoising Model

We propose a Morphology-Aware Denoising Model to condition the iterative grasp generation on the morph representation M, which enables the model to adapt grasp generation to diverse hand structures <ref:2602.00915#pg4>. The conditioning inputs include the morph representation M ∈ Nc×D from Sec. 3 and the point cloud feature P ∈ Np×D extracted from the object point cloud O via a Point Transformer (Zhao et al., 2021) encoder, where Np represents the number of point groups <ref:2602.00915#pg5>. The initial hand feature hT ∈ R D is obtained by embedding the noised hand pose Hc T and the active joint mask δ, then concatenating them <ref:2602.00915#pg5>. The denoising process iteratively refines this feature from t = T to t = 0 through a series of Transformer blocks <ref:2602.00915#pg5>. At each timestep t, the current hand feature ht is processed through a UNet block containing a residual convolutional layer conditioned on the timestep embedding: h conv t = Conv(ht) + Emb(t), where Emb(t) ∈ R D that embeds timestep to the latent dimension <ref:2602.00915#pg5>. This is followed by cross-attention modules for morphology and point cloud conditioning <ref:2602.00915#pg5>. First, morphology-aware conditioning is applied via cross-attention with M: h m t = softmax qhKT√M D!V M, where qh = h conv t wq ∈ R 1×D is the query from the hand feature, and KM = Mwk ∈ R Nc×D, V M = Mwv ∈ R Nc×D are the key and value from M <ref:2602.00915#pg5>. Similarly, objectaware conditioning is then applied via cross-attention with P: h p t = softmax q′hKT√P D!V P, where q′h = h m t w′q ∈ R 1×D is the query from the morphology-conditioned hand feature, and KP = P w′k ∈ R Np×D and V P = P w'v ∈ R Np×D are derived from the grouped point cloud representation <ref:2602.00915#pg5>.

Morphology-Aware Loss Function

Given the kinematic structure of the hand, let ci denote the number of descendant joints for the i-th joint, where ci = 0 for inactive joints <ref:2602.00915#pg5>. We compute an adaptive weight for each joint as: wi = r(ci + 1)G, where G is defined as: G = exp(1/(PNc j=1 δj X Nc j=1 δj ln(cj + 1)) <ref:2602.00915#pg5>. This formulation ensures that the weights maintain a geometric mean of 1, preventing overall scale drift while adaptively emphasizing proximal joints with larger kinematic influence <ref:2602.00915#pg5>. The morphology-aware loss is then formulated as: Lm = t − ˆt squared + R − Rˆ squared + X Nc i=1 δiwi(θi − ˆθi) squared, where (t, R, θ) and (ˆt, Rˆ, θˆ) define Hc 0 predicted by the morphology-aware denoising model and the ground truth canonical hand pose Hˆc 0 <ref:2602.00915#pg5>. The total training loss is formulated as: L = Lrecon + Lm + αspfLspf + αerfLerf + αsrfLsrf, where Lrecon denotes the standard noise reconstruction loss, and the physical constraint losses include the surface pulling force loss Lspf, the external-penetration repulsion force loss Lerf, and the self-penetration repulsion force loss Lsrf <ref:2602.00915#pg5>.

Experiments and Results

Extensive experiments demonstrate that UniMorphGrasp achieves state-of-the-art (SoTA) performance on existing benchmarks and exhibits strong generalization to previously unseen hand structures <ref:2602.00915#pg4>. In the in-domain comparison shown in Tab. 1, DRO-Grasp achieves the highest success rate among baselines but exhibits limited diversity <ref:2602.00915#pg5>. UniMorphGrasp outperforms all evaluated baselines across all metrics, achieving an overall success rate of 92.4%, representing improvements ranging from 4.9% to 35.0% over the baselines <ref:2602.00915#pg5>.

Effectiveness of Morphology Encoding

To validate the effectiveness of our proposed morphology encoding mechanism, we conduct ablation studies of: 1) morph encoding in UniMorphGrasp, and 2) the Graphormer (Ying et al., 2021) employed in the morphology encoder compared to a basic Transformer (Dosovitskiy, 2020) on the MultiDex (Li et al., 2023) dataset <ref:2602.00915#pg5>. Tab. 3 demonstrates that incorporating morphology encoding results in a 7.3% improvement in success rate and a 2.

Improvements for AI systems

  1. The UniMorphGrasp framework can generate stable and diverse grasps for robotic hands with heterogeneous kinematic structures by mapping diverse hand morphologies into a unified human-like five-finger kinematic tree, which serves as a canonical pose space. This allows the system to achieve strong zeroshot generalization to previously unseen hand structures.

  2. The system can synthesize grasps conditioned on both graph-encoded hand structures and object geometry, enabling it to adapt grasp distributions to previous unseen hand structures during inference.

  3. The introduction of the morphology-aware loss function, defined as Lm = t − ˆt2 + R − Rˆ 2 + Σi wi(θi − ˆθi)2, guides joint-level supervision by adaptively weighting errors according to the kinematic tree, ensuring proximal joints with larger kinematic influence are prioritized.

  4. The morphology encoder, utilizing Graphormer, encodes hand morphologies as graph-structured features where attention is biased by the hand’s kinematic structure and the active joint mask, providing a richer understanding of hand structures for improved grasp quality.

  5. The system can perform zero-shot generalization to novel hand structures by maintaining a consistently high success rate with performance degradation limited to 7% across topological, geometrical, and embodiment variations of the Shadow Hand, as validated by ablation studies.

Sources

Related papers