Bridging the Morphology Gap: Adapting VLA Models to Dexterous Manipulation via Intent-Conditioned Fine-Tuning
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: I'm Rosa.
Dev: And I'm Dev, here with Taro.
Rosa: We're looking at the paper: "Bridging the Morphology Gap".
Dev: Here's the gist of the paper. Pre-trained Vision-Language-Action (VLA) models provide useful semantic and spatial priors, yet their parallel gripper action interfaces do not specify how those priors should be realized by a dexterous hand<ref:2606.12109#pg2>.
Rosa: InDex introduces an intent-conditioned adaptation framework that separates the decisions of when contact should be established and how a morphology-specific hand trajectory should establish it<ref:2606.12109#pg4>.
Dev: This framework derives a normalized grasp intent from retargeted demonstrations, predicting synchronized end-effector–intent chunks in Stage 1 and then generating multi-joint hand actions via conditional diffusion in Stage 2<ref:2606.12109#pg4>.
Rosa: The results identify post-reach arm–hand coordination, rather than object localization alone, as the principal bottleneck in adapting parallel-gripper VLAs to dexterous manipulation<ref:2606.12109#pg4>.
Paper summary: Rosa: So, wrapping up this discussion on "Bridging the Morphology Gap: Adapting VLA Models to Dexterous Manipulation via Intent-Conditioned Fine-Tuning," the main point is that we need a way to link what the vision language model understands about a task with the specific physical control needed for a hand to execute it.
Dev: They found that by deriving this normalized grasp intent from demonstrations and using it to condition a diffusion decoder, they successfully preserved the VLA's core reaching abilities while making the final grasp much more stable and task completion rates higher on tasks like Nut Assembly.
Taro: The real implication for us in autonomy research is that we don't need to just train these models to see objects; we need to teach them how to coordinate their arm motion with a specific, intent-driven hand trajectory when they get close enough for contact.
Rosa: It suggests that morphology transfer isn't just about copying shapes; it requires an explicit interface between the semantic task progress and the embodiment-specific control signals. This approach gives us a temporal reference for contact, which is useful when dealing with uncertain physical interactions.
Dev: I think what stands out is how they successfully separated the decision of *when* to establish contact from the decision of *how* to establish it via hand trajectory generation. That modularity seems really promising for building safer systems where we can predict potential failure modes at different stages.
Conclusion: Rosa: So, this paper is about bridging that gap between what Vision Language Action models know and how your actual physical hand needs to move for something like grasping an object correctly, using this intent conditioning idea they called InDex.
Dev: Yeah, it separates when you should even try to touch something from the specific way the fingers actually reach for it, which is a neat way to handle that coordination problem.
Taro: It seems they found that focusing on arm hand coordination instead of just figuring out where the object is located was actually the biggest sticking point in making these parallel grippers work for real tasks.
Rosa: Right, so the core idea is this normalized grasp intent which acts like a temporal signal telling the system when to engage and what kind of contact phase it needs to be in, without worrying about the exact finger configuration yet.
Dev: And they use that scalar intent to guide a diffusion process in a second stage, which lets them generate many different finger trajectories that all fit that same high-level intent.
Taro: It’s interesting because it means you don't have to design every single hand pose from scratch; you just need to provide the right temporal goal and the system figures out how to get there physically.
Rosa: The conclusion is really about treating this as a coordination problem, meaning we look at the whole process of getting there rather than just trying to make a single complex action move.
Dev: It suggests that for transferring skills from simulation or vision models to real dexterous manipulation, you need an interface that connects the semantic task progress with the physical control signals.
Taro: That means morphology transfer isn't just about copying shapes anymore; it requires this kind of coordination between what the system knows about the task and how it physically executes that task.
Rosa: It opens up a way to get much better grasp stability on real tasks like assembling nuts, because they focus on timing the hand movement right with the arm movement, not just pinpointing a location.
Dev: The next thing we need to see is how robust this works when things go wrong outside of that perfect simulation setup you saw in the validation experiments.
Chuanke Pang, Junyi Huang, Zhijun Zhao, Yaobing Wang, Kun Xu, Xilun Ding
Beihang University · China Academy of Space Technology
cs.RO, cs.AI
Submitted: 2026-06-10
Updated: 2026-11-04
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 65/100
The gist: Pre-trained Vision-Language-Action (VLA) models provide useful semantic and spatial priors, yet their parallel gripper action interfaces do not specify how those priors should be realized by a…
Key concepts
- InDex
- A framework that adapts VLA models for dexterous tasks by factoring control into a task-space plan and embodiment realization. It predicts synchronized end-effector–intent chunks in Stage 1 and uses this intent to generate the final hand trajectory via conditional diffusion in Stage 2.
- Normalized Grasp Intent (γ)
- A scalar value derived from retargeted human demonstrations that aligns the contact phase without specifying a finger configuration. This single scalar represents a temporal coordination interface, allowing multiple hand configurations to share the same intent while context disambiguates the exact finger movements required.
- Stage 1 and Stage 2
- The framework is sequential: Stage 1 predicts synchronized end-effector–intent chunks conditioned on VLA context and proprioception (predicting *when* to contact). Stage 2 uses these predictions to generate the multi-joint hand trajectory via conditional diffusion (determining *how* to establish contact).
- Post-reach Arm–Hand Coordination
- The paper identifies this as the main difficulty in adapting parallel gripper VLAs. It means coordinating the arm's movement with the hand's action immediately after reaching an object, rather than just focusing on where the object is located.
Terminology
Summary
Pre-trained Vision-Language-Action (VLA) models provide useful semantic and spatial priors, yet their parallel gripper action interfaces do not specify how those priors should be realized by a dexterous hand<ref:2606. p. 012109#pg2>. InDex introduces an intent-conditioned adaptation framework that separates the decisions of when contact should be established and how a morphology-specific hand trajectory should establish it<ref:2606. p. 012109#pg4>. This framework derives a normalized grasp intent from retargeted demonstrations, predicting synchronized end-effector–intent chunks in Stage 1 and then generating multi-joint hand actions via conditional diffusion in Stage 2<ref:2606. p. 012109#pg4>. The results identify post-reach arm–hand coordination, rather than object localization alone, as the principal bottleneck in adapting parallel-gripper VLAs to dexterous manipulation<ref:2606. p. 012109#pg4>.
Framework Architecture
InDex factorizes dexterous control into a task-space transition plan and its embodiment-specific realization<ref:2606.12109#pg4>. Stage 1 predicts synchronized EEF–intent chunks, conditioned on VLA context and proprioception, while Stage 2 uses these predictions to generate the hand trajectory by conditional diffusion<ref:2606. p. 012109#pg4>. The scalar intent is a temporal coordination interface rather than a compressed hand pose<ref:2606. p. 012109#pg4>.
Intent Realization Mechanism
The normalized grasp intent γ is derived after retargeting the human hand to the robot configuration, defined as γ = clip[0,1] d(e,τ) max − dv(q) d(e,τ) min<ref:2606.12109#pg4>. This scalar aligns the contact phase without prescribing a finger configuration<ref:2606. p. 012109#pg4>. Multiple hand configurations can share the same γ; object context and proprioception disambiguate their finger-level realization<ref:2606. p. 112109#pg4>.
Training Objectives and Roles
The training involves a sequential initialization where Stage 1 supervises EEF translation, rotation, and grasp intent with separate losses Lpos, Lrot, and Lintent<ref:2606.12109#pg4>. Stage 2 is trained using a standard noise-prediction objective for the diffusion decoder while conditioning on the predicted stages<ref:2606. p. 212109#pg4>. This allows the diffusion decoder to realize many hand trajectories compatible with the same intent<ref:2606. p. 012109#pg4>.
Experimental Validation
Experiments across four simulated tasks and a physical arm–hand platform show that InDex preserves the VLA’s reaching competence while markedly improving conversion from approach to stable grasp and task completion<ref:2606.12109#pg5>. The largest gain is observed on Nut Assembly, where the improvement is attributed to contact–hand synchronization rather than localization alone<ref:2606. p. 312109#pg6>. Cross-backbone and physical experiments confirm that this interface extends beyond the primary simulation model and embodiment<ref:2606. p. 012109#pg5>.
Conclusion
InDex treats dexterous VLA adaptation as a coordination problem rather than an increase in action dimension. Aperture-derived intent provides a temporal reference for contact, while conditional diffusion preserves the full, multimodal hand trajectory. p. 7 The broader implication is that morphology transfer requires an interface between semantic task progress and embodiment-specific control. p. 7
REFERENCES
arXiv:2606.12109v2 [cs. p. 0RO] 5,Pre-trained Vision-Language-Action (VLA) models provide useful semantic and spatial priors, yet their parallel gripper action interfaces do not specify how those priors should be realized by a dexterous hand<ref:2606. p. 012109#pg2>InDex introduces an intent-conditioned adaptation framework that separates the decisions of when contact should be established and how a morphology-specific hand trajectory should establish it<ref:2606. p. 012109#pg4>This framework derives a normalized grasp intent from retargeted demonstrations, predicting synchronized end-effector–intent chunks in Stage 1 and then generating multi-joint hand actions via conditional diffusion in Stage 2<ref:2606. p. 012109#pg4>The results identify post-reach arm–hand coordination, rather than object localization alone, as the principal bottleneck in adapting parallel-gripper VLAs to dexterous manipulation<ref:2606. p. 012109#pg4>Framework ArchitectureInDex factorizes dexterous control into a task-space transition plan and its embodiment-specific realization<ref:2606. p. 112109#pg4>Stage 1 predicts synchronized EEF–intent chunks, conditioned on VLA context and proprioception, while Stage 2 uses these predictions to generate the hand trajectory by conditional diffusion<ref:2606. p. 0 p. 012109#pg4>The scalar intent is a temporal coordination interface rather than a compressed hand pose<ref:2606. p. 0 p. 012109#pg4>Intent Realization MechanismThe normalized grasp intent γ is derived after retargeting the human hand to the robot configuration, defined as γ = clip[0,1] d(e,τ) max − dv(q) d(e,τ) min<ref:2606. p. 1 p. 112109#pg4>This scalar aligns the contact phase without prescribing a finger configuration<ref:2606. p. 0 p. 012109#pg4>Multiple hand configurations can share the same γ; object context and proprioception disambiguate their finger-level realization<ref:2606. p. 1 p. 112109#pg4>Training Objectives and RolesThe training involves a sequential initialization where Stage 1 supervises EEF translation, rotation, and grasp intent with separate losses Lpos, Lrot, and Lintent<ref:2606. p. 2 p. 212109#pg4>Stage 2 is trained using a standard noise-prediction objective for the diffusion decoder while conditioning on the predicted stages<ref:2606. p. 2 p. 212109#pg4>This allows the diffusion decoder to realize many hand trajectories compatible with the same intent<ref:2606. p. 0 p. 012109#pg4>Experimental ValidationExperiments across four simulated tasks and a physical arm–hand platform show that InDex preserves the VLA’s reaching competence while markedly improving conversion from approach to stable grasp and task completion<ref:2606. p. 0 p. 012109#pg5>The largest gain is observed on Nut Assembly, where the improvement is attributed to contact–hand synchronization rather than localization alone<ref:2606. p. 3 p. 312109#pg6>Framework ArchitectureInDex factorizes dexterous control into a task-space transition plan and its embodiment-specific realization<ref:2606. p. 112109#pg4>Stage 1 predicts synchronized EEF–intent chunks, conditioned on VLA context and proprioception, while Stage 2 uses these predictions to generate the hand trajectory by conditional diffusion<ref:2606.12109#pg4>The scalar intent is a temporal coordination interface rather than a compressed hand pose<ref:2606.12109#pg4>Intent Realization MechanismThe normalized grasp intent γ is derived after retargeting the human hand to the robot configuration, defined as γ = clip[0,1] d(e,τ) max − dv(q) d(e,τ) min<ref:2606.12109#pg4>This scalar aligns the contact phase without prescribing a finger configuration<ref:2606.12109#pg4>Multiple hand configurations can share the same γ; object context and proprioception disambiguate their finger-level realization<ref:2606.12109#pg4>Training Objectives and RolesThe training involves a sequential initialization where Stage 1 supervises EEF translation, rotation, and grasp intent with separate losses Lpos, Lrot, and Lintent<ref:2606.12109#pg4>Stage 2 is trained using a standard noise-prediction objective for the diffusion decoder while conditioning on the predicted stages<ref:2606.12109#pg4>This allows the diffusion decoder to realize many hand trajectories compatible with the same intent<ref:2606.12109#pg4>Experimental ValidationExperiments across four simulated tasks and a physical arm–hand platform show that InDex preserves the VLA’s reaching competence while markedly improving conversion from approach to stable grasp and task completion<ref:2606.12109#pg5>The largest gain is observed on Nut Assembly, where the improvement is attributed to contact–hand synchronization rather than localization alone<ref:2606.12109#pg6>ConclusionInDex treats dexterous VLA adaptation as a coordination problem rather than an increase in action dimensionAperture-derived intent provides a temporal reference for contact, while conditional diffusion preserves the full, multimodal hand trajectoryThe broader implication is that morphology transfer requires an interface between semantic task progress and embodiment-specific controlREFERENCES
[1] K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn et al. p. 7, “π0: A vision-language-action flow model for general robot control,” arXiv preprint arXiv:2410. p. 724164, 2024. p. 7[1]A Khazatsky, K Pertsch S Nair A Balakrishna S Dasari S Karamcheti S Nasiriany M K Srirama L Y Chen K Ellis et al. p. 7, “DROID: A large-scale in-the-wild robot manipulation dataset,” arXiv preprint arXiv:2403. p. 712945, 2024. p. 7[3]Open X-Embodiment Collaboration A O’Neill A Rehman A Maddukuri A Gupta A Padalkar A Lee A Pooley A Gupta A Mandlekar et al. p. 7, “Open X-embodiment: Robotic learning datasets and RT-X models,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). p. 7 IEEE, 2024, pp. p. 7 6892–6903. p. 7[4]A Schmitz B Bauml T Wimbock M Grebenstein M Fuchs C Borst and G Hirzinger, “The dlr hand arm system,” in 2010 10th IEEE-RAS International Conference on Humanoid Robots. p. 7 IEEE, 2010, pp. p. 7 317–321.[5]C Grothe D Kuehn G Ficht K Welke M Rolf T Asfour, “A humanoid two-arm system for dexterous manipulation,” in 2012 12th IEEE-RAS International Conference on Humanoid Robots. p. 7 IEEE, 2012, pp. p. 7 p. 7 p. 7 317–322.[6]Y Tomita T Tsuji T Sasaki K Doya, “On the representation of anthropomorphic robot hands: Shape versus function,” in 2012 12th IEEE-RAS International Conference on Humanoid Robots. p. 7 IEEE, 2012, pp. 701–708.[7]Y Bai and C K Liu, “Dexterous manipulation using both palm and fingers,” in 2014 IEEE International Conference on Robotics and Automation (ICRA). p. 7 IEEE, 2014, pp. p. 7 p. 7 1560–1565. p. 7[8]Y Chen T Wu S Wang X Feng J Jiang Z Lu et al. p. 7, “Towards human-level bimanual dexterous manipulation with reinforcement learning,” in Advances in Neural Information Processing Systems, vol. p. 7 35, 2022, pp. p. 7 5150–5163. p. 7[9]V Kumar Y Tassa T Erez and E Todorov, “Real-time behaviour synthesis for dynamic hand-manipulation,” in 2014 IEEE International Conference on Robotics and Automation (ICRA). p. 7 IEEE, 2014, pp. 6808–6815. p. 7 p. 7[10]T Chen J Xu P Agrawal, “A system for general in-hand object re-orientation,” in Proceedings of the 5th Conference on Robot Learning, ser. p. 7 Proceedings of Machine Learning Research, vol. p. 7 p. 7 p. 7 164. PMLR, 2022, pp. p. 7 297–307.[11]W Zhao J P Queralta T Westerlund, “Sim-to-real transfer in deep reinforcement learning for robotics: A survey,” in 2020 IEEE Symposium Series on Computational Intelligence (SSCI). p. 7 IEEE, 2020, pp. p. 7 737–744.[12]C To and G D Hager, “Dexterous telemanipulation with a multitouch interface,” in 2012 12th IEEE-RAS International Conference on Humanoid Robots. p. 7 IEEE, 2012, pp. 270–277.[13]M Seo S Han K Sim S H Bang C Gonzalez L Sentis and Y Zhu, “Deep imitation learning for humanoid loco-manipulation through human teleoperation,” in 2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids). p. 7 IEEE, 2023, pp. p. 7 1–8.[14]Y Qin W Yang B Huang K Van Wyk H Su X Wang et al. p. 7, “AnyTeleop: A general vision-based dexterous robot arm-hand teleoperation system,” arXiv preprint arXiv:2307. p. 704577, 2023. p. 7[15]T Pearce T Rashid A Kanervisto D Bignell M Sun R Georgescu et al. p. 7, “Imitating human behaviour with diffusion models,” arXiv preprint arXiv:2301. p. 710677, 2023. p. 7[16]C Chi Z Xu S Feng E Cousineau Y Du B Burchfiel et al. p. 7, “Diffusion policy: Visuomotor policy learning via action diffusion,” The International Journal of Robotics Research, vol. p. 7 44, no. 10–11, pp. 1684–1704, 2025. p. 7[17]M Reuss M X Li X Jia and R Lioutikov, “Goal-conditioned imitation learning using score-based diffusion policies,” in Robotics: Science and Systems XIX, 2023,[18]B Zitkovich T Yu S Xu P Xu T Xiao F Xia J Wu P Wohlhart S Welker A Wahid et al. p. 7, “RT-2: Vision-language-action models transfer web knowledge to robotic control,” in Proceedings of The 7th Conference on Robot Learning, ser. p. 7 Proceedings of Machine Learning Research, vol. 229. PMLR, 2023, pp. p. 7 2165–2183. p. 7[19]M J Kim K Pertsch S Karamcheti T Xiao A Balakrishna S Nair et al. p. 7, “OpenVLA: An open-source vision-language-action model,” arXiv preprint arXiv:2406. p. 709246, 2024. p. 7[20]K Black N Brown J Darpinian K Dhabalia D Driess A Esmail et al. p. 7, “π0: A vision-language-action flow model for general robot control,” in Proceedings of The 9th Conference on Robot Learning, ser. p. 7 Proceedings of Machine Learning Research, vol. 305. PMLR, 2025, pp. p. 7 17–40.[21]E J Hu Y Shen P Wallis Z Allen-Zhu Y Li S Wang et al. p. 7, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations, 2022. p. 7[23]Y Zhu J Wong A Mandlekar R Mart´ın-Mart´ın A Joshi K Lin et al. p. 7, “robosuite: A modular simulation framework and benchmark for robot learning,” arXiv preprint arXiv:2009. p. 712293, 2019, pp. p. 7 6808–6815.[24]A Mandlekar D Xu J Wong S Nasiriany C Wang R Kulkarni et al. p. 7, “What matters in learning from offline human demonstrations for robot manipulation,” arXiv preprint arXiv:2108. p. 703298, 2021. p. 7[25]T Z Zhao V Kumar S Levine and C Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” in Robotics: Science and Systems XIX, 2023,[26]Q Bu Y Yang J Cai S Gao G Ren M Yao et al. p. 7, “UniVLA: Learning to act anywhere with task-centric latent actions,” arXiv preprint arXiv:2505. p. 706111, 2025. p. 7
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Intent-conditioned adaptation framework: InDex separates
when contact should be established and how a morphology-specific hand trajectory should establish it
by deriving a normalized grasp intent gamma from retargeted demonstrations. This allows the system to align contact transition timing without prescribing finger configurations, enabling the diffusion decoder to realizemany hand trajectories compatible with that intent.
-
Sequential architecture: The system employs a sequential initialization where Stage 1 predicts
synchronized EEF–intent chunks
and Stage 2 generates corresponding hand trajectories using a conditional diffusion model. This structure allows fortemporal coupling
between the arm and hand streams while ensuring thatgamma aligns the contact phase without prescribing a finger configuration.
-
Targeted training objectives: By utilizing separate loss functions, InDex supervises EEF translation, rotation, and grasp intent with distinct learning rates. This ensures that Stage 1 is trained for
semantic approach and transition alignment,
while Stage 2 focuses onmorphology-specific hand realization
via the diffusion decoder. -
Improved grasp conversion: The framework preserves the VLA's reaching competence while
markedly improving conversion from approach to stable grasp and task completion,
with the largest gain observed on Nut Assembly, indicating that InDex addressespost-reach arm–hand coordination, rather than object localization alone.
-
Cross-backbone compatibility: Experiments show that InDex improves native models across different backbones by significant margins, such as increasing OpenVLA performance from 8.0% to 73.0% on Pick & Place, demonstrating that the interface extends beyond a single simulation model and is compatible with various VLA representations.
Sources
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
- AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
- Imitating Human Behaviour with Diffusion Models
- OpenVLA: An Open-Source Vision-Language-Action Model
- Language Conditioned Multi-Finger Dexterous Manipulation Enabled by Physical Compliance and Switching of Controllers
- robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
- What Matters in Learning from Offline Human Demonstrations for Robot Manipulation
- UniVLA: Learning to Act Anywhere with Task-centric Latent Actions
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation