CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation".
Title and authors: Rosa: So we're looking at this paper, "CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation." It sounds like they are tackling that big problem where robot demonstrations teach you how to do something on one robot, but then you have to teach it on a different robot.
Dev: Exactly. The title tells us they aren't just doing action reconstruction; the core idea is creating a shared space for intent so that the knowledge transfers across different control interfaces. It’s about bridging those differences in how each robot is physically controlled, which usually requires a specific decoder for every single robot.
Taro: I wonder if this shared space actually captures the right kind of information to make it work robustly when you move from one hardware setup to another.
The paper's summary: Rosa: The paper explains that they use three different encoders—one for the action, one for visual changes, and one for language grounded in what you see—all mapping those inputs into this shared intent space. They align them using contrastive losses and then train a single decoder to reconstruct the actions from all three representations at once.
Dev: So the training objective is pretty comprehensive; they're combining masked action and text reconstruction losses with symmetric InfoNCE losses to make sure all those different signals are talking about the same underlying intent. The complete loss is defined as L equals X m in a, v, t L m act plus Ltext plus zero point one alpha lambda v plus zero point zero five alpha lambda t plus zero point zero five lambda v t >
Rosa: That structure shows they are not just checking if the action looks right, but also making sure the visual and language signals align with the action in that shared space. It’s trying to make alignment functional rather than purely geometric so it actually helps with control.
The paper's improvements: Dev: They suggest a few key things for improvement. First, they pretrain CIALLO on eight different robot embodiments with balanced sampling across them, which helps the shared codec represent a broader range of actions, leading to lower reconstruction errors even on the original robots.
Taro: So pretraining on more hardware seems crucial for getting that representation broad enough to handle new things later. What about adapting it?
Rosa: When you move to a new robot, you don't retrain everything; instead, you just calibrate the new robot's essential vector and arm and axis embeddings while keeping the shared network frozen. Then, a policy learns to predict intent from observations and instructions.
Dev: It’s this separation of embodiment calibration from task learning that they emphasize as key to making this transfer mechanism work efficiently. It also points out that Qwen-CIALLO achieves ninety-seven point three percent average success across the four LIBERO suites, which is their highest average in Table two <ref:2611.13732#pg2>.
Conclusion: Rosa: So, to wrap up on "CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation," the main implication is that you can separate what an action actually is from how a specific robot executes it. This allows you to have one shared representation of short-horizon behavioral intent that works across different bodies and control schemes.
Dev: And the practical result is they show this separation makes adaptation much faster; you only need to update the embodiment embeddings, and the policy fine-tunes its intent projector on top of a frozen action encoder. This means you don't need massive robotaction pretraining before you even start finetuning your policy backbone.
Taro: From my point of view, seeing it adapt from one recording of at most ten seconds just by updating the embodiment embeddings is pretty efficient for real-world deployment where data collection isn't always easy <ref:2611.13732#pg1,from one recording of at most 10 seconds>.
Rosa: Exactly. The CIALLO paper shows that this shared action codec can be a very useful way to manage and transfer manipulation knowledge across different physical systems, provided you handle that initial calibration step correctly.
Choo Sin Wai, Dake Zhong, Jingye Zhang, Zhifang Liu, Siyao Xiao, Zirong Ding, Yuhong Zhang, Kangjie Chen, Dongbin Zhang, Guorui Song, Guocun Wang, Peng Jiao–Haoqian Wang
Tsinghua University
cs.RO
Submitted: 2026-11-10
Updated: 2026-11-10
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 100/100
The gist: Robot demonstrations contain reusable manipulation knowledge, but transferring this knowledge across embodiments requires representations that bridge differences in control interfaces…
Key concepts
- Action Codec
- A mechanism that maps different control signals and observations into a unified representation of intent. It takes actions, visual changes, and language and aligns them into a single shared space of short-horizon behavioral intent.
- Shared Intent Space
- A common representation where actions, visual changes, and language are mapped together. This space is independent of the specific robot's control interface or physical body. It allows knowledge learned on one robot to be understood by a policy on another robot.
- Embodiment-Conditioned Decoder
- The component that takes the shared intent representation and translates it back into actions specific to a particular robot. This decoder uses embodiment-specific context, such as the essential vector and arm embeddings, to generate the correct control signals for that physical system.
Terminology
Summary
Robot demonstrations contain reusable manipulation knowledge, but transferring this knowledge across embodiments requires representations that bridge differences in control interfaces <ref:2611. p. 013732#pg2>. CIALLO introduces an action codec that aligns actions, visual changes, and observation-grounded language in a shared space of shorthorizon behavioral intent <ref:2611. p. 013732#pg6>. This shared representation supports different control interfaces via an embodiment-conditioned decoder <ref:2611. p. 013732#pg6>.
CIALLO Architecture and Alignment
CIALLO aligns actions, visual changes, and observationgrounded language in one space by using three encoders to map a demonstrated chunk into a shared intent space <ref:2611.13732#pg6>. The action encoder uses the essential vector E to map control signals into shared intent, incorporating context from the proprioceptive encoder <ref:2611. p. 313732#pg5>. The decoder reconstructs actions from all three representations, preserving control information needed for execution <ref:2611. p. 013732#pg6>. The essential vector E, arm and axis embeddings, and execution times supply the embodiment-specific context <ref:2611. p. 013732#pg6>.
Training Objectives and Pretraining
The training objective combines masked action and text reconstruction losses with symmetric InfoNCE losses to align the three modalities <ref:2611.13732#pg5>. The complete training objective is defined as L = X m∈[a,v,t] L m act + Ltext + 0. p. 41La↔v + 0. p. 4 p. 405La↔t + 0. p. 4 p. 405Lv↔t <ref:2611. p. 0 p. 013732#pg5>. CIALLO is pretrained on eight embodiments with balanced sampling across them <ref:2611. p. 513732#pg9>. This pretraining helps the shared codec represent a broader range of actions, which explains lower reconstruction errors even on original embodiments <ref:2611. p. 513732#pg9>.
Cross-Embodiment Transfer Mechanism
To adapt CIALLO to a new robot, one first calibrates its essential vector and arm and axis embeddings while keeping the shared network fixed <ref:2611.13732#pg6>. The calibrated codec is then frozen while a policy learns to predict intent from observations and instructions <ref:2611. p. 013732#pg6>. This separation of embodiment calibration from task learning is key to the method <ref:2611. p. 013732#pg6>.
Policy Learning and Performance
Qwen-CIALLO, using a Qwen3.5-4B backbone, predicts intent from observations and instructions, supervised by the frozen action encoder <ref:2611. p. 113732#pg7>. With 20 demonstrations per task, Qwen-CIALLO achieves 76. p. 07% average success across real SO-101 and single-arm Piper robots <ref:2611. p. 013732#pg5>. The results support a shared action codec as a transferable control interface that separates embodiment calibration from task learning <ref:2611. p. 013732#pg6>.
Key Findings on Efficiency and Robustness
The CIALLO action decoder uses a distribution-based regression readout that predicts softmax weights over 256 ordered bins in a single forward pass <ref:2611.13732#pg15>. CIALLO achieves lower latency than flow-matching baselines, taking 4. p. 289 ms compared to 19. p. 088 ms for π0. p. 05. Qwen-CIALLO attains 97. p. 63% average success across the four LIBERO suites, the highest average in Table 2 <ref:2611. p. 6 p. 613732#pg5>. The method shows that greater embodiment diversity helps CIALLO learn shared crossembodiment intent representations <ref:2611. p. 013732#pg9>.
Conclusion and Future Work
CIALLO presents an action codec that separates shared behavioral intent from embodiment-specific control <ref:2611.13732#pg10>. Future work will align human egocentric videos with humanoid actions in the shared intent space and scale CIALLO pretraining to larger, more diverse datasets <ref:2611. p. 8 p. 813732#pg10>. The findings support separating embodiment calibration from task learning <ref:2611. p. 8 p. 813732#pg6>.
LLM Usage Statement
We used OpenAI Codex for language editing, manuscript organization, LATEX formatting, and summarizing experiment records provided by the authors for presentation in the manuscript <ref:2611.13732#pg10>. We did not use generative AI tools to design the methodology, write or modify experimental code, run experiments, or generate experimental data or results <ref:2611. p. 813732#pg10>. We reviewed all work produced with AI assistance <ref:2611. p. 813732#pg10>. The authors checked text produced with AI assistance against the manuscript plan and original experiment records, verified citation metadata against primary sources, and compiled and visually inspected the affected pages <ref:2611. p. 813732#pg10>. We take responsibility for the final content of this work, including text, claims, or artifacts produced with the aid of generative AI <ref:2611. p. 813732#pg10>.
References
[4] Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Maria Elisabeth Bechtle (Page 12).
[5] Qingwen Bu et al. p. 8 Learning to act anywhere with task-centric latent actions (Page 12). p. 8
[6] Boyu Chen et al. p. 8 UniT: Toward a unified physical language for human-to-humanoid policy learning and world modeling (Page 13). p. 8
[7] Tianxing Chen et al. p. 8 RoboTwin 2. p. 00: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation (Page 13). p. 8
[8] Zesen Cheng et al. p. 8 Breaking the memory barrier of contrastive loss via tilebased strategy (Page 16). p. 8
[9] Cheng Chi et al. p. 8 BERT: Pre-training of deep bidirectional transformers for language understanding (Page 9). p. 8 p. 8
[10] Tri Dao et al. p. 8 FlashAttention: Fast and memory-efficient exact attention with IO-awareness (Page 10). p. 8
[11] Jacob Devlin et al. p. 8 BERT: Pre-training of deep bidirectional transformers for language understanding (Page 9).
[12] Ria Doshi et al. p. 9 Scaling cross-embodied learning: One policy for manipulation, navigation, locomotion and aviation (Page 9). p. 9
[13] Senyu Fei et al. p. 9 LIBERO-Plus: A progressive robustness benchmark for visual-languageaction models (Page 10). p. 9
[14] Dibya Ghosh et al. p. 9 Octo: An open-source generalist robot policy (Page 10). p. 9
[15] Ankit Goyal et al. p. 9 RVT: Robotic view transformer for 3D object manipulation (Page 12). p. 9
[16] Kerui Gu et al. p. 9 Bias-compensated integral regression for human pose estimation (Page 12). p. 9
[17] Yanjiang Guo et al. p. 9 Ctrl-World: A controllable generative world model for robot manipulation (Page 13). p. 9
[18] Danijar Hafner et al. p. 9 Dream to control: Learning behaviors by latent imagination (Page 9). p. 9
[19] Xun Huang and Serge Belongie. p. 9 Arbitrary style transfer in real-time with adaptive instance normalization (Page 13). p. 9
[20] Stephen James et al. p. 9 Coarse-to-Fine Q-attention: Efficient learning for visual robotic manipulation via discretisation (Page 13). p. 9
[21] Moo Jin Kim et al. p. 9 Fine-tuning vision-language-action models: Optimizing speed and success (Page 13). p. 9
[22] Moo Jin Kim et al. p. 9 OpenVLA: An open-source vision-language-action model (Page 9). p. 9
[23] Seungjae Lee et al. p. 9 Behavior generation with latent actions (Page 10). p. 9
[24] Xuanlin Li et al. p. 9 Evaluating realworld robot manipulation policies in simulation (Page 10). p. 9
[25] Yue Liao et al. p. 9 Genie Envisioner: A unified world foundation platform for robotic manipulation (Page 13). p. 9
[26] Bo Liu et al. p. 0 LIBERO: Benchmarking knowledge transfer for lifelong robot learning (Page 13). p. 9
[27] Songming Liu et al. p. 9 RDT-1B: a diffusion foundation model for bimanual manipulation (Page 14). p. 9
[28] Zhuang Liu et al. p. 9 A ConvNet for the 2020s (Page 14). p. 9
[29] Shangchen Miao et al. p. 9 JEPA-VLA: Video predictive embedding is needed for VLA models (Page 15). p. 9
[30] Aiden Nibali et al. p. 9 Numerical coordinate regression with convolutional neural networks (Page 14). p. 9
[31] NVIDIA et al. p. 9 GR00T N1: An open foundation model for generalist humanoid robots (Page 14). p. 9
[32] Open X-Embodiment Collaboration et al. p. 9 Open X-Embodiment: Robotic learning datasets and RT-X models (Page 14). p. 8
[33] Maxime Oquab et al. DINOv2: Learning robust visual features without supervision (Page 15). p. 4
[34] William Peebles and Saining Xie. p. 9 Scalable diffusion models with transformers (Page 15). p. 8
[35] Ethan Perez et al. p. 8 FiLM: Visual reasoning with a general conditioning layer (Page 15).
[36] Karl Pertsch et al. p. 8 FAST: Efficient action tokenization for vision-language-action models (Page 14). p. 0
[37] Physical Intelligence, Kevin Black et al. p. 8 π0.5: a vision-language-action model with open-world generalization (Page 15). p. 9
[38] Qwen Team. p. 0 Qwen3.5: Towards native multimodal agents (Page 9). p. 2
[39] Alec Radford et al. Learning transferable visual models from natural language supervision (Page 9). p. 1
[40] Dominik Schmidt and Minqi Jiang. p. 9 Learning to act without actions (Page 12). p. 8
[41] Mohit Shridhar et al. PerceiverActor: A multi-task transformer for robotic manipulation (Page 12). p. 9
[42] Haoming Song et al. p. 0 Hume: Introducing system-2 thinking in visual-language-action model (Page 13). p. 2
[43] StarVLA Community. p. 1 StarVLA: A Lego-like codebase for vision-language-action model developing (Page 10). p. 9
[44] Jiaming Sun et al. LoFTR: Detector-free local feature matching with transformers (Page 12).
[45] Xiao Sun et al. p. 0 Integral human pose regression (Page 12). p. 9
[46] Lirui Wang et al. p. 0 Scaling proprioceptive-visual learning with heterogeneous pre-trained transformers (Page 13). p. 8
[47] Junjie Wen et al. TinyVLA: Toward fast, data-efficient vision-language-action models for robotic manipulation (Page 13). p. 9
[48] Pei Yang et al. p. 8 ActionMap: Robot policy learning via voxel action heatmap (Page 12). p. 8
[49] Seonghyeon Ye et al. p. 9 Latent action pretraining from videos (Page 12). p. 8
[50] Shanglin Yuan et al. p. 9 DreamWAM: Beyond RGB future prediction for world action models (Page 13). p. 1
[51] Tianyuan Yuan et al. p. 9 Fast-WAM: Do world action models need test-time future imagination? p. 1 (Page 14).
[52] Chubin Zhang et al. p. 0 CLAP: Contrastive latent action pretraining for learning visionlanguage-action models from human videos (Page 14). p. 8
[53] Jingye Zhang et al. p. 0 SURE: Spatiotemporal uncertainty representation for efficient visionlanguage-action learning (Page 14). p. 2
[54] Tony Z. Zhao et al. p. 8 Learning fine-grained bimanual manipulation with low-cost hardware (Page 18). p. 8
--- Page 1 ---
CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation
Choo Sin Wai, Dake Zhong, Jingye Zhang, Zhifang Liu, Siyao Xiao, Zirong Ding, Yuhong Zhang (Page 1), Equal contribution. †Corresponding author <ref:2611. p. 013732#pg2>.
Abstract Robot demonstrations contain reusable manipulation knowledge <ref:2611. p. 013732#pg6>. Action reconstruction alone does not ensure that similar behaviors across robots share a representation <ref:2611. p. 0 p. 013732#pg6>. To address this gap, we introduce CIALLO, an action codec that aligns actions, visual changes, and observation-grounded language in a shared space of shorthorizon behavioral intent <ref:2611. p. 0 p. 013732#pg6>. An embodiment-conditioned decoder maps this intent back to robot-specific actions <ref:2611. p. 013732#pg6>.
Pruning redundant reasoning tokens
Robot demonstrations contain reusable manipulation knowledge, but transferring this knowledge across embodiments requires representations that bridge differences in control interfaces <ref:2611.13732#pg6>. Action reconstruction alone does not ensure that similar behaviors across robots share a representation <ref:2611.13732#pg6>. To address this gap, we introduce CIALLO, an action codec that aligns actions, visual changes, and observation-grounded language in a shared space of shorthorizon behavioral intent <ref:2611.13732#pg6>.
How the training signal is built
The training signal is built by combining visual and textual intent with action reconstruction losses and symmetric InfoNCE losses <ref:2611.13732#pg5>. The complete training objective is L = X m∈[a,v,t] L m act + Ltext + 0. p. 41La↔v + 0.05La↔t + 0.05Lv↔t <ref:2611.13732#pg5>. The shared decoder makes alignment functional rather than purely geometric <ref:2611. p. 413732#pg5>.
Where the gains come from
The gains come from pretraining CIALLO on eight embodiments, which helps the shared codec represent a broader range of actions <ref:2611.13732#pg9>. The results show that training on more embodiments and a wider range of demonstrations helps the shared codec represent a broader range of actions <ref:2611. p. 513732#pg9>. The policy learning stage involves calibrating the new robot's essential vector and arm and axis embeddings while keeping the shared network fixed <ref:2611.13732#pg6>. The results show that training on more embodiments and a wider range of demonstrations helps CIALLO learn shared crossembodiment intent representations <ref:2611. p. 513732#pg9>.
Policy Success Metrics
Qwen-CIALLO achieves 97.3% average success across real SO-101 and single-arm Piper robots <ref:2611. p. 013732#pg5>. Qwen-CIALLO has the highest camera, robot, background, and layout success rates among listed methods <ref:2611. p. 613732#pg9>. The results show that a single policy trained through the shared intent interface can handle diverse manipulation tasks <ref:2611. p. 713732#pg5>. The results show that Qwen-CIALLO attains 97. p. 63% average success across the four LIBERO suites, the highest average in Table 2 <ref:2611.13732#pg5>.
Adaptation to New Embodiments
When transferring to a new robot, one first calibrates its essential vector and arm and axis embeddings while keeping the shared network fixed <ref:2611.13732#pg6>. The results show that Qwen-CIALLO achieves 76. p. 77% average success, compared with 70. p. 00% for π0 (Table 5) <ref:2611. p. 713732#pg5>. The results show that Qwen-CIALLO transfers the intent space of a separately pretrained codec and uses the target demonstrations to train only Qwen3. p. 75-4B and its intent projector to select intents for the frozen, calibrated codec <ref:2611. p. 713732#pg9>.
Action Head Efficiency
CIALLO uses 19.89M parameters, approximately 7. p. 16× fewer than OFT and 21. p. 66× fewer than the π0. p. 05 action head. p. 2 CIALLO decodes an action chunk in 4. p. 289 ms, showing that the implementation framework materially affects latency even with an unchanged architecture <ref:2611.13732#pg6>. The single-pass expectation readout avoids the iterative action refinement used by flow matching <ref:2611. p. 213732#pg6>.
Real Robot Tasks
The evaluation on SO-101 and single-arm Piper shows that QwenCIALLO achieves 85% success on SO-101 and 80% success on Piper <ref:2611.13732#pg5>. The results show that QwenCIALLO achieves higher success, reaching 70% versus 35% on SO-101 and 40% versus 20% on Piper <ref:2611. p. 713732#pg9>. The results show that the current evaluation measures overall robustness and does not separate these two possible contributions <ref:2611. p. 713732#pg9>.
Conclusion
CIALLO presents an action codec that separates shared behavioral intent from embodiment-specific control <ref:2611.13732#pg10>. Future work will align human egocentric videos with humanoid actions in the shared intent space and scale CIALLO pretraining to larger, more diverse datasets <ref:2611.13732#pg10>. The findings support separating embodiment calibration from task learning <ref:2611.13732#pg6>.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
A shared action codec allows policies to
consume heterogeneous data
by mapping actions, visual changes, and observation-grounded language into ashared space of shorthorizon behavioral intent.
This enables cross-embodiment learning by separatinga shared representation of short-horizon behavior, intent, from the body and control schema used to realize it.
-
The system can adapt to new robots with minimal data by fitting only the embodiment parameters, as it allows one to
adapt to a new robot from one recording of at most 10 seconds by updating only its embodiment embeddings.
This is achieved by keeping thecalibrated codec frozen while a policy learns to predict intent from observations and instructions.
-
The Qwen-CIALLO architecture enables policy learning without requiring
large-scale robotaction pretraining before policy finetuning
for the backbone, as it uses a frozen action encoder that providesteacher tokens from the demonstrated actions
to supervise intent prediction.
Sources
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- SSR: Scaling Surefooted and Symmetric Humanoid Traversal Toward the Open World