Robots that Collaborate: Sequential Asymmetric Imitation for Learning Coupled Robot Policies
summary
The gist
Collaborative mobile manipulation requires robots to coordinate with a partially observed partner while physically interacting through shared objects This is difficult because failures often arise…
In short
Sequential Asymmetric Imitation (SAI) learns how two robots can work together physically by training them sequentially without needing synchronized demonstrations or explicit communication. It trains Robot A first, then trains Robot B against Robot A's learned behavior, and finally refines both using targeted corrections near coordination failures. This method enables coupled multi-robot manipulation through a structured data curriculum.
Key concepts
- Sequential Asymmetric Imitation (SAI)
- A training strategy where Robot A learns from a single demonstration first, then Robot B learns against the resulting policy of Robot A, and finally, both are refined using sparse human interventions near coordination errors. This decomposes complex coordination into three asymmetric stages.
- Partner-masking Augmentation
- A technique used in Stage 1 where the visual identity of a compliant human partner is masked or replaced with randomized elements like Gaussian blur or Stable Diffusion Inpainting. This forces Robot A to focus on the actual task object and state rather than relying on the specific appearance of its supporting partner.
- Data Curriculum
- The structured sequence of training data used in SAI, moving from unilateral demonstrations (Robot A) to paired demonstration (Robot B|A), and finally to sparse interventions near failures. This curriculum is designed to systematically expose policies to increasingly realistic coordination challenges.
- Temporal History Encoding
- An architecture component using a GRU network with a history window of 30 steps to encode the sequence of past states. This allows the policy to distinguish between intermediate transport phases and final release states, which helps stabilize primitive execution and reduce premature actions like releasing an object too early.
Terminology used across episodes
This episode discusses
- Robots that Collaborate: Sequential Asymmetric Imitation for Learning Coupled Robot Policies · Paper Radio
- Constraining Streaming Flow Models for Adapting Learned Robot Trajectory Distributions
- TriPilot-FF: Coordinated Whole-Body Teleoperation with Force Feedback
- DINOv3
The paper
Robots that Collaborate: Sequential Asymmetric Imitation for Learning Coupled Robot Policies · Read on arXiv
Yincong Chen, Ranpeng Qiu, Zihao Li, Yanan Zhou
Zhejiang University of Technology · University of Sydney
Collaborative mobile manipulation requires robots to coordinate with a partially observed partner while physically interacting through shared objects. This is difficult because failures often arise not from poor local skills, but from mistimed waiting, yielding, pulling, releasing, or repositioning. We study this problem with two bimanual mobile manipulators coupled through rigid and deformable objects. We propose Sequential Asymmetric Imitation (SAI), a single-teleoperator curriculum for learning coupled multi-robot behaviors without synchronized dual-operator demonstrations or explicit inter-robot communication. SAI trains Robot A from unilateral demonstrations with a compliant human partner, trains Robot B against the deployed Robot A policy, and then refines Robot A using sparse interventions near coordination failures. This staged process exposes the policies to increasingly realistic partner behaviors, including delay, phase mismatch,insufficient yielding, and interaction conflict. Across real-world dual-robot manipulation tasks, SAI improves task success, phase synchronization, and partner-contingent yielding over independent imitation and curriculum-ablation baselines. These results suggest that physically coupled collaboration can be learned through the structure of the imitation curriculum, rather than through synchronized multi-operator demonstrations or explicit coordination mechanisms. More videos on project page:http://cyc0429.github.io/sai-project-page/
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: I'm Rosa.
Dev: And I'm Dev, here with Taro.
Rosa: We're looking at the paper: "Robots that Collaborate".
Dev: Here's the gist of the paper. Collaborative mobile manipulation requires robots to coordinate with a partially observed partner while physically interacting through shared objects This is difficult because failures often arise not from poor local skills, but from mistimed waiting, yielding, pulling, releasing, or repositioning We propose Sequential Asymmetric Imitation (SAI), a single-teleoperator curriculum for learning coupled multi-robot behaviors without synchronized dual-operator demonstrations or explicit inter-robot communication SAI trains Robot A from unilateral demonstrations with a compliant human partner, trains Robot B against the deployed Robot A policy, and then refines Robot A using sparse interventions near coordination failures.
Rosa: The Core Problem Addressed Dual-robot manipulation success requires more than two locally competent policies because each robot's motion changes the load, contact state, and feasible motion of the other Small timing errors can therefore produce large interaction forces, object deformation, loss of grasp, or task failure.
Dev: Independent policies often fail because each robot follows its own nominal motion phase, even when the partner is delayed, stalled, or out of position.
Rosa: The key coordination challenges include temporal phase coupling, partner-contingent yielding, and interaction conflict mitigation.
Title and authors: Rosa: So we're talking about this paper now, "Robots that Collaborate: Sequential Asymmetric Imitation for Learning Coupled Robot Policies." The title itself suggests they’re tackling how robots can work together when they have to interact physically through shared objects while one of them is partially observed.
Dev: Right, and the authors are Rosa and Dev. They're focusing on a specific way to learn this collaboration without needing two operators guiding both robots at the same time or any direct communication between them.
Rosa: Exactly. The implication here is that we don't need those synchronized demonstrations anymore to teach robots how to handle those tricky coordination issues, like waiting or yielding when the partner moves unexpectedly.
Dev: It sounds like they’re proposing a staged learning process where Robot A gets started first from a single human demonstration, then Robot B learns against that learned behavior before the system is refined with targeted corrections.
The paper's summary: Rosa: Basically, the core idea of this "Robots that Collaborate: Sequential Asymmetric Imitation for Learning Coupled Robot Policies" paper is to learn coupled multi-robot behaviors without needing synchronized dual-operator demonstrations or explicit inter-robot communication. They propose something called Sequential Asymmetric Imitation, or SAI.
Dev: It breaks the coordination learning down into three asymmetric stages: first, Robot A learns basic task execution from unilateral demonstrations with a compliant human partner; then, Robot B learns against the deployed Robot A policy; and finally, they refine Robot A using sparse interventions near coordination failures.
Rosa: That's what I mean. The paper says this staged process is key because it exposes the policies to increasingly realistic partner behaviors, including delay, phase mismatch, insufficient yielding, and interaction conflict.
Dev: And the main question they’re trying to answer is whether coupled dual-robot policies can actually be learned from single-teleoperator demonstrations without having that synchronized dual operator data or reinforcement learning or explicit inter-robot communication.
The paper's improvements: Rosa: Now, what they suggest as improvements is this entire curriculum structure itself. They hypothesize that inducing inter-robot coordination can happen by structuring the partner distribution seen during imitation learning. It shifts the training from just compliant support to a deployed learned partner, and finally to closed-loop robot-robot execution with targeted corrections.
Dev: That shift exposes the policies to realistic coordination errors, such as partner delay, phase mismatch, insufficient yielding, and interaction conflict. The paper shows that SAI actually improves task success and process-level coupling metrics across several real-world tasks like deformable spreading and rigid-object transport compared to just using independent imitation.
Rosa: And they also touched on the policy architecture itself. They use an ACT style action chunking backbone with temporal history encoding and a cascaded whole-body action head, which helps stabilize primitive execution by conditioning arm commands on the predicted base motion rather than having an independent flat head.
Dev: That architecture actually shows some specific wins, too. The authors found that the temporal history encoding reduces premature release in tasks like Laundry Collection from sixty-four point five percent down to sixteen point one percent because the policy can distinguish intermediate transport states from terminal release states.
Conclusion: Rosa: So to wrap up, this paper shows that Sequential Asymmetric Imitation successfully learns physically coupled dual-robot manipulation without needing synchronized dual-operator demonstrations or explicit inter-robot communication. The staged curriculum is effective because it exposes policies to those realistic coordination errors through the different stages.
Dev: It suggests that coordinated multi-robot behavior can emerge from structured imitation data, without needing reinforcement learning, explicit inter-robot communication, or dense synchronized demonstrations for every single interaction.
Rosa: I think the most important sentence is that Sequential Asymmetric Imitation trains Robot A from unilateral demonstrations with a compliant human partner, trains Robot B against the deployed Robot A policy, and then refines Robot A using sparse interventions near coordination failures.
Dev: Taro, what’s your take on these results? Does this work outside of the lab seem practical for real-world deployment?
Taro: Well, it seems like they assume each robot can learn useful standalone primitives and that failures can be corrected through meaningful single-agent interventions. The main limitation they flag is failure-mode coverage, because the Stage three intervention set is only about thirty percent of the baseline dataset size, meaning recovery hasn't fully saturated yet.
Rosa: That makes sense for real-world deployment concerns about robustness when things go seriously wrong. What does this mean for our next steps?
Taro: They suggest that more targeted interventions around rare delays and near-failure contact states might actually improve robustness, and they also note that the current A centric design might not fit tasks where both robots need simultaneous correction.
Dev: So we’re left with a method that learns coupling through data structure rather than direct communication or reinforcement learning, but it still has some gaps in covering every possible failure mode. That's what we’ve got for now on this paper, "Robots that Collaborate: Sequential Asymmetric Imitation for Learning Coupled Robot Policies."
More episodes
- 2608.13438-ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
- 2610.34593-Future Prediction as Training-Only Video Supervision for Pretrained VLA Policies
- 2610.35385-EmbodiedLab: Beyond Manipulation, Benchmarking Embodied Agents That Act to Know
- 2610.36378-Data-Driven Hamiltonian for Neural Optimal Control
- 2610.37275-A Heuristic for Scheduling Autonomous Vehicles at Intersections
- 2610.37288-Dynamic Coordination of Autonomous Vehicles at Intersections
- 2610.37632-Real-Time Coordination of Autonomous Vehicles at Intersections
- 2611.13779-Softness is something you also need! Why compliance helps control: information metrics of a passive walker with compliant ankle
- 2611.13411-Residual-Corrected Physics-Informed Neural Network KKL Observers with Input-to-State Stability Guarantees
- 2611.14488-Is Your Robot's Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies