DIDO: Distilling Interaction-Centric Dynamics into One-Step Denoising for World Action Models

arXiv:2609.15570 · cs.RO · Submitted 2026-09-14 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa, and with me are Dev and Taro, guest researcher.

Dev: Today's paper: "DIDO: Distilling Interaction-Centric Dynamics into One-Step Denoising for World Action Models".

Rosa: The gist The authors propose DIDO,

Dev: First, who's behind it and why it matters.

Title and authors: Rosa: Let's start with who wrote this, which is Jing Lyu, Shuanghao Bai, Runze Xiao, Zhenyu Liao, Wenxing Tan, Zihan Tang, Ruochuan Shi—and Cheng Peng and Yuheng Ji—all from the Institute of Automation at Chinese Academy of Sciences.

Dev: And they’re also pulling in some contributors from BAAI and Xi’an Jiaotong University. It shows a really strong collaboration between different AI groups on this kind of vision-action modeling work.

Taro: When you see authors coming from so many places, it suggests they're trying to build a model that works robustly across different kinds of visual data, not just one specific lab setup.

Rosa: Right, and the title itself really tells you the core idea: distilling interaction-centric dynamics into one-step denoising for World Action Models. It’s about taking complexity and simplifying it while keeping the important stuff.

Dev: It’s about taking something that takes several steps to converge visually and forcing it to learn that convergence in a single shot, but making sure that what they distill is the manipulation interaction itself.

Taro: So, if I'm hearing this right, they are trying to solve the problem where you need fast control loops, but you can’t just stop the video prediction early because that kills the dynamic interaction information.

Rosa: That’s it precisely. They argue that naively truncating a multi-step model loses the interaction-centric dynamics most critical for manipulation, and this paper introduces DIDO to recover those specific dynamics in one step.

Dev: So, we’re looking at a method that combines distillation with some kind of focused reasoning to keep the fine details of how the object and gripper move together sharp.

The paper's summary: Rosa: The paper summarizes DIDO by explaining that they are using distribution matching distillation combined with interaction-centric representation guidance to achieve this goal. It’s a two-stage process, Stage I and Stage II.

Taro: I want to know more about that two-stage setup, because if it's just one step, how do they ensure the model still has enough information to make sense of what’s happening in the video?

Dev: Stage I seems to be where they do the heavy lifting of distillation—transferring the converged dynamics from a multi-step model—and grounding that distilled knowledge with specific tokens for objects, grippers, and alignment.

Rosa: They introduce four interaction-centric tokens: Tinter, which stands for Target object, Tint for gripper-object interaction, Tgrip for the gripper itself, and Talign for object-centric visual alignment.

Taro: Those specific tokens sound important because they seem to be explicitly telling the model where to pay attention in the video frames related to the physical action.

Dev: Stage I uses trajectory supervision on those gripper and object branches, predicting future bounding boxes for both, and an alignment loss that ties them to features from a pre-trained DINOv3 encoder.

Rosa: In Stage II, they take this distilled single-step model and couple it with an action expert through a mixture-of-transformers architecture. This lets the policy directly access those representations without having to decode any future frames.

Taro: So, the whole point of Stage II is to make sure that even though we only have one step's prediction, the policy gets a rich enough signal for the action expert to make a good decision right away.

The paper's improvements: Dev: Now let’s talk about what they actually achieved, because the paper highlights some specific improvements over previous methods. They achieved an average success rate of ninety-nine point zero percent on LIBERO, seventy-six point six percent on LIBERO-Plus, and ninety-two point zero percent on RoboTwin <ref:2609.15570#pg3,99.0% on LIBERO, 76.6% on LIBERO-Plus, and 92.0>.

Rosa: That’s a solid performance across those different benchmarks, but the ablation study is where I think the real magic is in showing *why* it works so well. They found that the Dynamics-Based Token Refinement component was key to getting those high scores on LIBERO and LIBERO-Plus.

Taro: So what does that refinement actually do? Does it just smooth things out, or is it something more specific about how the dynamics are represented?

Dev: It’s about preserving fine-grained tokens in regions where the predicted dynamics are strong while compressing the rest, which helps maintain high quality in the action prediction. They even showed that reducing the conditioning context from three hundred twenty-three to one hundred ninety-one tokens improved performance significantly.

Rosa: That context reduction is interesting because it suggests that you don't need to feed the model every single piece of visual data if you focus on where the dynamics are changing most rapidly.

Taro: And what about speed? The paper reports that DIDO predicts an action chunk in three hundred eighty-four milliseconds end-to-end, which is a thirty-two percent latency reduction compared to the four-step teacher which took five hundred sixty-two milliseconds.

Dev: That latency reduction is a big deal for closed-loop control because it means the robot can react much quicker to things happening in real time. It really shows that this distillation approach keeps the efficiency of imaginationfree policies while adding this extra level of interaction awareness.

Conclusion: Rosa: So, to wrap up, DIDO successfully distills the converged dynamics from a multi-step video model into a single denoising step and pairs it with interaction-centric reasoning over the gripper and manipulated object.

Dev: It means they’ve found a way to keep the manipulation dynamics information even when you only have one step of prediction, which is crucial for making future plans in real-time systems.

Taro: I think what this really changes for autonomy is that we can move closer to systems that are fast enough for direct physical control, because they aren't waiting on multiple iterative refinement steps to figure out how the object will behave.

Rosa: Exactly. And while the paper shows great results, they do flag some limitations. They mention that relying on bounding-box supervision might not be enough when dealing with multi-object interaction or objects that can deform easily in the real world.

Dev: And they also noted that because of the three separate optimization phases involved in training, the final predictive capability still depends heavily on how good and how comprehensive their teacher model is.

Taro: So, even with this distillation technique, you still need a very high-quality starting point to get good results out of DIDO.

Rosa: That’s the summary of DIDO: it’s a powerful tool for making world action models more efficient and better at capturing the physical interactions that matter when robots are actually moving.

Institute of Automation, Chinese Academy of Sciences

cs.RO

Submitted: 2026-09-14

Updated: 2026-10-08

Comments: Preprint. 22 pages, 8 figures, 5 tables

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 88/100

The gist: The gist The authors propose DIDO, which distills converged dynamics of a multi-step video model into a single denoising step and augments them with interaction-centric reasoning to recover critical

Key concepts

World Action Models (WAMs)
These models use video generation techniques to predict what will happen visually in the future during a robotic task. They are used for planning and controlling robots by generating sequences of future frames based on current states, aiming to enable complex manipulation.
Iterative Denoising Latency
When these models generate video frames iteratively through denoising, each step adds delay. The paper found that the static background appears quickly, but the dynamic interaction between the robot's gripper and the object only becomes clear in later steps, making iterative denoising slow for real-time control.
Interaction-Centric Reasoning Tokens (Tinter)
These are specialized tokens introduced by DIDO to explicitly model the relationship between key elements: the target object, its interaction with the gripper, and alignment features. These tokens guide the model to focus on how these specific entities move together during manipulation.
Distribution Matching Distillation (DMD)
This technique is used in Stage I of DIDO to transfer the learned dynamics from a slow multi-step video model into a single step. It involves matching the distribution of future visual content while grounding the resulting distillation with specific manipulation-relevant tokens like object and gripper bounding boxes.

Terminology

Summary

The gist The authors propose DIDO, which distills converged dynamics of a multi-step video model into a single denoising step and augments them with interaction-centric reasoning to recover critical manipulation dynamics.

Motivation and Problem

World Action Models (WAMs) use video generation models to predict future visual dynamics for robotic manipulation, but iterative denoising introduces additional latency for closed-loop control <ref:2609.15570#pg2> The paper empirically finds that visual content converges at different rates during denoising, with static background structure forms early, whereas the gripper and manipulated object remain blurry after the first step, with their interaction dynamics emerging only through subsequent denoising <ref:2609.15570#pg3>. Consequently, naively truncating a multi-step video model to one step preserves scene structure but loses the interaction-centric dynamics most critical for manipulation <ref:2609.15570#pg4>.

DIDO Methodology

DIDO combines distribution matching distillation with interaction-centric representation guidance <ref:2609.15570#pg4>. The approach is structured in two stages, Stage I and Stage II <ref:2609.15570#pg5>.

(Stage I)

DIDO jointly combines distribution matching distillation (DMD) Yin et al. (2024b) with interaction-centric visual reasoning <ref:2609.15570#pg5>. DMD transfers the converged dynamics of a multi-step video model, while dedicated object, interaction, gripper, and alignment tokens explicitly ground the distilled dynamics in manipulation-relevant entities <ref:2609.15570#pg5>. The object and gripper tokens are supervised through future bounding-box trajectories to capture their spatial and temporal evolution, while the alignment tokens are aligned with target-object features from a pretrained DINOv3 encoder across multiple transformer layers <ref:2609.15570#pg5>.

(Stage II)

The resulting single-step world model is coupled with an action expert through a mixture-of-transformers architecture, allowing the policy to directly access its predictive representations without decoding future frames <ref:2609.15570#pg5>. In Stage II, the video-generation objective Lvideo Yuan et al. (2026) is retained together with the interaction-centric objectives <ref:2609.15570#pg5>.

Key Components

DIDO introduces four interaction-centric token groups Tinter = [Tobj; Tint; Tgrip; Talign], representing the target object, gripper-object interaction, gripper, and object-centric visual alignment, respectively <ref:2609.15570#pg5>.

  1. Trajectory Supervision: The gripper branch [Tgrip; Tint] predicts a sequence of Ha future bounding boxes ˆb grip = ˆb grip h for h=1, while the object branch [Tobj; Tint] predicts ˆb obj = ˆb obj h for h=1 <ref:2609.15570#pg6>. The interaction trajectory objective is Linter = λobjLbox ˆb obj, b obj + λgripLbox ˆb grip, b grip <ref:2609.15570#pg5>.

  2. Multi-Layer Visual Alignment: Talign is supervised with object-centric features extracted by a frozen DINOv3 encoder Simeoni et al. (2026) <ref:2609.15570#pg5>. The loss Lalign = 1/S X l∈S Pl T l align - FDINO 1, (5) <ref:2609.15570#pg5>.

  3. Dynamics-Based Token Refinement: This mechanism preserves fine-grained tokens in regions with strong predicted dynamics and compresses the remaining regions <ref:2609.15570#pg7>. For each cell r, we measure its predicted change as stk(r) = Vtk(r) - Vt0(r) squared (6) <ref:2609.15570#pg5>.

  4. Flow-Matching Action Generation: The action expert generates an action chunk of length Ha via flow matching <ref:2609.15570#pg9>.

Results and Analysis

DIDO achieves an average success rate of 99.0% on LIBERO, 76.6% on LIBERO-Plus, and 92.0% on RoboTwin <ref:2609.15570#pg4>. The component ablation study shows that Dynamics-Based Token Refinement achieves 99.0% and 76.6%, while reducing the conditioning context from 323 to 191 tokens and end-to-end latency by approximately 30 ms <ref:2609.15570#pg9>. DIDO predicts an action chunk in 384 ms end to end, compared with 562 ms for the four-step teacher, yielding a 32% latency reduction while improving LIBERO performance from 98.4% to 99.0% <ref:2609.15570#pg10>.

Limitations

DIDO has several limitations, including reliance on bounding-box supervision which may be insufficient for multi-object interaction or deformable objects <ref:2609.15570#pg19>. Furthermore, the training involves three separate optimization phases and the predictive capability remains dependent on the quality and coverage of the teacher <ref:2609.15570#pg19>.

Conclusion

DIDO distills converged multi-step dynamics into a single denoising step and augments them with interaction-centric reasoning over the gripper and manipulated object, achieving strong generalization in real-world manipulation <ref:2609.15570#pg11>. The results show that efficient predictive dynamics can retain manipulation-relevant interaction information, enabling WAMs to support fast and effective closed-loop control.

--- Page 1 ---

The gist The authors propose DIDO, which distills converged dynamics of a multi-step video model into a single denoising step and augments them with interaction-centric reasoning to recover critical manipulation dynamics.

How it works

World Action Models (WAMs) use video generation models to predict future visual dynamics for robotic manipulation, but iterative denoising introduces additional latency for closed-loop control <ref:2609.15570#pg2>. The paper empirically finds that visual content converges at different rates during denoising, with static background structure forms early, whereas the gripper and manipulated object remain blurry after the first step, with their interaction dynamics emerging only through subsequent denoising <ref:2609.15570#pg3>. Consequently, naively truncating a multi-step video model to one step preserves scene structure but loses the interaction-centric dynamics most critical for manipulation <ref:2609.15570#pg4>.

Improvements for AI systems

  1. Interaction-Centric Dynamics Recovery: DIDO distills the converged dynamics of a multi-step video model into a single denoising step and explicitly models interaction using supervised bounding-box visual reasoning tokens, allowing the system to recover interaction-centric dynamics that are otherwise missed by naive truncation.

  2. Enhanced Inference Efficiency: By combining distillation with interaction guidance, DIDO reduces inference latency significantly, achieving a 32% latency reduction compared to the four-step teacher and demonstrating that single-step future modeling retains much of the efficiency of imaginationfree policies.

  3. Robust Distribution Shift Handling: The system achieves superior robustness under perturbations, notably on LIBERO-Plus where DIDO improves over Faster-WAM by 6.4% under robot initialization and by 2.3% under both language and lighting shifts, showing it retains stronger manipulation-relevant predictive representations.

  4. Task-Specific Performance Gains: The component ablation study confirms that the final stage, Dynamics-Based Token Refinement, is crucial for performance, achieving the highest scores (99.0% on LIBERO and 76.6% on LIBERO-Plus), indicating that selectively compressing near-static regions is particularly beneficial under visual distribution shifts.

  5. Action Expert Grounding: The architecture utilizes a mixture-of-transformers where the action expert directly accesses its intermediate representations for action prediction without decoding and re-encoding future video, ensuring the policy is conditioned on rich, interaction-aware latent streams.

Sources

Related papers