DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training

arXiv:2610.12468 · cs.RO, cs.CV · Submitted 2026-10-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training".

Dev: The gist: DREAMTRUE presents a multi-view, cross-embodiment robot world model that improves action following and interaction plausibility without additional real-robot training data collection.

Rosa: First, who's behind it and why it matters.

Paper summary: Rosa: We’re diving into the DreamTrue paper, which is a multi-view, cross-embodiment robot world model designed for action faithful and physically plausible video prediction without needing tons of new real-robot data.

Dev: The authors are tackling two major hurdles here: imprecise calibration that messes up what the robot actually does and limited coverage of unsuccessful interactions biasing predictions toward just successful outcomes.

Taro: They solve that by rendering action trajectories into image-space conditions and introducing offline geometric calibration to align these conditions with the target videos.

Rosa: That alignment step is key because it lets them train across different robot embodiments in Stage I, and then they use counterfactual post-training in Stage II to broaden interaction coverage.

Dev: In Stage II, they introduce a human-annotated video reward model to provide feedback on these predictions without needing paired ground-truth futures.

Taro: This model learns from annotations covering robot defects, object consistency, and interaction plausibility to guide the reinforcement learning post-training toward more physically plausible interaction outcomes.

Rosa: The results are strong: on AgiBot, DreamTrue attains a state of the art action following score of zero point eight seven seven two among evaluated methods for action following and reduces the human-assessed interaction defect rate from forty-eight point one two percent to six point two five percent <ref:2610.12468#pg3>

Dev: That’s a big drop in error, especially when you look at how much better the object defects are—dropping from thirty-one point eight eight percent to three point one two percent and interaction defects went down from forty-eight point one two percent to six point two five percent >

Taro: The implication is that we can build these models that not only follow the right motions but also predict interactions that are physically believable, which makes it a big step for autonomous systems.

Rosa: But they do flag a limitation: because they operate in image space for the action representation and the reward model, if things get occluded or you have limited views, the framework might just generate something that looks plausible but isn't actually physically correct.

Dev: That dependency on image space is a major constraint; it means physical plausibility guarantees aren't absolute when vision is incomplete.

Taro: And they also point out that the reward model’s performance really hinges on how good those human annotations are for spotting those defects in embodiment, object consistency, and interaction plausibility.

Conclusion: Rosa: We’re wrapping up on DreamTrue, focusing on how this robot world model aims at action faithfulness and physical plausibility without needing tons of new real-robot data.

Dev: Yeah, the authors tackle two main issues: imprecise calibration that messes up what the robot actually does and limited coverage of unsuccessful interactions biasing predictions toward just successful outcomes.

Taro: They solve that by doing offline geometric calibration to make sure the rendered actions match the target videos spatially, which helps fix those alignment errors.

Rosa: Right, so they use those aligned conditions to train a unified world model across different robot types in Stage I and then in Stage II, they introduce counterfactual post-training guided by a human-annotated defect dataset.

Dev: That reward model is crucial because it checks for defects in embodiment, object consistency, and interaction plausibility to guide the reinforcement learning post-training toward better outcomes.

Taro: It's interesting that this framework works well when you generalize across different robot embodiments and environments without needing extra fine-tuning.

Rosa: The main result is that on the AgiBot benchmark, DreamTrue gets a state of the art action following score of zero point eight seven seven two among evaluated methods for action following and reduces the human-assessed interaction defect rate from forty-eight point one two percent to six point two five percent <ref:2610.12468#pg3>

Dev: That’s a big drop in error, especially when you look at how much better the object defects are—dropping from thirty-one point eight eight percent to three point one two percent and interaction defects went down from forty-eight point one two percent to six point two five percent >

Taro: The implication is that we can build these models that not only follow the right motions but also predict interactions that are physically believable, which makes it a big step for autonomous systems.

Rosa: But they do flag a limitation: because they operate in image space for the action representation and the reward model, if things get occluded or you have limited views, the framework might just generate something that looks plausible but isn't actually physically correct.

Dev: That dependency on image space is a major constraint; it means physical plausibility guarantees aren't absolute when vision is incomplete.

Taro: And they also point out that the reward model’s performance really hinges on how good those human annotations are for spotting those defects in embodiment, object consistency, and interaction plausibility.

Rosa: So to wrap up this DreamTrue paper: Action-Faithful Robot World Model with Counterfactual Post-Training, they successfully integrated offline geometric calibration to improve spatial alignment and counterfactual post-training guided by human annotations to enhance interaction plausibility in robot world models >

Dev: It’s a robust mechanism for generating action faithful and physically plausible video predictions that helps us get a more accurate estimation of policy outcomes before we actually execute the actions.

Taro: This work gives us a tool that improves action following while significantly improving interaction quality, which makes it valuable for evaluating robot actions.

Rosa: That’s what it is. The DreamTrue paper shows how to use these specific techniques—offline geometric calibration and reward-guided post-training—to build a world model that actually understands physical plausibility better than just learning from successful demonstrations alone >

Dev: It opens up some interesting questions about how we can make these image-space models more robust when they encounter real-world visual noise and occlusion.

Junyan Li, Ruizhi Li, Yu Liu, Xiangshuo Liu, Mingchao Sun, Hongyu Pan, Mu Xu

NLPR Institute of Automation, Chinese Academy of Sciences (CASIA) · AMAP, Alibaba Group

cs.RO, cs.CV

Submitted: 2026-10-08

Updated: 2026-10-08

Comments: project page: https://brave-eai.github.io/DreamTrue; code: https://github.com/brave-eai/DreamTrue

Code: https://github.com/haosulab/MPlib

Project page: https://brave-eai.github.io/DreamTrue

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 90/100

The gist: The gist: DREAMTRUE presents a multi-view, cross-embodiment robot world model that improves action following and interaction plausibility without additional real-robot training data collection.

Key concepts

Offline Geometric Calibration
This process refines the camera parameters and robot mounting offsets by rendering the robot's model using recorded states. It uses constraints like Palign, Ptime, and Pview to find pixel correspondences between renderings and real videos, ensuring that the visual conditions used for training are spatially aligned with the target videos across different embodiments.
Unified World Model (Stage I)
This stage learns a single representation for actions across various robot types. It converts actions from different embodiments into a common image-space condition by rendering them into RGB images, depth maps, and masks. This unified action representation is then used to train a video generator that predicts future videos from multiple camera views simultaneously.
Counterfactual Post-Training
To improve interaction coverage, this technique modifies recorded robot trajectories by applying small perturbations to the final end-effector pose. This generates new, counterfactual actions. A human-annotated reward model then evaluates these modified actions based on predicted defects (like poor object consistency), guiding reinforcement learning to favor more physically plausible outcomes.
Embodied Video Reward Model
Since ground-truth future videos are unavailable, this model is trained using human annotations that label robot, object, and interaction defects. It learns to predict three defect types: L1 Embodiment consistency, L2 Object consistency, and L3 Interaction plausibility. This learned reward guides the post-training phase toward generating physically realistic interactions.

Terminology

Summary

The gist: DREAMTRUE presents a multi-view, cross-embodiment robot world model that improves action following and interaction plausibility without additional real-robot training data collection.

How it works

DREAMTRUE addresses two major obstacles in training robot world models: imprecise calibration which impairs action following, and limited coverage of unsuccessful interactions which biases predictions toward successful outcomes. To improve action following across embodiments, the framework introduces offline geometric calibration to align rendered action trajectories with target videos. This calibration involves rendering the robot's URDF model using recorded states and initial camera parameters, establishing pixel correspondences via Palign, Ptime, and Pview constraints to jointly refine camera parameters Θ and relative arm mounting offsets ξ. The optimized parameters are then used to render action trajectories, providing spatially aligned visual conditions for supervised training across robot datasets.

Stage I: Unified world model

Stage I focuses on learning action-conditioned prediction across different robot embodiments using a unified imagespace action representation. Actions from different embodiments are converted into image-space conditions by rendering the robot according to the action trajectory a1:T, producing RGB images Rv t, depth maps Dv t, and amodal masks Mv t. Gripper states and camera motion are also encoded into this representation through R˜v t and ¨ Pv t. The resulting action representation C is then incorporated into the video generator via a VACE conditioning branch, which produces residuals injected into the DiT layers to predict multi-view future videos.

Stage II: Counterfactual Post-Training with Embodied Video Rewards

Stage II broadens interaction coverage by introducing counterfactual post-training using an embodied video reward model. Counterfactual actions are constructed by applying an SE(3) perturbation to the final end-effector pose of a recorded trajectory and interpolating to a perturbed endpoint, then using inverse kinematics to obtain the modified joint sequence. Since paired ground-truth futures are unavailable, a human-annotated video defect dataset covering robot, object, and interaction defects is constructed to train an embodied video reward model. This reward model predicts defects in three dimensions: L1 Embodiment, L2 Object consistency, and L3 Interaction plausibility. The continuous reward Rk(ˆx) is defined as the negative defect-label probability, Rk(ˆx) = −p kϕ(ˆx), where lower predicted defect probabilities yield higher rewards. This feedback guides reinforcement learning post-training toward more physically plausible interaction outcomes.

Key Contributions and Results

The model achieves state-of-the-art performance on the AgiBot benchmark, attaining an nDTW score of 0.8772 among evaluated methods for action following. Furthermore, the model reduces the human-assessed interaction defect rate from 48.12% to 6.25% through reward-guided post-training. The framework is released alongside refined calibration results for 153,666 episodes across three datasets and a video defect annotation corpus covering 44.9K videos. Experiments show that the full model reduces object defects from 31.88% to 3.12% and interaction defects from 48.12% to 6.25% compared to Stage I. The model also demonstrates generalization across different robot embodiments and environments, with shared checkpoints generating rollouts on four datasets and generalizing without additional fine-tuning to unseen real-world scenes and an unseen robot embodiment.

Limitations

A limitation of the framework is that the action representation, generator, and VLM reward model all operate in image space, meaning that under occlusion or limited views, the framework may generate or reward visually plausible but physically incorrect interactions. This inherent dependency on image-space representations is a key constraint for the model's physical plausibility guarantees. The framework also requires substantial time and resources for collecting new real-robot trajectories with accurate calibration and broad coverage of failed interactions, though DREAMTRUE seeks to improve existing data quality without such collection. The model's performance relies on the quality of the human annotations used to train the reward model, as evidenced by its defect discrimination performance across L1, L2, and L3 dimensions.

Conclusion

DREAMTRUE successfully integrates offline geometric calibration to improve spatial alignment and counterfactual post-training guided by human annotations to enhance interaction plausibility in robot world models. This approach provides a robust mechanism for generating action-faithful and physically plausible video predictions, enabling more accurate estimation of policy outcomes before execution. The release of the calibration parameters and the defect annotation corpus further supports its utility in advancing embodied AI research. The framework's ability to maintain high action following while significantly improving interaction quality makes it a valuable tool for evaluating robot actions.

--- Page 1 ---

DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training

Junyan Li Ruizhi Li Yu Liu Xiangshuo Liu Mingchao Sun Hongyu Pan Mu Xu Lue Fan B Zhaoxiang Zhang NLPR Institute of Automation Chinese Academy of Sciences CASIA Amap Alibaba Group Equal contribution Co-Project Leads lue.fan zhaoxiang.zhang@ia.ac.cn

--- Page 2 ---

Stage I: Unified world model Stage II: Counterfactual Post-Training with Embodied Video Rewards Video evaluation Object consistency Appearance consistency Object persistence Interaction plausibility Contact plausibility Support stability Robot embodiment Structural integrity Motion consistency AgiBot-G1 Franka Piper AgileX UR5 Unseen Robot RoboMind DROID RoboTwin Unseen Real Data Shared checkpoint Unified model across embodiments and datasets Initial observation Counterfactual editing Expected: no grasp Object stays in rack 1 Replay & render 2 Dense matching 3 Joint refinement Observation Intrinsics & Extrinsics Arm offsets Recorded states Render Offline geometric calibration + Counterfactual post-training Embodied video reward model Multi-view observations Robot actions Joint modeling Multi-view future video Incorrect lift Floats in air Ph Stays in rack ysically correct interaction Unsupported suspension Spurious attachment

--- Page 3 ---

3.1 Problem Formulation and Overview Given initial multi-view observations x 1:V0, a task instruction l, and an action sequence a1:T, we model the distribution of future videos as x ˆ1:V 1:T ∼ pθ x 1:V 1:T x 1:V0, l, a1:T (3) where V is the number of camera views, T is the prediction horizon, and θ denotes model parameters. Our framework consists of offline geometric calibration and two training stages. We first refine camera parameters and optional robot mounting offsets to improve spatial alignment between robot renderings and recorded video frames. Stage I represents action sequences as robot renderings and trains a multi-view generator across embodiments for accurate action following. Stage II uses a learned video reward model to guide post-training toward more plausible robot–object interactions.

--- Page 4 ---

action conditions inconsistent with the training targets. Second, training data dominated by successful demonstrations provide limited coverage of missed grasps, slipping objects, and unsuccessful contact (Peng et al., 2026). Recent work identifies a tendency to predict successful outcomes even under actions that lead to failure. For example, a model may predict that an object rises with the gripper despite a missed grasp.

Improvements for AI systems

  1. Bold header: Offline geometric calibration for action condition alignment

This method refines camera parameters using image correspondences from existing recordings to improve spatial alignment between robot renderings and video frames, without dedicated calibration sequences. This ensures that action conditions must accurately represent the robot’s intended trajectory in the target video, which directly addresses a key obstacle in training.

  1. Bold header: Counterfactual post-training with embodied video rewards

This stage improves interaction plausibility by using feedback from human annotations: We construct a human-annotated video defect dataset covering robot, object, and interaction defects and use it to train an embodied video reward model. This allows the model to learn physically plausible interaction outcomes by guiding reinforcement learning post-training toward actions beyond recorded trajectories.

  1. Bold header: Unified multi-view action representation

The framework introduces a common format for actions: Actions are defined differently across robot embodiments, making it difficult to use the same action encoder across datasets, by rendering them into RGB images Rv t, depth maps Dv t, and amodal masks Mv t and encoding gripper states into R˜v t. This enables a single generator to predict future videos across different robot embodiments.

  1. Bold header: Action-conditioned prediction fidelity

The model achieves high fidelity by conditioning the video generation on the unified representation: We incorporate the action representation C into the generator through a VACE (Jiang et al., 2025b) conditioning branch to produce residuals injected into DiT layers, allowing it to generate action-faithful and physically plausible video prediction.

  1. Bold header: Policy outcome evaluation with reduced bias

The model demonstrates superior policy evaluation by showing that our model yields a linear fit that virtually matches the ideal diagonal (y = 1.032x−0.009), whereas baseline predictions exhibit a substantial upward shift, thus reducing optimistic hallucination in failed manipulations, leading to an outcome precision of 87.63%.

Sources

Related papers