WorldAlign: Decoupled 4D Reward for World-Consistent Video Generation

arXiv:2610.12382 · cs.CV · Submitted 2026-10-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "WorldAlign: Decoupled 4D Reward for World-Consistent Video Generation".

Jane: The gist The introduction states that faithful visual world simulation requires 4D world consistency, encompassing both static and dynamic consistency, which is essential for applications like robotics and autonomous driving.

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we're looking at this paper called "WorldAlign: Decoupled 4D Reward for World-Consistent Video Generation <ref:2610.12382#pg1,WorldAlign: Decoupled 4D Reward for World-Consistent Video Generation>." It sounds like they're tackling the problem of making AI generate videos that look consistent over time, both in the static parts and the moving parts.

Jane: Yeah, that’s right. The authors, Jing He and his team at places like Tsinghua University and Hong Kong University of Science and Technology, are proposing a new way to handle this consistency using a decoupled reward framework.

Lu: They are essentially breaking the problem down into two separate consistency checks: static regions versus dynamic subjects, so they can align each one with a world prior that makes sense for its specific assumptions.

Meng: It’s interesting because most methods try to handle everything at once, but this approach suggests separating the geometry from the motion when we're training these video generators.

Lalam: This decoupling is significant because it lets them use different types of feedback for static things versus dynamic things, which should lead to much more precise results.

The paper's summary: Tom: So what’s the core idea here? Basically, WorldAlign introduces a "decoupled 4D reward framework" that semantically separates the static regions from the dynamic subjects in a generated video <ref:2610.12382#pg1>.

Jane: Exactly. For static areas, they align them with a geometric world prior using something called semantically guided masked reprojection to check for static consistency.

Lu: And they also use an auxiliary camera-motion reward there to discourage solutions that are almost completely still, which is something existing methods often struggle with when assuming a static scene.

Meng: For the dynamic subjects, they use a strong vision-language model or VLM as their world prior and create a "VLM-as-a-judge" reward based on specific checklists.

Lalam: This checklist assesses things like dynamicity, physical plausibility, shape, and texture consistency for the moving parts of the video.

Tom: So it’s not just checking if the whole video looks good; they are giving distinct feedback signals for static geometry and dynamic motion separately.

The paper's improvements: Jane: One big improvement they highlight is how WorldAlign handles static consistency by using a geometry foundation model to measure consistency through cross-view reprojection.

Lu: They specifically construct a shared point cloud using only the static regions, excluding the three dee points corresponding to the dynamic subject before doing that reprojection <ref:2610.12382#pg1>. This way, you penalize inconsistent static structures without accidentally penalizing legitimate subject motion or movement in the scene.

Meng: Then they add this auxiliary camera-motion reward, which acts like a hinge penalty if the predicted camera motion falls below some threshold tau, effectively discouraging solutions that are nearly static when there should be movement.

Tom: And for the dynamic parts, instead of just relying on motion cues, they use that VLM-as-a-judge reward based on those specific checklists—dynamicity, physical plausibility, shape and texture consistency.

Lalam: This means you get feedback that checks not only if the subject looks right but also if its movement makes sense in the real world according to what the VLM expects.

Conclusion: Jane: So to wrap up on "WorldAlign: Decoupled 4D Reward for World-Consistent Video Generation," they show that this decoupled design allows for more effective online post-training without needing human preference annotations at all <ref:2610.12382#pg1,WorldAlign: Decoupled 4D Reward for World-Consistent Video Generation>.

Tom: They tested it across two generators, Wan2 point 1 and Wan2 point 2, and found that WorldAlign jointly improves static and dynamic consistency over existing methods while keeping the overall motion of the subjects intact <ref:2610.12382#pg1>.

Lu: The results show that this approach works well for improving both reward-related metrics like Masked Epipolar Error and reward-independent metrics like Subject DINO similarity.

Meng: The ablation study confirmed that having both components—the static alignment and the dynamic VLM alignment—together leads to lower geometric errors and much stronger dynamic consistency compared to just using direct reprojection alone.

Lalam: It really demonstrates that by aligning each part with a world prior suited for its assumptions, you can achieve better visual simulation without suppressing motion in any of them.

Jing He, Kaixin Ding, Xingye Tian, Guibao Shen, Wenhang Ge, Xin Tao, Pengfei Wan

The Hong Kong University of Science and Technology (Guangzhou) · KlingAI

cs.CV

Submitted: 2026-10-08

Updated: 2026-10-08

Comments: Includes an appendix with implementation details, human evaluation, and additional ablation analysis. Project page: https://worldalign.github.io/

Project page: https://worldalign.github.io

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 78/100

The gist: The gist The introduction states that faithful visual world simulation requires 4D world consistency, encompassing both static and dynamic consistency, which is essential for applications like

Key concepts

Static Consistency
This refers to ensuring that the 3D structure of unchanging parts of a scene remains coherent across different viewpoints. WorldAlign achieves this by aligning static geometry with a geometric world prior using semantically guided masked reprojection, which checks if the static parts look correct from various angles.
Dynamic Consistency
This focuses on making subject motion plausible and consistent over time. For moving objects, WorldAlign uses a Vision-Language Model (VLM) as a dynamic world prior. It then assesses the generated video based on a checklist to ensure the movement is physically plausible, maintains correct shape, and has consistent texture.
Decoupled Reward Framework
Instead of one reward for the whole video, WorldAlign separates feedback into distinct components: Rstatic (for static regions), Rcam (for camera motion), and Rdyn (for dynamic subjects). This separation allows the model to learn specific improvements for each aspect independently, leading to more effective online post-training.
VLM-as-a-Judge Reward
For dynamic subjects, a strong Vision-Language Model is used as an automated judge. The VLM answers sample-specific questions about the generated video's dynamicity, physical plausibility, shape, and texture. The resulting reward measures how well the generated video satisfies these criteria based on the VLM's assessment.

Terminology

Summary

The gist The introduction states that faithful visual world simulation requires 4D world consistency, encompassing both static and dynamic consistency, which is essential for applications like robotics and autonomous driving.

WorldAlign Framework

WorldAlign introduces a decoupled 4D reward framework that semantically separates static regions and dynamic subjects to provide feedback by aligning each with a world prior suited to its assumptionsFigure 2 Overview of the proposed WorldAlign>. For static regions, WorldAlign aligns them with a geometric world prior through semantically guided masked reprojection, enabling more reliable static-consistency evaluation and uses an auxiliary camera-motion reward discourages nearly static solutions. This approach avoids the limitations of existing methods that often rely on a static-scene assumption.

Static World-Prior Alignment

The framework handles static consistency by using a geometry foundation model (GFM) to measure consistency through cross-view reprojection. To prevent motion artifacts from fusing the dynamic subject into the point cloud, WorldAlign constructs a shared point cloud using only static regions by applying the tracked mask to exclude 3D points corresponding to the dynamic subject. The resulting masked reprojection reward penalizes inconsistent static structures without penalizing legitimate subject motion. Furthermore, an auxiliary camera-motion reward is defined as a hinge penalty when the predicted motion falls below a threshold τ, thus penalizing insufficient viewpoint changes.

Dynamic World-Prior Alignment

For dynamic subjects, WorldAlign uses a strong vision-language model (VLM) as a dynamic world prior and constructs a VLM-as-a-judge reward based on sample-specific checklists that assess dynamicity, physical plausibility, shape, and texture consistency. This checklist consists of subject-centric binary questions covering four complementary aspects: Dynamicity, Physical plausibility, Shape, and Texture. The VLM receives the full generated video and the checklist Q to answer each question with YES or NO. The resulting reward Rdyn measures the fraction of checklist criteria satisfied by the generated video.

Online Policy Update

The framework optimizes the pretrained generator using online GRPO following Flow-GRPO. For each conditioning pair, WorldAlign computes three reward components Rstatic, Rcam, and Rdyn. These rewards are normalized within each sample group to obtain group-relative signals and are aggregated into an advantage Ai using the formula Ai = λsRbstatic(vi) + λcRbcam(vi) + λdRbdyn(vi).

Evaluation and Results

Experiments compare WorldAlign against state-of-the-art post-training methods across two pretrained generators, Wan2.1 and Wan2.2. Evaluation metrics include reward-related metrics like Masked Epipolar Error (M.Epi.) and Masked Reprojection Error (M.Reproj.), and reward-independent metrics such as Tracked-Subject DINOv3 Similarity (Subject DINO) and the Checklist “Yes” Rate. Results demonstrate that WorldAlign improves static geometry and dynamic consistency over existing methods without suppressing camera or subject motion. Human evaluation further confirms this, showing WorldAlign achieves win rates of 71.3%, 75.0%, and 76.0% for static consistency, dynamic consistency, and overall preference on Wan2.1-I2V-14B-480P. The ablation study confirms that the full design achieves both lower geometric errors and substantially stronger dynamic consistency compared to direct reprojection alone.

Conclusion

WorldAlign successfully decouples static and dynamic consistency by aligning each with a world prior suited to its assumptions. These rewards enable online post-training without architectural changes or human preference annotations, supporting more faithful visual world simulation through decoupled world-prior alignment. The framework shows that this approach is effective for improving both reward-related and reward-independent consistency metrics while preserving overall and subject motion.

How it works

WorldAlign introduces a decoupled 4D reward framework that semantically separates static regions and dynamic subjects to provide feedback by aligning each with a world prior suited to its assumptions. For static regions, WorldAlign aligns them with a geometric world prior through semantically guided masked reprojection, enabling more reliable static-consistency evaluation and uses an auxiliary camera-motion reward discourages nearly static solutions. This approach avoids the limitations of existing methods that often rely on a static-scene assumption.

Conclusion

WorldAlign successfully decouples static and dynamic consistency by aligning each with a world prior suited to its assumptions. These rewards enable online post-training without architectural changes or human preference annotations, supporting more faithful visual world simulation through decoupled world-prior alignment.

Improvements for AI systems

  1. Decoupled 4D Consistency Training: WorldAlign introduces a decoupled 4D reward framework that semantically separates static regions and dynamic subjects. This allows for simultaneous optimization of static geometry (aligned with a geometric world prior through semantically guided masked reprojection) and dynamic motion (aligned with a VLM-based world prior), enabling online post-training without human preference annotations.

  2. Geometry-Aware Static Consistency: By using masked reprojection, the system ensures that rewards are calculated only on static regions, which is defined as points assigned to static regions after excluding the subject’s motion trajectory from the shared point cloud, leading to a reward that penalizes inconsistent static structures without penalizing legitimate subject motion.

  3. Plausible Dynamic Consistency Assessment: WorldAlign employs a VLM-as-a-judge reward based on sample-specific checklists to assess dynamic consistency, specifically evaluating criteria like Dynamicity, Physical plausibility, and Texture consistency, providing feedback that assesses not only appearance preservation but also whether subjects move as intended and whether their motion is physically plausible.

  4. Robust Online Policy Updates: The system optimizes the generator using online GRPO where the advantage is a weighted sum of three normalized reward components: Rstatic, Rcam, and Rdyn. This allows for online post-training that can jointly improve static and dynamic consistency without suppressing overall or subject motion, as demonstrated by results showing improvements across both reward-related and reward-independent metrics.

  5. Motion Suppression Mitigation: The inclusion of an auxiliary camera-motion reward, defined as applying a hinge penalty when predicted motion falls below a threshold τ, explicitly discourages nearly static solutions to prevent the static collapse observed with masked reprojection alone.

Sources

Related papers