Phase-aware video generation for physics-grounded dynamics and interactions

arXiv:2610.11791 · cs.CV · Submitted 2026-10-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Phase-aware video generation for physics-grounded dynamics and interactions".

Jane: The gist Generating physically plausible videos for solid–gas dynamics is challenging as different phases exhibit distinct dynamics yet remain coupled through physical interactions PAVG,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, looking at "Phase-aware video generation for physics-grounded dynamics and interactions" by Oua and colleagues, the paper successfully introduced PAVG as a dual-branch architecture that separates solid and gas dynamics while using spatiotemporal cross-attention to model their physical coupling.

Jane: The authors achieved better motion adherence, greater physical plausibility, and higher video quality when tested on single-phase and cross-phase scenarios compared to previous methods.

Lu: The real implication here is that by learning these physical trajectory priors directly from data instead of doing online simulation, we can have efficient and controllable video generation under external physical interventions.

Meng: For me, the practical impact is that this means AI can generate more realistic simulations for things like fluid dynamics or material movement in videos without needing a massive supercomputer running complex physics solvers for every frame.

Lalam: Lalam suggests that because the model learns these physical priors, it can be used to improve how we generate content across different domains where physical consistency matters.

Tom: The paper moves beyond just visual fidelity and shows that you can achieve something that respects the distinct dynamics of different phases while still modeling their interactions correctly.

Jane: It sets a standard for models trying to bridge the gap between beautiful visuals and actual physical behavior in complex scenes.

Conclusion: Tom: So we're wrapping up this look at "Phase-aware video generation for physics-grounded dynamics and interactions."

Jane: It’s really about how they managed to separate the movements of solids and gases while still making them interact physically in the video.

Lu: The authors, Oua and their team, used a dual-branch architecture to do that separation explicitly.

Meng: They lift the image into these phase-specific three dee point representations before predicting where they go next.

Lalam: Instead of just guessing motion like a standard generator does, this AI learns those physical paths directly from data.

Tom: So it's not simulating physics frame by frame; it's learning the underlying physical rules from seeing how things move in videos.

Jane: That’s right, and they do this by using spatiotemporal cross-attention to link the solid and gas predictions together.

Lu: It’s a pretty clever way to keep the dynamics of each phase distinct while still letting them influence each other spatially and temporally.

Meng: From an engineering standpoint, that separation means you don't have to build one massive solver for everything; you can handle the two different physics separately and then connect the results.

Lalam: And what this means for us is that we can create video content where the fluid flows look right, or the material deforms correctly, without needing a supercomputer running constant complex simulations.

Tom: It seems like they did a lot of heavy lifting by training this model on a large corpus of trajectories.

Jane: They showed that this approach leads to better motion adherence and way more physical plausibility in the final videos compared to other models we've seen.

Lu: The quantitative results across single-phase and solid-gas interaction settings showed PAVG performing best on all three metrics we tested.

Meng: So, it’s not just about making things look pretty; it's about making the movement physically consistent for the objects in the scene.

Lalam: And when you think about what this means for culture, it opens up ways to generate complex visual effects that feel genuinely real and dynamic.

Tom: It definitely sets a new benchmark for models trying to ground video generation in actual physical principles.

Jingfeng Oua, Kun Wangc, Rui Zhaoc, Jingwei Guand, Limin Wanga, Chao Dongb, *Xingyu Zeng

School of Computer Science, Nanjing University

cs.CV

Submitted: 2026-10-08

Updated: 2026-10-08

Comments: 31 pages, 5 figures, 9 tables, including appendix

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 83/100

The gist: The gist Generating physically plausible videos for solid–gas dynamics is challenging as different phases exhibit distinct dynamics yet remain coupled through physical interactions PAVG, a

Key concepts

Dual-Branch Architecture
This structure separates the modeling of solid dynamics from gas dynamics into two distinct branches. Each branch specializes in predicting the movement and behavior specific to its respective phase, ensuring that unique physical characteristics are accurately captured independently before they are combined.
Spatiotemporal Cross-Attention
This mechanism connects the separate solid and gas models to capture how they physically influence each other across both space (within a frame) and time (across frames). It allows the model to understand the coupling between phases, such as how a solid's motion affects the surrounding gas.
Phase-Aware Point-Trajectory Predictor
This component uses point-cloud diffusion branches for each phase. It predicts where points in a specific phase will go in the future by learning from data, rather than running complex physics simulations. This prediction is then used to guide the motion of a video generator.
Cross-Phase Interaction (CPI)
These modules are designed to link the solid and gas branches while maintaining their individual phase-specific dynamics. They use gated residual cross-attention to allow information flow between phases, ensuring that interactions are physically grounded without destroying the unique properties of either phase.

Terminology

Summary

The gist Generating physically plausible videos for solid–gas dynamics is challenging as different phases exhibit distinct dynamics yet remain coupled through physical interactions PAVG, a Phase-Aware Video Generator for solid–gas dynamics and interactions, employs a dual-branch architecture to explicitly model the distinct dynamics of solids and gases, while spatiotemporal cross-attention captures their physical interactions.

How it works

PAVG lifts the image to phase-specific 3D point representations and predicts their future trajectories, which are then injected into a pretrained video generator as motion controls. Instead of performing online physical simulation, PAVG learns physical trajectory priors directly from data. Its dual-branch architecture separately models solid and gas dynamics, while spatiotemporal cross-attention couples the branches to capture their interactions.

The framework involves a two-stage training strategy. Stage 2 mixes single-solid, single-gas, and paired solid–gas trajectories, applying Eq. 3 to present branches and independent noise at a shared τ. The local coupling model transfers solid-induced momentum to gas in the solid-affected region.

Phase-aware point-trajectory predictor

PAVG couples two phase-specific point-cloud diffusion branches with spatiotemporal attention. Spatial and temporal cross-phase interaction (CPI) modules connect the branches while preserving phase-specific dynamics. Each of L blocks applies spatial self-attention, spatial CPI, a feedforward network, temporal self-attention, and temporal CPI. The Cross-phase interaction (CPI) modules use gated residual cross-attention. Spatial CPI reads all solid points in the same frame; temporal CPI reads matching indices across frames after spatial mixing, not physical solid–gas correspondences.

Training and constraints

Stage 2 objectives include Ltraj = Pq∈s,g wqLq stage1 ws + wg, Lstage2 = Ltraj + λcouplingLcoupling. For solids, the deformation constraint reconstructs grid velocities from predicted point motion and recorded material state. For gas, the divergence loss imposes MantaFlow’s pressure-projection incompressibility constraint. Stage 2 includes Lcoupling = mean(f,p)∈omegacpl Huber (Eq. 3 to Eq. 13).

Experiments and results

Experiments demonstrate that PAVG achieves better motion adherence, greater physical plausibility, and higher video quality than other approaches. Quantitative evaluation shows that PAVG achieves the best results on all three metrics and settings, outperforming both interaction-capable baselines and phase-specific models on the single-phase subsets. Human preference studies show PAVG ranks first in SA and PC and second in VQ.

Conclusion

In this work, PAVG (Phase-Aware Video Generation), a framework for physics-grounded dynamics and interactions, explicitly separates the distinct dynamics of solids and gases through a dual-branch trajectory predictor, while spatiotemporal cross-attention captures the physical coupling between phases. By learning physical trajectory priors from the proposed largescale trajectory corpus, PAVG enables efficient and controllable video generation under external physical interventions. Experiments across single-phase and solid–gas interaction scenarios show consistent improvements in motion adherence, physical plausibility, and visual quality. This work provides a step toward more general physics-grounded video generation models capable of reasoning about heterogeneous physical dynamics and their interactions>

The gist Generating physically plausible videos for solid–gas dynamics is challenging as different phases exhibit distinct dynamics yet remain coupled through physical interactions.

How it works

PAVG lifts the image to phase-specific 3D point representations and predicts their future trajectories, which are then injected into a pretrained video generator as motion controls. Instead of performing online physical simulation, PAVG learns physical trajectory priors directly from data. Its dual-branch architecture separately models solid and gas dynamics, while spatiotemporal cross-attention captures their interactions.

Phase-aware point-trajectory predictor

PAVG couples two phase-specific point-cloud diffusion branches with spatiotemporal attention. Spatial and temporal cross-phase interaction (CPI) modules connect the branches while preserving phase-specific dynamics.

Improvements for AI systems

  1. Phase-aware trajectory prediction allows for physically plausible solid–gas interaction by using a dual-branch architecture to explicitly model the distinct dynamics of solids and gases, while spatiotemporal cross-attention captures their physical interactions. This enables the system to produce physically consistent responses across phases, which is superior to baseline methods that either move both phases together or leave dust stationary.

  2. Dual-stage training supports learning complex dynamics sequentially: Stage 1 learns phase-specific motion priors, and Stage 2 initializes PAVG from them and jointly learns single-phase motion and cross-phase interaction. This allows the model to preserve phasespecific motion characteristics while producing physically consistent responses by first establishing distinct dynamics before coupling them.

  3. The system can perform physics-grounded video generation of single-phase motion and solid–gas interactions by using a shared point-trajectory interface for physically controllable video generation. This means the AI can generate videos where the moving solid induces or alters the motion of surrounding gas, capturing complex physical coupling that standard models fail to achieve.

  4. Trajectory prediction evaluation metrics like vIoU, CD, and Corr enable precise refinement: PAVG achieves the best results on all three metrics and settings in trajectory prediction, indicating it produces trajectories with improved motion adherence, greater physical plausibility, and higher video quality.

  5. The system can generate high-fidelity videos by using a process that involves projecting predicted 3D trajectories into image space to guide a generator: PAVG lifts the image to phase-specific 3D point representations and predicts their future trajectories, which are then injected into a pretrained video generator as motion controls.

Sources

Related papers