Do Joint Audio-Video Generation Models Understand Physics?
cs.SD, cs.AI, cs.CV, cs.MM
Submitted: 2026-05-08
Updated: 2026-11-07
Code: https://github.com/ZijunCui02/AV-Phys-Bench
Terminology
Sources
- Cosmos World Foundation Model Platform for Physical AI
- VideoPhy: Evaluating Physical Commonsense for Video Generation
- VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation
- T2AV-Compass: Towards Unified Evaluation for Text-to-Audio-Video Generation
- SAVVY: Spatial Awareness via Audio-Visual LLMs through Seeing and Hearing
- Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- "PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Models
- T2VPhysBench: A First-Principles Benchmark for Physical Consistency in Text-to-Video Generation
- LTX-2: Efficient Joint Audio-Visual Foundation Model
- VABench: A Comprehensive Benchmark for Audio-Video Generation
- JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronization
- JavisDiT++: Unified Modeling and Optimization for Joint Audio-Video Generation
- Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation
- Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation
- OmniSonic: Towards Universal and Holistic Audio Generation from Video and Text
- Seedance 2.0: Advancing Video Generation for World Complexity
- OpenAI GPT-5 System Card
- From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and Generation
- SonicBench: Dissecting the Physical Perception Bottleneck in Large Audio Language Models
Related papers
- Speech-DRAME: A Framework for Human-Aligned Benchmarks in Speech Role-Play
- MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control
- Optimising Neural Speech Codecs for 300bps Communication using Reinforcement Learning
- Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement
- Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems
- Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stems