Real2SAM2Real: Generative 3D Caches as Complementary Context for Video Diffusion

arXiv:2606.00299 · cs.CV, cs.AI · Submitted 2026-05-29 · Read on arXiv

cs.CV, cs.AI

Submitted: 2026-05-29

Updated: 2026-10-13

Project page: https://jiayi-wu-leo.github.io/real2sam2real

Terminology

• Consistency — 1× in this paper · explained in 3 episodes
• Decoupling — 1× in this paper · explained in 2 episodes
• Diffusion Model — 1× in this paper · explained in 9 episodes
• Diffusion Models — 1× in this paper · explained in 6 episodes
• Fidelity — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes

Sources

• SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpoints
• Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
• Depth Pro: Sharp Monocular Metric Depth in Less Than a Second
• SAM 3: Segment Anything with Concepts
• SAM 3D: 3Dfy Anything in Images
• 3DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generation
• LTX-Video: Realtime Video Latent Diffusion
• CameraCtrl: Enabling Camera Control for Text-to-Video Generation
• EX-4D: EXtreme Viewpoint 4D Video Synthesis via Depth Watertight Mesh
• ViPE: Video Pose Engine for 3D Geometric Perception
• Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material
• HunyuanVideo: A Systematic Framework For Large Video Generative Models
• Depth Anything 3: Recovering the Visual Space from Any Views
• Novel View Extrapolation with Video Diffusion Priors
• Motion-2-To-3: Leveraging 2D Motion Data for 3D Motion Generations
• Taming Video Models for 3D and 4D Generation via Zero-Shot Camera Control
• Wan: Open and Advanced Large-Scale Video Generative Models
• MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details
• Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attention
• Trajectory Attention for Fine-grained Video Motion Control

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models