WarpHammer: Densifying Scene Warps with 3D Object Priors for Extreme View Synthesis
cs.CV
Submitted: 2026-06-30
Updated: 2026-09-26
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Projection-conditioned novel view synthesis (NVS) warps an explicit 3D reconstruction of the input view into the target camera and conditions a generator on the warped rendering.
Terminology
Abstract
Projection-conditioned novel view synthesis (NVS) warps an explicit 3D reconstruction of the input view into the target camera and conditions a generator on the warped rendering. This works well for small viewpoint changes but degrades sharply under large orbital motion: the warp becomes sparse around the orbited object, where hidden surfaces dominate the new view and mirror-like artifacts emerge, causing the generator to lose both pixel content and the implicit camera cue carried by the warp. We introduce WarpHammer, a training-free framework that resolves this failure mode by augmenting the warped scene with an explicit 3D reconstruction of the object obtained from a native 3D generative prior (e.g., SAM3D). The reconstructed object adds missing foreground surfaces and occludes background points that should no longer be visible, restoring both appearance and camera cues without fine-tuning the base model. The same explicit object representation further unlocks a capability current NVS pipelines do not support: incorporating auxiliary views of the object from sources outside the target scene, for example, a casual snapshot of a car paired with a manufacturer studio shot of the same model. We process the reference and auxiliary images jointly with a pretrained multi-view geometry foundation model, which predicts a unified point cloud that we fuse into the 3D object reconstruction. This yields substantially more faithful geometry than single-image reconstruction, without requiring user-provided camera poses for the auxiliary views. On five benchmarks, WarpHammer produces stable novel views at viewpoint deviations where strong baselines collapse, and is the first scene-level NVS method that can naturally fuse auxiliary, pose-unknown object views from an external source.
Sources
- Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
- MVGenMaster: Scaling Multi-View Generation from Any Image via 3D Priors Enhanced Diffusion Model
- ShapeNet: An Information-Rich 3D Model Repository
- VideoCrafter1: Open Diffusion Models for High-Quality Video Generation
- FlexWorld: Progressively Expanding 3D Scenes for Flexiable-View Synthesis
- SAM 3D: 3Dfy Anything in Images
- CAT3D: Create Anything in 3D with Multi-View Diffusion Models
- CameraCtrl: Enabling Camera Control for Text-to-Video Generation
- EX-4D: EXtreme Viewpoint 4D Video Synthesis via Depth Watertight Mesh
- RayZer: A Self-supervised Large View Synthesis Model
- LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias
- OrbitNVS: Harnessing Video Diffusion Priors for Novel View Synthesis
- Depth Anything 3: Recovering the Visual Space from Any Views
- 3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors
- CamCloneMaster: Enabling Reference-based Camera Control for Video Generation
- You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scale
- DINOv2: Learning Robust Visual Features without Supervision
- SAM 2: Segment Anything in Images and Videos
- Taming Video Models for 3D and 4D Generation via Zero-Shot Camera Control
- CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models