StitchEdit: Stitching Depth Priors into Image Editors via Per-Layer Gradient Probing
cs.CV
Submitted: 2026-11-05
Updated: 2026-11-05
Project page: https://guoqincode.github.io/StitchEdit
License: http://creativecommons.org/licenses/by/4.0/
The gist: Text-guided image editing with diffusion models struggles with spatial edits (e.g.
Terminology
Abstract
Text-guided image editing with diffusion models struggles with spatial edits (e.g. manipulating objects, changing viewpoints, manipulating occlusions) that depend on the 3D structure understanding of a scene. Marigold and follow-up work demonstrate that a pretrained diffusion backbone already harbors strong geometric priors that can be activated for depth estimation, yet how to redirect these priors toward editing remains unresolved. Through per-layer gradient probing, we discover that the editing and depth objectives are compatible in early transformer layers but actively conflict in late layers, explaining why naive depth-conditioned editing either ignores geometry or degrades visual quality. We present StitchEdit, which first activates the backbone's geometric capability via a lightweight depth-denoising adapter, then selectively fine-tunes this adapter for editing using a probe-guided per-layer learning rate schedule that preserves depth knowledge where it helps and prunes it where it conflicts. On the proposed StitchEditBench, a benchmark of 150 geometry-dependent edits spanning spatial transformation, occlusion manipulation, and viewpoint change, StitchEdit consistently surpasses both instruction-based and depth-conditioned baselines on perceptual quality and geometric fidelity.
Sources
- InstructPix2Pix: Learning to Follow Image Editing Instructions
- DepthFM: Fast Monocular Depth Estimation with Flow Matching
- Lotus-2: Advancing Geometric Dense Prediction with Powerful Image Generative Model
- Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Prediction
- Prompt-to-Prompt Image Editing with Cross Attention Control
- Flow Matching for Generative Modeling
- DDT: Decoupled Diffusion Transformer
- Qwen-Image Technical Report
- Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers
- Depth Anything V2
- Make Geometry Matter for Spatial Reasoning
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs