Object-Aware Background-Controlled Editing via Weighted Velocity Guidance

arXiv:2609.06288 · cs.CV · Submitted 2026-09-05 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Object-Aware Background-Controlled Editing via Weighted Velocity Guidance".

Jane: Here's the gist of the paper. Training-free image editing steers diffusion or flow-matching generative models at inference time by modifying prompt-conditioned denoising velocities.

Tom: OAVC proposes a training-free framework that introduces object-level control into the velocity-integration process to achieve object-centric editing while preserving background stability.

Jane: Core Problem and Proposed Solution Existing velocity-based editors often apply prompt-induced residuals globally over the latent space, which can cause background drift and unstable object boundaries during multi-step integration.

Tom: The authors propose ObjectAware Velocity Control (OAVC), a training-free framework that decouples where semantic residuals are allowed to act from how they are injected into the dynamics by constructing a background-anchored reference interface and performing object-localized safe semantic injection.

Paper summary: Tom: So we’re looking at this paper called "Object-Aware Background-Controlled Editing via Weighted Velocity Guidance." It's about how to steer diffusion or flow matching models at inference time just by changing how fast they move when they're denoising, based on a prompt.

Jane: That sounds like it could be useful for making edits without having to retrain anything per image. The paper claims this training-free approach can introduce object-level control into that velocity integration process and achieve object-centric editing while keeping the background stable.

Lu: It tackles a problem where existing velocity editors apply prompt-induced residuals globally across the latent space, which means those small changes outside the target object can build up over several steps during editing, causing drift and unstable boundaries.

Meng: So if you’re trying to remove a car from a video frame, and you only want to change the car but keep everything else exactly as it was before, this sounds like it addresses that background drift issue directly.

Lalam: The paper proposes ObjectAware Velocity Control or OAVC, which decouples where the semantic residuals are allowed to act from how they are actually injected into the dynamics. They do this by building a background-anchored reference interface and then doing object-localized safe semantic injection to stabilize transitions near object boundaries using time-adaptive spatial weighting.

Tom: Decoupling where the changes happen from how they get in seems like a clever way to manage that trade-off between making an edit and keeping the scene consistent. It moves away from just letting the model figure it out everywhere at once.

Jane: Right, so instead of applying a general prompt correction across everything, they construct this reference interface first, which acts as a stable baseline for the background. Then they apply targeted changes only where appropriate using some kind of safe injection operator to smooth those transitions near the object edges.

Lu: The mechanism involves constructing that background-anchored reference interface under the source prompt to ensure a stable comparison between source and target velocities, while relaxing that alignment force inside the editable object itself.

Meng: I wonder how robust this is when we actually deploy it. If you use an arbitrary segmentation model for the support, does it still work well? We need something that doesn't break if the segmentation is imperfect at scale.

Paper summary: Lalam: The paper shows that they are not tied to a single base pipeline because the same where/how velocity-control principle can be transferred to another rectified flow based editor, which suggests OAVC is quite flexible. Also, it’s support-source agnostic; performance stays strong even when you swap out the default segmentation model for something derived from attention.

Tom: That's good news because it means we don't have to lock ourselves into one specific segmentation tool for this kind of control. But what about the actual editing quality? The results on the benchmarks are pretty compelling, right?

Jane: They show improvements in background preservation, structural fidelity, and boundary stability when testing on object-centric image and video benchmarks using rectified-flow backbones. Quantitatively, they say OAVC reduces structure distance from fourteen point one nine to four point one one on SD3 point 5 and boosts PSNR from twenty-six point six six decibels to thirty-two point six four decibels compared to the baseline DNAEdit paper’s method.

Lu: That drop in structure distance is significant, showing that this method really helps maintain the shape of things you aren't changing, like the background texture. But we have to remember that while OAVC significantly improves background preservation, its CLIP scores can sometimes be lower than global-steering baselines because it prioritizes localized semantic editing over maximizing similarity to the target prompt everywhere.

Meng: Lower CLIP scores mean it might look less 'perfect' compared to some other methods if you're just looking at overall text alignment, but that trade-off seems acceptable given the background stability gain. What about video?

Lalam: For video editing on DAVIS benchmarks, OAVC improved background PSNR by four point six six decibels and reduced background L1 error by thirty-nine point two percent relative to the UniEdit-Flow method. That reduction in L1 error is pretty concrete for temporal consistency.

Tom: So we're talking about cleaner boundaries and better temporal consistency in video edits, which is usually a tricky spot when you’re modifying motion or objects over time. What does this imply for someone just using these models to do simple edits?

Jane: It means that if you use a training-free system for editing, you can expect the things you aren't editing to stay much more stable across multiple steps, which should lead to less noticeable drift in the background. It makes the localized edits cleaner and less messy.

Paper summary: Lu: This framework provides a structured way to navigate that preservation versus editability trade-off by controlling where prompt-induced residuals accumulate and how they are injected during velocity integration, effectively removing target semantics that global steering tends to leak into non-target regions.

Meng: It sounds like the core contribution here is managing the leakage of prompt instructions into the background when you only intend to change a specific object. That’s something we worry about constantly in production deployments because unintended changes are a nightmare to debug and deploy.

Lalam: And they confirm that this control mechanism effectively removes those target semantics from leaking into non-target areas, which results in a larger foreground-background gap and a smaller leakage ratio. It’s focused precisely on keeping the edit localized.

Tom: So to wrap up this paper, "Object-Aware Background-Controlled Editing via Weighted Velocity Guidance," the authors have developed OAVC to introduce object control directly into how diffusion or flow matching models are steered during inference. They do this by using a background-anchored reference interface combined with an object-localized safe semantic injection process.

Jane: The core idea is that instead of letting prompt instructions affect the entire image, you explicitly constrain where those edits are allowed to happen and how they are applied to prevent background drift and boundary artifacts during multi-step integration.

Lu: This paper shows that this method improves background preservation, structural fidelity, and boundary stability on object-centric benchmarks like PIE-Bench and DAVIS while still keeping the editing localized. It also proves the where/how velocity control principle is transferable to other rectified flow based editors.

Meng: The main thing I'm seeing from a deployment angle is that it gives us a more reliable way to get clean edits without needing intensive per-image fine-tuning, provided we can get our segmentation support reasonably accurate. We have to keep an eye on those segmentation errors though because the paper mentions that under-segmentation limits edit coverage, and over-segmentation increases non-target drift.

Lalam: Exactly, so it's not a magic fix; it depends on the quality of the object support you feed it into. But overall, OAVC gives us a structured way to manage that preservation versus editability trade-off while maintaining effective localized editability across different tasks.

Conclusion: Tom: So we’re looking at this paper called "Object-Aware Background-Controlled Editing via Weighted Velocity Guidance." It’s about steering those diffusion or flow matching models at inference time by changing how fast they move, based on a prompt, to get object control while keeping the background stable.

Jane: That sounds like it could be useful for making edits without having to retrain anything per image. The authors propose this training-free framework that introduces object-level control into the velocity integration process.

Tom: Right, they decouple where the semantic changes are allowed to happen from how they actually get injected into the dynamics, which is key for stability.

Jane: They do this by building a background-anchored reference interface and then using object-localized safe semantic injection to stabilize transitions near object boundaries with time-adaptive spatial weighting.

Tom: So it’s basically a structured way to tell the model exactly where to apply the prompt's instructions without letting them mess up everything else.

Jane: And when you look at the results, they show improvements in background preservation and structural fidelity on object-centric image and video benchmarks like SD3 point five and DAVIS.

Tom: The numbers are pretty solid there, showing a reduction in structure distance from fourteen point one nine to four point one one on SD3 point five compared to the baseline DNAEdit method.

Jane: That improvement in structure distance is significant because it means the shape of things you aren't changing stays much better preserved during the edit.

Tom: But we also have to watch out for a trade-off they mention, where their CLIP scores can be lower than global steering baselines because they prioritize localized editing over making the whole image look perfectly like the target prompt.

Jane: That makes sense; you get better local control, but maybe you lose some of that overall global text alignment score.

Tom: It’s a choice between perfect overall similarity and precise object-centric editing, which is what they are pushing here.

Jane: This method gives us a concrete way to manage that preservation versus editability trade-off while maintaining localized editability across different tasks.

Tom: And it seems pretty flexible, because the authors show this velocity control principle can be transferred to other rectified flow based editors.

Jane: So what's next for these kinds of methods? We need to see if they can handle more complex scenarios than just one object at a time, like multi-object editing.

Carnegie Mellon University · Institute of Computing Technology, Chinese Academy of Sciences

cs.CV

Submitted: 2026-09-05

Updated: 2026-11-13

Comments: Code: https://github.com/wwj-code/OAVC

Code: https://github.com/black-forest-labs/flux

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 98/100

The gist: Training-free image editing steers diffusion or flow-matching generative models at inference time by modifying prompt-conditioned denoising velocities.

Key concepts

Velocity-based Editors
These existing editors apply prompt changes globally across the entire image latent space at every step of generation. This global application often causes unintended drift, leading to unstable object boundaries and a messy background when editing multiple things simultaneously.
Object-level Control (OAVC)
OAVC decouples where semantic edits are allowed from how they are injected into the model's dynamics. It uses an object support to define specific regions for prompt residuals, allowing targeted changes on foreground objects while preserving the background stability.
Background-Anchored Reference Interface
This stage constructs a stable reference interface based on the source prompt to ensure consistent velocity comparisons. It relaxes the alignment force within an editable object while softly suppressing background components that move parallel to the original flow.

Terminology

Summary

Training-free image editing steers diffusion or flow-matching generative models at inference time by modifying prompt-conditioned denoising velocities. p. 0 OAVC proposes a training-free framework that introduces object-level control into the velocity-integration process to achieve object-centric editing while preserving background stability. p. 0

Core Problem and Proposed Solution

Existing velocity-based editors often apply prompt-induced residuals globally over the latent space, which can cause background drift and unstable object boundaries during multi-step integration. The authors propose ObjectAware Velocity Control (OAVC), a training-free framework that decouples where semantic residuals are allowed to act from how they are injected into the dynamics by constructing a background-anchored reference interface and performing object-localized safe semantic injection. p. 0 This approach suppresses drift-inducing velocity components using a constrained injection operator and stabilizes transitions near object boundaries with time-adaptive spatial weighting. p. 0

Mechanism of Object-Aware Velocity Control

OAVC operates by controlling where prompt-induced residuals may accumulate using an object support, denoted as the where/how factorization: u t = W t ⊙ Γ t (Δv t; R t). Stage-1 constructs a background-anchored reference interface R t under the source prompt to ensure stable source–target velocity comparison, relaxing the source-alignment force inside the editable object while softly suppressing background components parallel to the source flow. p. 1 Stage-2 then applies this control by computing target semantic residuals on offset-aligned states and transforming them using safe injection operators that employ flow-orthogonal projection, boundary smoothing, and a scalar confidence gate. p. 5

Performance on Benchmarks

Experiments show that OAVC improves background preservation, structural fidelity, boundary stability, and temporal consistency on object-centric image and video benchmarks with rectified-flow backbones. Quantitatively, OAVC reduces structure distance from 14. p. 819 to 4.11 on SD3.5 and increases PSNR from 26. p. 766 dB to 32.64 dB compared to the baseline DNAEdit. p. 7 For video editing on DAVIS, OAVC improves BG-PSNR by 4. p. 866 dB and reduces BG-L1 by 39. p. 62% relative to UniEdit-Flow. p. 8

Analysis of Trade-offs and Robustness

The analysis clarifies the editability–preservation trade-off, noting that while OAVC substantially improves background preservation, its CLIP scores can be lower than global-steering baselines because it prioritizes localized semantic editing over globally maximizing target-prompt similarity. Furthermore, the framework is support-source agnostic; performance remains strong even when replacing the default SAM3 support with attention-derived supports, confirming that the contribution lies in object-aware velocity control rather than reliance on a specific segmentation model. p. 4 The method is robust to imperfect supports, as moderate errors are mitigated by foreground dilation or buffered by the ring support, showing that under-segmentation limits edit coverage while over-segmentation increases non-target drift. p. 3

Generalization and Future Scope

OAVC demonstrates feasibility for multi-object editing, where multiple object supports can be incorporated by summing the object-specific semantic residuals, allowing both foreground objects to change while preserving the non-target background. However, the framework is not designed for full-image transformations like global style transfer because its spatial selectivity vanishes when the support becomes nearly global. Future work will explore adaptive support discovery and refinement alongside multistep compositing to reduce sensitivity to segmentation errors.

Conclusion

OAVC successfully controls both where prompt-induced residuals accumulate and how they are injected during velocity integration by combining a background-anchored reference interface with object-localized safe semantic injection, leading to cleaner boundaries and stronger structural fidelity. This framework provides a structured way to navigate the preservation–editability trade-off while maintaining effective localized editability across various editing tasks. p. 0 The controlled injection mechanism effectively removes target semantics that global steering tends to leak into non-target regions, resulting in a larger FG-BG Gap and a smaller Leakage Ratio. p. 9

References

[1] Tim Brooks, Aleksander Holynski, and Alexei A Efros. 2022. InstructPix2Pix: Learning to Follow Image Editing <ref:2609. p. 006288#pg2> [1] Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, Jie Lei, Tengyu Ma <ref:2609.06288#pg2> [2] Nicolas Carion et al. 2023. Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models <ref:2609. p. 006288#pg2> [3] Yingying Deng et al. 2024. FireFlow: Fast Inversion of Rectified Flow for Image Semantic Editing <ref:2609. p. 906288#pg2> [4] Patrick Esser et al. 2024. Scaling rectified flow transformers for high-resolution image synthesis <ref:2609. p. 0 p. 006288#pg2> [5] Patrick Esser et al. 2024. Scaling rectified flow transformers for high-resolution image synthesis <ref:2609.06288#pg2> [6] Yiming Gong, Zhen Zhu, and Minjia Zhang. 2025. InstantEdit: Text-Guided Few-Step Image Editing with Piecewise Rectified Flow <ref:2609. p. 206288#pg2> [7] Amir Hertz et al. 2022. Promptto-Prompt Image Editing with Cross-Attention Control <ref:2609. p. 006288#pg2> [8] Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Probabilistic Models <ref:2609. p. 006288#pg2> [9] Zhangqi Jiang et al. 2026. GEditBench v2: A Human-Aligned Benchmark for General Image Editing <ref:2609. p. 406288#pg2> [10] Guanlong Jiao et al. 2025. UniEdit-Flow: Unleashing Inversion and Editing in the Era of Flow Models <ref:2609. p. 006288#pg2> [11] Xuan Ju et al. 2024. BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion <ref:2609. p. 206288#pg2> [12] Xuan Ju et al. 2023. Direct Inversion: Boosting Diffusion-Based Editing with 3 Lines of Code <ref:2609. p. 206288#pg2> [13] Alexander Kirillov et al. 2023. Segment Anything <ref:2609. p. 306288#pg2> [14] Vladimir Kulikov et al. 2025. Flowedit: Inversion-free text-based editing using pre-trained flow models <ref:2609. p. 006288#pg2> [15] Black Forest Labs. 2024. FLUX <ref:2609. p. 006288#pg2> [16] Hongyu Li et al. 2025. EditThinker: Unlocking Iterative Reasoning for Any Image Editor <ref:2609. p. 406288#pg2> [17] Yaron Lipman et al. 2023. Flow Matching for Generative Modeling <ref:2609. p. 006288#pg2> [18] Xiaoming Liu et al. 2024. FlowAlign: Aligning Flow-Based Generative Models <ref:2609. p. 006288#pg2> [19] Chenlin Meng et al. 2022. SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations <ref:2609. p. 006288#pg2> [20] Zhi Ouyang et al. 2025. ProEdit: Inversion-based Editing From Prompts Done Right <ref:2609. p. 206288#pg2> [21] Gaurav Parmar et al. 2023. Zero-shot Image-to-Image Translation <ref:2609. p. 606288#pg2> [26] Narek Tumanyan et al. 2023. Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation <ref:2609. p. 206288#pg2> [27] Jiangshan Wang et al. p. 0 2024. Taming Rectified Flow for Inversion and Editing <ref:2609. p. 006288#pg2> [28] Chenxi Xie et al. 2025. DNAEdit: Direct Noise Alignment for Text-Guided Rectified Flow Editing <ref:2609. p. 306288#pg2> [29] Pengcheng Xu et al. 2025. Unveil inversion and invariance in flow transformer for versatile image editing <ref:2609. p. 006288#pg2> [30] Sihan Xu et al. 2024. Inversion-Free Image Editing with Natural Language <ref:2609. p. 006288#pg2> [31] Sung-Hoon Yoon et al. 2025. SplitFlow: Flow Decomposition for Inversion-Free Text-to-Image Editing <ref:2609. p. 206288#pg2> [32] Yue Zhang et al. p. 0 2025. SAMControl: Controlling Pose and Object for Image Editing with Soft Attention Mask <ref:2609. p. 206288#pg2> [33] Tianrui Zhu et al. 2025. KV-Edit: Training-Free Image Editing for Precise Background Preservation <ref:2609. p. 006288#pg2> [47] Exact Tikhonov shrinkage (closed form). p. 5 Consider the diagonal Tikhonov-regularized objective Δv(1) t = arg min Δv - Δv DNA t squared + Γ 1/2 ⊙ Δv squared, where Γ >= 0 is diagonal and ⊙ denotes element-wise multiplication (masks are broadcast over channels) <ref:2609.06288#pg2> [48] Eq. (48) makes the effect explicit: on background pixels (Mfg = 0), Δv(1) t = Δv DNA t; on foreground pixels (Mfg = 1), the correction is shrunk by a factor 1/(1+lambda fg) <ref:2609.06288#pg2> [49] Eq. (49) preserves the same masking structure as the exact shrinkage and can be interpreted as a first-order approximation of Eq. p. 6 (48): 1/(1 + lambda fg Mfg) = 1 - lambda fg Mfg + O(lambda 2 fg), so when lambda fg is small, (1 - lambda fg Mfg) closely matches the exact Tikhonov shrink factor <ref:2609.06288#pg2> [50] Eq. (50) makes the effect explicit: on background pixels (Mbg = 0), Δv(1) t = Δv DNA t; on foreground pixels (Mfg = 1), the correction is shrunk by a factor 1/(1+lambda fg) <ref:2609.06288#pg2> [54] Eq. (54) is identical to the Stage-1 construction in the main text (Eq. p. 3 (5)) and makes explicit that Stage-1 anchors the background by partially removing only the drift-inducing component parallel to the source flow, while leaving the remaining (approximately orthogonal) background correction intact <ref:2609. p. 506288#pg2> [55] Latent recursion in Stage-1. p. 3 Using Δv safe t, Stage-1 forms a corrected latent by adding the step-size–scaled correction to the proxy point: Z t = Z∗ t + Δt Δv safe t <ref:2609. p. 506288#pg2> [56] Auxiliary noise-state recursion. p. 4 Following DNAEdit’s noise-space bookkeeping, we maintain an auxiliary noise state S t aligned with timestep t and update it using the same safe correction <ref:2609. p. 406288#pg2> [57] Stored offset (interface to Stage-2). p. 4 We cache the per-timestep offset between the corrected latent and the proxy point: Δx safe t = Z t - Z∗ t = Δt Δv safe t <ref:2609. p. 506288#pg2> [58] Offset-aligned proxy state (aligned with the main text). p. 5 At timestep t, the edited trajectory provides the current latent Z edit t. p. 6 To evaluate prompt-conditioned velocities consistently with Stage-1, we reuse the cached Stage-1 offset Δx safe t and form the offset-aligned proxy state Z edit t+ = Z edit t + Δx safe t, vtgt t = v(Z'edit t, ptgt) <ref:2609.06288#pg2> [59] Why controlling Δx safe t stabilizes semantic differences. p. 0 Assume v(·,p) is locally Lipschitz around Z edit t for both psrc and ptgt. p. 1 Then the semantic difference evaluated at the offset-aligned state is stable w. p. 5r.t. the offset magnitude: Δv(Z'edit t + Δx safe t) - Δv(Z edit t) [60] First Projection: Transport-Orthogonal Semantic Direction. semantic direction by projecting the prompt difference Δv(t) onto the subspace orthogonal to v src t: Δv⊥ t = Δv(t) - / + epsilon, and provides the semantic direction used for controlled accumulation (e. p. 5g., the reference update) <ref:2609. p. 306288#pg2> [65] Final edited field and timestep update. p. 6 The final edited velocity field matches the main text: v edit t = v src t + W edit t ⊙ g t Proj⊥ v src t v fg t - v src t, and the edited latent is then updated by the timestep-based Euler step (Eq. p. 6 (41)) <ref:2609. p. 006288#pg2> [66]

Appendix Details

Algorithm 1 Stage-1: Background-Anchored Reference Interface <ref:2609.06288#pg2> Algorithm 2 Stage-2: Object-Localized Safe Semantic Injection <ref:2609. p. 506288#pg2> B Implementation of the Safe Injection Operator This section specifies the implementation of the safe injection operator Γ t (·; R t) used in Algorithms 1 and 2 <ref:2609. p. 306288#pg2> A Details of Algorithm Algorithm 1 Stage-1: Background-Anchored Reference Interface <ref:2609. p. 406288#pg2> Support prior, not hard editing mask. p. 3 Given an input image I and an edit request (psrc, ptgt), OAVC constructs an object support M corresponding to the region where the semantic change is expected to occur <ref:2609. p. 306288#pg2> Latent-space support construction. p. 3 The pixel-space support is resized to the latent resolution, producing M in [0, 1] <ref:2609. p. 106288#pg2> Velocity-level weight maps. p. 0 We define two weight maps, broadcast over channels: W edit t = Mfg, W ref t = Mfg + β (σt) Mring <ref:2609. p. 406288#pg2> Boundary coupling schedule. p. 0 We use a fixed monotone decay schedule for the boundary coupling <ref:2609. p. 5

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Object-aware background-controlled editing can be implemented by introducing object-level control into the velocity-integration process, decoupling where semantic residuals act from how they are injected into the dynamics. This allows for object-centric edits such as object replacement, material change, deletion, or localized attribute modification while preserving surrounding regions.

  2. The system can construct a background-anchored reference interface under the source prompt and then perform object-localized safe semantic injection under the target prompt. This utilizes a constrained injection operator to suppress drift-inducing velocity components and time-adaptive spatial weighting to stabilize transitions near object boundaries.

  3. The framework requires no training or modification of pretrained model parameters, making it a training-free method for image editing steering diffusion or flow-matching generative models at inference time by modifying prompt-conditioned denoising velocities.

  4. By using the same where/how velocity-control principle on different rectified-flow backbones, the OAVC framework can be transferred to other pipelines like FlowEdit, improving localized editability and reducing unnecessary changes in non-target regions.

  5. The system can leverage object priors from various sources, such as SAM3 for zero-shot support, attention maps derived from the editing backbone itself for internal alternatives, or user-provided brush-refined masks to define the editable region.

Sources

Related papers