Scalable Minimal-Change Learning for Controllable Image Editing
cs.CV, cs.AI
Submitted: 2026-10-05
Updated: 2026-10-05
Comments: 29 pages, 2 figures. Accepted at NeurIPS 2026. Revised version with additional off-target, reward and SFT control, human evaluation, and OmniGen2 transfer results; clarified related work and experimental scope
Code: https://github.com/Showwwwwwwww/ARRO
License: http://creativecommons.org/licenses/by/4.0/
The gist: Image editing should change only the attributes specified by an instruction while preserving everything else, yet current methods often make unintended changes.
Abstract
Image editing should change only the attributes specified by an instruction while preserving everything else, yet current methods often make unintended changes. We treat this minimal-change principle as an optimization objective for instruction-based editing. Latent L1 regularization is a poor proxy for output locality in modern nonlinear generators and often requires supervision unavailable at scale. We instead optimize edit outcomes with reinforcement learning. An agentic vision-language reward model audits each source image, instruction, and edited image for two failure types: unimplemented requested changes and unintended changes. A group-level rubric merges and verifies these issues to provide consistent rewards across candidate edits without per-instruction human annotations. On FLUX.1 Kontext-dev, ARRO raises average EditScore from 5.21 to 5.88 across MinEval, MagicBrush, AnyBench, and Emu-Edit. On 600 evaluation examples, it reduces off-target pixel change by 8.4% relative to the base editor. Reward and SFT controls, blinded human evaluations, and transfer to OmniGen2 provide complementary evidence. Code: https://github.com/Showwwwwwwww/ARRO
Sources
- SPIE: Semantic and Structural Post-Training of Image Editing Diffusion Models with AI feedback
- FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space
- Emerging Properties in Unified Multimodal Pretraining
- Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization
- OneReward: Unified Mask-Guided Image Generation via Multi-Task Human Preference Learning
- AdLift: Lifting Adversarial Perturbations to Safeguard 3D Gaussian Splatting Assets Against Instruction-Driven Editing
- Delving into the Temporal Challenges of Unified Video Protection Against Image-to-Video and Fine-Tuning-based Customization
- SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models
- CompBench: Benchmarking Complex Instruction-guided Image Editing
- GEditBench v2: A Human-Aligned Benchmark for General Image Editing
- FlexCausal: Flexible Causal Disentanglement via Structural Flow Priors and Manifold-Aware Interventions
- CounterScene: Counterfactual Causal Reasoning in Generative World Models for Safety-Critical Closed-Loop Evaluation
- SAEdit: Token-level control for continuous image editing via Sparse AutoEncoder
- Analyzing and Improving the Image Quality of StyleGAN
- Causal Diffusion Autoencoders: Toward Counterfactual Generation via Diffusion Probabilistic Models
- Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies
- ThinkRL-Edit: Thinking in Reinforcement Learning for Reasoning-Centric Image Editing
- InstructAny2Pix: Flexible Visual Editing via Multimodal Instruction Following
- Beyond Optimal Transport: Model-Aligned Coupling for Flow Matching
- SubFlow: Sub-mode Conditioned Flow Matching for Diverse One-Step Generation
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models