Local Content-Style Control for Diffusion-based Image Stylization
cs.GR, cs.CV
Submitted: 2026-10-06
Updated: 2026-10-06
Comments: SIGGRAPH Asia 2026 Technical Communications. 4 pages, 4 figures, 1 table. Supplemental material included as an ancillary file
License: http://creativecommons.org/licenses/by/4.0/
The gist: Image stylization with latent-diffusion models entangles two independently refined axes: what a region depicts and how it is depicted.
Abstract
Image stylization with latent-diffusion models entangles two independently refined axes: what a region depicts and how it is depicted. Such pipelines expose only global controls, yet professional retouching demands deliberate, region-specific control. We lift two conditioning weights already present in a ControlNet + IP-Adapter stylization pipeline from global scalars to per-location spatial maps, yielding local, per-axis control of content and style in a single generative pass. Because the two weights act on disjoint pathways, adjusting them independently spans a 2x2 retouching vocabulary, from free regeneration to identity preservation. We validate that edits stay confined to the retouched region and that each weight predominantly steers its own axis. Our approach requires no retraining and drops unchanged into any such pipeline.
Sources
- eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers
- SDXL-Lightning: Progressive Adversarial Diffusion Distillation
- InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generation
- InstantStyle-Plus: Style Transfer with Content-Preserving in Text-to-Image Generation
- CSGO: Content-Style Composition in Text-to-Image Generation
- IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models
Related papers
- SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-Based Humanoid Control
- CADReasoner: Iterative Program Editing for CAD Reverse Engineering
- QuadLink: Autoregressive Quad-Dominant Mesh Generation via Point-Relation Learning
- DrawVideo: Grounded and Faithful Multi-Shot Video Generation from Storyboard Keyframe Sketches
- MotionPersona: Real-Time Locomotion Control across Personas, Bodies, and Styles
- GaussianBench: Physics-Fidelity Evaluation for Gaussian Scene Representations