Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews
summary
The gist
Ask a commercial image editor to preview a cosmetic procedure and it will often change more of the face than the request names<ref:2608.02841#pg2> The paper investigates how much control is possible…
In short
The study tested three methods for client-side control over AI image editing APIs: prompt-only, masked composite, and native inpainting. It found that using a mask to composite the edited region back onto the original photograph significantly reduced unintended changes outside the target area compared to just using prompts. This demonstrates that client-side masking can effectively confine edits to specific regions.
Key concepts
- Client-Side Control Levels
- These are three ways a user can try to limit what an AI editor changes before sending the request. The first level is just giving a text prompt, which offers little control. The second level involves using a mask to cut out the edited area and paste it back in, which provides much better confinement. The third level uses native inpainting features if available.
- Masked Composite
- This technique involves creating a region mask based on landmarks and then compositing the edited area back onto the original photo using that mask. This process acts as a post-hoc control mechanism, enforcing boundaries by copying pixels outside the target region, which helps keep edits localized.
- Localization Ratio (CIELAB Pixel Change)
- This metric measures how much of the pixel change lands inside the requested editing region versus protected facial zones. A high ratio indicates that most of the edit occurred where intended, showing good localization and less off-target alteration.
Terminology used across episodes
This episode discusses
- Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews · Paper Radio
- Blended Diffusion for Text-driven Editing of Natural Images
- DiffEdit: Diffusion-based semantic image editing with mask guidance
- Deep Learning-based Facial Appearance Simulation Driven by Surgically Planned Craniomaxillofacial Bony Movement
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
- High-Resolution Image Synthesis with Latent Diffusion Models
- SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
- RePaint: Inpainting using Denoising Diffusion Probabilistic Models
- Blended Latent Diffusion
- Prompt-to-Prompt Image Editing with Cross Attention Control
- Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation
- InstructPix2Pix: Learning to Follow Image Editing Instructions
- MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing
- Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting
- Emu Edit: Precise Image Editing via Recognition and Generation Tasks
- ImagenHub: Standardizing the evaluation of conditional image generation models
- I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing
- HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing
- Addressing Text Embedding Leakage in Diffusion-based Image Editing
- ArcFace: Additive Angular Margin Loss for Deep Face Recognition
- Sample and Computation Redistribution for Efficient Face Detection
The paper
Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews · Read on arXiv
Northeastern University
Ask a commercial image editor to preview a cosmetic procedure and it will often change more of the face than the request names: a jawline edit can also smooth the forehead or alter the eyes. Existing methods for confining an edit to one region require access to the model's internals, which a public editing API does not expose. We ask how much control is possible from the client side alone. In a pilot benchmark, five current commercial editing configurations perform facelift-style jaw-neck and rhinoplasty edits on synthetic faces under two levels of client-side control: the prompt alone, and a masked composite that cuts the edited region out of the response and pastes it back onto the original photograph through a landmark-derived mask. Of 162 attempted edits, 151 could be scored. ArcFace cosine measures identity preservation; a CIELAB pixel-change ratio measures how much change lands inside the requested region rather than a protected facial zone. On 15 faces, the masked composite improved localization over the paired prompt-only output by a median of 0.315 (95% face-clustered bootstrap interval 0.279-0.366) while retaining most of the target-region change. Editors differed in edit strength and identity retention, and one returned near-copies for the facelift request. This is a study of control, not clinical accuracy: the faces are synthetic, no surgeons rated the outputs, and each condition was generated once. Within that scope, keeping a surgical preview inside its intended region needs no access to the model; a mask and composite on the client enforce it across every editor tested, at low provider cost.
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Localize, Don't Beautify".
Paper summary: Tom: So, we're looking at this paper called "Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews." Basically, they’re asking how much control a user has when they ask a commercial image editor to make a cosmetic change.
Jane: That sounds really practical. Because as we know from experience, you ask an AI to smooth skin or change lighting on your face, and it often changes way more than you actually asked for, right?
Tom: Exactly. The paper investigates this control from the client side alone because the public editing APIs don't let you see what the model is doing internally. They test six different commercial editing setups and one mask based inpainting model for things like jaw neck edits and rhinoplasty.
Lu: What I find interesting is their three levels of client-side control they tested: prompt only, a masked composite, and native masked inpainting where it was supported. These are the tools they use to try and keep the edit confined to just one area.
Meng: From a production standpoint, that's important because if you can't guarantee that an edit stays put, how do you deploy it? I'd want to know what actually breaks at scale with these different control levels.
Lalam: My internal literature search points me toward the core idea: they are measuring how much off-target change happens when you use these different settings before or after a request is sent.
Tom: Right, so before we get into the results, what's the main point they're trying to make about this situation? Is it that client control is hard to get?
Jane: They're showing that even with a black-box API where you have very little control, you can still enforce region confinement entirely on the client side if you use the right technique.
Lu: Their setup involves using a target region, like the jaw and neck for a facelift edit, and then comparing what happens when you only give it a prompt versus when you use that feathered mask to cut out the edited area and paste it back onto your original photo.
Paper summary: Meng: That sounds like a post-hoc fix rather than true control. If you have to manually composite things after the model responds, isn't that just a workaround? How much does that actually save time or money in a real workflow?
Tom: The paper suggests that using this masked composite method performs really well. They found that for those six commercial configurations, the median localization ratio—which measures how much change lands inside the requested region versus protected facial zones—was about zero point nine eight five across all of them <ref:2608.02841#pg2>.
Jane: That's a high number, suggesting they are very good at keeping the edit where you want it on average. But Tom, what does that mean when you compare it to their prompt only setting?
Tom: The prompt only outputs had a median localization ratio of zero point five three eight, and after using the masked composite—that post-hoc control—the median off-target change was reduced to three point five zero compared to three point five zero for the prompt only outputs.
Lu: So, they're demonstrating that the preservation is enforced by that blend in the masked composite because it copies off-mask pixels by design, which means it keeps everything outside your target region from being edited too.
Meng: I see what you mean about the design enforcing it. But this study is small, with only fifteen faces for the main comparisons, so we can't really say anything about anatomical accuracy or clinical plausibility yet.
Jane: That’s a big caveat they mention. They are also very clear that their metrics, like the ArcFace cosine and CIELAB pixel-change ratio, just measure identity preservation and pixel change ratios, not actual clinical outcome or realism.
Tom: Right, so we're measuring if the face looks like the original person still is—the identity preservation—and how much of the pixel change lands inside your target area versus protected facial zones. They are explicitly stating they aren't measuring clinical plausibility or postoperative appearance.
Lu: And they point out that ArcFace’s zero point is a heuristic reference and that edits moved slightly away from the outcome in identity-embedding space rather than closer to it, which is important context for interpreting those scores.
Paper summary: Jane: It sounds like the paper's main contribution isn't a new model or algorithm, but proving that client-side compositing can be a control mechanism to reduce off-target pixel changes when using these black-box APIs.
Meng: So what does this mean for someone who just uses an AI editor without reading the research? It means if you use the post-hoc masking technique, you get much better results than just typing a prompt alone.
Tom: It means that when dealing with black-box editing APIs, the client has some levers it can pull to keep edits confined to a specific region without needing access to whatever complex model is running on the provider's end.
Jane: The authors suggest that for surgeons or users, this client-side compositing should be seen as a control for off-target pixel changes, not as evidence that a preview is clinically safe or accurate enough for patient counseling.
Lu: And they do admit that the study doesn't validate cosmetic surgery outcome prediction at all. They say the next steps should involve things like repeated generations and anatomy aware postoperative accuracy metrics.
Tom: So, the main finding, summarizing this paper "Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews," is that landmark derived masks and client side feathered composites reduced that measured off target change while keeping the target region pixel change similar across all six editors tested.
Jane: It establishes that when an editor doesn't expose its internals, you can still enforce region confinement entirely from the client using methods like masked composite.
Lu: And they suggest future work needs to add repeated generations and pose robust whole image localization to get closer to what's needed for true clinical accuracy.
Meng: For deployment, the cost of this control is low; the median provider cost for these edits was about zero point zero four five per image across all tests. That's a manageable operational factor if you can rely on it.
Tom: So, to wrap up, this paper shows that client side techniques offer a way to gain measurable control over where an image editor makes changes when dealing with cosmetic surgery previews using public APIs, even without seeing the model internals.
Conclusion: Tom: So, we're wrapping up this talk on "Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews."
Jane: The authors are looking at how much control a regular user has when they ask an AI editor to make cosmetic changes.
Tom: They test six different commercial editing setups and one mask based inpainting model using three levels of client side control.
Lu: The main idea is showing that even with a black box API, you can still enforce region confinement entirely from the client side if you use the right technique.
Meng: So what does this actually mean for someone who just uses an AI editor every day?
Jane: It means that when you use the right tools, like a feathered mask to cut out and paste back the edited area, you can reduce off target pixel changes a lot.
Tom: They found that this masked composite method produced a high localization ratio of about zero point nine eight five across all six setups.
Lu: Their prompt only settings only had about half that, so using the composite actually enforces preservation by design because it copies the pixels outside your target area.
Jane: But we have to be careful here; this is still just measuring pixel change and identity preservation in embedding space, not actual clinical plausibility or surgery outcomes.
Meng: Right, so it’s a control mechanism for pixel changes, not a guarantee that the preview looks anatomically correct or safe for a patient.
Tom: That's the big caveat; you need to see this as a way to control what pixels are modified, not proof that the result is clinically perfect.
Lu: The authors admit they haven't done much on repeated generations or pose robust localization yet, which means this is just a pilot study for now.
Jane: The next step in this research should be adding more generations and better whole image localization to get closer to what's needed for real-world use.
Tom: So the main point is that client side techniques can give you measurable control over where an editor makes changes when dealing with cosmetic surgery previews.
Jane: It shows that we don't have zero control, but there are specific ways to manage the off target pixel changes using standard tools.
More episodes
- 2603.28113-The multiplicative Lipschitz bound as an inductive bias
- 2603.28595-Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound