Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Localize, Don't Beautify".
Paper summary: Tom: So, we're looking at this paper called "Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews." Basically, they’re asking how much control a user has when they ask a commercial image editor to make a cosmetic change.
Jane: That sounds really practical. Because as we know from experience, you ask an AI to smooth skin or change lighting on your face, and it often changes way more than you actually asked for, right?
Tom: Exactly. The paper investigates this control from the client side alone because the public editing APIs don't let you see what the model is doing internally. They test six different commercial editing setups and one mask based inpainting model for things like jaw neck edits and rhinoplasty.
Lu: What I find interesting is their three levels of client-side control they tested: prompt only, a masked composite, and native masked inpainting where it was supported. These are the tools they use to try and keep the edit confined to just one area.
Meng: From a production standpoint, that's important because if you can't guarantee that an edit stays put, how do you deploy it? I'd want to know what actually breaks at scale with these different control levels.
Lalam: My internal literature search points me toward the core idea: they are measuring how much off-target change happens when you use these different settings before or after a request is sent.
Tom: Right, so before we get into the results, what's the main point they're trying to make about this situation? Is it that client control is hard to get?
Jane: They're showing that even with a black-box API where you have very little control, you can still enforce region confinement entirely on the client side if you use the right technique.
Lu: Their setup involves using a target region, like the jaw and neck for a facelift edit, and then comparing what happens when you only give it a prompt versus when you use that feathered mask to cut out the edited area and paste it back onto your original photo.
Paper summary: Meng: That sounds like a post-hoc fix rather than true control. If you have to manually composite things after the model responds, isn't that just a workaround? How much does that actually save time or money in a real workflow?
Tom: The paper suggests that using this masked composite method performs really well. They found that for those six commercial configurations, the median localization ratio—which measures how much change lands inside the requested region versus protected facial zones—was about zero point nine eight five across all of them <ref:2608.02841#pg2>.
Jane: That's a high number, suggesting they are very good at keeping the edit where you want it on average. But Tom, what does that mean when you compare it to their prompt only setting?
Tom: The prompt only outputs had a median localization ratio of zero point five three eight, and after using the masked composite—that post-hoc control—the median off-target change was reduced to three point five zero compared to three point five zero for the prompt only outputs.
Lu: So, they're demonstrating that the preservation is enforced by that blend in the masked composite because it copies off-mask pixels by design, which means it keeps everything outside your target region from being edited too.
Meng: I see what you mean about the design enforcing it. But this study is small, with only fifteen faces for the main comparisons, so we can't really say anything about anatomical accuracy or clinical plausibility yet.
Jane: That’s a big caveat they mention. They are also very clear that their metrics, like the ArcFace cosine and CIELAB pixel-change ratio, just measure identity preservation and pixel change ratios, not actual clinical outcome or realism.
Tom: Right, so we're measuring if the face looks like the original person still is—the identity preservation—and how much of the pixel change lands inside your target area versus protected facial zones. They are explicitly stating they aren't measuring clinical plausibility or postoperative appearance.
Lu: And they point out that ArcFace’s zero point is a heuristic reference and that edits moved slightly away from the outcome in identity-embedding space rather than closer to it, which is important context for interpreting those scores.
Paper summary: Jane: It sounds like the paper's main contribution isn't a new model or algorithm, but proving that client-side compositing can be a control mechanism to reduce off-target pixel changes when using these black-box APIs.
Meng: So what does this mean for someone who just uses an AI editor without reading the research? It means if you use the post-hoc masking technique, you get much better results than just typing a prompt alone.
Tom: It means that when dealing with black-box editing APIs, the client has some levers it can pull to keep edits confined to a specific region without needing access to whatever complex model is running on the provider's end.
Jane: The authors suggest that for surgeons or users, this client-side compositing should be seen as a control for off-target pixel changes, not as evidence that a preview is clinically safe or accurate enough for patient counseling.
Lu: And they do admit that the study doesn't validate cosmetic surgery outcome prediction at all. They say the next steps should involve things like repeated generations and anatomy aware postoperative accuracy metrics.
Tom: So, the main finding, summarizing this paper "Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews," is that landmark derived masks and client side feathered composites reduced that measured off target change while keeping the target region pixel change similar across all six editors tested.
Jane: It establishes that when an editor doesn't expose its internals, you can still enforce region confinement entirely from the client using methods like masked composite.
Lu: And they suggest future work needs to add repeated generations and pose robust whole image localization to get closer to what's needed for true clinical accuracy.
Meng: For deployment, the cost of this control is low; the median provider cost for these edits was about zero point zero four five per image across all tests. That's a manageable operational factor if you can rely on it.
Tom: So, to wrap up, this paper shows that client side techniques offer a way to gain measurable control over where an image editor makes changes when dealing with cosmetic surgery previews using public APIs, even without seeing the model internals.
Conclusion: Tom: So, we're wrapping up this talk on "Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews."
Jane: The authors are looking at how much control a regular user has when they ask an AI editor to make cosmetic changes.
Tom: They test six different commercial editing setups and one mask based inpainting model using three levels of client side control.
Lu: The main idea is showing that even with a black box API, you can still enforce region confinement entirely from the client side if you use the right technique.
Meng: So what does this actually mean for someone who just uses an AI editor every day?
Jane: It means that when you use the right tools, like a feathered mask to cut out and paste back the edited area, you can reduce off target pixel changes a lot.
Tom: They found that this masked composite method produced a high localization ratio of about zero point nine eight five across all six setups.
Lu: Their prompt only settings only had about half that, so using the composite actually enforces preservation by design because it copies the pixels outside your target area.
Jane: But we have to be careful here; this is still just measuring pixel change and identity preservation in embedding space, not actual clinical plausibility or surgery outcomes.
Meng: Right, so it’s a control mechanism for pixel changes, not a guarantee that the preview looks anatomically correct or safe for a patient.
Tom: That's the big caveat; you need to see this as a way to control what pixels are modified, not proof that the result is clinically perfect.
Lu: The authors admit they haven't done much on repeated generations or pose robust localization yet, which means this is just a pilot study for now.
Jane: The next step in this research should be adding more generations and better whole image localization to get closer to what's needed for real-world use.
Tom: So the main point is that client side techniques can give you measurable control over where an editor makes changes when dealing with cosmetic surgery previews.
Jane: It shows that we don't have zero control, but there are specific ways to manage the off target pixel changes using standard tools.
Northeastern University
cs.CV
Submitted: 2026-08-03
Updated: 2026-11-17
Comments: 9 pages, 7 figures, 2 tables. Pilot study; no surgeon ratings. Code and paper source: https://github.com/suxrobGM/localize-dont-beautify
Code: https://github.com/suxrobGM/localize-dont-beautify
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 76/100
The gist: Ask a commercial image editor to preview a cosmetic procedure and it will often change more of the face than the request names<ref:2608.02841#pg2> The paper investigates how much control is possible…
Key concepts
- Client-Side Control Levels
- These are three ways a user can try to limit what an AI editor changes before sending the request. The first level is just giving a text prompt, which offers little control. The second level involves using a mask to cut out the edited area and paste it back in, which provides much better confinement. The third level uses native inpainting features if available.
- Masked Composite
- This technique involves creating a region mask based on landmarks and then compositing the edited area back onto the original photo using that mask. This process acts as a post-hoc control mechanism, enforcing boundaries by copying pixels outside the target region, which helps keep edits localized.
- Localization Ratio (CIELAB Pixel Change)
- This metric measures how much of the pixel change lands inside the requested editing region versus protected facial zones. A high ratio indicates that most of the edit occurred where intended, showing good localization and less off-target alteration.
Terminology
Summary
Ask a commercial image editor to preview a cosmetic procedure and it will often change more of the face than the request names<ref:2608. p. 0 s@northeastern. edu
Abstract—Ask a commercial image editor to preview a cosmetic procedure and it will often change more of the face than
the request names: a nose edit can also smooth skin or alter
lighting. Existing methods for confining an edit to one region
require access to the model’s internals, which a public editing
API does not expose. We ask how much control is possible from
the client side alone. We ask how much control is possible from
the client side alone. In a pilot benchmark, six commercial editing
configurations and one mask-based inpainting model perform
facelift-style jaw–neck and rhinoplasty edits at three levels of
client-side control: the prompt alone; cutting the edited region
out of the response and pasting it back onto the original photograph through a landmark-derived mask (a masked composite);
and asking the model itself to inpaint inside the mask where
supported. In a pilot benchmark, six commercial editing
configurations and one mask-based inpainting model perform
facelift-style jaw–neck and rhinoplasty edits at three levels of
client-side control: the prompt alone; cutting the edited region
out of the response and pasting it back onto the original photograph through a landmark-derived mask (a masked composite);
and asking the model itself to inpaint inside the mask where
supported. Of 210 attempted edits, 196 could be scored. This is a study of control, not clinical accuracy: no
surgeons rated the outputs, and each condition was generated
once. Within that scope, keeping a surgical preview inside its
intended region needs no access to the model; a mask and
composite on the client enforce it across every editor tested, at
low provider cost.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The study evaluates control over off-target change on the client side before or after a request is sent using three rungs of increasing control<ref:2608.02841#pg4> The three levels of client-side control are prompt only, a masked composite that cuts the edited region out along a landmark-derived region mask and pastes it back onto the original photograph through a masked composite, and native masked inpainting where supported<ref:2608. p. 0 Our contributions are:
• A model-agnostic control ladder. Three rungs over
the same black-box APIs: prompt-only, a client-side
masked composite that cuts the model’s edit out along
a landmark-derived region mask and pastes it back onto
the original photograph (with alignment, feathered blending, and pose gating), and native masked inpainting
where supported. The photo and instruction go to the endpoint unmodified and the response is used as-is. This
measures what the model does when instructed to edit
one region. A
black-box API exposes none of those surfaces. Our masked
composite is deliberately the post-hoc remnant of this family,
the final pixel-space blend applied after the API returns, and
we measure how much control that remnant still provides.Where the paper says it
Where the paper says it
Where the paper says it
How the training signal is built
The evaluation protocol employs a two-part diagnostic protocol pairing ArcFace identity with a CIELAB region-localization ratio and its target/off-target components<ref:2608.02841#pg4> The ArcFace cosine measures identity preservation, while the CIELAB pixel-change ratio measures how much change lands inside the requested region rather than a protected facial zone<ref:2608. p. 0 Of 210 attempted edits, 196 could be scored. ArcFace
cosine measures identity preservation; a CIELAB pixel-change
ratio measures how much change lands inside the requested
region rather than a protected facial zone. ArcFace identity is
paired with a CIELAB region-localization ratio and its
target/off-target components. The metrics detect identity
loss, off-target pixel change, and near-copies, but do not
measure clinical plausibility. The metrics detect identity
loss, off-target pixel change, and near-copies, but do not
measure clinical plausibility. Postoperative photographs
provide an identity reference with a per-face input baseline (Sec.Where the paper says it
Where the paper says it
Where the paper says it
Where the gains come from
Masked compositing produces a median localization ratio of 0.985 across the six primary editing configurations<ref:2608. p. 4 985 (range 0. 998) across the six primary editing
configurations (Fig. 998) across the six primary editing
configurations (Fig. Their prompt-only outputs have a median of 0. The top inpaint example
has low identity similarity. landmark-derived mask, alignment, and client-side feathered
composite reduced that measured off-target change while retaining similar target-region pixel change, uniformly across the
six editors tested and at a median provider cost of $0. A two-order, single-face chain is reported
only as a feasibility probe (Sec. To our knowledge, this is the first controlled comparison
of client-side region confinement across multiple commercial,
arXiv:2608. Prompt instructions alone did not preserve the measured keep
zone. Post-hoc compositing did so because preservation is
enforced by the blend; this result does not assess anatomical
correctness inside the mask or changes outside the limited keep
zone. 045 per
image. The tested Qwen inpainting model did not outperform
that post-hoc control, but one model cannot support a general
conclusion about masked inpainting.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Limitations and Context
The study is a small pilot with 15 faces supporting the main comparisons, and the results do not establish clinical plausibility or anatomical accuracy<ref:2608.02841#pg8> The localization ratio measures CIELAB pixel change, not anatomical direction, clinical plausibility, realism, or expected postoperative appearance<ref:2608. p. 5 Each cell contains one generated output, and two
models ignore the supplied seed, so stochastic variation is
unknown. The localization ratio measures CIELAB pixel change, not
anatomical direction, clinical plausibility, realism, or expected
postoperative appearance. Of 210 attempted edits, 196 could be scored. ArcFace
cosine measures identity preservation; a CIELAB pixel-change
ratio measures how much change lands inside the requested
region rather than a protected facial zone. 6 line is heuristic. The
postoperative comparison, though baselined against each face’s
input photograph, lives entirely in identity-embedding space;
an edit could approximate the surgical result anatomically
while losing embedding similarity, and neither score is a
validated surgical-outcome metric. The tested Qwen inpainting model did not outperform
that post-hoc control, but one model cannot support a general
conclusion about masked inpainting. The study does not validate cosmetic-surgery outcome prediction, and client-side compositing should be read as a control
for off-target pixel changes, not as evidence that a preview is
clinically accurate or safe for patient counseling.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion
The main finding is that landmark-derived mask, alignment, and client-side feathered composite reduced that measured off-target change while retaining similar target-region pixel change uniformly across the six editors tested and at a median provider cost of 0.045 per image<ref:2608. p. 3 Across scoreable outputs, median latency is 21 s and
median provider cost is $0. 045 per image. It will also
need multiple blinded surgeons with inter-rater agreement,
a larger and demographically characterized cohort, and immutable or self-hosted model baselines. What the pilot does
establish is narrower: when an editor exposes no internals,
region confinement can still be enforced entirely from the
client. The main
analysis contains 15 faces, one generated output per cell, no
surgeon ratings, limited profile support, and only an identityembedding measure of movement toward the postoperative
photographs; by that measure the edits moved slightly away
from the outcome, not closer. The next study should add
repeated generations, pose-robust whole-image localization,
and anatomy-aware postoperative accuracy metrics.Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Client-side control ladder for image editing APIs allows users to enforce region confinement by using a masked composite that
cuts the model’s edit out along a landmark-derived region mask and pastes it back onto the original photograph.
This prevents off-target changes while keepingnothing inside the model is reachable
for black-box editors. -
Two-part diagnostic protocol enables systems to measure identity preservation using ArcFace cosine alongside a CIELAB pixel change ratio to detect both
identity loss, off-target pixel change, and near-copies.
This allows for a system diagnostic that detects whether an edit isanatomically or clinically plausible.
-
Implementing prompt guardrails by appending fixed instruction fragments—an avoidance fragment listing unwanted behaviors and a positive fragment anchoring realism—can mitigate generic beautification outputs like
smoothed skin, brightened teeth, and subtly enlarged eyes.
This forces the model to focus onchange only facial tissue and contour
as requested.
Abstract
Ask a commercial image editor to preview a cosmetic procedure and it will often change more of the face than the request names: a jawline edit can also smooth the forehead or alter the eyes. Existing methods for confining an edit to one region require access to the model's internals, which a public editing API does not expose. We ask how much control is possible from the client side alone. In a pilot benchmark, five current commercial editing configurations perform facelift-style jaw-neck and rhinoplasty edits on synthetic faces under two levels of client-side control: the prompt alone, and a masked composite that cuts the edited region out of the response and pastes it back onto the original photograph through a landmark-derived mask. Of 162 attempted edits, 151 could be scored. ArcFace cosine measures identity preservation; a CIELAB pixel-change ratio measures how much change lands inside the requested region rather than a protected facial zone. On 15 faces, the masked composite improved localization over the paired prompt-only output by a median of 0.315 (95% face-clustered bootstrap interval 0.279-0.366) while retaining most of the target-region change. Editors differed in edit strength and identity retention, and one returned near-copies for the facelift request. This is a study of control, not clinical accuracy: the faces are synthetic, no surgeons rated the outputs, and each condition was generated once. Within that scope, keeping a surgical preview inside its intended region needs no access to the model; a mask and composite on the client enforce it across every editor tested, at low provider cost.
Sources
- Blended Diffusion for Text-driven Editing of Natural Images
- DiffEdit: Diffusion-based semantic image editing with mask guidance
- Deep Learning-based Facial Appearance Simulation Driven by Surgically Planned Craniomaxillofacial Bony Movement
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
- High-Resolution Image Synthesis with Latent Diffusion Models
- SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
- RePaint: Inpainting using Denoising Diffusion Probabilistic Models
- Blended Latent Diffusion
- Prompt-to-Prompt Image Editing with Cross Attention Control
- Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation
- InstructPix2Pix: Learning to Follow Image Editing Instructions
- MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing
- Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting
- Emu Edit: Precise Image Editing via Recognition and Generation Tasks
- ImagenHub: Standardizing the evaluation of conditional image generation models
- I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing
- HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing
- Addressing Text Embedding Leakage in Diffusion-based Image Editing
- ArcFace: Additive Angular Margin Loss for Deep Face Recognition
- Sample and Computation Redistribution for Efficient Face Detection
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs