Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews

arXiv:2608.02841 · cs.CV · Submitted 2026-08-03 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Localize, Don't Beautify".

Paper summary: Tom: So, we're looking at this paper called "Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews." Basically, they’re asking how much control a user has when they ask a commercial image editor to make a cosmetic change.

Jane: That sounds really practical. Because as we know from experience, you ask an AI to smooth skin or change lighting on your face, and it often changes way more than you actually asked for, right?

Tom: Exactly. The paper investigates this control from the client side alone because the public editing APIs don't let you see what the model is doing internally. They test six different commercial editing setups and one mask based inpainting model for things like jaw neck edits and rhinoplasty.

Lu: What I find interesting is their three levels of client-side control they tested: prompt only, a masked composite, and native masked inpainting where it was supported. These are the tools they use to try and keep the edit confined to just one area.

Meng: From a production standpoint, that's important because if you can't guarantee that an edit stays put, how do you deploy it? I'd want to know what actually breaks at scale with these different control levels.

Lalam: My internal literature search points me toward the core idea: they are measuring how much off-target change happens when you use these different settings before or after a request is sent.

Tom: Right, so before we get into the results, what's the main point they're trying to make about this situation? Is it that client control is hard to get?

Jane: They're showing that even with a black-box API where you have very little control, you can still enforce region confinement entirely on the client side if you use the right technique.

Lu: Their setup involves using a target region, like the jaw and neck for a facelift edit, and then comparing what happens when you only give it a prompt versus when you use that feathered mask to cut out the edited area and paste it back onto your original photo.

Paper summary: Meng: That sounds like a post-hoc fix rather than true control. If you have to manually composite things after the model responds, isn't that just a workaround? How much does that actually save time or money in a real workflow?

Tom: The paper suggests that using this masked composite method performs really well. They found that for those six commercial configurations, the median localization ratio—which measures how much change lands inside the requested region versus protected facial zones—was about zero point nine eight five across all of them <ref:2608.02841#pg2>.

Jane: That's a high number, suggesting they are very good at keeping the edit where you want it on average. But Tom, what does that mean when you compare it to their prompt only setting?

Tom: The prompt only outputs had a median localization ratio of zero point five three eight, and after using the masked composite—that post-hoc control—the median off-target change was reduced to three point five zero compared to three point five zero for the prompt only outputs.

Lu: So, they're demonstrating that the preservation is enforced by that blend in the masked composite because it copies off-mask pixels by design, which means it keeps everything outside your target region from being edited too.

Meng: I see what you mean about the design enforcing it. But this study is small, with only fifteen faces for the main comparisons, so we can't really say anything about anatomical accuracy or clinical plausibility yet.

Jane: That’s a big caveat they mention. They are also very clear that their metrics, like the ArcFace cosine and CIELAB pixel-change ratio, just measure identity preservation and pixel change ratios, not actual clinical outcome or realism.

Tom: Right, so we're measuring if the face looks like the original person still is—the identity preservation—and how much of the pixel change lands inside your target area versus protected facial zones. They are explicitly stating they aren't measuring clinical plausibility or postoperative appearance.

Lu: And they point out that ArcFace’s zero point is a heuristic reference and that edits moved slightly away from the outcome in identity-embedding space rather than closer to it, which is important context for interpreting those scores.

Paper summary: Jane: It sounds like the paper's main contribution isn't a new model or algorithm, but proving that client-side compositing can be a control mechanism to reduce off-target pixel changes when using these black-box APIs.

Meng: So what does this mean for someone who just uses an AI editor without reading the research? It means if you use the post-hoc masking technique, you get much better results than just typing a prompt alone.

Tom: It means that when dealing with black-box editing APIs, the client has some levers it can pull to keep edits confined to a specific region without needing access to whatever complex model is running on the provider's end.

Jane: The authors suggest that for surgeons or users, this client-side compositing should be seen as a control for off-target pixel changes, not as evidence that a preview is clinically safe or accurate enough for patient counseling.

Lu: And they do admit that the study doesn't validate cosmetic surgery outcome prediction at all. They say the next steps should involve things like repeated generations and anatomy aware postoperative accuracy metrics.

Tom: So, the main finding, summarizing this paper "Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews," is that landmark derived masks and client side feathered composites reduced that measured off target change while keeping the target region pixel change similar across all six editors tested.

Jane: It establishes that when an editor doesn't expose its internals, you can still enforce region confinement entirely from the client using methods like masked composite.

Lu: And they suggest future work needs to add repeated generations and pose robust whole image localization to get closer to what's needed for true clinical accuracy.

Meng: For deployment, the cost of this control is low; the median provider cost for these edits was about zero point zero four five per image across all tests. That's a manageable operational factor if you can rely on it.

Tom: So, to wrap up, this paper shows that client side techniques offer a way to gain measurable control over where an image editor makes changes when dealing with cosmetic surgery previews using public APIs, even without seeing the model internals.

Conclusion: Tom: So, we're wrapping up this talk on "Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews."

Jane: The authors are looking at how much control a regular user has when they ask an AI editor to make cosmetic changes.

Tom: They test six different commercial editing setups and one mask based inpainting model using three levels of client side control.

Lu: The main idea is showing that even with a black box API, you can still enforce region confinement entirely from the client side if you use the right technique.

Meng: So what does this actually mean for someone who just uses an AI editor every day?

Jane: It means that when you use the right tools, like a feathered mask to cut out and paste back the edited area, you can reduce off target pixel changes a lot.

Tom: They found that this masked composite method produced a high localization ratio of about zero point nine eight five across all six setups.

Lu: Their prompt only settings only had about half that, so using the composite actually enforces preservation by design because it copies the pixels outside your target area.

Jane: But we have to be careful here; this is still just measuring pixel change and identity preservation in embedding space, not actual clinical plausibility or surgery outcomes.

Meng: Right, so it’s a control mechanism for pixel changes, not a guarantee that the preview looks anatomically correct or safe for a patient.

Tom: That's the big caveat; you need to see this as a way to control what pixels are modified, not proof that the result is clinically perfect.

Lu: The authors admit they haven't done much on repeated generations or pose robust localization yet, which means this is just a pilot study for now.

Jane: The next step in this research should be adding more generations and better whole image localization to get closer to what's needed for real-world use.

Tom: So the main point is that client side techniques can give you measurable control over where an editor makes changes when dealing with cosmetic surgery previews.

Jane: It shows that we don't have zero control, but there are specific ways to manage the off target pixel changes using standard tools.

Northeastern University

cs.CV

Submitted: 2026-08-03

Updated: 2026-11-17

Comments: 9 pages, 7 figures, 2 tables. Pilot study; no surgeon ratings. Code and paper source: https://github.com/suxrobGM/localize-dont-beautify

Code: https://github.com/suxrobGM/localize-dont-beautify

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 76/100

The gist: Ask a commercial image editor to preview a cosmetic procedure and it will often change more of the face than the request names<ref:2608.02841#pg2> The paper investigates how much control is possible…

Key concepts

Client-Side Control Levels
These are three ways a user can try to limit what an AI editor changes before sending the request. The first level is just giving a text prompt, which offers little control. The second level involves using a mask to cut out the edited area and paste it back in, which provides much better confinement. The third level uses native inpainting features if available.
Masked Composite
This technique involves creating a region mask based on landmarks and then compositing the edited area back onto the original photo using that mask. This process acts as a post-hoc control mechanism, enforcing boundaries by copying pixels outside the target region, which helps keep edits localized.
Localization Ratio (CIELAB Pixel Change)
This metric measures how much of the pixel change lands inside the requested editing region versus protected facial zones. A high ratio indicates that most of the edit occurred where intended, showing good localization and less off-target alteration.

Terminology

Summary

Ask a commercial image editor to preview a cosmetic procedure and it will often change more of the face than the request names<ref:2608. p. 0

Where the paper says it

s@northeastern. edu Abstract—Ask a commercial image editor to preview a cosmetic procedure and it will often change more of the face than the request names: a nose edit can also smooth skin or alter lighting.

02841#pg2> The paper investigates how much control is possible from the client side alone in confining edits to one region when using public editing APIs<ref:2608. p. 0
Where the paper says it

Existing methods for confining an edit to one region require access to the model’s internals, which a public editing API does not expose. We ask how much control is possible from the client side alone.

02841#pg2> In a pilot benchmark, six commercial editing configurations and one mask-based inpainting model perform facelift-style jaw–neck and rhinoplasty edits at three levels of client-side control<ref:2608. p. 0
Where the paper says it

We ask how much control is possible from the client side alone. In a pilot benchmark, six commercial editing configurations and one mask-based inpainting model perform facelift-style jaw–neck and rhinoplasty edits at three levels of client-side control: the prompt alone; cutting the edited region out of the response and pasting it back onto the original photograph through a landmark-derived mask (a masked composite); and asking the model itself to inpaint inside the mask where supported.

02841#pg2> Of 210 attempted edits, 196 could be scored<ref:2608. p. 0
Where the paper says it

In a pilot benchmark, six commercial editing configurations and one mask-based inpainting model perform facelift-style jaw–neck and rhinoplasty edits at three levels of client-side control: the prompt alone; cutting the edited region out of the response and pasting it back onto the original photograph through a landmark-derived mask (a masked composite); and asking the model itself to inpaint inside the mask where supported. Of 210 attempted edits, 196 could be scored.

02841#pg4> Within that scope, keeping a surgical preview inside its intended region needs no access to the model; a mask and composite on the client enforce it across every editor tested, at low provider cost<ref:2608. p. 0
Where the paper says it

This is a study of control, not clinical accuracy: no surgeons rated the outputs, and each condition was generated once. Within that scope, keeping a surgical preview inside its intended region needs no access to the model; a mask and composite on the client enforce it across every editor tested, at low provider cost.

02841#pg2>

Pruning redundant reasoning tokens

The study evaluates control over off-target change on the client side before or after a request is sent using three rungs of increasing control<ref:2608.02841#pg4> The three levels of client-side control are prompt only, a masked composite that cuts the edited region out along a landmark-derived region mask and pastes it back onto the original photograph through a masked composite, and native masked inpainting where supported<ref:2608. p. 0

Where the paper says it

Our contributions are: • A model-agnostic control ladder. Three rungs over the same black-box APIs: prompt-only, a client-side masked composite that cuts the model’s edit out along a landmark-derived region mask and pastes it back onto the original photograph (with alignment, feathered blending, and pose gating), and native masked inpainting where supported.

02841#pg4> The prompt only setting measures what the model does when instructed to edit one region<ref:2608. p. 2
Where the paper says it

The photo and instruction go to the endpoint unmodified and the response is used as-is. This measures what the model does when instructed to edit one region.

02841#pg4> The masked composite is the post-hoc remnant of this family, the final pixel-space blend applied after the API returns, and we measure how much control that remnant still provides<ref:2608. p. 1
Where the paper says it

A black-box API exposes none of those surfaces. Our masked composite is deliberately the post-hoc remnant of this family, the final pixel-space blend applied after the API returns, and we measure how much control that remnant still provides.

02841#pg4>

How the training signal is built

The evaluation protocol employs a two-part diagnostic protocol pairing ArcFace identity with a CIELAB region-localization ratio and its target/off-target components<ref:2608.02841#pg4> The ArcFace cosine measures identity preservation, while the CIELAB pixel-change ratio measures how much change lands inside the requested region rather than a protected facial zone<ref:2608. p. 0

Where the paper says it

Of 210 attempted edits, 196 could be scored. ArcFace cosine measures identity preservation; a CIELAB pixel-change ratio measures how much change lands inside the requested region rather than a protected facial zone.

02841#pg4> The metrics detect identity loss, off-target pixel change, and near-copies but do not measure clinical plausibility<ref:2608. p. 0
Where the paper says it

ArcFace identity is paired with a CIELAB region-localization ratio and its target/off-target components. The metrics detect identity loss, off-target pixel change, and near-copies, but do not measure clinical plausibility.

02841#pg4> Postoperative photographs provide an identity reference with a per-face input baseline<ref:2608. p. 0
Where the paper says it

The metrics detect identity loss, off-target pixel change, and near-copies, but do not measure clinical plausibility. Postoperative photographs provide an identity reference with a per-face input baseline (Sec.

02841#pg5>

Where the gains come from

Masked compositing produces a median localization ratio of 0.985 across the six primary editing configurations<ref:2608. p. 4

Where the paper says it

985 (range 0. 998) across the six primary editing configurations (Fig.

02841#pg7> Their prompt-only outputs have a median of 0. p. 4
Where the paper says it

998) across the six primary editing configurations (Fig. Their prompt-only outputs have a median of 0.

538, while after compositing the median off-target change is reduced to 3. p. 7
Where the paper says it

The top inpaint example has low identity similarity. landmark-derived mask, alignment, and client-side feathered composite reduced that measured off-target change while retaining similar target-region pixel change, uniformly across the six editors tested and at a median provider cost of $0.

50 compared to 3.50 for prompt-only outputs<ref:2608. p. 0
Where the paper says it

A two-order, single-face chain is reported only as a feasibility probe (Sec. To our knowledge, this is the first controlled comparison of client-side region confinement across multiple commercial, arXiv:2608.

02841#pg7> This result demonstrates that preservation is enforced by the blend in masked composite because it copies off-mask pixels by design<ref:2608. p. 4
Where the paper says it

Prompt instructions alone did not preserve the measured keep zone. Post-hoc compositing did so because preservation is enforced by the blend; this result does not assess anatomical correctness inside the mask or changes outside the limited keep zone.

02841#pg4> The tested Qwen inpainting model did not outperform that post-hoc control, but one model cannot support a general conclusion about masked inpainting<ref:2608. p. 7
Where the paper says it

045 per image. The tested Qwen inpainting model did not outperform that post-hoc control, but one model cannot support a general conclusion about masked inpainting.

02841#pg7>

Limitations and Context

The study is a small pilot with 15 faces supporting the main comparisons, and the results do not establish clinical plausibility or anatomical accuracy<ref:2608.02841#pg8> The localization ratio measures CIELAB pixel change, not anatomical direction, clinical plausibility, realism, or expected postoperative appearance<ref:2608. p. 5

Where the paper says it

Each cell contains one generated output, and two models ignore the supplied seed, so stochastic variation is unknown. The localization ratio measures CIELAB pixel change, not anatomical direction, clinical plausibility, realism, or expected postoperative appearance.

02841#pg8> ArcFace’s 0. p. 0
Where the paper says it

Of 210 attempted edits, 196 could be scored. ArcFace cosine measures identity preservation; a CIELAB pixel-change ratio measures how much change lands inside the requested region rather than a protected facial zone.

6 line is a heuristic reference and the postoperative comparison lives entirely in identity-embedding space where edits moved slightly away from the outcome rather than closer to it<ref:2608. p. 5
Where the paper says it

6 line is heuristic. The postoperative comparison, though baselined against each face’s input photograph, lives entirely in identity-embedding space; an edit could approximate the surgical result anatomically while losing embedding similarity, and neither score is a validated surgical-outcome metric.

02841#pg8> The study does not validate cosmetic-surgery outcome prediction, and client-side compositing should be read as a control for off-target pixel changes, not as evidence that a preview is clinically accurate or safe for patient counseling<ref:2608. p. 7
Where the paper says it

The tested Qwen inpainting model did not outperform that post-hoc control, but one model cannot support a general conclusion about masked inpainting. The study does not validate cosmetic-surgery outcome prediction, and client-side compositing should be read as a control for off-target pixel changes, not as evidence that a preview is clinically accurate or safe for patient counseling.

02841#pg8>

Conclusion

The main finding is that landmark-derived mask, alignment, and client-side feathered composite reduced that measured off-target change while retaining similar target-region pixel change uniformly across the six editors tested and at a median provider cost of 0.045 per image<ref:2608. p. 3

Where the paper says it

Across scoreable outputs, median latency is 21 s and median provider cost is $0. 045 per image.

02841#pg8> The pilot establishes that when an editor exposes no internals, region confinement can still be enforced entirely from the client<ref:2608. p. 7
Where the paper says it

It will also need multiple blinded surgeons with inter-rater agreement, a larger and demographically characterized cohort, and immutable or self-hosted model baselines. What the pilot does establish is narrower: when an editor exposes no internals, region confinement can still be enforced entirely from the client.

02841#pg8> The next study should add repeated generations, pose-robust whole-image localization, and anatomy-aware postoperative accuracy metrics<ref:2608. p. 7
Where the paper says it

The main analysis contains 15 faces, one generated output per cell, no surgeon ratings, limited profile support, and only an identityembedding measure of movement toward the postoperative photographs; by that measure the edits moved slightly away from the outcome, not closer. The next study should add repeated generations, pose-robust whole-image localization, and anatomy-aware postoperative accuracy metrics.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Client-side control ladder for image editing APIs allows users to enforce region confinement by using a masked composite that cuts the model’s edit out along a landmark-derived region mask and pastes it back onto the original photograph. This prevents off-target changes while keeping nothing inside the model is reachable for black-box editors.

  2. Two-part diagnostic protocol enables systems to measure identity preservation using ArcFace cosine alongside a CIELAB pixel change ratio to detect both identity loss, off-target pixel change, and near-copies. This allows for a system diagnostic that detects whether an edit is anatomically or clinically plausible.

  3. Implementing prompt guardrails by appending fixed instruction fragments—an avoidance fragment listing unwanted behaviors and a positive fragment anchoring realism—can mitigate generic beautification outputs like smoothed skin, brightened teeth, and subtly enlarged eyes. This forces the model to focus on change only facial tissue and contour as requested.

Abstract

Ask a commercial image editor to preview a cosmetic procedure and it will often change more of the face than the request names: a jawline edit can also smooth the forehead or alter the eyes. Existing methods for confining an edit to one region require access to the model's internals, which a public editing API does not expose. We ask how much control is possible from the client side alone. In a pilot benchmark, five current commercial editing configurations perform facelift-style jaw-neck and rhinoplasty edits on synthetic faces under two levels of client-side control: the prompt alone, and a masked composite that cuts the edited region out of the response and pastes it back onto the original photograph through a landmark-derived mask. Of 162 attempted edits, 151 could be scored. ArcFace cosine measures identity preservation; a CIELAB pixel-change ratio measures how much change lands inside the requested region rather than a protected facial zone. On 15 faces, the masked composite improved localization over the paired prompt-only output by a median of 0.315 (95% face-clustered bootstrap interval 0.279-0.366) while retaining most of the target-region change. Editors differed in edit strength and identity retention, and one returned near-copies for the facelift request. This is a study of control, not clinical accuracy: the faces are synthetic, no surgeons rated the outputs, and each condition was generated once. Within that scope, keeping a surgical preview inside its intended region needs no access to the model; a mask and composite on the client enforce it across every editor tested, at low provider cost.

Sources

Related papers