Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization

arXiv:2606.09909 · cs.CR, cs.AI, cs.CV · Submitted 2026-06-06 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization".

Elias: Here's the gist of the paper. Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization introduces TS-LFO, a two-stage attack framework that systematically recovers the compromised latent-image mapping through a two-stage approach to bypass adversarial perturbation-based copyright protection methods for LDMs.

Nadia: This method achieves unauthorized reconstruction of protected content by exploiting the vulnerability inherent in the objective phenomenon of latent-image mapping disruption.

Elias: Pruning redundant reasoning tokens Existing defenses primarily disrupt the mapping between input images and their latent representations, thereby degrading the model’s ability to produce personalized outputs.

Nadia: To counter this, TS-LFO restores the broken mapping through a two-stage optimization process.

Paper summary: Nadia: So we're looking at this paper called "Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization." It seems like the core idea is tackling how existing defenses try to stop people from creating personalized content with diffusion models.

Elias: Exactly. The authors are proposing a two-stage attack framework, TS-LFO, which systematically recovers that compromised latent image mapping. They claim this method can bypass adversarial perturbation-based copyright protection for Latent Diffusion Models, which is the core architecture for most modern generative systems nineteen twenty-one thirty-five <ref:2606.09909#pg2,adversarial perturbation-based copyright protection>.

Priya: But what does that mean in practice? You’re talking about circumventing defenses designed to suppress unauthorized personalized generation by adding targeted adversarial perturbations based on the loss function of diffusion model personalized training <ref:2606.09909#pg2>.

Nadia: Right, so they argue that these existing protection methods, while showing preliminary effectiveness against naive attempts, have underlying mechanisms that aren't fully robust <ref:2606.09909#pg2>. This paper suggests there are exploitable vulnerabilities in how those defenses work.

Elias: The proposal is to restore the broken mapping through a two-stage optimization process one. In the first stage, they have a Latent Denoising Stage where they enhance semantic consistency between latent codes and input images one.

Priya: How do they actually build that signal for the first stage? I mean, what are these losses that guide the denoising process?

Nadia: They use two specific loss terms for this stage: a Latent-Image Alignment Loss and a Latent Diffusion Loss one. The alignment loss forces an alignment between protected images and their latent representations, which they call Lalign = M(D(z), xadv).

Elias: And the second term is the Latent Diffusion Loss, LDM = E t epsilon epsilon theta (z t, t) minus epsilon squared one. They combine these using a timestep-dependent weight factor lambda t equals lambda l plus (lambda r minus lambda l) T times t to ensure consistent optimization across all timesteps.

Priya: That sounds complex. So, what’s the actual goal of that combination? What are they trying to achieve by minimizing both losses simultaneously?

Nadia: They are effectively suppressing the high-frequency noise that those defenses introduce one. The alignment loss handles one thing, and the diffusion loss simulates the forward noising step in latent space. It's about cleaning up the corrupted latent features over time.

Elias: And then they move to a second stage, which is called Latent Reconstruction Stage one. Here, they recover low-frequency semantic information using pixel-level constraints to refine the latent features one.

Priya: That sounds like they are trying to pull back the structural and low-frequency semantic information that got lost or distorted in the initial perturbation process. Is that how they ensure the final output still looks like what it should?

Nadia: Precisely. This stage is crucial because it makes sure that the semantic meaning of their optimized latent features does not deviate significantly from the original input image one. It maintains latent quality by focusing on structure and semantics.

Paper summary: Elias: The algorithm outlines two main stages: Stage I, Latent Denoising, which optimizes the distribution parameters mu and sigma of the latent feature z through a loop that minimizes Ltotal equals Lalign plus lambda t LDM one.

Priya: So, the process is optimizing those distribution parameters in stage one before they reconstruct the final image in stage two? What kind of architectures can this method handle?

Nadia: It supports diverse autoencoder architectures because they extend the optimization from a deterministic latent feature z to just those latent distribution parameters one. It's flexible enough to work across different setups.

Elias: Now, look at the experimental results mentioned in section B. They show that TS-LFO consistently bypasses state of the art defenses and outperforms attacks like DiffPure, GrIDPure, and IMPRESS across different settings one.

Priya: I’m interested in some specific numbers you mentioned. For example, on CelebA-HQ against SimAC defense, TS-LFO achieves an ISM of zero point four four eight seven and an FID of one hundred seventy-one point six seven eight five, which surpasses IMPRESS with an ISM of zero point three two seven two and an FID of one hundred eighty-nine point two two eight nine one.

Nadia: That comparison shows a significant improvement in metrics when testing against SimAC one. And it’s not just about the main attacks; they also tested it on clean images without defenses, and it improved all the standard metrics there too one.

Elias: They also looked at LSUN-cat against AdvDM defense and showed a FID of two hundred forty-seven point five seven, which is a three point eight improvement over the suboptimal DiffPure one. It’s showing good performance even against methods that aren't the absolute best.

Priya: The visualization results they show that TS-LFO successfully learns original image features under all the protection schemes, and users agree with this, showing a proportion of seventy-eight point one percent of selected samples by users for SimAC defense, compared to forty-nine point four percent for the baseline noise method one.

Nadia: That user study data suggests that people actually perceive a higher similarity to the original images when using TS-LFO one. It's not just a technical metric; it's about perceived quality.

Elias: They also tested it against semantic-level copyright protection, like the DIAGNOSIS method, where the proportion of images classified as containing that semantic watermark dropped to zero point nine two percent after applying TS-LFO one.

Priya: So, if we look at what this paper does for the world, it’s showing how to bypass a specific type of copyright defense in diffusion systems by exploiting latent space vulnerabilities. It suggests that current defenses rely too heavily on perturbing the latent space representation to suppress personalization <ref:2606.09909#pg2>.

Nadia: The implication is that we need more sophisticated defense mechanisms that account for feature vulnerabilities in latent spaces, because this paper shows how to exploit them one. It points toward a necessary balance between copyright protection and open innovation in generative AI.

Elias: Ultimately, TS-LFO systematically recovers the compromised latent image mapping using this two-stage approach one. It highlights that existing defenses have weaknesses that can be exploited if you understand the structure of the latent space.

Conclusion: Nadia: So we're wrapping up this session on TS-LFO, which is about bypassing copyright protections in diffusion models using a two-stage method to recover latent image mappings.

Elias: Yeah, the title itself is pretty descriptive, focusing on how they use latent feature optimization to get around those adversarial attacks that try to stop people from making personalized content.

Priya: It sounds like the authors are saying that these defenses work by messing with the latent space representation between a real image and its model code, but they found a way to fix that disruption.

Nadia: Exactly. The core idea is exploiting a vulnerability in how those mapping disruptions happen, which lets them reconstruct the protected content.

Elias: They did this through two main stages: first, Latent Denoising where they optimize the distribution parameters of the latent features, and second, Latent Reconstruction to pull back low-frequency semantic information using pixel-level constraints.

Priya: So when you look at the overall impact, this suggests that current copyright defenses are focusing too much on just messing with latent space perturbations rather than actually understanding how those features are structured.

Nadia: Right. It points toward a need for more sophisticated defense mechanisms that account for these feature vulnerabilities in latent spaces, not just simple noise injections.

Elias: The authors show this method consistently outperforms existing attacks like DiffPure and GrIDPure on datasets like CelebA-HQ, hitting high scores on metrics like FID and ISM.

Priya: But we have to remember the caveats here, because it's not a magic fix for everything. They also showed that against semantic protection methods, they can reduce the classification of copyrighted images by a significant amount.

Nadia: That's a big win because it shows that this approach isn't just theoretical; it actually works on real-world image quality and user studies show people perceive the reconstructed images as much better.

Elias: It really highlights that balancing copyright protection with the open innovation in generative AI is something we need to think about.

Priya: And it makes you wonder how far this optimization goes, and what new kinds of vulnerabilities might be found in those latent spaces when we start looking at these recovery methods.

Harbin Institute of Technology, Shenzhen University of Science and Technology, Tsinghua Shenzhen International Graduate School

cs.CR, cs.AI, cs.CV

Submitted: 2026-06-06

Updated: 2026-06-06

Comments: accepted by KDD 2026

Journal ref: Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining, V.2, pp. 5867-5878, 2026

DOI: 10.1145/3770855.3817760

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 98/100

The gist: Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization introduces TS-LFO…

Key concepts

Latent Denoising Stage
This first stage optimizes the distribution parameters of the latent feature 'z' by minimizing a combined loss function. It uses a Latent-Image Alignment Loss to match protected images with their latents and a Latent Diffusion Loss to suppress noise introduced by defenses, ensuring semantic consistency.
Latent Reconstruction Stage
The second stage recovers low-frequency semantic information from the perturbed latent features. It uses pixel-level constraints to refine the latent features, ensuring that the reconstructed image's meaning closely matches the original input image.
Latent-Image Alignment Loss (Lalign)
This loss term enforces alignment between protected images and their corresponding latent representations. It ensures that the latent code accurately reflects the semantic content of the original copyrighted image, which is crucial for bypassing defenses.
TS-LFO Framework
This is a two-stage attack method designed to systematically recover the disrupted latent mapping. It optimizes distribution parameters in Stage I and then reconstructs the final image in Stage II, providing a robust way to bypass various adversarial copyright protections.

Terminology

Summary

Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization introduces TS-LFO, a two-stage attack framework that systematically recovers the compromised latent-image mapping through a two-stage approach to bypass adversarial perturbation-based copyright protection methods for LDMs. p. 1

Where the paper says it

In this paper, we take this important finding as the core starting point, and propose a systematic and effective attack method named Two-Stage Latent Feature Optimization (TS-LFO) to bypass the adversarial perturbation-based copyright protection methods for LDMs. The TS-LFO framework is meticulously designed to exploit the vulnerability inherent in the objective phenomenon of latent-image mapping disruption, and it achieves the unauthorized reconstruction of protected content through two stages.

This method achieves unauthorized reconstruction of protected content by exploiting the vulnerability inherent in the objective phenomenon of latent-image mapping disruption. p. 1
Where the paper says it

In this paper, we take this important finding as the core starting point, and propose a systematic and effective attack method named Two-Stage Latent Feature Optimization (TS-LFO) to bypass the adversarial perturbation-based copyright protection methods for LDMs. The TS-LFO framework is meticulously designed to exploit the vulnerability inherent in the objective phenomenon of latent-image mapping disruption, and it achieves the unauthorized reconstruction of protected content through two stages.

Pruning redundant reasoning tokens

Existing defenses primarily disrupt the mapping between input images and their latent representations, thereby degrading the model’s ability to produce personalized outputs. To counter this, TS-LFO restores the broken mapping through a two-stage optimization process. p. 0

Where the paper says it

We begin by observing that existing defenses primarily disrupt the mapping between input images and their latent representations, thereby degrading the model’s ability to produce personalized outputs. To counteract this, TS-LFO restores the broken mapping through a two-stage optimization process.

In the Latent Denoising Stage, the method enhances semantic consistency between latent codes and input images by jointly minimizing a Latent-Image Alignment Loss and a Latent Diffusion Loss with timestep-dependent weights, effectively suppressing the high-frequency noise introduced by defenses. p. 0
Where the paper says it

To counteract this, TS-LFO restores the broken mapping through a two-stage optimization process. In the Latent Denoising Stage, we enhance semantic consistency between latent codes and input images by jointly minimizing a Latent-Image Alignment Loss and a Latent Diffusion Loss with timestep-dependent weights, effectively suppressing the high-frequency noise introduced by defenses.

In the subsequent Latent Reconstruction Stage, it recovers low-frequency semantic information using pixel-level constraints to refine the latent features. p. 0
Where the paper says it

In the Latent Denoising Stage, we enhance semantic consistency between latent codes and input images by jointly minimizing a Latent-Image Alignment Loss and a Latent Diffusion Loss with timestep-dependent weights, effectively suppressing the high-frequency noise introduced by defenses. In the Latent Reconstruction Stage, we recover low-frequency semantic information using pixel-level constraints to refine the latent features.

How the training signal is built

The Latent Denoising Stage utilizes two loss terms: a Latent-Image Alignment Loss and a Latent Diffusion Loss. The Latent-Image Alignment Loss, formulated as Lalign = M(D(z), xadv), enforces alignment between protected images and their latent representations. p. 4

Where the paper says it

3 The First Stage of TS-LFO: Latent Denoising Stage 3. 1 Latent-Image Alignment Loss: Restore the Mapping between Protected Images and Their Latent Representations.

The Latent Diffusion Loss, LDM = E t,ϵ ϵθ (z t, t) − ϵ 22, simulates the forward noising step of the diffusion process in the latent space. p. 5
Where the paper says it

This loss directly simulates and minimizes a key loss term from the downstream DreamBooth training process. Given a latent feature 𝑧 being trained, we simulate the forward noising step of the diffusion process in the latent space: randomly sample a timestep 𝑡 ∼ U (1,𝑇 ), and add noise according to a predefined schedule to obtain the noisy latent feature 𝑧𝑡 .

These two losses are combined using a timestep-dependent weight factor λt = λl + (λr − λl) T · t, ensuring consistent optimization across all timesteps. p. 5
Where the paper says it

(5) In our preliminary experiments, we observe that for a fixed 𝜆, L𝐷𝑀 naturally decays as timestep 𝑡 increases, causing imbalanced optimization: the training prioritizes L𝐷𝑀 reduction at smaller 𝑡 but switches focus to L𝑎𝑙𝑖𝑔𝑛 at larger 𝑡. To enforce consistent optimization across all timesteps, we introduce timestep-dependent weight factors with linear scheduling: Initialization Decoder Unet Noise Image Space Alignment Diffusion Latnet Space Alignment Stage I: Latent Denoising Encoder Latent Decoder Encoder Clip Stage II: Latent Reconstruction Forward Backward Downstream Customization Customize Generator High Fidelity Images ℒ�� ℒ����� ℒ Figure 4: An overview of the complete TS-LFO pipeline, which consists of the Latent Denoising Stage, Latent Reconstruction Stage, and Downstream Customization Stage.

Where the gains come from

The Latent Reconstruction Stage leverages structural and low-frequency semantic information preserved in perturbed latent features to restore low-frequency semantic fidelity through pixel-level constraints. This stage is crucial because it ensures that the semantic meaning of optimized z does not deviate significantly from x, thereby maintaining latent quality. p. 5

Where the paper says it

Here, 𝜖𝑟𝑒𝑐 is a hyperparameter, typically set to 8/255. This constraint ensures the semantic meaning of optimized 𝑧 does not deviate significantly from 𝑥, thereby maintaining latent quality.

The use of a timestep-varying weight factor λt = λl + (λr − λl) T · t, where λl and λr are initial and final weight factors, enforces consistent optimization across all timesteps. p. 5
Where the paper says it

(5) In our preliminary experiments, we observe that for a fixed 𝜆, L𝐷𝑀 naturally decays as timestep 𝑡 increases, causing imbalanced optimization: the training prioritizes L𝐷𝑀 reduction at smaller 𝑡 but switches focus to L𝑎𝑙𝑖𝑔𝑛 at larger 𝑡. To enforce consistent optimization across all timesteps, we introduce timestep-dependent weight factors with linear scheduling: Initialization Decoder Unet Noise Image Space Alignment Diffusion Latnet Space Alignment Stage I: Latent Denoising Encoder Latent Decoder Encoder Clip Stage II: Latent Reconstruction Forward Backward Downstream Customization Customize Generator High Fidelity Images ℒ�� ℒ����� ℒ Figure 4: An overview of the complete TS-LFO pipeline, which consists of the Latent Denoising Stage, Latent Reconstruction Stage, and Downstream Customization Stage.

A Algorithm of our method

The TS-LFO framework is outlined in Fig. 4. The algorithm involves two main stages: Stage I, Latent Denoising, which optimizes the distribution parameters (mu, σ) of the latent feature z through a loop that minimizes Ltotal = Lalign + λt LDM. p. 9

Where the paper says it

A Algorithm of our method Our TS-LFO framework is outlined in Fig. While the main manuscript assumes deterministic latent feature 𝑧 from autoencoder, practical implementations may involve probabilistic encoders like variational autoencoders (VAEs) that output Gaussian distribution parameters (𝜇, 𝜎).

Stage II, Latent Reconstruction, samples a new latent feature znew and then reconstructs the final image xrec ← clamp(D(z), xadv − ϵrec, xadv + ϵrec). p. 5
Where the paper says it

(7) Here 𝑡 ∼ U (1,𝑇 ) denotes the current timestep, 𝑇 is the total diffusion steps, and 𝜆𝑙 and 𝜆𝑟 represent the initial and final weight factors (typically set to 1×10−4 and 3×10−3 respectively). 4 The Second Stage of TS-LFO: Latent Reconstruction Stage Unlike classification tasks, diffusion-based customization is a generative task where output quality critically depends on the input image quality.

This process supports diverse autoencoder architectures by extending the optimization from deterministic z to latent distribution parameters. p. 9
Where the paper says it

1, extending the optimization from deterministic 𝑧 to latent distribution parameters. This enhanced version maintains TS-LFO’s core objective: reconstructing the mapping between input images and latent space while supporting diverse autoencoder architectures.

B More Experimental Results

Extensive experiments show that TS-LFO consistently bypasses state-of-the-art (SOTA) copyright defenses and outperforms SOTA copyright attacks such as DiffPure, GrIDPure and IMPRESS across diverse settings. For example, on CelebA-HQ against SimAC defense, TS-LFO attains an ISM of 0. p. 7

Where the paper says it

On generation quality metrics, TS-LFO outperforms all baselines in both ISM and FID. For example, under PhotoGuard, TS-LFO attains an ISM of 0.

4487 and FID of 171. p. 7
Where the paper says it

For example, under PhotoGuard, TS-LFO attains an ISM of 0. 4487 and FID of 171.

6785, surpassing IMPRESS (ISM: 0. p. 7
Where the paper says it

4487 and FID of 171. 6785, surpassing IMPRESS (ISM: 0.

3272, FID: 189. p. 7
Where the paper says it

6785, surpassing IMPRESS (ISM: 0. 3272, FID: 189.

2289). Furthermore, TS-LFO exhibits general image enhancement properties when applied to clean images (w/o def), improving all metrics. p. 7
Where the paper says it

Furthermore, TS-LFO exhibits general image enhancement properties. When applied to clean images (w/o def), it improves all metrics: FDFR decreases to 0.

The method's performance on LSUN-cat against AdvDM defense shows a FID of 247.57, significantly surpassing the suboptimal DiffPure with a 3. p. 9
Where the paper says it

On LSUN-cat, our FID value of 247. 57 significantly surpasses the suboptimal DiffPure with a 3.

8 improvement. p. 9
Where the paper says it

57 significantly surpasses the suboptimal DiffPure with a 3. 8 improvement.

B.2 Visualization Results and User Study

Visualization results show that TS-LFO has successfully learned original image features under all the protection schemes. The user study also indicates that TS-LFO has again surpassed all the other methods in the user study regarding image quality. p. 8

Where the paper says it

5 Visualization Results and User Study Due to the length constraint policy, the visualization and user study results are presented in Appendix B. 5 Conclusion In this paper, we address the critical challenge of bypassing adversarial copyright defenses in diffusion-based customization by proposing the Two-Stage Latent Feature Optimization (TS-LFO) framework.

Table 9 shows that for SimAC defense, TS-LFO achieves a proportion of 78. p. 9
Where the paper says it

First, our method demonstrates superior attack performance. On CelebAHQ against SimAC defense, our method achieves an FID score of 170.

1% of selected samples by users, which is higher than the baseline Noise method's 49.4%.

B.3 Experiments about Semantic-Level Copyright Protection

When tested against the DIAGNOSIS protection method, TS-LFO can effectively attack it, as the proportion of images classified as containing the DIAGNOSIS semantic watermark decreases to 0.92% after applying TS-LFO. p. 6

Where the paper says it

35, verifying the defenses’ ability to disrupt identifiable features, consistent with existing literature. Notably, applying our TS-LFO causes substantial degradation of defense effectiveness.

This demonstrates that TS-LFO can effectively attack the DIAGNOSIS copyright defense method. p. 8
Where the paper says it

Limitations, Future Direction, and Social Impacts. Although the TS-LFO method demonstrates stronger copyright attack capabilities, its requirement of 6 minutes to process a single 512 × 512 image is indeed a drawback.

B.1 Attack Experiments on More Datasets

The results in Table 8 reveal that our method demonstrates superior attack performance, such as achieving an FID score of 170.15 on CelebA-HQ against SimAC defense, outperforming DiffPure and GrIDPure by margins of 35. p. 9

Where the paper says it

On CelebAHQ against SimAC defense, our method achieves an FID score of 170. 15, outperforming DiffPure and GrIDPure by margins of 35.

35 and 31.35, respectively. p. 5
Where the paper says it

𝜆𝑡 = 𝜆𝑙 + (𝜆𝑟 − 𝜆𝑙) 𝑇 · 𝑡, (6) L = L𝑎𝑙𝑖𝑔𝑛 + 𝜆𝑡 L𝐷𝑀 . (7) Here 𝑡 ∼ U (1,𝑇 ) denotes the current timestep, 𝑇 is the total diffusion steps, and 𝜆𝑙 and 𝜆𝑟 represent the initial and final weight factors (typically set to 1×10−4 and 3×10−3 respectively).

The traditional Gaussian noise method also shows unexpected performance, surpassing SOTA copyright attack methods to become the second-best method behind our TS-LFO on CelebA-HQ. p. 9
Where the paper says it

Furthermore, the traditional Gaussian noise method shows unexpected performance. On the CelebA-HQ dataset, this method can even surpass the SOTA copyright attack methods to become the second-best method behind our TS-LFO.

This indicates that there is still great potential for improvement in the performance of current copyright attack and protection methods. p. 9
Where the paper says it

On the CelebA-HQ dataset, this method can even surpass the SOTA copyright attack methods to become the second-best method behind our TS-LFO. This indicates that there is still great potential for improvement in the performance of current copyright attack and protection methods.

4.5 Visualization Results and User Study

The visualization results show that our method has successfully learned original image features under all the protection schemes. The user study confirms that our TS-LFO has again surpassed all the other methods in the user study. p. 8

Where the paper says it

5 Visualization Results and User Study Due to the length constraint policy, the visualization and user study results are presented in Appendix B. 5 Conclusion In this paper, we address the critical challenge of bypassing adversarial copyright defenses in diffusion-based customization by proposing the Two-Stage Latent Feature Optimization (TS-LFO) framework.

This indicates that our method can generate images with significantly higher similarity to the original ones compared to other copyright attack methods. p. 6
Where the paper says it

We evaluate similarity between generated and original images using FID [16] (lower ↓ better), CLIP [27] Image-Image Similarity (higher ↑ better), and LPIPS [42] (lower ↓ better). Higher similarity indicates more successful copyright theft.

5 Conclusion

The TS-LFO framework systematically recovers the compromised latent-image mapping through a two-stage approach. The investigation reveals that existing defense mechanisms rely on perturbing latent space representations to disrupt the alignment between input images and latent features to suppress personalized generation, leaving exploitable vulnerabilities. p. 8

Where the paper says it

5 Conclusion In this paper, we address the critical challenge of bypassing adversarial copyright defenses in diffusion-based customization by proposing the Two-Stage Latent Feature Optimization (TS-LFO) framework. Our investigation reveals that existing defense mechanisms rely on perturbing latent space representations to disrupt the alignment between input images and latent features to suppress personalized generation, leaving exploitable vulnerabilities.

This work marks an effective solution for circumventing copyright defenses in diffusion-based customization. p. 8
Where the paper says it

Extensive experiments validate that our method successfully bypasses state-of-the-art defenses across diverse settings, achieving high-fidelity content generation even when models are protected by adversarial perturbations. This work marks an effective solution for circumventing copyright defenses in diffusion-based customization, challenging the robustness of current adversarial protection strategies.

Our findings highlight the need for more sophisticated defense mechanisms that account for feature vulnerabilities in latent spaces. p. 8
Where the paper says it

This work marks an effective solution for circumventing copyright defenses in diffusion-based customization, challenging the robustness of current adversarial protection strategies. Our findings highlight the need for more sophisticated defense mechanisms that account for feature vulnerabilities in latent spaces.

Ultimately, this study underscores the importance of balancing copyright protection with open innovation in the era of generative AI. p. 8
Where the paper says it

We hope our work serves as a benchmark for evaluating protection robustness and anticipate more robust defenses against TS-LFO in the future. Ultimately, this study underscores the importance of balancing copyright protection with open innovation in the era of generative AI [24, 44], calling for collaborative efforts to establish ethical frameworks for responsible model customization.

Acknowledgments

This work is supported in part by the National Natural Science Foundation of China under grant 62301189, 62576122,62571298, Guangdong Basic and Applied Basic Research Foundation under grant 2026A1515011139.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Latent Denoising Stage Enhancement: Implement a Latent-Image Alignment Loss to restore semantic consistency between latent codes and input images by minimizing L a l i g n = M(D(z), xadv). This ensures that the optimized latent feature z closely preserves the semantic content of the perturbed image.

  2. Latent Diffusion Loss for Training Dynamics: Introduce a Latent Diffusion Loss to guide optimization by simulating the forward noising step in latent space, defined as LDM = E t, eps epstheta (zt, t) - eps 2. This loss directly minimizes the training loss of the Unet denoiser when using a corrupted latent feature z.

  3. Timestep-Varying Weight Factors: Employ timestep-dependent weight factors lambda t = lambda l + (lambda r - lambda l) T · t to enforce consistent optimization across all diffusion timesteps. This balances the effects of the Latent-Image Alignment Loss and Latent Diffusion Loss, ensuring both high-frequency denoising and low-frequency fidelity restoration occur appropriately during training.

  4. Latent Reconstruction Stage Refinement: Utilize pixel-level constraints in the second stage to recover low-frequency semantic information using znew = E(clamp(D(z), xadv - epsrec, xadv + epsrec)). This step refines the latent features to restore the low-frequency semantic fidelity of protected content while maintaining quality.

  5. Latent Distribution Optimization: Extend the optimization from a deterministic latent feature z to Gaussian distribution parameters (mu, sigma) by using an encoder that outputs these parameters, as described in Algorithm 1 TS-LFO Variant with Gaussian Latent Distribution. This allows the framework to support diverse autoencoder architectures while maintaining the core objective of reconstructing the mapping between input images and latent space.

Abstract

With the growing concerns over copyright infringement in diffusion-based customization, adversarial attacks have emerged as a prominent defense strategy to prevent malicious content forgery in personalized image generation. However, current defenses typically introduce persistent perturbations in the latent space of Latent Diffusion Models (LDMs), which remain susceptible to adaptive bypasses by adversaries. In this paper, we introduce Two-Stage Latent Feature Optimization (TS-LFO), an efficient and effective copyright-stealing attack against protected diffusion-based customization. We begin by observing that existing defenses primarily disrupt the mapping between input images and their latent representations, thereby degrading the model's ability to produce personalized outputs. To counteract this, TS-LFO restores the broken mapping through a two-stage optimization process. In the Latent Denoising Stage, we enhance semantic consistency between latent codes and input images by jointly minimizing a Latent-Image Alignment Loss and a Latent Diffusion Loss with timestep-dependent weights, effectively suppressing the high-frequency noise introduced by defenses. In the Latent Reconstruction Stage, we recover low-frequency semantic information using pixel-level constraints to refine the latent features. Extensive experiments show that TS-LFO consistently bypasses state-of-the-art (SOTA) copyright defenses and outperforms SOTA copyright attacks such as DiffPure, GrIDPure and IMPRESS across diverse settings.

Sources

Related papers