Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: We're looking at the paper: "Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization".
Elias: Here's the gist of the paper. Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization introduces TS-LFO, a two-stage attack framework that systematically recovers the compromised latent-image mapping through a two-stage approach to bypass adversarial perturbation-based copyright protection methods for LDMs.
Nadia: This method achieves unauthorized reconstruction of protected content by exploiting the vulnerability inherent in the objective phenomenon of latent-image mapping disruption.
Elias: Pruning redundant reasoning tokens Existing defenses primarily disrupt the mapping between input images and their latent representations, thereby degrading the model’s ability to produce personalized outputs.
Nadia: To counter this, TS-LFO restores the broken mapping through a two-stage optimization process.
Paper summary: Nadia: So we're looking at this paper called "Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization." It seems like the core idea is tackling how existing defenses try to stop people from creating personalized content with diffusion models.
Elias: Exactly. The authors are proposing a two-stage attack framework, TS-LFO, which systematically recovers that compromised latent image mapping. They claim this method can bypass adversarial perturbation-based copyright protection for Latent Diffusion Models, which is the core architecture for most modern generative systems nineteen twenty-one thirty-five <ref:2606.09909#pg2,adversarial perturbation-based copyright protection>.
Priya: But what does that mean in practice? You’re talking about circumventing defenses designed to suppress unauthorized personalized generation by adding targeted adversarial perturbations based on the loss function of diffusion model personalized training <ref:2606.09909#pg2>.
Nadia: Right, so they argue that these existing protection methods, while showing preliminary effectiveness against naive attempts, have underlying mechanisms that aren't fully robust <ref:2606.09909#pg2>. This paper suggests there are exploitable vulnerabilities in how those defenses work.
Elias: The proposal is to restore the broken mapping through a two-stage optimization process one. In the first stage, they have a Latent Denoising Stage where they enhance semantic consistency between latent codes and input images one.
Priya: How do they actually build that signal for the first stage? I mean, what are these losses that guide the denoising process?
Nadia: They use two specific loss terms for this stage: a Latent-Image Alignment Loss and a Latent Diffusion Loss one. The alignment loss forces an alignment between protected images and their latent representations, which they call Lalign = M(D(z), xadv).
Elias: And the second term is the Latent Diffusion Loss, LDM = E t epsilon epsilon theta (z t, t) minus epsilon squared one. They combine these using a timestep-dependent weight factor lambda t equals lambda l plus (lambda r minus lambda l) T times t to ensure consistent optimization across all timesteps.
Priya: That sounds complex. So, what’s the actual goal of that combination? What are they trying to achieve by minimizing both losses simultaneously?
Nadia: They are effectively suppressing the high-frequency noise that those defenses introduce one. The alignment loss handles one thing, and the diffusion loss simulates the forward noising step in latent space. It's about cleaning up the corrupted latent features over time.
Elias: And then they move to a second stage, which is called Latent Reconstruction Stage one. Here, they recover low-frequency semantic information using pixel-level constraints to refine the latent features one.
Priya: That sounds like they are trying to pull back the structural and low-frequency semantic information that got lost or distorted in the initial perturbation process. Is that how they ensure the final output still looks like what it should?
Nadia: Precisely. This stage is crucial because it makes sure that the semantic meaning of their optimized latent features does not deviate significantly from the original input image one. It maintains latent quality by focusing on structure and semantics.
Paper summary: Elias: The algorithm outlines two main stages: Stage I, Latent Denoising, which optimizes the distribution parameters mu and sigma of the latent feature z through a loop that minimizes Ltotal equals Lalign plus lambda t LDM one.
Priya: So, the process is optimizing those distribution parameters in stage one before they reconstruct the final image in stage two? What kind of architectures can this method handle?
Nadia: It supports diverse autoencoder architectures because they extend the optimization from a deterministic latent feature z to just those latent distribution parameters one. It's flexible enough to work across different setups.
Elias: Now, look at the experimental results mentioned in section B. They show that TS-LFO consistently bypasses state of the art defenses and outperforms attacks like DiffPure, GrIDPure, and IMPRESS across different settings one.
Priya: I’m interested in some specific numbers you mentioned. For example, on CelebA-HQ against SimAC defense, TS-LFO achieves an ISM of zero point four four eight seven and an FID of one hundred seventy-one point six seven eight five, which surpasses IMPRESS with an ISM of zero point three two seven two and an FID of one hundred eighty-nine point two two eight nine one.
Nadia: That comparison shows a significant improvement in metrics when testing against SimAC one. And it’s not just about the main attacks; they also tested it on clean images without defenses, and it improved all the standard metrics there too one.
Elias: They also looked at LSUN-cat against AdvDM defense and showed a FID of two hundred forty-seven point five seven, which is a three point eight improvement over the suboptimal DiffPure one. It’s showing good performance even against methods that aren't the absolute best.
Priya: The visualization results they show that TS-LFO successfully learns original image features under all the protection schemes, and users agree with this, showing a proportion of seventy-eight point one percent of selected samples by users for SimAC defense, compared to forty-nine point four percent for the baseline noise method one.
Nadia: That user study data suggests that people actually perceive a higher similarity to the original images when using TS-LFO one. It's not just a technical metric; it's about perceived quality.
Elias: They also tested it against semantic-level copyright protection, like the DIAGNOSIS method, where the proportion of images classified as containing that semantic watermark dropped to zero point nine two percent after applying TS-LFO one.
Priya: So, if we look at what this paper does for the world, it’s showing how to bypass a specific type of copyright defense in diffusion systems by exploiting latent space vulnerabilities. It suggests that current defenses rely too heavily on perturbing the latent space representation to suppress personalization <ref:2606.09909#pg2>.
Nadia: The implication is that we need more sophisticated defense mechanisms that account for feature vulnerabilities in latent spaces, because this paper shows how to exploit them one. It points toward a necessary balance between copyright protection and open innovation in generative AI.
Elias: Ultimately, TS-LFO systematically recovers the compromised latent image mapping using this two-stage approach one. It highlights that existing defenses have weaknesses that can be exploited if you understand the structure of the latent space.
Conclusion: Nadia: So we're wrapping up this session on TS-LFO, which is about bypassing copyright protections in diffusion models using a two-stage method to recover latent image mappings.
Elias: Yeah, the title itself is pretty descriptive, focusing on how they use latent feature optimization to get around those adversarial attacks that try to stop people from making personalized content.
Priya: It sounds like the authors are saying that these defenses work by messing with the latent space representation between a real image and its model code, but they found a way to fix that disruption.
Nadia: Exactly. The core idea is exploiting a vulnerability in how those mapping disruptions happen, which lets them reconstruct the protected content.
Elias: They did this through two main stages: first, Latent Denoising where they optimize the distribution parameters of the latent features, and second, Latent Reconstruction to pull back low-frequency semantic information using pixel-level constraints.
Priya: So when you look at the overall impact, this suggests that current copyright defenses are focusing too much on just messing with latent space perturbations rather than actually understanding how those features are structured.
Nadia: Right. It points toward a need for more sophisticated defense mechanisms that account for these feature vulnerabilities in latent spaces, not just simple noise injections.
Elias: The authors show this method consistently outperforms existing attacks like DiffPure and GrIDPure on datasets like CelebA-HQ, hitting high scores on metrics like FID and ISM.
Priya: But we have to remember the caveats here, because it's not a magic fix for everything. They also showed that against semantic protection methods, they can reduce the classification of copyrighted images by a significant amount.
Nadia: That's a big win because it shows that this approach isn't just theoretical; it actually works on real-world image quality and user studies show people perceive the reconstructed images as much better.
Elias: It really highlights that balancing copyright protection with the open innovation in generative AI is something we need to think about.
Priya: And it makes you wonder how far this optimization goes, and what new kinds of vulnerabilities might be found in those latent spaces when we start looking at these recovery methods.
Harbin Institute of Technology, Shenzhen University of Science and Technology, Tsinghua Shenzhen International Graduate School
cs.CR, cs.AI, cs.CV
Submitted: 2026-06-06
Updated: 2026-06-06
Comments: accepted by KDD 2026
Journal ref: Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining, V.2, pp. 5867-5878, 2026
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 98/100
The gist: Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization introduces TS-LFO…
Key concepts
- Latent Denoising Stage
- This first stage optimizes the distribution parameters of the latent feature 'z' by minimizing a combined loss function. It uses a Latent-Image Alignment Loss to match protected images with their latents and a Latent Diffusion Loss to suppress noise introduced by defenses, ensuring semantic consistency.
- Latent Reconstruction Stage
- The second stage recovers low-frequency semantic information from the perturbed latent features. It uses pixel-level constraints to refine the latent features, ensuring that the reconstructed image's meaning closely matches the original input image.
- Latent-Image Alignment Loss (Lalign)
- This loss term enforces alignment between protected images and their corresponding latent representations. It ensures that the latent code accurately reflects the semantic content of the original copyrighted image, which is crucial for bypassing defenses.
- TS-LFO Framework
- This is a two-stage attack method designed to systematically recover the disrupted latent mapping. It optimizes distribution parameters in Stage I and then reconstructs the final image in Stage II, providing a robust way to bypass various adversarial copyright protections.
Terminology
Summary
Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization introduces TS-LFO, a two-stage attack framework that systematically recovers the compromised latent-image mapping through a two-stage approach to bypass adversarial perturbation-based copyright protection methods for LDMs. p. 1 In this paper, we take this important finding as the core starting point, and propose a systematic and effective attack method
named Two-Stage Latent Feature Optimization (TS-LFO) to bypass
the adversarial perturbation-based copyright protection methods
for LDMs. The TS-LFO framework is meticulously designed to exploit the vulnerability inherent in the objective phenomenon of
latent-image mapping disruption, and it achieves the unauthorized
reconstruction of protected content through two stages. In this paper, we take this important finding as the core starting point, and propose a systematic and effective attack method
named Two-Stage Latent Feature Optimization (TS-LFO) to bypass
the adversarial perturbation-based copyright protection methods
for LDMs. The TS-LFO framework is meticulously designed to exploit the vulnerability inherent in the objective phenomenon of
latent-image mapping disruption, and it achieves the unauthorized
reconstruction of protected content through two stages.Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
Existing defenses primarily disrupt the mapping between input images and their latent representations, thereby degrading the model’s ability to produce personalized outputs. To counter this, TS-LFO restores the broken mapping through a two-stage optimization process. p. 0 We
begin by observing that existing defenses primarily disrupt the mapping between input images and their latent representations, thereby
degrading the model’s ability to produce personalized outputs. To
counteract this, TS-LFO restores the broken mapping through a
two-stage optimization process. To
counteract this, TS-LFO restores the broken mapping through a
two-stage optimization process. In the Latent Denoising Stage, we
enhance semantic consistency between latent codes and input images by jointly minimizing a Latent-Image Alignment Loss and a
Latent Diffusion Loss with timestep-dependent weights, effectively
suppressing the high-frequency noise introduced by defenses. In the Latent Denoising Stage, we
enhance semantic consistency between latent codes and input images by jointly minimizing a Latent-Image Alignment Loss and a
Latent Diffusion Loss with timestep-dependent weights, effectively
suppressing the high-frequency noise introduced by defenses. In
the Latent Reconstruction Stage, we recover low-frequency semantic information using pixel-level constraints to refine the latent
features.Where the paper says it
Where the paper says it
Where the paper says it
How the training signal is built
The Latent Denoising Stage utilizes two loss terms: a Latent-Image Alignment Loss and a Latent Diffusion Loss. The Latent-Image Alignment Loss, formulated as Lalign = M(D(z), xadv), enforces alignment between protected images and their latent representations. p. 4 3 The First Stage of TS-LFO: Latent Denoising
Stage
3. 1 Latent-Image Alignment Loss: Restore the Mapping
between Protected Images and Their Latent Representations. This loss directly simulates and minimizes a key loss term
from the downstream DreamBooth training process. Given a latent feature 𝑧 being trained, we simulate the forward
noising step of the diffusion process in the latent space: randomly
sample a timestep 𝑡 ∼ U (1,𝑇 ), and add noise according to a predefined schedule to obtain the noisy latent feature 𝑧𝑡
. (5)
In our preliminary experiments, we observe that for a fixed 𝜆,
L𝐷𝑀 naturally decays as timestep 𝑡 increases, causing imbalanced
optimization: the training prioritizes L𝐷𝑀 reduction at smaller
𝑡 but switches focus to L𝑎𝑙𝑖𝑔𝑛 at larger 𝑡. To enforce consistent
optimization across all timesteps, we introduce timestep-dependent
weight factors with linear scheduling:
Initialization
Decoder
Unet
Noise
Image Space Alignment
Diffusion Latnet Space Alignment
Stage I: Latent Denoising
Encoder
Latent
Decoder Encoder
Clip
Stage II: Latent Reconstruction
Forward
Backward
Downstream
Customization
Customize
Generator
High Fidelity
Images
ℒ��
ℒ�����
ℒ
Figure 4: An overview of the complete TS-LFO pipeline,
which consists of the Latent Denoising Stage, Latent Reconstruction Stage, and Downstream Customization Stage.Where the paper says it
Where the paper says it
Where the paper says it
Where the gains come from
The Latent Reconstruction Stage leverages structural and low-frequency semantic information preserved in perturbed latent features to restore low-frequency semantic fidelity through pixel-level constraints. This stage is crucial because it ensures that the semantic meaning of optimized z does not deviate significantly from x, thereby maintaining latent quality. p. 5 Here,
𝜖𝑟𝑒𝑐 is a hyperparameter, typically set to 8/255. This constraint ensures the semantic meaning of optimized 𝑧
does not deviate significantly from 𝑥, thereby maintaining latent
quality. (5)
In our preliminary experiments, we observe that for a fixed 𝜆,
L𝐷𝑀 naturally decays as timestep 𝑡 increases, causing imbalanced
optimization: the training prioritizes L𝐷𝑀 reduction at smaller
𝑡 but switches focus to L𝑎𝑙𝑖𝑔𝑛 at larger 𝑡. To enforce consistent
optimization across all timesteps, we introduce timestep-dependent
weight factors with linear scheduling:
Initialization
Decoder
Unet
Noise
Image Space Alignment
Diffusion Latnet Space Alignment
Stage I: Latent Denoising
Encoder
Latent
Decoder Encoder
Clip
Stage II: Latent Reconstruction
Forward
Backward
Downstream
Customization
Customize
Generator
High Fidelity
Images
ℒ��
ℒ�����
ℒ
Figure 4: An overview of the complete TS-LFO pipeline,
which consists of the Latent Denoising Stage, Latent Reconstruction Stage, and Downstream Customization Stage.Where the paper says it
Where the paper says it
A Algorithm of our method
The TS-LFO framework is outlined in Fig. 4. The algorithm involves two main stages: Stage I, Latent Denoising, which optimizes the distribution parameters (mu, σ) of the latent feature z through a loop that minimizes Ltotal = Lalign + λt LDM. p. 9 A Algorithm of our method
Our TS-LFO framework is outlined in Fig. While the main manuscript assumes deterministic latent feature 𝑧 from autoencoder,
practical implementations may involve probabilistic encoders like
variational autoencoders (VAEs) that output Gaussian distribution
parameters (𝜇, 𝜎). (7)
Here 𝑡 ∼ U (1,𝑇 ) denotes the current timestep, 𝑇 is the total
diffusion steps, and 𝜆𝑙 and 𝜆𝑟 represent the initial and final weight
factors (typically set to 1×10−4
and 3×10−3
respectively). 4 The Second Stage of TS-LFO: Latent
Reconstruction Stage
Unlike classification tasks, diffusion-based customization is a generative task where output quality critically depends on the input
image quality. 1, extending the optimization from deterministic 𝑧 to latent distribution parameters. This enhanced version maintains TS-LFO’s
core objective: reconstructing the mapping between input images and
latent space while supporting diverse autoencoder architectures.Where the paper says it
Where the paper says it
Where the paper says it
B More Experimental Results
Extensive experiments show that TS-LFO consistently bypasses state-of-the-art (SOTA) copyright defenses and outperforms SOTA copyright attacks such as DiffPure, GrIDPure and IMPRESS across diverse settings. For example, on CelebA-HQ against SimAC defense, TS-LFO attains an ISM of 0. p. 7 On generation quality metrics, TS-LFO outperforms all baselines
in both ISM and FID. For example, under PhotoGuard, TS-LFO
attains an ISM of 0. For example, under PhotoGuard, TS-LFO
attains an ISM of 0. 4487 and FID of 171. 4487 and FID of 171. 6785, surpassing IMPRESS
(ISM: 0. 6785, surpassing IMPRESS
(ISM: 0. 3272, FID: 189. Furthermore, TS-LFO exhibits general image enhancement properties. When applied to clean images (w/o def), it improves all
metrics: FDFR decreases to 0. On LSUN-cat, our FID value of 247. 57 significantly surpasses the suboptimal DiffPure with a 3. 57 significantly surpasses the suboptimal DiffPure with a 3. 8 improvement.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
B.2 Visualization Results and User Study
Visualization results show that TS-LFO has successfully learned original image features under all the protection schemes. The user study also indicates that TS-LFO has again surpassed all the other methods in the user study regarding image quality. p. 8 5 Visualization Results and User Study
Due to the length constraint policy, the visualization and user study
results are presented in Appendix B. 5 Conclusion
In this paper, we address the critical challenge of bypassing adversarial copyright defenses in diffusion-based customization by
proposing the Two-Stage Latent Feature Optimization (TS-LFO)
framework. First,
our method demonstrates superior attack performance. On CelebAHQ against SimAC defense, our method achieves an FID score of
170.Where the paper says it
Where the paper says it
B.3 Experiments about Semantic-Level Copyright Protection
When tested against the DIAGNOSIS protection method, TS-LFO can effectively attack it, as the proportion of images classified as containing the DIAGNOSIS semantic watermark decreases to 0.92% after applying TS-LFO. p. 6 35, verifying the defenses’ ability
to disrupt identifiable features, consistent with existing literature. Notably, applying our TS-LFO causes substantial degradation
of defense effectiveness. Limitations, Future Direction, and Social Impacts. Although
the TS-LFO method demonstrates stronger copyright attack capabilities, its requirement of 6 minutes to process a single 512 × 512
image is indeed a drawback.Where the paper says it
Where the paper says it
B.1 Attack Experiments on More Datasets
The results in Table 8 reveal that our method demonstrates superior attack performance, such as achieving an FID score of 170.15 on CelebA-HQ against SimAC defense, outperforming DiffPure and GrIDPure by margins of 35. p. 9 On CelebAHQ against SimAC defense, our method achieves an FID score of
170. 15, outperforming DiffPure and GrIDPure by margins of 35. 𝜆𝑡 = 𝜆𝑙 +
(𝜆𝑟 − 𝜆𝑙)
𝑇
· 𝑡, (6)
L = L𝑎𝑙𝑖𝑔𝑛 + 𝜆𝑡 L𝐷𝑀 . (7)
Here 𝑡 ∼ U (1,𝑇 ) denotes the current timestep, 𝑇 is the total
diffusion steps, and 𝜆𝑙 and 𝜆𝑟 represent the initial and final weight
factors (typically set to 1×10−4
and 3×10−3
respectively). Furthermore, the traditional Gaussian noise method shows unexpected performance. On the CelebA-HQ dataset, this method can
even surpass the SOTA copyright attack methods to become the
second-best method behind our TS-LFO. On the CelebA-HQ dataset, this method can
even surpass the SOTA copyright attack methods to become the
second-best method behind our TS-LFO. This indicates that there is
still great potential for improvement in the performance of current
copyright attack and protection methods.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
4.5 Visualization Results and User Study
The visualization results show that our method has successfully learned original image features under all the protection schemes. The user study confirms that our TS-LFO has again surpassed all the other methods in the user study. p. 8 5 Visualization Results and User Study
Due to the length constraint policy, the visualization and user study
results are presented in Appendix B. 5 Conclusion
In this paper, we address the critical challenge of bypassing adversarial copyright defenses in diffusion-based customization by
proposing the Two-Stage Latent Feature Optimization (TS-LFO)
framework. We evaluate similarity between generated and original images using FID [16] (lower ↓ better), CLIP [27] Image-Image
Similarity (higher ↑ better), and LPIPS [42] (lower ↓ better). Higher
similarity indicates more successful copyright theft.Where the paper says it
Where the paper says it
5 Conclusion
The TS-LFO framework systematically recovers the compromised latent-image mapping through a two-stage approach. The investigation reveals that existing defense mechanisms rely on perturbing latent space representations to disrupt the alignment between input images and latent features to suppress personalized generation, leaving exploitable vulnerabilities. p. 8 5 Conclusion
In this paper, we address the critical challenge of bypassing adversarial copyright defenses in diffusion-based customization by
proposing the Two-Stage Latent Feature Optimization (TS-LFO)
framework. Our investigation reveals that existing defense mechanisms rely on perturbing latent space representations to disrupt the
alignment between input images and latent features to suppress
personalized generation, leaving exploitable vulnerabilities. Extensive experiments
validate that our method successfully bypasses state-of-the-art defenses across diverse settings, achieving high-fidelity content generation even when models are protected by adversarial perturbations. This work marks an effective solution for circumventing copyright defenses in diffusion-based customization, challenging the
robustness of current adversarial protection strategies. This work marks an effective solution for circumventing copyright defenses in diffusion-based customization, challenging the
robustness of current adversarial protection strategies. Our findings
highlight the need for more sophisticated defense mechanisms that
account for feature vulnerabilities in latent spaces. We hope our work serves as a benchmark for evaluating protection
robustness and anticipate more robust defenses against TS-LFO in
the future. Ultimately, this study underscores the importance of
balancing copyright protection with open innovation in the era of
generative AI [24, 44], calling for collaborative efforts to establish
ethical frameworks for responsible model customization.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Acknowledgments
This work is supported in part by the National Natural Science Foundation of China under grant 62301189, 62576122,62571298, Guangdong Basic and Applied Basic Research Foundation under grant 2026A1515011139.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Latent Denoising Stage Enhancement: Implement a Latent-Image Alignment Loss to restore semantic consistency between latent codes and input images by minimizing L a l i g n = M(D(z), xadv). This ensures that the optimized latent feature z closely preserves the semantic content of the perturbed image.
-
Latent Diffusion Loss for Training Dynamics: Introduce a Latent Diffusion Loss to guide optimization by simulating the forward noising step in latent space, defined as LDM = E t, eps epstheta (zt, t) - eps 2. This loss directly minimizes the training loss of the Unet denoiser when using a corrupted latent feature z.
-
Timestep-Varying Weight Factors: Employ timestep-dependent weight factors lambda t = lambda l + (lambda r - lambda l) T · t to enforce consistent optimization across all diffusion timesteps. This balances the effects of the Latent-Image Alignment Loss and Latent Diffusion Loss, ensuring both high-frequency denoising and low-frequency fidelity restoration occur appropriately during training.
-
Latent Reconstruction Stage Refinement: Utilize pixel-level constraints in the second stage to recover low-frequency semantic information using znew = E(clamp(D(z), xadv - epsrec, xadv + epsrec)). This step refines the latent features to restore the low-frequency semantic fidelity of protected content while maintaining quality.
-
Latent Distribution Optimization: Extend the optimization from a deterministic latent feature z to Gaussian distribution parameters (mu, sigma) by using an encoder that outputs these parameters, as described in Algorithm 1 TS-LFO Variant with Gaussian Latent Distribution. This allows the framework to support diverse autoencoder architectures while maintaining the core objective of reconstructing the mapping between input images and latent space.
Abstract
With the growing concerns over copyright infringement in diffusion-based customization, adversarial attacks have emerged as a prominent defense strategy to prevent malicious content forgery in personalized image generation. However, current defenses typically introduce persistent perturbations in the latent space of Latent Diffusion Models (LDMs), which remain susceptible to adaptive bypasses by adversaries. In this paper, we introduce Two-Stage Latent Feature Optimization (TS-LFO), an efficient and effective copyright-stealing attack against protected diffusion-based customization. We begin by observing that existing defenses primarily disrupt the mapping between input images and their latent representations, thereby degrading the model's ability to produce personalized outputs. To counteract this, TS-LFO restores the broken mapping through a two-stage optimization process. In the Latent Denoising Stage, we enhance semantic consistency between latent codes and input images by jointly minimizing a Latent-Image Alignment Loss and a Latent Diffusion Loss with timestep-dependent weights, effectively suppressing the high-frequency noise introduced by defenses. In the Latent Reconstruction Stage, we recover low-frequency semantic information using pixel-level constraints to refine the latent features. Extensive experiments show that TS-LFO consistently bypasses state-of-the-art (SOTA) copyright defenses and outperforms SOTA copyright attacks such as DiffPure, GrIDPure and IMPRESS across diverse settings.
Sources
- Editable-DeepSC: Reliable Cross-Modal Semantic Communications for Facial Editing
- Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defenses
- Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization
- An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examples
- DreamFusion: Text-to-3D using 2D Diffusion
- Raising the Cost of Malicious AI-Powered Image Editing
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data