TrojanEdit: Multimodal Backdoor Attack Against Image Editing Model
summary
The gist
Multimodal diffusion models for image editing generate outputs conditioned on both textual instructions and visual inputs…
In short
TrojanEdit addresses a weakness where image editing models ignore visual triggers when combined with text triggers during backdoor training. The authors propose Backdoor Gradients (BKG) Multimodal Balance Learning to dynamically adjust gradient strengths, ensuring the model learns both modalities equally. This results in a highly effective multimodal backdoor attack that maintains normal image editing capabilities.
Key concepts
- Backdoor Modality Bias
- This occurs when combining text and visual triggers directly during training causes the model to rely too heavily on one modality, usually the textual trigger, while suppressing the contribution of the other. This leads to a unimodal backdoor instead of a truly multimodal one.
- Backdoor Gradients (BKG) Multimodal Balance Learning
- This is a proposed solution that dynamically rebalances gradient magnitudes during training. It computes a discrepancy ratio ($ ho$) between modalities and rescales the gradients ($ ilde{G}_{ ext{text}}$ and $ ilde{G}_{ ext{vis}}$) so that both visual and textual signals contribute equally to the learning process.
- Attack Success Rate (ASR)
- ASR measures how often the TrojanEdit attack successfully modifies an image according to the desired backdoor instruction. The framework achieves an ASR exceeding 90% on tested models, demonstrating its high effectiveness in injecting a hidden trigger.
- Dynamic Gradient Balancing
- This technique involves calculating a discrepancy ratio ($ ho$) and using it to scale the gradients of different modalities. By setting $ ilde{G}_{ ext{text}} = 1/ ho G_{ ext{text}}$ and $ ilde{G}_{ ext{vis}} = ho G_{ ext{vis}}$, the method ensures that both textual and visual gradient signals have balanced influence on the model's updates.
Terminology used across episodes
This episode discusses
- TrojanEdit: Multimodal Backdoor Attack Against Image Editing Model · Paper Radio
- Prompt-to-Prompt Image Editing with Cross Attention Control
- SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
- LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs
The paper
TrojanEdit: Multimodal Backdoor Attack Against Image Editing Model · Read on arXiv
Laboratory of Intelligent Collaborative Computing, University of Electronic Science and Technology of China, Chengdu, 611731, China · School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China · School of Computer Science and Artificial Intelligence, Wuhan University of Technology
DOI: 10.1016/j.neucom.2026.133346
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: We're looking at the paper: "TrojanEdit: Multimodal Backdoor Attack Against Image Editing Model".
Paper summary: Nadia: So, to wrap up on TrojanEdit: Multimodal Backdoor Attack Against Image Editing Model, what's the final word on what they proved? Elias They demonstrated that they can build a backdoor attack by implanting triggers into both textual prompts and corresponding input images simultaneously.
Priya: The key finding seems to be their method, Backdoor Gradients Multimodal Balance Learning, which dynamically adjusts the gradient magnitudes during training to ensure the model learns both modalities equally.
Elias: That dynamic rebalancing is what allows them to achieve high attack effectiveness, showing an attack success rate of ninety-eight point zero zero on InstructPix2Pix and ninety-seven point seven five on SDEdit-E, while keeping normal editing performance intact.
Nadia: It means we can now build attacks that are genuinely multimodal without sacrificing the model's ability to perform standard image editing tasks on clean inputs. Priya That’s the big implication for safety research, showing that focusing only on unimodal vulnerabilities leaves a gap in what we can actually exploit.
Elias: The authors concluded that a well-balanced multimodal trigger is more suitable for backdoor attacks in these models than any unimodal counterpart they've tested. Nadia So, the main point is achieving strong attack effectiveness while minimally affecting normal functionality by balancing the learning process between text and visual signals.
Conclusion: Nadia: So we're wrapping up on TrojanEdit, looking at that title again—"TrojanEdit: Multimodal Backdoor Attack Against Image Editing Model." Basically, this paper is about how you can build a backdoor attack that uses both text and images at the same time against these image editing models.
Elias: Right. It’s not just injecting a trigger into the text prompt, or just messing with the image itself; they are combining them dynamically to make the attack stronger across both channels simultaneously.
Priya: What this really shows us is that we can move beyond just testing one type of vulnerability in these multimodal systems and start thinking about attacks that exploit how different data types interact.
Nadia: Exactly. The core idea is that by forcing the model to learn from both modalities at the same time, you can create a backdoor that doesn't just rely on one input—say, just the text—but is truly multimodal in its trigger mechanism.
Elias: And their method, this Backdoor Gradients Multimodal Balance Learning thing they propose... it’s about dynamically adjusting how much influence each modality has during training so the model learns to pay attention to both equally.
Priya: From a measurement standpoint, what's interesting is that even with this balanced approach, they still maintain normal image editing capabilities for clean inputs, which keeps the attack practical.
Nadia: That’s crucial because it means we’re not just creating a broken model; we’re creating an exploitable backdoor that doesn't immediately ruin the model's everyday function.
Elias: The authors show that this balanced approach works better than simply stacking triggers on top of each other, which usually makes the model ignore the weaker signal.
Priya: So, the big implication for me is that if we build defenses against one type of trigger—say, text-based ones—they might still be vulnerable to a balanced multimodal attack if they aren't looking at the interaction between modalities.
Nadia: Right. It shifts our focus from just finding triggers to understanding how these different inputs are woven together by the model during its learning process.
Elias: This leads us nicely into what kind of triggers actually work best—are they semantic matches, or something else entirely?
More episodes
- 2608.27496-ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection
- Daily Summary for 2026-11-19
- 2607.07085-Is Randomness Necessary for Adaptive Data Analysis?
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2609.10601-Threshold Choice, Not Sample Size, Bounds Trustless Verification of Nondeterministic Compound AI Workflows
- 2610.38786-Adversarial Process Disturbances at a Deployable Alarm Budget: Detector Evaluation on a Simulator and a Real Plant
- 2511.08660-Binary and Multiclass Cyberattack Classification on GeNIS Dataset
- 2604.27302-Static Attribution of Android Residential Proxy Malware Using Graph Kernels
- Daily Summary for 2026-11-18