Mirage: Targeted Jailbreaks Against Post-Generation Safety Checkers in Multimodal Diffusion Models

summary

The gist

The gist The proposed attack, Mirage, demonstrates that post-generation safety checkers in multimodal diffusion models can be bypassed using targeted jailbreak attacks while preserving fine-grained

In short

Mirage is a multimodal jailbreak attack that bypasses post-generation safety checkers in diffusion models. It works by hiding an adversarial target, like an NSFW image or text, within a small change to a benign input image. Mirage uses shared encoder spaces to align the perturbed image with the target while pushing it away from unsafe concepts, effectively evading various safety filters without needing internal model details.

Key concepts

Multimodal Jailbreak Attack: This attack targets diffusion models that handle both images and text. Instead of just changing the text prompt, Mirage modifies an input image slightly. It uses this small change to carry a hidden malicious meaning, allowing it to bypass safety checks designed for either modality.
Shared Representation Space: Multimodal encoders (like those in diffusion models) use a common mathematical space to understand both images and text simultaneously. Mirage exploits this shared space by aligning the perturbed image representation with the target's representation while simultaneously pushing it away from representations associated with unsafe content like NSFW concepts.
Contrastive Objective: This is a mathematical goal used during attack construction. Mirage uses this objective to create an adversarial perturbation that achieves two things: first, it keeps the meaning of the original benign image close to the target's meaning; second, it ensures this new representation is mathematically distant from representations of unsafe concepts.
Black-box Query Refinement: Since Mirage operates without knowing the safety checker's internal workings, it uses a technique called black-box query refinement. This involves making small adjustments to the attack perturbation based only on the final feedback received from the target diffusion pipeline, helping it navigate unknown safety boundaries more effectively.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 26 episodes
Flow Matching — explained in 22 episodes
Quantum Fisher Information (QFI) — explained in 21 episodes
Retrieval-Augmented Generation (RAG) — explained in 21 episodes
Sample Complexity — explained in 20 episodes
Federated Learning — explained in 18 episodes

This episode discusses

The paper

Mirage: Targeted Jailbreaks Against Post-Generation Safety Checkers in Multimodal Diffusion Models · Read on arXiv

Washington University in St. Louis · Virginia Tech

Post-generation safety checkers are widely employed in diffusion-model pipelines to prevent unsafe generated content from reaching users. In this paper, we show that these safety checkers can still be bypassed by our proposed multimodal jailbreak attack, Mirage. Given an adversarial target, which can be either a reference NSFW image or a malicious text description, Mirage conceals the target within an imperceptible perturbation applied to a benign input image. Mirage exploits the shared representation space of multimodal encoders to jointly align the perturbed image with the cross-modal adversarial target while repelling its representation from commonly filtered NSFW concepts. Such semantic separation can transfer across representation spaces, enabling Mirage to evade heterogeneous safety checkers without accessing their internal details. We further employ lightweight black-box query refinement to improve evasion near the unknown safety boundary. We implemented Mirage on Stable Diffusion 1.5, Stable Diffusion XL, and Kandinsky 2.2, and evaluated it on two cross-modal NSFW datasets, covering both text-to-image and image-to-image generation scenarios. The experimental results demonstrate that Mirage achieves strong targeted jailbreak performance across diverse post-generation safety checkers with limited black-box queries while preserving high semantic alignment with the adversarial targets.

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "Mirage: Targeted Jailbreaks Against Post-Generation Safety Checkers in Multimodal Diffusion Models".

Elias: Here's the gist of the paper. The gist The proposed attack, Mirage, demonstrates that post-generation safety checkers in multimodal diffusion models can be bypassed using targeted jailbreak attacks while preserving fine-grained malicious target semantics.

Nadia: How it works Mirage conceals the adversarial target within a small perturbation applied to a benign input image while leaving the text prompt unchanged <ref:2610.30348#pg2>.

Elias: It exploits the shared representation space of multimodal encoders to jointly align the perturbed image with the cross-modal adversarial target while repelling its representation from commonly filtered NSFW concepts <ref:2610.30348#pg3>.

Nadia: This semantic separation can transfer across representation spaces, enabling Mirage to evade heterogeneous safety checkers without accessing their internal details <ref:2610.30348#pg6>.

Paper summary: Nadia: So, we just covered the basics of what this paper claims about Mirage—that it bypasses post-generation safety checkers by using a targeted attack that aligns an image perturbation with a malicious target in the shared encoder space and pushes it away from unsafe concepts.

Elias: It’s important to remember that Mirage doesn't need access to the safety checker’s model details or decision threshold, which is a big deal because usually these checks are designed to be opaque.

Priya: But what about the actual mechanism of encoding that cross-modal target into that small image perturbation? The abstract hints at a challenge there.

Nadia: They address it by exploiting the shared embedding space of multimodal encoders, specifically optimizing a norm-bounded perturbation so that both textual and visual targets can be handled in the same optimization framework. This guides the diffusion model toward generating content aligned with whatever target you chose.

Elias: And then for evading the checker, they use a contrastive learning objective. They formulate it as L contras = - (y adv, y t) + lambda sum j=one (y adv, y j NSFW) which means they are minimizing the distance to the target while maximizing the distance from known NSFW concepts <ref:2610.30348#pg1>.

Priya: That contrastive loss setup is interesting because it’s explicitly trying to create that separation in the unified embedding space where features from different inputs are mapped close together.

Nadia: Right. And they claim this semantic separation can transfer across representation spaces, which is what lets Mirage evade heterogeneous safety checkers without knowing their internal details.

Elias: It means you don't need to know the specific rule that blocks something; you just need to find a representation gap that points somewhere else entirely.

Priya: What does this mean for the real-world data? If the results show strong performance, how confident are we in those claims when it comes to actual content generation?

Nadia: The experimental results show Mirage achieves high attack success rates and strong CLIP similarity between the generated content and the intended targets. Specifically, they found that successful attack images remain highly aligned with the adversarial targets, achieving CLIP similarity scores close to the None setting with only negligible degradation.

Elias: And they’re quite query-efficient too; in most cases, it requires only a few queries to bypass the safety checker. That’s a nice practical result for deployment because it keeps the cost low for running this type of attack.

Conclusion: Nadia: So, wrapping up on "Mirage: Targeted Jailbreaks Against Post-Generation Safety Checkers in Multimodal Diffusion Models," what we've seen is that these post-generation safety checkers aren't foolproof against targeted attacks.

Elias: It really shows that even when you have these checks running inside a diffusion pipeline, if the attack is clever enough, it can exploit the shared representation space to achieve its goal.

Priya: For someone focused on privacy and measurement, what does this vulnerability mean for how we trust AI-generated content in general?

Nadia: It highlights that relying solely on post-generation safety checks might not be enough against adaptive multimodal jailbreak attacks. The paper suggests we need more robust defense mechanisms, something called defense-in-depth, for these kinds of pipelines.

Elias: And the implication is that the way these models are structured—with those shared encoders and adapters—creates a pathway where semantic manipulation can bypass layered defenses without needing internal model access.

Priya: It sounds like we need to look at how we measure robustness in terms of representation space manipulation, not just final output filtering.

Nadia: Exactly. The work on Mirage demonstrates that post-generation safety checking alone isn't sufficient protection against these adaptive attacks, and it points toward the need for deeper layers of defense within multimodal diffusion pipelines.

More episodes

← Home