RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion".
Jane: Here's the gist of the paper. Large Vision-Language Models (VLMs) are increasingly deployed in open-ended environments, where ensuring reliable safety under multimodal inputs is critical <ref:2503.06223#pg2> This work studies whether VLMs can maintain safe behavior when partial toxic text is paired with visual context <ref:2503.06223#pg2> RedDiffuser (RedDiff) is a reinforcement-based framework that leverages diffusion models to generate semantically coherent visual inputs for black-box safety testing <ref:2503.06223#pg2> Pruning redundant reasoning tokens The paper addresses the limitation of existing instruction-centric evaluations by studying whether safety alignment remains robust under harmful contextual exposure <ref:2503.06223#pg2> This is important because visual inputs can substantially steer model behavior and render text-only auditing insufficient <ref:2503.06223#pg2> The research focuses on whether partial harmful text, when paired with visual context, leads VLMs to exhibit unsafe behavior <ref:2503.06223#pg2> This is treated as an audit of whether existing alignment mechanisms remain effective under realistic multimodal exposure <ref:2503.06223#pg2> How the training signal is built RedDiffuser operates in two stages to systematically audit this failure mode <ref:2503.06223#pg2> First, it uses an LLM to perform greedy search over candidate image prompts and identify those that most strongly expose unsafe model behavior <ref:2503.06223#pg2> Second, it reinforcement fine-tunes the diffusion model with a dual objective <ref:2503.06223#pg2> The two objectives are the Toxicity Reward and the Alignment Reward <ref:2503.06223#pg2> The Toxicity Reward captures the degree of unsafe behavior exposed under the audited context, while the Alignment Reward preserves semantic consistency between the generated image and its prompt <ref:2503.06223#pg2> The total reward is formulated as a weighted sum of the toxicity reward and the alignment reward, controlled by a hyperparameter lambda <ref:2503.06223#pg2> Where the gains come from The methodology involves several key components for systematic auditing <ref:2503.06223#pg2> 1.
Tom: Greedy Search for High-Risk Contexts: This procedure iteratively refines candidate prompts based on the toxicity scores of the target VLM’s responses <ref:2503.06223#pg2> It uses Gemini Pro with safety settings disabled as an auxiliary LLM for candidate prompt generation <ref:2503.06223#pg2> 2.
Jane: Reward Functions for Safety Auditing: The Toxicity Reward is calculated using Detoxify to evaluate six toxicity attributes, and the Alignment Reward uses BERTScore to measure semantic similarity between the image prompt and the VLM’s interpretation of the generated image <ref:2503.06223#pg2> 3.
Tom: External Guardrail Auditing: For practical deployments, a checker-aware variant is introduced that incorporates guardrail feedback during optimization <ref:2503.06223#pg2> This allows the framework to audit whether upstream safety filters are sufficient to prevent harmful multimodal context from propagating into unsafe behavior <ref:2503.06223#pg2> Pruning redundant reasoning tokens The experiments show that RedDiffuser reveals substantial safety vulnerabilities in current VLMs <ref:2503.06223#pg2> On LLaVA, it increases unsafe response rates by up to 10.69% on the original set and 8.91% on a hold-out set <ref:2503.06223#pg2> These vulnerabilities persist even under external safety guardrails, suggesting that current system-level safety mechanisms remain insufficient for realistic multimodal risks <ref:2503.06223#pg2> Where the gains come from The results demonstrate strong transferability to Gemini and LLaMA-Vision <ref:2503.06223#pg2> RedDiff w/ alignment (H) reaches the highest ‘Any‘ rate of 62.12% on the primary evaluation subset under the Perspective API <ref:2503.06223#pg2> Furthermore, checker-aware training enables RedDiffuser to identify inputs that remain admissible under external filtering while still revealing context-conditioned safety failures in the target VLM <ref:2503.06223#pg2> The checker-aware variant improves the guardrail pass rate by 40.4% over Stable Diffusion, indicating that existing safety filtering remains insufficient under multimodal contextual exposure <ref:2503.06223#pg2> Pruning redundant reasoning tokens The findings reveal a critical blind spot in existing safety evaluations and establish context-aware multimodal auditing as an essential paradigm for diagnosing hidden vulnerabilities in modern VLM systems <ref:2503.06223#pg2> The identified failure patterns also transfer to Gemini-1.5-flash and LLaMA-Vision-Instruct, indicating that they are not specific to a single target model <ref:2503.06223#pg2> Overall, our findings highlight an important blind spot in instruction-centric safety evaluation and demonstrate the value of context-aware auditing as a complementary perspective to conventional jailbreak evaluation <ref:2503.06223#pg2> The codes are available at https://github.com/roywang021/RedDiffuser <ref:2503.06223#pg2> Where the gains come from The results show that RedDiff w/o alignment achieves the highest ‘Any‘ rate of 79.45% under Detoxify, together with notable increases in attributes such as Obscene (67.44%) and Insult (42.02%) <ref:2503.06223#pg2> The results show that RedDiff w/ alignment (M) achieves the highest ‘Any‘ rate of 68.70% under Detoxify, showing that reinforcement-based optimization can more effectively expose context-conditioned safety failures <ref:2503.06223#pg2> This suggests that the context-conditioned safety failures identified by our framework are not limited to a narrow set of prompts, but reflect a more persistent weakness in current VLM safety alignment <ref:2503.06223#pg2> The results show that RedDiff w/ alignment (M) achieves 81.33% and 82.52% under Detoxify and Perspective API for Gemini-1.5-flash, showing that multimodal context can further aggravate unsafe behavior even in a commercial VLM <ref:2503.06223#pg2> The checker-aware variant achieves a 94.50% guardrail pass rate while continuing to expose unsafe model behavior <ref:2503.06223#pg2> Pruning redundant reasoning tokens The paper presents RedDiffuser, a reinforcement-based framework for auditing multimodal safety in Vision-Language Models (VLMs) <ref:2503.06223#pg2> Rather than evaluating whether models comply with explicit malicious instructions, RedDiffuser examines whether harmful textual context, when paired with visual inputs, leads to contextconditioned safety degradation <ref:2503.06223#pg2> The identified failure patterns also transfer to Gemini-1.5-flash and LLaMA-Vision-Instruct <ref:2503.06223#pg2> Under external NSFW guardrails, the checker-aware variant achieves a 94.50% guardrail pass rate while continuing to expose unsafe behavior <ref:2503.06223#pg2> Overall, our findings highlight an important blind spot in instruction-centric safety evaluation and demonstrate the value of context-aware auditing as a complementary perspective to conventional jailbreak evaluation <ref:2503.06223#pg2> Where the gains come from The results show that RedDiff w/o alignment achieves the highest ‘Any‘ rate of 79.45% under Detoxify, together with notable increases in attributes such as Obscene (67.44%) and Insult (42.02%) <ref:2503.06223#pg2> The results show that RedDiff w/o alignment reaches the highest ‘Any‘ rate of 79.36% under Detoxify, while also attaining leading scores on Profanity (68.13%) and Severe Toxicity (7.12%) for Gemini-1.5-flash <ref:2503.06223#pg2> These results indicate that RedDiffuser generalizes effectively to unseen prompts and remains consistent across toxicity classifiers <ref:2503.06223#pg2> The results show that RedDiff w/o alignment reaches the highest ‘Any‘ rate of 79.45% under Detoxify, together with notable increases in attributes such as Obscene (67.44%) and Insult (42.02%) <ref:2503.06223#pg2> The results show that RedDiff w/o alignment reaches the highest ‘Any‘ rate of 79.36% under Detoxify, while also attaining leading scores on Profanity (68.13%) and Severe Toxicity (7.12%) <ref:2503.06223#pg2> The results show that RedDiff w/o alignment reaches the highest ‘Any‘ rate of 79.45% under Detoxify, together with notable increases in attributes such as Obscene (67.44%) and Insult (42.02%) <ref:2503.
Paper summary: Tom: To recap, the RedDiffuser paper is essentially about moving beyond just checking if a model follows explicit malicious instructions. They are looking at context-conditioned safety degradation >
Jane: They introduce this framework to see if VLMs can stay safe when they get partially toxic text alongside visual information >
Lu: The methodology involves using a diffusion model to generate images that cause the VLM to produce unsafe outputs, and then using reward functions based on toxicity and semantic alignment to teach the diffusion model how to generate those risky images >
Meng: So, if I'm looking at this from a deployment standpoint, what does this actually achieve? Is it just a clever way to test prompts?
Tom: It’s more about creating inputs that are hard for existing safety filters to catch because they look like normal, coherent visual context >
Jane: The paper shows that their diffusion model generates images that elicit harmful responses from the VLM, whereas images made by a standard diffusion model don't >
Conclusion: Tom: So, to wrap up the RedDiffuser paper, it’s about showing that context matters a lot when we test safety in these multimodal models >
Jane: They found that this approach reveals substantial safety vulnerabilities in current VLMs on things like LLaVA and Gemini >
Lu: The authors highlight a critical blind spot in existing instruction-centric evaluations and suggest that context-aware auditing is necessary for finding hidden problems >
Meng: What does this mean for us when we're actually shipping models? Are we supposed to be generating these kinds of inputs constantly?
Tom: No, it means that current system-level safety mechanisms might not be sufficient when dealing with realistic multimodal risks from context >
Jane: It suggests that testing needs to evolve to include how visual context interacts with harmful text, not just the text itself >
Fudan University · Shanghai Innovation Institute
cs.CV
Submitted: 2025-03-08
Updated: 2026-11-10
Code: https://github.com/roywang021/RedDiffuser
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 89/100
The gist: Large Vision-Language Models (VLMs) are increasingly deployed in open-ended environments, where ensuring reliable safety under multimodal inputs is critical <ref:2503.06223#pg2> This work studies…
Key concepts
- RedDiffuser
- A reinforcement-based framework that employs diffusion models to create semantically coherent visual prompts for testing VLMs. It uses a dual objective reward system—Toxicity Reward and Alignment Reward—to guide the generation process, allowing it to probe safety failures systematically.
- Toxicity Reward
- A metric used during training that quantifies the degree of unsafe behavior exposed by a given context. This reward is calculated using Detoxify to evaluate six specific toxicity attributes, helping the framework measure how harmful the generated input is.
- Alignment Reward
- A metric designed to preserve semantic consistency between the generated image and its corresponding text prompt. It uses BERTScore to measure this similarity, ensuring that while toxicity is being tested, the visual context remains semantically relevant to the prompt.
Terminology
Summary
Large Vision-Language Models (VLMs) are increasingly deployed in open-ended environments, where ensuring reliable safety under multimodal inputs is critical <ref:2503. p. 006223#pg2> This work studies whether VLMs can maintain safe behavior when partial toxic text is paired with visual context <ref:2503. p. 006223#pg2> RedDiffuser (RedDiff) is a reinforcement-based framework that leverages diffusion models to generate semantically coherent visual inputs for black-box safety testing <ref:2503. p. 006223#pg2>
Pruning redundant reasoning tokens
The paper addresses the limitation of existing instruction-centric evaluations by studying whether safety alignment remains robust under harmful contextual exposure <ref:2503.06223#pg2> This is important because visual inputs can substantially steer model behavior and render text-only auditing insufficient <ref:2503. p. 006223#pg2> The research focuses on whether partial harmful text, when paired with visual context, leads VLMs to exhibit unsafe behavior <ref:2503. p. 006223#pg2> This is treated as an audit of whether existing alignment mechanisms remain effective under realistic multimodal exposure <ref:2503. p. 006223#pg2>
How the training signal is built
RedDiffuser operates in two stages to systematically audit this failure mode <ref:2503.06223#pg2> First, it uses an LLM to perform greedy search over candidate image prompts and identify those that most strongly expose unsafe model behavior <ref:2503. p. 006223#pg2> Second, it reinforcement fine-tunes the diffusion model with a dual objective <ref:2503. p. 006223#pg2> The two objectives are the Toxicity Reward and the Alignment Reward <ref:2503. p. 106223#pg2>
The Toxicity Reward captures the degree of unsafe behavior exposed under the audited context, while the Alignment Reward preserves semantic consistency between the generated image and its prompt <ref:2503.06223#pg2> The total reward is formulated as a weighted sum of the toxicity reward and the alignment reward, controlled by a hyperparameter lambda <ref:2503. p. 306223#pg2>
Where the gains come from
The methodology involves several key components for systematic auditing <ref:2503.06223#pg2>
-
Greedy Search for High-Risk Contexts: This procedure iteratively refines candidate prompts based on the toxicity scores of the target VLM’s responses <ref:2503. p. 206223#pg2> It uses Gemini Pro with safety settings disabled as an auxiliary LLM for candidate prompt generation <ref:2503. p. 206223#pg2>
-
Reward Functions for Safety Auditing: The Toxicity Reward is calculated using Detoxify to evaluate six toxicity attributes, and the Alignment Reward uses BERTScore to measure semantic similarity between the image prompt and the VLM’s interpretation of the generated image <ref:2503. p. 306223#pg2>
-
External Guardrail Auditing: For practical deployments, a checker-aware variant is introduced that incorporates guardrail feedback during optimization <ref:2503. p. 106223#pg2> This allows the framework to audit whether upstream safety filters are sufficient to prevent harmful multimodal context from propagating into unsafe behavior <ref:2503. p. 106223#pg2>
The experiments show that RedDiffuser reveals substantial safety vulnerabilities in current VLMs <ref:2503.06223#pg2> On LLaVA, it increases unsafe response rates by up to 10. p. 069% on the original set and 8. p. 091% on a hold-out set <ref:2503. p. 006223#pg2> These vulnerabilities persist even under external safety guardrails, suggesting that current system-level safety mechanisms remain insufficient for realistic multimodal risks <ref:2503. p. 006223#pg2>
The results demonstrate strong transferability to Gemini and LLaMA-Vision <ref:2503.06223#pg2> RedDiff w/ alignment (H) reaches the highest ‘Any‘ rate of 62. p. 412% on the primary evaluation subset under the Perspective API <ref:2503. p. 406223#pg2> Furthermore, checker-aware training enables RedDiffuser to identify inputs that remain admissible under external filtering while still revealing context-conditioned safety failures in the target VLM <ref:2503. p. 706223#pg2> The checker-aware variant improves the guardrail pass rate by 40. p. 14% over Stable Diffusion, indicating that existing safety filtering remains insufficient under multimodal contextual exposure <ref:2503. p. 106223#pg2>
The findings reveal a critical blind spot in existing safety evaluations and establish context-aware multimodal auditing as an essential paradigm for diagnosing hidden vulnerabilities in modern VLM systems <ref:2503.06223#pg2> The identified failure patterns also transfer to Gemini-1. p. 6 p. 65-flash and LLaMA-Vision-Instruct, indicating that they are not specific to a single target model <ref:2503. p. 706223#pg2> Overall, our findings highlight an important blind spot in instruction-centric safety evaluation and demonstrate the value of context-aware auditing as a complementary perspective to conventional jailbreak evaluation <ref:2503. p. 7 p. 706223#pg2> The codes are available at https://github. p. 7com/roywang021/RedDiffuser <ref:2503. p. 706223#pg2>
The results show that RedDiff w/o alignment achieves the highest ‘Any‘ rate of 79.45% under Detoxify, together with notable increases in attributes such as Obscene (67. p. 6 p. 6 p. 6 p. 644%) and Insult (42. p. 2 p. 2 p. 2 p. 202%) <ref:2503. p. 0 p. 0 p. 0 p. 006223#pg2> The results show that RedDiff w/ alignment (M) achieves the highest ‘Any‘ rate of 68. p. 670% under Detoxify, showing that reinforcement-based optimization can more effectively expose context-conditioned safety failures <ref:2503. p. 406223#pg2> This suggests that the context-conditioned safety failures identified by our framework are not limited to a narrow set of prompts, but reflect a more persistent weakness in current VLM safety alignment <ref:2503. p. 606223#pg2> The results show that RedDiff w/ alignment (M) achieves 81. p. 633% and 82.52% under Detoxify and Perspective API for Gemini-1. p. 65-flash, showing that multimodal context can further aggravate unsafe behavior even in a commercial VLM <ref:2503. p. 606223#pg2> The checker-aware variant achieves a 94. p. 750% guardrail pass rate while continuing to expose unsafe model behavior <ref:2503. p. 706223#pg2>
The paper presents RedDiffuser, a reinforcement-based framework for auditing multimodal safety in Vision-Language Models (VLMs) <ref:2503.06223#pg2> Rather than evaluating whether models comply with explicit malicious instructions, RedDiffuser examines whether harmful textual context, when paired with visual inputs, leads to contextconditioned safety degradation <ref:2503. p. 706223#pg2> The identified failure patterns also transfer to Gemini-1.5-flash and LLaMA-Vision-Instruct <ref:2503. p. 706223#pg2> Under external NSFW guardrails, the checker-aware variant achieves a 94. p. 750% guardrail pass rate while continuing to expose unsafe behavior <ref:2503. p. 706223#pg2> Overall, our findings highlight an important blind spot in instruction-centric safety evaluation and demonstrate the value of context-aware auditing as a complementary perspective to conventional jailbreak evaluation <ref:2503.06223#pg2>
The results show that RedDiff w/o alignment achieves the highest ‘Any‘ rate of 79.45% under Detoxify, together with notable increases in attributes such as Obscene (67.44%) and Insult (42.02%) <ref:2503.06223#pg2> The results show that RedDiff w/o alignment reaches the highest ‘Any‘ rate of 79. p. 6 p. 6 p. 6 p. 636% under Detoxify, while also attaining leading scores on Profanity (68. p. 6 p. 613%) and Severe Toxicity (7. p. 5 p. 512%) for Gemini-1. p. 05-flash <ref:2503. p. 006223#pg2> These results indicate that RedDiffuser generalizes effectively to unseen prompts and remains consistent across toxicity classifiers <ref:2503. p. 606223#pg2> The results show that RedDiff w/o alignment reaches the highest ‘Any‘ rate of 79.45% under Detoxify, together with notable increases in attributes such as Obscene (67.44%) and Insult (42.02%) <ref:2503.06223#pg2> The results show that RedDiff w/o alignment reaches the highest ‘Any‘ rate of 79.36% under Detoxify, while also attaining leading scores on Profanity (68.13%) and Severe Toxicity (7.12%) <ref:2503. p. 006223#pg2> The results show that RedDiff w/o alignment reaches the highest ‘Any‘ rate of 79.45% under Detoxify, together with notable increases in attributes such as Obscene (67.44%) and Insult (42.02%) <ref:2503.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Reinforce safety alignment using RedDiffuser to uncover context-conditioned safety failures by leveraging a reinforcement-based framework that leverages diffusion models to generate semantically coherent visual inputs for black-box safety testing. This allows the system to audit whether
safety alignment remains robust under harmful contextual exposure.
-
Implement a two-stage pipeline combining greedy prompt search with reinforcement fine-tuning, using toxicity and alignment objectives to uncover high-risk multimodal contexts while preserving natural image quality. This process enables the system to surface
high-risk multimodal inputs that expose latent safety failures.
-
Develop a checker aware variant of RedDiffuser that incorporates guardrail feedback during optimization, enabling the system to audit not only the target VLM itself but also
whether upstream safety filters are sufficient to prevent harmful multimodal context from propagating into unsafe behavior.
-
Enhance prompt generation by iteratively refining candidate prompts based on toxicity scores using an auxiliary LLM like Gemini Pro, which allows the system to refine image prompts using
the target VLM’s responses and their toxicity scores from a classifier (e.g., Detoxify [13]).
-
Optimize the diffusion model using a dual reward structure consisting of the Toxicity Reward and Alignment Reward, balancing
the exposure of unsafe model behavior with semantic consistency.
This ensures that discovered failures are not only harmful but also visually plausible, as RedDiffusersurface high-risk multimodal inputs while preserving visual plausibility.
-
Adapt auditing to realistic deployment conditions by introducing checker aware reward shaping that masks the toxicity reward when a generated image is blocked by external safety checkers, allowing RedDiffuser to reveal
high-risk inputs that remain admissible under external filtering while still revealing context-conditioned safety failures in the target VLM.
Sources
- GPT-4 Technical Report
- Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs
- Image Hijacks: Adversarial Images can Control Generative Models at Runtime
- Training Diffusion Models with Reinforcement Learning
- Language Models are Few-Shot Learners
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
- FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts
- CLIPScore: A Reference-free Evaluation Metric for Image Captioning
- Privacy in Large Language Models: Attacks, Defenses and Future Directions
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends
- MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models
- Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Character
- Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety
- Robust Conversational Agents against Imperceptible Toxicity Triggers
- Jailbreaking Attack against Multimodal Large Language Model
- Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks
- Gemini: A Family of Highly Capable Multimodal Models
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs