RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion
summary
The gist
Large Vision-Language Models (VLMs) are increasingly deployed in open-ended environments, where ensuring reliable safety under multimodal inputs is critical <ref:2503.06223#pg2> This work studies…
In short
RedDiffuser is a reinforcement-based framework that uses diffusion models to generate visual inputs for black-box safety testing of Vision-Language Models (VLMs). It systematically audits whether VLMs maintain safe behavior when exposed to partial toxic text paired with visual context, revealing significant safety vulnerabilities in current VLM alignment mechanisms.
Key concepts
- RedDiffuser
- A reinforcement-based framework that employs diffusion models to create semantically coherent visual prompts for testing VLMs. It uses a dual objective reward system—Toxicity Reward and Alignment Reward—to guide the generation process, allowing it to probe safety failures systematically.
- Toxicity Reward
- A metric used during training that quantifies the degree of unsafe behavior exposed by a given context. This reward is calculated using Detoxify to evaluate six specific toxicity attributes, helping the framework measure how harmful the generated input is.
- Alignment Reward
- A metric designed to preserve semantic consistency between the generated image and its corresponding text prompt. It uses BERTScore to measure this similarity, ensuring that while toxicity is being tested, the visual context remains semantically relevant to the prompt.
Terminology used across episodes
This episode discusses
- RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion · Paper Radio
- GPT-4 Technical Report
- Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs
- Image Hijacks: Adversarial Images can Control Generative Models at Runtime
- Training Diffusion Models with Reinforcement Learning
- Language Models are Few-Shot Learners
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
- FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts
- CLIPScore: A Reference-free Evaluation Metric for Image Captioning
- Privacy in Large Language Models: Attacks, Defenses and Future Directions
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends
- MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models
- Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Character
- Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety
- Robust Conversational Agents against Imperceptible Toxicity Triggers
- Jailbreaking Attack against Multimodal Large Language Model
- Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks
- Gemini: A Family of Highly Capable Multimodal Models
The paper
RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion · Read on arXiv
Fudan University · Shanghai Innovation Institute
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion".
Jane: Here's the gist of the paper. Large Vision-Language Models (VLMs) are increasingly deployed in open-ended environments, where ensuring reliable safety under multimodal inputs is critical <ref:2503.06223#pg2> This work studies whether VLMs can maintain safe behavior when partial toxic text is paired with visual context <ref:2503.06223#pg2> RedDiffuser (RedDiff) is a reinforcement-based framework that leverages diffusion models to generate semantically coherent visual inputs for black-box safety testing <ref:2503.06223#pg2> Pruning redundant reasoning tokens The paper addresses the limitation of existing instruction-centric evaluations by studying whether safety alignment remains robust under harmful contextual exposure <ref:2503.06223#pg2> This is important because visual inputs can substantially steer model behavior and render text-only auditing insufficient <ref:2503.06223#pg2> The research focuses on whether partial harmful text, when paired with visual context, leads VLMs to exhibit unsafe behavior <ref:2503.06223#pg2> This is treated as an audit of whether existing alignment mechanisms remain effective under realistic multimodal exposure <ref:2503.06223#pg2> How the training signal is built RedDiffuser operates in two stages to systematically audit this failure mode <ref:2503.06223#pg2> First, it uses an LLM to perform greedy search over candidate image prompts and identify those that most strongly expose unsafe model behavior <ref:2503.06223#pg2> Second, it reinforcement fine-tunes the diffusion model with a dual objective <ref:2503.06223#pg2> The two objectives are the Toxicity Reward and the Alignment Reward <ref:2503.06223#pg2> The Toxicity Reward captures the degree of unsafe behavior exposed under the audited context, while the Alignment Reward preserves semantic consistency between the generated image and its prompt <ref:2503.06223#pg2> The total reward is formulated as a weighted sum of the toxicity reward and the alignment reward, controlled by a hyperparameter lambda <ref:2503.06223#pg2> Where the gains come from The methodology involves several key components for systematic auditing <ref:2503.06223#pg2> 1.
Tom: Greedy Search for High-Risk Contexts: This procedure iteratively refines candidate prompts based on the toxicity scores of the target VLM’s responses <ref:2503.06223#pg2> It uses Gemini Pro with safety settings disabled as an auxiliary LLM for candidate prompt generation <ref:2503.06223#pg2> 2.
Jane: Reward Functions for Safety Auditing: The Toxicity Reward is calculated using Detoxify to evaluate six toxicity attributes, and the Alignment Reward uses BERTScore to measure semantic similarity between the image prompt and the VLM’s interpretation of the generated image <ref:2503.06223#pg2> 3.
Tom: External Guardrail Auditing: For practical deployments, a checker-aware variant is introduced that incorporates guardrail feedback during optimization <ref:2503.06223#pg2> This allows the framework to audit whether upstream safety filters are sufficient to prevent harmful multimodal context from propagating into unsafe behavior <ref:2503.06223#pg2> Pruning redundant reasoning tokens The experiments show that RedDiffuser reveals substantial safety vulnerabilities in current VLMs <ref:2503.06223#pg2> On LLaVA, it increases unsafe response rates by up to 10.69% on the original set and 8.91% on a hold-out set <ref:2503.06223#pg2> These vulnerabilities persist even under external safety guardrails, suggesting that current system-level safety mechanisms remain insufficient for realistic multimodal risks <ref:2503.06223#pg2> Where the gains come from The results demonstrate strong transferability to Gemini and LLaMA-Vision <ref:2503.06223#pg2> RedDiff w/ alignment (H) reaches the highest ‘Any‘ rate of 62.12% on the primary evaluation subset under the Perspective API <ref:2503.06223#pg2> Furthermore, checker-aware training enables RedDiffuser to identify inputs that remain admissible under external filtering while still revealing context-conditioned safety failures in the target VLM <ref:2503.06223#pg2> The checker-aware variant improves the guardrail pass rate by 40.4% over Stable Diffusion, indicating that existing safety filtering remains insufficient under multimodal contextual exposure <ref:2503.06223#pg2> Pruning redundant reasoning tokens The findings reveal a critical blind spot in existing safety evaluations and establish context-aware multimodal auditing as an essential paradigm for diagnosing hidden vulnerabilities in modern VLM systems <ref:2503.06223#pg2> The identified failure patterns also transfer to Gemini-1.5-flash and LLaMA-Vision-Instruct, indicating that they are not specific to a single target model <ref:2503.06223#pg2> Overall, our findings highlight an important blind spot in instruction-centric safety evaluation and demonstrate the value of context-aware auditing as a complementary perspective to conventional jailbreak evaluation <ref:2503.06223#pg2> The codes are available at https://github.com/roywang021/RedDiffuser <ref:2503.06223#pg2> Where the gains come from The results show that RedDiff w/o alignment achieves the highest ‘Any‘ rate of 79.45% under Detoxify, together with notable increases in attributes such as Obscene (67.44%) and Insult (42.02%) <ref:2503.06223#pg2> The results show that RedDiff w/ alignment (M) achieves the highest ‘Any‘ rate of 68.70% under Detoxify, showing that reinforcement-based optimization can more effectively expose context-conditioned safety failures <ref:2503.06223#pg2> This suggests that the context-conditioned safety failures identified by our framework are not limited to a narrow set of prompts, but reflect a more persistent weakness in current VLM safety alignment <ref:2503.06223#pg2> The results show that RedDiff w/ alignment (M) achieves 81.33% and 82.52% under Detoxify and Perspective API for Gemini-1.5-flash, showing that multimodal context can further aggravate unsafe behavior even in a commercial VLM <ref:2503.06223#pg2> The checker-aware variant achieves a 94.50% guardrail pass rate while continuing to expose unsafe model behavior <ref:2503.06223#pg2> Pruning redundant reasoning tokens The paper presents RedDiffuser, a reinforcement-based framework for auditing multimodal safety in Vision-Language Models (VLMs) <ref:2503.06223#pg2> Rather than evaluating whether models comply with explicit malicious instructions, RedDiffuser examines whether harmful textual context, when paired with visual inputs, leads to contextconditioned safety degradation <ref:2503.06223#pg2> The identified failure patterns also transfer to Gemini-1.5-flash and LLaMA-Vision-Instruct <ref:2503.06223#pg2> Under external NSFW guardrails, the checker-aware variant achieves a 94.50% guardrail pass rate while continuing to expose unsafe behavior <ref:2503.06223#pg2> Overall, our findings highlight an important blind spot in instruction-centric safety evaluation and demonstrate the value of context-aware auditing as a complementary perspective to conventional jailbreak evaluation <ref:2503.06223#pg2> Where the gains come from The results show that RedDiff w/o alignment achieves the highest ‘Any‘ rate of 79.45% under Detoxify, together with notable increases in attributes such as Obscene (67.44%) and Insult (42.02%) <ref:2503.06223#pg2> The results show that RedDiff w/o alignment reaches the highest ‘Any‘ rate of 79.36% under Detoxify, while also attaining leading scores on Profanity (68.13%) and Severe Toxicity (7.12%) for Gemini-1.5-flash <ref:2503.06223#pg2> These results indicate that RedDiffuser generalizes effectively to unseen prompts and remains consistent across toxicity classifiers <ref:2503.06223#pg2> The results show that RedDiff w/o alignment reaches the highest ‘Any‘ rate of 79.45% under Detoxify, together with notable increases in attributes such as Obscene (67.44%) and Insult (42.02%) <ref:2503.06223#pg2> The results show that RedDiff w/o alignment reaches the highest ‘Any‘ rate of 79.36% under Detoxify, while also attaining leading scores on Profanity (68.13%) and Severe Toxicity (7.12%) <ref:2503.06223#pg2> The results show that RedDiff w/o alignment reaches the highest ‘Any‘ rate of 79.45% under Detoxify, together with notable increases in attributes such as Obscene (67.44%) and Insult (42.02%) <ref:2503.
Paper summary: Tom: To recap, the RedDiffuser paper is essentially about moving beyond just checking if a model follows explicit malicious instructions. They are looking at context-conditioned safety degradation >
Jane: They introduce this framework to see if VLMs can stay safe when they get partially toxic text alongside visual information >
Lu: The methodology involves using a diffusion model to generate images that cause the VLM to produce unsafe outputs, and then using reward functions based on toxicity and semantic alignment to teach the diffusion model how to generate those risky images >
Meng: So, if I'm looking at this from a deployment standpoint, what does this actually achieve? Is it just a clever way to test prompts?
Tom: It’s more about creating inputs that are hard for existing safety filters to catch because they look like normal, coherent visual context >
Jane: The paper shows that their diffusion model generates images that elicit harmful responses from the VLM, whereas images made by a standard diffusion model don't >
Conclusion: Tom: So, to wrap up the RedDiffuser paper, it’s about showing that context matters a lot when we test safety in these multimodal models >
Jane: They found that this approach reveals substantial safety vulnerabilities in current VLMs on things like LLaVA and Gemini >
Lu: The authors highlight a critical blind spot in existing instruction-centric evaluations and suggest that context-aware auditing is necessary for finding hidden problems >
Meng: What does this mean for us when we're actually shipping models? Are we supposed to be generating these kinds of inputs constantly?
Tom: No, it means that current system-level safety mechanisms might not be sufficient when dealing with realistic multimodal risks from context >
Jane: It suggests that testing needs to evolve to include how visual context interacts with harmful text, not just the text itself >
More episodes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound
- 2605.15085-From Data to Action: Accelerating Refinery Optimization with AI
- 2605.23500-GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation