GARD: Gradient-Guided Adversarial Representation Defense against Backdoors in Multimodal Contrastive Learning Models

arXiv:2611.14143 · cs.CR, cs.CV · Submitted 2026-11-10 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "GARD: Gradient-Guided Adversarial Representation Defense against Backdoors in Multimodal Contrastive Learning Models".

Elias: Here's the gist of the paper. Multimodal contrastive learning models such as CLIP are vulnerable to backdoor attacks with diverse trigger structures, ranging from localized patches to globally distributed perturbations, with recent attacks increasingly optimizing these triggers in the representation space<ref:2611.14143#pg3>.

Nadia: GARD optimizes complementary global and local adversarial probes to maximally displace cleanimage representations, and then realigns the perturbed representations with detached clean anchors while preserving image–text alignment<ref:2611.14143#pg6>.

Elias: Pruning redundant reasoning tokens Existing post-hoc defenses based on clean fine-tuning may leave backdoors active, while trigger-inversion methods depend on accurately estimating the attack target and trigger<ref:2611.14143#pg3>.

Nadia: The paper observes that backdoored encoders exhibit structure-dependent representation sensitivity: global perturbations preferentially expose distributed backdoors, whereas local patch perturbations are more effective for localized backdoors<ref:2611.14143#pg4>.

Title and authors: Nadia: So we're looking at the paper "GARD: Gradient-Guided Adversarial Representation Defense against Backdoors in Multimodal Contrastive Learning Models" again. We just touched on the idea of it being trigger and target agnostic, which is a big deal for defense.

Elias: The paper shows that existing methods are limited because they either rely on trigger inversion or trying to guess the target class, which we know isn't reliable >

Priya: And what I noticed in the summary is that GARD alternates between these two things—probing and realignment—which suggests it’s trying to cover different kinds of input variations simultaneously.

Nadia: Right. They are using a global probe for distributed triggers and a local probe for spatially concentrated ones, which is smart because it addresses both types of backdoors >

Elias: The authors observe that backdoored encoders show different sensitivities depending on the trigger structure; global perturbations expose distributed backdoors while local patches work better for localized ones >

Priya: So, by using both global and local probes under one training setup, they are tackling the full spectrum of possible triggers in a unified way >

Nadia: That’s the point. It moves away from needing an exact trigger estimation to just suppressing the representation vulnerabilities that trigger-like inputs expose >

The paper's summary: Elias: Looking at what they summarize, GARD alternates between adversarial probing and representation realignment to construct these complementary global and local adversarial views >

Nadia: They define this whole process using a training objective called LGARD which balances three things: the clean contrastive loss, the global probe loss, and the local probe loss >

Priya: And specifically for that alignment part, they use a representation alignment loss where they try to match perturbed representations to clean counterparts while making sure the image and text still align properly >

Elias: The results show that when you use this method, triggered images don't have a target estimation or trigger inversion happening, but they get strong backdoor suppression while keeping competitive clean performance >

Nadia: It seems the main contribution is that they are suppressing these vulnerabilities by focusing on the representation itself rather than trying to recover the specific trigger input >

The paper's improvements: Priya: I was looking at their improvements, and it really hammers home that this method isn't just one trick; it uses both global and local probes to cover distributed and spatially concentrated vulnerabilities simultaneously >

Elias: And they emphasize that the representation realignment step is what brings everything together by aligning the perturbed representations with a detached clean-image anchor while preserving image text correspondence >

Nadia: It seems like the main improvement is moving away from explicit trigger recovery or target-class estimation, which makes it much more robust against attacks where those things are unknown >

Elias: They also mention that this alternating procedure doesn't introduce any extra computation when you actually use the model for inference, which is a huge plus for practical deployment >

Conclusion: Nadia: So to wrap up the discussion on "GARD: Gradient-Guided Adversarial Representation Defense against Backdoors in Multimodal Contrastive Learning Models", it shows that identifying and suppressing representation vulnerabilities offers a practical, target-free way to defend these contrastive models >

Elias: It successfully mitigates backdoor attacks in the image text retrieval task while maintaining competitive clean retrieval performance, even when you only have a small amount of clean data available >

Priya: And I think the fact that it works across diverse trigger forms and doesn't need clean examples of the attack target class is really important because it makes defense much more realistic for us >

Nadia: Right. The method proves that focusing on cleaning up the representation vulnerabilities themselves is a viable path for defending these models >

Elias: It’s a solid result, showing how complementary global and local views can tackle both distributed and localized triggers effectively >

Yuhong Wang, Yujun Cai, Lei Feng, Songze Li

Southeast University

cs.CR, cs.CV

Submitted: 2026-11-10

Updated: 2026-11-10

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 86/100

The gist: Multimodal contrastive learning models such as CLIP are vulnerable to backdoor attacks with diverse trigger structures, ranging from localized patches to globally distributed perturbations…

Key concepts

Multimodal Contrastive Learning Models
These are AI models that learn to match images and text together by comparing their features in a shared space. Backdoor attacks exploit these models by adding hidden triggers that cause the model to misclassify inputs based on those triggers, even when the input is clean.
Adversarial Probing
This involves using adversarial perturbations—small, carefully crafted changes to an image—to test how sensitive the model's internal representations are. GARD uses both global and local probes to find where these vulnerabilities exist, distinguishing between triggers spread across the whole image versus those concentrated in small areas.
Representation Realignment
This is the core defense step where the system takes a corrupted (perturbed) representation and adjusts it back towards a clean version. It does this by aligning it with a 'detached clean-image anchor' while strictly maintaining the original relationship between the image and its corresponding text description.

Terminology

Summary

Multimodal contrastive learning models such as CLIP are vulnerable to backdoor attacks with diverse trigger structures, ranging from localized patches to globally distributed perturbations, with recent attacks increasingly optimizing these triggers in the representation space<ref:2611. p. 014143#pg3>. GARD optimizes complementary global and local adversarial probes to maximally displace cleanimage representations, and then realigns the perturbed representations with detached clean anchors while preserving image–text alignment<ref:2611. p. 014143#pg6>.

Pruning redundant reasoning tokens

Existing post-hoc defenses based on clean fine-tuning may leave backdoors active, while trigger-inversion methods depend on accurately estimating the attack target and trigger<ref:2611.14143#pg3>. The paper observes that backdoored encoders exhibit structure-dependent representation sensitivity: global perturbations preferentially expose distributed backdoors, whereas local patch perturbations are more effective for localized backdoors<ref:2611. p. 014143#pg4>. This suggests that different trigger structures produce distinct adversarial sensitivity patterns<ref:2611. p. 014143#pg6>.

How the training signal is built

GARD alternates between adversarial probing and representation realignment, constructing complementary global and local adversarial views to cover distributed and spatially concentrated input variations, respectively<ref:2611.14143#pg3>. The overall training objective is defined as LGARD = λcmLcm + λgLg + λpLp, which balances the three objectives<ref:2611. p. 414143#pg10>. Specifically, the representation alignment loss Lq = 1/B Σ (1 − sim¯fθ(x q i),sg¯fθ(xi)) is used to align perturbed representations with clean counterparts while preserving clean image–text alignment<ref:2611. p. 414143#pg6>.

Where the gains come from

The effectiveness of GARD stems from its ability to identify and suppress backdoor-induced representation vulnerabilities rather than relying on explicit trigger recovery or target-class estimation<ref:2611.14143#pg7>. The complementary global and local adversarial probing modules are designed to probe distributed and spatially concentrated representation vulnerabilities under a unified training framework<ref:2611. p. 114143#pg6>. Ablation studies confirm that the two representation alignment objectives address complementary vulnerabilities induced by different trigger patterns<ref:2611. p. 514143#pg7>.

Evaluation and Robustness

Across six visual backdoor attacks and two CLIP visual backbones, GARD reduces the attack success rate below 1% in all 12 evaluated attack–backbone combinations while maintaining competitive clean accuracy<ref:2611.14143#pg7>. The results demonstrate effective backdoor removal across diverse trigger forms, and the ablations verify the complementary roles of two adversarial views<ref:2611. p. 114143#pg6>. Furthermore, GARD is robust to adaptive attacks by continually probing and suppressing backdoor behavior during purification rather than relying on a fixed trigger pattern<ref:2611. p. 514143#pg6>. The defense runtime for GARD is 42. p. 050 minutes, which corresponds to speedups of 5.07× compared with CleanCLIP<ref:2611. p. 514143#pg9>.

Conclusion

GARD demonstrates that identifying and suppressing backdoor-induced representation vulnerabilities offers a practical target-free direction for defending MCL models<ref:2611.14143#pg8>. The method successfully mitigates backdoor attacks in the image–text retrieval task, maintaining competitive clean retrieval performance even with a limited clean-data budget<ref:2611. p. 014143#pg8>. GARD remains effective without clean images from the attacker’s target class, consistent with preliminary studies<ref:2611. p. 214143#pg9>. The paper concludes that identifying and suppressing backdoor-induced representation vulnerabilities offers a practical target-free direction for defending MCL models<ref:2611. p. 714143#pg8>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Backdoor defense formulation is changed to trigger-agnostic adversarial probing followed by representation realignment, avoiding both target-class estimation and explicit trigger recovery. This allows for a defense that does not rely on accurate target estimation or assumptions about the trigger structure.

  2. Complementary global and local perturbation modules are developed to probe distributed and spatially concentrated representation vulnerabilities under a unified training framework. This enables the system to effectively handle both localized and distributed triggers simultaneously.

  3. Representation realignment is achieved by aligning perturbed representations with a detached clean-image anchor while preserving image–text alignment, which ensures that the defense suppresses abnormal representation sensitivity exposed during perturbation generation without requiring trigger recovery or target-class estimation.

  4. The system can reduce the attack success rate below 1% across six visual backdoor attacks and two CLIP visual backbones while maintaining competitive clean accuracy, demonstrating effective backdoor removal across diverse trigger forms.

  5. The defense is robust against adaptive attacks, such as Adap-1 (simulated purification) and Adap-2 (defense-signal evasion), which aim to preserve backdoor effectiveness after purification or weaken the signals used by the defense.

  6. GARD can be applied effectively with a limited clean data budget, as it remains effective across different trigger structures and benefits from keeping the defense set clean, requiring only a small collection of clean images rather than the large dataset used by the fine-tuning baselines.

  7. The system can operate successfully without using clean images from the attacker’s target class, as global and local probes can expose backdoor-sensitive directions using images from other classes, without requiring clean examples of the attack target.

Sources

Related papers