PhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems
summary
The gist
Multimodal Large Language Models (MLLMs) are becoming integral to autonomous driving (AD) systems due to their strong vision-language reasoning capabilities However…
In short
PhysPatch creates a physically realizable and transferable adversarial patch attack for autonomous driving systems using Multimodal Large Language Models (MLLMs). It jointly optimizes patch location, shape, and content by using semantic initialization and SVD-based alignment to ensure the patches are placed realistically in the scene. This method significantly improves attack effectiveness and real-world applicability compared to existing techniques.
Key concepts
- Physically Realizable Adversarial Patch Framework
- This framework optimizes three aspects of an adversarial patch: its location, shape, and content. It uses a semantic-based mask initialization and a potential field update to ensure the patches are placed in physically feasible regions of driving scenes, making the attack practical for real-world deployment.
- SVD-based Local Alignment Loss
- This loss function uses truncated Singular Value Decomposition (SVD) to achieve optimal semantic compression of local visual features. It helps make the generated adversarial patches robust against variations in different vision encoders used by various MLLMs, enhancing the attack's transferability.
- Patch-Guided Crop-Resize Strategy
- This strategy ensures that the chosen patch center is included in every sampled crop during optimization. This prevents gradient vanishing issues and guarantees that the adversarial patch remains visible and relevant across different image crops, improving overall attack reliability.
Terminology used across episodes
This episode discusses
- PhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems · Paper Radio
- Qwen2.5-VL Technical Report
- Adversarial Patch
- PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximization
- advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
- How Robust is Google's Bard to Adversarial Image Attacks?
- A Survey on LLM-as-a-Judge
- SURDS: Benchmarking Spatial Understanding and Reasoning in Driving Scenarios with Vision Language Models
- OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization
- Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignment
- A Frustratingly Simple Yet Highly Effective Attack Baseline: Over 90% Success Rate Against the Strong Black-box Models of GPT-4.5/4o/o1
- Towards Deep Learning Models Resistant to Adversarial Attacks
- Probing Multimodal LLMs as World Models for Driving
- Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving
- Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning
- Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
- Visual Adversarial Attack on Vision-Language Models for Autonomous Driving
The paper
PhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems · Read on arXiv
School of Software Engineering, Xi’an Jiaotong University · School of Computer Science and Engineering, Nanyang Technological University School of Computer Science and Engineering, Northeastern University, Hangzhou Dianzi University School of Software Engineering at Xi’an Jiaotong University Center for Frontier AI Research at A*STAR
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "PhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems".
Jane: Here's the gist of the paper. Multimodal Large Language Models (MLLMs) are becoming integral to autonomous driving (AD) systems due to their strong vision-language reasoning capabilities However, MLLMs are vulnerable to adversarial attacks—particularly adversarial patch attacks—which can pose serious threats in real-world scenarios PhysPatch, a physically realizable and transferable adversarial patch framework tailored for MLLMbased AD systems, jointly optimizes patch location, shape, and content to enhance attack effectiveness and real-world applicability PhysPatch Framework Overview The proposed PhysPatch framework is designed to address the limitations of existing patch-based methods by jointly optimizing three key aspects of an adversarial patch: location, shape, and content To overcome weak attack effectiveness and poor transferability, the method replaces naive PGD-based optimization with a Global-Local Feature Alignment strategy This involves introducing a theoretically grounded SVD-based Local Alignment Loss inspired by optimal semantic compression principles Furthermore, it incorporates a Patch-Guided Crop-Resize Strategy to guarantee patch inclusion in every sampled crop, mitigating gradient vanishing issues Semantic Initialization and Refinement The method employs a Semantic-Based Mask Initialization strategy to identify physically deployable regions for patch placement in AD scenarios This initialization combines MLLM reasoning with potential field modeling Specifically, it utilizes SoM to extract semantic and spatial information, leverages GPT-4o to infer suitable patch placement regions Rj, and applies a region-centric potential field algorithm R to compute the centroid coordinate p and corresponding Gaussian potential field Φ The binary mask M is then generated by thresholding the potential field, with post-processing applied to enhance mask quality Transferability Enhancement Mechanisms To improve cross-model transferability, PhysPatch introduces a novel SVD-based local alignment loss and a patch-guided crop-resize strategy The SVD-based loss uses truncated SVD to achieve optimal semantic compression of local features and robustness to encoder variations This is achieved by forming representations f adv i and f tar i using left singular vectors U and singular values Σ from the rank-k SVD of local feature matrices The patch-guided crop-resize strategy, Tpatch, ensures the patch center p is included in every sampled crop by sampling the top-left corner (x, y) within constrained ranges based on a target crop area Ar Optimization and Real-World Applicability The optimization process involves an adaptive potential field update algorithm to refine the patch shape within physically feasible regions The mask is updated by computing the gradient G of the loss L with respect to M, updating Φ as Φ ← Φ + lr · max(0, G), and iteratively refining M based on P(Φ, τ) until a predefined threshold is met Extensive experiments demonstrate that PhysPatch consistently outperforms state-of-the-art (SOTA) methods in attack success rate and semantic alignment Moreover, the method reliably places adversarial patches in physically feasible regions of AD scenes, ensuring strong real-world applicability and deployability Experimental Validation Experiments were conducted across open-source, commercial, and reasoning-capable MLLMs on the nuScenes dataset The evaluation uses the LLM-as-a-Judge framework with GPT-4o to assess attack success rate (ASR) and average similarity (AvgSim) PhysPatch consistently outperforms existing SOTA methods across various model types, achieving superior results in both perception and planning tasks Furthermore, real-world case studies confirm the physical realizability of the attack, showing that PhysPatch successfully induces target-aligned responses in diverse driving environments Ablation and Trade-offs Ablation experiments confirm the necessity of each component; removing the potential-field-based mask update, SVD-based local alignment loss, or patch-guided cropping results in a noticeable decline in attack performance Hyperparameter studies show that while larger patches yield higher ASR, they lead to increased FID scores, illustrating a clear trade-off between attack effectiveness and image quality The method maintains high visual fidelity, with adversarial patches occupying no more than 1% of the entire image area PhysPatch proposes a physically realizable and transferable adversarial patch attack targeting MLLM-based AD systems by combining semantic-aware mask initialization, SVD-based local alignment, and patch-guided cropping The findings expose critical vulnerabilities in current MLLM-based AD systems and underscore the urgent need for robust physical-world defenses Future work plans to explore road-level scene understanding from a bird’s-eye view (BEV) perspective to expand potential placement areas The research emphasizes the need for responsible disclosure and collaborative research to mitigate potential risks associated with these techniques References Anthropic.
Jane: Introducing Claude 4.
Tom: https://www.anthropic.com/news/claude-4 Accessed 2025-06-16 Athalye, A.; Engstrom, L.; Ilyas, A.; and Kwok, K.
Title and authors: Tom: So the title is PhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems, which basically means they are tackling the challenge of making adversarial patches that actually work well on these massive vision language models for autonomous driving, but with a focus on real-world applicability.
Jane: It’s important that whatever they design can be physically realized and transferable across different models in this complex domain.
Lu: I find interesting how they are addressing the weakness of current patch attacks, which often don't transfer well from object detection models to these more complicated multimodal systems.
Meng: That’s a key hurdle for any research in this area; if an attack works on one model but fails on another, it doesn't help us deploy anything useful.
Lalam: I see how the authors are trying to create something that solves this by focusing specifically on MLLM-based autonomous driving systems rather than just general vision models.
The paper's summary: Tom: So the main point of PhysPatch is that they move away from just using standard patch optimization methods and instead use a global-local feature alignment strategy to improve how effective the attack is, which sounds like they are trying to make the attack smarter about where it puts the patch.
Jane: They introduce two big ideas for this: first, a SVD based local alignment loss to handle local feature redundancy, and second, they have this semantic-aware mask initialization combined with an adaptive potential field update algorithm for placing the patch in physically deployable spots.
Lu: It sounds like they are using the model's reasoning ability itself to suggest where the patch should go, combining things like using a model to get semantic information and then feeding that into a potential field algorithm that guides how the mask updates.
Meng: It’s about making sure the AI uses its understanding of what’s physically possible when deciding where to place the patch, which is much more grounded than random placement.
The paper's improvements: Tom: So the improvements they highlight are pretty focused on making it both better at fooling the model and better at being transferable across different models, specifically by using that SVD based local alignment loss which is inspired by optimal semantic compression principles to handle those local features.
Jane: And then they have this patch guided crop resize strategy, which guarantees the patch stays visible in every sampled crop, cutting out that gradient vanishing issue that happens with naive cropping methods.
Lu: That cropping thing sounds like a very practical fix for when the optimization process gets messy and you lose information during training.
Meng: It’s a good move if it solves those gradient vanishing issues because they are always a headache when you're trying to fine-tune things at scale.
Conclusion: Tom: So to wrap up, PhysPatch aims to be a physically realizable and transferable adversarial patch attack targeting MLLM based autonomous driving systems by combining semantic-aware mask initialization, SVD based local alignment, and patch guided cropping.
Jane: It shows that using the AI's reasoning capabilities to place the patch in feasible regions helps significantly with attack success rate across different models.
Lu: And it’s really about moving from just a theoretical paper to something that has real-world applicability for autonomous vehicles because of this approach.
Meng: From my side, I worry about how much we rely on that semantic initialization; if the reasoning model gets confused, the patch placement might be totally wrong in a real scene.
Lalam: I’m pretty sure this is solid because the results show it consistently outperforms state of the art methods in attack success rate and semantic alignment when tested on datasets like nuScenes.
Jane: It’s not just better, it reliably places adversarial patches in physically feasible regions of driving scenes, which is crucial for deployment.
Lu: Yeah, that reliability is what sells it for real-world use because the paper addresses the transferability problem head-on by using those SVD and cropping strategies.
Tom: PhysPatch is a solid piece of work because it addresses the transferability problem head-on by using those SVD and cropping strategies.
Jane: It leaves a clear path forward for making these attacks more robust against model changes, which is important when you are shipping these systems out to the road.
More episodes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound
- 2605.15085-From Data to Action: Accelerating Refinery Optimization with AI
- 2605.23500-GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation