MOAT: Model-Agnostic Randomized Transformations for preventing Efficiency Degradation Attacks on ViTs
Anadi Goyal, Nandish Chattopadhyay, Chandan Karfa, Anupam Chattopadhyay, Norrathep Rattanavipanon
cs.CR, cs.CV
Submitted: 2026-08-05
Comments: This paper has been accepted for publication at IEEE ISVLSI 2026
License: http://creativecommons.org/licenses/by/4.0/
The gist: To adopt the Vision Transformers (ViTs) in resource-constrained environment, token pruning is widely used to reduce computational cost without impacting accuracy.
Terminology
Abstract
To adopt the Vision Transformers (ViTs) in resource-constrained environment, token pruning is widely used to reduce computational cost without impacting accuracy. However, adversaries have developed targeted attacks against said token pruning techniques to undermine such attempts to make ViTs efficient. In this paper, we propose MOAT, a model-agnostic pre-processing defense pipeline that applies a combination of input transformations to protect efficient ViT implementations against adversarial efficiency attacks. MOAT operates directly on the input without requiring modifications to the model architecture or token pruning mechanism. Experimental results demonstrate that, across all evaluated ViT models, MOAT limits GFLOPs degradation under adversarial attacks to within 3.4% of the original unattacked model.
Sources
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Anomaly Unveiled: Securing Image Classification against Adversarial Patch Attacks
- A Panda? No, It's a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network Inference
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Estimating the Social Cost of Corporate Data Breaches
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills