On the Existence of Consistent Adversarial Attacks in High-Dimensional Linear Classification
stat.ML, cond-mat.dis-nn, cs.CR, cs.LG
Submitted: 2025-06-14
Updated: 2026-09-01
Journal ref: ICML 2026
License: http://creativecommons.org/licenses/by/4.0/
The gist: What fundamentally distinguishes an adversarial attack from a misclassification due to limited model expressivity or finite data? In this work, we investigate this question in the setting of
Terminology
Abstract
What fundamentally distinguishes an adversarial attack from a misclassification due to limited model expressivity or finite data? In this work, we investigate this question in the setting of high-dimensional binary classification, where statistical effects due to limited data availability play a central role. We introduce a new error metric that precisely capture this distinction, quantifying model vulnerability to consistent adversarial attacks -- perturbations that preserve the ground-truth labels. Our main technical contribution is an exact and rigorous asymptotic characterization of these metrics in both well-specified models and latent space models, revealing different vulnerability patterns compared to standard robust error measures. The theoretical results demonstrate that as models become more overparameterized, their vulnerability to label-preserving perturbations grows, offering theoretical insight into the mechanisms underlying model sensitivity to adversarial attacks.
Sources
- The Gaussian min-max theorem in the Presence of Convexity
- Asymptotics of Non-Convex Generalized Linear Models in High-Dimensions: A proof of the replica formula
- Adversarially Robust Generalization Just Requires More Unlabeled Data
- Why adversarial training can hurt robust accuracy
- On the Geometry of Regularization in Adversarial Training: High-Dimensional Asymptotics and Generalization Bounds
- Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness
- Robust Linear Regression: Phase-Transitions and Precise Tradeoffs for General Norms
- The Price of Implicit Bias in Adversarially Robust Generalization
- Over-parameterization and Adversarial Robustness in Neural Networks: An Overview and Empirical Analysis
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Efficient quadratic entropy with distance sketches
- Embedding-Bias in Conditional Independence Testing
- From Log-Odds to Shapley Values: An Explanatory Geometry for the Weighted Naive Bayes Classifier