Adversarial Contamination Meets Hard Thresholding: An Iterative Algorithm with Signal Adaptivity and Minimax Optimality
Shixiang Liu, Hanming Yang
stat.ML, cs.LG
Submitted: 2026-06-26
Comments: 56 pages, 6 figures
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Pervasive data contamination -- stemming from measurement errors, outliers, or adversarial corruption -- has motivated the development of robust statistical methods.
Terminology
Abstract
Pervasive data contamination -- stemming from measurement errors, outliers, or adversarial corruption -- has motivated the development of robust statistical methods. In this context, we propose a two-stage Adversarial Contamination-resistant Iterative Hard Thresholding (AC-IHT) algorithm for high-dimensional regression with contamination. Our nonconvex algorithm achieves minimax near-optimal (up to logarithmic terms) estimation by iteratively updating the coefficient vector and the contamination vector with different thresholding scales. We further demonstrate that our AC-IHT estimator is signal-adaptive: under proper signal conditions, it adaptively attains a sharper estimation rate and more accurate support recovery. Moreover, it enjoys the strong oracle property, laying a theoretical foundation for asymptotic inference. Numerical experiments confirm its superior finite-sample performance. Finally, we discuss theoretical extensions of the proposed procedure to generalized linear models and to heavy-tailed noise settings.
Sources
- Robust estimation with Lasso when outputs are adversarially contaminated
- Robust-to-outliers square-root LASSO, simultaneous inference with a MOM approach
- Robust Regression under Adversarial Contamination: Theory and Algorithms for the Welsch Estimator
- Outlier-robust sparse/low-rank least-squares regression and robust matrix completion
- Scaled minimax optimality in high-dimensional linear regression: A non-convex algorithmic regularization approach
- The noise barrier and the large signal bias of the Lasso and other convex estimators
- Introduction to the non-asymptotic analysis of random matrices
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Scalable Posterior Uncertainty for Flexible Density-Based Clustering
- A Transport Geometry of Benign Overfitting
- Efficient quadratic entropy with distance sketches