Sum-of-Squares Degree Barriers for the Reweighted-Hinge Method in Robust Halfspace Learning: A Christoffel-Function Characterization
Xiaoyu Li
cs.LG, cs.DS, stat.ML
Submitted: 2026-08-17
Updated: 2026-08-18
License: http://creativecommons.org/licenses/by/4.0/
The gist: A certificate that removes outliers sees the data only through its low-degree moments, and an adversary exploits exactly this, hiding corruption where the clean data already looks typical, in the
Terminology
Abstract
A certificate that removes outliers sees the data only through its low-degree moments, and an adversary exploits exactly this, hiding corruption where the clean data already looks typical, in the blind spot no bounded-degree test resolves. That blind spot turns out to have an exact size: the Christoffel function of the clean marginal, the very quantity modern data analysis thresholds to detect outliers, here read from the adversary's side as the corruption a bounded-degree certificate cannot remove. We turn this inversion into the organizing principle of the reweighted-hinge approach to robustly learning gamma-margin halfspaces under malicious noise (Shen, 2025; Zeng and Shen, 2025): the governing resource is the Sum-of-Squares degree of the outlier-removal certificate, and the resolution principle states that the maximal corruption mass which can hide at a center c from a degree- 2t certificate is exactly the Christoffel function lambda t+1(c) of the clean marginal. Three consequences follow, all against the certificate method (not information-theoretic). A margin-degree tradeoff: certifying the dense pancake to error epsilon costs SoS degree ((1/epsilon)) or margin (sqrt (1/epsilon) /sqrt d), explaining why the (1/epsilon) margin Shen (2025) records is forced, with a weighted-Chebyshev reduction making the threshold 2t= ((c/s) 2) tight modulo one classical weighted-extremal estimate. A degree- 2 outlier barrier: the resolution principle realized as an explicit instance on which degree 2 is stuck at eta 1/2 while degree 4 escapes, locating the method's small breakdown rate in the degree, not the analysis. And a degree- 2t algorithm tracing the frontier eta 1-1/2t (recovering Shen (2025) at t=1), whose gain is an explicit constant, capped by the pancake density and shown unimprovable by the degree- 2 barrier.
Sources
- Efficient and Noise-Tolerant PAC Learning of Multiclass Linear Classifiers
- Efficient Learning of Linear Separators under Bounded Noise
- The Power of Localization for Efficiently Learning Linear Separators with Noise
- Robustly Learning Mixtures of $k$ Arbitrary Gaussians
- Efficient Certificates of Anti-Concentration Beyond Gaussians
- Active and passive learning of linear separators under log-concave distributions
- Perturbations of Christoffel-Darboux kernels. I: detection of outliers
- Recent Advances in Algorithmic High-Dimensional Robust Statistics
- Near-Optimal Statistical Query Hardness of Learning Halfspaces with Massart Noise
- Learning Geometric Concepts with Nasty Noise
- Distribution-Independent PAC Learning of Halfspaces with Massart Noise
- Robust Estimators in High Dimensions without the Computational Intractability
- Sever: A Robust Meta-Algorithm for Stochastic Optimization
- Near-Optimal SQ Lower Bounds for Agnostically Learning Halfspaces and ReLUs under Gaussian Marginals
- Robust Sparse Mean Estimation via Sum of Squares
- Learning General Halfspaces with General Massart Noise under the Gaussian Distribution
- Cryptographic Hardness of Learning Halfspaces with Massart Noise
- Information-Computation Tradeoffs for Learning Margin Halfspaces with Random Classification Noise
- SoS Certifiability of Subgaussian Distributions and its Algorithmic Applications
- Mixture Models, Robustness, and Sum of Squares Proofs
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks