Neural networks become centroid classifiers when trained on heavily corrupted inputs

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Neural networks become centroid classifiers when trained on heavily corrupted inputs".

Jane: Here's the gist of the paper. Neural networks become centroid classifiers when trained on heavily corrupted inputs, maintaining well-above-chance accuracy even when training inputs are > 90% corrupted Empirical Findings The study empirically demonstrates that multi-layer perceptrons (MLPs) remain strikingly robust even when trained on severely corrupted versions of standard image datasets like MNIST, KMNIST, and FashionMNIST Across all three datasets, the networks retain predictive power substantially higher than random chance when inputs are nearly indistinguishable to the human eye Specifically, even when replacement noise has corrupted up to 80% of each image in the training set, test accuracy remains at or above ∼70% across all three datasets Furthermore, when considering simultaneous training-set and testing-set corruption, the optimal value of train corruption is found to be near the test corruption level rather than zero This suggests that when both sets are corrupted, it is optimal to use a training set corruption strength that matches that of the test set Analytical Treatment and Decision Rule The robustness is explained analytically using the neural tangent kernel (NTK) framework In the infinite-width limit, training with mean-squared error is equivalent to kernel regression with the NTK The derivation leads to an explicit analytical form for the learned predictor in the high-noise limit This leading-order decision rule is simple: a linear nearest-class-mean—or centroid—classifier on top of a competing stochastic background The analytical form of the predictor is a linear nearest-class-mean—or centroid classifier on top of a competing stochastic background This universal rule holds for any depth, activation function, and noise distribution Symmetry and Universality The universality of the decision rule is understood through the lens of symmetry and linear response At pure noise, training examples become exchangeable, forcing the leading class-dependent response to depend on centered class averages This connects the results to prototype-based views of classification, as the learned rule itself becomes simple, universal, and interpretable The leading class-dependent signal depends only on the clean dataset through these clean centered-class means Scaling and Performance Metrics The theory predicts an effective signal-to-noise ratio (SNR) that scales as SNR ∼ (1 − p)(2/d N/C) This suggests improved classification accuracy as the surviving signal fraction, feature dimension d, or the number of training examples per class N/C are increased Empirical validation confirms these trends for MNIST in the high-noise regime p = 0.97, showing that test accuracy improves systematically with both N and d Activation Function Dependence The analysis shows that the leading class-dependent term in the output logit is controlled by a coefficient whose sign determines the orientation of the centroid score For unit-variance noise models, this coefficient is guaranteed to be nonnegative, meaning that different activations change only subleading architecturedependent prefactors multiplying the same centroid rule Consequently, for a generic case where the pure-noise off-diagonal NTK is nonzero, the large-N relation implies that this term is suppressed by O(N −2) Thus, a broad class of activations should have nearly identical signal-to-noise ratios and similar test accuracy Variance Analysis The non-monotonic dependence of the test-accuracy variance on the noise probability p follows directly from the general form of the output logits in Eq.

Tom: (13) This arises because the correct class receives a positive deterministic offset, reflecting its larger centroid overlap, while incorrect classes do not Since the population variance is of the form m(1 − m) with m = E

Acc: , it is necessarily a non-monotonic function of p This proves that the test-accuracy variance is generically non-monotonic in p Simultaneous Corruption When both training and testing sets are corrupted, the model is trying to generalize within the same data distribution, which is more likely to succeed than out-of-distribution generalization The optimal value of train corruption ptrain is found to be near ptest rather than 0 This finding supports the centroid classification perspective, as testing on actual examples from the clean distribution is optimal for the nearest-mean-class classifier The leading decision rule learned by the network in this joint limit is i⋆ = arg max i=1,..., C 1/d X d k=1 xk∗ x¯i k − x¯k Experimental Validation The theoretical prediction for the logit output in the high-noise regime is numerically verified by finite-width experiments The empirical ensemble's mean output logits are well described by the mean centroid-based predictor in both replacement noise and additive-Gaussian settings The agreement between the empirical ensemble and the fitted prediction from Eq.

Jane: (C76) indicates that the centroid description accounts for both average logits and noise-induced variations that determine classification performance This confirms that the same centroid mechanism remains visible in finite-width networks once experimental settings match the calculation assumptions The leading-order decision rule is universal across depth, activation, and noise distribution Controllability and Outlook The analytical control enabled in the strong corruption regime may be extended to higher orders by systematically carrying out the expansion in (1 − p) This systematic expansion may provide access to the intermediate-corruption regime or even suggest a controlled route toward the clean-data limit using corruption strength as an explicit parameter An analogous expansion around the pure-noise limit could reveal what statistics of the clean data are recovered at leading order, and how progressively finer denoising information emerges at higher orders in the signal strength The leading high-noise classifier is not strongly controlled by the detailed choice of activation; the activation mainly changes subleading architecturedependent prefactors multiplying the same centroid rule This suggests that linear response around the high-noise fixed point identifies the first class-dependent correction to the predictor, which is precisely the centroid rule The success of the NTK prediction for finite-width networks suggests that this mechanism has an effective universality beyond exactly solvable infinite-width limits The leading class-dependent response remains a centroid-overlap rule for any depth, activation, and i.i.d.

Tom: noise distribution This indicates that the essential ingredients are not the particular MLP recursion alone but rather the exchangeability of the pure-noise kernel and permutation equivariance of the NTK recursion The leading decision rule is obtained by reversing the orientation of the centroid score if B(L) < 0, with its sign fixing this orientation The centroid term in Eq.

Paper summary: Tom: So we've touched on the high-level idea of this paper, "Neural networks become centroid classifiers when trained on heavily corrupted inputs." It establishes that neural networks stay robust against attribute noise, even when training inputs are over ninety percent corrupted.

Jane: They specifically examine two types of corruption models: additive noise and replacement noise, looking at datasets like MNIST, Fashion-MNIST, and KMNIST.

Lu: The paper shows that for these multi-layer perceptrons trained with cross entropy loss on those corrupted datasets, they maintain test accuracy well above random guess when the training set is heavily corrupted.

Meng: That sounds promising for handling noisy sensor data or real-world images where corruption is common, but what about the specific numbers they are using?

Tom: They show that even with replacement noise corrupting up to eighty percent of each image in the training set, test accuracy remains at or above approximately seventy percent across all three datasets <ref:2606.11319#pg2>.

Jane: And they look at simultaneous training-set and testing-set corruption, finding that the optimal value for train corruption is actually near the test corruption level rather than zero.

Lu: This suggests that when both sets are corrupted, it's better to use a training set corruption strength that matches what you expect on your test set.

Meng: If we have to balance training and testing noise, matching them sounds like a pragmatic way to optimize performance without needing perfect clean data for the whole training set.

Lalam: They also predict an effective signal-to-noise ratio scales as SNR proportional to one minus p raised to the power of two over d times n over c.

Tom: That scaling suggests that accuracy improves systematically if you increase the surviving signal fraction, or if you have more features, or more training examples per class.

Jane: And they show that for unit variance noise models, a coefficient controlling the output logit's orientation is guaranteed to be non-negative.

Lu: This means different activations only change subleading architecture dependent prefactors multiplying the same centroid rule.

Conclusion: Tom: So, wrapping up this paper "Neural networks become centroid classifiers when trained on heavily corrupted inputs," we're seeing that these models fall into a very simple pattern under heavy corruption.

Jane: The core idea is that the network essentially acts as a nearest class mean classifier on top of some competing stochastic background.

Lu: This leading order decision rule is universal across depth, activation functions, and noise distributions, which is a strong claim about the underlying mechanism.

Meng: For someone who deploys these models in production, this means we don't need incredibly clean training data to get decent results if the corruption level is high.

Lalam: The implication for me is that it confirms the centroid description accounts for both average logits and noise induced variations that determine classification performance in finite width networks when experimental settings match the calculation assumptions.

Tom: It really brings back how simple these systems can become when you push them to their limits with noise. We should think about how this affects our robustness testing protocols next.