Confusion-Geometry Rebalancing for Long-Tailed Adversarial Training

Adversarial training under long-tailed distributions suffers from a dual imbalance: the class imbalance skews the training objective toward head classes, and the adversarial inner maximization may further amplify this bias. Existing methods mitigate this issue by correcting class priors or adapting class-wise robust supervision, yet they treat each class in isolation and fail to identify which boundaries drive long-tailed collapse. We propose a Confusion-Geometry Rebalancing method (CGRm) for long-tail adversarial training, a plug-in framework that leverages directed robust errors as training signals. CGRm leverages periodic robust evaluations to derive source-class loss weights, class-wise robust coefficients, and a directed confusion-geometry graph. The method then couples feedback-weighted robust optimization with graph-guided margin correction, thereby boosting the robustness of vulnerable classes and sharpening the critical boundaries that drive long-tailed performance degradation. Experiments on long-tailed benchmarks show that CGRm achieves consistent robust performance gains over existing methods, with ablations validating the contribution of each component. We provide the code in the supplement.

The vulnerability of deep neural networks to imperceptible adversarial perturbations has raised critical concerns about their deployment in safety-critical systems. Adversarial training (AT) is widely regarded as one of the most effective defenses against this threat. It is typically formulated as a min-max optimization problem. Despite its empirical success, AT has been predominantly studied under an implicit assumption of balanced class priors. However, this assumption rarely holds in real-world scenarios, where sampled datasets are intrinsically long-tailed: a few head classes account for the majority of samples, while numerous tail classes are represented by very few instances. When AT is applied directly to such skewed data, it introduces a two-fold imbalance: head classes not only dominate the training objective, but also disproportionately influence the adversarial decision boundaries during the inner maximization. Consequently, the learned model may achieve satisfactory average robust accuracy, yet its performance on tail classes remains notably inferior.

Recent studies improve the long-tailed AT from complementary perspectives. One line of work focuses on prior correction, adjusting logits, losses, or sample weights to counteract label imbalance. Another thread develops class-aware robust calibration, adapting perturbation budgets, regularization strengths, or class-specific weights based on each class’s robust behavior. Most recently, RobustLT shows that the adversarial inner maximization itself should also be class-dependent, reallocating perturbation radii and attack schedules across the imbalance spectrum. Despite these advances, existing methods share a common limitation: each class’s robust error is treated in isolation, without considering where those errors are directed. This directional information matters in long-tailed AT. Under adversarial attack, an example from a source class is driven across the decision boundary toward different target classes. In long-tailed recognition, these errors are often structured rather than random: tail-class examples tend to collapse into frequent classes, especially when their feature representations lie close in the adversarial space. Consequently, treating all errors from an under-robust class equally may waste robust supervision on irrelevant negatives, while scalar class feedback alone cannot identify which target classes actually absorb these errors. We argue that robust long-tailed training should consider not only whether a class is vulnerable, but also where its adversarial examples are directed and whether the resulting confusion is geometrically plausible.

Motivated by this observation, we propose a Confusion-Geometry Rebalancing method (CGRm) for long-tail AT, a plug-in framework that explicitly accounts for directed robust errors. The method periodically evaluates the model under attack and collects three forms of feedback—class-wise robust accuracy, confusion counts, and adversarial feature centers—which jointly characterize each class’s vulnerability and its error direction. This feedback serves two complementary purposes. On one hand, it produces class-adaptive weights and robust coefficients that amplify the contribution and regularization of under-robust classes. On the other hand, it builds a directed confusion-geometry graph whose edges capture tail-to-head misclassifications that are both frequent and geometrically plausible; a margin loss is then applied exclusively to these selected pairs. In this way, CGRm rebalances not only how much each class is supervised, but also which boundaries are explicitly hardened.

Our contributions are summarized as follows:

• We formulate long-tailed AT as a directed robust error-correction problem and propose CGRm, modeling not only per-class vulnerability but also the target direction and geometric plausibility of robust confusions.

• We introduce a dynamic feedback mechanism that decouples per-class gradient contribution from clean-robust regularization by separately updating loss weights and robust coefficients based on evolving robust-error statistics.

• We design a confusion-geometry boundary correction module that builds a directed graph from confusion frequency, target-class headness, and adversarial feature affinity, and then applies a margin loss exclusively to systematic source-target confusions.

• Extensive experiments on long-tailed benchmarks show that CGRm consistently outperforms existing methods.

Existing methods treat all misclassifications from a vulnerable class equally, ignoring both which competing class the error goes to and whether that confusion is geometrically plausible in feature space. CGRm addresses these two missing dimensions. It jointly tracks robust errors and their source-target geometry: class frequency only initializes adversarial exploration, while subsequent corrections are driven by observed robust errors, their target classes, and the feature-space proximity of the confused classes.

This work propose CRGm, which periodically evaluates the model under attack and translates the robust errors into optimization objectives: class-adaptive weights and coefficients that rebalance per-class loss contribution and regularization toward clean-robust consistency, and a directed graph that applies margin loss to systematic source-target confusions. To mitigate the adverse effects of uniformly assigned perturbation budgets in imbalanced data scenarios, we reuse Eqs. (7) and (8) to calculate the attack budgets ϵc and step size ηc. For a sample (x, y), we then generate xadv using such class-conditioned budget ϵy and step size ηy. Given these adversarial examples, CGRm optimizes three coupled signals: a prior-calibrated natural classification term for long-tailed recognition, a feedback-weighted robust consistency term for vulnerable source classes, and a confusion-geometry margin term for the target classes that absorb adversarial errors. The objective is expressed as LCGR = E [lbal (fθ (x), y)] + Lrob + λm Lcgm. The first term denotes a balanced prior-calibrated classification term. Let πc = nc/N and bc = log(πc). For logits z = fθ (x), we use lbal (z, y) = −log exp(zy + by)/Σj exp(zj + bj). Lrob reallocates the consistency between clean and adversarial predictions across classes. We introduce two independent factors: the source-class weight wy modulates the overall gradient contribution from samples of class y, while the class-wise robust coefficient βy controls the strength of the consistency constraint imposed on that class. This allows the model to emphasize an under-robust class and independently enforce a stronger clean-adversarial prediction agreement on it. For example, when adopting TRADES as the base learner, Lrob is expressed as Lrob = E [wy(t) βy(t) DKL (fθ (x) ∥ fθ (xadv))]. The confusion-geometry margin complements this source-class correction with target-class information. A low robust accuracy indicates that class y is vulnerable, but it does not identify which wrong class should be separated from y. The graph G(t) selects target classes that adversarial examples from y are frequently confused with and that are geometrically plausible competitors. CGRm then applies a margin penalty only to these selected targets: Lcgm = E(x,y) [wy(t) Σj≠y Gyj(t) [m − zyadv + zjadv]+], where m denotes the margin threshold, z adv = fθ (xadv). At the beginning of training, wy = 1, βy = β, and Gy,j = 0. As training progresses, these variables are updated from robust-error statistics.

The frequency prior is only a proxy for robust difficulty. Some rare classes may be clearly separable, whereas some medium-frequency classes may remain vulnerable due to feature overlap or semantic ambiguity. CGRm thus updates its training emphasis using the model’s own robust behavior. At an evaluation epoch t, we attack a monitoring split (subset of the training dataset) and compute the class-wise robust accuracy ac(t) and robust confusion matrix M(t), where Mij(t) counts examples from class i predicted as class j under attack. We convert these measurements into a feedback score using two complementary signals. The first measures how far a class falls below the mean robust accuracy, gc(t) = max(ā(t) − ac(t), 0), ā(t) = (1/C) Σj aj(t). The second measures how often adversarial examples from class c leave the correct class: ec(t) = 1 − Mcc(t) / (Σj Mcj(t) + ξ), where ξ is a negligible positive constant for numerical stability. We combine the two signals as sc(t) = (gc(t) + ec(t)) / ((1/C) Σj (gj(t) + ej(t)) + ξ). Because robust evaluation can be noisy, especially for tail classes, we use an exponential moving average s̃c(t) = µs̃c(t−1) + (1 − µ)sc(t). Based on s̃c(t), we calculate the per-class loss weight in Eq. (11), which answers the question of how much examples from class c should contribute to the optimization objective: wc(t) = clip((1 + s̃c(t)) / ((1/C) Σj (1 + s̃j(t))), 0, wmax). Clipping prevents noisy tail estimates from producing excessive gradients, while normalization keeps the average weight stable across epochs. Additionally, s̃c(t) is used to determine the regularization strength for each class. A single global coefficient β applies the same clean-robust trade-off across all classes, which becomes suboptimal when robust difficulty varies unevenly: increasing β universally may hurt clean accuracy on easier classes, while decreasing it leaves harder classes under-regularized. To address this, we adapt β per class based on the proposed robust-error score: βc(t) = β · clip(1 + s̃c(t)/s̄(t), 1, rmax) / ((1/C) Σj clip(1 + s̃j(t)/s̄(t), 1, rmax)), where s̄(t) = (1/C) Σj s̃j(t). Unlike wc, which modulates a class’s contribution to the overall objective, βc adjusts the clean-robust trade-off within the robust term.

Class-wise feedback identifies vulnerable source classes, yet it treats each class independently, overlooking the inherently relational nature of robust failure. An adversarial example from class i becomes harmful precisely because it crosses the decision boundary toward a specific target class j. In long-tailed recognition, the most detrimental errors are often not random misclassifications, but rather tail classes being absorbed by neighboring head classes. To address this, CGRm constructs a directed confusion-geometry graph that selects source-target pairs requiring explicit margin correction. The graph integrates three complementary signals: empirical confusion probabilities, class-frequency bias, and feature-geometry affinity. The graph at epoch t is formulated as the element-wise product of these three terms: Ĝij(t) = Pij(t) hj Aij(t), i ≠ j, Ĝii = 0. Here, the row-normalized robust confusion probability Pij(t) = Mij(t) / (Σk Mik(t) + ξ) reflects the empirical tendency of adversarial examples from i to cross the decision boundary toward j. The headness score hj aims to penalize confusions that are more damaging under long-tailed imbalance: hj = sqrt(nj/nmax), where nc is the sample count of class c and nmax = maxc nc. The geometry affinity Aij(t) measures the geometric proximity between the two classes in the adversarial feature space. Let mc(t) denote the feature center of class c computed from adversarial examples, and define Aij(t) = 1 / (∥m̂i(t) − m̂j(t)∥2 + ξ), where m̂ is l2-normalized. To reduce noise and maintain sparsity, we retain only the top-k outgoing edges for each source class and normalize the nonzero entries to unit mean. The graph is then updated smoothly across epochs via exponential moving average: G(t) = µG(t−1) + (1 − µ)Ĝ(t). The target classes selected by Eq. (19) are those that (i) are frequently confused with adversarial examples from the source class, as measured by Eqs. (20) and (21), and (ii) serve as geometrically plausible competitors, as quantified by Eq. (22). Thus, this graph serves as a targeted description of long-tailed robust collapse.

Following RobustLT, we apply the exponential long-tailed sampling strategy to derive CIFAR10-LT, CIFAR100-LT, and TinyImageNet-LT (200 classes) from CIFAR10, CIFAR100, and TinyImageNet, respectively. For C classes with imbalance ratio K, the number of training samples for class i is ni = nmax K−i/(C−1), where classes are sorted from head to tail. We set the imbalance ratio to 50 for CIFAR10-LT and 10 for CIFAR100-LT and TinyImageNet-LT. The test sets retain the original balanced class distributions. We compare against advanced AT techniques, including conventional debiased AT methods (RoBal, REAT, BSL, TAET) and plug-in methods (UDR, CFA, DAFA, RobustLT) that are applied on base AT algorithms such as PGD-AT, TRADES, and AWP. For fair comparison, each plug-in method shares the same learner, backbone, and attack setting as its corresponding baseline. We report natural and robust accuracy on all test classes (Nat.(all) and Rob.(all)), as well as on the 80% of classes with fewest training samples (Nat.(tail) and Rob.(tail)). Robust accuracy is measured under a 20-step l∞ PGD attack with radius 8/255 and step size 2/255. As an additional sanity check, we also evaluate the final models with AutoAttack to rule out gradient masking. The main experiments use ResNet18, PreActResNet, WideResNet as backbones, optimized with SGD (momentum 0.9, weight decay 5 × 10−4) and an initial learning rate of 0.1. All models are trained for 110 epochs. The perturbation radius is set to ϵ = 8/255. Adversarial examples are generated by 10-step PGD during training and evaluated by 20-step PGD at test time. The base AT configuration is kept unchanged. For CGRm-specific settings, the robust-error feedback and the confusion-geometry graph are updated every 10 epochs. For each source class, the graph retains the top-3 most relevant target classes. The resulting graph is used only in the margin regularization term with weight λm = 10/C, where C denotes the number of classes. All other CGRm-specific hyperparameters are fixed across datasets and listed in the appendix.

Tables 1-3 present a comprehensive comparison across datasets, backbones, and base learners. Compared with existing plug-in methods (e.g., UDR, CFA, DAFA, and RobustLT), CGRm consistently enhances both clean and adversarial performance, with the most stable gains observed on tail classes. For example, under ResNet-based PGD-AT on CIFAR10-LT, CGRm increases Nat.(tail)/Rob.(tail) from 46.81%/18.60% (RobustLT) to 66.15%/27.93%. On CIFAR100-LT with TRADES, it improves the best competitor’s Rob.(all)/Rob.(tail) from 20.68%/18.25% to 21.54%/20.73%, while also raising Nat.(all) from 45.50% to 48.81%. These gains indicate that the proposed method allocates robust learning pressure more effectively toward vulnerable classes and ambiguous boundaries. The improvement persists when switching from ResNet to Pre-ResNet and WideResNet. On Pre-ResNet, CGRm attains strong tail robustness. In the sole case where RobustLT marginally surpasses CGRm in overall robustness—TinyImageNet-LT with TRADES (14.08% vs. 13.…

Based on this paper, here are the specific improvements you can make to AI systems:

1. Directed Error-Aware Training

• Instead of treating all misclassifications equally, the system now tracks where errors go (source→target class pairs) and uses this directional information to guide training.

• The system identifies systematic confusions (e.g., tail classes collapsing into head classes) and applies targeted margin penalties only to those specific class pairs, rather than applying uniform regularization.

2. Dynamic Class-Adaptive Feedback

• The system periodically evaluates its own robust performance under attack and converts this into three adaptive signals: per-class loss weights, per-class regularization coefficients, and a confusion graph.

• These signals are updated via exponential moving averages to reduce noise, especially for rare classes, and are clipped to prevent unstable gradients.

3. Decoupled Clean-Robust Trade-off

• The system separates how much a class contributes to the overall loss (via weights) from how strongly clean-adversarial consistency is enforced (via class-specific coefficients).

• This allows under-robust classes to receive both more gradient contribution and stronger regularization without penalizing easier classes.

4. Confusion-Geometry Graph Construction

• The system builds a directed graph using three signals: empirical confusion probabilities, class-frequency bias (headness), and feature-space proximity of adversarial examples.

• Only the top-k most relevant target classes per source are retained, ensuring sparsity and focusing on geometrically plausible confusions rather than random errors.

5. Class-Conditioned Attack Budgets

• Perturbation radii and step sizes are computed per class based on robust-error statistics, preventing uniform attack budgets that disadvantage tail classes.

---

• Achieve significantly higher robustness on rare/tail classes (e.g., +9.3% tail robust accuracy on CIFAR10-LT, +2.5% on CIFAR100-LT) while maintaining or improving overall accuracy.

• Maintain robustness under stronger attacks (AutoAttack, C&W), not just PGD, showing resistance to gradient masking.

• Scale with model capacity—gains increase when using larger backbones like WideResNet.

• Generalize across datasets and base learners (PGD-AT, TRADES, AWP) without requiring per-dataset hyperparameter tuning.

• Avoid over-regularization of easy classes, preserving clean accuracy while hardening vulnerable boundaries.

• Provide interpretable feedback on which class pairs are most problematic, enabling targeted debugging of model failure modes.

• Operate as a plug-in module, meaning it can be added to existing adversarial training frameworks without architectural changes.