Test-Time Collective Action: Proxy-Based Perturbations for Correcting Algorithmic Harms

The gist: Test-Time Collective Action (TTCA) is a framework allowing a group of users to correct algorithmic harms affecting under-served subgroups by generating per-class perturbations at test time without requiring platform retraining.

Introduction and Motivation

Machine learning systems can exhibit performance disparities across different user groups, with documented examples in computer vision [Buolamwini and Gebru, 2018], safety features of language models [Chehbouni et al., 2024] and automated speech recognition [Koenecke et al., 2020]. When users encounter these harms, provider-centric fairness interventions such as pre-processing, in-processing, and post-processing methods are commonly used to address them [Caton and Haas, 2024]. This could be considered reasonable if platforms are willing to act towards reducing these harms; however, when remediation is unavailable, it leaves affected users with no lever of their own to address the situation [Page 1]. Recent work in Algorithmic Collective Action shows that coordinated users can steer an algorithmic system toward a collective goal, but the existing mechanisms require the provider to retrain on the collective’s modified data which users may not have control over [Page 1]. Test-Time Collective Action (TTCA) is proposed as a framework through which a group of users who share query access to the platform can correct disparities affecting under-served subgroup without participating in the platform’s training loop [Page 1].

How it works

The TTCA framework operates in two main phases, as illustrated in Figure 1 [Page 2]. Phase 1 involves distilling a collective proxy and optimizing y through collective queries on the platform model plat which yields a proxy, and then optimizing y against this proxy [Page 2]. Phase 2 is Test-Time Collective Action where a misclassified member applies y to recover their true class by submitting x + y at test time, requiring no cooperation from the platform [Page 2]. The high-level pipeline involves extracting a collective proxy from pooled queries, then applying the resulting perturbation at test-time [Page 2].

The mechanism for crafting the perturbation involves several steps:

1. Query pooling: Each member uses their query budget to label a subset of Dpub with predictions from fplat, which are aggregated into a shared pool Dproxy = (x, fplat(x)) of Q pairs [Page 4].

2. Proxy extraction: The collective then trains a surrogate fproxy on Dproxy via KnockoffNets random sampling [Orekondy et al., 2019], fitting a network with cross-entropy against the platform’s soft-probability outputs, and additionally trains K = 5 proxies with independent random seeds and ensembling their outputs as fens = (Page 4].

3. Perturbation optimization: For each class y in G, the collective optimizes a perturbation δy against the proxy ensemble under an l∞ budget: δy = arg min δ∞≤ε Ex∼Dy proxy CE(fens(˜x), y) [Page 4].

4. Test-time application: A member with input x of class y checks whether y ∈ G; if so, they submit clip(x + δy, 0, 1) instead of x; otherwise the input is sent unchanged [Page 4].

Empirical Evaluation and Performance

The mechanism was empirically evaluated on CIFAR-10, CIFAR-100, and FairFace under both balanced and unbalanced platform training regimes [Page 3]. The results show that modestly-sized collectives close most of the subgroup accuracy gap, transfer across architectures (a small proxy can attack a larger platform), and improve worst-group accuracy, equal-opportunity gap, and disparate impact [Page 1]. Specifically, subgroup accuracy improves even at the smallest pool size and once a proxy is faithful enough to the predictions of the platform model, additional queries do not significantly increase subgroup accuracy [Page 6].

A query-budget analysis comparing a per-user black-box attack baseline shows that pooling is cheaper than each subgroup member attacking alone [Page 1]. The collective’s intervention requires far fewer platform queries than individuals attacking alone; for instance, the collective spends N = 22,000 queries once to train a shared proxy, while individual SimBA requires approximately 2.3 million cumulative queries from the 1,000-user subgroup [Page 6].

Fairness and Comparison to Baselines

The primary outcome is the subgroup accuracy from Equation 1: SubgroupAcc = Pr fplat(x′) = y for y ∈ G, (1) [Page 4]. The intervention is corrective if subgroup accuracy rises while these metrics improve or hold [Page 6]. Most settings move in the improving direction for all three metrics: above the y=x diagonal for WGA and DI, and below it for EOD [Page 13].

The individual baseline is instantiated with targeted SimBA using the pixel variant, which models an uncoordinated scenario in which users attempt to correct their own misclassifications without sharing data, queries, or perturbations with others [Page 9]. The collective’s advantage is that it pays the proxy-extraction and perturbation-generation cost once and produces a reusable perturbation δy that can be applied by non-contributing affected users at test time [Page 12].

Conclusion

Test-Time Collective Action offers corrective intervention to users when platform-side remediation is unavailable or delayed [Page 1]. The work demonstrates that a small group of coordinated users can improve model performance and fairness without requiring model retraining, or even knowledge of its architecture [Page 2]. TTCA contributes a collective, black-box, query-budgeted formulation of corrective perturbations rather than a new platform-side fairness intervention [Page 2]. The result is consistent across CIFAR-10, CIFAR-100, and FairFace; under both balanced and deliberately skewed platform-training regimes; and even when the collective’s proxy is smaller than the platform it queries [Page 6].

The work contributes to the ongoing discussion on external, user-driven mechanisms to correct harms [Page 17]. The proposed mechanism shows that a small pool is already enough for that local recovery, which is why subgroup accuracy lifts at Q = 500 in §6.1 without the proxy’s overall fidelity being high [Page 19]. The collective reaches its ceiling with roughly 23,000 total platform queries paid once and reused across users, whereas bringing the 1,000 subgroup users to the individual ceiling requires approximately 2.3 million cumulative queries [Page 6].

The paper also notes that when the task itself is harmful, user-side action may need to move beyond the corrective framing developed in this paper and instead take forms such as refusal, opt-out, or contestation [Page 14]. The intervention focuses on quality-of-service harms where the underlying task objective is broadly useful to users [Page 2].

The work also suggests that platform defenses can be effective in preventing TTCA but also cost the platform in terms of accuracy degradation [Page 18]. The framework acts as a complementary approach to the ACA framework in the test-time setting, showing that collective action remains a viable option even when users cannot influence the training loop or know whether their data was used to train the model [Page 18].

The paper concludes by proposing one specific way of implementing a test-time collective action framework where we use a KnockoffNets ensemble with a per-class optimized perturbation [Page 13]. The authors emphasize that the corrective rate substantially exceeds the cross-target rate at every pool size, indicating that the perturbation is a stronger lever for correction than for misuse on these benchmarks [Page 18].

The paper also suggests that stronger individual attacks would shift the break-even K⋆ upward and tighten the per-contributor cost threshold [Page 12]. The comparison highlights the main advantage of pooling, as it amortizes the proxy-extraction and perturbation-generation cost across contributors, producing a reusable perturbation δy that can be applied by non-contributing affected users at test time [Page 12].

The authors also note that in settings where the task itself is harmful, user-side action may need to move beyond the corrective framing developed in this paper and instead take forms such as refusal, opt-out, or contestation [Page 14]. The intervention focuses on quality-of-service harms where the underlying task objective is broadly useful to users [Page 2].

The paper also suggests that platform defenses can be effective in preventing TTCA but also cost the platform in terms of accuracy degradation [Page 18]. The framework acts as a complementary approach to the ACA framework in the test-time setting, showing that collective action remains a viable option even when users cannot influence the training loop or know whether their data was used to train the model [Page 18].

1. The Test-Time Collective Action (TTCA) framework allows a group of users who share query access to the platform, can correct disparities affecting under-served subgroup without participating in the platform’s training loop. This enables users to correct quality-of-service harms by applying a per-class universal perturbation at inference time.

2. The system can generate corrective perturbations using only API query access by first distilling a collective proxy from pooled queries, which is then used to optimize a perturbation against it. This process requires the collective to pool their data contributions to generate a perturbation that mitigates the algorithmic harms of a frozen black-box model using only API query access.

3. The system can be deployed across different platform architectures because a small proxy can attack a larger platform, as demonstrated by cross-architecture transfer where perturbations crafted on a smaller proxy transfer effectively to the larger ResNet-50 or RegNet-Y-16GF platform.

4. The collective intervention translates subgroup accuracy gains into tangible fairness improvements, showing that the mechanism can improve worst-group accuracy and equal-opportunity gap by achieving results in Figure 5 that show metrics move in the improving direction for all three metrics.

5. The system offers significant query efficiency compared to individual attacks, as pooling is cheaper than each subgroup member attacking alone, requiring a collective size of only n ≥ 6 at B=4,000 to match individual performance on CIFAR-100.

When machine learning systems under-perform for particular subgroups, affected users typically have no way to correct these disparities without relying on platform-level fixes. Existing approaches to algorithmic fairness rely on provider-centric approaches to correct these failures, leaving users with no external lever when faced with harm. Recent work in Algorithmic Collective Action shows that coordinated users can steer an algorithmic system toward a collective goal, but the existing mechanisms require the provider to retrain on the collective's modified data which users may not have control over. We propose Test-Time Collective Action (TTCA), a framework through which a group of users who share query access to the platform, can correct disparities affecting under-served subgroup without participating in the platform's training loop. We implement this through a proxy-based mechanism where the collective pools query access to a black-box API to extract a proxy of the platform, then optimizes a per-class universal perturbation against the proxy. Each member applies this perturbation to their own inputs at submission time, requiring no cooperation from the platform. We empirically evaluate the mechanism on CIFAR-10, CIFAR-100, and FairFace, showing that modestly-sized collectives close most of the subgroup accuracy gap, transfer across architectures (a small proxy can attack a larger platform), and improve worst-group accuracy, equal-opportunity gap, and disparate impact. A query-budget analysis comparing a per-user black-box attack baseline shows that pooling is cheaper than each subgroup member attacking alone. Test-time collective action thus offers corrective intervention to users when platform-side remediation is unavailable or delayed.