Subtract the Corruption: Training-Data-Free Corrective Machine Unlearning using Task Arithmetic
cs.LG, stat.ML
Submitted: 2025-11-24
Updated: 2025-11-25
Journal ref: Machine Learning and Knowledge Discovery in Databases. Research Track. ECML PKDD 2026, LNCS 16942, pp. 340-355, Springer, 2027
DOI: 10.1007/978-3-032-37657-2_19
Code: https://github.com/mosix11/CUTS
License: http://creativecommons.org/licenses/by/4.0/
The gist: Corrupted training data are ubiquitous.
Terminology
Abstract
Corrupted training data are ubiquitous. Corrective Machine Unlearning (CMU) seeks to remove the influence of such corruption post-training. Prior CMU typically assumes access to identified corrupted training samples (a "forget set"). However, in many real-world scenarios the training data are no longer accessible. We formalize source-free CMU, where the original training data are unavailable and, consequently, no forget set of identified corrupted training samples can be specified. Instead, we assume a small proxy (surrogate) set of corrupted samples that reflect the suspected corruption type without needing to be the original training samples. In this stricter setting, methods relying on forget set are ineffective or narrow in scope. We introduce Corrective Unlearning in Task Space (CUTS), a lightweight weight space correction method guided by the proxy set using task arithmetic principles. CUTS treats the clean and the corruption signal as distinct tasks. Specifically, we briefly fine-tune the corrupted model on the proxy to amplify the corruption mechanism in the weight space, compute the difference between the corrupted and fine-tuned weights as a proxy task vector, and subtract a calibrated multiple of this vector to cancel the corruption. Without access to clean data or a forget set, CUTS recovers a large fraction of the lost utility under label noise and, for backdoor triggers, nearly eliminates the attack with minimal damage to utility, outperforming state-of-the-art specialized CMU methods in source-free setting.
Sources
- Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- A Survey on Backdoor Threats in Large Language Models (LLMs): Attacks, Defenses, and Evaluations
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks