A Few Steps Further: Why Defenses Against Malicious Finetuning Erode Under Continued Training
Existing defenses against malicious fine-tuning (MFT) are incomplete because they only test against fixed attacks. These defenses share a common weakness: they obscure harmful behavior without removing it entirely. The paper introduces an adaptive attack that optimizes for both harmful and useful outcomes simultaneously, proving that robustness must be measured against this combined objective.