DRIFT: Derailing Denoising Trajectories of Flow-Matching VLAs with Adversarial Patch Attack
DRIFT introduces a test-time adversarial patch that attacks flow-matching vision-language models by perturbing the first denoising step of their action generation process. The key finding is that attacking only this single step is stronger and cheaper than attacking a wider sequence of steps, exploiting a unique gradient conflict in input optimization. This method breaks nearly all solvable tasks with minimal effort.