Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning
The episode analyzes how LLM backdoors function by exploiting attention sinks, noting that trigger placement is key. Prefix triggers, placed at the start of input, are shown to be highly effective because they capture early-layer attention. The discussion concludes that defense requires 'value-norm regularization' to stabilize training and strengthen unlearning without compromising the model's general utility.