Robust LLM Unlearning Against Relearning Attacks: The Minor Components in Representations Matter
Existing LLM unlearning methods fail because they only modify dominant model components, which attackers can easily reverse during relearning attacks. This work analyzes this fragility using representation geometry and proposes Minor Component Unlearning (MCU), a method that explicitly targets minor components to create robust unlearned models.