Anatomy of Unlearning: The Dual Impact of Fact Salience and Model Fine-Tuning
The episode discusses 'Anatomy of Unlearning,' a study showing that erasing knowledge from AI is not uniform. The difficulty depends on fact salience (popularity) and training origin (pretraining vs. fine-tuning). A key finding is that supervised fine-tuning (SFT) makes unlearning popular facts more stable and reliable than using a raw pretrained model.