Mitigating Memorization In Language Models
The study investigated three memorization mitigation strategies: regularization, fine-tuning, and machine unlearning. Using small models called TinyMem, researchers found that unlearning methods are the fastest and most effective way to remove memorized training data from large language models without significantly hurting performance.