Confess What You Know: Forget-Set Misalignment with Model Knowledge in LLM Unlearning
The episode discusses the paper "Confess What You Know," which addresses misalignment in Large Language Model (LLM) unlearning. The researchers identify two failure modes—Under Unlearning (omitting key facts) and Out-of-Knowledge Unlearning (including irrelevant information). They propose the CONFS framework, a systematic approach using iterative probing and gradient methods to ensure precise, verifiable memory management.