Unlearning at Scale: State-Exact Trace-Preserving Deletion in Billion-Parameter Language Models
The hosts discuss a paper detailing 'state-exact trace-preserving deletion' in large language models. The authors propose a system that allows for bit-for-bit identical removal of data from training sets, proving it works on billion-parameter models like Llama. The discussion focuses on the technical requirements, limitations (like high cost for scattered deletions), and the paper' providing a verifiable standard for data deletion.