Do LLMs Really Forget? Hidden-State Leakage in Model Unlearning and How to Fix it
cs.LG
Submitted: 2026-09-29
Updated: 2026-09-29
Code: https://github.com/OptimAI-Lab/HiddenStateUnlearning
Sources
- Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
- Understanding intermediate layers using linear classifier probes
- Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate
- SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders
- Do Unlearning Methods Remove Information from Language Model Weights?
- MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering
- OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics
- Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond
- Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning
- Applying sparse autoencoders to unlearn knowledge in language models
- Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning
- Copyright Violations and Large Language Models
- The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
- LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Construction
- An Adversarial Perspective on Machine Unlearning for AI Safety
- Eight Methods to Evaluate Robust Unlearning in LLMs
- Towards Deep Learning Models Resistant to Adversarial Attacks
- Scalable Extraction of Training Data from (Production) Language Models
- Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks
- BLUR: A Bi-Level Optimization Approach for LLM Unlearning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks