Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearning
cs.LG
Submitted: 2025-10-01
Updated: 2026-04-18
Code: https://github.com/OPTML-Group/Unlearn_Optimizer
Terminology
Sources
- GPT-4 Technical Report
- Invariant Risk Minimization
- Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities
- DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Do Unlearning Methods Remove Information from Language Model Weights?
- GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection
- Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning
- Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond
- Sharpness-Aware Minimization for Efficiently Improving Generalization
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- Measuring Massive Multitask Language Understanding
- Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning
- TrustLLM: Trustworthiness in Large Language Models
- Adam: A Method for Stochastic Optimization
- Proactive Privacy Amnesia for Large Language Models: Safeguarding PII with Negligible Impact on Model Utility
- Protecting Privacy Through Approximating Optimal Parameters for Sequence Unlearning in Language Models
- The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
- TruthfulQA: Measuring How Models Mimic Human Falsehoods
- Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis