Removing Information Content Does Not Certify Tamper Resistance in Open-Weight Models
cs.LG, cs.CR
Submitted: 2026-10-06
Updated: 2026-10-06
Comments: Under submission AISTATS 2026
License: http://creativecommons.org/licenses/by/4.0/
The gist: Does removing harmful information make open-weight models resistant to fine-tuning attacks? We show that mutual information at release alone cannot universally certify slow recovery.
Abstract
Does removing harmful information make open-weight models resistant to fine-tuning attacks? We show that mutual information at release alone cannot universally certify slow recovery. Function-preserving reparameterizations leave information unchanged while altering gradient-descent geometry, so an invariant certificate is bounded by the fastest reachable parameterization. We apply this principle to weight--data mutual information under training-data filtering and label--representation mutual information under capability removal. Training order can change recovery time at fixed weight--data information, while exact representation-level independence can preserve the entire parameter Jacobian. An explicit construction has both information quantities equal to zero and recovers in one gradient step. Controlled experiments illustrate order-dependent recovery and parameterization-dependent attack speed. These results identify the missing requirement for certification: constraints on attack dynamics beyond mutual information at release.
Sources
- Open Problems in Machine Unlearning for AI Safety
- Open Technical Problems in Open-Weight AI Model Risk Management
- Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities
- Understanding Machine Unlearning Through the Lens of Mode Connectivity
- Do Unlearning Methods Remove Information from Language Model Weights?
- The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
- An Adversarial Perspective on Machine Unlearning for AI Safety
- Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!
- Distribution-Specific Curvature Control with Finite-Sample Guarantees for Open-Weight Safety
- Limits of Convergence-Rate Control for Open-Weight Safety
- The information bottleneck method
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Symmetry in Neural Network Parameter Spaces
- A Few Steps Further: Why Defenses Against Malicious Finetuning Erode Under Continued Training
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks