An Information Bottleneck for Post-hoc Memorization Mitigation in LLMs
cs.LG
Submitted: 2026-11-05
Updated: 2026-11-05
Comments: 13 pages, 4 figures, including appendix
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent studies have shown that large language models (LLMs) can memorize and reproduce copyrighted training data, raising serious privacy and intellectual property concerns.
Terminology
Abstract
Recent studies have shown that large language models (LLMs) can memorize and reproduce copyrighted training data, raising serious privacy and intellectual property concerns. In this work, we investigate whether memorization can be mitigated without modifying or retraining the underlying model through an information bottleneck. We propose a modular plug-and-play framework that inserts a nonparametric variational information bottleneck (NVIB) between a frozen LLM backbone and its language modeling head. By compressing the final hidden representations before token prediction, the proposed bottleneck aims to preserve general linguistic knowledge while filtering out highly specific memorized sequences. We evaluate our approach on memorization extraction tasks involving book passages and on standard language modeling benchmarks to measure the trade-off between memorization reduction and downstream utility. Our experiments show that the NVIB layer substantially reduces extractability while maintaining competitive performance on benchmark tasks. These results suggest that adding an information bottleneck offers a practical and architecture agnostic direction for post-hoc memorization mitigation in pretrained LLMs.
Sources
- Extracting books from production language models
- Differential Privacy for Transformer Embeddings of Text with Nonparametric Variational Information Bottleneck
- Measuring Massive Multitask Language Understanding
- Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework
- GPQA: A Graduate-Level Google-Proof Q&A Benchmark
- MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
- Copyright Protection for Large Language Models: A Survey of Methods, Challenges, and Trends
- Instruction-Following Evaluation for Large Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis