Sequential Membership Inference Attacks
cs.LG, cs.CR, math.ST, stat.ML, stat.TH
Submitted: 2026-02-18
Updated: 2026-11-12
Comments: Accepted at NeurIPS 2026. Camera-ready version. Code at https://github.com/Thomick/pysemi
License: http://creativecommons.org/licenses/by/4.0/
The gist: Modern AI models are not static.
Terminology
Abstract
Modern AI models are not static. They go through multiple updates in their lifecycles. We propose to design Sequential Membership Inference (SeMI) attacks leading to tighter privacy audits by exploiting the sequence of models and injecting a target canary at a controlled insertion time. First, for empirical mean computation, we develop SeMI*, an optimal SeMI attack to identify the presence of a target inserted at a specific insertion step. We derive the power of SeMI* to show that accessing the model sequence yields more powerful MI attacks than scrutinising only the final model. SeMI* exhibits an isolation property. Its power depends on the statistics obtained right before and after insertion of the target. Leveraging this insight, we develop practical white-box (accessing model gradients) and black-box (accessing loss) SeMI attacks against models trained with (DP-)SGD. Across datasets and models trained with (DP-)SGD, our experiments show that SeMI attacks achieve higher power than snapshot-independent baselines, and yield tighter privacy audits thanks to (a) control over the insertion time and (b) observations across the model sequence. Code is available at https://github.com/Thomick/pysemi.
Sources
- When Machine Unlearning Jeopardizes Privacy
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Debugging Differential Privacy: A Case Study for Privacy Auditing
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Related papers
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis
- (Mis)generalization of Helpful-Only Fine-tuning