Leveraging Soft Prompts for Privacy Attacks in Federated Prompt Tuning
cs.LG, cs.CR
Submitted: 2026-01-10
Updated: 2026-09-27
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Membership inference attacks (MIAs) pose a serious privacy threat in federated learning (FL).
Terminology
Abstract
Membership inference attacks (MIAs) pose a serious privacy threat in federated learning (FL). While MIAs have been extensively studied in standard FL, the recent shift toward federated fine-tuning introduces new and largely unexplored attack surfaces. In this work, we show that federated prompt-tuning, which adapts pre-trained foundation models using lightweight input prefixes, exposes a novel and effective vector for membership inference. We propose PromptMIA, a membership inference attack tailored to federated prompt-tuning, in which a malicious server introduces adversarially crafted prompts and exploits their updates during collaborative training to determine whether a target data point belongs to a client's private dataset. We formalize this threat via a security game and demonstrate that PromptMIA achieves consistently high attack advantage across diverse benchmark datasets, substantially outperforming current SOTA federated MIAs. We also provide a theoretical lower bound on the attack advantage that explains the observed empirical behavior. Finally, we show that existing MIA defenses are often ineffective against PromptMIA, highlighting the need for defense mechanisms specifically tailored to prompt-tuning in federated settings.
Sources
- FedAdapter: Efficient Federated Learning for Modern NLP
- CINIC-10 is not ImageNet or CIFAR-10
- Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models
- DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer
- The Power of Scale for Parameter-Efficient Prompt Tuning
- Active Membership Inference Attack under Local Differential Privacy in Federated Learning
- FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuning
- Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation
- Privacy-Preserving In-Context Learning for Large Language Models
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks