Efficient Provably Private Classification with a Tabular Foundation Model
cs.LG, cs.CR
Submitted: 2026-10-07
Updated: 2026-10-07
Comments: 74 pages, 18 figures; includes supplementary information
Code: https://github.com/TrustworthyMLHelsinki/PrivTab
License: http://creativecommons.org/licenses/by/4.0/
The gist: Tabular data underpin prediction and decision-making in medicine, finance, government and science, but often contain sensitive individual-level information, creating a need for accurate prediction
Terminology
Abstract
Tabular data underpin prediction and decision-making in medicine, finance, government and science, but often contain sensitive individual-level information, creating a need for accurate prediction while preserving privacy. Traditional private learning provides formal privacy guarantees, but requires slow dataset-specific optimisation, suffers substantial utility loss under strong privacy, and is often difficult to apply correctly. Tabular foundation models adapt rapidly to new datasets, but existing models lack formal privacy guarantees, and are highly vulnerable to membership-inference attacks, limiting their use on sensitive data. Here we introduce PrivTab, an easy to use tabular foundation model for differentially private classification that embeds a privacy mechanism within its architecture. Pretrained on simulated datasets, PrivTab uses in-context learning to transform sensitive rows into compact, provably private summaries---effectively learning how to learn under privacy. PrivTab outperforms private linear and neural-network baselines under moderate-to-strong privacy, shows negligible membership leakage, maintains well-calibrated predictions under strong privacy, and reduces dataset fitting time by 10,000 times, requiring only a single forward pass. By combining formal privacy, speed, and easy of use, PrivTab brings recent advances in AI to applications where sensitive individual-level data have limited their adoption.
Sources
- Debugging Differential Privacy: A Case Study for Privacy Auditing
- Neural Processes
- On Reliability of Membership Inference Vulnerability Evaluation
- TabPFN-3: Technical Report
- On First-Order Meta-Learning Algorithms
- Differentially Private In-Context Learning with Nearest Neighbor Search
- Differential Privacy for Transformer Embeddings of Text with Nonparametric Variational Information Bottleneck
- Layer Normalization
- GLU Variants Improve Transformer
- Gaussian Error Linear Units (GELUs)
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks