Position: Privacy Is a Claim, Not a Property of Synthetic Data
cs.LG, cs.CR
Submitted: 2026-09-01
Updated: 2026-09-01
Journal ref: Proceedings of the 43rd International Conference on Machine Learning (ICML 2026), 2026
License: http://creativecommons.org/licenses/by/4.0/
The gist: Synthetic data has become a common component of machine learning research.
Terminology
Abstract
Synthetic data has become a common component of machine learning research. While widely adopted, its use in privacy-sensitive contexts has quietly shifted from a claim of residual inference risk under stated assumptions to an appearance-based property inferred from data generation itself. In this position paper, we argue that this shift reflects an implicit change in community standards for what counts as sufficient privacy evidence, rather than a misunderstanding of well-established privacy principles. Drawing on an empirical analysis of recent publications across major ML venues, we show that synthetic data is frequently used in privacy-sensitive settings without explicit articulation of threat models, inference risks, or falsifiable privacy claims. As a result, privacy assurance often remains implicit, difficult to verify, and unevenly distributed, with heightened exposure for rare and minority records. We argue for treating privacy as an explicit, evidence-based scientific claim and recommend that ML venues adopt norms requiring privacy-relevant assertions to be clearly scoped, testable, and contestable.
Sources
- Membership Inference over Diffusion-models-based Synthetic Tabular Data
- TAPAS: a Toolbox for Adversarial Privacy Auditing of Synthetic Data
- The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text
- Benchmarking Synthetic Tabular Data: A Multi-Dimensional Evaluation Framework
- Synth-MIA: A Testbed for Auditing Privacy Leakage in Tabular Data Synthesis
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks