Optimal Rates for Adaptive Private k-PCA
cs.LG, stat.ML
Submitted: 2026-11-09
Updated: 2026-11-09
License: http://creativecommons.org/licenses/by/4.0/
The gist: Given n i.i.d.
Terminology
Abstract
Given n i.i.d. random matrices A i in R d times d with common expectation Σ, the goal of Differentially Private Stochastic PCA is to identify a k-dimensional subspace capturing the leading variance directions of Σ, while preserving differential privacy (DP) for each individual sample A i. introduced, the first algorithm to simultaneously (1) achieve sample complexity n= O(d) for sub-Gaussian data, (2) adapt its privacy noise to the intrinsic randomness of the data, and (3) extend seamlessly to any target dimension k at most d. However, its sample complexity has a suboptimal dependence on k. We propose the first algorithm that achieves optimal sample complexity in both d and k, while retaining properties (2) and (3) of. Additionally, prior utility guarantees require a minimum sample size with exponential dependence on the eigengap. Our method removes this dependence. It also improves the dependence on spectral parameters, matching known lower bounds for the spiked covariance model. Unlike deflation-based approaches like, our algorithm updates the full d times k subspace jointly rather than one eigenvector at a time. This non-deflation structure simplifies the algorithm, reduces the number of hyper-parameters, and improves computational efficiency, particularly when using block linear algebra libraries.
Sources
- Optimal Differentially Private PCA and Estimation for Spiked Covariance Matrices
- Decentralized Differentially Private Power Method
- Differentially private and decentralized randomized power method
- Spectral Perturbation Bounds for Low-Rank Approximation with Applications to Privacy
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis