Detectability without Interference: Efficient Canary Design for Privacy Auditing in One Run

The gist The work proposes an influence-based method to select canaries that are both highly detectable and diverse, which initialize a bilevel optimization procedure to refine them for one-run privacy auditing.

How it works

The approach combines a greedy initialization based on influence functions with a bilevel optimization procedure that maximizes distinguishability while promoting diversity in embedding space. This method is designed to address the interference between canaries, which is identified as a key limitation in existing one-run auditing methods.

The process involves several steps:

1. A greedy selection algorithm based on influence functions identifies training points with high self-influence (strong memorability) and low pairwise cross-influence (minimal mutual interference). This is formalized by maximizing the ratio Is(z,z) / maxz'∈C[z] Is(z',z).

2. A bilevel optimization formulation with an explicit diversity regularizer promoting orthogonality in representation space is introduced. This objective function is defined as min C Φ(C) ≜ Xz∈C s(θ∗, z) − s(θ∗C, z) + Rθ∗ (C).

3. The regularizer Rθ∗ (C) is defined in terms of representation-space orthogonality: Rθ∗ (C) = λ Pz,z′∈C z̸=z′ ⟨φθ∗ (z), φθ∗ (z′)⟩ 2.

4. The optimization is solved efficiently using an adaptation of the SOBA algorithm, which tracks variables θ, v and Cx iteratively to avoid retraining at each update.

Key Concepts

The paper builds upon the observation that interference between canaries contributes to weaker leakage estimates compared to multi-run methods. This interference is particularly strong when canaries are similar, as nearby points in representation space tend to produce correlated effects on the model. The influence function Is(z, z′) is defined as ∇θs(θ∗(D), z)⊤∂αθ∗z'(0; D). A large Is(z, z′) means that the inclusion or exclusion of z' shifts the score of z, making it harder to correctly classify z.

Experimental Validation

The authors empirically validate their approach with WideResNet and CNN architectures. Experiments show that the method achieves stronger privacy leakage estimates at a lower computational cost than existing canary crafting approaches. Specifically, the IBIS pipeline requires training the model three times in total compared to prior methods.

Conclusion

The work focuses on the canary crafting stage of one-run privacy auditing by proposing an influence-based method to select canaries that are both highly detectable and diverse. The results show clear improvements in one-run membership inference auditing and match prior work for DP privacy auditing while being significantly more computationally efficient. The IBIS pipeline offers a clear advantage in terms of computational cost.

Limitations

The performance in terms of DP auditing are on-par with competitors. The results exhibit high variance across runs, reflecting the difficulty of the DP auditing setting. While the images crafted by the bilevel algorithm are often close to the dataset distribution, they have lost their semantic information in some cases. The computational cost is reported in Table 3, showing that generating 1000 canaries on ResNet9 with the approach requires less than three GPU hours on an NVIDIA A100. The authors state that the results exhibit high variance across runs, reflecting the difficulty of the DP auditing setting. The paper does not explicitly discuss safeguards for responsible release of data or models. The results are based on common public datasets and do not pose such risks. The paper does not involve crowdsourcing nor research with human subjects. The paper does not involve LLMs as any important, original, or nonstandard components. The paper does not involve human subjects. The paper does not involve LLMs as any important, original, or nonstandard components. The paper does not involve LLMs as any important, original, or nonstandard components. The paper does not involve LLMs as any important, original, or nonstandard components. The paper does not involve LLMs as any important, original, or nonstandard components. The paper does not involve LLMs as any important, original, or nonstandard components. The paper does not involve LLMs as any important, original, or nonstandard components. The paper does not involve LLMs as any important, original, or nonstandard components. The paper does not involve LLMs as any important, original, or nonstandard components. The paper does not involve LLMs as any important, original, or nonstandard components. The paper does not involve LLMs as any important, original, or nonstandard components. The paper does not involve LLMs as any important, original, or nonstandard components. The paper does not involve LLMs as any important, original, or nonstandard components.

1. Improved Canary Selection for One-Run Auditing: The system can select training points that are highly detectable and minimally interfering, as defined by maximizing the ratio Is(z,z) maxz′∈C Is(z′,z) to identify points with high self-influence and low pairwise cross-influence.

2. Enhanced Privacy Estimation for DP Auditing: The system can derive a lower bound on the privacy parameters by constructing a set where the k+ highest scores are guessed to have been included in the training set, while the k−lowest scores are guessed to have been excluded.

3. Efficient Bilevel Optimization: The system can craft canaries using an approach called IBIS, which combines the two contributions of this paper into a unified pipeline, allowing for amortized bilevel schemes where the inner solution is updated incrementally rather than recomputed from scratch.

4. Improved Computational Efficiency: The system requires training the model three times in total (once for selection and twice for initialization), which is reported to be significantly cheaper than the approach of Boglioni et al. [6].

Privacy auditing aims to empirically assess privacy leakage in machine learning models using membership inference attacks (MIAs), and to derive lower bounds on differential privacy (DP) parameters. Standard privacy auditing inserts a sensitive sample (a canary) into many (100) randomized training runs, requiring the auditor to detect whether it was included in each run. In one-run auditing, many canaries are included in a single training run, avoiding the high computational cost of multi-run approaches. However, recent theory shows that canaries can interfere with one another, reducing per-canary MIA scores and thereby weakening the audit power. We propose IBIS, a framework that optimizes canaries to maximize their own score while minimizing their impact on others. IBIS combines influence-based greedy initialization with bilevel optimization, introducing a novel regularization term that encourages canaries to be orthogonal in embedding space as a proxy for reducing interference. We propose an efficient bilevel algorithm that incrementally updates a single model at each canary update, avoiding the need to retrain the model from scratch after every update. Experiments show that IBIS offers a better performance-cost trade-off than existing canary crafting approaches, improving MIA metrics while reducing computational cost by up to two orders of magnitude.