Estimating Model-Level Membership Inference Vulnerability Without Reference Models
The paper proposes estimating AI model vulnerability to membership inference attacks using only training and testing loss distributions, avoiding expensive reference models. It finds that samples from the high-loss region of the training data, which shift to low-loss regions during training, are key indicators of risk. A simple metric, the True Negative Rate (TNR) of a basic loss attack, accurately predicts the performance of state-of-the-art attacks.