Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation
The paper introduces Policy Alignment Membership Auditing (PAMA) for on-policy distillation, a method to detect if a prompt was used during training. It uses Teacher Alignment Gain (TAG) and other signals to measure how much the student model's update follows the teacher's guidance when processing specific prompts. This provides a robust way to audit data usage without needing access to training gradients.