Unmasking the Secret Code: Using Canaries to Show How Masked Context Controls Memorization in SFT
Researchers used synthetic codes as 'canaries' in fine-tuning models to see how context presentation affects memorization risk. Findings show that verbatim secrets are highly predictive, while encoded or absent secrets can still trigger memorization if the model must decode them from surrounding information. This requires auditors to test multiple contexts and timing settings.