Quantifying the Effect of Test Set Contamination on Generative Evaluations
Researchers studied how including test data in a model's training set (contamination) affects its performance and reliability. They found that contamination artificially boosts scores through fragile memorization, but this gain does not reflect real mathematical reasoning or generalization capabilities.