MERGE: Minimal Expression-Replacement GEneralization Test for Natural Language Inference
MERGE tests how well reasoning models generalize to minimal changes in Natural Language Inference (NLI) datasets. The method automatically creates slightly altered versions of original sentences using contextually probable replacements. Results show that strong models struggle to maintain accuracy when variants are minimally different, failing to generalize consistently beyond a 50% correctness threshold.