Shortcuts or Memorisation? Revisiting Data Corruption in Natural Language Understanding with Large Language Models

arXiv:2611.16843 · cs.CL · Submitted 2026-11-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Shortcuts or Memorisation? Revisiting Data Corruption in Natural Language Understanding with Large Language Models".

Jane: Here's the gist of the paper. Pre-trained encoders fine-tuned on natural language inference (NLI) and other language understanding tasks keep much of their accuracy after the input is perturbed, e.g.

Tom: by removing all nouns or shuffling the words The main contributions of this paper are as follows.

Jane: Robustness to Perturbation and Benchmark Differences Regardless of model size, every system that answers directly retains 36–72% of its above-chance performance without nouns on MNLI compared to only 14–29% on ANLI Only a model that reasons before answering departs from this pattern.

Tom: The paper demonstrates that the gap between MNLI and ANLI lies in the dataset, not the model Robustness against perturbation alone cannot prove language understanding or memorisation What it reveals depends on which cues remain intact even after perturbation is applied.

Paper summary: Tom: So we just talked about how these models handle losing nouns, and the paper "Shortcuts or Memorisation? Revisiting Data Corruption in Natural Language Understanding with Large Language Models" claims that even when you strip out nouns, most systems keep a lot of their accuracy on MNLI >

Jane: They found that every system that answers directly retains thirty six to seventy two percent of its above chance performance without nouns on MNLI compared to only fourteen to twenty nine percent on ANLI >

Lu: The paper also points out that the gap between those two benchmarks is in the dataset not necessarily the model itself, regardless of how large or small we make the LLM >

Meng: They also checked some contamination indicators—like exact n-gram counts in the pre-training data—and found no association with robustness >

Tom: That means they aren't seeing a simple link between how much data corruption a model sees and how robust it becomes against losing parts of the input >

Jane: Instead, they focus on items solvable solely from the hypothesis, which they found to be more robust in all three of the most robust LLMs on MNLI when compared to when function words are removed >

Lu: They built three hundred sixty eight new human-validated NLI items and found that all eight LLMs show the same shortcut pattern across these unseen items >

Tom: That means they aren't seeing memorization as the main driver here, but something else is at play in how those models work >

Jane: They compare MNLI with BioASQ, which is also robust to perturbation, but their results suggest that the structure of the data matters more than just having a large model >

Conclusion: Tom: So when we look at this paper, "Shortcuts or Memorisation? Revisiting Data Corruption in Natural Language Understanding with Large Language Models," they are suggesting that the survival of familiar NLI benchmarks isn't because the models memorized specific answers >

Jane: They argue instead that certain structural shortcuts in how the data is presented allow these models to maintain performance even when you corrupt the input, like removing all nouns >

Lu: The key evidence they offer is that for the most robust LLMs on MNLI, items that can be solved just from the initial hypothesis are more robust than items where you remove function words >

Meng: What this means for us in production is that we shouldn't just look at memorization as a failure mode when models get weird input; we should also be looking at what structural assumptions they make about the language structure itself >

Tom: And they show that these shortcut patterns are consistent across all eight models even on brand new items they haven't seen, which is pretty strong evidence for this idea >

Jane: It implies that understanding the underlying structure of natural language is a more stable thing than remembering every specific sentence in the training set >

University of Helsinki · Institute of Language and Speech Processing Athena Research Centre Athens

cs.CL

Submitted: 2026-11-12

Updated: 2026-11-19

Code: https://github.com/aarnetalman/shortcuts-or-memorisation

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 89/100

The gist: Pre-trained encoders fine-tuned on natural language inference (NLI) and other language understanding tasks keep much of their accuracy after the input is perturbed, e.g.

Key concepts

Robustness to Perturbation
This tests how well a model's answer holds up when parts of the input text are deliberately changed or removed, like taking out all nouns. The paper found that many models keep most of their accuracy even with these changes, indicating they rely on more than just exact memorization.
Contamination Indicators
These are specific metrics used to check for potential data corruption or shortcuts within the model's responses. The research found no significant link between these indicators and overall robustness, suggesting that the presence of these signs doesn't reliably predict how well a model handles noisy input.
Hypothesis-Only Analysis
This method involves testing if an item can be correctly classified using only the initial guess or hypothesis generated by the model, without needing further complex reasoning. The paper showed that for certain challenging items, this direct approach reveals patterns related to memorization versus shortcuts.
MNLI vs. ANLI Gap
This refers to the difference in performance between two specific natural language inference benchmarks. The research concluded that this performance gap is caused by the differences in the datasets used for each benchmark, not necessarily a fundamental limitation of how well a model understands language generally.

Terminology

Summary

Pre-trained encoders fine-tuned on natural language inference (NLI) and other language understanding tasks keep much of their accuracy after the input is perturbed, e. p. 0

Where the paper says it

chatzikyriakidis@athenarc. gr Abstract Pre-trained encoders fine-tuned on natural language inference (NLI) and other language understanding tasks keep much of their accuracy after the input is perturbed, e.

g. by removing all nouns or shuffling the words The main contributions of this paper are as follows. p. 1
Where the paper says it

The main contributions of this paper are as follows: • We demonstrate that the gap between MNLI and ANLI lies in the dataset, not the model. Regardless of model size, all systems that answer directly keep much more of their performance on MNLI than on ANLI when nouns are removed (36–72% vs.

Robustness to Perturbation and Benchmark Differences

Regardless of model size, every system that answers directly retains 36–72% of its above-chance performance without nouns on MNLI compared to only 14–29% on ANLI Only a model that reasons before answering departs from this pattern. The paper demonstrates that the gap between MNLI and ANLI lies in the dataset, not the model Robustness against perturbation alone cannot prove language understanding or memorisation What it reveals depends on which cues remain intact even after perturbation is applied. p. 1

Where the paper says it

In other words, robustness against perturbation alone cannot prove language understanding or memorisation. What it reveals depends on which cues remain intact even after perturbation is applied.

Contamination Indicators and Shortcut Mechanisms

The paper finds no link between the three contamination indicators and robustness In all three of the most robust LLMs on MNLI, items solvable solely from the hypothesis are more robust, and this advantage is reversed when function words are removed. The strongest evidence comes from 368 new, human-validated items that none of the models can have seen All eight LLMs exhibit the same shortcut pattern in these unseen items. p. 0

Where the paper says it

Our strongest evidence comes from 368 new, human-validated items that none of the models can have seen. All eight LLMs keep more of their performance without nouns on items written in MNLI’s style than on items written without its cues.

Hypothesis-Only Analysis and Fresh Items

The new items enable the most direct test: no model has seen them, but the MNLI-style items have the same type of cues as MNLI If memorisation causes the robustness on MNLI, it should disappear on Clean R no nouns R nouns+verbs Easy− Model MNLI free MNLI free MNLI free hard Table 2 shows the second pattern in all models. Within MNLI-style items, items correctly labelled by the MNLI hypothesis-only classifier show 11–28 points less loss than the remaining items when nouns are absent The new items are easier than MNLI (93–100% clean accuracy). p. 6

Where the paper says it

62–74%), as on MNLI. Within MNLI-style items, items correctly labelled by the MNLI hypothesis-only classifier show 11– 28 points less loss than the remaining items when nouns are absent, and for 7 of the 8 models the 95% interval excludes zero.

Conclusion on Memorisation vs. p. 7

Where the paper says it

We would therefore argue that a benchmark score neither confirms nor refutes understanding, which is better seen as a matter of degree, and that perturbation experiments are most useful for finding out what a given benchmark can and cannot distinguish. 7 Conclusion In this paper we have revisited the effect of input perturbations on NLI and other language understanding tasks with LLMs and fine-tuned encoders.

Shortcuts

Taken together, the results point away from memorisation and towards shortcuts as the reason why familiar NLI benchmarks survive perturbation First, the contrast between MNLI and ANLI is shared by every system that answers directly Second, none of the contamination indicators is related to robustness Third, for the open LLMs most robust on MNLI, items that can be solved from the hypothesis alone are more robust Fourth, and most directly, every LLM shows the same signature on fresh items that it cannot have seen.

Limitations and Future Directions

Our contamination checks are partial The shortcut split depends on one simple classifier; items it labels incorrectly may still contain cues that a stronger classifier would find The hosted models are analysed on the 2,000-item MNLI sample, which gives the comparison less power than for the open models Open questions remain: a larger, multiply annotated fresh set would sharpen the comparison other cues, such as lexical overlap (McCoy et al., 2019), may explain why the hosted models show no hypothesis-only effect on MNLI and it remains to be seen whether reasoning models in general escape the MNLI–ANLI contrast. p. 7

Where the paper says it

Open questions remain: a larger, multiply annotated fresh set would sharpen the comparison; other cues, such as lexical overlap (McCoy et al. , 2019), may explain why the hosted models show no hypothesis-only effect on MNLI; and it remains to be seen whether reasoning models in general escape the MNLI–ANLI contrast.

The paper is 2611. p. 0
Where the paper says it

First, we reproduce previously published encoder results on the MNLI dataset within 1 percentage point, and then re-examine the perturbation experiments of Talman et al. (2021, 2022) using four open-weight 1 arXiv:2611.

p. 0
Where the paper says it

First, we reproduce previously published encoder results on the MNLI dataset within 1 percentage point, and then re-examine the perturbation experiments of Talman et al. (2021, 2022) using four open-weight 1 arXiv:2611.

16843A.

Shortcuts or Memorisation? p. 0

Where the paper says it

Shortcuts or Memorisation? Revisiting Data Corruption in Natural Language Understanding with Large Language Models Aarne Talman University of Helsinki aarne.

Revisiting Data Corruption in Natural Language Understanding with Large Language Models Shortcuts or Memorisation? p. 0
Where the paper says it

Shortcuts or Memorisation? Revisiting Data Corruption in Natural Language Understanding with Large Language Models Aarne Talman University of Helsinki aarne.

Revisiting Data Corruption in Natural Language Understanding with Large Language Models The main contributions of this paper are as follows. p. 0
Where the paper says it

Shortcuts or Memorisation? Revisiting Data Corruption in Natural Language Understanding with Large Language Models Aarne Talman University of Helsinki aarne.

Robustness to perturbation and benchmark differences. p. 5

Where the paper says it

Among all 80 comparisons, 7 have 95% confidence intervals excluding zero, with differences in both directions and an overall average of −0. We therefore find no evidence that exposure to the source text increases robustness.

Contamination indicators and shortcut mechanisms. p. 0

Where the paper says it

Only a model that reasons before answering departs from this pattern. We also check contamination indicators (including exact n-gram counts in the public pre-training data of two of the models) and find no link between exposure and robustness.

Hypothesis-only analysis and fresh items. p. 7

Where the paper says it

We recommend that robustness results on NLI benchmarks be reported together with a hypothesis-only analysis and, where possible, on items written after the models’ training data were collected. Open questions remain: a larger, multiply annotated fresh set would sharpen the comparison; other cues, such as lexical overlap (McCoy et al.

Conclusion on memorisation vs. p. 7

Where the paper says it

We would therefore argue that a benchmark score neither confirms nor refutes understanding, which is better seen as a matter of degree, and that perturbation experiments are most useful for finding out what a given benchmark can and cannot distinguish. 7 Conclusion In this paper we have revisited the effect of input perturbations on NLI and other language understanding tasks with LLMs and fine-tuned encoders.

shortcuts.

Limitations and future directions. p. 8

Where the paper says it

Limitations Our contamination checks are partial. The n-gram counts cover the OLMo 2 pre-training mix rather than that of OLMo 3, sample only two 13-grams per field, and detect only verbatim overlap; paraphrased or reformatted copies of a benchmark, for example in code repositories, would be missed.

The paper is 2611.16843. EVERY sentence must end with a reference token. p. 2

Where the paper says it

3 Models Fine-tuned encoders. As a reference we finetune RoBERTa-base (Liu et al.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Robustness analysis should incorporate hypothesis-only classification to distinguish between memorization and shortcuts, as robustness to perturbation on familiar NLI benchmarks reflects dataset shortcuts rather than memorisation. This allows systems to better understand that a high score on MNLI might stem from superficial cues rather than deep language understanding of the sentence pair.

  2. Develop methods for assessing how different cues survive perturbations, as robustness against perturbation alone cannot prove language understanding or memorisation. What it reveals depends on which cues remain intact even after perturbation is applied. This enables the creation of benchmarks that specifically test the necessity of different linguistic elements (like function words) for correct inference.

  3. Implement a mechanism to detect answer bias in open-weight LLMs, as Perturbed input not only removes information but also biases LLMs toward a single answer. This allows systems to be evaluated on centred accuracy rather than raw accuracy, which conflates two things: losing the information needed for the task, and shifting towards one answer.

  4. Utilize fresh, unseen items written in MNLI style as a critical test for generalization, as on 368 new, human-validated items that none of the models can have seen all eight LLMs exhibit the same shortcut pattern in these unseen items. This provides a direct measure of whether performance is due to memorization or learned patterns from familiar data.

  5. Prioritize training on ablated data for encoders when seeking robustness, as for encoders, training on ablated data recovers most of the loss (Talman et al., 2021, 2022). This demonstrates that fine-tuning can mitigate the effects of data corruption by forcing the model to rely on more fundamental linguistic structures.

Abstract

Pre-trained encoders fine-tuned on natural language inference (NLI) and other language understanding tasks keep much of their accuracy after the input is perturbed, e.g. by removing all nouns or shuffling the words. Does the same hold for large language models (LLMs), and if so, is it because they have seen the benchmarks during pre-training? In this paper we revisit these perturbation experiments with eight LLMs alongside fine-tuned encoders on GLUE and the three rounds of ANLI. Regardless of model size, every system that answers directly retains 36-72% of its above-chance performance without nouns on MNLI compared to only 14-29% on ANLI. Only a model that reasons before answering departs from this pattern. We also check contamination indicators (including exact n-gram counts in the public pre-training data of two of the models) and find no link between exposure and robustness. Our strongest evidence comes from 368 new, human-validated items that none of the models can have seen. All eight LLMs keep more of their performance without nouns on items written in MNLI's style than on items written without its cues. When the perturbation instead removes the function words carrying such cues, this advantage reverses. In light of these results, we argue that robustness to perturbation on familiar NLI benchmarks reflects dataset shortcuts rather than memorisation, and tells us more about the benchmark than about the model.

Sources

Related papers