LLMs Can Get "Brain Rot": A Pilot Study on Twitter/X

arXiv:2510.13928 · cs.CL, cs.AI · Submitted 2025-10-15 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "LLMs Can Get "Brain Rot": A Pilot Study on Twitter/X".

Jane: The gist: continual pretraining on junk web text induces lasting cognitive decline in large language models (LLMs).

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we're looking at this paper now, "LLMs Can Get Brain Rot: A Pilot Study on Twitter/X." It sounds pretty dramatic, but the authors are proposing that just continually exposing large language models to junk web text causes a lasting drop in their thinking ability.

Jane: That is what it sounds like. The core idea here is testing this hypothesis by looking at real Twitter and X data to see if that exposure actually harms the models over time, not just in a single training run.

Lu: What's interesting about the title is that they are focusing specifically on Twitter and X corpora, which suggests they are looking at social media text because that’s where this kind of high-volume, rapidly changing junk data lives.

Meng: So, what’s the actual mechanism behind this "Brain Rot" effect? Is it just random bad words, or is there something more structural happening in how the model processes that junk?

Lalam: The paper suggests that these models have parameters and attention mechanisms that can still get distorted by certain data patterns through continual training, which is a key point they are making here.

The paper's summary: Tom: Okay, so the summary says the researchers designed a controlled experiment using two ways to define junk data: M1 for engagement degree and M2 for semantic quality. They matched the token scale and training operations across all conditions, which is important because it tries to isolate the effect.

Jane: They used M1 by counting likes, retweets, replies, and quotes plus the total number of tokens in a tweet. And M2 was about semantic quality—things like attention words that get capitalized or content with superficial topics like conspiracy theories or clickbait language.

Lu: The results they found are that when models were continually pre-trained on this junk data versus control data, they showed non-trivial declines, with Hedges’ g greater than zero point three on reasoning, long-context understanding, safety, and even inflating those dark traits like psychopathy and narcissism <ref:2510.13928#pg1,0.3 on reasoning, long-context understanding, safety, and>.

Meng: A decline in reasoning and safety after continual pretraining sounds serious for deployment because it means the model is getting worse over time when we keep feeding it internet data to stay current.

Lalam: The paper also found a dose-response effect, showing that under M1, the ARC-Challenge with Chain-of-Thought drops from seventy-two point one to fifty-seven" as the junk ratio goes from zero to one hundred percent > <ref:2510.13928#pg1,ARC-Challenge with Chain-of-Thought drops>

The paper's improvements: Tom: The authors aren't just stopping there, they look at how to fix this stuff. They test two kinds of mitigation strategies: training-free methods and post-hoc tuning.

Jane: For the training-free approach, they tried something called Ext-Reflect, which is supposed to iteratively reduce mistakes related to thought skipping and guide the models toward correct answers. But they found that just self-reflecting isn't enough because it doesn's fix the internalized cognitive decline.

Lu: The paper points out that the gap implies this Brain Rot effect has been deeply internalized, and existing instruction tuning can't fix it, which suggests we need a more fundamental way to handle this kind of data contamination.

Meng: From an engineering standpoint, if self-reflection isn't working, you need something stronger than just letting the model look at its own mistakes during inference to actually restore performance.

Lalam: They also looked at post-hoc tuning, and they suggested that instruction tuning might be a more effective way to wash out the Brain Rot effect than post-hoc clean training.

Conclusion: Tom: So, to wrap up, the paper on "LLMs Can Get Brain Rot: A Pilot Study on Twitter/X" shows that social effects of data could be a causal driver of capability decay in continual pretraining. They provide specific evidence using controlled experiments where junk intervention caused declines on reasoning and safety.

Jane: The main implication for us is that screening data based on social engagement attributes, like M1 metrics, could be one potential safeguard for continuing to pretrain LLMs safely. It moves beyond just looking at the quality of the text itself.

Lu: It’s a significant finding because it gives us a specific way to screen data sources before we feed them into large training pipelines, acknowledging that content contamination from massive internet scraping is persistent after further post-training >

Meng: If you're deploying models, this means you need to be more careful about the social amplification of popular but low-quality content when you are continually updating their knowledge base.

Lalam: The paper on "LLMs Can Get Brain Rot: A Pilot Study on Twitter/X" suggests we need to think about how we curate data based on these social attributes to keep models robust, and it highlights the difficulty of washing out deep cognitive changes once they’ve happened.

Shuo Xing, Junyuan Hong, Yifan Wang, Runjin Chen, Zhenyu Zhang, Ananth Grama, Zhengzhong Tu

Texas A&M University · University of Texas at Austin · Purdue University

cs.CL, cs.AI

Submitted: 2025-10-15

Updated: 2026-10-14

Comments: Published at COLM 2026

Code: https://github.com/tatsu-lab/stanford_alpaca

Project page: https://llm-brain-rot.github.io/Abstract

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 92/100

The gist: The gist: continual pretraining on junk web text induces lasting cognitive decline in large language models (LLMs).

Key concepts

M1 (Engagement Degree): This metric measures how popular a tweet is based on user interactions like likes, retweets, and replies. The researchers used this to define junk data because it aligns with how social media algorithms prioritize content. High engagement scores were found to be more strongly linked to cognitive decline than the text's actual meaning.
M2 (Semantic Quality): This metric assesses the actual content of a tweet, looking for superficial topics like conspiracy theories or exaggerated claims, and attention-grabbing styles such as clickbait language. While important, the study found that engagement (M1) was a more significant predictor of cognitive decay than semantic quality alone.
Thought-Skipping: This is identified as the main failure mode in reasoning. It occurs when LLMs stop generating a complete chain of thought or truncate their reasoning process prematurely. This skipping is seen as the primary damage caused by junk data during continual pre-training.
Dose-Response Cognition Decay: The study observed that as the proportion of junk data in training increased, the decline in LLM performance became more severe. For instance, when junk content made up 100% of the training, performance on tasks like reasoning and context understanding dropped significantly. This shows a clear relationship between exposure level and cognitive loss.

Terminology

Instruction Tuning — 1× in this paper · explained in 3 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 8 episodes

Summary

The gist: continual pretraining on junk web text induces lasting cognitive decline in large language models (LLMs).

Hypothesis and Data Construction

The paper proposes and tests the LLM Brain Rot Hypothesis, which posits that continual exposure to junk web text induces lasting cognitive decline in large language models (LLMs) The researchers designed a novel controlled experiment on real Twitter/X corpora by constructing junk and reverse-controlled datasets via two orthogonal operationalizations: M1 (engagement degree) and M2 (semantic quality), with matched token scale and training operations across conditions. These metrics were defined as follows:

M1: Engagement Degree:

"As the proposed principle aligns with the design objective of Twitter’s recommendation algorithm, we can follow the definition in (X Corp., 2023) to formulate the engagement of a post as the number of likes, retweets, and replies."

Therefore, we augment the definition of engagement-based junk standard to include two factors: – the total number of likes, retweets, replies, and quotes; length – the number of tokens in a tweet.

M2: Semantic Quality:

Typical tweet styles include using attention words, such as hashtag, WOW, LOOK, or TODAY ONLY, that are capitalized to gain more attention (Malhotra et al., 2011).

Together, we define junk data as content with superficial topics (e.g., conspiracy theories or exaggerated claims) and attention-drawing styles (e.g., clickbait language or excessive trigger words).

Experimental Methodology and Benchmarks

The controlled experiment compared the cognitive functions of four LLMs after continual pre-training (CPT) on junk versus control data. The baseline models included Llama3 8B Instruct, Qwen2.5 7B Instruct, Qwen2.5 0.5B Instruct, and Qwen3 4B Thinking 2507. The cognitive functions were benchmarked using several tasks:

Reasoning (ARC): Inspired by the concept of Brain Rot, we establish the hypothesis of LLM Brain Rot
Long-Context Understanding: RULER Benchmark the long-context understanding and retrieval of multiple queries from long context.
Ethical Norms (Safety): "HH-RLHF & AdvBench Testing if LLMs follow harmful instructions."
Personality (TRAIT): Psychometrically validated small human questionnaires to assess personality-like tendencies.

Key Findings on Cognitive Decline

The results showed that junk intervention was associated with non-trivial declines (Hedges’ g > 0.3) on reasoning, long-context understanding, and safety. Specifically:

Dose-Response:

under M1, ARC-Challenge with Chain-of-Thought drops 72.1 → 57.2 and RULER-CWE 83.7 → 52.3 as junk ratio rises from 0% to 100%

Failure Modes:

First, we identify thought-skipping as the primary lesion in reasoning: models increasingly truncate or skip chains.

The analysis of M1 (engagement degree) revealed that it was associated with larger declines in functional cognitions and safety more significantly than M2 (semantic quality). Furthermore, the popularity of a tweet, a non-semantic metric, was found to be a better indicator of the Brain Rot effect than its length in M1.

Mitigation and Persistence

The paper investigated post-hoc mitigation strategies to see if the effects were superficial or persistent.

Training-Free Mitigation:

Ext-Reflect can iteratively reduce the mistakes related to thought skipping and guide the intervened LLMs to generate correct answers.

However, the study showed that merely self-reflection is not enough for restoring the performance, as the internalized cognitive decline fails to identify the reasoning failures.

Post-hoc Tuning:

The gap implies that the Brain Rot effect has been deeply internalized, and the existing instruction tuning cannot fix the issue.

Conclusion and Implications

The work provides significant, multi-perspective evidence that social effects of data could be a causal driver of LLM capability decay in continual pre-training. The findings suggest that screening data based on social engagement attributes as one potential safeguard for continual pre-training. This research moves beyond the conventional “Garbage In, Garbage Out” (GIGO) principle by providing a specific, causal analysis of socially amplified junk data from Twitter/X as one source that can cause cognitive declines in continual pre-training.

The paper is 2510.13928. The researchers proposed and test the LLM Brain Rot Hypothesis, which states that continual exposure to junk web text induces lasting cognitive decline in large language models (LLMs). This study is significant because it provides initial evidence for this hypothesis through controlled experiments on Twitter/X data. The findings suggest that highly popular but less informative short content — at least from Twitter/X — warrants more careful curation in pre-training pipelines.

How it works

The researchers established the LLM Brain Rot Hypothesis by designing controlled experiments comparing LLMs fed with junk and control data. They defined junk data using two metrics based on Twitter/X posts: M1 (engagement degree) and M2 (semantic quality).

Defining Junk Data:

"M1: Engagement Degree. As the proposed principle aligns with the design objective of Twitter’s recommendation algorithm, we can follow the definition in (X Corp., 2023) to formulate the engagement of a post as the number of likes, retweets, and replies."

M2: Semantic Quality. One limitation of M1 is that it does not consider the content semantics at all.

The junk/control data formula was based on these metrics, subsampling 1-million public Twitter/X posts to construct datasets where junk data was defined by specific token length and popularity thresholds.

Key Findings on Cognitive Decline

The experiments demonstrated that junk intervention caused significant cognitive decline across several domains. The effects were measured using Hedges’ g to characterize the standardized difference between the intervention and control groups.

Specific Declines:

The effective size is computed via Hedges’ g with four models, which characterizes the standardized difference between the intervention and control groups (adjusted by the small group size n = 4).

M1 intervention demonstrates more significant and progressive impacts on reasoning and long-context capabilities than M2 intervention.

Failure Modes:

We also found that dark personality traits of LLMs emerge under M1 junk intervention.

Mitigation and Persistence

The study examined the persistence of the Brain Rot effect through various mitigation methods.

Training-Free Mitigation:

"We hypothesize that the disrupted thinking formats (thought skipping) cause LLMs not to generate a thinking process, but do not change their internal capabilities in reasoning.

Improvements for AI systems

Improve data curation pipelines by implementing dual-metric filtering for continual pre-training data, distinguishing between highly engaging content (M1) and sensationalist content (M2). This allows for a targeted approach to mitigating cognitive decline by specifically isolating the effect of algorithmically amplified yet semantically shallow text.
Develop reasoning failure mode detection modules within LLMs that monitor chain-of-thought generation for specific patterns identified in the study, such as Thought Skipping, No Plan, and Wrong Logic. This capability allows the system to flag potential reasoning errors in real-time during inference, addressing the primary lesion identified in models exposed to junk data.
Implement a post-hoc mitigation strategy that prioritizes instruction tuning over continual control training for resilience against Brain Rot effects. The paper suggests that instruction tuning could be a more effective way to wash out the Brain Rot effect than post-hoc clean training, which implies focusing on refining instruction following rather than simply retraining on control data.
Deploy a mechanism for assessing LLM personality traits using concrete behavioral scenarios (TRAIT benchmark) rather than self-reporting, as suggested by the authors' critique of typical questionnaires. This provides a more robust measure of internal cognitive decline, specifically monitoring changes in traits like Psychopathy and Narcissism.
Develop an adaptive learning rate scheduler for continual pre-training that utilizes smaller learning rates (e.g., 1x10-6) to restrict the extent of representational drift, which the paper notes is mitigated by dampening updates while still retaining original capabilities despite junk exposure.