Analyzing Memory Effects in Large Language Models through the lens of Cognitive Psychology
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.
Ines: We're looking at the paper: "Analyzing Memory Effects in Large Language Models through the lens of Cognitive Psychology".
Marcus: Here's the gist of the paper. My analysis will be meticulous, focusing on extracting and integrating all relevant comparative data while maintaining the integrity of the original study's conclusions.
Ines: Here is a detailed and comprehensive summary of the paper, combining the provided textual excerpts: Detailed Summary of "Analyzing Memory Effects in Large Language Models through the Lens of Cognitive Psychology" This study undertakes a systematic investigation into seven key memory phenomena by using established human memory research paradigms as a comparative lens to evaluate state-of-the-art Large Language Models (LLMs).
Marcus: The core objective is to determine how LLM performance aligns with, or diverges from, documented human cognitive behaviors across various experimental conditions.
Ines: The seven specific phenomena analyzed are: list length effect, list strength effect, fan effect, nonsense effect, position effect (primacy/recency), DRM-style false memories, and cross-domain generalization ability.
Paper summary: Ines: So the paper "Analyzing Memory Effects in Large Language Models through the lens of Cognitive Psychology" sets out to systematically investigate seven key memory phenomena using human memory research as a comparative lens for large language models.
Marcus: The core thesis is that by testing LLMs against documented human cognitive behaviors, we can figure out how well these models emulate or deviate from natural memory processes in specific experimental conditions.
Yuki: So they are checking things like the list length effect, the fan effect, and even some false memory susceptibility through paradigms like DRM. It’s an effort to map LLM behavior onto known cognitive science.
Ines: They found that the models show characteristics strikingly similar to human memory in several areas. For instance, they display a fan effect where recall accuracy goes down as the number of associations increases, just like people do when overloaded with information.
Marcus: And they also saw comparable sensitivity to variations in list strength and length effects, meaning recall accuracy drops predictably as the list gets longer, mirroring human performance trends.
Yuki: They also noted that the models show a similar cross-domain generalization ability, which suggests they have a capacity to recognize critical words even when those words aren't in the specific test list.
Ines: That’s one part of it. But they also pointed out where things differ significantly, especially concerning how information is ordered or presented.
Marcus: Right, because they found two major divergences too. First, the position effect; unlike humans who show a primacy and recency effect—remembering better at the start and end of a sequence—the LLMs are position-invariant.
Yuki: That’s interesting because positional bias is usually considered a very fundamental feature of biological memory systems.
Ines: And the second divergence is related to nonsense content, where the models showed superior robustness when processing random or semantically meaningless material compared to human participants who get confused more easily by that kind of interference.
Marcus: So they’re establishing a baseline for comparison: LLMs mimic certain aspects of associative interference but fail on sequence-based positional biases and are actually better at handling pure noise than we expect.
Conclusion: Ines: To wrap up, the paper "Analyzing Memory Effects in Large Language Models through the lens of Cognitive Psychology" concludes that LLMs exhibit some human-like patterns in certain memory tasks but also show clear structural differences.
Marcus: The title and authors suggest they are trying to bridge the gap between cognitive psychology and artificial intelligence by using these memory effects as a benchmark for alignment.
Yuki: The implication is that while we might be able to build models that capture some of the associative interference seen in humans, we still have significant gaps, particularly when it comes to sequence order.
Ines: It suggests that understanding how LLMs handle list length and association is useful, but it doesn't automatically mean they are replicating human memory exactly.
Marcus: So what this means for us is that when we build systems, we need to be careful about imposing cognitive biases onto them because the underlying architecture seems to handle positional effects differently than our brains do.
Yuki: It’s a caution, really. We can study these models' memory as a tool, but we shouldn't assume they operate on the same fundamental cognitive principles as humans just because some surface behaviors align.
Syracuse University
q-bio.NC
Submitted: 2025-09-21
Updated: 2026-11-14
Code: https://github.com/zycao29/LLM_
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 77/100
The gist: My analysis will be meticulous, focusing on extracting and integrating all relevant comparative data while maintaining the integrity of the original study's conclusions.
Key concepts
- Fan Effect
- This is a pattern where performance drops as the number of associations increases. LLMs showed this human-like behavior, meaning their ability to recall information gets worse when you try to link too many items together.
- Position Effect (Primacy/Recency)
- This refers to how memory performance changes based on where information is placed in a sequence. Humans show strong effects (better at the start and end). LLMs, however, are position-invariant, meaning they don't care if the list starts at the beginning or the end.
- DRM-style False Memories
- This tests how easily models create false memories based on lures. LLMs showed a similar pattern of error generation to humans in this area, suggesting a shared underlying mechanism for generating these types of errors.
Terminology
Summary
My analysis will be meticulous, focusing on extracting and integrating all relevant comparative data while maintaining the integrity of the original study's conclusions.
Here is a detailed and comprehensive summary of the paper, combining the provided textual excerpts:
Detailed Summary of Analyzing Memory Effects in Large Language Models through the Lens of Cognitive Psychology
This study undertakes a systematic investigation into seven key memory phenomena by using established human memory research paradigms as a comparative lens to evaluate state-of-the-art Large Language Models (LLMs). The core objective is to determine how LLM performance aligns with, or diverges from, documented human cognitive behaviors across various experimental conditions. p. 6 Figure 4: LLM performance across memory effects measured by d’. Mean d’ values are shown for three experimental groups:
(a) memory sensitivity and discriminability decrease as list length increases, particularly under lower fan conditions; (b)
LLM shows that repetition progressively reduces discriminability; and (c) fan effect, demonstrating lower overall d’ for Fan 2
compared with Fan 1, consistent with classical interference patterns in human associative recognition.Where the paper says it
The seven specific phenomena analyzed are: list length effect, list strength effect, fan effect, nonsense effect, position effect (primacy/recency), DRM-style false memories, and cross-domain generalization ability. p. 1 Our investigation evaluates experimental paradigms from human memory research - primarily associative recognition studies
of the fan effect [2, 6] and the Deese-Roediger-McDermott (DRM)
false memory [38] - to evaluate LLM memory architectures. We
evaluate seven critical phenomena: list length effect (scaling performance with increasing memory load), list strength effect (interference in repetitive retrieval), fan effect (associative interference),
nonsense effect (handling of non-meaningful stimuli), position effect (primacy/recency patterns in sequential memory), DRM-style
false memories (false memory in a list learning paradigm), and
cross-domain generalization ability (transfer of schema-consistent
details).Where the paper says it
Findings on Human-Like Patterns in LLMs:
The research compellingly demonstrates that LLMs exhibit several characteristics strikingly similar to those observed in human memory:
-
Associative Interference (Fan Effect): LLMs display human-like patterns of associative interference, specifically exhibiting the fan effect, where retrieval accuracy decreases as the number of associations increases.
-
List Sensitivity: Models show a comparable sensitivity to variations in list length and list strength effects; for instance, recall accuracy declines predictably as list length increases, mirroring human performance trends.
-
False Memory Susceptibility (DRM Effect): LLMs demonstrate parallel susceptibility to DRM-style false memories, showing lure responses consistent with the DRM paradigm. This suggests a similarity in the pattern of error generation between humans and LLMs; however, the study notes that LLMs exhibit greater accuracy and less interference in recognition tasks compared to human subjects.
-
Generalization Ability: The models show comparable cross-domain generalization ability, indicating they possess a human-similar capacity to recognize critical lure words even when those words are absent from the specific study list.
Key Divergences from Human Cognition:
Crucially, the study identifies two primary areas where LLMs diverge significantly from human memory processes:
-
Positional Bias (Position Effect): LLMs show no sensitivity to the order in which information is presented. Unlike humans, who exhibit classic serial position effects—specifically primacy (higher recall at the beginning) and recency (higher recall at the end)—LLMs are position-invariant. p. 5
Where the paper says it
Generally, as shown in Figure 3c, human memory shows a Ushaped serial position curve [21], with higher recall at the beginning and end of lists [38]. In contrast, LLMs show no sensitivity to the positional bias.
-
Robustness to Anomalous Content (Nonsense Effect): LLMs demonstrate superior robustness when processing random or semantically meaningless material. They show a stronger resistance to errors induced by semantically anomalous content compared to human participants, who are more susceptible to such interference. p. 1
Where the paper says it
While LLMs mirror the qualitative patterns of human memory, they are more accurate, less affected by interference, and show stronger generalization, even as their pattern of errors remains similar to those of human cognition. • Our experiments reveal where LLM memory systems fundamentally differ: i): Superior robustness to semantically anomalous content (nonsense effect).
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improve memory retrieval by implementing mechanisms to mitigate list length effect by dynamically adjusting search strategies based on predicted information load, as
recall accuracy decreases as list size increases.
-
Enhance associative interference handling by designing retrieval systems that prioritize lower fan values during recognition tasks, as
higher associations to an item reduce recall accuracy.
-
Increase robustness against nonsensical inputs by training models with stronger semantic scaffolding, counteracting the finding that LLMs show
superior robustness to semantically anomalous content.
-
Develop better generalization capabilities by fine-tuning systems on diverse datasets to improve performance on cross-domain generalization, as "LLMs exhibit similar characteristics of human memory effects: i) Human-like patterns in associative interference - fan effect. ii) Similar sensitivity to list length and list strength variations. iii) Parallel susceptibility to DRM-style false memories. iv) Comparable generalization ability."
-
Implement positional awareness mechanisms to correct for the observed lack of sensitivity to order, aiming to replicate human memory's
Ushaped serial position curve
by incorporating sequential processing biases into retrieval architectures.
Abstract
Memory, a fundamental component of human cognition, exhibits adaptive yet fallible characteristics as illustrated by Schacter's memory "sins".These cognitive phenomena have been studied extensively in psychology and neuroscience, but the extent to which artificial systems, specifically Large Language Models (LLMs), emulate these cognitive phenomena remains underexplored. This study uses human memory research as a lens for understanding LLMs and systematically investigates human memory effects in state-of-the-art LLMs using paradigms drawn from psychological research. We evaluate seven key memory phenomena, comparing human behavior to LLM performance. Both people and models remember less when overloaded with information (list length effect) and remember better with repeated exposure (list strength effect). They also show similar difficulties when retrieving overlapping information, where storing too many similar facts leads to confusion (fan effect). Like humans, LLMs are susceptible to falsely "remembering" words that were never shown but are related to others (false memories), and they can apply prior learning to new, related situations (cross-domain generalization). However, LLMs differ in two key ways: they are less influenced by the order in which information is presented (positional bias) and more robust when processing random or meaningless material (nonsense effect). These results reveal both alignments and divergences in how LLMs and humans reconstruct memory. The findings help clarify how memory-like behavior in LLMs echoes core features of human cognition, while also highlighting the architectural differences that lead to distinct patterns of error and success.
Sources
- GPT-4 Technical Report
- Centaur: a foundation model of human cognition
- SongComposer: A Large Language Model for Lyric and Melody Generation in Song Composition
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- DeepSeek-V3 Technical Report
- Large Language Models and Cognitive Science: A Comprehensive Review of Similarities, Differences, and Challenges
- Summarization is (Almost) Dead
- Do Large Language Models Mirror Cognitive Language Processing?
- Humanlike Cognitive Patterns as Emergent Phenomena in Large Language Models
- ChatMusician: Understanding and Generating Music Intrinsically with LLM
Related papers
- Self-Supervised Learning of Local Predictive Directions as a Neural Computation
- Synchronized Hearts, Desynchronized Brains: A Nonlinear Dynamics Perspective for Function and Failure
- Message interaction dynamics covary with brain volume across mammalian connectomes
- Planning as Dynamics Relaxation: Hippocampal Recurrent Network Realizes Optimal Goal-Directed Navigation
- The Platonic brain bridge hypothesis: brain networks as architectural priors for multimodal large language models
- BrainPFN: Amortized Brain-Behaviour Prediction via Prior-Data Fitted Networks