On the use of foundation models in cognitive science
Raj Sanjay Shah, Alex Warstadt, Michael Frank, Sashank Varma
cs.CL
Submitted: 2026-08-07
Updated: 2026-08-11
License: http://creativecommons.org/licenses/by/4.0/
The gist: A host of recent studies have evaluated the cognitive and developmental alignment of Foundation Models (FMs).
Terminology
Abstract
A host of recent studies have evaluated the cognitive and developmental alignment of Foundation Models (FMs). These investigations include evaluations of their correspondence to adult performance across a range of cognitive domains, as well as whether aspects of model training track children's cognitive development. However, using FMs as candidate cognitive models poses significant methodological and conceptual challenges. A key question underlies this effort: under what conditions does behavioral alignment justify treating FMs as explanatory models of cognition? In this paper, we articulate a four-stage inferential framework for evaluating FMs as cognitive and developmental models: adapting human experimental tasks to model-compatible formats, specifying linking hypotheses that map model outputs to human measures, evaluating behavioral correspondence, and comparing across candidate models or manipulations. We clarify the role of linking hypotheses in mapping model outputs to human behavioral measures, identify challenges that constrain alignment claims, and propose principles for theory-driven and comparative evaluation. Throughout, we argue that behavioral fit alone is insufficient. Alignment becomes scientifically meaningful only when embedded within explicit theoretical commitments, theory-diagnostic tasks, and systematic contrastive evaluation across candidate models.
Sources
- Antagonistic AI
- BabyLM Turns 4 and Goes Multilingual: Call for Papers for the 2026 BabyLM Workshop
- CogBench: a large language model walks into a psychology lab
- HLB: Benchmarking LLMs' Humanlikeness in Language Use
- A Primer on the Inner Workings of Transformer-based Language Models
- The Representational Geometry of Number
- Are More Tokens Rational? Inference-Time Scaling in Language Models as Adaptive Resource Rationality
- Representation biases: will we achieve complete understanding by analyzing representations?
- Rethinking Thinking Tokens: LLMs as Improvement Operators
- N-gram-like Language Models Predict Reading Time Best
- To model human linguistic prediction, make LLMs less superhuman
- Perturbation: A simple and efficient adversarial tracer for representation learning in language models
- Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering