CogBattery: Human-Referenced Evaluation of Cognitive Task Behavior and Structure in Language Models
cs.CL, cs.AI, q-bio.NC
Submitted: 2026-07-27
Updated: 2026-10-30
Comments: 13 pages, 3 figures, 5 tables. Code at https://github.com/dengzhe-hou/CogBattery. Renamed from CogArena to avoid confusion with an unrelated benchmark of the same name
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Revealing the structure of language model capabilities
- An LLM-Native Psychometric Instrument Reveals a Self-Report--Behavior Gap Across 25 Models
- A Neuropsychologically Grounded Evaluation of LLM Cognitive Abilities
- WMIP: A Working-Memory-Inspired Programmable Probe of Cumulative State Tracking in Large Language Models
- NeuReasoner: Theory-grounded Mapping of Reasoning Elicitation Boundaries
- ActTraitBench: Quantifying the Knowledge-Decision Gap in Large Language Models via Human-Grounded Behavioral Validation
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data