Real Computation or Merely Naming? A Pre-Registered Mechanism Audit of 25 Cognitive-Architecture Modules in an LLM-Based Agent System
cs.AI
Submitted: 2026-10-30
Updated: 2026-10-30
Comments: 40 pages. Static audit of 25 cognitive-architecture modules (18 carry no computed value to any production output), pre-registered interventions on 11 modules, 61 frozen criteria (53 confirmed / 5 falsified), two repair controls (0.028 -> 1.000 task success; -52% error). Artifact: https://doi.org/10.5281/zenodo.23433900
Code: https://github.com/youngbryan97/au
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Evaluating Large Language Models Trained on Code
- Are We Measuring Scientific Intelligence? Rethinking the Evaluation of AI Scientists
- Kernel Autoresearch for Open-Ended Model Discovery
- When Agents Fail: Model or Harness--Who's to Blame?
- Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attribution
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents