Encoding and Propagation of Grammatical Number in Large Language Models
cs.CL, cs.AI
Submitted: 2026-11-05
Updated: 2026-11-05
Comments: 50 pages, 19 figures
Code: https://github.com/solin020/dissertation_sae
Project page: http://skylion007.github.io/OpenWebTextCorpus
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
- Learning Multi-Level Features with Matryoshka Sparse Autoencoders
- Sparse Autoencoders Find Highly Interpretable Features in Language Models
- Neuron-based explanations of neural networks sacrifice completeness and interpretability
- PURE: Turning Polysemantic Neurons Into Pure Features by Identifying Relevant Circuits
- Toy Models of Superposition
- Scaling and evaluating sparse autoencoders
- The Llama 3 Herd of Models
- Verbalizable Representations Form a Global Workspace in Language Models
- Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders
- Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
- Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models
- Gemma 2: Improving Open Language Models at a Practical Size
- Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
- Attention Is All You Need
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data