A Fine-Grained Hierarchy and Benchmark for State Tracking in Language Models
cs.AI
Submitted: 2026-11-02
Updated: 2026-11-02
Terminology
Sources
- CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution
- Why are Sensitive Functions Hard for Transformers?
- Training Compute-Optimal Large Language Models
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- Efficiently Representing Algorithms With Chain-of-Thought Transformers
- The Expressive Power of Transformers with Chain of Thought
- A Little Depth Goes a Long Way: The Expressive Power of Log-Depth Transformers
- Why Are Linear RNNs More Parallelizable?
- Olmo Hybrid: From Theory to Practice and Back
- An Algebraic View of the Expressivity of Recurrent Language Models
- A Compositional Theory of Causally Masked Transformers
- DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Products
- Learning State-Tracking from Code Using Linear RNNs
- Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
- Olmo 3
- Logic Meets Algebra: the Case of Regular Languages
- Length Generalization of Causal Transformers without Position Encoding
- Knee-Deep in C-RASP: A Transformer Depth Hierarchy
- Algebraic Decomposition Theory for Transformer Length Generalization
- Gated Delta Networks: Improving Mamba2 with Delta Rule
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents