Dynamical Memory Transformer: Selective Recurrent Memories with Constant Cost Attention
cs.AI
Submitted: 2026-10-30
Updated: 2026-10-30
Code: https://github.com/karpathy/char-rnn
Project page: https://skylion007.github.io/OpenWebTextCorpus
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- CogScale: Scalable Benchmark for Sequence Processing
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- RWKV: Reinventing RNNs for the Transformer Era
- Retentive Network: A Successor to Transformer for Large Language Models
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents