DeltaTTT: Layerwise Optimization for Nonlinear Recurrent Memory
cs.LG, cs.AI, cs.CL
Submitted: 2026-10-06
Updated: 2026-10-06
Sources
- ATLAS: Learning to Optimally Memorize the Context at Test Time
- It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization
- Titans: Learning to Memorize at Test Time
- Decoupled Greedy Learning of CNNs
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- Going Beyond Linear Transformers with Recurrent Fast Weight Programmers
- Linearizing Vision Transformer with Test-Time Training
- TNT: Improving Chunkwise Training for Test-Time Memorization
- Test-Time Training with KV Binding Is Secretly Linear Attention
- Learning to (Learn at Test Time): RNNs with Expressive Hidden States
- End-to-End Test-Time Training for Long Context
- Modular TTT: Rethinking Test-Time Training as Composable Modules
- Learning What to Remember: Test-Time Training via Context Distillation
- Parallelizing Linear Transformers with the Delta Rule over Sequence Length
- Test-Time Training Done Right
- Rethinking Expressivity and Efficiency in Test-Time Training
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks