Modular TTT: Rethinking Test-Time Training as Composable Modules
Bohao Tang, Zhen Qin, Yuqi Pan, Zheng Li, Pengfei Liu, Ya Zhang
cs.LG, cs.CL
Submitted: 2026-08-07
Updated: 2026-08-10
Comments: Code: https://github.com/ByteDance-Seed/Modular-TTT
Code: https://github.com/ByteDance-Seed/Modular-TTT
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- ATLAS: Learning to Optimally Memorize the Context at Test Time
- It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization
- Nested Learning: The Illusion of Deep Learning Architectures
- Titans: Learning to Memorize at Test Time
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- ViT$^3$: Unlocking Test-Time Training in Vision
- TNT: Improving Chunkwise Training for Test-Time Memorization
- RWKV-7 "Goose" with Expressive Dynamic State Evolution
- GLU Variants Improve Transformer
- Learning to (Learn at Test Time): RNNs with Expressive Hidden States
- Retentive Network: A Successor to Transformer for Large Language Models
- End-to-End Test-Time Training for Long Context
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Test-Time Training Done Right
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks