Anchor: Augmented Cross-Layer Recurrent State for Long-Context Retrieval in Test-Time Training
cs.LG, cs.AI
Submitted: 2026-11-03
Updated: 2026-11-03
Comments: 10 Pages main paper with reference and appendix
Code: https://github.com/dvlab-research/LongLoRA
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Longformer: The Long-Document Transformer
- Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
- Generating Long Sequences with Sparse Transformers
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- Native Hybrid Attention for Efficient Sequence Modeling
- Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
- Mechanistic Design and Scaling of Hybrid Architectures
- End-to-End Test-Time Training for Long Context
- Kimi Linear: An Expressive, Efficient Attention Architecture
- GLIDE: Guided Layerwise Hybrid Attention for Efficient LLM Inference
- Gated Linear Attention Transformers with Hardware-Efficient Training
- Attention Amnesia in Hybrid LLMs: When CoT Fine-Tuning Breaks Long-Range Recall, and How to Fix It
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis