ASCENT: Scale-Complementary External State for Frozen Long-Context Language Models
cs.CL, cs.LG
Submitted: 2026-11-06
Updated: 2026-11-06
Terminology
Sources
- Attention Is All You Need
- Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
- Compressive Transformers for Long-Range Sequence Modelling
- Longformer: The Long-Document Transformer
- Big Bird: Transformers for Longer Sequences
- Reformer: The Efficient Transformer
- Rethinking Attention with Performers
- Linformer: Self-Attention with Linear Complexity
- Nystr\"omformer: A Nystr\"om-Based Algorithm for Approximating Self-Attention
- Efficient Content-Based Sparse Attention with Routing Transformers
- ETC: Encoding Long and Structured Inputs in Transformers
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- Hyena Hierarchy: Towards Larger Convolutional Language Models
- RWKV: Reinventing RNNs for the Transformer Era
- Retentive Network: A Successor to Transformer for Large Language Models
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
- Ring Attention with Blockwise Transformers for Near-Infinite Context
- Efficient Memory Management for Large Language Model Serving with PagedAttention
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data