POEM: Output-Relative Block-Sparse Attention for Long-Context Prefilling
cs.CR, cs.CL
Submitted: 2026-11-10
Updated: 2026-11-10
Terminology
Sources
- ReST-KV: Robust KV Cache Eviction with Layer-wise Output Reconstruction and Spatial-Temporal Smoothing
- Generating Long Sequences with Sparse Transformers
- Challenges in Deploying Long-Context Transformers: A Theoretical Peak Performance Analysis
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs
- SeerAttention-R: Sparse Attention Adaptation for Long Reasoning
- CAOTE: KV Cache Selection for LLMs via Attention Output Error-Based Token Eviction
- The Llama 3 Herd of Models
- Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling
- SALE : Low-bit Estimation for Efficient Sparse Attention in Long-context LLM Prefilling
- FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
- Scout Before You Attend: Sketch-and-Walk Sparse Attention for Efficient LLM Inference
- SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- A Unified Sparse Attention via Multi-Granularity Compression
- Stem: Rethinking Causal Information Flow in Sparse Attention
- AlphaEvolve: A coding agent for scientific and algorithmic discovery
- Compressive Transformers for Long-Range Sequence Modelling
- KV-Compress: Paged KV-Cache Compression with Variable Compression Rates per Attention Head
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data