Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression
cs.LG, cs.AI
Submitted: 2026-09-28
Updated: 2026-09-28
Sources
- MELODI: Exploring Memory Compression for Long Contexts
- Generating Long Sequences with Sparse Transformers
- Kwai Summary Attention Technical Report
- DeepSeek-V3 Technical Report
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- SnapKV: LLM Knows What You are Looking for Before Generation
- MoBA: Mixture of Block Attention for Long-Context LLMs
- Learning to Compress Prompts with Gist Tokens
- Controllably Efficient Language Models
- Qwen3 Technical Report
- Specialization of softmax attention heads: insights from the high-dimensional single-location model
- RazorAttention: Efficient KV Cache Compression Through Retrieval Heads
- Transformers as Support Vector Machines
- Gradient Flow Polarizes Softmax Outputs towards Low-Entropy Solutions
- Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
- LoMA: Lossless Compressed Memory Attention
- SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget
- DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks