Block-GTQ: RoPE-Block Rate Allocation for KV-Cache Quantization
cs.LG, cs.CL
Submitted: 2026-06-23
Updated: 2026-10-29
Comments: 41 pages, 8 figures; revised formulation and deployment evaluation; expanded experiments and appendices. Code: https://github.com/JIA-Lab-research/blockgtq
Code: https://github.com/JIA-Lab-research/blockgtq
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
- FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
- MiniCache: KV Cache Compression in Depth Dimension for Large Language Models
- PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs
- Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time
- TriAttention: Efficient Long Reasoning with Trigonometric KV Compression
- Efficiently Scaling Transformer Inference
- FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU
- SQuat: Subspace-orthogonal KV Cache Quantization
- Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost
- RAP: KV-Cache Compression via RoPE-Aligned Pruning
- No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization
- SubGen: Token Generation in Sublinear Time and Memory
- MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoning
- EliteKV: Scalable KV Cache Compression via RoPE Frequency Selection and Joint Low-Rank Projection
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis