OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
cs.LG, cs.AI, cs.DC, cs.PF
Submitted: 2026-05-18
Updated: 2026-11-10
Code: https://github.com/vllm-project/vllm
Terminology
Sources
- MiniKV: Pushing the Limits of LLM Inference via 2-Bit Layer-Discriminative KV Cache
- Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
- WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
- Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost
- RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations
- SpinQuant: LLM quantization with learned rotations
- SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving
- FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
- FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling
- KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
- Qwen3 Technical Report
- ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
- Evaluating Large Language Models Trained on Code
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate
- SnapKV: LLM Knows What You are Looking for Before Generation
- PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
- GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
- Palu: Compressing KV-Cache with Low-Rank Projection
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis