BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design
cs.AI, cs.AR, cs.CL, cs.PF
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/OpenBitSys/BitDecoding-FP4
Terminology
Sources
- LongBench Pro: A More Realistic and Comprehensive Bilingual Long-Context Evaluation Benchmark
- DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
- Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
- GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving
- Online normalizer calculation for softmax
- Pretraining Large Language Models with NVFP4
- Microscaling Data Formats for Deep Learning
- OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond
- Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries
- Quantization-Aware Distillation for NVFP4 Inference Accuracy Recovery
- Qwen3 Technical Report
- MXAttention: Data-Free Optimal Scaling and Pre-Normalization Quantization for MXFP4 Attention
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate
- OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents