Can KV Cache Be Reused Across Model Updates? A DeepSeek-V4-Flash Case Study
cs.DC, cs.AI
Submitted: 2026-11-10
Updated: 2026-11-10
License: http://creativecommons.org/licenses/by/4.0/
The gist: Reusing key--value (KV) caches across language model updates could avoid repeated computation of shared context…
Terminology
Abstract
Reusing key--value (KV) caches across language model updates could avoid repeated computation of shared context, but changes in model weights raise a fundamental question about what these caches preserve. We investigate direct cache reuse across four model updates, with a detailed study of DeepSeek-V4-Flash to Flash-0731. Our central finding is that caches can retain useful task information without faithfully reproducing the updated model's behavior. Retrieval accuracy varies substantially across updates, ranging from near-native performance to large losses. With source caches already available, reuse achieves median time-to-first-token speedups of 2.10 -- 5.08 times on three longer-prefix QA benchmarks. To characterize the behavioral differences behind these outcomes, we introduce Functional Cache Divergence, which compares next-token distributions under reused and native caches. Direct KV cache reuse better preserves native model behavior in natural-text continuation, whereas compatibility degrades with increasing prefix length in single-token retrieval tasks. Answer-margin analyses help explain how correct retrieval decisions survive substantial internal state changes. Yet even agreement on the first generated token does not ensure agreement in answer content or stopping behavior. These findings establish task utility and fidelity to native inference as distinct criteria for evaluating cross-version KV cache reuse.
Sources
- LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- Accelerating Cross-Model LLM Inference via Attention Map Reuse
- A Universal Context-Reuse Layer for Cross-Model KV Sharing
- DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving
- Locating and Editing Factual Associations in GPT
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
- CacheBridge: Efficient Cross-Model KV Transfer for Multi-Model LLM Serving
- RaReCache: Bridging the Gap in Cross-Model KV Cache Reuse via Rank disagreement-based Selective Recomputation
- KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints
- KVComm: Enabling Efficient LLM Communication through Selective KV Sharing
- Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs
- Towards Best Practices of Activation Patching in Language Models: Metrics and Methods
- Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends