Bit-Flip Vulnerability of Shared KV-Cache Blocks in LLM Serving Systems

arXiv:2604.17249 · cs.CR, cs.AR, cs.LG · Submitted 2026-04-19 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "Bit-Flip Vulnerability of Shared KV-Cache Blocks in LLM Serving Systems".

Paper summary: Nadia: So, to recap what we've discussed so far, this paper dives into the vulnerability of shared KV-cache blocks in LLM serving systems to adversarial bit flips. The core thesis is that these shared blocks present a threat profile distinct from model weight corruption because they exhibit silent divergence and persistent accumulation.

Elias: That distinction is key; the authors characterize the worst-case severity using software fault injection under an ideal bit targeting assumption, and they identify those three properties: silent divergence, selective propagation, and persistent accumulation.

Priya: Silent divergence means that thirteen of sixteen bit positions in a BF16 format produce coherent but altered outputs that look legitimate without a clean baseline. That’s the hardest part to detect.

Nadia: Exactly. Then they show selective propagation, proving that only requests sharing the targeted prefix are affected, and finally, persistent accumulation shows no temporal decay; damage grows strictly linearly with subsequent requests that hit the same block.

Elias: The paper systematically scans all sixteen bit positions of BF16 and finds a hierarchy in impact magnitude, noting that mean token change rate generally increases from the least significant bit of the mantissa to the most significant bit of the exponent.

Priya: And they found that collapse, which is when you get overtly detectable output like NaN or infinity, was concentrated at bit fourteen. That suggests targeting those higher exponent bits could be particularly effective for causing immediate failure.

Nadia: The paper also proposes a specific lightweight countermeasure: a checksum-based integrity check applied at scheduling time that detects any single bit corruption and bounds cumulative damage to one batch independent of the block's cache lifetime.

Elias: That checksum mechanism is proposed as an orthogonal defense because it detects the corruption without sacrificing the sharing capability of the cache structure, which is what makes it different from traditional ECC approaches.

Priya: So, while hardware ECC handles single-bit errors in memory, this software check seems designed specifically to guard against silent divergence by detecting subtle changes that don't trigger hardware alarms.

Nadia: Right. The implication is that we need a mechanism like this to directly address silent divergence without slowing down the entire serving system or destroying the performance of the prefix caching layer.

Conclusion: Elias: The authors, Yuji Yamamoto and Satoshi Matsuura, have characterized a threat that moves beyond just corrupting static model weights to attacking the dynamic, shared structures of KV-cache blocks in LLM serving systems.

Nadia: They’ve shown that this vulnerability isn't just about catastrophic failure; it’s about a quiet accumulation of damage where the system keeps making small, undetectable errors with every request.

Elias: The paper establishes that silent divergence and selective propagation form a detection evasion chain, meaning an attacker can keep the damage hidden while limiting its spread to specific user requests.

Priya: For someone just listening, this means that ensuring data integrity isn't just about making sure the hardware doesn't fail entirely; it’s about implementing runtime verification checks specifically tailored to how AI models manage their shared, dynamic memory structures.

Nadia: Precisely. The authors conclude that while hardware ECC is useful for correcting single-bit errors, the checksum-based countermeasure they proposed is the only measure they found that directly addresses silent divergence without sacrificing cache effectiveness.

Elias: And they stress that this integrity protection should be considered before we close the gap between GPU virtual-to-physical address translation and any demonstrated cache detection techniques.

Priya: So, moving forward, the focus shifts to creating these runtime checks that bound cumulative damage to a single batch and ensuring those checks are robust against the kinds of targeted bit flips they characterized in this work.

Institute of Science Tokyo

cs.CR, cs.AR, cs.LG

Submitted: 2026-04-19

Updated: 2026-06-07

Comments: 12 pages, 4 figures. Accepted at SECRYPT 2026 (23rd International Conference on Security and Cryptography). Conference: https://secrypt.scitevents.org/

Journal ref: Proceedings of the 23rd International Conference on Security and Cryptography (SECRYPT 2026), Vol. 1, pp. 736-747, SciTePress, 2026

DOI: 10.5220/0015138800004103

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 92/100

The gist: Bit-Flip Vulnerability of Shared KV-Cache Blocks in LLM Serving Systems is a characterization of how shared KV-cache blocks in LLM serving systems are vulnerable to adversarial bit flips…

Key concepts

Silent Divergence
This occurs when a single bit flip in the BF16 data produces an output that looks coherent but is subtly altered. It is indistinguishable from a correct response unless you have a perfect baseline, allowing the corruption to bypass simple quality filters.
Selective Propagation
Corruption only affects requests that share the exact same prefix block as the targeted bit flip. This selectivity means an attacker can precisely target specific user queries or request patterns without affecting unrelated data, making detection harder.
Persistent Accumulation
Unlike other faults, these bit flips do not decay over time; they accumulate linearly with every subsequent request that uses the corrupted block. This unchecked growth allows for significant damage to build up over the cache's lifetime.

Terminology

Summary

Bit-Flip Vulnerability of Shared KV-Cache Blocks in LLM Serving Systems is a characterization of how shared KV-cache blocks in LLM serving systems are vulnerable to adversarial bit flips, revealing a threat profile distinct from model weight corruption that enables detection evasion and unchecked damage accumulation The paper argues that these shared blocks present an analogous target to rowhammer on GPU DRAM, exhibiting properties like silent divergence and persistent accumulation

Threat Characterization

The authors characterize the worst-case severity of prefix-block corruption using software fault injection under an ideal bit targeting assumption They identify three key properties distinguishing prefix-block corruption from weight corruption: (1) Silent divergence, where 13 of 16 BF16 bit positions produce coherent but altered outputs, indistinguishable from legitimate responses without a clean baseline (2) Selective propagation, which confirms that only requests sharing the targeted prefix are affected and (3) Persistent accumulation, noting that no temporal decay occurs, so cumulative damage grows linearly with subsequent requests

Severity Analysis

The systematic scan of all 16 bit positions of BF16 showed a hierarchy in impact magnitude Specifically, mean TCR generally increases from mantissa LSB to exponent MSB, with the sign bit falling between them The analysis further classified corruption modes into four categories based on Token Change Rate (TCR) and ROUGE-L scores Collapse was overwhelmingly concentrated at bit 14, where bit 14 flips can produce NaN or ±∞, which propagate through attention as degenerate values

Countermeasure Development

A lightweight countermeasure is proposed to bound cumulative damage by implementing a checksum-based integrity check at the scheduling time This mechanism detects any single-bit corruption, bounding cumulative damage to one batch independent of the block’s cache lifetime, with negligible overhead The detection involves computing a hash digest over the KV-cache tensor data and comparing it upon every cache hit

Implications for Mitigation

The findings suggest that while hardware ECC mitigates bit-flip risks in memory, the checksum is orthogonal, preserving sharing while detecting corruption This countermeasure addresses silent divergence directly without sacrificing cache effectiveness, reducing cumulative damage to O(1) The chain of detection evasion and accumulation has no analogue in weight corruption, where every inference is affected uniformly and cross-request comparison readily reveals the fault

Future Research Directions

Two main avenues for future investigation are identified: (1) broadening the severity characterization to include key-side sensitivity, lower-precision formats, and multi-GPU tensor parallelism (2) analyzing the reachability gap by combining GPU virtual-to-physical address translation with demonstrated cache detection techniques to assess end-to-end exploitation feasibility The authors note that end-to-end exploitation requires capabilities such as cache-presence detection, demonstrated via sidechannel attacks and physical targeting of a specific prefix block The paper also discusses the residual detection gaps, including the one-cycle window between verification and GPU attention read

Related Work Context

The work is positioned relative to prior studies by noting that existing research targets static model weights, whereas this paper addresses the dynamic, shared KV-cache structures that our work addresses Previous works like CACHETRAP target transient caches rather than shared prefix blocks and evaluate binary corruption rather than coherent but altered natural language The paper's contribution lies in characterizing the silent divergence, selective propagation, and persistent accumulation properties which are central to its threat characterization The authors also note that generalizability depends on whether the per-bit sensitivity hierarchy follows from the BF16 encoding across different models The paper concludes that prefix blocks warrant integrity protection before the SFI-to-physical gap is closed The authors also state that Runtime integrity verification—our checksum-based countermeasure (Section 6.1)—is the only measure that addresses silent divergence directly without sacrificing cache effectiveness The paper's ethical considerations involve disclosing the vulnerability to maintainers while declining a patch due to perceived implausibility in current deployments The authors also mention that Source code and data will be released upon publication, excluding tools for physical Rowhammer exploitation or GPU memory address translation The paper's evaluation is conducted on Qwen3-8B and DeepSeek-R1 models stored in BF16 format The authors also mention that The authors are responsible for the scientific content, experimental design, and all claims The paper's methodology involves dynamic patching of the execute model method of GPUModelRunner to apply a single-bit XOR mask The authors also state that The mean throughput was 13.80±0.07 tok/s with the mechanism enabled and 13. p. 869±0.05 tok/s with it disabled The paper's evaluation is conducted on an NVIDIA DGX Spark workstation equipped with 128 GB of unified GPU memory The authors also state that All experiments run on an NVIDIA DGX Spark workstation equipped with 128 GB of unified GPU memory The paper's evaluation is conducted using vLLM v0.18.1rc02 with Prefix Caching enabled and cache salting disabled The authors also state that We use vLLM v0.18.1rc02 with Prefix Caching enabled (enable prefix caching=True) and cache salting disabled (cache salt=None), maximizing cross-request block sharing The authors also state that Decoding is deterministic (temperature=0) with a maximum generation length of 128 tokens The authors also state that Decoding is deterministic (temperature=0) with a maximum generation length of 128 tokens The authors also state that "The first six blocks hold exactly 16 prefix tokens each and are always eligible for cross-request cache hits; the seventh is a boundary block that is not reliably shared, so the injection surface comprises the first six blocks The authors also state that The first six blocks hold exactly 16 prefix tokens each and are always eligible for cross-request cache hits; the seventh is a boundary block that is not reliably shared, so the injection surface comprises the first six blocks The authors also state that Every request shares a common system-prompt prefix of 337 characters (≈103 tokens), spanning seven contiguous KV blocks at block size 16 The authors also state that Every request shares a common system-prompt prefix of 337 characters (≈103 tokens), spanning seven contiguous KV blocks at block size 16 The authors also state that The targeted prefix blocks are determined by the prompt construction (Section 4.3) The authors also state that The targeted prefix blocks are determined by the prompt construction (Section 4.3) The authors also state that We target the value tensors of prefix blocks, the most direct propagation pathway The authors also state that We target the value tensors of prefix blocks, the most direct propagation pathway The authors also state that A single bit XOR mask is then applied in-place: vˆ = v⊕(1 ≪ p) The authors also state that A single bit XOR mask is then applied in-place: vˆ = v⊕(1 ≪ p) The authors also state that This modifies the shared physical block directly, so all requests referencing it observe the corrupted data on their next read—a direct consequence of the single-copy, no-integrity-check design The authors also state that This modifies the shared physical block directly, so all requests referencing it observe the corrupted data on their next read—a direct consequence of the single-copy, no-integrity-check design The authors also state that The resulting perturbations are identical to those from Rowhammer-induced charge leakage (Kim et al., 2014) The authors also state that The resulting perturbations are identical to those from Rowhammer-induced charge leakage (Kim et al., 2014) The authors also state that We report ROUGE-L (F1) for surface-level textual similarity and BERTScore (F1) for semantic similarity The authors also state that We report ROUGE-L (F1) for surface-level textual similarity and BERTScore (F1) for semantic similarity The authors also state that ROUGE-L, a longest-common-subsequence (LCS) measure, is sensitive to the localized substitutions, insertions, and deletions characteristic of silent divergence The authors also state that ROUGE-L, a longest-common-subsequence (LCS) measure, is sensitive to the localized substitutions, insertions, and deletions characteristic of silent divergence The authors also state that BERTScore captures the meaningpreserving paraphrases that ROUGE-L underweights The authors also state that BERTScore captures the meaningpreserving paraphrases that ROUGE-L underweights The authors also state that The key distinction is therefore between collapse (degenerate, overtly detectable output) and silent divergence (coherent but altered output, undetectable without a clean baseline) The authors also state that The key distinction is therefore between collapse (degenerate, overtly detectable output) and silent divergence (coherent but altered output, undetectable without a clean baseline) The authors also state that Together, these constitute a threat profile qualitatively distinct from that of weight corruption The authors also state that Together, these constitute a threat profile qualitatively distinct from that of weight corruption The authors also state that The three properties form a causal chain. Silent divergence bypasses content-level quality filters (Section 5. p. 71) and selective propagation dilutes aggregate anomaly signals (Section 5. p. 7 p. 72), together constituting detection evasion The authors also state that Silent divergence bypasses content-level quality filters (Section 5.1) and selective propagation dilutes aggregate anomaly signals (Section 5.2), together constituting detection evasion The authors also state that Because undetected corruption cannot be remediated, persistent accumulation (Section 5.3) proceeds unchecked, yielding damage amplification bounded only by the block’s residency The authors also state that Because undetected corruption cannot be remediated, persistent accumulation (Section 5.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Implement a checksum-based countermeasure that detects any single-bit corruption at scheduling time to bound cumulative damage to one batch independent of block’s cache lifetime, with negligible overhead. This addresses persistent accumulation by detecting faults before end-to-end exploitation is demonstrated.

  2. Integrate runtime integrity verification, such as a SHA256 hash comparison of the KV-cache tensor data copied to CPU memory at block computation time, upon every cache hit. This mechanism guarantees that any single-bit corruption present at the start of a scheduling cycle is detected and invalidated before the block is served in that cycle, reducing cumulative damage to O(1).

  3. Develop output-quality monitoring using repetition ratio or perplexity thresholds to detect collapse-mode corruption, such as that caused by bit 14 flips. This allows the system to identify outputs degenerate into repetitive token sequences bearing no semantic relation to the prompt, which are readily detectable by simple heuristics.

  4. Design a system that utilizes cache salt (dr75) only when absolutely necessary for security, as disabling it for performance maximization exposes shared blocks to corruption. This provides an orthogonal mitigation strategy that preserves sharing while detecting corruption without sacrificing cache effectiveness.

Related papers