CacheSolidarity: Preventing Prefix Caching Side Channels in Multi-Tenant LLM Serving Systems

summary

The gist

Large Language Models (LLMs) rely on optimizations like Automatic Prefix Caching (APC) to accelerate inference…

In short

Automatic Prefix Caching (APC) in multi-tenant LLM systems creates timing side channels that allow attackers to steal sensitive information by observing cache hit/miss patterns across users. CacheSolidarity introduces PrefixWall, a system that secures these systems by selectively isolating only suspicious shared prefixes rather than isolating entire users. This approach maintains high performance while closing the timing leak.

Key concepts

Automatic Prefix Caching (APC)
APC is an optimization used in LLM serving that reuses previously computed model states for the beginning part of a request, called a prefix. While it speeds up inference by avoiding redundant computation, it creates observable latency differences between cache hits and misses that attackers can exploit.
Timing Side Channel
This is an unintended information leak where the time taken to process a request reveals properties of secret data. In this context, APC causes timing variations based on whether a prefix is cached or not, allowing an attacker to infer parts of another user's sensitive prompt by measuring these small latency differences.
PrefixWall
A system designed to prevent APC side channels by monitoring cache reuse across users. Instead of disabling caching entirely, it tracks suspicious sharing and selectively isolates only the prefixes involved in potential attacks, balancing security with high performance.

Terminology used across episodes

This episode discusses

The paper

CacheSolidarity: Preventing Prefix Caching Side Channels in Multi-Tenant LLM Serving Systems · Read on arXiv

IMDEA Software Institute · Universidad Politécnica de Madrid

Large Language Models (LLMs) rely on optimizations like Automatic Prefix Caching (APC) to accelerate inference. APC works by reusing previously computed states for the beginning part of a request (prefix), when another request starts with the same text. While APC improves throughput, it introduces timing side channels: cache hits are faster than misses, creating observable latency differences. In multi-tenant systems, attackers can exploit these differences to infer sensitive information, e.g., by incrementally reconstructing another user's request by observing hit/miss patterns. Current defenses take a sledgehammer approach: they disable APC and cache sharing, isolating users, and sacrificing efficiency for regular users. This paper presents CacheSolidarity, a system that secures multi-tenant LLM serving systems against APC side channels without sacrificing performance and efficiency. CacheSolidarity monitors cache reuse across users, flags suspicious sharing, and selectively isolates prefixes, restricting their reuse only when necessary. Evaluation shows that CacheSolidarity enables up to 70% higher cache reuse and 30% lower inference latency compared to existing user-isolation defenses. CacheSolidarity's lightweight design demonstrates how LLM serving can be secured without sacrificing performance or incurring unbearable overheads.

DOI: 10.1145/3801927.3858638

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "CacheSolidarity: Preventing Prefix Caching Side Channels in Multi-Tenant LLM Serving Systems".

Elias: Here's the gist of the paper. Large Language Models (LLMs) rely on optimizations like Automatic Prefix Caching (APC) to accelerate inference, but this introduces timing side channels that attackers can exploit to infer sensitive information in multi-tenant systems The paper presents PrefixWall, a system that secures multi-tenant LLM serving systems against APC side channels without sacrificing performance and efficiency Problem Statement The core issue is that APC creates observable latency differences between cache hits and misses, which form a timing side channel that attackers can use to reconstruct another user’s request by observing hit/miss patterns Current defenses take a "sledgehammer approach" by disabling APC and isolating users, which sacrifices efficiency for regular users Existing defenses suffer from significant overhead because they isolate entire users instead of just the shared prefixes that might lead to attacks PrefixWall Design PrefixWall is a system-level solution based on the insight that isolating entire users is not necessary, but rather sufficient to protect the shared prefixes that might lead to attacks The design involves three main components: 1.

Nadia: KV Cache Extension: Augments each cache entry with metadata like OwnerID and AttackFlag to track ownership and flag suspicious reuse 2.

Elias: Detector: Examines prefix reuse to identify cross-user access and enforces selective isolation when suspicious reuse occurs 3.

Nadia: Activator: Dynamically activates selective isolation only under conditions that create exploitable timing side-channels, balancing security with performance Performance and Evaluation Evaluation shows that PrefixWall enables up to 70% higher cache reuse and 30% lower inference latency compared to existing defenses that isolate users The system is implemented on top of vLLM

64: and demonstrates negligible time and memory overheads during runtime PrefixWall preserves most of the performance benefits of prefix caching while enforcing strong security guarantees, even under diverse workload patterns Security Guarantees PrefixWall's selective isolation scheme is designed to provide security against prompt-stealing attacks performed through APC timing side channels Security guarantees are provided when conditions (1) and (2) are met, ensuring that executing all probes in the sequence s from user identifiers different from u will always result in misses for entries v1,..., vn The system successfully closes the timing leak on an "attacker workload," where both hit rate and TTFT are uniform across all attacker requests Sensitivity Analysis The selection of the KDE overlap threshold represents a tradeoff between security and performance With very low thresholds, PrefixWall performs comparably to Prefix Caching for the same workload since selective-isolation is almost always disabled For high threshold (close to 100%), the system achieves the hit rate and performance of Selective Cache Isolation The activator component provides the system administrator with a way of effectively balancing performance and security Overhead Analysis In terms of per-request latency, PrefixWall’s modifications introduce an average per-request Component Time of 0.007 ms (0.004 ms due to the detector component and 0.003 ms due to the activator component) In terms of memory, the metadata introduced by the detector component amounts to 32 bytes per cache entry The overall overhead is minimal, demonstrating that security can be achieved without sacrificing performance when implemented in a lightweight and principled way PrefixWall builds on the key insight that preventing timing side-channel attacks does not require isolating entire users, but only requires isolating prefixes that might lead to attacks This approach enables efficient sharing for benign users while selectively isolating only those prefixes associated with suspicious behavior PrefixWall’s lightweight design demonstrates how security in LLM serving does not have to come at the cost of unnecessarily reduced performance or unbearable overheads The specific paper contributions include a detailed analysis of timing side channels, the design of PrefixWall, a comprehensive security analysis, and an extensive evaluation across diverse workloads Page 1 --- PrefixWall: Mitigating Prefix Caching Side Channels in Shared LLM Systems Panagiotis Georgios Pennas IMDEA Software Institute Universidad Politécnica de Madrid panagiotis.pennas@imdea.org Konstantinos Papaioannou IMDEA Software Institute Universidad Politécnica de Madrid konstatinos.papaioannou@imdea.org Marco Guarnieri IMDEA Software Institute marco.guarnieri@imdea.org Thaleia Dimitra Doudali IMDEA Software Institute thaleia.doudali@imdea.org ABSTRACT Large Language Models (LLMs) rely on optimizations like Automatic Prefix Caching (APC) to accelerate inference APC works by reusing previously computed states for the beginning part of a request (prefix), when another request starts with the same text While APC improves throughput, it introduces timing side channels: cache hits are faster than misses, creating observable latency differences In multi-tenant systems, attackers can exploit these differences to infer sensitive information, e.g., by incrementally reconstructing another user’s request by observing hit/miss patterns Current defenses take a sledgehammer approach: they disable APC and cache sharing, isolating users, and sacrificing efficiency for regular users This paper presents PrefixWall, a system that secures multi-tenant LLM serving systems against APC side channels without sacrificing performance and efficiency PrefixWall monitors cache reuse across users, flags suspicious sharing, and selectively isolates prefixes, restricting their reuse, only when necessary Evaluation shows that PrefixWall enables up to 70% higher cache reuse and 30% lower inference latency compared to existing defenses that isolate users PrefixWall’s lightweight design demonstrates how security in LLM serving does not have to come at the cost of unnecessarily reduced performance or unbearable overheads 1 INTRODUCTION Large Language Models (LLMs) now power applications such as conversational assistants, code generation, and enterprise analytics

13: These services operate at massive scale under strict latency and throughput requirements, making inference serving a critical systems challenge

76: Modern serving systems employ system-level optimizations such as cache management

39, 40, 59: and scheduling

2, 3, 33: to accelerate LLM inference Among these optimizations, prefix sharing

10, 37, 71: , or otherwise known as Automatic Prefix Caching (APC), is widely used by state-of-the-art frameworks and commercial APIs APC accelerates inference by caching and reusing previously computed model states for the beginning part of a request (the request’s prefix) whenever another request starts with the same text The use of a prefix cache avoids redundant computation and significantly reduces latency for long prompts and multi-turn conversations

37, 71: While APC delivers substantial performance benefits, it also introduces security risks similar to other caching optimizations [12, Name: Password: Address: Email: Prompt Constructor Hit Miss FEEDBACK Shared Prefix Cache Time Analyzer Victim Attacker Serving System LLM Secret Inputs Request Response Request Response … Response Generation Time Request Diff Response LLM Figure 1 Timing side-channel leakage in prefix-sharing LLM inference.

Paper summary: Elias: So to wrap up the discussion on "CacheSolidarity: Preventing Prefix Caching Side Channels in Multi-Tenant LLM Serving Systems", the authors are really pushing the idea that you don't need full user isolation to stop prompt-stealing attacks exploiting APC timing side channels.

Priya: It’s about moving away from that sledgehammer approach and toward a more surgical security method where you only isolate specific prefixes when there's actual suspicious behavior, which they achieve with the PrefixWall system architecture.

Nadia: Right. The implication is that for high-performance, multi-tenant LLM serving environments, this selective isolation can be the right balance to maintain efficiency while closing those timing leaks without imposing massive penalties on benign users.

Elias: They show that even with their lightweight design, they can achieve up to seventy percent higher cache reuse and lower latency compared to defenses that enforce user-level isolation <ref:2603.10726#pg1>.

Priya: So, the takeaway for us is that security in this area can be achieved through a principled design focused on monitoring prefix reuse across users and selectively isolating only those prefixes associated with suspicious behavior, as outlined in "CacheSolidarity: Preventing Prefix Caching Side Channels in Multi-Tenant LLM Serving Systems".

Conclusion: Nadia: So, we're wrapping up this look at "CacheSolidarity: Preventing Prefix Caching Side Channels in Multi-Tenant LLM Serving Systems". The main idea is that you don't need to isolate every single user completely to stop prompt-stealing attacks when using Automatic Prefix Caching.

Elias: Right. They’re arguing that isolating entire users is too much overhead for regular users, and the actual security comes from selectively isolating just the shared prefixes when they look suspicious across different users.

Priya: From a measurement standpoint, what this means is that instead of crippling performance with heavy isolation, you can maintain pretty good cache reuse while still mitigating that timing leak risk.

Nadia: Exactly. The authors show that this selective approach actually allows for up to seventy percent higher cache reuse and thirty percent lower inference latency compared to the other defenses they tested.

Elias: I like the numbers, but we gotta look at how they do it technically, you know? They extend the cache entry with some metadata like an Owner ID and an Attack Flag.

Priya: That metadata is key. It lets them track ownership and flag reuse patterns between users, which feeds into a detector that isolates prefixes only when things get too suspicious.

Nadia: The system has three parts: the cache extension, the detector that flags suspicious sharing, and then the activator that decides when to actually enforce isolation based on whether a timing side channel is exploitable.

Elias: And they validate it with a security analysis showing that under certain conditions—specifically when certain probes are made from different user identifiers—the system ensures misses for those entries.

Priya: The sensitivity analysis on that overlap threshold shows the trade-off between performance and security, letting the admin fine-tune how strict the isolation gets.

Nadia: So, what does this mean for real world deployment? It suggests that securing LLM serving doesn't always have to mean sacrificing speed for every single request.

Elias: It means finding a middle ground where you get strong protection against these specific timing attacks without making the system unusable for everyone who just wants fast inference.

Priya: If this holds up under diverse workloads, it could be a significant way to make shared LLM infrastructure safer in practice.

More episodes

← Home