Privacy Leakage from Recurrent State in LLM Inference

arXiv:2611.16048 · cs.CR · Submitted 2026-11-11 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "Privacy Leakage from Recurrent State in LLM Inference".

Elias: Here's the gist of the paper. Personalized LLM services place private facts and application instructions into a prompt, then retain representations of that context for later inference The information preserved by its learned updates is implicit, and later computation can expose it even when an observer receives neither the private text nor the responses Experiments with released hybrid implementations show that a small state observation supports multiple predictions about private facts and response rules, and that personal attributes generalize to unseen authors Complete-read power recovers private facts under a fixed execution schedule Pruning redundant reasoning tokens The paper investigates privacy leakage from the recurrent state in latent-state LLMs by examining partial-state probing and physical observation of its use The methodology separates recoverable information, persistence through public dialogue, causal influence on later decisions, and leakage through power The core question is what information remains recoverable from that state and from the later computations that read it How the training signal is built The recurrent component summarizes preceding tokens in fixed-size numerical state, where each update combines that state with the current token representation and produces an output for later layers This state is a history-dependent latent representation whose dimensions are fixed by the model, but its values depend on the session Learned gates can also decay or overwrite information during continued execution Private prompt content can enter this state during prefill and remain available when a later public request is processed Where the gains come from The research establishes that retained recurrent state as a confidentiality boundary for personalized LLM services Private facts and response rules remain inferable after the conversation moves to public content, and state transplantation connects retained information to later decisions The study develops a methodology for tracing private information from retained state to execution signals Physical observation tests two consequences of using the state: data-dependent power during a complete recurrent read and changes in expert execution during public requests Pruning redundant reasoning tokens Software probing tests whether restricted state observations predict private content, and retention measures persistence after public dialogue To measure contribution to later computation, the authors transplant recurrent state between matched histories before the future query Physical observability tests whether later use of retained state exposes private information without tensor access Where the gains come from The complete-read power test shows that a fixed execution schedule can recover a protected secret The GPU side channel experiment attributes the routing signal to the recurrent state rather than to the attention cache or convolution state These findings extend context confidentiality to the lifetime and execution of retained state Pruning redundant reasoning tokens The results demonstrate that personal attributes can be inferred from whole-device power during public requests The study identifies access-control responsibilities for serving systems, telemetry restrictions for GPU operators, and data-dependent power leakage for accelerator designers Where the gains come from The paper concludes that serving systems that cache, offload, migrate, or share recurrent state should give it the access control and isolation of the prompt that produced it and clear or recompute it at session boundaries GPU cloud operators should treat whole-device parent power as tenant data Accelerator designers should note that fixed control flow removed schedule dependence while data-dependent switching still disclosed private facts, so a constant-schedule datapath does not protect confidential state

Title and authors: Nadia: So we're looking at this paper, "Privacy Leakage from Recurrent State in LLM Inference," and the core idea is that when you use personalized AI services, you put private stuff—like personal facts or specific instructions—into the prompt, and the model keeps a representation of that context for later.

Elias: Right. It's about how this retained history, this latent state LLMs use to remember what they've seen, can leak information even if you never see the private text or the final response. The paper is investigating privacy leakage from that recurrent state through two main avenues: partial-state probing and physical observation of how it’s being used.

Priya: I'm curious about what kind of information they think is actually recoverable from that state, because if we can't control what the model remembers, then the privacy promise falls apart.

Nadia: Well, they set up their methodology to separate four things: information that is actually recoverable, how much stuff persists through public conversations you have with the AI, how it influences later decisions you make, and leakage that happens through physical power usage. They’re trying to map out all those different ways a secret could slip out.

Elias: The paper then focuses on what remains recoverable from that state itself and from the computations that read it later. That’s the main question driving their work here in "Privacy Leakage from Recurrent State in LLM Inference."

Priya: What does this mean for someone just using an AI service every day? Is it a warning about how much they should trust what they put into these personalized systems?

Nadia: It’s a warning that the state isn't just a temporary buffer; it can become a persistent boundary. They show experiments with released hybrid implementations where observing this small state gives multiple predictions about private facts and response rules, and personal attributes generalize even to authors they haven't seen before.

Elias: And they found something about persistence too, which is interesting for me because I’m focused on the parameters of these models. They found that the information remains inferable after public dialogue happens, and it also changes later decision margins.

Title and authors: Priya: So if I have a private fact about my health or a specific preference, and then I have a public chat with the AI, does that public chat somehow make it easier for the model to guess my private fact later?

Nadia: That’s exactly what they show: state transplantation connects that retained information to those later decisions. They developed a way to trace private information from that recurrent state all the way through to the actual execution signals.

Elias: And they tested this causal influence by transplanting the recurrent state between matched histories before asking a future query, and they quantified that contribution as the change in decision margin between the donor logit and the source logit.

Priya: That sounds like a strong way to measure causality, but what about physical leakage? I mean, if it’s all happening in software or computation, how do we actually see this state leaking outside of just looking at the code?

Nadia: They looked at physical observability testing two things: first, whether restricted state observations can predict private content, and second, how retention measures persistence after public dialogue. But they also did a complete-read power profiling experiment.

Elias: That power experiment was pretty telling because it found that a fixed execution schedule can recover a protected secret even when the model is running under specific access conditions.

Priya: Wait, so if we’re talking about physical observation, how does that translate to something a regular user notices on their device? Is this about things like the power draw of their phone or laptop while they're using these services?

Nadia: It means that on a GPU side channel experiment, they attributed the routing signal—the way data flows through the hardware—to that recurrent state rather than to other parts of the system like the attention cache or convolution state.

Elias: That’s significant because it extends context confidentiality to not just what's in memory, but how that information dictates physical operations during a session. They found personal attributes can even be inferred from whole-device power usage during public requests under certain access controls.

Title and authors: Priya: So, if I’m using this AI for a public query, and someone is monitoring my device's power draw, could they infer something about my private profile? That sounds like a real deployment concern.

Nadia: They identified specific access-control responsibilities for serving systems and telemetry restrictions for GPU operators because of these findings. They also told accelerator designers that while fixed control flow removes schedule dependence, data-dependent switching still discloses private facts.

Elias: That last point is crucial because it means a constant-schedule datapath doesn't protect confidential state if the switching behavior itself leaks something about the private information. It’s not just about what's stored; it's about how it moves and computes.

Priya: So, putting that all together, the implication for me is that we need to think differently about where we set those security boundaries—not just at the prompt level, but potentially around the entire lifetime of that recurrent state.

Nadia: Exactly. The paper suggests serving systems that cache or offload or migrate recurrent state should treat it with the same access control and isolation as the original prompt that created it, and they should clear or recompute it when a session boundary is crossed.

Elias: And for those of us working on accelerator design, the conclusion is that we need to be aware that data-dependent switching can still leak facts even if we keep the timing constant. It forces a rethink on how we isolate state during inference.

Priya: I just think this paper gives us a solid framework for how privacy isn't just about hiding text; it’s about controlling the physical and computational life of the data representation itself.

Nadia: That’s the gist of "Privacy Leakage from Recurrent State in LLM Inference." It shows that for personalized AI, we need to manage state retention as a key confidentiality boundary.

Elias: We learned how to trace private information from that retained state through execution signals and how physical power measurements can reveal secrets under specific conditions.

Priya: And it points toward needing stricter controls on state access, whether that's software isolation or hardware design choices, because the retention itself is a vector for leakage.

The paper's summary: Nadia: So, this paper is basically saying that when an AI keeps a memory of your private stuff—like instructions or facts from earlier in the chat—that memory itself becomes a potential leak.

Elias: Right, they're looking at how that recurrent state, this history the model builds over time, can expose things even if you never see the original prompt or the final answer.

Priya: So what’s their main angle on what kind of information is actually recoverable from that stored history? Is it just literal text, or is it something more subtle?

Nadia: They break down what they think is recoverable into four buckets—what you can actually recover, how much stuff stays through public back-and-forth, how that history influences later decisions you make, and leakage through physical power usage.

Elias: That’s a structured way to look at it. The core idea they're driving at is figuring out which of those things are still recoverable from the state and what happens when later computations read that state.

Priya: I’m interested in the causal part—how they measure that influence on future decisions you make, because if you can trace a private fact to a later choice, that’s dangerous.

Nadia: Exactly. They developed this methodology for tracing that private information from the retained state all the way to the actual execution signals of the model.

Elias: And they test it by transplanting that recurrent state between two different histories before asking a new question and quantifying that contribution as a change in decision margin, which is how much it shifts the outcome.

Priya: So, for someone just listening to this show, what does this mean practically? Does it mean my private preference gets leaked later during a public query I’m having?

Nadia: It means that retained recurrent state acts like a confidentiality boundary for personalized AI services. They argue that serving systems should treat that state with the same access control and isolation they gave the original prompt.

Elias: That’s a big claim, Nadia. It suggests we need to manage the lifetime and access to this retained information, not just the input prompt itself.

Priya: And they tested this on hardware too, looking at power usage during public requests to see if personal attributes can be inferred from whole-device power consumption under certain conditions.

Nadia: They did a complete-read power experiment and found that a fixed execution schedule can actually recover a protected secret even when access controls are in place.

Elias: That’s interesting because it shows the leakage isn't just about what you see on the screen or what data is stored, but how the hardware computes things over time.

Priya: So, if I’m using an AI for something public, and someone is monitoring my device's power draw, they might be able to infer details about my private profile because of that retained state?

Nadia: That’s what they found. They pointed out access control responsibilities for serving systems and telemetry restrictions for the hardware operators because of this leakage.

Elias: And there’s a caveat on accelerator design too—they showed that even with a fixed control flow, if the switching between states depends on data, it can still disclose private facts.

Priya: So, what’s the immediate action here for folks building or using these personalized AI tools?

Nadia: The conclusion is that serving systems caching or offloading this recurrent state should treat it with the same isolation as the original prompt and clear it at session boundaries.

Elias: It pushes us to rethink how we manage state retention across different execution environments.

The paper's improvements: Tom: So, we're looking at how the authors suggest ways to fix or improve on their initial findings about leakage from that recurrent state in AI inference.

Nadia: They propose strengthening that concept of a confidentiality boundary for personalized AI services by making sure serving systems have real access control and isolation for that state.

Elias: That makes sense, Nadia. So if the system running the AI can't freely read or reuse that history, it should be treated as highly protected data, just like the original prompt it came from.

Priya: But how do you actually implement that control? Is this some software tweak, or is it something deeper in how the hardware handles those state transitions?

Nadia: They suggest they should clear or even recompute that recurrent state whenever there's a session boundary, like when a user starts a new conversation.

Elias: That’s the operational part. It moves from just analyzing the leakage to actually building systems that enforce those limits on state retention.

Priya: And for the measurement folks, what does this mean for how we test these systems? Do we need new ways to measure if that state is persisting after public dialogue?

Nadia: They developed a new methodology for tracing private information from the retained state all the way to the execution signals by transplanting that state between matched histories before a future query.

Elias: That’s building on what they did before, pushing it toward causal influence tracing. They want to quantify exactly how much that history is actually steering a later decision.

Priya: So it moves from just saying "it might leak" to providing a measurable way to prove "this specific piece of history caused that specific output."

Nadia: Right. And they also push the physical leakage experiment further, looking at how whole-device power usage can reveal private information even under certain access conditions.

Elias: The big takeaway here is that we’re moving toward a model where state management isn't just about keeping data safe in a database, but about controlling its entire lifecycle across software and hardware.

Priya: So, if we take this to the next level, what does that imply for how we design these AI services going forward?

Nadia: It suggests that serving systems need to be fundamentally designed with the isolation of the prompt in mind, treating that history as a sensitive resource from start to finish.

Elias: And for us working on cryptography and proofs, it means we have to consider how state persistence affects the parameters of our models.

Priya: It shifts the focus toward measuring not just what's in the final output, but what’s happening in the background computation that leads to it.

Conclusion: Tom: So, we’re wrapping up our look at "Privacy Leakage from Recurrent State in LLM Inference," summarizing what this research actually means for us right now.

Nadia: Basically, the paper shows that that hidden memory inside AI models is a real vector for leakage, and we need to treat it like a physical boundary.

Elias: It’s not just about the text you put in; it’s about how the model keeps track of everything from that prompt, and that history can tell us things later.

Priya: I just want to stress what this means for data—we can't trust just the final output when we don't know if a private fact was retained somewhere in the background state.

Nadia: Exactly. The paper gives us tools to trace that private information from the stored state all the way through to how it actually affects later decisions.

Elias: It gives us a framework for testing that causal influence by transplanting states between different conversations and measuring that change in decision margin.

Priya: And on the measurement side, they show that physical power usage during public requests can expose those private attributes under specific observation conditions.

Nadia: They conclude that serving systems need to treat this recurrent state with the same high level of access control as the original prompt and clear it when a session ends.

Elias: It pushes us to think about state management not just in terms of data security, but in terms of hardware and system isolation too.

Priya: It’s a strong warning that privacy isn't just about hiding what you show, but controlling the very life and computation of the information being stored.

Nadia: So, for anyone building these personalized AI services, you need to seriously look at how you manage that history and who gets access to it.

Elias: And for those of us working on hardware, it’s a reminder that even with fixed control flow, data-dependent switching can still expose secrets if we aren't careful.

Priya: It really solidifies the idea that privacy defense needs to be layered—software controls for state and physical observation for leakage.

Tom: That covers the main points of "Privacy Leakage from Recurrent State in LLM Inference."

Nadia: We’ve seen how persistent recurrent state can act as a boundary, and how physical power usage can reveal secrets under certain conditions.

Elias: The paper gives us concrete ways to trace that influence and test for causal effects on later decisions.

Priya: It makes it clear that we have to look beyond the final answer and examine the entire computation process.

Weimin Fu, Shikun Liu, Qiying Li, Yuntao Liu, Pan Li, Xiaolong Guo

Lehigh University · Georgia Institute of Technology

cs.CR

Submitted: 2026-11-11

Updated: 2026-11-11

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 100/100

The gist: Personalized LLM services place private facts and application instructions into a prompt, then retain representations of that context for later inference The information preserved by its learned…

Key concepts

Recurrent State
This is a fixed-size numerical summary of preceding tokens in an LLM session. It's history-dependent, meaning each update combines the current token with the previous state. This state retains information from earlier parts of the conversation, even as new tokens are processed.
Confidentiality Boundary
The retained recurrent state is established as a boundary for personalized LLM services. This means that controlling access to this specific state—through proper isolation and access control on serving systems—can protect the private facts and response rules stored within it from being exposed later.
State Transplantation
This methodology involves moving the recurrent state between two matched histories just before a future query. By comparing the resulting execution signals, researchers can trace exactly how the retained information causally influences subsequent decisions, quantifying its contribution to those later actions.
Data-Dependent Power Leakage
This refers to physical leakage where using specific hardware access patterns or switching between different computational paths during inference exposes private information. The study found that whole-device power and complete-read power can disclose private content under certain conditions, highlighting risks for accelerator designers.

Terminology

Summary

Personalized LLM services place private facts and application instructions into a prompt, then retain representations of that context for later inference The information preserved by its learned updates is implicit, and later computation can expose it even when an observer receives neither the private text nor the responses Experiments with released hybrid implementations show that a small state observation supports multiple predictions about private facts and response rules, and that personal attributes generalize to unseen authors Complete-read power recovers private facts under a fixed execution schedule

Pruning redundant reasoning tokens

The paper investigates privacy leakage from the recurrent state in latent-state LLMs by examining partial-state probing and physical observation of its use The methodology separates recoverable information, persistence through public dialogue, causal influence on later decisions, and leakage through power The core question is what information remains recoverable from that state and from the later computations that read it

How the training signal is built

The recurrent component summarizes preceding tokens in fixed-size numerical state, where each update combines that state with the current token representation and produces an output for later layers This state is a history-dependent latent representation whose dimensions are fixed by the model, but its values depend on the session Learned gates can also decay or overwrite information during continued execution Private prompt content can enter this state during prefill and remain available when a later public request is processed

Where the gains come from

The research establishes that retained recurrent state as a confidentiality boundary for personalized LLM services Private facts and response rules remain inferable after the conversation moves to public content, and state transplantation connects retained information to later decisions The study develops a methodology for tracing private information from retained state to execution signals Physical observation tests two consequences of using the state: data-dependent power during a complete recurrent read and changes in expert execution during public requests

Software probing tests whether restricted state observations predict private content, and retention measures persistence after public dialogue To measure contribution to later computation, the authors transplant recurrent state between matched histories before the future query Physical observability tests whether later use of retained state exposes private information without tensor access

The complete-read power test shows that a fixed execution schedule can recover a protected secret The GPU side channel experiment attributes the routing signal to the recurrent state rather than to the attention cache or convolution state These findings extend context confidentiality to the lifetime and execution of retained state

The results demonstrate that personal attributes can be inferred from whole-device power during public requests The study identifies access-control responsibilities for serving systems, telemetry restrictions for GPU operators, and data-dependent power leakage for accelerator designers

The paper concludes that serving systems that cache, offload, migrate, or share recurrent state should give it the access control and isolation of the prompt that produced it and clear or recompute it at session boundaries GPU cloud operators should treat whole-device parent power as tenant data Accelerator designers should note that fixed control flow removed schedule dependence while data-dependent switching still disclosed private facts, so a constant-schedule datapath does not protect confidential state

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Confidentiality Boundary Establishment: Establish retained recurrent state as a confidentiality boundary for personalized LLM services, meaning serving systems can extend prompt protection to state access, reuse, and lifetime by ensuring access control and isolation of the prompt that produced it.

  2. Causal Influence Tracing: Develop a methodology for tracing private information from retained state to execution signals by transplanting recurrent state between matched histories before the future query to measure its causal contribution to later decisions, quantified as the change in decision margin (donor logit minus source logit).

  3. Physical Leakage Identification: Demonstrate physical leakage under deployment-relevant controls by identifying that Complete-read power and whole-device GPU power disclose private content under their stated access conditions, allowing accelerator designers to note that fixed control flow removed schedule dependence while data-dependent switching still disclosed private facts.

Sources

Related papers