Privilege and confidentiality in generative AI workflows
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: We're looking at the paper: "Privilege and confidentiality in generative AI workflows".
Elias: Here's the gist of the paper. Generative AI systems store and process client data in three distinct ways: in the model’s parameters through training and memorisation, in the context window during a live session, and in knowledge databases for retrieval-augmented generation (RAG) Each mode creates different and often counter-intuitive risks to confidentiality and legal professional privilege, necessitating specific governance responses.
Nadia: The Three Modes of Data Processing The paper examines three distinct ways in which data is stored and processed in generative AI systems: (1) model memorisation; (2) the context window; and (3) retrieval-augmented generation (RAG).
Elias: The headline message is twofold, stating that risks are manageable only if one understands the difference between these three modes of data processing.
Nadia: Model Memorisation Risks Memorisation is described as "the most permanent of the three modes of data processing considered here, and the most familiar".
Paper summary: Nadia: So, to recap what we just heard, we talked about the three distinct ways client data gets stored in generative AI systems: memorisation in the model parameters, the context window during a live session, and knowledge databases for retrieval augmented generation or RAG.
Elias: And they argue that these three modes create different risks to confidentiality and privilege, so understanding them is what manages those risks. The paper emphasizes that you can't manage anything if you don't know the difference between data baked into the model, data transiting a provider’s infrastructure during a session, or data persisting in a vector database.
Priya: I think it’s important to focus on how these persistence layers affect legal practice because the risks aren't all equal across those modes. Memorisation is described as the most permanent and familiar mode of processing here.
Nadia: Right, so when we talk about memorisation, it means statistical patterns from training material get encoded into the model’s parameters, which can then be extracted using prompting techniques to get verbatim content. It assumes that just because you type something in doesn't mean it’s gone once it's used for training data.
Elias: That assumption is a big problem because there isn't a delete button on a trained AI model, and you can extract that stored data with the right prompt, so the user can’t undo that storage permanently.
Priya: And then we have the context window, which is like its short-term working memory during a live session. The risks there include disclosure to the model provider, shared context between users, and prompt injection attacks.
Nadia: There's also this practical issue called the long-input problem where materials bigger than that window get truncated or processed in batches, which leads to what they call context rot. That’s a risk specific to the session itself.
Elias: And then there’s retrieval augmented generation, or RAG. This mode uses a separate searchable knowledge store to handle long inputs, but it introduces specific risks like embedding persistence and embedding inversion which can mean the embeddings reveal almost as much information as the original text itself.
Conclusion: Nadia: So, looking at the whole paper "Privilege and confidentiality in generative AI workflows," Janeček and Melham are essentially arguing that the risks are manageable if you grasp these three distinct modes of data processing.
Elias: They claim that putting client material into a consumer-grade tool whose terms allow retention or training use is a breach of confidentiality, which puts privilege at serious risk, no matter what legal route a court ultimately prefers.
Priya: What this means for us in practice is that on limited disclosure authorities, an enterprise-grade deployment with no training use and minimal retention shouldn't itself imperil privilege if you have the right documentation proving it.
Nadia: And the paper points out that the persistence layers that lawyers don't see, like fine-tuned parameters, shared workspaces, and vector stores, are where those unintuitive breaches are actually going to happen.
Elias: So the final message is that understanding this difference between memorisation, context window use, and RAG stores is becoming part of what technological competence looks like in two thousand twenty-six <ref:2607.05479#pg1>.
Priya: It’s about moving the focus from the AI tool's brand or source code to those technical data-handling terms and controls behind them.
Nadia: Exactly. The paper's big implication is that the duty of confidentiality remains the anchor, but you have to account for all these different channels through which a breach can occur.
University of Bristol Law School · University of Oxford
cs.CR, cs.AI
Submitted: 2026-07-06
Updated: 2026-11-11
Comments: Version with minor amendments
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Importance score: 100/100
The gist: Generative AI systems store and process client data in three distinct ways: in the model’s parameters through training and memorisation, in the context window during a live session…
Key concepts
- Model Memorisation
- This is permanent storage where training material becomes encoded into the AI's core parameters. Data can be extracted verbatim using specific prompts. Since there is no 'delete' button on a trained model, this mode presents the most enduring risk to confidentiality.
- Context Window
- This represents the AI’s short-term working memory during an active session. Risks include disclosure to providers, shared context between users, and prompt injection. A practical issue is 'context rot,' where long inputs are truncated or processed in batches.
- Retrieval-Augmented Generation (RAG)
- RAG uses a separate searchable knowledge store to provide information during generation. Risks include embedding persistence, where data stays in the vector database, and semantic caching. Governance must manage these specific persistence layers separately from standard document management systems.
Terminology
Summary
Generative AI systems store and process client data in three distinct ways: in the model’s parameters through training and memorisation, in the context window during a live session, and in knowledge databases for retrieval-augmented generation (RAG) Each mode creates different and often counter-intuitive risks to confidentiality and legal professional privilege, necessitating specific governance responses. p. 0 Generative AI (GenAI) systems store and process client data in three distinct ways: in
the model’s parameters through training and memorisation, in the context window during a live
session, and in knowledge databases for retrieval-augmented generation (RAG). Each mode
creates different and often counter-intuitive risks to confidentiality and legal professional
privilege, and each calls for specific governance responses.Where the paper says it
The Three Modes of Data Processing
The paper examines three distinct ways in which data is stored and processed in generative AI systems: (1) model memorisation; (2) the context window; and (3) retrieval-augmented generation (RAG). The headline message is twofold, stating that risks are manageable only if one understands the difference between these three modes of data processing. p. 1 0)
2
The headline message is twofold. First, the risks are manageable, but only if one understands the
difference between the three modes of data processing.Where the paper says it
Model Memorisation Risks
Memorisation is described as Some specific, verbatim content can be encoded, distributed in non-trivial ways across
those parameters. And this can be extracted from the model with the right prompting techniques. A common assumption among law professionals is that a generative AI tool works like a search
engine: query in, result out, and nothing more. But in certain circumstances, what a user types in
can become part of the model’s permanent memory. And this can then be used as further training data in a subsequent
training run for the model. Once the stored data is used for model training, the user cannot undo
this: there is no delete button on a trained AI model.the most permanent of the three modes of data processing considered here, and the most familiar
. This occurs when statistical patterns in training material are encoded into the model’s parameters, which can be extracted with prompting techniques to emit verbatim content. p. 2Where the paper says it
Where the paper says it
Where the paper says it
Context Window Risks
The context window is described as But while the
model itself retains nothing, it does not mean the context window is risk-free. As will be shown,
at least three categories of risk arise: disclosure to the model provider; shared and persistent
context; and prompt injection.its short-term working memory, operating at inference time—during the live session
. Three categories of risk arise from this mode: disclosure to the model provider; shared and persistent context; and prompt injection. p. 5Where the paper says it
long-input problem,
where materials bigger than the window are truncated or processed in batches, leading to context rot
.
Retrieval-Augmented Generation Risks
RAG solves the long-input problem by using a separate, searchable knowledge store, but it introduces five specific risks. These risks include embedding persistence, embedding inversion, semantic caching, access control issues, and retrieval mismatch. p. 7 19 The leading vendors have, to
their credit, built their offerings around precisely the contractual architecture we describe in this
article: Harvey contractually guarantees that customer inputs, outputs and uploaded documents
are not used to train the underlying models, requires zero data retention from its upstream model
providers, and offers regional data residency, audit logs and customer-set retention policies;
Legora makes equivalent no-training and no-retention commitments, holds an ISO 42001 AIgovernance certification, and provides configurable retention with deletion of all data at contract
end. 20
But unless the organisation runs its own sealed-off instance of a model—for example via local
hosting and deployment of an AI model on its exclusively controlled computers—the data still
18 A Modarressi and others ‘NoLiMa: Long-Context Evaluation Beyond Literal Matching’ in Proceedings of the
42nd International Conference on Machine Learning (2025) PMLR:267:44554–44570; J Su and others ‘RoFormer:
Enhanced Transformer with Rotary Position Embedding’ (2024) 568 Neurocomputing 127063; D Aumiller and
others ‘Structural Text Segmentation of Legal Documents’ in Proceedings of the Eighteenth International
Conference on Artificial Intelligence and Law (2021) 2.Where the paper says it
Professional Stakes and Governance
The professional stakes involve the regulatory framework, negligence analysis, and jurisdiction extension. The duty of confidentiality is the anchor, and governance must account for training, context transit, shared workspaces, vector stores, caches—all channels through which a breach can occur. The standard of care is temporal and requires understanding documented failure modes. p. 0 Drawing on the first English and
American decisions to address privilege and generative AI, UK and Munir v Secretary of State
for the Home Department and United States v Heppner, on the orthodox privilege authorities
against which those decisions must be read, and on recent computer science research, we explain
the three modes of data storage and processing in terms accessible to practitioners and analyse
the legal consequences of each. We then situate the analysis within the regulatory framework
governing solicitors in England and Wales and within the ordinary principles of professional
negligence, arguing that the standard of effective information governance (and with it the
benchmark against which negligence and misconduct will be measured) is changing. Drawing on the first English and
American decisions to address privilege and generative AI, UK and Munir v Secretary of State
for the Home Department and United States v Heppner, on the orthodox privilege authorities
against which those decisions must be read, and on recent computer science research, we explain
the three modes of data storage and processing in terms accessible to practitioners and analyse
the legal consequences of each. We then situate the analysis within the regulatory framework
governing solicitors in England and Wales and within the ordinary principles of professional
negligence, arguing that the standard of effective information governance (and with it the
benchmark against which negligence and misconduct will be measured) is changing. Drawing on the first English and
American decisions to address privilege and generative AI, UK and Munir v Secretary of State
for the Home Department and United States v Heppner, on the orthodox privilege authorities
against which those decisions must be read, and on recent computer science research, we explain
the three modes of data storage and processing in terms accessible to practitioners and analyse
the legal consequences of each. We then situate the analysis within the regulatory framework
governing solicitors in England and Wales and within the ordinary principles of professional
negligence, arguing that the standard of effective information governance (and with it the
benchmark against which negligence and misconduct will be measured) is changing.Where the paper says it
Where the paper says it
Where the paper says it
Conclusion on Governance
The ultimate aim is to help legal services professionals understand salient data leakage risks and facilitate a more responsible deployment of GenAI on client data. The paper concludes that the persistence layers are where the unintuitive breaches will happen, and understanding these modes is increasingly part of what technological competence in 2026 looks like. p. 2 Figure 1
provides an overview of the three modes and their persistence characteristics. Figure 1: Three modes of data persistence in generative AI systems
Memorisation: The model’s permanent memory
How data becomes part of the model
Memorisation is the most permanent of the three modes of data processing considered here, and
the most familiar.Where the paper says it
Summary of Conclusions
(a) the doctrinal tests for privilege have not changed, and are unlikely to; what has changed is the factual substrate to which they apply, and the knowledge a competent practitioner must have to apply them.
(b) putting client material into a consumer-grade tool—one whose terms permit retention, training use and onward disclosure—is a breach of confidentiality and puts privilege at serious risk, whichever doctrinal route (loss of confidentiality, waiver, or destruction of the reasonable expectation) a court ultimately prefers.
(c) on the limited-disclosure authorities, a properly configured enterprise-grade deployment—no training use, zero or minimal retention, confidentiality obligations holding down the subprocessing chain, and documentation to prove it—should not in itself imperil privilege. p. 1 A note on terminology is needed at the outset, because loose labels have already caused judicial
confusion. We distinguish between consumer-grade tools—services used on standard public
terms, under which the provider may typically retain inputs, use them for model training (subject
to opt-outs), and disclose them in defined circumstances—and enterprise-grade deployments, in
which inputs are processed under a negotiated contractual framework that typically excludes
training use, restricts retention, and imposes confidentiality obligations on the provider.Where the paper says it
(d) the persistence layers that lawyers do not see—fine-tuned parameters, shared workspaces, vector stores, caches—are where the unintuitive breaches will happen, and they must be governed with the same rigour as the document management system.
(e) because these failure modes are now documented and publicised by the regulator, the professional bodies and the senior judiciary, the standard of reasonable care has already begun (and will no doubt continue) to absorb them. p. 1 A note on terminology is needed at the outset, because loose labels have already caused judicial
confusion. We distinguish between consumer-grade tools—services used on standard public
terms, under which the provider may typically retain inputs, use them for model training (subject
to opt-outs), and disclose them in defined circumstances—and enterprise-grade deployments, in
which inputs are processed under a negotiated contractual framework that typically excludes
training use, restricts retention, and imposes confidentiality obligations on the provider.Where the paper says it
Adjusted Governance Requirements
For memorisation, one must know each tool’s training policy, and do not fine-tune on client or personal data until erasure is technically solvable
. For the context window, practitioners should use enterprise-grade tools with strict privacy controls; check for context rot on long documents; disable or scope prompt caching; govern shared workspaces so that membership tracks matter access rights
. For RAG, one must disable or partition semantic caching; manage embedding persistence under explicit retention schedules
.
Regulatory and Professional Obligations
The duty of confidentiality is the anchor, and firms must have effective governance structures ensuring compliance across all data flows. The professional negligence analysis focuses on whether the loss falls within the risk that the relevant duty existed to guard against. p. 9 28 As Millett J observed in Price
Waterhouse v BCCI, privilege is normally an answer to compulsory disclosure, whereas
‘confidentiality alone affords protection only against voluntary disclosure without the consent of
the person to whom the duty of confidentiality is owed’. 29
Within that architecture, two doctrinally distinct routes can defeat the privilege: loss of
confidentiality, which destroys the precondition (once information is genuinely in the public
domain, there is nothing left for privilege to bite on), and waiver, ie, conduct of the privilegeholder treating the material in a manner inconsistent with maintaining the privilege. We distinguish between consumer-grade tools—services used on standard public
terms, under which the provider may typically retain inputs, use them for model training (subject
to opt-outs), and disclose them in defined circumstances—and enterprise-grade deployments, in
which inputs are processed under a negotiated contractual framework that typically excludes
training use, restricts retention, and imposes confidentiality obligations on the provider. The
1 UK and R (on the application of Munir) v Secretary of State for the Home Department (AI hallucinations;
supervision; Hamid) [2026] UKUT 81 (IAC) at [60] (hereafter Munir).Where the paper says it
Where the paper says it
Jurisdictional Scope
The structural claims are deliberately jurisdiction-neutral because they attach to the technology rather than the law, and wherever protection depends on demonstrable confidentiality, the three modes of data persistence determine what must be governed. The portable core is this: wherever protection depends on demonstrable confidentiality, the three modes of data persistence determine what must be governed. p. 0 We then situate the analysis within the regulatory framework
governing solicitors in England and Wales and within the ordinary principles of professional
negligence, arguing that the standard of effective information governance (and with it the
benchmark against which negligence and misconduct will be measured) is changing. Although
we write primarily for SRA-regulated practitioners, our data-governance analysis is framed to
extend to any jurisdiction in which the protection of privilege or professional secrecy depends on
demonstrable confidentiality. We then situate the analysis within the regulatory framework
governing solicitors in England and Wales and within the ordinary principles of professional
negligence, arguing that the standard of effective information governance (and with it the
benchmark against which negligence and misconduct will be measured) is changing. Although
we write primarily for SRA-regulated practitioners, our data-governance analysis is framed to
extend to any jurisdiction in which the protection of privilege or professional secrecy depends on
demonstrable confidentiality.Where the paper says it
Where the paper says it
Final Recommendations
The paper concludes that lawyers who understand the difference between a model’s parameters, its context window, and its retrieval stores can manage these risks much better and maintain the confidentiality of material processed by GenAI systems. This understanding is increasingly part of what technological competence in 2026 looks like. p. 3 A common assumption among law professionals is that a generative AI tool works like a search
engine: query in, result out, and nothing more. But in certain circumstances, what a user types in
can become part of the model’s permanent memory. A common assumption among law professionals is that a generative AI tool works like a search
engine: query in, result out, and nothing more. But in certain circumstances, what a user types in
can become part of the model’s permanent memory. A common assumption among law professionals is that a generative AI tool works like a search
engine: query in, result out, and nothing more. But in certain circumstances, what a user types in
can become part of the model’s permanent memory. 19 The leading vendors have, to
their credit, built their offerings around precisely the contractual architecture we describe in this
article: Harvey contractually guarantees that customer inputs, outputs and uploaded documents
are not used to train the underlying models, requires zero data retention from its upstream model
providers, and offers regional data residency, audit logs and customer-set retention policies;
Legora makes equivalent no-training and no-retention commitments, holds an ISO 42001 AIgovernance certification, and provides configurable retention with deletion of all data at contract
end. 20
But unless the organisation runs its own sealed-off instance of a model—for example via local
hosting and deployment of an AI model on its exclusively controlled computers—the data still
18 A Modarressi and others ‘NoLiMa: Long-Context Evaluation Beyond Literal Matching’ in Proceedings of the
42nd International Conference on Machine Learning (2025) PMLR:267:44554–44570; J Su and others ‘RoFormer:
Enhanced Transformer with Rotary Position Embedding’ (2024) 568 Neurocomputing 127063; D Aumiller and
others ‘Structural Text Segmentation of Legal Documents’ in Proceedings of the Eighteenth International
Conference on Artificial Intelligence and Law (2021) 2. 19 The leading vendors have, to
their credit, built their offerings around precisely the contractual architecture we describe in this
article: Harvey contractually guarantees that customer inputs, outputs and uploaded documents
are not used to train the underlying models, requires zero data retention from its upstream model
providers, and offers regional data residency, audit logs and customer-set retention policies;
Legora makes equivalent no-training and no-retention commitments, holds an ISO 42001 AIgovernance certification, and provides configurable retention with deletion of all data at contract
end. 20
But unless the organisation runs its own sealed-off instance of a model—for example via local
hosting and deployment of an AI model on its exclusively controlled computers—the data still
18 A Modarressi and others ‘NoLiMa: Long-Context Evaluation Beyond Literal Matching’ in Proceedings of the
42nd International Conference on Machine Learning (2025) PMLR:267:44554–44570; J Su and others ‘RoFormer:
Enhanced Transformer with Rotary Position Embedding’ (2024) 568 Neurocomputing 127063; D Aumiller and
others ‘Structural Text Segmentation of Legal Documents’ in Proceedings of the Eighteenth International
Conference on Artificial Intelligence and Law (2021) 2. 19 The leading vendors have, to
their credit, built their offerings around precisely the contractual architecture we describe in this
article: Harvey contractually guarantees that customer inputs, outputs and uploaded documents
are not used to train the underlying models, requires zero data retention from its upstream model
providers, and offers regional data residency, audit logs and customer-set retention policies;
Legora makes equivalent no-training and no-retention commitments, holds an ISO 42001 AIgovernance certification, and provides configurable retention with deletion of all data at contract
end. 20
But unless the organisation runs its own sealed-off instance of a model—for example via local
hosting and deployment of an AI model on its exclusively controlled computers—the data still
18 A Modarressi and others ‘NoLiMa: Long-Context Evaluation Beyond Literal Matching’ in Proceedings of the
42nd International Conference on Machine Learning (2025) PMLR:267:44554–44570; J Su and others ‘RoFormer:
Enhanced Transformer with Rotary Position Embedding’ (2024) 568 Neurocomputing 127063; D Aumiller and
others ‘Structural Text Segmentation of Legal Documents’ in Proceedings of the Eighteenth International
Conference on Artificial Intelligence and Law (2021) 2.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Concrete Governance Actions
The paper outlines specific actions across all three modes to manage risk. These include training staff to recognise failure modes, maintaining an incident-response protocol that includes self-reporting, and verifying retrieval sources on outputs that matter. The cost objection is countered by comparing governance costs against breach costs. p. 0 Each mode
creates different and often counter-intuitive risks to confidentiality and legal professional
privilege, and each calls for specific governance responses. Drawing on the first English and
American decisions to address privilege and generative AI, UK and Munir v Secretary of State
for the Home Department and United States v Heppner, on the orthodox privilege authorities
against which those decisions must be read, and on recent computer science research, we explain
the three modes of data storage and processing in terms accessible to practitioners and analyse
the legal consequences of each. Drawing on the first English and
American decisions to address privilege and generative AI, UK and Munir v Secretary of State
for the Home Department and United States v Heppner, on the orthodox privilege authorities
against which those decisions must be read, and on recent computer science research, we explain
the three modes of data storage and processing in terms accessible to practitioners and analyse
the legal consequences of each. We then situate the analysis within the regulatory framework
governing solicitors in England and Wales and within the ordinary principles of professional
negligence, arguing that the standard of effective information governance (and with it the
benchmark against which negligence and misconduct will be measured) is changing.Where the paper says it
Where the paper says it
The Core Principle
Three modes, three sets of risks, one overarching principle: wherever the client’s data goes, the lawyer’s professional obligations go with it. An AI system does not know by itself if it is processing privileged material, and a responsible data governance framework needs to account for that. p. 7 0)
8
The danger for legal practice is that this failure mode is invisible to traditional supervisory
review: the error does not resemble anything a supervising partner has been trained to catch and
yet this error can compromise the quality of the output. Moreover, if the system silently preprocesses the input material to fit it within the context window, the pre-processing itself may
pose risks to data confidentiality, depending on what technical methods are used to achieve the
compression goal. The doctrinal baseline
Confidential communications between lawyers and clients attract special protection—legal
professional privilege in England and Wales, attorney-client privilege in the United States,
professional secrecy in civilian systems. The labels and doctrinal details vary, but the underlying
principle is close to universal.Where the paper says it
Where the paper says it
Extensibility
The paper notes that the extendibility is visible in the early cases already, reaching Heppner by orthodox American means. The American Bar Association’s first ethics opinion requires lawyers to evaluate the risk that client information will be disclosed to or accessed by others. Consent in engagement letters will not do, and the EDPB’s position on trained models supplies the same analytical move for the EU’s data protection regime. p. 3 This is broadly the technical reason for the argument
that feeding data into consumer AI tools is like putting those data in the public domain. The European Data Protection Board has recognised the data protection significance of this
point: in Opinion 28/2024 it concluded that AI models trained on personal data cannot
automatically be regarded as anonymous; whether personal data can be extracted from a model
must be assessed case by case, having regard (among other things) to the likelihood of
regurgitation of training data.Where the paper says it
Final Summary of Governance
What does adjusted governance look like in concrete terms? For memorisation, know each tool’s training policy, and do not fine-tune on client or personal data until erasure is technically solvable. p. 4 How that works, and what risks it carries, is the subject
of the next section. The memorisation risk therefore reduces to two practical rules: first, know the tool’s training
policy before using it for any client or otherwise sensitive work; secondly, hold off fine-tuning
on client or personal data until the erasure problem has a workable technical solution. Moreover, if the system silently preprocesses the input material to fit it within the context window, the pre-processing itself may
pose risks to data confidentiality, depending on what technical methods are used to achieve the
compression goal. The practical responses are straightforward: when working with long documents, test the tool
with a reference to something near the end of the document; for very large document sets,
RAG—discussed in section 4—may be a safer architecture than squeezing everything into one
context window; and for enterprise tools it is best to ask directly for details about how the AI tool
in question tackles the context rot challenge. 19 The leading vendors have, to
their credit, built their offerings around precisely the contractual architecture we describe in this
article: Harvey contractually guarantees that customer inputs, outputs and uploaded documents
are not used to train the underlying models, requires zero data retention from its upstream model
providers, and offers regional data residency, audit logs and customer-set retention policies;
Legora makes equivalent no-training and no-retention commitments, holds an ISO 42001 AIgovernance certification, and provides configurable retention with deletion of all data at contract
end. 20
But unless the organisation runs its own sealed-off instance of a model—for example via local
hosting and deployment of an AI model on its exclusively controlled computers—the data still
18 A Modarressi and others ‘NoLiMa: Long-Context Evaluation Beyond Literal Matching’ in Proceedings of the
42nd International Conference on Machine Learning (2025) PMLR:267:44554–44570; J Su and others ‘RoFormer:
Enhanced Transformer with Rotary Position Embedding’ (2024) 568 Neurocomputing 127063; D Aumiller and
others ‘Structural Text Segmentation of Legal Documents’ in Proceedings of the Eighteenth International
Conference on Artificial Intelligence and Law (2021) 2.Where the paper says it
Where the paper says it
Where the paper says it
Final Conclusion
The ultimate aim is to help legal services professionals understand salient data leakage risks and thereby facilitate a more responsible deployment of GenAI on client data. The underlying challenge, in short, is not the artificial intelligence but the data: moving through systems that are not designed for transparency. Lawyers who understand the difference between a model’s parameters, its context window, and its retrieval stores can manage these risks much better and maintain the confidentiality of material processed by GenAI systems wherever they practise. p. 0 Generative AI (GenAI) systems store and process client data in three distinct ways: in
the model’s parameters through training and memorisation, in the context window during a live
session, and in knowledge databases for retrieval-augmented generation (RAG). Each mode
creates different and often counter-intuitive risks to confidentiality and legal professional
privilege, and each calls for specific governance responses. Generative AI (GenAI) systems store and process client data in three distinct ways: in
the model’s parameters through training and memorisation, in the context window during a live
session, and in knowledge databases for retrieval-augmented generation (RAG). Each mode
creates different and often counter-intuitive risks to confidentiality and legal professional
privilege, and each calls for specific governance responses.Where the paper says it
Where the paper says it
Final Takeaway
The ultimate aim is to help legal services professionals understand salient data leakage risks and thereby facilitate a more responsible deployment of GenAI on client data. The underlying challenge, in short, is not the artificial intelligence but the data: moving through systems that are not designed for transparency. Lawyers who understand the difference between a model’s parameters, its context window, and its retrieval stores can manage these risks much better and maintain the confidentiality of material processed by GenAI systems wherever they practise. The standard of care is temporal and requires understanding documented failure modes. This understanding is increasingly part of what technological competence in 2026 looks like.
Final Summary Sentence
Three modes, three sets of risks, one overarching principle: wherever the client’s data goes, the lawyer’s professional obligations go with it. The single most informative sentence about the paper's subject and finding is that an AI system does not know by itself if it is processing privileged material, and a responsible data governance framework needs to account for that.
Final Sentence
Three modes, three sets of risks, one overarching principle: wherever the client’s data goes, the lawyer’s professional obligations go with it. The single most informative sentence about the paper's subject and finding is that an AI system does not know by itself if it is processing privileged material, and a responsible data governance framework needs to account for that.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Model Memorisation Governance: Implement strict policies to prohibit fine-tuning on client or personal data until a workable technical solution for machine unlearning is available, as
honouring that right in respect of a trained model is technically very difficult, if not practically impossible.
-
Context Window Integrity Checks: Integrate mechanisms to check for context rot by testing the tool with reference to material near the end of long documents and disable or scope prompt caching on any RAG pipeline used for client work.
-
Vector Database Access Control: Ensure that retrieval databases are treated as part of the organization's system of record, subject to the same access controls, encryption standards, and retention schedules as traditional document management systems to prevent embedding inversion risks.
-
Prompt Injection Defenses: Treat AI-assisted analysis of externally sourced documents with skepticism and implement data cleaning or conversion before inputting material into secure systems to mitigate prompt injection risks where
the underlying weakness cannot be fully patched.
-
RAG Pipeline Isolation: Build per-matter RAG pipelines where each pipeline is populated only with documents the relevant people have the right to use, thereby limiting the blast radius of any leak and ensuring that
access controls follow documents into the vector store.
Abstract
Generative AI (GenAI) systems store and process client data in three distinct ways: in the model's parameters through training and memorisation, in the context window during a live session, and in knowledge databases for retrieval-augmented generation (RAG). Each mode creates different and often counter-intuitive risks to confidentiality and legal professional privilege, and each calls for specific governance responses. Drawing on the first English and American decisions to address privilege and generative AI, UK and Munir v Secretary of State for the Home Department and United States v Heppner, on the orthodox privilege authorities against which those decisions must be read, and on recent computer science research, we explain the three modes of data storage and processing in terms accessible to practitioners and analyse the legal consequences of each. We then situate the analysis within the regulatory framework governing solicitors in England and Wales and within the ordinary principles of professional negligence, arguing that the standard of effective information governance (and with it the benchmark against which negligence and misconduct will be measured) is changing. Although we write primarily for SRA-regulated practitioners, our data-governance analysis is framed to extend to any jurisdiction in which the protection of privilege or professional secrecy depends on demonstrable confidentiality. The ultimate aim of this article is to help legal services professionals understand salient data leakage risks in GenAI systems and thereby facilitate a more responsible deployment of GenAI on client data and other sensitive material.
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data
- Honeyval: A Comprehensive Evaluation Framework for LLM-powered HTTP Honeypots