Privilege and confidentiality in generative AI workflows

arXiv:2607.05479 · cs.CR, cs.AI · Submitted 2026-07-06 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "Privilege and confidentiality in generative AI workflows".

Elias: Here's the gist of the paper. Generative AI systems store and process client data in three distinct ways: in the model’s parameters through training and memorisation, in the context window during a live session, and in knowledge databases for retrieval-augmented generation (RAG) Each mode creates different and often counter-intuitive risks to confidentiality and legal professional privilege, necessitating specific governance responses.

Nadia: The Three Modes of Data Processing The paper examines three distinct ways in which data is stored and processed in generative AI systems: (1) model memorisation; (2) the context window; and (3) retrieval-augmented generation (RAG).

Elias: The headline message is twofold, stating that risks are manageable only if one understands the difference between these three modes of data processing.

Nadia: Model Memorisation Risks Memorisation is described as "the most permanent of the three modes of data processing considered here, and the most familiar".

Paper summary: Nadia: So, to recap what we just heard, we talked about the three distinct ways client data gets stored in generative AI systems: memorisation in the model parameters, the context window during a live session, and knowledge databases for retrieval augmented generation or RAG.

Elias: And they argue that these three modes create different risks to confidentiality and privilege, so understanding them is what manages those risks. The paper emphasizes that you can't manage anything if you don't know the difference between data baked into the model, data transiting a provider’s infrastructure during a session, or data persisting in a vector database.

Priya: I think it’s important to focus on how these persistence layers affect legal practice because the risks aren't all equal across those modes. Memorisation is described as the most permanent and familiar mode of processing here.

Nadia: Right, so when we talk about memorisation, it means statistical patterns from training material get encoded into the model’s parameters, which can then be extracted using prompting techniques to get verbatim content. It assumes that just because you type something in doesn't mean it’s gone once it's used for training data.

Elias: That assumption is a big problem because there isn't a delete button on a trained AI model, and you can extract that stored data with the right prompt, so the user can’t undo that storage permanently.

Priya: And then we have the context window, which is like its short-term working memory during a live session. The risks there include disclosure to the model provider, shared context between users, and prompt injection attacks.

Nadia: There's also this practical issue called the long-input problem where materials bigger than that window get truncated or processed in batches, which leads to what they call context rot. That’s a risk specific to the session itself.

Elias: And then there’s retrieval augmented generation, or RAG. This mode uses a separate searchable knowledge store to handle long inputs, but it introduces specific risks like embedding persistence and embedding inversion which can mean the embeddings reveal almost as much information as the original text itself.

Conclusion: Nadia: So, looking at the whole paper "Privilege and confidentiality in generative AI workflows," Janeček and Melham are essentially arguing that the risks are manageable if you grasp these three distinct modes of data processing.

Elias: They claim that putting client material into a consumer-grade tool whose terms allow retention or training use is a breach of confidentiality, which puts privilege at serious risk, no matter what legal route a court ultimately prefers.

Priya: What this means for us in practice is that on limited disclosure authorities, an enterprise-grade deployment with no training use and minimal retention shouldn't itself imperil privilege if you have the right documentation proving it.

Nadia: And the paper points out that the persistence layers that lawyers don't see, like fine-tuned parameters, shared workspaces, and vector stores, are where those unintuitive breaches are actually going to happen.

Elias: So the final message is that understanding this difference between memorisation, context window use, and RAG stores is becoming part of what technological competence looks like in two thousand twenty-six <ref:2607.05479#pg1>.

Priya: It’s about moving the focus from the AI tool's brand or source code to those technical data-handling terms and controls behind them.

Nadia: Exactly. The paper's big implication is that the duty of confidentiality remains the anchor, but you have to account for all these different channels through which a breach can occur.

University of Bristol Law School · University of Oxford

cs.CR, cs.AI

Submitted: 2026-07-06

Updated: 2026-11-11

Comments: Version with minor amendments

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 100/100

The gist: Generative AI systems store and process client data in three distinct ways: in the model’s parameters through training and memorisation, in the context window during a live session…

Key concepts

Model Memorisation
This is permanent storage where training material becomes encoded into the AI's core parameters. Data can be extracted verbatim using specific prompts. Since there is no 'delete' button on a trained model, this mode presents the most enduring risk to confidentiality.
Context Window
This represents the AI’s short-term working memory during an active session. Risks include disclosure to providers, shared context between users, and prompt injection. A practical issue is 'context rot,' where long inputs are truncated or processed in batches.
Retrieval-Augmented Generation (RAG)
RAG uses a separate searchable knowledge store to provide information during generation. Risks include embedding persistence, where data stays in the vector database, and semantic caching. Governance must manage these specific persistence layers separately from standard document management systems.

Terminology

Summary

Generative AI systems store and process client data in three distinct ways: in the model’s parameters through training and memorisation, in the context window during a live session, and in knowledge databases for retrieval-augmented generation (RAG) Each mode creates different and often counter-intuitive risks to confidentiality and legal professional privilege, necessitating specific governance responses. p. 0

Where the paper says it

Generative AI (GenAI) systems store and process client data in three distinct ways: in the model’s parameters through training and memorisation, in the context window during a live session, and in knowledge databases for retrieval-augmented generation (RAG). Each mode creates different and often counter-intuitive risks to confidentiality and legal professional privilege, and each calls for specific governance responses.

The Three Modes of Data Processing

The paper examines three distinct ways in which data is stored and processed in generative AI systems: (1) model memorisation; (2) the context window; and (3) retrieval-augmented generation (RAG). The headline message is twofold, stating that risks are manageable only if one understands the difference between these three modes of data processing. p. 1

Where the paper says it

0) 2 The headline message is twofold. First, the risks are manageable, but only if one understands the difference between the three modes of data processing.

Model Memorisation Risks

Memorisation is described as the most permanent of the three modes of data processing considered here, and the most familiar. This occurs when statistical patterns in training material are encoded into the model’s parameters, which can be extracted with prompting techniques to emit verbatim content. p. 2

Where the paper says it

Some specific, verbatim content can be encoded, distributed in non-trivial ways across those parameters. And this can be extracted from the model with the right prompting techniques.

A common assumption is that a generative AI tool works like a search engine, but data typed in can become part of the model’s permanent memory if stored as verbatim text. p. 3
Where the paper says it

A common assumption among law professionals is that a generative AI tool works like a search engine: query in, result out, and nothing more. But in certain circumstances, what a user types in can become part of the model’s permanent memory.

Once stored data is used for training, the user cannot undo this because there is no delete button on a trained AI model. p. 3
Where the paper says it

And this can then be used as further training data in a subsequent training run for the model. Once the stored data is used for model training, the user cannot undo this: there is no delete button on a trained AI model.

Context Window Risks

The context window is described as its short-term working memory, operating at inference time—during the live session. Three categories of risk arise from this mode: disclosure to the model provider; shared and persistent context; and prompt injection. p. 5

Where the paper says it

But while the model itself retains nothing, it does not mean the context window is risk-free. As will be shown, at least three categories of risk arise: disclosure to the model provider; shared and persistent context; and prompt injection.

A practical limitation is the long-input problem, where materials bigger than the window are truncated or processed in batches, leading to context rot.

Retrieval-Augmented Generation Risks

RAG solves the long-input problem by using a separate, searchable knowledge store, but it introduces five specific risks. These risks include embedding persistence, embedding inversion, semantic caching, access control issues, and retrieval mismatch. p. 7

Where the paper says it

19 The leading vendors have, to their credit, built their offerings around precisely the contractual architecture we describe in this article: Harvey contractually guarantees that customer inputs, outputs and uploaded documents are not used to train the underlying models, requires zero data retention from its upstream model providers, and offers regional data residency, audit logs and customer-set retention policies; Legora makes equivalent no-training and no-retention commitments, holds an ISO 42001 AIgovernance certification, and provides configurable retention with deletion of all data at contract end. 20 But unless the organisation runs its own sealed-off instance of a model—for example via local hosting and deployment of an AI model on its exclusively controlled computers—the data still 18 A Modarressi and others ‘NoLiMa: Long-Context Evaluation Beyond Literal Matching’ in Proceedings of the 42nd International Conference on Machine Learning (2025) PMLR:267:44554–44570; J Su and others ‘RoFormer: Enhanced Transformer with Rotary Position Embedding’ (2024) 568 Neurocomputing 127063; D Aumiller and others ‘Structural Text Segmentation of Legal Documents’ in Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law (2021) 2.

Embedding persistence means data persists in the vector database until explicitly deleted, and embedding inversion shows that embeddings can reveal almost as much as the text itself.

Professional Stakes and Governance

The professional stakes involve the regulatory framework, negligence analysis, and jurisdiction extension. The duty of confidentiality is the anchor, and governance must account for training, context transit, shared workspaces, vector stores, caches—all channels through which a breach can occur. The standard of care is temporal and requires understanding documented failure modes. p. 0

Where the paper says it

Drawing on the first English and American decisions to address privilege and generative AI, UK and Munir v Secretary of State for the Home Department and United States v Heppner, on the orthodox privilege authorities against which those decisions must be read, and on recent computer science research, we explain the three modes of data storage and processing in terms accessible to practitioners and analyse the legal consequences of each. We then situate the analysis within the regulatory framework governing solicitors in England and Wales and within the ordinary principles of professional negligence, arguing that the standard of effective information governance (and with it the benchmark against which negligence and misconduct will be measured) is changing.

p. 0
Where the paper says it

Drawing on the first English and American decisions to address privilege and generative AI, UK and Munir v Secretary of State for the Home Department and United States v Heppner, on the orthodox privilege authorities against which those decisions must be read, and on recent computer science research, we explain the three modes of data storage and processing in terms accessible to practitioners and analyse the legal consequences of each. We then situate the analysis within the regulatory framework governing solicitors in England and Wales and within the ordinary principles of professional negligence, arguing that the standard of effective information governance (and with it the benchmark against which negligence and misconduct will be measured) is changing.

p. 0
Where the paper says it

Drawing on the first English and American decisions to address privilege and generative AI, UK and Munir v Secretary of State for the Home Department and United States v Heppner, on the orthodox privilege authorities against which those decisions must be read, and on recent computer science research, we explain the three modes of data storage and processing in terms accessible to practitioners and analyse the legal consequences of each. We then situate the analysis within the regulatory framework governing solicitors in England and Wales and within the ordinary principles of professional negligence, arguing that the standard of effective information governance (and with it the benchmark against which negligence and misconduct will be measured) is changing.

Conclusion on Governance

The ultimate aim is to help legal services professionals understand salient data leakage risks and facilitate a more responsible deployment of GenAI on client data. The paper concludes that the persistence layers are where the unintuitive breaches will happen, and understanding these modes is increasingly part of what technological competence in 2026 looks like. p. 2

Where the paper says it

Figure 1 provides an overview of the three modes and their persistence characteristics. Figure 1: Three modes of data persistence in generative AI systems Memorisation: The model’s permanent memory How data becomes part of the model Memorisation is the most permanent of the three modes of data processing considered here, and the most familiar.

Summary of Conclusions

(a) the doctrinal tests for privilege have not changed, and are unlikely to; what has changed is the factual substrate to which they apply, and the knowledge a competent practitioner must have to apply them.

(b) putting client material into a consumer-grade tool—one whose terms permit retention, training use and onward disclosure—is a breach of confidentiality and puts privilege at serious risk, whichever doctrinal route (loss of confidentiality, waiver, or destruction of the reasonable expectation) a court ultimately prefers.

(c) on the limited-disclosure authorities, a properly configured enterprise-grade deployment—no training use, zero or minimal retention, confidentiality obligations holding down the subprocessing chain, and documentation to prove it—should not in itself imperil privilege. p. 1

Where the paper says it

A note on terminology is needed at the outset, because loose labels have already caused judicial confusion. We distinguish between consumer-grade tools—services used on standard public terms, under which the provider may typically retain inputs, use them for model training (subject to opt-outs), and disclose them in defined circumstances—and enterprise-grade deployments, in which inputs are processed under a negotiated contractual framework that typically excludes training use, restricts retention, and imposes confidentiality obligations on the provider.

(d) the persistence layers that lawyers do not see—fine-tuned parameters, shared workspaces, vector stores, caches—are where the unintuitive breaches will happen, and they must be governed with the same rigour as the document management system.

(e) because these failure modes are now documented and publicised by the regulator, the professional bodies and the senior judiciary, the standard of reasonable care has already begun (and will no doubt continue) to absorb them. p. 1

Where the paper says it

A note on terminology is needed at the outset, because loose labels have already caused judicial confusion. We distinguish between consumer-grade tools—services used on standard public terms, under which the provider may typically retain inputs, use them for model training (subject to opt-outs), and disclose them in defined circumstances—and enterprise-grade deployments, in which inputs are processed under a negotiated contractual framework that typically excludes training use, restricts retention, and imposes confidentiality obligations on the provider.

Adjusted Governance Requirements

For memorisation, one must know each tool’s training policy, and do not fine-tune on client or personal data until erasure is technically solvable. For the context window, practitioners should use enterprise-grade tools with strict privacy controls; check for context rot on long documents; disable or scope prompt caching; govern shared workspaces so that membership tracks matter access rights. For RAG, one must disable or partition semantic caching; manage embedding persistence under explicit retention schedules.

Regulatory and Professional Obligations

The duty of confidentiality is the anchor, and firms must have effective governance structures ensuring compliance across all data flows. The professional negligence analysis focuses on whether the loss falls within the risk that the relevant duty existed to guard against. p. 9

Where the paper says it

28 As Millett J observed in Price Waterhouse v BCCI, privilege is normally an answer to compulsory disclosure, whereas ‘confidentiality alone affords protection only against voluntary disclosure without the consent of the person to whom the duty of confidentiality is owed’. 29 Within that architecture, two doctrinally distinct routes can defeat the privilege: loss of confidentiality, which destroys the precondition (once information is genuinely in the public domain, there is nothing left for privilege to bite on), and waiver, ie, conduct of the privilegeholder treating the material in a manner inconsistent with maintaining the privilege.

Self-reporting material as seen in Munir shows the practical value of taking reporting obligations seriously. p. 1
Where the paper says it

We distinguish between consumer-grade tools—services used on standard public terms, under which the provider may typically retain inputs, use them for model training (subject to opt-outs), and disclose them in defined circumstances—and enterprise-grade deployments, in which inputs are processed under a negotiated contractual framework that typically excludes training use, restricts retention, and imposes confidentiality obligations on the provider. The 1 UK and R (on the application of Munir) v Secretary of State for the Home Department (AI hallucinations; supervision; Hamid) [2026] UKUT 81 (IAC) at [60] (hereafter Munir).

Jurisdictional Scope

The structural claims are deliberately jurisdiction-neutral because they attach to the technology rather than the law, and wherever protection depends on demonstrable confidentiality, the three modes of data persistence determine what must be governed. The portable core is this: wherever protection depends on demonstrable confidentiality, the three modes of data persistence determine what must be governed. p. 0

Where the paper says it

We then situate the analysis within the regulatory framework governing solicitors in England and Wales and within the ordinary principles of professional negligence, arguing that the standard of effective information governance (and with it the benchmark against which negligence and misconduct will be measured) is changing. Although we write primarily for SRA-regulated practitioners, our data-governance analysis is framed to extend to any jurisdiction in which the protection of privilege or professional secrecy depends on demonstrable confidentiality.

p. 0
Where the paper says it

We then situate the analysis within the regulatory framework governing solicitors in England and Wales and within the ordinary principles of professional negligence, arguing that the standard of effective information governance (and with it the benchmark against which negligence and misconduct will be measured) is changing. Although we write primarily for SRA-regulated practitioners, our data-governance analysis is framed to extend to any jurisdiction in which the protection of privilege or professional secrecy depends on demonstrable confidentiality.

Final Recommendations

The paper concludes that lawyers who understand the difference between a model’s parameters, its context window, and its retrieval stores can manage these risks much better and maintain the confidentiality of material processed by GenAI systems. This understanding is increasingly part of what technological competence in 2026 looks like. p. 3

Where the paper says it

A common assumption among law professionals is that a generative AI tool works like a search engine: query in, result out, and nothing more. But in certain circumstances, what a user types in can become part of the model’s permanent memory.

p. 3
Where the paper says it

A common assumption among law professionals is that a generative AI tool works like a search engine: query in, result out, and nothing more. But in certain circumstances, what a user types in can become part of the model’s permanent memory.

p. 3
Where the paper says it

A common assumption among law professionals is that a generative AI tool works like a search engine: query in, result out, and nothing more. But in certain circumstances, what a user types in can become part of the model’s permanent memory.

The underlying challenge, in short, is not the artificial intelligence but the data: moving through systems that are not designed for transparency. p. 7
Where the paper says it

19 The leading vendors have, to their credit, built their offerings around precisely the contractual architecture we describe in this article: Harvey contractually guarantees that customer inputs, outputs and uploaded documents are not used to train the underlying models, requires zero data retention from its upstream model providers, and offers regional data residency, audit logs and customer-set retention policies; Legora makes equivalent no-training and no-retention commitments, holds an ISO 42001 AIgovernance certification, and provides configurable retention with deletion of all data at contract end. 20 But unless the organisation runs its own sealed-off instance of a model—for example via local hosting and deployment of an AI model on its exclusively controlled computers—the data still 18 A Modarressi and others ‘NoLiMa: Long-Context Evaluation Beyond Literal Matching’ in Proceedings of the 42nd International Conference on Machine Learning (2025) PMLR:267:44554–44570; J Su and others ‘RoFormer: Enhanced Transformer with Rotary Position Embedding’ (2024) 568 Neurocomputing 127063; D Aumiller and others ‘Structural Text Segmentation of Legal Documents’ in Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law (2021) 2.

p. 7
Where the paper says it

19 The leading vendors have, to their credit, built their offerings around precisely the contractual architecture we describe in this article: Harvey contractually guarantees that customer inputs, outputs and uploaded documents are not used to train the underlying models, requires zero data retention from its upstream model providers, and offers regional data residency, audit logs and customer-set retention policies; Legora makes equivalent no-training and no-retention commitments, holds an ISO 42001 AIgovernance certification, and provides configurable retention with deletion of all data at contract end. 20 But unless the organisation runs its own sealed-off instance of a model—for example via local hosting and deployment of an AI model on its exclusively controlled computers—the data still 18 A Modarressi and others ‘NoLiMa: Long-Context Evaluation Beyond Literal Matching’ in Proceedings of the 42nd International Conference on Machine Learning (2025) PMLR:267:44554–44570; J Su and others ‘RoFormer: Enhanced Transformer with Rotary Position Embedding’ (2024) 568 Neurocomputing 127063; D Aumiller and others ‘Structural Text Segmentation of Legal Documents’ in Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law (2021) 2.

p. 7
Where the paper says it

19 The leading vendors have, to their credit, built their offerings around precisely the contractual architecture we describe in this article: Harvey contractually guarantees that customer inputs, outputs and uploaded documents are not used to train the underlying models, requires zero data retention from its upstream model providers, and offers regional data residency, audit logs and customer-set retention policies; Legora makes equivalent no-training and no-retention commitments, holds an ISO 42001 AIgovernance certification, and provides configurable retention with deletion of all data at contract end. 20 But unless the organisation runs its own sealed-off instance of a model—for example via local hosting and deployment of an AI model on its exclusively controlled computers—the data still 18 A Modarressi and others ‘NoLiMa: Long-Context Evaluation Beyond Literal Matching’ in Proceedings of the 42nd International Conference on Machine Learning (2025) PMLR:267:44554–44570; J Su and others ‘RoFormer: Enhanced Transformer with Rotary Position Embedding’ (2024) 568 Neurocomputing 127063; D Aumiller and others ‘Structural Text Segmentation of Legal Documents’ in Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law (2021) 2.

Concrete Governance Actions

The paper outlines specific actions across all three modes to manage risk. These include training staff to recognise failure modes, maintaining an incident-response protocol that includes self-reporting, and verifying retrieval sources on outputs that matter. The cost objection is countered by comparing governance costs against breach costs. p. 0

Where the paper says it

Each mode creates different and often counter-intuitive risks to confidentiality and legal professional privilege, and each calls for specific governance responses. Drawing on the first English and American decisions to address privilege and generative AI, UK and Munir v Secretary of State for the Home Department and United States v Heppner, on the orthodox privilege authorities against which those decisions must be read, and on recent computer science research, we explain the three modes of data storage and processing in terms accessible to practitioners and analyse the legal consequences of each.

The standard of care accommodates proportionate, documented risk decisions rather than ignorance of where the data goes. p. 0
Where the paper says it

Drawing on the first English and American decisions to address privilege and generative AI, UK and Munir v Secretary of State for the Home Department and United States v Heppner, on the orthodox privilege authorities against which those decisions must be read, and on recent computer science research, we explain the three modes of data storage and processing in terms accessible to practitioners and analyse the legal consequences of each. We then situate the analysis within the regulatory framework governing solicitors in England and Wales and within the ordinary principles of professional negligence, arguing that the standard of effective information governance (and with it the benchmark against which negligence and misconduct will be measured) is changing.

The Core Principle

Three modes, three sets of risks, one overarching principle: wherever the client’s data goes, the lawyer’s professional obligations go with it. An AI system does not know by itself if it is processing privileged material, and a responsible data governance framework needs to account for that. p. 7

Where the paper says it

0) 8 The danger for legal practice is that this failure mode is invisible to traditional supervisory review: the error does not resemble anything a supervising partner has been trained to catch and yet this error can compromise the quality of the output. Moreover, if the system silently preprocesses the input material to fit it within the context window, the pre-processing itself may pose risks to data confidentiality, depending on what technical methods are used to achieve the compression goal.

This overarching principle dictates the required level of rigour in managing all data flows. p. 8
Where the paper says it

The doctrinal baseline Confidential communications between lawyers and clients attract special protection—legal professional privilege in England and Wales, attorney-client privilege in the United States, professional secrecy in civilian systems. The labels and doctrinal details vary, but the underlying principle is close to universal.

Extensibility

The paper notes that the extendibility is visible in the early cases already, reaching Heppner by orthodox American means. The American Bar Association’s first ethics opinion requires lawyers to evaluate the risk that client information will be disclosed to or accessed by others. Consent in engagement letters will not do, and the EDPB’s position on trained models supplies the same analytical move for the EU’s data protection regime. p. 3

Where the paper says it

This is broadly the technical reason for the argument that feeding data into consumer AI tools is like putting those data in the public domain. The European Data Protection Board has recognised the data protection significance of this point: in Opinion 28/2024 it concluded that AI models trained on personal data cannot automatically be regarded as anonymous; whether personal data can be extracted from a model must be assessed case by case, having regard (among other things) to the likelihood of regurgitation of training data.

The portable core is this: wherever protection depends on demonstrable confidentiality, the three modes of data persistence determine what must be governed.

Final Summary of Governance

What does adjusted governance look like in concrete terms? For memorisation, know each tool’s training policy, and do not fine-tune on client or personal data until erasure is technically solvable. p. 4

Where the paper says it

How that works, and what risks it carries, is the subject of the next section. The memorisation risk therefore reduces to two practical rules: first, know the tool’s training policy before using it for any client or otherwise sensitive work; secondly, hold off fine-tuning on client or personal data until the erasure problem has a workable technical solution.

For the context window, use enterprise-grade tools with strict privacy controls; check for context rot on long documents; disable or scope prompt caching; govern shared workspaces so that membership tracks matter access rights. p. 7
Where the paper says it

Moreover, if the system silently preprocesses the input material to fit it within the context window, the pre-processing itself may pose risks to data confidentiality, depending on what technical methods are used to achieve the compression goal. The practical responses are straightforward: when working with long documents, test the tool with a reference to something near the end of the document; for very large document sets, RAG—discussed in section 4—may be a safer architecture than squeezing everything into one context window; and for enterprise tools it is best to ask directly for details about how the AI tool in question tackles the context rot challenge.

For RAG, disable or partition semantic caching; manage embedding persistence under explicit retention schedules. p. 7
Where the paper says it

19 The leading vendors have, to their credit, built their offerings around precisely the contractual architecture we describe in this article: Harvey contractually guarantees that customer inputs, outputs and uploaded documents are not used to train the underlying models, requires zero data retention from its upstream model providers, and offers regional data residency, audit logs and customer-set retention policies; Legora makes equivalent no-training and no-retention commitments, holds an ISO 42001 AIgovernance certification, and provides configurable retention with deletion of all data at contract end. 20 But unless the organisation runs its own sealed-off instance of a model—for example via local hosting and deployment of an AI model on its exclusively controlled computers—the data still 18 A Modarressi and others ‘NoLiMa: Long-Context Evaluation Beyond Literal Matching’ in Proceedings of the 42nd International Conference on Machine Learning (2025) PMLR:267:44554–44570; J Su and others ‘RoFormer: Enhanced Transformer with Rotary Position Embedding’ (2024) 568 Neurocomputing 127063; D Aumiller and others ‘Structural Text Segmentation of Legal Documents’ in Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law (2021) 2.

And across all three: train staff to recognise these failure modes, and maintain an incident-response protocol that includes self-reporting. The paper's ultimate aim is to help legal services professionals understand salient data leakage risks and thereby facilitate a more responsible deployment of GenAI on client data.

Final Conclusion

The ultimate aim is to help legal services professionals understand salient data leakage risks and thereby facilitate a more responsible deployment of GenAI on client data. The underlying challenge, in short, is not the artificial intelligence but the data: moving through systems that are not designed for transparency. Lawyers who understand the difference between a model’s parameters, its context window, and its retrieval stores can manage these risks much better and maintain the confidentiality of material processed by GenAI systems wherever they practise. p. 0

Where the paper says it

Generative AI (GenAI) systems store and process client data in three distinct ways: in the model’s parameters through training and memorisation, in the context window during a live session, and in knowledge databases for retrieval-augmented generation (RAG). Each mode creates different and often counter-intuitive risks to confidentiality and legal professional privilege, and each calls for specific governance responses.

p. 0
Where the paper says it

Generative AI (GenAI) systems store and process client data in three distinct ways: in the model’s parameters through training and memorisation, in the context window during a live session, and in knowledge databases for retrieval-augmented generation (RAG). Each mode creates different and often counter-intuitive risks to confidentiality and legal professional privilege, and each calls for specific governance responses.

The standard of care is temporal and requires understanding documented failure modes. This understanding is increasingly part of what technological competence in 2026 looks like.

Final Takeaway

The ultimate aim is to help legal services professionals understand salient data leakage risks and thereby facilitate a more responsible deployment of GenAI on client data. The underlying challenge, in short, is not the artificial intelligence but the data: moving through systems that are not designed for transparency. Lawyers who understand the difference between a model’s parameters, its context window, and its retrieval stores can manage these risks much better and maintain the confidentiality of material processed by GenAI systems wherever they practise. The standard of care is temporal and requires understanding documented failure modes. This understanding is increasingly part of what technological competence in 2026 looks like.

Final Summary Sentence

Three modes, three sets of risks, one overarching principle: wherever the client’s data goes, the lawyer’s professional obligations go with it. The single most informative sentence about the paper's subject and finding is that an AI system does not know by itself if it is processing privileged material, and a responsible data governance framework needs to account for that.

Final Sentence

Three modes, three sets of risks, one overarching principle: wherever the client’s data goes, the lawyer’s professional obligations go with it. The single most informative sentence about the paper's subject and finding is that an AI system does not know by itself if it is processing privileged material, and a responsible data governance framework needs to account for that.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Model Memorisation Governance: Implement strict policies to prohibit fine-tuning on client or personal data until a workable technical solution for machine unlearning is available, as honouring that right in respect of a trained model is technically very difficult, if not practically impossible.

  2. Context Window Integrity Checks: Integrate mechanisms to check for context rot by testing the tool with reference to material near the end of long documents and disable or scope prompt caching on any RAG pipeline used for client work.

  3. Vector Database Access Control: Ensure that retrieval databases are treated as part of the organization's system of record, subject to the same access controls, encryption standards, and retention schedules as traditional document management systems to prevent embedding inversion risks.

  4. Prompt Injection Defenses: Treat AI-assisted analysis of externally sourced documents with skepticism and implement data cleaning or conversion before inputting material into secure systems to mitigate prompt injection risks where the underlying weakness cannot be fully patched.

  5. RAG Pipeline Isolation: Build per-matter RAG pipelines where each pipeline is populated only with documents the relevant people have the right to use, thereby limiting the blast radius of any leak and ensuring that access controls follow documents into the vector store.

Abstract

Generative AI (GenAI) systems store and process client data in three distinct ways: in the model's parameters through training and memorisation, in the context window during a live session, and in knowledge databases for retrieval-augmented generation (RAG). Each mode creates different and often counter-intuitive risks to confidentiality and legal professional privilege, and each calls for specific governance responses. Drawing on the first English and American decisions to address privilege and generative AI, UK and Munir v Secretary of State for the Home Department and United States v Heppner, on the orthodox privilege authorities against which those decisions must be read, and on recent computer science research, we explain the three modes of data storage and processing in terms accessible to practitioners and analyse the legal consequences of each. We then situate the analysis within the regulatory framework governing solicitors in England and Wales and within the ordinary principles of professional negligence, arguing that the standard of effective information governance (and with it the benchmark against which negligence and misconduct will be measured) is changing. Although we write primarily for SRA-regulated practitioners, our data-governance analysis is framed to extend to any jurisdiction in which the protection of privilege or professional secrecy depends on demonstrable confidentiality. The ultimate aim of this article is to help legal services professionals understand salient data leakage risks in GenAI systems and thereby facilitate a more responsible deployment of GenAI on client data and other sensitive material.

Related papers