Representation Drift in Continual Learning: LoRA versus N-gram Memory

arXiv:2610.14921 · cs.LG · Submitted 2026-10-10 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Representation Drift in Continual Learning".

Jane: The gist The base stays fixed and every change is local and reversible,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're looking at this paper called "Representation Drift in Continual Learning: LoRA versus N-gram Memory." It's about how a language model that keeps learning online doesn't slowly lose its basic abilities. The authors are asking which part of the model should absorb new knowledge when it updates continually by measuring how each choice changes the model's internal representations.

Jane: That sounds really important because we all use these models more and more, and if they start forgetting things, that could cause big problems for how we build and use AI systems. So what exactly are they testing here?

Tom: They set up a forty-eight-layer mixture-of-experts model with a native pretrained N-gram memory <ref:2610.14921#pg1,a 48-layer mixture-of-experts model with a native pretrained N>. They then train seven different tasks using three different update mechanisms. Those mechanisms are updating only the N-gram memory table, continually training one shared LoRA adapter, and training a fresh LoRA per task that gets merged into the backbone.

Lu: The core idea here is to capture fixed-context representations at every layer and every task boundary. They measure these representations against two references: general math and code inputs for big-K drift, and learned task inputs for small-k drift (<ref:2610.14921#pg2>).

Meng: When I think about this from a deployment side, I'm thinking about what kind of memory mechanism actually scales well. The authors are comparing the N-gram approach against two LoRA strategies under matched data exposure to see where new knowledge should be written for an AI that keeps learning online.

Lalam: My literature sweep points toward representation drift as a key concept, and this paper directly addresses how that drift manifests in different update strategies. The paper claims that updating only the N-gram memory changes the base model’s representations of general math and code inputs by twenty-six percent/fifteen percent relative RMS, against sixty-four percent/thirty-five percent for merged LoRA and ninety-seven percent/thirty-nine percent for shared LoRA (<ref:2610.14921#pg1>).

Jane: Those numbers are what grab my attention right now. So, the paper suggests that updating just the N-gram memory causes a big-K drift of twenty-six percent to fifteen percent on math and code inputs, which is much smaller than the drift seen with LoRA, which is sixty-four percent to ninety-seven percent <ref:2610.14921#pg1>. What does that actually mean for us?

Tom: It means that updating only the N-gram memory leaves the base model's representations of general math and code inputs nearly untouched. And this drift doesn't grow with the number of tasks; it reaches twenty-five percent after just the first task and stays there for all seven (<ref:2610.14921#pg1>).

Lu: That finding is interesting because generic behavior stays essentially unchanged, meaning no code problem gets lost, which is something we really want. But they also found that the per-round changes cancel out in the N-gram case, with a realized squared drift being fifteen percent of the sum of per-round squared changes, compared to forty percent to fifty-six percent for LoRA (<ref:2610.14921#pg2>).

Paper summary: Meng: From an engineering standpoint, that cancellation sounds good for stability, but they also point out a cost: the cost is interference among stored tasks rather than damage to the base model itself because it acquires difficult tasks more weakly and its stored tasks interfere (<ref:2610.14921#pg3>).

Jane: That's a subtle distinction. So, for N-gram memory, the interference happens between existing stored tasks, not necessarily eroding the general knowledge base as much as those LoRA methods do. But if it does interfere, how does that look in terms of representation drift?

Tom: They define representation drift as key drift Kt minus Ki at each layer. The paper shows that every output change comes from this key drift, and in particular at the head (<ref:2610.14921#pg3>). Small-ki drift compares each learned task’s inputs with the checkpoint at which that task had just been learned, measuring what happens to acquired knowledge afterwards.

Lalam: The paper connects this small-ki drift directly to retention loss through a bound: ∆NLLt i is less than or equal to sqrt(two) times the norm of U times the norm of e t i (<ref:2610.14921#pg3>). This means that even with small key drift, there's still a connection between that drift and how much knowledge you lose when you add new things.

Lu: And they show that for N-gram memory, an input whose addressed rows were never rewritten is exactly unchanged, which is why generic drift stays small (<ref:2610.14921#pg3>). But old-task inputs share rows with new tasks, leading to large small-ki drift because those old inputs share rows with later tasks that rewrite them (<ref:2610.14921#pg8>).

Meng: So for a system running on real traffic, that means the performance might degrade specifically for the data patterns that overlap between old and new tasks, even if the overall math capabilities look fine. That sounds like a specific failure mode we'd have to debug at scale.

Jane: What does that tell us about the architecture? The paper says N-gram memory has one source term, which is the injection change ∆J(x) of Equation (twenty-eight) carried through the frozen suffix (<ref:2610.14921#pg8>). If ∆T R(x) is zero on the full causal prefix of an old input, its entire forward pass is unchanged (<ref:2610.14921#pg8>).

Tom: That single source term idea simplifies things a lot because it means we can track the drift down to that injection change. It's small on generic inputs—the stuff that doesn't overlap with new tasks—but large on old-task inputs which read rows later tasks rewrite (<ref:2610.14921#pg8>).

Lu: The layer profile they show is pretty telling, showing that on old-task inputs, the N-gram drift reaches thirty-one percent within seven layers of reading, while generic inputs at the same depth only move by four percent (<ref:2610.14921#pg8>). That's a significant difference in how sensitive the model is to new knowledge depending on what it's processing.

Meng: So, if we were building a system, that suggests we might need to be very careful about how we structure our memory retrieval or update mechanisms to avoid those high drift points on old data patterns. It’s not just about training; it’s about the structure of the knowledge store itself.

Paper summary: Jane: So, moving toward the conclusion of this paper, what's the main point they are trying to drive home with this comparison between N-gram and LoRA? What's really new here?

Tom: The paper is titled "Representation Drift in Continual Learning: LoRA versus N-gram Memory." The authors are comparing these three update mechanisms under identical data and budgets to determine where new knowledge should be written for a language model that keeps learning online (<ref:2610.14921#pg1>).

Lu: The main implication is that N-gram memory leaves the base model nearly untouched, and its big-K drift is twenty-six percent/fifteen percent on math and code inputs against sixty-four percent/thirty-five percent for merged LoRA and ninety-seven percent/thirty-nine percent for shared LoRA (<ref:2610.14921#pg1>).

Meng: It seems the authors are arguing that N-gram memory is actually a more stable way to absorb new knowledge into an existing model compared to the other methods because its generic behavior stays essentially unchanged (<ref:2610.14921#pg1>).

Jane: So, thinking about what this means for us using these models every day, it suggests that if you use a native N-gram memory, you're less likely to erode the general capabilities of your model over time compared to the other ways of handling continual learning.

Tom: Exactly. The paper shows that for N-gram memory, generic drift stays small and interference concentrates on inputs that share rows with new tasks (<ref:2610.14921#pg8>). It’s a single source term picture where the injection change is small on generic inputs but large on old-task inputs that read rows later tasks rewrite (<ref:2610.14921#pg8>).

Lu: The authors are emphasizing that N-gram memory can have very small big-K drift because generic inputs rarely read rows the tasks write, and still have large small-ki drift because old-task inputs share rows with new tasks (<ref:2610.14921#pg8>).

Meng: From a production standpoint, that points toward a design choice for memory retrieval that favors stability for the core knowledge while accepting localized interference patterns. It's not about achieving perfect retention everywhere, but managing where the drift is happening.

Jane: So, to wrap up, this paper suggests that N-gram memory is a viable option because its big-K drift against general math and code inputs is relatively small compared to LoRA methods (<ref:2610.14921#pg1>).

Tom: And the conclusion they draw is that for online learning, updating only the N-gram memory changes the base model’s representations of general math and code inputs by twenty-six percent/fifteen percent against sixty-four percent/thirty-five percent for merged LoRA and ninety-seven percent/thirty-nine percent for shared LoRA (<ref:2610.14921#pg1>).

Lu: The title itself, "Representation Drift in Continual Learning: LoRA versus N-gram Memory," sets up the core tension we're seeing here about how these different systems handle knowledge acquisition over time.

Meng: It's a comparison of update strategies to see where new knowledge should be written for an AI that keeps learning online (<ref:2610.14921#pg1>).

Jane: So, this research gives us a clear picture of the trade-offs when we decide how to keep our models updated continuously. It shows that N-gram memory has specific behavior concerning drift and interference that is worth paying attention to (<ref:2610.14921#pg8>).

Conclusion: Tom: So we've been looking at this paper, "Representation Drift in Continual Learning: LoRA versus N-gram Memory." It’s comparing three ways to update a language model when it keeps learning online.

Jane: Right, so it looks at N-gram memory against training a shared LoRA adapter and training a new LoRA for every task. The main thing they're showing is where the new knowledge actually writes itself in the model.

Tom: Exactly, and they found that updating only the N-gram memory leaves the base model nearly untouched, especially for general math and code inputs. It also sees a much smaller big-K drift there compared to what happens with LoRA methods.

Jane: That means if you use that N-gram approach, your core understanding of things like math or coding isn't being eroded as quickly when the AI is constantly getting new tasks.

Tom: Precisely, and they show that while generic behavior stays stable, there’s this specific interference issue where old tasks share rows with new ones in the memory.

Jane: So, it’s not a total protection against forgetting; it’s more like localized friction between old and new data patterns. That's a key distinction to make.

Tom: Yeah, and Lalam confirms that this drift is tied to how much knowledge you lose when you add things on top. It’s a specific kind of retention loss driven by those overlapping inputs.

Jane: It makes me wonder how this applies to real-world systems where models are learning continuously from user interactions. What does that mean for the long-term stability of deployed AI?

Tom: Well, it suggests that if you want to keep the general capabilities solid while still allowing the model to adapt, you have to manage those specific row overlaps in your memory structure carefully. We need to figure out how much interference we can tolerate.

Hao Wu

Illinois Institute of Technology

cs.LG

Submitted: 2026-10-10

Updated: 2026-10-10

The gist: The gist The base stays fixed and every change is local and reversible, as seen in the conclusion > How it works The study compares three update mechanisms—updating only N-gram memory, continually

Key concepts

Representation Drift: This is the key measure of forgetting. It quantifies how much the meaning or representation of existing knowledge changes when new knowledge is added. It's measured by comparing the 'keys' (learned representations) of old data with those of newly written data within each layer.
Big-K Drift: This refers to a large, general drift across the entire model. For N-gram memory, this drift is small for generic inputs but large for old tasks because old tasks share rows with new ones. It indicates how much the overall knowledge base shifts.
Small-K Drift: This measures the change in representation specific to individual, sensitive inputs. Even if the big-K drift is low, small-K drift can be large for certain old inputs because they share specific rows with new tasks, causing localized forgetting.

Terminology

Continual Learning — 1× in this paper · explained in 2 episodes

Summary

The gist The base stays fixed and every change is local and reversible, as seen in the conclusion >

How it works

The study compares three update mechanisms—updating only N-gram memory, continually training one shared LoRA adapter, and training a fresh LoRA per task that is merged into the backbone—to determine where new knowledge should be written for a language model that keeps learning online >

N-gram memory leaves the base nearly untouched, and its big-K drift is 26%/15% on math/code versus 64–97%/35–39% for LoRA >
Its per-round changes cancel, with the realized squared drift being 15% of the sum of per-round squared changes, against 40–56% for LoRA >
Its cost is interference among stored tasks, not damage to the base, as it acquires difficult tasks more weakly and its stored tasks interfere >

Representation Drift and How It Enters Forgetting

The paper defines representation drift as key drift Kt − Ki at each layer, which is measured by comparing the keys of existing knowledge with those of newly written knowledge >

**/Key drift Kt − Ki at each layer, which is measured by comparing the keys of existing knowledge with those of newly written knowledge > (Page 3) The change of response between rounds i and t splits exactly as Yt − Yi = Ki (Wt − Wi) weight change at fixed keys + (Kt − Ki) Wt key drift through new weights > (Page 3) Every output change comes from key drift, which the paper calls representation drift, and in particular at the head > (Page 3) With a frozen head the logit change at the head input is exact, δzt(x) = zt(x) − zi(x) = Uet i (4) every output change comes from key drift > (Page 3) The second-order expansion makes this explicit: ∆l = g⊤ i e t i + 1/2 (e t i)⊤U⊤ΛiUet i + O(∥e t i)3, gi = U⊤(pi − ey), Λi = diag(pi) − pip⊤ i > (Page 3) Small drift along a sensitive head direction can cause large forgetting > (Page 3) With a frozen head the logit change at the head input is exact, δzt(x) = zt(x) − zi(x) = Uet i (10) every output change comes from key drift > (Page 3) For a teacher-forced answer token y, since ∇zl = p − ey has norm at most √2, the cross-entropy obeys the strict bound l(zt, y) − l(zi, y) ≤ √2∥U∥2∥e t i (11) and averaging over the task set (Jensen) gives ∆NLLt i ≤ √2∥U∥2 Dt i > (Page 3) The bound connects small-ki drift directly to retention loss > (Page 3) For N-gram memory, an input whose addressed rows were never rewritten is exactly unchanged, which is why generic drift stays small, and why interference concentrates on inputs that share rows > (Page 4) This is why generic drift stays small, and why interference concentrates on inputs that share rows > (Page 4) The whole right-hand side reduces to a single term, the injection change ∆J(x) of Equation (28) carried through the frozen suffix > (Page 8) If ∆T R(x) = 0 on the full causal prefix of an old input, its entire forward pass is unchanged > (Page 8) N-gram memory can therefore have very small big-K drift (generic inputs rarely read rows the tasks write) and still large small-ki drift (old-task inputs share rows with new tasks) > (Page 8) Old-task inputs share rows with later tasks: 1,277–4,911 rows per task by count, and in particular the template and chat-format rows that every example reads, including at the last prompt token > (Page 23) The layer profile shows it: on old-task inputs N-gram drift reaches 31% within seven layers of the read, while generic inputs at the same depth move by 4% (Figure 4) > (Page 17) N-gram has one source term, the table injection, and it is small on generic inputs and large on old-task inputs, which read rows that later tasks rewrite > (Page 17) This is the single-source picture of Proposition 2: N-gram has one source term, the table injection, and it is small on generic inputs and large on old-task inputs, which read rows that later tasks rewrite > (Page 17) The whole right-hand side reduces to a single term, the injection change ∆J(x) of Equation (28) carried through the frozen suffix > (Page 8) If ∆T R(x) = 0 on the full causal prefix of an old input, its entire forward pass is unchanged > (Page 8) N-gram memory can therefore have very small big-K drift (generic inputs rarely read rows the tasks write) and still large small-ki drift (old-task inputs share rows with new tasks) > (Page 8) The table is read through addresses fixed by raw tokens, and everything upstream of the read is frozen > (Page 8) If ∆T R(x) = 0 on the full causal prefix of an old input, its entire forward pass is unchanged > (Page 8) N-gram memory can therefore have very small big-K drift (generic inputs rarely read rows the tasks write) and still large small-ki drift (old-task inputs share rows with new tasks) > (Page 8) The whole right-hand side reduces to a single term, the injection change ∆J(x) of Equation (28) carried through the frozen suffix > (Page 8) If ∆T R(x) = 0 on the full causal prefix of an old input, its entire forward pass is unchanged > (Page 8) N-gram memory can therefore have very small big-K drift (generic inputs rarely read rows the tasks write) and still large small-ki drift (old-task inputs share rows with new tasks) > (Page 8) The whole right-hand side reduces to a single term, the injection change ∆J(x) of Equation (28) carried through the frozen suffix > (Page 8) If ∆T R(x) = 0 on the full causal prefix of an old input, its entire forward pass is unchanged > (Page 8) N-gram memory can therefore have very small big-K drift (generic inputs rarely read rows the tasks write) and still large small-ki drift (old-task inputs share rows with new tasks) > (Page 8) The whole right-hand side reduces to a single term, the injection change ∆J(x) of Equation (28) carried through the frozen suffix > (Page 8) If ∆T R(x) = 0 on the full causal prefix of an old input, its entire forward pass is unchanged > (Page 8) N-gram memory can therefore have very small big-K drift (generic inputs rarely read rows the tasks write) and still large small-ki drift (old-task inputs share rows with new tasks) > (Page 8) The whole right-hand side reduces to a single term, the injection change ∆J(x) of Equation (28) carried through the frozen suffix > (Page 8) If ∆T R(x) = 0 on the full causal prefix of an old input, its entire forward pass is unchanged > (Page 8) N-gram memory can therefore have very small big-K drift (generic inputs rarely read rows the tasks write) and still large small-ki drift (old-task inputs share rows with new tasks) > (Page 8) The whole right-hand side reduces to a single term, the injection change ∆J(x) of Equation (28) carried through the frozen suffix > (Page 8) If ∆T R(x) = 0 on the full causal prefix of an old input, its entire forward pass is unchanged > (Page 8) N-gram memory can therefore have very small big-K drift (generic inputs rarely read rows the tasks write) and still large small-ki drift (old-task inputs share rows with new tasks) > (Page 8) The whole right-hand side reduces to a single term, the injection change ∆J(x) of Equation (28) carried through the frozen suffix > (Page 8) If ∆T R(x) = 0 on the full causal prefix of an old input, its entire forward pass is unchanged > (Page 8) N-gram memory can therefore have very small big-K drift (generic inputs rarely read rows the tasks write) and still large small-ki drift (old-task inputs share rows with new tasks) > (Page 8) The whole right-hand side reduces to a single term, the injection change ∆J(x) of Equation (28) carried through the frozen suffix > (Page 8) If ∆T R(x) = 0 on the full causal prefix of an old input, its entire forward pass is unchanged > (Page 8) N-gram memory can therefore have very small big-K drift (generic inputs rarely read rows the

Improvements for AI systems

Begin online learning by writing new knowledge only into the memory table and keep the backbone fixed, as this choice ensures the base stays intact as updates pile up. This design principle maintains generic capabilities because generic drift stays small when new knowledge is written only to the memory table, which is read at a fixed early layer.
Implement a two-speed design where online updates are restricted to the native memory table, and the backbone and head frozen, while managing interference via memory-side mechanisms. This isolates updates from eroding general capabilities by ensuring every change is local and reversible.
Address inter-task interference by focusing management on interference between stored tasks, then lies inside the memory, rather than editing the backbone. By inspecting old-task inputs share rows with later tasks within the memory, systems can manage this cost through memory management, such as addressing, capacity and per-row update rules.
Develop a system that monitors small-ki drift rather than just generic drift to track forgetting; the paper states that forgetting follows small-ki drift and that it is measured by task age is not the only factor. This allows for targeted mitigation of task-specific knowledge loss.
Utilize the exact margin identity derived from frozen heads: with a frozen head, every pairwise logit margin changes by the projection of drift onto a head-difference direction, which allows for precise control over prediction stability and accuracy preservation during continual learning.