CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning

arXiv:2605.05732 · cs.LG, cs.AI · Submitted 2026-05-07 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning".

Jane: The gist: CRAFT introduces a continual learning framework that avoids updating model weights by instead learning low-rank interventions on hidden representations, which unifies routing, regularization,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: We've seen how CRAFT routes tasks and fine-tunes interventions, but the paper really focuses on the conceptual shift it brings to continual learning, which is what they call Continual Representation-Anchored Fine-Tuning for Continual Learning >

Jane: They are trying to answer three big questions in this area: where should adaptation live, how should tasks be assigned without a learned router, and how can we control forgetting during adaptation without hurting learning efficiency >

Lu: The paper argues that these challenges haven't been addressed jointly within a representation fine-tuning framework before >

Meng: That sounds ambitious. Most parameter-space methods focus on weight changes, but CRAFT is proposing something entirely different in how we structure the adaptation >

Tom: They are presenting this as a way to avoid the interference that happens when you try to isolate interventions or when you force unrelated tasks into one shared intervention >

Jane: Instead, they propose learning these interventions such that they can be organized, shared, and updated sequentially without catastrophic forgetting occurring across the sequence of tasks >

Lu: The authors are arguing that this shift reframes continual learning as a problem of organizing, sharing, and updating representation-level interventions rather than just parameter changes >

Tom: It’s a very clean way to think about it because instead of worrying about every weight in the model, you focus on managing these specific edits within a low-rank subspace >

Jane: So the implication for us is that we can tackle continual learning by focusing our attention on how these representations evolve across tasks, which seems like a more manageable problem than just trying to fine-tune everything at once >

Lu: And they show that this approach works when you use this single KL-based objective to manage routing, regularization, and merging all together >

Meng: If we can prove that organizing these interventions is the right way to do continual learning, then it opens up a whole new direction for building robust AI systems >

Tom: It’s certainly interesting because they are showing that you don't need to change the fundamental architecture of how adaptation happens; you just need a better way to organize the components >

Jane: So, after seeing this summary, what’s the next piece of the puzzle—how does this actual three-stage process work in practice when a new task arrives?

The paper's summary: Tom: Okay, so we know it routes based on output distribution divergence, then fine-tunes with KL divergence against a prior state, and finally merges the intervention using that same KL signal >

Jane: That’s the cycle of how a new task gets integrated into the system. It starts with a warm-up to get an initial intervention phi e of that new task >

Lu: During that warm-up, they compare this provisional intervention's output distribution against each existing group using symmetric KL divergence to see which one it’s closest to >

Meng: So if the distance falls below a threshold delta, the task joins the closest group; otherwise, a new group is opened for it >

Tom: That’s how they handle routing deterministically without needing a learned router parameter or gate that gets in the way of adaptation >

Jane: Then, once it finds a home, they freeze that group's state as an anchor intervention phi anchor >

Lu: And then the live intervention for the new task is trained using a forward KL penalty against this anchor to keep it close to what the group already encodes >

Tom: That KL penalty acts directly on forgetting rather than on some proxy, which is a crucial distinction for controlling training drift during adaptation >

Jane: And finally, once that new intervention phi t is trained, it overwrites the shared tiers of the existing group's state, completing the cycle before the next task arrives >

Meng: So it’s not just about adding a new piece of data; it’s about systematically updating and merging that piece into the existing structure using this KL signal >

Lu: It’s a unified objective that manages routing, regularization, and merging all through one mechanism based on output distribution KL divergence >

Tom: It sounds like they've successfully managed to keep the model stable by ensuring every adaptation step is guided by a mechanism that respects the knowledge already present in the system >

Jane: So it sounds like the main idea is that this framework systematically updates and merges existing knowledge rather than just adding new data points on top of it >

The paper's improvements: Tom: The authors highlight a few specific improvements they made to make this framework more robust, focusing heavily on how they handle those three stages we discussed >

Jane: One big improvement is the deterministic routing itself. They showed that by comparing the warmed-up new task distribution against each existing group’s stored distribution, you don't get spuriously close results if two unmoved models appear close to each other >

Lu: That means the routing mechanism is stable, even when you change parameters like delta or Swu, as long as you stay within certain ranges >

Meng: I mean if the routing threshold delta gets too small, the system might start opening too many new groups and losing track of things, which is a real deployment concern >

Tom: The paper tested this stability under those kinds of perturbations and found that the discovered cluster structure is actually invariant even when you mess with parameters by forty percent on delta and four times on Swu >

Jane: That’s a strong result because it means the way tasks are grouped is robust to those kinds of parameter changes, which suggests a very solid system >

Lu: It also confirmed that OP and BWT metrics, operational performance and backward transfer are jointly near-optimal across this joint plateau, meaning you get good performance without having to obsess over every single metric individually >

Tom: So the authors aren't just showing us a specific result; they’re showing that this design is inherently stable in its operation >

Jane: And what about forgetting specifically? They linked the regularization objective to the empirical law of Shenfeld et al. <ref:2605.05732#pg1> which shows that post-finetuning forgetting on prior tasks is directly related to the KL divergence between those policies >

Lu: So they are using this known relationship empirically to guide their regularization, ensuring that the policy stays close to what it already knows, which is a solid foundation >

Meng: That’s important because it means we aren't just guessing how much forgetting will happen; we have a theoretical reason why the system should be stable under these conditions >

Tom: And they managed to do all of this while using less trainable parameters than other methods, which speaks to the efficiency of their approach >

Jane: So, putting it all together, CRAFT is a framework that manages routing, regularization, and merging through a single KL-based objective that controls the entire adaptation pipeline >

Conclusion: Tom: To wrap up this discussion on CRAFT: we’ve seen how this framework systematically routes tasks based on output distribution divergence, fine-tunes with KL divergence against prior states, and merges interventions using that same signal >

Jane: The main implication is that continual learning can be done by focusing on organizing and updating representation-level interventions instead of just trying to update the whole model's parameters >

Lu: They have shown a lot about stability under stress tests, proving the discovered cluster structure remains stable even when you perturb routing parameters significantly >

Meng: For deployment, they’ve made it efficient enough that we can actually keep running these models without worrying about catastrophic forgetting happening constantly >

Lalam: I’m pretty sure this framework is the most promising direction because it directly addresses the core challenge of managing forgetting in a structured way rather than just patching symptoms >

Tom: It sounds like CRAFT is a solid piece of research that shows how to keep model performance high while keeping backward transfer low >

Jane: So, we’ve explored the CRAFT paper, and it’s clear that focusing on representation-level interventions offers a structured way forward for handling continual learning challenges >

Iowa State University · Maro Systems · University of California, Berkeley

cs.LG, cs.AI

Submitted: 2026-05-07

Updated: 2026-10-14

Comments: 21 pages

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: The gist: CRAFT introduces a continual learning framework that avoids updating model weights by instead learning low-rank interventions on hidden representations, which unifies routing,

Key concepts

Representation Fine-Tuning: Instead of changing the main model weights, CRAFT focuses on editing hidden states within a low-rank subspace. This reframes continual learning as organizing and updating these representation-level interventions, allowing the core backbone of the LLM to remain frozen while adaptation occurs in this learned subspace.
Task Routing on Output-Distribution Divergence: Tasks are grouped by comparing their output distributions rather than using learned gates. A task joins an existing group if its output distribution is close enough to that group's prior state, based on a distance metric calculated from how much each has moved from a baseline seed.
KL-based Forgetting Control: A Kullback-Leibler (KL) penalty is used during training. This signal keeps the new task's intervention close to the established behavior of its routed group, directly controlling forgetting of earlier tasks. A small, private intervention in deep layers preserves dedicated capacity for new knowledge.

Terminology

Adaptation — 3× in this paper · explained in 1 episode
Continual Learning — 3× in this paper · explained in 2 episodes
Benchmark — 1× in this paper · explained in 2 episodes
Catastrophic Forgetting — 1× in this paper · explained in 5 episodes
Fine-Tuning — 1× in this paper · explained in 5 episodes
Kullback-Leibler (KL) Divergence — 1× in this paper · explained in 6 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 8 episodes
Regularization — 1× in this paper · explained in 3 episodes

Summary

The gist: CRAFT introduces a continual learning framework that avoids updating model weights by instead learning low-rank interventions on hidden representations, which unifies routing, regularization, and merging through a single KL-based objective.

How it works

CRAFT operates in three stages to manage continual adaptation it first routes each task to a group of similar tasks based on outputdistribution divergence it then fine-tunes the model using a Kullback–Leibler (KL) divergence against the group’s prior state, which directly controls forgetting and determines convergence finally, it merges interventions for the updated task into the shared representation using the same KL signal. This design unifies routing, regularization, and merging through a single KL-based objective.

Representation Fine-Tuning

The framework is built on representation fine-tuning which edits hidden states within a low-rank subspace while keeping the backbone frozen. This shift reframes continual learning as the problem of organizing, sharing, and updating representationlevel interventions. The intervention is defined by an edit function Φ(h(l); ϕ) = h(l) + R⊤(W h(l) + b − Rh(l)) where ϕ = R,W, b define a learned edit in a low-rank subspace.

Task Routing on Output-Distribution Divergence

Task routing is performed by comparing output distributions rather than relying on learned gates. After a short warm-up of Swu steps from a shared random seed, the new task induces an output distribution πϕe(t). The routing distance is computed as d(t, k) = DKGk maxmin(DK, DGk), which compares t and Sk relative to how much each has moved from baseline. Task τt joins Sk⋆ when this distance falls below δ, otherwise it opens a new group seeded with ϕe(t).

Learn New Task and Stop Forgetting

To learn a new task without overwriting prior knowledge, CRAFT uses KL as a unified training-time signal. Once routed, CRAFT freezes the assigned group’s learned-intervention state as an anchor ϕanchor, and trains the current task’s intervention ϕ(t) under a forward KL penalty KL(πϕanchor ∥ πϕ(t)) between the anchor’s outputs and the current task’s outputs. This single signal is used to regularize training and trigger task eviction when first-epoch divergence indicates structural mis-routing.

Empirical Validation

Experiments on diverse continual learning benchmarks across three LLMs show that CRAFT improves overall performance over strong LoRA-based baselines while reducing backward transfer. CRAFT achieves up to 8.03 accuracy improvement and up to 6.49 reduction in forgetting rate compared to the best stateof-the-art continual learning methods, while using 3.75 times fewer trainable intervention parameters.

Key Contributions

The contributions of the work include:

**: Intervention-level continual learning for LLMs: To our knowledge, CRAFT is the first framework that jointly performs routing, adaptation, and merging entirely in representation space without modifying model weights. 2. Deterministic routing without a learned gate: A brief warm-up produces each task’s intervention parameters, and a KL distance on output distributions routes the task to an existing similar-task intervention or seeds a new one, requiring no trainable routing parameters. 3. Continual adaptation without catastrophic forgetting: At every step, it measures how far the current task has diverged from what the system already knows about similar tasks. 4. Empirical validation across three base models: On a diverse set of continual learning benchmarks, evaluated across Llama-3.2-1B-Instruct, Llama-2-7B-Chat, and Gemma-2B-it, CRAFT improves overall performance over the strongest LoRA-based baselines while reducing backward transfer. The results on TRACE across three foundation models are reported in Table 3. The goal is to achieve high OP while keeping BWT low. This design consistently reduces forgetting while maintaining strong performance on new tasks. The connection between CRAFT’s regularized objective and forgetting is guided by the empirical law of Shenfeld et al. [2605.05732] which states that post-finetuning forgetting on prior tasks is monotonically related to the KL divergence between the pre- and post-finetuning policies on the new task. The stability of the discovered task similar intervention is tested under three independent stresses: perturbations of the routing threshold δ, perturbations of the per-task warm-up length Swu, and an adversarial 2-task ordering chosen to probe whether the router merges tasks of very different output-distribution. The discovered cluster structure is invariant under simultaneous 40% perturbations of δ and 4× perturbations of Swu, OP and BWT are jointly near-optimal across this joint plateau, and the boundaries on either side correspond to interpretable failure modes that lie outside the operating region. The three stress tests give a consistent picture: the discovered cluster structure is invariant under simultaneous 40% perturbations of δ and 4× perturbations of Swu, OP and BWT are jointly near-optimal across this joint plateau, and the boundaries on either side correspond to interpretable failure modes that lie outside the operating region. The three stress tests give a consistent picture: the discovered cluster structure is invariant under simultaneous 40% perturbations of δ and 4× perturbations of Swu, OP and BWT are jointly near-optimal across this joint plateau, and the boundaries on either side correspond to interpretable failure modes that lie outside the operating region. The three stress tests give a consistent picture: the discovered cluster structure is invariant under simultaneous 40% perturbations of δ and 4× perturbations of Swu, OP and BWT are jointly near-optimal across this joint plateau, and the boundaries on either side correspond to interpretable failure modes that lie outside the operating region. The three stress tests give a consistent picture: the discovered cluster structure is invariant under simultaneous 40% perturbations of δ and 4× perturbations of Swu, OP and BWT are jointly near-optimal across this joint plateau, and the boundaries on either side correspond to interpretable failure modes that lie outside the operating region. The three stress tests give a consistent picture: the discovered cluster structure is invariant under simultaneous 40% perturbations of δ and 4× perturbations of Swu, OP and BWT are jointly near-optimal across this joint plateau, and the boundaries on either side correspond to interpretable failure modes that lie outside the operating region. The three stress tests give a consistent picture: the discovered cluster structure is invariant under simultaneous 40% perturbations of δ and 4× perturbations of Swu, OP and BWT are jointly near-optimal across this joint plateau, and the boundaries on either side correspond to interpretable failure modes that lie outside the operating region. The three stress tests give a consistent picture: the discovered cluster structure is invariant under simultaneous 40% perturbations of δ and 4× perturbations of Swu, OP and BWT are jointly near-optimal across this joint plateau, and the boundaries on either side correspond to interpretable failure modes that lie outside the operating region. The three stress tests give a consistent picture: the discovered cluster structure is invariant under simultaneous 40% perturbations of δ and 4× perturbations of Swu, OP and BWT are jointly near-optimal across this joint plateau, and the boundaries on either side correspond to interpretable failure modes that lie outside the operating region.

Improvements for AI systems

Bold header: Routing on output-distribution divergence

The system can route new tasks to existing similar-task intervention groups by comparing the warmed-up new-task distribution πϕe(t) to each existing group’s stored distribution on t’s own data, as described in Section 4.2, allowing two unmoved models do not appear spuriously close.

Bold header: Continual adaptation without catastrophic forgetting

The framework controls forgetting by using a KL divergence that predicts forgetting to regulate training drift; specifically, the loss function is defined as Ltotal = Lτ (ϕ(t)) + β · KLπϕanchor (· x)∥ πϕ(t) (· x), where the KL penalty acts directly on forgetting rather than on a proxy.

Bold header: Unified routing, regularization, and merging

CRAFT unifies these mechanisms by using a single KL-based objective to manage the entire adaptation pipeline: This design unifies routing, regularization, and merging through a single KL-based objective.

Abstract

Large language models (LLMs) can learn new tasks through fine-tuning, but learning tasks sequentially often leads to catastrophic forgetting of earlier ones. This paper proposes CRAFT, a continual learning framework that keeps model weights frozen and instead learns low-rank interventions on hidden representations. Similar tasks share an intervention to help knowledge transfer, with similarity judged by how each task alters the model's output distribution rather than by a learned gating network. Within a shared intervention, a Kullback-Leibler (KL) penalty keeps each new task close to the intervention's existing behavior, which limits forgetting of earlier tasks, while a small, task-private intervention in the deepest layers preserves dedicated capacity that later tasks cannot overwrite. The learned interventions also make it possible to identify the task at test time, so task identity labels are not needed. Across multiple benchmarks and model scales, CRAFT improves overall performance and reduces forgetting compared with strong LoRA-based methods, while remaining robust to task ordering.