Contextual Linear Activation Steering of Language Models
summary
The gist
Contextual Linear Activation Steering (CLAS) Contextual Linear Activation Steering (CLAS) is a novel method introduced to dynamically adapt linear activation steering within large language models…
In short
Contextual Linear Activation Steering (CLAS) dynamically adapts how a language model's internal computations are guided. It addresses fixed steering limitations by using context-aware sensing vectors to modulate steering strength based on the input and model block. This allows for precise, nuanced control over the LLM's output without extensive full fine-tuning.
Key concepts
- Contextual Linear Activation Steering (CLAS)
- A novel method that changes how a language model is steered by making the steering strength dependent on both the input context and which part of the model is being processed. It uses learned vectors to dynamically adjust the influence of steering directions.
- Block-specific steering vectors ($\mathbf{d}_\ell$)
- These are learned vectors specific to each layer or block within a large language model. They are extracted by training an activation probe on labeled data, helping the system understand how each part of the model responds to different concepts.
- Context-dependent sensing vectors ($\mathbf{c}_\ell$)
- These vectors are learned during training by minimizing prediction errors on prompt-completion pairs. They act as a mechanism to sense and determine how much steering should be applied at a particular point, based on the current context.
Terminology used across episodes
This episode discusses
- Contextual Linear Activation Steering of Language Models · Paper Radio
- Understanding intermediate layers using linear classifier probes
- Activation Steering for Chain-of-Thought Compression
- Representation Engineering for Large-Language Models: Survey and Research Challenges
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- Training Verifiers to Solve Math Word Problems
- Efficient and accurate steering of Large Language Models through attention-guided feature learning
- The Llama 3 Herd of Models · Paper Radio
- Designing and Interpreting Probes with Control Tasks
- Steering Language Models With Activation Engineering
- Extending Activation Steering to Broad Skills and Multiple Behaviours
- Qwen2.5 Technical Report
- A Survey of Large Language Models
- Representation Engineering: A Top-Down Approach to AI Transparency
The paper
Contextual Linear Activation Steering of Language Models · Read on arXiv
UC San Diego · MIT · Broad Institute of MIT and Harvard
Linear activation steering is a powerful approach for controlling and specializing large language models by adding linear representations of concepts to their activations at inference. While effective, existing methods often steer with a fixed strength, independent of how much steering any given input requires. In this work, we introduce Contextual Linear Activation Steering (CLAS), which decomposes activation steering into a fixed steering direction and a context-dependent steering coefficient. This decomposition preserves the interpretability of the steering direction while enabling the steering strength to vary with context. It also enables CLAS to perform selective steering: steering multiple behaviors across different portions of a single generation, which cannot be achieved using a fixed steering strength. In addition, across a wide range of steering benchmarks and language models, CLAS consistently outperforms standard linear activation steering and matches or exceeds ReFT and LoRA in settings with limited labeled data. Notably, CLAS achieves this using directions that also serve as reliable concept monitors, a property that directions learned via fine-tuning lack. We therefore propose CLAS as an interpretable and accurate method for controlling and specializing large language models.
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Contextual Linear Activation Steering of Language Models".
Jane: Here's the gist of the paper. Detailed Research Summary: Contextual Linear Activation Steering (CLAS) Contextual Linear Activation Steering (CLAS) is a novel method introduced to dynamically adapt linear activation steering within large language models (LLMs), addressing the critical limitation of using fixed steering strengths in prior approaches.
Tom: The core innovation of CLAS lies in its ability to modulate the amount of steering applied based on both the input context and the specific model block being processed.
Jane: This dynamic adaptation is achieved by utilizing a learned sensing vector (c) that interacts with block-specific steering vectors (d) via an equation such as: h tilde,t = h,t + (c times
h,t, 1: ) d.
Tom: The methodology involves a two-stage learning process.
Paper summary: Tom: So, Jane, this paper is about this new method called Contextual Linear Activation Steering of Language Models. The main idea is that instead of using one fixed steering strength for every part of the model, we can change how much we steer based on what's in the input context and which specific model block we are looking at.
Jane: That sounds really precise, Tom. But before we get into the technical bits, what's the big picture here? What problem is this paper trying to fix with this contextual steering idea?
Tom: Well, existing methods use a fixed steering strength for every token in the model, and that leads to inconsistent steering quality when you have different types of prompts. This paper proposes Contextual Linear Activation Steering, or CLAS, which dynamically adjusts those steering strengths based on the input context and the specific model block.
Lu: What I find interesting is how they achieve this dynamic adjustment. They use something they call a learned sensing vector c sub ell that interacts with block-specific steering vectors d sub ell through an equation like h t tilde equals h t plus c sub ell dot h t one d sub ell <ref:2604.24693#pg1>.
Jane: That sounds like it's adding a layer of context into the calculation. So, how do they actually learn this sensing vector, and what kind of data are they using for that?
Tom: The learning happens in two stages. First, they extract the block-specific steering vectors d sub ell by training an activation probe on a dataset of prompt concept label pairs using something called Recursive Feature Machines or RFM to derive those vectors.
Meng: So, they're using a probe trained on concept labels to get the steering directions for each block. That sounds like a solid starting point if you need task-specific knowledge for each part of the model. But what about learning that context vector?
Lu: For the context-dependent sensing vectors c sub ell, they learn them by minimizing the standard next-token prediction objective on a separate dataset consisting of prompt completion pairs. This means they use a small amount of data to train these vectors, and it requires only one forward backward pass per prompt completion pair to update them.
Paper summary: Jane: That sounds quite efficient for learning those context vectors, especially since it's much lighter than tuning the coefficients in standard linear activation steering methods that rely on an evaluator selecting a fixed coefficient.
Tom: Exactly. They compare their CLAS against standard linear activation steering and two popular parameter-efficient finetuning baselines, ReFT and LoRA, and they show CLAS outperforms both in settings with limited labeled data across eleven diverse steering tasks using four different instruction-tuned language models.
Meng: Eleven tasks is a lot of tests. That’s impressive if it holds up across all those different behaviors they are steering the models toward, like step-by-step reasoning or translation. But I wonder what happens when you try to deploy this at scale.
Lalam: From where I'm standing, the context vector learns how to modulate that linear activation based on the input context and specific model block using a learned sensing vector c sub ell. It enables context-dependent activation steering: h t tilde equals h t plus c sub ell dot h t one d sub ell > <ref:2604.24693#pg1>
Jane: So, for someone just listening to the show, what does that mean in practical terms? What’s the benefit of this contextual modulation over a fixed coefficient?
Tom: It means we can exert more precise and nuanced control over the language model's generation process. Instead of a one-size-fits-all steering direction, you can have fine-grained, context-aware control for things like stylistic constraints or tone modulation without needing extensive full fine tuning.
Lu: The paper also points out that the prompt contains the concept and y equals zero indicating that it does not. They pass the prompts x sub i N i equals one to an LLM and obtain a per-block dataset of LLM activations H l t =
h (one) l t, <ref:2604.24693#pg1>...,h (N) l t: in R k times N.
Meng: That data structure, the dataset of last-token residual stream activations, seems like a lot to manage for production. What does that mean for the computational cost when you're actually running this model?
Lalam: The method is computationally efficient because learning the sensing vectors only requires updating them through minimizing the standard next-token prediction objective on a small dataset of prompt completion pairs. It differs from tuning coefficients in linear activation steering, which uses an evaluator to select a fixed coefficient that produces the best generated outputs over a set of prompts.
Paper summary: Jane: So, we have this efficient learning process for the context vector, and then we have these specific steering vectors derived from RFM for each block. What’s the main point about what this does compared to what ReFT or LoRA can do?
Tom: The paper shows that CLAS consistently outperforms standard linear activation steering and matches or exceeds the performance of ReFT and LoRA in settings where labeled data is limited across eleven steering benchmarks. Plus, those probe-derived steering vectors used by CLAS achieve much higher accuracy for concept monitoring tasks than those learned by ReFT or LoRA.
Lu: The paper highlights that this contextual modulation is shown to enable more precise, scalable, and general control over LLM behavior than existing fixed steering directions. It establishes CLAS as a scalable, interpretable, and accurate technique for specializing and steering large language models by making the steering coefficient context-dependent >
Jane: So, to wrap up on what this means for us right now in the AI space, it seems like we have a method that lets us steer these massive language models with much finer control using only limited labeled data because the steering itself adapts to the context.
Tom: Yeah, it’s about making sure that when you ask an AI something specific, its internal mechanism is guided along the most relevant path for that exact moment in time. It shifts the control from a static setting to a dynamic one based on what's happening right now in the input data stream >
Meng: From my point of view, as someone shipping models to production, I'm still concerned about how robust these context vectors are when faced with completely novel inputs that weren't seen during training. Does this dynamic adaptation hold up for out-of-distribution behavior?
Lalam: The paper shows that the method is scalable because it doesn't require extensive full fine tuning to achieve high accuracy on those tasks >
Jane: That leads perfectly into what we should think about next. This paper by Brandon Hsu, Daniel Beaglehole, Adityanarayanan Radhakrishnan, and Mikhail Belkin is proposing Contextual Linear Activation Steering of Language Models. It seems like a significant step in moving toward more adaptable steering for AI models >
Tom: Exactly. It’s about providing a scalable and interpretable way to specialize these models without needing massive amounts of labeled data for every single task >
Conclusion: Tom: So we've been talking about Contextual Linear Activation Steering, or CLAS, and now we're wrapping up this segment to talk about what this paper actually says and who wrote it.
Jane: The title is Contextual Linear Activation Steering of Language Models. Basically, the authors are showing how you can steer a language model’s internal math based on what the input context is right then.
Tom: Right, so we're moving away from those fixed steering strengths we talked about earlier and showing this dynamic adjustment based on context and model block.
Jane: It’s about making the steering coefficient depend on where you are in the input and which part of the model is processing that information.
Lu: The authors introduce two main things they train. First, block specific steering vectors derived from an activation probe using something called Recursive Feature Machines to get those initial directions for each layer or block.
Meng: And second, context dependent sensing vectors learned by minimizing the standard next-token prediction objective on a separate set of prompt completion pairs. That’s the learning mechanism we talked about earlier, right?
Tom: Exactly. It’s a two stage learning process to get those specific vectors d and then the context dependent ones c.
Jane: So what does this actually change for us if we're building these models? It means we can get much finer control over things like tone or style without needing massive fine tuning.
Tom: It shifts the control from a static setting to something dynamic based on what's happening in the input data stream right now.
Lu: The authors show that CLAS consistently outperforms standard linear activation steering across nearly all model and task pairs, often by substantial margins.
Jane: That’s strong, especially when compared against other methods like ReFT and LoRA, particularly when you have limited labeled data for those eleven steering benchmarks they tested.
Tom: And the paper highlights that the specific steering vectors derived from the probe method achieve much higher accuracy for concept monitoring tasks than those learned by ReFT or LoRA.
Lu: So in essence, CLAS is presented as a scalable and interpretable way to specialize these models by making those steering coefficients context dependent.
Jane: It sets up a lot of questions about how robust these context vectors are when the model sees something completely new it hasn't encountered before during its learning phase.
Tom: That’s the big caveat we need to watch for as we try to deploy this kind of dynamic system in a real production environment.
More episodes
- 2603.28595-Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound
- 2605.15085-From Data to Action: Accelerating Refinery Optimization with AI