Adversarial Robustness of Activation Steering in Large Language Models
cs.LG, cs.AI
Submitted: 2026-06-05
Updated: 2026-10-31
Comments: EMNLP Manuscript update
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Activation steering has become a popular training-free method to control LLMs' behavior by injecting precomputed direction vectors into the model's residual stream at inference time.
Terminology
Abstract
Activation steering has become a popular training-free method to control LLMs' behavior by injecting precomputed direction vectors into the model's residual stream at inference time. Yet, its robustness to realistic input variation remains under-explored. We present the first systematic evaluation of activation steering robustness under adversarial text perturbations on the inputs, covering four extraction methods, three attack strategies, six personas from the Anthropic Model-Written Evaluations dataset, and five models ranging from 1.5B to 30B parameters. Attacks succeed broadly across all settings: directional robustness drops by up to 64%, post-attack confidence collapses near or below 0.25 across all methods and models, and steering strength degrades on nearly every steerable input. Layer selection is equally fragile, with the optimal layer identified by an automated method on clean inputs shifting by up to 17 positions under perturbation, a failure that compounds the vector-level breakdown. Extracting vectors from adversarially perturbed inputs partially recovers steerability for PCA and MD on mid-to-large models, but they consistently fail to locate the improved optimal layer, limiting the practical benefit of this mitigation. Together, these findings reveal that the brittleness of activation steering is structural rather than method-specific, and that current layer selection strategies are not robust enough for real-world deployment. Our code is available at github.com/kien206/steering-adv-robustness.
Sources
- Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RL
- Understanding (Un)Reliability of Steering Vectors in Language Models
- Discovering Latent Knowledge in Language Models Without Supervision
- Explaining and Harnessing Adversarial Examples
- The Llama 3 Herd of Models
- TextBugger: Generating Adversarial Text Against Real-world Applications
- Towards Deep Learning Models Resistant to Adversarial Attacks
- Open Character Training: Shaping the Persona of AI Assistants through Constitutional AI
- ATLAS: Verifier-Guided Adaptive Latent Activation Steering for Efficient LLM Reasoning
- Towards Reliable Evaluation of Behavior Steering Interventions in LLMs
- Qwen2.5 Technical Report
- Steering Language Models With Activation Engineering
- Qwen3 Technical Report
- ODESteer: A Unified ODE-Based Steering Framework for LLM Alignment
- Representation Engineering: A Top-Down Approach to AI Transparency
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis