Shifting the Gradient: Understanding How Defensive Training Methods Protect Language Model Integrity
cs.LG, cs.AI
Submitted: 2026-04-03
Updated: 2026-11-07
Terminology
Sources
- Recontextualization Mitigates Specification Gaming without Modifying the Specification
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- Context Structure Reshapes the Representational Geometry of Language Models
- Linear representations in language models can change dramatically over a conversation
- Priors in Time: Missing Inductive Biases for Language Model Interpretability
- Natural Emergent Misalignment from Reward Hacking in Production RL
- Steering Llama 2 via Contrastive Activation Addition
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples
- Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time
- Steering Language Models With Activation Engineering
- Inoculation Prompting: Instructing LLMs to misbehave at train-time improves test-time alignment
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
- AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
- Qwen3 Technical Report
- Representation Engineering: A Top-Down Approach to AI Transparency
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis