Goal Encoding in LLMs
cs.CL, cs.AI, cs.LG
Submitted: 2026-11-12
Updated: 2026-11-12
Code: https://github.com/davidbau/baukit
Terminology
Sources
- Eliciting Latent Predictions from Transformers with the Tuned Lens
- Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs
- Reasoning Models Don't Always Say What They Think
- nnterp: A Standardized Interface for Mechanistic Interpretability of Transformers
- Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space
- Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
- Measuring Faithfulness in Chain-of-Thought Reasoning
- What's the plan? Metrics for implicit planning in LLMs and their application to rhyme generation and question answering
- The Hydra Effect: Emergent Self-repair in Language Model Computations
- In-context Learning and Induction Heads
- Language Models use Lookbacks to Track Beliefs
- LLMs Process Lists With General Filter Heads
- Convergent Linear Representations of Emergent Misalignment
- Model Organisms for Emergent Misalignment
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data