A Circuit for In-Context Learning
cs.CL, cs.LG
Submitted: 2026-11-03
Updated: 2026-11-03
Code: https://github.com/andrewgcodes/in-context-learning-circuit
Terminology
Sources
- What learning algorithm is in-context learning? Investigations with linear models
- Language Models are Few-Shot Learners
- Skill Path: Unveiling Language Skills from Circuit Graphs
- Revisiting In-context Learning Inference Circuit in Large Language Models
- What Can Transformers Learn In-Context? A Case Study of Simple Function Classes
- How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model
- In-Context Learning Creates Task Vectors
- Understanding In-context Learning of Addition via Activation Subspaces
- Understanding the Dynamics of Demonstration Conflict in In-Context Learning
- Task Schema and Binding: A Double Dissociation Study of In-Context Learning
- Copy Suppression: Comprehensively Understanding an Attention Head
- Locating and Editing Factual Associations in GPT
- Progress measures for grokking via mechanistic interpretability
- In-context Learning and Induction Heads
- Necessary, Decodable and Reversible, Yet Not Transferable: A Stress Test for Attention-Head Role Claims
- Function Vectors in Large Language Models
- Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias
- Transformers learn in-context by gradient descent
- Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
- Larger language models do in-context learning differently
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data