Prototype Language Models
Dan Ley, Giang Nguyen, Himabindu Lakkaraju, Julius Adebayo
cs.LG, stat.ML
Submitted: 2026-07-01
License: http://creativecommons.org/licenses/by/4.0/
The gist: Knowing which training examples drive outputs is fundamental to auditing, correcting, and understanding language models, yet for modern LLMs this remains expensive, approximate, and largely post-hoc.
Terminology
Abstract
Knowing which training examples drive outputs is fundamental to auditing, correcting, and understanding language models, yet for modern LLMs this remains expensive, approximate, and largely post-hoc. Standard language models generate tokens through a dense network pathway, causing training data's influence to be distributed across parameters rather than organized along explicit, traceable components. We introduce a prototype language model architecture, Prototypes for Interpretable Sequence Modeling (PRISM), that forms each prediction via a sparse, non-negative mixture of learned prototypes, trained with clustering objectives that anchor each prototype to coherent neighborhoods of training examples. Across architectures from 130M to 1.6B parameters trained on up to 50B tokens, prototype language models either surpass or remain within 2.5 percentage points on average downstream accuracy of matched dense baselines. We show that sparse prototype structure localizes curvature in the loss landscape, yielding a more tractable Hessian and enabling training data attribution that is 500x faster than post hoc baselines when consuming equivalent memory. Calibrating linear prototype controllers can improve downstream accuracy by roughly 3 points while tracing those corrections back to training neighborhoods, and targeted prototype suppression can remove model behaviors without finetuning or measurable loss in generation quality.
Sources
- If Influence Functions are the Answer, Then What is the Question?
- Eliciting Latent Predictions from Transformers with the Tuned Lens
- Large-Scale Training Data Attribution for Music Generative Models via Unlearning
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Adaptive Gradient Methods at the Edge of Stability
- TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
- Natively Unlearnable Large Language Models
- Studying Large Language Model Generalization with Influence Functions
- Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Models
- MAGIC: Near-Optimal Data Attribution for Deep Learning
- Interpretable Next-token Prediction via the Generalized Induction Head
- Revisiting inverse Hessian vector products for calculating influence functions
- Generalized Group Data Attribution
- LoRIF: Low-Rank Influence Functions for Scalable Training Data Attribution
- Automated Interpretability and Feature Discovery in Language Models with Agents
- Scaling Up Influence Functions
- FlexOlmo: Open Language Models for Flexible Data Use
- Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs
- Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methods
- This Looks Better than That: Better Interpretable Models with ProtoPNeXt
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks