Differential Backpack Language Models: Sense Inhibition via Signed Contextualization Weights
cs.CL
Submitted: 2026-11-11
Updated: 2026-11-11
Terminology
Sources
- Controlling Gender Bias in Retrieval via a Backpack Architecture
- Shared DIFF Transformer
- Multilinguality as Sense Adaptation
- Sense Representations Are Inducible Interfaces
- Model Editing with Canonical Examples
- Threshold Differential Attention for Sink-Free, Ultra-Sparse, and Non-Dispersive Language Modeling
- Understanding Differential Transformer Unchains Pretrained Self-Attentions
- More Expressive Attention with Negative Weights
- Differential Gated Self-Attention
- In the LLM era, Word Sense Induction remains unsolved
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data