Decodability Is Not Validity: Evaluating Nuisance Sensitivity in Language Model Concept Probes
cs.CL, cs.LG
Submitted: 2026-11-05
Updated: 2026-11-05
Code: https://github.com/kleinay/decodability-is-not-validity
Terminology
Sources
- Gemma 2: Improving Open Language Models at a Practical Size
- Probing the Probes: Methods and Metrics for Concept Alignment
- When Decodability Is Not Enough: Logical Validity Representations, Behavioral Dissociation, and Causal Tests in Language Models
- A Probe Direction Is a Property of Its Prompt
- Qwen2.5 Technical Report
- Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe
- Decodability is Not Causality: Dissociating Probe Readouts from Behavioral Drivers via SAE Decomposition
- Representation Engineering: A Top-Down Approach to AI Transparency
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data