Geometry of Confabulation and Refusal in Small Language Models
cs.LG, cs.CL
Submitted: 2026-10-31
Updated: 2026-10-31
Code: https://github.com/vkazei/confabqa
Terminology
Sources
- Understanding intermediate layers using linear classifier probes
- Refusal in Language Models Is Mediated by a Single Direction
- The Internal State of an LLM Knows When It's Lying
- A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders
- Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models
- Scaling and evaluating sparse autoencoders
- The Llama 3 Herd of Models
- TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
- Language Models (Mostly) Know What They Know
- Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs
- Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
- Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
- When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
- LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations
- Safety Alignment Should Be Made More Than Just a Few Tokens Deep
- Qwen3 Technical Report
- Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Models
- Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States
- Can LLMs Introspect? A Reality Check
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis