Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?
cs.LG
Submitted: 2026-02-15
Updated: 2026-11-02
Project page: http://skylion007.github.io/OpenWebTextCorpus
License: http://creativecommons.org/licenses/by/4.0/
The gist: Sparse Autoencoders (SAEs) have emerged as a promising tool for interpreting neural networks by decomposing their activations into sparse sets of human-interpretable features.
Terminology
Abstract
Sparse Autoencoders (SAEs) have emerged as a promising tool for interpreting neural networks by decomposing their activations into sparse sets of human-interpretable features. Recent work has introduced multiple SAE variants and successfully scaled them to frontier models. Despite much excitement, a growing number of negative results in downstream tasks casts doubt on whether SAEs recover meaningful features. To directly investigate this, we perform two complementary evaluations. On a synthetic setup with known ground-truth features, we demonstrate that SAEs recover only 9% of true features despite achieving 71% explained variance, showing that they fail at their core task even when reconstruction is strong. To evaluate SAEs on real activations, we introduce three baselines that constrain SAE feature directions or their activation patterns to random values. Through extensive experiments across multiple SAE architectures, we show that our baselines match fully-trained SAEs in interpretability (0.87 vs 0.90), sparse probing (0.69 vs 0.72), and causal editing (0.73 vs 0.72). Together, these results suggest that SAEs in their current state do not reliably decompose models' internal mechanisms.
Sources
- BatchTopK Sparse Autoencoders
- Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders
- Feature Hedging: Correlated Features Break Narrow Sparse Autoencoders
- Toy Models of Superposition
- I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders
- Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformers
- Do Sparse Autoencoders Generalize? A Case Study of Answerability
- GPT-4o System Card
- Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Video
- OrtSAE: Orthogonal Sparse Autoencoders Uncover Atomic Features
- Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
- Sparse Autoencoders Trained on the Same Data Learn Different Features
- Improving Dictionary Learning with Gated Sparse Autoencoders
- Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders
- Gemma 2: Improving Open Language Models at a Practical Size
- The Llama 3 Herd of Models
- Persona Features Control Emergent Misalignment
Related papers
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis
- (Mis)generalization of Helpful-Only Fine-tuning