Verifying the Linear Representation Hypothesis: How Interpretable Are Vision SAEs?
cs.CV
Submitted: 2026-09-28
Updated: 2026-09-28
Comments: 28 pages, 8 figures, 5 tables, preprint under review
Code: https://github.com/KempnerInstitute/overcomplete
License: http://creativecommons.org/licenses/by/4.0/
The gist: Vision Sparse Autoencoders (SAEs) have become a popular tool in Mechanistic Interpretability due to their presumed ability to disentangle complex features learned by a model into monosemantic
Terminology
Abstract
Vision Sparse Autoencoders (SAEs) have become a popular tool in Mechanistic Interpretability due to their presumed ability to disentangle complex features learned by a model into monosemantic concepts. Despite their growing popularity, evaluating their interpretability remains an active topic of research. The bedrock motivating the adoption of SAEs is the Linear Representation Hypothesis (LRH), which claims that polysemantic features can be projected onto a (near) orthogonal basis of sparse, human-understandable representations. Yet, most current frameworks evaluate proxies such as the sparsity of SAE features or the coherence of the inferred dictionary, implicitly assuming that these reflect alignment with human perception. In this paper, we provide empirical evidence that measuring the interpretability of SAE concepts is more difficult than these proxies suggest. To this end, we adapt the Autointerpretability Score (AIS) - previously shown to align with human judgments in Natural Language Processing - to vision tasks and validate our approach in a dedicated user study. We evaluate SAE concept quality using both standard metrics and our adapted AIS. We find that established interpretability metrics for SAEs correlate neither with one another nor with AIS, indicating that no single reference-free metric, whether grounded in the LRH or not, is sufficient for verifying the interpretability of vision SAEs. We argue these findings support recent calls for more verifiable, ground-truth-anchored design and evaluation of explanation methods.
Sources
- Beyond Explainable AI (XAI): An Overdue Paradigm Shift and Post-XAI Research Directions
- On the Robustness of Interpretability Methods
- Do Sparse Autoencoders Capture Concept Manifolds?
- Concept-SAE: A Controllable and Invertible Concept Interface for Sparse Autoencoders
- Toy Models of Superposition
- Causal Interpretation of Sparse Autoencoder Features in Vision
- Natural Language Descriptions of Deep Visual Features
- On the Robustness of Explanations of Deep Neural Network Models: A Survey
- Evaluating Adversarial Robustness of Concept Representations in Sparse Autoencoders
- GPT-4 Technical Report
- Interpretable and Testable Vision Features via Sparse Autoencoders
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models