Sparse Autoencoder Attribution Graphs Are Decided at the Margin
cs.LG, cs.AI, cs.CL
Submitted: 2026-10-31
Updated: 2026-11-14
Code: https://github.com/EleutherAI/sparsify
Terminology
Sources
- Git Re-Basin: Merging Models modulo Permutation Symmetries
- Mechanistic Permutability: Match Features Across Layers
- Finding Transformer Circuits with Edge Pruning
- Smooth and Sparse Optimal Transport
- A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders
- Transferring Linear Features Across Language Models With Model Stitching
- Towards Automated Circuit Discovery for Mechanistic Interpretability
- Monge, Bregman and Occam: Interpretable Optimal Transport in High-Dimensions with Feature-Sparse Maps
- Transcoders Find Interpretable LLM Feature Circuits
- Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activations
- Scaling and evaluating sparse autoencoders
- Complexity of Linear Regions in Deep Networks
- Deep ReLU Networks Have Surprisingly Few Activation Patterns
- Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
- Sparse Autoencoders Do Not Find Canonical Units of Analysis
- Convergent Learning: Do different neural networks learn the same representations?
- Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
- Sparsity-Constrained Optimal Transport
- Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis