ScopeSAE: Model-Scope Feature Discovery with Interpretable Layer Selection
cs.LG, cs.AI
Submitted: 2026-10-04
Updated: 2026-10-04
Code: https://github.com/Mechanical-Interpretability/ScopeSAE
Sources
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Improving Steering Vectors by Targeting Sparse Autoencoder Features
- Toy Models of Superposition
- The Pile: An 800GB Dataset of Diverse Text for Language Modeling
- The Llama 3 Herd of Models
- Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
- Adam: A Method for Stochastic Optimization
- The Remarkable Robustness of LLMs: Stages of Inference?
- Residual Stream Analysis with Multi-Layer SAEs
- Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models
- Steering Language Model Refusal with Sparse Autoencoders
- Improving Dictionary Learning with Gated Sparse Autoencoders
- Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders
- Polysemanticity and Capacity in Neural Networks
- Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
- Gemma 2: Improving Open Language Models at a Practical Size
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks