AlignSAE: Concept-Aligned Sparse Autoencoders
AlignSAE upgrades Sparse Autoencoders (SAEs) to create interpretable interfaces for LLMs by enforcing alignment with a predefined ontology. It uses a two-phase training process—initial unsupervised learning followed by supervised post-training—to bind specific concepts to dedicated latent slots. This allows researchers to causally control model behavior through concept swaps, achieving significant disentanglement improvements.