Cross-Layer Discrete Concept Discovery for Interpreting Language Models
CLVQ-VAE is a new framework that maps representations between different layers of language models using a discrete bottleneck. It collapses redundant features into compact, interpretable concept vectors, revealing cross-layer structures missed by single-layer analysis. This yields concepts that are both functionally important and semantically coherent.