Tree SAE: Learning Hierarchical Feature Structures in Sparse Autoencoders
Standard Sparse Autoencoders (SAEs) fail to capture hierarchical structures in data because they only check if a child feature activates when its parent does. This leads to false positive associations between unrelated concepts. Tree SAEs solve this by combining activation coverage with a new reconstruction condition, enforcing a deeper functional link between hierarchical levels to robustly learn true parent-child pairs.