Finding Usable Weight Mechanisms with Tiled SVD
The hosts discuss a paper from Aquin Labs on interpreting language model weights directly via column-tiled SVD. They explain mechanism mounts, honest evaluation metrics, and results showing all 182 Gemma-2-2B site-layers pass, while cautioning that the work offers reusable measurement tools rather than new concept discovery.