The Communication Map of a Transformer
The episode discusses the paper "The Communication Map of a Transformer," focusing on mapping every potential communication channel within language models using their weights. Hosts explain how this provides a direct map of internal connections, moving beyond guesswork to understanding model structure. Key findings include identifying patterned communication and developing tools for targeted model optimization.