The Transformer Revolution, Part 1: Dynamic Processing through OutputWeight Interconnections
The hosts discuss a paper titled "The Transformer Revolution," which introduces SIDPP, a new way Transformers operate. They explain how these models dynamically build transformations based on input prompts using output-weight interconnections. This dynamic processing is shown to be highly sensitive to context, challenging the idea that models are just repeating training data.