Transformers Struggle to Use Their Emergent World Models: Revisiting the Tower of Hanoi, and the Illusion of Thinking
The episode discusses a paper by Pereira and Zuidema showing that Transformers, including large reasoning models, form a perfect world model of the Tower of Hanoi puzzle but lose it during generation. The hosts explain the 'illusion of thinking' and how activation steering can partially restore performance.