When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction
The research investigates why large language models lose track of instructions during long conversations. They used a channel transition framework, measuring direct attention to goals via GAR and manipulating it with sliding-window masks. Findings show that when direct goal access closes, behavior degrades differently across models based on their architecture, revealing that residual stream information holds partial predictive power for post-failure outcomes.