Language Models Struggle to Use Representations Learned In-Context
The study tested if large language models (LLMs) can use representations learned from context to solve simple tasks like predicting the next word or modeling an adaptive world. Findings show that these in-context representations are largely inert, meaning they cannot be flexibly deployed to perform downstream tasks, even when they encode new information.