The Shape of Wisdom: How Language Models Converge on an Answer
This study tracked how language model answers move across their internal layers during factual question answering. Researchers found that answers follow specific trajectories characterized by state, motion, and boundary distance. Correctness and stability are distinct properties; the best performance comes from trajectories that not only reach the correct answer but also maintain it throughout the network.