Brain alignment of reasoning and action representations from vision-language and action models during naturalistic gameplay
The study compared vision-language models (VLMs) and large-action models (LAMs) against reinforcement learning baselines during video games. Both outperformed RL baselines in brain encoding, showing that action fine-tuning reorganizes neural representations toward motor control despite similar overall prediction accuracy. Prompting gains are strongest in higher-order frontal regions, and LAMs show a distinct action-asymmetric organization.