How do Transformers Perform Low-Level Control In-Context?
cs.LG, cs.SY, eess.SY
Submitted: 2026-11-05
Updated: 2026-11-05
Comments: 6 pages, 5 figures, Accepted to the 65th IEEE Conference on Decision and Control (CDC 2026)
Code: https://github.com/ebonyelsmith/transformer_control
License: http://creativecommons.org/licenses/by/4.0/
The gist: One of the remarkable abilities of Transformer-based neural networks is in-context learning.
Terminology
Abstract
One of the remarkable abilities of Transformer-based neural networks is in-context learning. This motivates us to think about "in-context" control. Can we control a system knowing only (1) that it belongs to a particular family of systems and (2) some data on system evolution (provided as the context trajectory) but no specific parameter information? Specifically, we investigate the ability of Transformer-models to perform "in-context" control of unstable, nonlinear dynamical systems, where context consists only of the unforced system dynamics. We leverage a multi-head, GPT-2-style architecture, and we evaluate performance for two classical nonlinear systems -- the cartpole system and the two-link acrobot system, where the system parameters are unknown to the Transformer. We consider a setting where trajectories from an expert controller are available at training time but not at inference time. Here, we identify training techniques that are essential to train Transformers to perform low-level control. Through an analysis over checkpoints, we show that in-context control is possible, and the learned policy performance generally improves as context increases. Furthermore, the Transformer models can generalize to stabilize systems that are out-of-distribution.
Sources
- Scaling Language Models: Methods, Analysis & Insights from Training Gopher
- In-Context Imitation Learning via Next-Token Prediction
- In-context Learning and Induction Heads
- In-Context Language Learning: Architectures and Algorithms
- Generalization to New Sequential Decision Making Tasks with In-Context Learning
- Vision-Language Foundation Models as Effective Robot Imitators
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis