Context Value Informed In-Context Reinforcement Learning
In-Context Reinforcement Learning methods suffer from performance drops during testing due to 'Contextual Ambiguity,' where random interactions mislead the model into suboptimal decisions. The authors introduce Context Value Informed ICRL (CV-ICRL), which adds an explicit 'Context Value' signal—the expected return given the context—to guide policy updates. This allows for lower-variance, more stable adaptation and higher returns across tasks.