Value as Memory: Online Value-Guided Post-training for General Robot Policies
Value as Memory (VaM) is a framework for improving generalist robot policies post-training by using value caching and online reinforcement learning. It solves two problems: action ambiguity due to partial observability and slow feedback from offline training. VaM uses recent value predictions as task memory to guide actions, enabling faster, more sample-efficient adaptation in the real world.