WALA Learning Executable Latent Actions from Action-Labeled Demonstrations and Action-Free Videos
WALA jointly learns executable latent actions from both action-labeled demonstrations and action-free videos. It pretrains a model to learn action-relevant representations by predicting future semantic and geometric changes in video sequences. This allows the policy to ground its latent actions in expected future scene evolution, improving performance even with limited robot demonstrations.