Static In, Dynamic Out: Counterfactual Action Augmentation for Moving Object Manipulation
SIDO introduces a counterfactual action augmentation that allows a policy trained only on static objects to handle moving ones during deployment. It achieves this by splitting manipulation into two parts: predicting where the object will move and then reaching that predicted position. The method preserves the hand-object relative pose by morphing the expert actions to satisfy constraints at future time steps.