Multisensory Continual Learning: Adapting Pretrained Visuomotor Policies to Force
MuSe is a framework that adapts robot policies trained with only vision to handle new sensors like force-torque (F/T) data. It uses multi-stage fusion, future prediction for world modeling, and experience replay to enable zero-shot F/T prediction on old tasks (backward transfer), improved performance on new contact tasks (forward transfer), and better generalization across modalities.