FOCUS: From Privileged States to RGB-D with Controlled Modality Switching and Representation Alignment
FOCUS is a single-stage PPO framework that trains an actor using privileged state information while intelligently regulating whether to collect rollouts from RGB-D or privileged state latents. It achieves this by aligning their representations and using a KL-guided switching mechanism to control the modality during training, leading to better sample efficiency and superior test performance without needing privileged states at test time.