CF-JEPA: Improving Robustness of JEPA World Models via Controllability Factorization
CF-JEPA is a world model that splits its latent space into controllable and uncontrollable parts to handle visual noise. It enforces this separation using specialized loss functions, allowing the agent to focus on task-relevant information while ignoring distracting background noise. This results in a model that remains robust even when faced with visual disturbances, unlike other models.