ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
summary
The gist
ContactGuard is a pre-contact execution monitoring system for chunked visuomotor policies in contact-rich robot manipulation tasks.
In short
ContactGuard is a monitoring system designed to prevent failures during robot manipulation before physical contact occurs. It uses a latent world model trained on unlabelled robot data to predict the visual outcome of planned actions in a compact latent space. By checking this prediction just before contact, ContactGuard can issue an abort signal without changing the main robot policy, improving safety in complex tasks.
Key concepts
- Latent World Model
- This is a predictive model that learns to forecast the future state of the environment using a compressed, abstract representation (latent space) instead of raw pixels. It is trained on many unlabelled real-world robot movements to understand how different actions will affect the scene visually, allowing it to predict what will happen after a planned move.
- Action-Conditioned Predictor
- This component takes the current situation and the specific action chunk planned by the robot as input. It uses a Transformer architecture to predict where the visual representation (latent) of that future state will be, based on what it has learned about how actions influence those states.
- Failure Probe
- This is a simple classifier, like logistic regression, trained on a small set of examples showing successful and failed pre-contact situations. It takes the predicted future latent state from the world model and quickly estimates the probability that contact will result in a failure, acting as a final safety check.
Terminology used across episodes
This episode discusses
- ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models · Paper Radio
- World Models
- Learning Latent Dynamics for Planning from Pixels
- Dream to Control: Learning Behaviors by Latent Imagination
- Mastering Diverse Domains through World Models
- Temporal Difference Learning for Model Predictive Control
- Deep Visual Foresight for Planning Robot Motion
- IRASim: A Fine-Grained World Model for Robot Manipulation
- Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imagination
- LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
- Vision-Language Models as Success Detectors
- AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation
- RoboReward: General-Purpose Vision-Language Reward Models for Robotics
- Failure Prediction at Runtime for Generative Robot Policies
- Rewind-IL: Online Failure Detection and State Respawning for Imitation Learning · Paper Radio
- PATCH: Action-Chunk-Conditioned Latent Patch Innovation Monitoring for Robot Manipulation
- Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
- Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
- TriPilot-FF: Coordinated Whole-Body Teleoperation with Force Feedback
- RoVer: Robot Reward Model as Test-Time Verifier for Vision-Language-Action Model
- AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World Models
The paper
ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models · Read on arXiv
Vanderbilt University · The University of Sydney · Australian Centre for Robotics, The University of Sydney
Contact-rich manipulation failures are often detected only after the robot has committed to contact. This is especially limiting in wrist-camera setups: close gripper--object views help observe contact, but a poor approach may already push, miss, slip, or disturb the object before conventional detectors react. We introduce ContactGuard, a pre-contact execution monitor for chunked visuomotor policies. Given the policy's planned action chunk, ContactGuard predicts its short-horizon consequence in latent visual space and aborts if the predicted future latent indicates likely failure. Its latent world model is trained from unlabelled robot trajectories to predict compact multi-view visual embeddings under planned actions, avoiding pixel-level video prediction. A lightweight failure probe is then trained from a small labelled set of pre-contact clips. At deployment, ContactGuard anchors prediction before an imminent contact event, rolls the model forward under the policy's own actions, and verifies the predicted post-contact latent. Across real-world contact-rich manipulation tasks, ContactGuard predicts failure more accurately than direct and corrupted-action ablations, and transfers to live robot as a pre-contact abort signal without modifying the underlying policy.
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: I'm Rosa.
Dev: And I'm Dev, here with Taro.
Rosa: We're looking at the paper: "ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models".
Paper summary: Dev: Looking at the whole picture of ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models, this research is really about shifting the monitoring problem from detection to prediction in a way that works before commitment.
Rosa: Yeah, they’ve introduced this action-conditioned latent world model monitor which conditions its predictions on the same planned chunk as an anchor k steps before the planned gripper closure Tg.
Taro: What this means for us is that we aren't just looking at what's happening now; we are using a model to imagine what will happen over the next h steps based on the specific action we plan to take.
Dev: It’s about getting failure information from imagined consequences of actions, which they demonstrate with counterfactual action-swap tests and corrupted action tests that collapse accuracy down to near chance.
Rosa: And they show this system works live on a real robot without having to modify the underlying visuomotor policy or search for candidate actions when it detects a risk.
Taro: The limitation they flag is that this method prevents failures by abstaining, but it doesn't recover from them or complete the task after an abort, and they suggest that extending this to longer-horizon skills would require some kind of hierarchical monitoring.
Dev: So, ContactGuard gives us a reliable way to intervene before contact in complex manipulation tasks where conventional detectors are too slow or unreliable for pre-contact checks.
Conclusion: Rosa: So, we’re wrapping up on ContactGuard, which is really about this pre-contact execution monitoring system for chunked visuomotor policies.
Dev: Yeah, it’s this system that predicts the latent space consequence of a planned action before actual contact happens and aborts if it thinks things are going to go wrong.
Rosa: The authors, they’re focused on this joint-embedding predictive architecture principle, using next-latent supervision to train a world model from unlabelled robot trajectories.
Dev: That means they’re not trying to predict pixels directly; they use a shared vision transformer and mean-pool the camera embeddings to get one latent representation per view.
Rosa: And then they use an action conditioned causal predictor, with those four AdaLN zero conditioned transformer blocks, to forecast future latents under the planned actions.
Dev: That prediction is then fed into a lightweight failure probe, which is just a logistic regression model trained on a small labelled set of pre-contact clips.
Rosa: So they anchor this prediction fifteen frames before planned gripper closure, roll the frozen latent world model forward thirty steps under those planned actions, and check if the predicted post-contact latent indicates likely failure.
Dev: The key result here is that it performs better than direct and corrupted action ablations on contact-rich manipulation tasks across four different setups like cup grasping or towel folding.
Rosa: And they showed that this prediction quality comes from imagining the consequence of the specific proposed action, not just looking at current observations or raw planned actions.
Dev: They even ran a counterfactual test where they replaced only the planned action with one from a failed attempt and saw the predicted failure probability go up.
Rosa: It’s a live robot system, it transfers to your existing policy as a pre-contact abort signal without you having to change the underlying policy or use pixel level video prediction.
Dev: The paper points out that while it’s good at preventing failures right before contact, it doesn't recover from them if they happen anyway and they suggest longer horizon skills would need hierarchical monitoring.
Rosa: So ContactGuard gives you a way to stop the robot from pushing or slipping before it even starts making contact in those tricky situations.
Dev: It’s a big step because it moves the safety check into the planning phase, which is crucial for real-time control loops.
More episodes
- 2610.34593-Future Prediction as Training-Only Video Supervision for Pretrained VLA Policies
- 2610.35385-EmbodiedLab: Beyond Manipulation, Benchmarking Embodied Agents That Act to Know
- 2610.36378-Data-Driven Hamiltonian for Neural Optimal Control
- 2610.37275-A Heuristic for Scheduling Autonomous Vehicles at Intersections
- 2610.37288-Dynamic Coordination of Autonomous Vehicles at Intersections
- 2610.37632-Real-Time Coordination of Autonomous Vehicles at Intersections
- 2611.13779-Softness is something you also need! Why compliance helps control: information metrics of a passive walker with compliant ankle
- 2611.13411-Residual-Corrected Physics-Informed Neural Network KKL Observers with Input-to-State Stability Guarantees
- 2611.14488-Is Your Robot's Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies
- 2611.14470-TanCRAFT: Context-Reasoned Anticipatory Feedback for Assembly Tasks