RESETTLE: Robotic Recovery through Disagreement-Triggered Retrieval and Efficient Corrective Control

arXiv:2610.12185 · cs.RO · Submitted 2026-10-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "RESETTLE: Robotic Recovery through Disagreement-Triggered Retrieval and Efficient Corrective Control".

Dev: The gist:

Rosa: First, who's behind it and why it matters.

Title and authors: Dev: So, putting the title and authors aside, let's look at the actual summary of "RESETTLE: Robotic Recovery through Disagreement-Triggered Retrieval and Efficient Corrective Control." It breaks it down into how they use disagreement to find a fix.

Rosa: Right. The paper summarizes that RESETTLE is an action-level recovery framework for frozen policies, and its main mechanism is triggering intervention when two action proposals sampled independently under identical conditions persistently disagree.

Taro: The researchers point out that this disagreement isn't just random noise; it’s a signal that something is really wrong with how the robot is deciding what to do next.

Dev: When that signal hits, they retrieve a successful demonstration reference using an adapted V-JEPA encoder and combine it with a state-servo prior and a visual residual to execute one corrective action.

Rosa: So they’re not just stopping the robot; they’re retrieving a proven successful path from training data for that exact situation and then using that reference to guide the physical correction.

Taro: The V-JEPA encoder part sounds like it's what maps the current messy situation into something that lets it find the closest good example quickly, which is pretty smart.

Dev: And they emphasize that this entire process happens without needing base-policy fine-tuning, no extra vision-language calls for the recovery itself, and no online trajectory optimization needed.

Rosa: That efficiency is key, especially when you're dealing with real hardware where latency matters a lot.

Taro: They also mention that they validate this in simulation and real-world manipulation across various tasks like LIBERO-Plus, Meta-World, and RoboCasa Tabletop.

Dev: And the numbers are pretty compelling for their latency reduction. They claim their monitoring and recovery computation latency is between seventy-four point zero four percent and ninety-three point five seven percent lower than VoLoAgent’s monitoring and planning time for grasp or place calls.

Rosa: That level of speed difference is significant when you're talking about rapid, in-the-moment adjustments needed during a manipulation task.

Taro: It’s interesting how they show this local recovery actually complements high-level agentic planning when they integrate it with Harness VLA on LIBERO-Pro Swap, raising the success rate from forty-two percent to fifty percent.

Dev: So the paper suggests that this type of localized correction can work well even when you have a higher-level planner coordinating the overall task.

Rosa: They also show that they actually validate this in simulation and real-world manipulation across various tasks like LIBERO-Plus, Meta-World, and RoboCasa Tabletop.

Taro: What they really push is that while disagreement is informative, it's not a perfect failure label; some successful trajectories can still show trigger rates under noise.

Dev: They also note that persistent disagreement doesn't automatically mean failure; some cases show heavier tails beyond their training-derived P95 threshold.

Rosa: They also point out a limitation: disagreement alone can’t tell you if a robot is doing something that looks stable but is actually incorrect behavior.

Taro: And they quantify this by showing that recovery usage only accounts for about two point six seven percent of the executed actions, meaning most of the time, the base policy just keeps running fine.

Dev: They also note that persistent disagreement doesn't automatically mean failure; some cases show heavier tails beyond their training-derived P95 threshold.

Rosa: And they quantify this by showing that recovery usage only accounts for about two point six seven percent of the executed actions, meaning most of the time, the base policy just keeps running fine.

Taro: They also point out a limitation: disagreement alone can’t tell you if a robot is doing something that looks stable but is actually incorrect behavior.

The paper's summary: Rosa: Now we get into what they suggest for making RESETTLE even better, moving beyond just the core mechanism, right? They’re talking about adding more context to that recovery process.

Dev: Right. The paper points out that the current system relies heavily on just state information when it finds that reference demonstration, and if the robot is in a slightly different physical configuration than what was in the training data, relying only on state displacement might not give you a good correction.

Taro: So they propose combining that state-servo prior with something visual, like a guarded visual residual.

Rosa: What does that mean for us listening? It means when the system tries to move toward the reference state, it checks what’s actually visible right now and adjusts its movement based on that view.

Dev: That adjustment is constrained by some rules—like keeping the visual difference small and not letting the correction become too aggressive relative to how far away you are from the target state.

Taro: It’s like giving the robot a sense of where it *should* be, but making sure that sense aligns with what its camera actually reports at that moment.

Rosa: It sounds like they’re making the correction more scene-aware, not just purely math-based based on coordinates.

Dev: And they use this visual feedback to refine the state-servo prior, which is a motion command in native robot coordinates.

Taro: That’s smart because it lets the system react to things happening in its immediate environment that aren't just tracked by its internal state variables.

Rosa: So, if the robot gets slightly bumped or something shifts visually, this mechanism helps it stay on track without needing a whole new plan.

Dev: It keeps the loop rate manageable because it’s just one quick correction step, not a full re-planning cycle.

Taro: The authors are also hinting at future work here—they mention that for really tricky failures, like an object getting lost or trapped, this local correction might just not be enough anymore.

Rosa: They acknowledge that disagreement monitoring is great for when the robot makes a mistake in its decision-making, but it can’t solve physical problems where the path itself is blocked.

Dev: So they’re suggesting that while RESETTLE handles execution errors well, you still need to think about high-level planning for things like sequencing a release and then a regrasp move.

Taro: Exactly. This paper shows how to handle the "mid-execution stumble," but it leaves the "stuck in a corner" problem for later research.

The paper's improvements: Rosa: So to wrap up this look at RESETTLE, we’re talking about how this framework uses disagreement between two AI action samples to trigger a fast, reference-based correction when things go wrong during robot execution.

Dev: Yeah, it’s basically a low-latency way to patch policy errors without needing massive re-training or heavy online planning.

Taro: It changes the autonomy game by showing we can add this kind of targeted local fix on top of a frozen policy, which is something we really need for real robots operating in messy environments.

Rosa: The implication is that frozen policies don't have to be completely rigid; they can be made resilient through these selective recovery mechanisms.

Dev: We saw the numbers showing it’s incredibly efficient, with monitoring latency being much lower than some other systems we tested, and it only uses recovery for a small fraction of actions.

Taro: And while it’s great for execution errors, the authors are clear that disagreement alone doesn't solve every problem; you still have to plan out what to do when the object is physically stuck or lost.

Rosa: Exactly, so it’s a very useful tool for handling decision-making inconsistencies during movement, but it doesn't replace the need for complex task sequencing.

Dev: It’s a practical method for improving execution reliability while keeping the underlying policy structure intact, which is important when you can’t afford to retrain everything constantly.

Taro: I think that points toward a future where we build systems that are both smart enough to make decisions and resilient enough to recover intelligently when those decisions lead them astray.

Rosa: And for now, this paper on RESETTLE shows us how to add a layer of intelligent, data-driven repair right at the action interface.

Dev: It’s a solid piece of engineering that focuses on making the recovery mechanism as lean and fast as possible.

Taro: It definitely sets a good baseline for what local correction looks like in this area before we tackle those more complex, multi-step failures.

Conclusion: Tom: So we’ve looked at how RESETTLE uses disagreement between two AI action samples to trigger a fast, reference-based correction when things go wrong during robot execution.

Rosa: Yeah, it’s basically a low-latency way to patch policy errors without needing massive re-training or heavy online planning.

Dev: It’s a solid piece of engineering that focuses on making the recovery mechanism as lean and fast as possible.

Taro: It changes the autonomy game by showing we can add this kind of targeted local fix on top of a frozen policy, which is something we really need for real robots operating in messy environments.

Rosa: The implication is that frozen policies don't have to be completely rigid; they can be made resilient through these selective recovery mechanisms.

Dev: We saw the numbers showing it’s incredibly efficient, with monitoring latency being much lower than some other systems we tested, and it only uses recovery for a small fraction of actions.

Taro: And while it’s great for execution errors, the authors are clear that disagreement alone doesn't solve every problem; you still have to plan out what to do when the object is physically stuck or lost.

Rosa: Exactly, so it’s a very useful tool for handling decision-making inconsistencies during movement, but it doesn't replace the need for complex task sequencing.

Dev: It’s a practical method for improving execution reliability while keeping the underlying policy structure intact, which is important when you can’t afford to retrain everything constantly.

Taro: I think that points toward a future where we build systems that are both smart enough to make decisions and resilient enough to recover intelligently when those decisions lead them astray.

Rosa: And for now, this paper on RESETTLE shows us how to add a layer of intelligent, data-driven repair right at the action interface.

Dev: It definitely seems like they're showing how you can get significant gains in task success rates on different benchmarks just by improving the execution phase locally.

Taro: And looking ahead, the authors suggest that for things that are harder than a single action correction—like an object getting stuck—you might need to plan out a whole sequence of release and regrasp moves rather than just one quick fix.

Rosa: That’s what it boils down to for us—a way to make the execution phase of robotic tasks much more reliable without slowing things down too much.

Dev: The overall message of RESETTLE: Robotic Recovery through Disagreement-Triggered Retrieval and Efficient Corrective Control is that you can get efficient action-level recovery by focusing on disagreement monitoring and using retrieved successful demonstrations to guide a quick, reference-based correction.

Taro: It’s a practical framework that adds robust local correction to frozen policies without needing massive retraining or complex online planning loops.

The Hong Kong University of Science and Technology · The Chinese University of Hong Kong

cs.RO

Submitted: 2026-10-08

Updated: 2026-10-08

Code: https://github.com/JIA-Lab-research/RESETTLE

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 90/100

The gist: The gist: RESETTLE introduces a model-agnostic framework that provides computationally efficient recovery at the action-execution interface of frozen robot policies by triggering recovery when two

Key concepts

Disagreement-Based Intervention
The system monitors two separately sampled actions under the same conditions. If these actions disagree consistently beyond a set threshold, it signals potential uncertainty or instability in the policy's decision. This disagreement acts as an early warning signal to trigger a recovery attempt.
V-JEPA-Based Retrieval
When intervention is triggered, an adapted V-JEPA encoder maps the current observation into a representation space. It then searches for and retrieves the visual frame from successful training demonstrations of the same task. This retrieved frame provides both the visual context and robot state needed for guiding corrective action.
Reference-Guided Recovery
Recovery is achieved by combining a state-servo prior, which moves the robot toward a known good reference state in its native coordinates, with a guarded visual residual. The residual refines this motion based on current and reference observations while respecting constraints like minimum similarity to ensure safe and effective correction.

Terminology

Summary

The gist: RESETTLE introduces a model-agnostic framework that provides computationally efficient recovery at the action-execution interface of frozen robot policies by triggering recovery when two action proposals independently sampled under identical conditioning persistently disagree, retrieving a same-task demonstration reference using an adapted V-JEPA encoder and combining a state-servo prior with a guarded visual residual to execute one corrective action without online trajectory optimization or additional vision-language reasoning, then returns control to the base policy<ref:2610.12185#pg5>.

How it works

RESETTLE operates by monitoring two independently sampled action proposals under identical conditioning and triggering intervention when their disagreement persists beyond a calibrated threshold at two consecutive decisions<ref:2610.12185#pg5>. This disagreement serves as a proxy for decision uncertainty rather than a direct test of physical feasibility or task failure<ref:2610.12185#pg5>. When intervention is triggered, the framework proceeds through three main steps:

i) Disagreement-Based Intervention: It measures the temporal weighted disagreement between two action chunks sampled with independent seeds under identical observations and task conditioning<ref:2610.12185#pg5>. The threshold calibration uses training-data inference P95, except in the dedicated threshold-percentile ablation<ref:2610.12185#pg5>.

ii) V-JEPA-Based Retrieval: When triggered, the adapted V-JEPA-2 encoder maps the current observation into a pooled representation to retrieve the nearest representation by cosine distance from successful training demonstrations of the same task<ref:2610.12185#pg5>. This retrieved frame supplies both its visual observation and the associated robot state for corrective control<ref:2610.12185#pg5>.

iii) Reference-Guided Recovery: A state-servo prior, defined by the displacement from the current to the reference robot state, provides motion toward this reference state in native control coordinates<ref:2610.12185#pg5>. This is refined by a guarded visual residual that uses current and reference observations to adjust this prior, constrained by minimum cosine similarity and maximum residual-to-prior norm ratio<ref:2610.12185#pg5>.

Key Components and Training

The recovery module is trained jointly with the context encoder and auxiliary dynamics predictor from V-JEPA-2-AC, initialized from a checkpoint pretrained on DROID<ref:2610.12185#pg4>. The training objective is a combination of several losses: L = Lres + λnativeLnative + λdynLdyn + λgateLgate + λsatLsat<ref:2610.12185#pg5>. Specifically, the residual loss Lres is the diagonal Gaussian negative log-likelihood, while the native-action loss Lnative applies a Huber loss between candidate recovery command and supervised native command<ref:2610.12185#pg5>. The dynamics predictor objective Ldyn encourages shared representations to capture action-dependent visual transitions alongside residual-action supervision<ref:2610.12185#pg5>.

Performance and Efficiency

RESETTLE has demonstrated significant improvements in task success rates across various benchmarks, achieving up to 8.70%, 6.28%, and 6.83% absolute success-rate gains on LIBERO-Plus, Meta-World, and RoboCasa Tabletop, respectively<ref:2610.12185#pg5>. In QwenPI-based comparisons, its monitoring-and-recovery computation latency is 74.04%–93.57% lower than VoLoAgent’s monitoring-and-planning latency for grasp and place tool calls<ref:2610.12185#pg5>. Furthermore, the integration with Harness VLA on LIBERO-Pro Swap raised the LIBERO-Pro Swap success from 42% to 50%, demonstrating compatibility with high-level agentic planning<ref:2610.12185#pg5>.

Analysis of Monitoring and Recovery

The analysis of when to intervene shows that training-data inference and successful executions align in the dominant low-disagreement region, whereas failures exhibit heavier tails beyond the training-derived P95 threshold<ref:2610.12185#pg10>. Persistent disagreement is informative but not a definitive failure label, as successful trajectories can still show nonzero trigger rates under noise<ref:2610.12185#pg10>. The actual recovery usage shows that recovery accounts for only 49,528 of the 1,855,166 executed actions (2.67%), leaving 97.33% of actions under base-policy control<ref:2610.12185#pg10>. The distribution of takeovers also varies substantially across perturbations, with robot-initialization shifts requiring more repeated state corrections<ref:2610.12185#pg10>.

Limitations and Future Directions

The paper notes that disagreement alone cannot distinguish correct, stable behavior from stable but incorrect behavior<ref:2610.12185#pg31>. Failures despite triggered recovery show that the intervention criterion does not guarantee completion within the evaluation budget, as seen in cases where object loss or entrapment remains unresolved<ref:2610.12185#pg31>. Future improvements could involve contact-aware reference selection or explicit planning of release and regrasp sequences to address configurations that repeated local corrections do not resolve<ref:2610.12185#pg33>. The present design focuses on selective local correction with low measured computation cost and leaves broader mechanisms outside the evaluated system<ref:2610.12185#pg33>.

Real-World Validation

Experiments on a Franka Research 3 robot across four single-arm manipulation tasks show that RESETTLE improves on all four tasks for both policies, with the average task metric rising from 66.67% to 78.75% for QwenPI and from 86.25% to 92.92% for VLAct<ref:2610.12185#pg35>. These gains cover binary success on the three short-horizon tasks and the long-horizon task ‘Use Spoon’<ref:2610.12185#pg35>. The results demonstrate that local corrective control remains useful when a high-level planner already coordinates execution<ref:2610.12185#pg9>.

Computational Tradeoffs

The total latency for RESETTLE is approximately 83.6 ms, which is 61.30% lower than Agentic Robot’s and 82.57% lower than adapted LPB’s, and 74.04–93.57% lower than VoLoAgent’s place and grasp branches<ref:2610.12185#pg10>. This sparse use of single-action corrections supports RESETTLE as a lightweight recovery mechanism that intervenes selectively when the disagreement criterion indicates instability, while preserving base-policy control for most of execution<ref:2610.12185#pg10>. The monitoring overhead is approximately zero in these implementations, with 83.6 ms recovery<ref:2610.12185#pg10>.

Conclusion

We introduced RESETTLE, a model-agnostic framework for efficient action-level recovery with frozen robot policies that combines disagreement-based intervention, retrieval of successful same-task references, and corrective control through a state-servo prior with a guarded visual residual<ref:2610.12185#pg10>. Each intervention executes one corrective action before returning the control to the base policy, requiring neither base-policy fine-tuning nor online trajectory optimization or additional VLM reasoning calls for recovery<ref:2610.12185#pg10>. Experiments across six policies and three manipulation benchmarks demonstrate that our method can greatly improve task completion, and the integration of our method with Harness VLA shows that the local recovery complements high-level planning<ref:2610.12185#pg10>. The analyses support action disagreement as an informative and low-overhead monitoring signal, and demonstrate the benefit of adding a guarded visual residual to reference-derived state-servo control<ref:2610.12185#pg10>. Together, these findings highlight that the reference-guided corrections can serve as a practical way to improve robotic execution while preserving the capabilities of the deployed policy<ref:2610.12185#pg10>.

References

Christopher Agia, Rohan Sinha, Jingyun Yang, Zi-ang Cao, Rika Antonova, Marco Pavone, and Jeannette Bohg. Unpacking failure modes of generative policies: Runtime monitoring of consistency and progress<ref:2610.12185#pg11>. Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, et al. V-jepa 2:

Improvements for AI systems

  1. Identification of unstable action selection using persistent disagreement: The system can be improved by monitoring two independently sampled proposals under identical conditioning persistently disagree and triggering recovery when disagreement exceeds a calibrated threshold at two consecutive decisions, which suppresses responses to isolated sampling fluctuations.

  2. Retrieval of task-relevant demonstrations: The system can be enhanced by using the adapted V-JEPA encoder to retrieve the nearest same-task demonstration reference in its representation space upon triggering recovery, thereby providing an empirical anchor of successful behavior for corrective control.

  3. Refinement of corrective action with scene context: Recovery actions can be improved by combining a state-servo prior with a guarded visual residual, where the residual is conditioned on current and reference observations to adjust this prior, allowing the system to execute actions that are scene-conditioned rather than just state-based.

  4. Latency reduction in monitoring and recovery: The system can achieve significantly lower computation overhead by ensuring that RESETTLE produces one corrective action instead of complex planning or trajectory preparation, resulting in a monitoring latency that is 74.04%–93.57% lower than VoLoAgent’s monitoring-and-planning latency.

  5. Compatibility with high-level planning: The system can be integrated into agentic frameworks by demonstrating that local recovery complements high-level planning, showing that the local corrective control is effective even when paired with a GPT-5.5 planner in Harness VLA settings.

Sources

Related papers