VLD: Visual Learned Distance as an Interface for Reinforcement Learning Navigation

summary

The gist

Training end-to-end policies from image data to directly predict navigation actions for robotic systems has proven inherently difficult The VLD framework decouples perception learning from policy…

In short

The VLD framework decouples perception from policy by training a self-supervised distance predictor on internet-scale video data to estimate distances to goals. This distance signal is then used by a reinforcement learning policy for navigation in simulation and the real world. The system achieves high success rates and robust transferability, proving that using learned semantic goal information can create reliable, multimodal navigation policies.

Key concepts

VLD Framework
This framework separates perception learning from policy learning. It first trains a predictor on massive video data to estimate the distance between an image and a specified goal. This distance prediction then serves as a compact signal for the reinforcement learning policy to learn how to navigate, allowing policies to be trained in simulation using these learned signals.
Temporal Distance Function (Td)
This function estimates the expected time or horizon between where the robot is currently observing and a specific goal. It is trained by sampling observation pairs along the same path for positive examples and using negative mining to identify mismatched goals, producing calibrated predictions with uncertainty estimates.
Geometric Overlap Noise (GeoNoise)
This noise model mimics the uncertainty patterns of the VLD predictor by conditioning perturbations on features like projection success ratio and camera rotation. It generates structured errors, such as lower confidence when a goal is out of view, which helps train the RL policy to behave correctly in challenging visual situations.

Terminology used across episodes

This episode discusses

The paper

VLD: Visual Learned Distance as an Interface for Reinforcement Learning Navigation · Read on arXiv

Lazar Milikic, Manthan Patel, Jonas Frey

ETH Zurich · EPFL · Stanford University · UC Berkeley

Training end-to-end policies from image data to directly predict navigation actions for robotic systems has proven inherently difficult. Existing approaches often suffer from either the sim-to-real gap during policy transfer or limited action-labeled training data. To address this problem, we introduce Visual Learned Distance (VLD), a scalable framework for image-goal navigation that decouples perception learning from policy learning. We first train a self-supervised distance-to-goal predictor on about 3,000 hours of internet-scale and simulated video, providing a distance signal that a reinforcement learning (RL) policy can minimize. The RL policy is trained separately in simulation using privileged geometric distances perturbed by GeoNoise, a learned noise model that reproduces the viewpoint- and overlap-dependent errors of the trained distance predictor. At deployment, the policy consumes VLD predictions zero-shot, without retraining, inheriting ``where to go'' from large-scale visual training while retaining the robust navigation behaviors learned in simulation. This error modeling is key to the transfer: with VLD inputs, GeoNoise-trained policies reach 58.7% success on HM3D and 76.2% on Gibson, whereas policies trained with generic temporally correlated noise collapse to 4.9% and 11.5%. On a physical robot, the VLD-driven policy succeeds in 88% of trials, compared with 60% for a strong end-to-end image-goal baseline that outperforms it in simulation. Our decoupled design thus provides an alternative and scalable path toward reliable navigation policies that transfer from simulation to the real world.

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "VLD: Visual Learned Distance as an Interface for Reinforcement Learning Navigation".

Dev: Here's the gist of the paper. Training end-to-end policies from image data to directly predict navigation actions for robotic systems has proven inherently difficult The VLD framework decouples perception learning from policy learning by training a self-supervised distance-to-goal predictor on internet-scale video data and using it as a compact signal for reinforcement learning navigation policies This design allows the RL policy to be trained entirely in simulation using privileged geometric distance signals, with injected noise to mimic the uncertainty of the trained distance predictor The framework enables a scalable path toward reliable, multimodal navigation policies by inheriting semantic goal information from large-scale visual training while retaining robust low-level navigation behaviors learned in simulation.

Rosa: Pruning redundant reasoning tokens The core idea of the Vision–Language Distance (VLD) learning is to decouple perception learning from policy learning by training a self-supervised distance-to-goal predictor on internet-scale video data and using it as a compact signal for reinforcement learning navigation policies.

Dev: Instead of relying on raw sensory inputs during policy training, the framework first trains this predictor to generalize across both image- and text-based goals, providing a distance signal that can be minimized by an RL policy.

Rosa: This allows the RL policy to consume VLD predictions to navigate in simulated and real-world environments, inheriting semantic goal information from large-scale visual training while retaining robust low-level navigation behaviors learned in simulation.

Paper summary: Rosa: So we're looking at this paper called "VLD: Visual Learned Distance as an Interface for Reinforcement Learning Navigation". The main idea here is to tackle how hard it is to train these end-to-end navigation policies directly from images, especially because of that gap when you move from simulation to the real world.

Dev: Right. It's about decoupling perception learning from policy learning by training a distance-to-goal predictor on massive amounts of video data and using that as a signal for reinforcement learning policies. This approach lets the RL policy train entirely in simulation using these geometric distance signals, even with some noise to mimic how uncertain the predictor actually is.

Taro: I'm curious about what this means for when we actually put these systems on a physical robot. If we can get robust low-level navigation learned in simulation, but then replace those simulator signals with VLD predictions at deployment, that sounds like a big win for sim-to-real transfer.

Rosa: Exactly. The authors are using internet scale video data to train this distance predictor so it understands both image and text goals. This gives the RL policy something much richer than just raw sensor inputs during training, allowing it to inherit semantic goal information from that large visual training set while keeping its basic navigation skills solid from simulation.

Dev: They build this temporal distance function, which we call Td of observation time and goal specification, by doing a self-supervised regression problem. They create positive examples by just taking observation pairs along the same path so the temporal distance is simply the difference in time between them.

Taro: And they also use negative mining to make sure the model recognizes when goals are mismatched across different paths, assigning a maximum temporal distance to those outliers. How do they actually train this predictor to give us reliable distances?

Rosa: They use an inlier-outlier Gaussian mixture Negative Log-Likelihood loss. That's what gives them calibrated predictions and uncertainty estimates, which is important because it tells the policy how much it can trust the distance signal.

Dev: Before we get into how they train the policy, Rosa mentioned something about validating these distance predictors on their own merits, not just by seeing if the downstream navigation policy succeeds. They introduce a direct evaluation method based on ordinal consistency using Kendall's tau, which checks if predicted distances maintain the correct ordering of progress toward the goal.

Paper summary: Taro: So they are treating the distance estimator as a product unto itself? That seems like it could be really useful for debugging perception models later on.

Rosa: It is. They show that VLD outperforms prior temporal distance approaches like ViNT and VIP by giving us a principled way to judge the predictor in isolation, not just looking at how well the navigation policy does its job downstream.

Dev: And for deployment, they inject noise into the geometric distance signals with a lightweight MLP trained to mimic the error distribution of their learned predictor. That prepares the RL policy for when it sees those noisy signals in reality.

Taro: What kind of structured errors are we talking about? Does this noise model capture things like when the goal is just out of view or behind the agent?

Rosa: Yes, they design this geometric overlap noise to mimic those uncertainty patterns by conditioning it on features like projection success ratio and relative camera rotation. This means the error structure isn't just random; it has specific shapes that match what their VLD model learns under that mixture negative log-likelihood objective.

Dev: That structured error pattern is key because it induces behaviors in the policy, like performing periodic three hundred sixty degree scans at important locations, which is something you want for real-world navigation.

Taro: So the noise isn't just random static; it’s mimicking the actual failure modes of their perception model. That makes sense for making a robot behave reasonably when things get messy outside the lab.

Rosa: It also helps with robustness even when the goal is visually out of view, because VLD can still maintain ordinal consistency by correctly identifying the closer viewpoint based on ground truth geodesic distance, even without direct visual overlap.

Dev: That's a strong claim for long-horizon perception gaps. The multimodal version of their work tracks the image and language goals very closely, suggesting that linguistic cues can reinforce the visual distance estimation while keeping that ordering structure intact.

Taro: If text alone can still provide semantically grounded distances, even with weaker supervision than a full image goal, that opens up some possibilities for systems where visual input is unreliable.

Rosa: It shows they are building a scalable bridge between large scale perception models and reinforcement learning control by training the predictor on diverse datasets like synthetic environments, internet walking videos, and embodied datasets.

Paper summary: Dev: The deployment stage involves training policies conditioned on sensory information and that distance-to-goal signal, where the simulator signal gets swapped out for VLD predictions. They report a success rate of zero point seven three when swapping the distance signal because policies trained on noisy ground truth distances transfer effectively to the VLD at deployment without retraining.

Taro: So they are showing zero-shot replacement capability based on geometric overlap noise inducing robustness to those characteristic error patterns of VLD, which is pretty powerful for field testing a robot.

Rosa: And in real-world experiments on a physical robot, the VLD-driven policy achieved a success rate of ninety-three point three percent, which outperforms image-based navigation policies they tested. That's where the practical validation comes in for me and Dev.

Dev: The paper also explores how this works across different goal specifications, including image-only and text-only goals, showing that the text-only VLD model still achieves ordinal consistency above random performance even after those baselines are fine tuned.

Taro: That suggests VLD can form meaningful navigation distances from text alone despite having a weaker supervision signal than when you have an image goal.

Rosa: It confirms their hypothesis that decoupling perception from control helps improve sim-to-real robustness, and it gives us this scalable path toward reliable, multimodal navigation policies by combining robust and transferable reinforcement learning control with scalable semantic understanding.

Dev: The authors conclude that the VLD framework successfully bridges large scale perception models and embodied action by training a distance-to-goal predictor on large scale visual data and using it as a compact signal for reinforcement learning navigation policies.

Taro: What about the limitations? Where does this framework stop working?

Rosa: They flag that while the multimodal model closely tracks the image goal variant, they still have to carefully manage how those different goal specifications interact during deployment. That is one thing they point out as an area needing more work.

Dev: The real limitation is that the performance still depends heavily on the quality and diversity of the internet scale video data used for training their predictor; if that data doesn't cover a specific scene type, the generalization might not hold up as well as expected.

Taro: So it’s not a perfect solution out of the box; you still need good data to keep those robust performance metrics high.

Rosa: Exactly. But overall, they show that even with this interface, you get effective navigation behavior and a scalable bridge between large scale perception models and reinforcement learning control. That's what we saw in these results for the VLD paper.

Conclusion: Rosa: So we're wrapping up on VLD, which is this framework that uses learned distance predictions from images to guide reinforcement learning policies for navigation in robots.

Dev: Yeah, so basically, they decoupled how the robot learns to move from what it sees by training a separate predictor on massive video data and feeding that signal into the control loop.

Rosa: The authors call it VLD—Visual Learned Distance as an Interface for Reinforcement Learning Navigation. It’s really about creating this bridge between huge perception models and actual robot action.

Dev: It seems like the core idea is that you don't need the policy to understand every single pixel perfectly; you just need a reliable distance estimate, and they showed how you can train that distance estimator on diverse data—both images and text—to be really robust.

Rosa: And what’s interesting is how they tested this predictor separately using ordinal consistency checks, which means they were judging the quality of the distance signal before even putting it into a robot loop.

Dev: That's crucial for safety, because if the distance estimate is wrong in a weird way, you need to know why before you let it drive anything.

Rosa: The results show that when you take that signal and deploy it on a real robot, the policy transfers pretty well from simulation to reality without needing retraining.

Dev: That’s the sim-to-real transfer part we were talking about; they found a success rate of ninety three point three percent in physical tests, which is solid for this kind of interface.

Rosa: It proves that even if you have a minimal interface like this distance signal, you can get effective navigation behavior that handles things like goal being out of view reasonably well.

Dev: And it also shows that the system explores by doing things like spinning around periodically when it's uncertain about the direction, which is a good way to handle those real-world perceptual gaps.

Rosa: So, VLD seems to be building a scalable path toward navigation policies that have both strong low-level control and high-level semantic understanding from big data.

Dev: It’s definitely a step in making these systems more reliable across different real-world conditions, but we still need to see how this holds up when the visual data is completely different from what it was trained on.

More episodes

← Home