RoboAtlas: Contextual Active SLAM

arXiv:2606.26046 · cs.RO, cs.CV · Submitted 2026-06-24 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "RoboAtlas: Contextual Active SLAM".

Dev: The gist RoboAtlas presents a contextual Active SLAM framework that adaptively balances geometric exploration and semantic reasoning using a scalable 3D semantic mapping system, OpenRoboVox.

Rosa: First, who's behind it and why it matters.

Title and authors: Rosa: So we're looking at RoboAtlas today, which is this contextual Active SLAM paper. It seems like the core idea is balancing physical exploration with using semantic understanding to actually find what someone wants in the environment.

Dev: Exactly. They call it a mixture-of-experts framework where you get frontier exploration, global map reasoning, and egocentric visual language model reasoning working together through this contextual multi-armed bandit thing.

Taro: What I’m hearing is that instead of just wandering around randomly, the robot gets smarter about where to go next based on what it knows and what it sees right now.

Rosa: Right. The paper describes how this works by having three experts proposing a goal, and then the bandit choosing which one to follow based on the current state of the robot and its surroundings.

Dev: It’s all about that context vector c t, which summarizes things like grid coverage and similarity metrics like Vsimt, used to decide whether to explore more or look for something specific.

Taro: And that reward function they use, it’s weighted by things like the change in occupancy and the confidence from the VLM, which means it rewards actions that cover new ground but also keep an eye on what's semantically relevant.

Rosa: That transition from broad exploration to targeted semantics-driven navigation is a big part of what they’re showing here, as it improves performance as the map gets more complete.

Dev: And to make this happen in the real world, they rely on OpenRoboVox for their three dee semantic mapping system <ref:2606.26046#pg1>. This framework handles memory constraints on platforms like the Unitree Go2 by using things like truncated signed distance field optimization and parallel dynamic-instance pruning.

Taro: So the hardware side is important here because a huge map can be computationally expensive, and they’ve put in specific architectural changes to manage that load.

Rosa: They also introduce a Scene-Dictionary St, which is this compact way of representing objects with their semantic descriptions and spatial properties instead of just millions of voxels. This makes the global reasoning much more feasible.

Dev: And to help the semantic map expert query that dictionary, they use an asynchronous interface that filters candidates based on semantic potential fields before feeding them to a large language model backend.

Title and authors: Taro: That filtering stage is smart because it prevents the LLM from being overwhelmed by looking at every single object in the scene.

Rosa: The LLM then ranks these candidates based on both semantic relevance and spatial plausibility, and the system picks the centroid location of the highest-scoring instance among a top N set.

Dev: And for when you’re looking at what the robot sees right now, they have an egocentric VLM expert that takes a query and an RGB image to estimate target relevance.

Taro: That VLM expert is also crucial because it can generate textual scene descriptions which feed back into the semantic map context, helping the system recover from things like occlusions.

Rosa: The results they got on GOAT-Bench “Val Unseen” with GPT-4o showed a highest reported success rate of ninety point six percent, improving over prior baselines by seventeen point eight percentage points in success rate <ref:2606.26046#pg1,by 17.8 percentage points in>.

Dev: And that’s impressive, but they also tested it on the Qwen2 point 5-VL-7B model, and it still managed an eighty-eight point eight percent success rate using GPT-4o, showing the value of the information gained by their system <ref:2606.26046#pg1,Qwen2.5-VL-7B model>.

Taro: It shows that grounding foundation models with these large-scale three dee semantic maps can lead to robust and efficient contextual Active SLAM in large-scale real-world environments exceeding one thousand eight hundred m2 with about 30k mapped semantic instances <ref:2606.26046#pg1,in large-scale real-world environments exceeding 1800 m2 with>.

Rosa: The paper’s main point is that this adaptive switching between geometric exploration and semantic guidance is effective across different stages of exploration, which means it’s not just a one-size-fits-all approach.

Dev: And they show that the contextual multi-armed bandit strategy adapts its behavior according to both the environment state and the reward structure, achieving higher success rates and shorter path lengths compared to any single fixed approach.

Taro: I noticed in their analysis that the egocentric VLM expert receives a large share of picks, about forty-five point zero percent of those decisions, but it only contributes about thirty point six percent of the total path length, acting more like a fine-grained corrector for local positioning.

Rosa: That makes sense because it’s focused on immediate visual cues and local refinement rather than long-range planning or global map coverage.

Title and authors: Dev: The semantic map expert is selected about thirty-eight point seven percent of the time, but it accounts for the majority of total path length at about forty-six point eight percent, which confirms its role as the primary channel for exploitation once things get more informed.

Taro: So what’s the real constraint they flag? They admit that both LLM-based reasoning and similarity-based retrieval depend strongly on the quality of the underlying three dee semantic map <ref:2606.26046#pg1>.

Rosa: Right, that’s a big caveat because if the map itself is poor, all that high-level reasoning might be built on shaky ground.

Dev: They also point out that relying on LLM and VLM inference introduces latency and variability in response time, which could limit deployment in settings that require very fast global decision making.

Taro: So the idea is adaptive switching based on map completeness, but the reliance on these models means you can’t get perfectly fast decisions everywhere yet.

Rosa: In summary, RoboAtlas introduces this contextual Active SLAM framework combining frontier exploration and semantic reasoning via a contextual multi-armed bandit to adapt its strategy as scene understanding improves across simulation and hardware experiments.

Dev: It bridges geometric exploration and semantic reasoning by adapting its decision policy based on the environment's state, showing that this adaptive switching between guidance is effective across different stages of exploration.

Taro: The final result is a ninety point six percent success rate and fifty-three point four percent path length at the GOAT-Bench benchmark using GPT-4o, exceeding the strongest prior baseline by seventeen point eight percentage points on success rate, which indicates adaptive switching between geometric and semantic guidance is effective across different stages of exploration.

Rosa: So this paper is really showing how context can drive navigation in a complex environment by mixing physical exploration with high-level semantic reasoning to get better results than fixed policies.

Dev: It’s about making the robot adapt its navigation strategy rather than relying on a fixed policy that might not suit the current state of understanding, which is what this paper does.

Taro: I just think the reliance on those foundation models means we still have some work to do to make these decisions faster and more reliable for real-time deployment across all kinds of scenarios.

The paper's summary: Rosa: So, to wrap up what we just heard, RoboAtlas is this framework that mixes physical exploration—the kind where you’re mapping out an unknown area—with using semantic understanding to actually find what someone wants in that area.

Dev: Exactly. They use a contextual multi-armed bandit system to dynamically switch between three modes: frontier exploration, global map reasoning, and egocentric visual language model reasoning.

Rosa: It really changes the navigation policy from something fixed to something that adapts based on how much it knows about the environment in real-time.

Dev: That's the core idea. They have a mixture-of-experts setup where the contextual multi-armed bandit decides which expert gets to propose the next goal, conditioned on things like how much area is covered or if there's a similarity metric score.

Rosa: It’s not just about mapping anymore; it’s about using that map context to decide whether to keep moving blindly into new territory or start looking for a specific object based on what the LLM can infer.

Dev: And they handle the computational load on physical robots by using OpenRoboVox, which is this semantic mapping framework that has these clever optimizations like Truncated Signed Distance Field and dynamic instance pruning to keep memory usage low.

Rosa: That makes sense because running a massive three dee map in real-time on hardware like a Go2 is tough, so they have to be smart about how the map is built and accessed.

Dev: They also have this Scene-Dictionary St that condenses millions of voxels into something queryable, which lets the semantic map expert talk about objects without having to process every single tiny detail all the time.

Rosa: And for when it needs to make a big decision, like finding a goal, they use this asynchronous interface with an LLM backend.

Dev: The LLM doesn't just look at everything; they filter the candidates first using semantic potential fields to make sure the model isn't overwhelmed by noise before it even starts ranking things.

Rosa: So when the robot sees something interesting, like a red chair, it gets a high-level suggestion from the map expert based on what it knows is plausible spatially and semantically.

Dev: And if the VLM expert takes over, it’s because you’re looking at something right in front of you, and that visual cue can guide the robot to a goal even if the global map context isn't fully formed yet.

Rosa: The results on GOAT-Bench were pretty strong with a ninety point six percent success rate using GPT-4o, which shows that this adaptive switching works really well in challenging real-world scenarios <ref:2606.26046#pg1>.

Dev: But they also admit the limitation—the performance of all that high-level reasoning depends heavily on how good that underlying three dee semantic map actually is.

Rosa: And they also mentioned that because they rely on LLM and VLM inference, there’s still some latency and variability in response time, which might be a problem for tasks needing super fast global decisions.

Dev: So the big picture here is that RoboAtlas shows how you can bridge the gap between simple geometric exploration and complex semantic understanding by making the robot's decision policy change dynamically as it learns about its surroundings.

Rosa: It suggests that future robotic systems don’t need one fixed way to navigate; they need a system that knows when to explore and when to use its deep understanding of what it's seeing.

Dev: And this adaptive strategy, driven by the contextual multi-armed bandit, is what allows it to achieve better success rates and shorter paths than any single approach alone.

Rosa: So the takeaway for us is that context is key in robotics—the robot needs to know not just where it is geometrically, but what it *should* be doing semantically at any given moment.

The paper's improvements: Tom: So, we've talked about how RoboAtlas uses that bandit system to switch between exploring new areas and looking for specific things based on context, but now we're looking at what they suggest changing next.

Rosa: They point out that the biggest limitation is still the quality of that three dee map—if the map itself is messy or incomplete, all that high-level reasoning from the LLM and VLM won't be very reliable.

Dev: That makes sense because those foundation models are only as good as the data they’re looking at, so if your semantic map is poor, you get poor guidance.

Rosa: They propose improving how the map itself is built on resource-constrained robots by adding things like memory efficient TSDF optimization and parallel dynamic instance pruning to keep the map construction feasible in real-time.

Dev: That’s practical; they're addressing the hardware side directly, making sure that building a large semantic map doesn't just crash your system or run too slow on something like a Unitree Go2.

Rosa: And they introduce this Scene-Dictionary St to help manage that data, turning millions of voxels into this compact, queryable representation of objects and their properties.

Dev: That helps the semantic map expert interact with the environment more efficiently, so it doesn't have to do heavy lifting just to answer a simple query about an object.

Rosa: On the reasoning side, they’re looking at making that LLM grounding even better by using a filtered prompt context when querying the Scene-Dictionary St.

Dev: They are essentially trying to make sure the AI doesn't get confused by too much information and only focuses on what is actually relevant to the current query Q.

Rosa: The egocentric VLM expert also gets some tweaks, focusing on how it generates those textual scene descriptions so that they feed back into the semantic map context more effectively.

Dev: They’re trying to create a tighter loop between what the robot sees and what the global map understands, which should make that transition from exploration to targeted search smoother.

Rosa: These improvements suggest a path toward more robust systems where geometric exploration and deep semantic reasoning are truly integrated, not just running side-by-side.

Dev: If they can get those foundation models grounded better with high-quality three dee maps, it means robots could handle much more complex, ambiguous tasks in the real world without needing a fixed plan.

Rosa: So what’s next for them is to prove that this adaptive switching works reliably across a wider variety of environments and hardware than they have done so far.

Dev: And we need to see how stable those latency issues are when you push the loop rate higher, because that's where the safety case gets tricky for real-time control.

Conclusion: Rosa: So we’re wrapping up our look at RoboAtlas, which is this framework that uses a contextual multi-armed bandit to let a robot intelligently switch between physical exploration and semantic reasoning based on the current map understanding.

Dev: It really shows how context drives navigation in active SLAM by making the robot adapt its strategy instead of relying on a fixed policy.

Rosa: The implication is that future robots don't need one way to navigate; they need a system that knows when to explore and when to use its deep understanding of what it's seeing.

Dev: And the success rate numbers, like the ninety point six percent on GOAT-Bench, show that this adaptive switching between geometric and semantic guidance is actually effective across different stages of exploration.

Taro: I just want to stress that the system handles when things go wrong in real life; it doesn't just get stuck when the world misbehaves.

Rosa: Right, Taro mentioned how it recovers gracefully, which is important because we’re testing this outside the lab on hardware like the Go2.

Dev: I’m still thinking about those latency issues; if you need super fast global decisions, that reliance on VLM and LLM inference introduces variability that could be a real hurdle for real-time control loops.

Taro: The recovery mechanism is what really stands out to me, how it can generate scene descriptions to help the semantic map expert when things get occluded.

Rosa: Exactly; it’s not just about reaching a goal; it’s about the robot maintaining a coherent understanding of its situation even when the visual input is momentarily confusing.

Dev: So, while RoboAtlas is great for showing adaptive behavior in simulation and on hardware, we still have work to do to make those decisions faster and more reliable for deployment in settings that require very high-speed response times.

Rosa: That’s a fair point; the foundation models are powerful, but they introduce a dependency on their performance.

Taro: I think the next big thing will be taking this concept and grounding it even more tightly with better predictive models for what’s going to happen next in the environment.

Rosa: So that's our look at RoboAtlas, Contextual Active SLAM. It shows how context can drive navigation by mixing physical exploration with high-level semantic reasoning to get better results than fixed policies.

Dev: The core idea is using a contextual multi-armed bandit to dynamically switch between frontier exploration and semantically guided navigation as the scene representation becomes more informative.

Rosa: They bridge geometric exploration and semantic reasoning by adapting their decision policy based on the environment's state, which is what makes it flexible across different stages of exploration.

Taro: The final result is a ninety point six percent success rate and a fifty-three point four percent path length at the GOAT-Bench benchmark using GPT-4o, which exceeds the strongest prior baseline by seventeen point eight percentage points on success rate <ref:2606.26046#pg1,the strongest prior baseline by>.

Rosa: That indicates that adaptive switching between geometric and semantic guidance is effective across different stages of exploration.

Dev: The system successfully bridges geometric exploration and semantic reasoning by adapting its decision policy as the environment becomes better understood across simulation and hardware experiments.

Mitsubishi Electric Research Laboratories · University of California, Los Angeles

cs.RO, cs.CV

Submitted: 2026-06-24

Updated: 2026-10-15

Comments: Alexander Schperberg and Shivam K. Panda made equal contribution

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 89/100

The gist: The gist RoboAtlas presents a contextual Active SLAM framework that adaptively balances geometric exploration and semantic reasoning using a scalable 3D semantic mapping system, OpenRoboVox.

Key concepts

OpenRoboVox: This is a scalable 3D semantic mapping framework designed for real-world robots. It builds upon existing methods but includes optimizations like memory-efficient structures and parallel processing to handle millions of semantic instances efficiently, making it suitable for deployment on edge robotic platforms.
Contextual Multi-Armed Bandit (CMAB): The CMAB is the decision-making core that dynamically selects which expert module (exploration, semantic map, or VLM) to use next. It uses a context vector summarizing the robot's state—like coverage and similarity metrics—to choose an action that maximizes a reward combining map gain and semantic relevance.
Mixture-of-Experts Framework: RoboAtlas uses three specialized experts: one for frontier exploration (seeking new geometry), one for the semantic map (reasoning about objects), and one for egocentric VLM reasoning (interpreting the current view). The CMAB manages which of these experts is most useful at any given moment.
Egocentric VLM Expert: This expert uses a Vision-Language Model (VLM) to reason directly from the robot's current viewpoint. It interprets user queries and scene observations to estimate target relevance, generating textual descriptions that help refine the overall scene context.

Terminology

Active SLAM — 30× in this paper · explained in 1 episode
Success Rate — 28× in this paper
Contextual Multi-Armed Bandit — 21× in this paper
Foundation Model — 17× in this paper · explained in 3 episodes
Benchmark — 16× in this paper · explained in 2 episodes
Foundation Models — 11× in this paper · explained in 7 episodes
Grounding — 10× in this paper · explained in 3 episodes
Planner — 9× in this paper · explained in 3 episodes
Cosine Similarity — 6× in this paper · explained in 3 episodes
Information Gain — 6× in this paper · explained in 2 episodes
Abstraction — 5× in this paper · explained in 1 episode
Adaptive Strategy — 5× in this paper · explained in 1 episode
Pruning — 5× in this paper · explained in 4 episodes
Reward Function — 5× in this paper · explained in 4 episodes
Consistency — 4× in this paper · explained in 3 episodes
Core — 4× in this paper · explained in 2 episodes
Embeddings — 4× in this paper · explained in 2 episodes
Embodied AI — 4× in this paper · explained in 2 episodes
Robustness — 4× in this paper · explained in 5 episodes
Semantic Understanding — 4× in this paper · explained in 2 episodes
Contextual Bandits — 3× in this paper · explained in 3 episodes
World Model — 3× in this paper · explained in 7 episodes
Adaptation — 2× in this paper · explained in 1 episode
Aggregation — 2× in this paper
Failure Modes — 2× in this paper · explained in 3 episodes
Fidelity — 2× in this paper · explained in 2 episodes
Fine-Tuning — 2× in this paper · explained in 5 episodes
IEEE International Conference on Robotics and Automation — 2× in this paper
Semantic Segmentation — 2× in this paper · explained in 2 episodes
Simultaneous Localization and Mapping — 2× in this paper
Vision-language models — 2× in this paper · explained in 2 episodes
Visual Grounding — 2× in this paper · explained in 2 episodes
AI Agents — 1× in this paper · explained in 2 episodes
Ablation — 1× in this paper · explained in 1 episode
Accuracy — 1× in this paper · explained in 1 episode
Adaptability — 1× in this paper · explained in 1 episode
Aggregation Function — 1× in this paper
Computational overhead — 1× in this paper · explained in 2 episodes
Computer Vision and Pattern Recognition Conference — 1× in this paper
Contextual Understanding — 1× in this paper · explained in 3 episodes
Credit assignment — 1× in this paper · explained in 3 episodes
Decoupling — 1× in this paper · explained in 2 episodes
Edge Deployment — 1× in this paper · explained in 2 episodes
Generalization — 1× in this paper · explained in 13 episodes
International Conference on Computer Vision — 1× in this paper
Interpretability — 1× in this paper · explained in 15 episodes
MLLMs — 1× in this paper · explained in 2 episodes
Mask Generation — 1× in this paper · explained in 3 episodes
Maximum A Posteriori — 1× in this paper
Regularization — 1× in this paper · explained in 3 episodes
Reinforcement Learning — 1× in this paper · explained in 4 episodes
Vision-Language Models (VLMs) — 1× in this paper · explained in 6 episodes

Summary

The gist RoboAtlas presents a contextual Active SLAM framework that adaptively balances geometric exploration and semantic reasoning using a scalable 3D semantic mapping system, OpenRoboVox. This framework integrates frontier exploration, global semantic-map reasoning, and egocentric VLM-based reasoning through a contextual multi-armed bandit that transitions from exploration to semantically guided navigation as scene understanding improves.

How it works

RoboAtlas employs a mixture-of-experts framework where the experts include a frontier-based exploration module, a semantic map module, and an egocentric VLM module (Fig. 1). The system receives the environment state through OpenRoboVox, which is described as a real-time 3D semantic mapping framework, supporting efficient semantic map construction suitable for deployment on edge robotic platforms (Page 2). These experts dynamically propose a goal based on the robot's current contextual state, and a contextual multi-armed bandit selects the next goal position conditioned on the robot and environment state (Algorithm 1, Page 4).

The three experts are defined as follows:

Frontier Exploration Expert: This expert selects navigation goals that expand the robot’s knowledge of the environment by prioritizing geometric information gain while ensuring that exploration remains compatible with the robot’s energy constraints (Page 6).
Semantic Map Expert: This module reasons over a persistent semantic global map, allowing it to reason directly over the evolving semantic state of the environment by answering open-vocabulary queries and inferring spatial relationships between objects (Page 6).
Egocentric VLM Expert: This expert reasons directly over the robot’s current viewpoint by interpreting the user query and scene observation to estimate target relevance, and it generates textual scene descriptions that are accumulated into the scene context Dscene (Page 6).

OpenRoboVox Framework

OpenRoboVox is a open-vocabulary semantic mapping framework that enables long-horizon semantic navigation on real-world mobile robots (Page 2). It builds upon OpenVox [34], but introduces modifications to handle deployment constraints, including memory-efficient TSDF (i.e., Truncated Signed Distance Field) optimization, parallel dynamic-instance pruning and updates, and distance-prioritized historical refresh (Page 2). A key feature is the Scene-Dictionary St, which provides a compact, queryable representation of objects, their semantic descriptions, and their spatial properties, condensing millions of low-level voxels into a representation suitable for high-level reasoning (Page 5).

To manage computational load and memory constraints on physical robots like the Unitree Go2, OpenRoboVox introduces several architectural changes:

Scene-Dictionary Manager: This incremental background process decouples semantic reasoning from geometric tracking by selecting a bounded batch of instances based on a Priority-Based Scheduling Algorithm (Page 8).
GPU Optimization & Memory Safety: The framework utilizes a block-based volumetric transfer protocol that bounds peak VRAM during map expansion, decomposing the domain into disjoint cubic blocks to prevent out-of-memory failures (Page 9).
Asynchronous Parallel Architecture: Geometric integration and the perception stack run on independent threads using a custom concurrency control manager to enforce an isolation boundary between high-bandwidth tracking processes and latency-prone semantic processes (Page 9).

Contextual Multi-Armed Bandit (CMAB)

The system uses a Contextual Multi-Armed Bandit (CMAB) to dynamically select among the three experts, transitioning from exploration to targeted semantics-driven navigation as evidence accumulates (Page 4). The CMAB selects an arm based on a context vector ct, which summarizes the robot’s state, including occupancy grid coverage, change in grid coverage from time step t to t−1, and similarity metrics like Vsimt (Page 4).

The reward function is defined as a weighted combination of contextual state variables: rt = w1m˙occ t − w2Bt + w3Vvlm t + w4Vsim t + wsucc, (5) (Page 7). This formulation encourages actions that increase map coverage, improve semantic relevance, and favor confident VLM predictions while penalizing unnecessary revisitation of previously explored regions via the backtracking indicator Bt (Page 7).

Semantic Map Goal Selection

The Semantic Map Expert utilizes an asynchronous query interface to bridge perception with an LLM backend running in a background worker thread (Page 10). The system uses a pre-filtering stage based on semantic potential fields, defining a filtered subset S˜t containing only instances relevant to the current query Q (Page 10). This filtering ensures that the LLM is not overwhelmed by the entire Scene-Dictionary St, which is serialized into a context vector Ct (Page 10).

The LLM is tasked with ranking candidate instances according to both semantic relevance and spatial plausibility, returning a confidence score ΦLLM(Oi) (Page 10). The final navigation goal gt is then chosen as the centroid location of the highest-scoring instance among the top-N candidates, ensuring that the semantic map expert used in the bandits algorithm retains the top N instances according to ΦLLM (Page 10).

Egocentric VLM Goal Selection

The Egocentric VLM Expert enables goal selection directly from instantaneous visual observations by taking a user query, system prompt, and an RGB image as input (Page 11). The VLM returns confidence scores ϕVLM(Oi) for each annotated candidate Oi and a high-level action selection at ∈ A, where a "go" action leads to the goal corresponding to the highest confidence instance (Page 12). This capability allows the system to recover from challenging visual configurations, such as occlusions, by generating textual scene descriptions that are subsequently provided to the semantic map expert (Page 12).

Experimental Validation

RoboAtlas was evaluated on three platforms: Isaac Sim, photo-realistic Habitat simulator, and hardware with a Unitree Go2 robot (Page 13). On the GOAT-Bench “Val Unseen” benchmark, RoboAtlas achieved state-of-the-art performance with a highest reported success rate (SR) of 90.6% using GPT-4o, improving over the strongest prior baseline by 17.8 percentage points in SR (Page 1). The results demonstrate that grounding foundation models with large-scale 3D semantic maps enables robust and efficient contextual Active SLAM (Page 1). The CMAB demonstrated an adaptive strategy, transitioning from exploration to semantic reasoning as environmental understanding improved, achieving a 100% success rate across all trials while also yielding the shortest path lengths and fastest completion times (Page 15).

Discussion and Limitations

The results show that RoboAtlas can bridge geometric exploration and semantic reasoning by adapting its decision policy as the environment becomes better understood (Page 18). A limitation is that the performance of both LLM-based reasoning and similarity-based retrieval depends strongly on the quality of the underlying 3D semantic map (Page 18). Another constraint is that the framework relies on LLM and VLM inference, which introduces latency and variability in response time, potentially limiting deployment in settings requiring fast global decision making (Page 18).

Conclusion

RoboAtlas introduced a contextual Active SLAM framework that combines geometric exploration and semantic reasoning through a Contextual Multi-Armed Bandit (CMAB) (Page 17). By integrating frontier exploration, LLM-based semantic reasoning, and egocentric VLM cues, the system adapts its navigation strategy to the current state of the environment rather than relying on a fixed exploration policy (Page 17). The experimental results in simulation, on hardware, and on GOAT-Bench demonstrate that contextual decision-making can improve both robustness and flexibility in Active SLAM (Page 17).

How it works

The CMAB allocation analysis shows that the egocentric VLM expert receives the largest share of picks (45.0%) but contributes only 30.6% of total path length, reflecting its role as a fine-grained corrector that issues many short-range waypoints to refine local positioning (Page 18). The semantic map expert is selected on 38.7% of decisions yet accounts for the majority of total path length (46.8%), consistent with its role as the primary exploitation channel (Page 18).

The CMAB policy adapts its behavior according to both the environment state and the reward structure, transitioning from coverage-driven exploration to semantically guided navigation once the scene representation becomes sufficiently informative (Page 18). This adaptive strategy explains why CMAB achieves higher success rates and more efficient trajectories than any single fixed approach (Page 15).

The framework successfully bridges geometric exploration and semantic reasoning by adapting its decision policy as the environment becomes better understood across simulation and hardware experiments (Page 18). The action and reward analyses support this interpretation, showing that semantic signals become increasingly important as map completeness improves (Page 18). This adaptive strategy explains why CMAB achieves higher success rates and more efficient trajectories than any single fixed approach (Page 15).

The final result is that RoboAtlas achieves a 90.6% SR and 53.4% SPL at the GOAT-Bench benchmark using GPT-4o, exceeding the strongest prior baseline by 17.8 percentage points on SR (Page 17). This indicates that adaptive switching between geometric and semantic guidance is effective across different stages of exploration (Page 17). The system successfully bridges geometric exploration and semantic reasoning by adapting its decision policy as the environment becomes better understood across simulation and hardware experiments (Page 18).

Improvements for AI systems

3D Semantic Mapping on Resource-Constrained Robots: OpenRoboVox enables large-scale, real-time voxel-based semantic mapping by combining memoryefficient TSDF (i.e., Truncated Signed Distance Field) optimization, parallel dynamic-instance pruning and updates. This allows for the construction of a Scene Dictionary which provides a compact, queryable representation of objects, their semantic descriptions, and their spatial properties, making it suitable for deployment on physical robots.
Foundation Model-Driven Semantic Reasoning: The framework introduces two complementary experts powered by a shared foundation model: (i) a semantic map expert that reasons over a scene dictionary derived from a global 3D map and (ii) an egocentric VLM expert that reasons over the robot’s immediate observations. This enables languageguided, semantics-aware goal prediction by integrating global map context with local visual cues, allowing the system to transition from broad exploration to targeted semantics-driven search as evidence accumulates.
Contextual Goal Selection via Adaptive Bandits: By employing a contextual multi-armed bandit (CMAB), the system dynamically selects among three experts based on the robot's state, enabling a behavior that transitions from efficient coverage in early exploration to targeted semantics-driven search as evidence accumulates, as shown by the transition from frontier exploration dominance in low-coverage regimes to semantic map exploitation in high-coverage regimes.
Semantic Navigation via Similarity Maps: The system constructs 2D Pillar Map[s] which compute potential fields like Target Object(s) Similarity (Tsim), allowing for semantic navigation by projecting the semantic relevance of 3D instances onto the 2D Pillar grid to enable queries like “Go to the red chair” using precomputed text embeddings without incurring LLM query costs for every frame.
Context-Aware Goal Ranking via LLM Grounding: The system utilizes a filtered prompt context where an LLM is tasked to rank candidate instances based on both semantic relevance and spatial plausibility, returning a score ΦLLM(Oi) = LLMOi, Nr(Oi) QT, QN, σi, which allows the semantic map expert to select the final navigation goal as gt = pi∗, i∗ = arg max Oi∈Top-N ΦLLM(Oi), grounding reasoning in underlying vector space.