RAG-RUSS: A Retrieval-Augmented Robotic Ultrasound for Autonomous Carotid Examination
summary
The gist
Robotic ultrasound (US) has recently attracted increasing attention as a means to overcome the limitations of conventional US examinations, such as the strong operator dependence on page 1.
In short
RAG-RUSS is an interpretable framework designed to automate full carotid ultrasound examinations by following clinical workflows and explaining its decisions. It uses a Large Language Model, a vision model, and retrieval-augmented generation to plan probe movements autonomously. The system successfully explains the current scanning stage and the next required action, showing promise for trustworthy robotic US.
Key concepts
- RAG-RUSS
- A framework that combines Retrieval-Augmented Generation with robotic ultrasound. It allows the system to perform a complete carotid exam by following clinical steps while clearly explaining what it is doing now and what it plans to do next, making its actions understandable.
- Clinical Workflow Partitioning
- Breaking down the complex carotid examination into eight sequential stages based on anatomical cues. This structure guides the robotic process, ensuring that the ultrasound scan follows established medical protocols, moving systematically from transverse views to longitudinal assessments.
- Retrieval-Augmented Generation (RAG)
- A technique used to improve AI performance when medical data is scarce. Instead of relying only on its training data, the system retrieves relevant context from a database before generating an answer or planning an action, enhancing its accuracy and generalization.
- Cross-modality Projector
- A component that connects visual information (from the ultrasound image) and textual information (from the LLM). It aligns these two different types of data so the AI can effectively reason about the image while understanding its textual instructions.
Terminology used across episodes
This episode discusses
- RAG-RUSS: A Retrieval-Augmented Robotic Ultrasound for Autonomous Carotid Examination · Paper Radio
- Intelligent Virtual Sonographer (IVS): Enhancing Physician-Robot-Patient Communication
- From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence
- EndoVLA: Dual-Phase Vision-Language-Action Model for Autonomous Tracking in Endoscopy
- USPilot: An Embodied Robotic Assistant Ultrasound System with Large Language Model Enhanced Graph Planner
- Does CLIP Benefit Visual Question Answering in the Medical Domain as Much as it Does in the General Domain?
- LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models
The paper
RAG-RUSS: A Retrieval-Augmented Robotic Ultrasound for Autonomous Carotid Examination · Read on arXiv
Dianye Huang, Ziping Cong, Nassir Navab, Zhongliang Jiang
Technical University of Munich · Munich Center for Machine Learning
Robotic ultrasound (US) has recently attracted increasing attention as a means to overcome the limitations of conventional US examinations, such as the strong operator dependence. However, the decision-making process of existing methods is often either rule-based or relies on end-to-end learning models that operate as black boxes. This has been seen as a main limit for clinical acceptance and raises safety concerns for widespread adoption in routine practice. To tackle this challenge, we introduce the RAG-RUSS, an interpretable framework capable of performing a full carotid examination in accordance with the clinical workflow while explicitly explaining both the current stage and the next planned action. Furthermore, given the scarcity of medical data, we incorporate retrieval-augmented generation to enhance generalization and reduce dependence on large-scale training datasets. The method was trained on data acquired from 28 volunteers, while an additional four volumetric scans recorded from previously unseen volunteers were reserved for testing. The results demonstrate that the method can explain the current scanning stage and autonomously plan probe motions to complete the carotid examination, encompassing both transverse and longitudinal planes.
DOI: 10.1109/ICRA57385.2026.11697339
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "RAG-RUSS: A Retrieval-Augmented Robotic Ultrasound for Autonomous Carotid Examination".
Dev: Here's the gist of the paper. Robotic ultrasound (US) has recently attracted increasing attention as a means to overcome the limitations of conventional US examinations, such as the strong operator dependence on page 1.
Rosa: To tackle this challenge, we introduce the RAG-RUSS, an interpretable framework capable of performing a full carotid examination in accordance with the clinical workflow while explicitly explaining both the current stage and the next planned action on page 1.
Dev: Furthermore, given the scarcity of medical data, we incorporate retrieval-augmented generation to enhance generalization and reduce dependence on large-scale training datasets on page 1.
Rosa: Clinical Workflow and Data Preparation The clinical workflow for a carotid artery examination involves a standard procedure where sonographers generally follow established protocols
9: ,
11: on page 2.
Paper summary: Dev: It’s interesting how they structure the entire process into eight sequential stages guided by specific anatomical cues on page three of this paper <ref:2603.01153#pg3>.
Rosa: That’s a lot of specific steps, from "Examine CCA proximal" to "Longitudinal scan completed," which shows they mapped the clinical workflow directly into the system's logic.
Dev: They built a dataset from thirty-two human volumetric scans and created three types of textual annotations for each one: the current scanning stage, an explanation of the key feature for that stage, and what API is next to finish that task on page four.
Rosa: That’s how they got their training data—pairing visual information with detailed text descriptions to teach the model.
Dev: The model itself uses a setup with a Large Language Model like Vicuna v1 point 5 7B and a vision foundation model called PubMedCLIPViT-B/thirty-two which gets frozen on page five.
Rosa: And they use a cross-modality projector, which is just a two-layer MLP, to align the visual and textual features together on page five <ref:2603.01153#pg3>.
Dev: The training involved three steps: first training the retrieval engine's ResMLP using metric learning with triplet loss, then pre-training that cross-modality projector, and finally instruction fine-tuning on page six <ref:2603.01153#pg3>.
Conclusion: Rosa: Looking at this paper RAG-RUSS: A Retrieval-Augmented Robotic Ultrasound for Autonomous Carotid Examination by Dianye Huang and colleagues, the main point is how they integrated retrieval augmentation with vision–language reasoning to achieve autonomous scanning.
Dev: It moves beyond just making a decision; it makes the system explain its reasoning, which is important for clinical acceptance because it addresses those safety concerns we talked about earlier.
Rosa: And by using retrieval-augmented in-context learning, they show that you don't need huge training sets to make the system adaptable and generalize better to new situations.
Dev: The results show that the RAG-RUSS@two configuration achieved the highest average accuracy of seventy-nine point one percent, which they claim is strong performance on anatomically well-defined stages on page seven.
Rosa: So, what this means for us is that we’ve got a framework here that can perform full-stage carotid scanning according to clinical protocols autonomously while providing explanations of the current image and the subsequent action.
Dev: It’s a step toward building trustworthy and acceptable robotic ultrasound systems for routine deployment, which is what the authors say on page eight <ref:2603.01153#pg3>.
Rosa: Exactly, so it’s about moving from a black box to something that can articulate its process, giving clinicians confidence in its performance during a complex procedure.
More episodes
- 2608.13438-ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
- 2610.34593-Future Prediction as Training-Only Video Supervision for Pretrained VLA Policies
- 2610.35385-EmbodiedLab: Beyond Manipulation, Benchmarking Embodied Agents That Act to Know
- 2610.36378-Data-Driven Hamiltonian for Neural Optimal Control
- 2610.37275-A Heuristic for Scheduling Autonomous Vehicles at Intersections
- 2610.37288-Dynamic Coordination of Autonomous Vehicles at Intersections
- 2610.37632-Real-Time Coordination of Autonomous Vehicles at Intersections
- 2611.13779-Softness is something you also need! Why compliance helps control: information metrics of a passive walker with compliant ankle
- 2611.13411-Residual-Corrected Physics-Informed Neural Network KKL Observers with Input-to-State Stability Guarantees
- 2611.14488-Is Your Robot's Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies