RAG-RUSS: A Retrieval-Augmented Robotic Ultrasound for Autonomous Carotid Examination

summary

The gist

Robotic ultrasound (US) has recently attracted increasing attention as a means to overcome the limitations of conventional US examinations, such as the strong operator dependence on page 1.

In short

RAG-RUSS is an interpretable framework designed to automate full carotid ultrasound examinations by following clinical workflows and explaining its decisions. It uses a Large Language Model, a vision model, and retrieval-augmented generation to plan probe movements autonomously. The system successfully explains the current scanning stage and the next required action, showing promise for trustworthy robotic US.

Key concepts

RAG-RUSS
A framework that combines Retrieval-Augmented Generation with robotic ultrasound. It allows the system to perform a complete carotid exam by following clinical steps while clearly explaining what it is doing now and what it plans to do next, making its actions understandable.
Clinical Workflow Partitioning
Breaking down the complex carotid examination into eight sequential stages based on anatomical cues. This structure guides the robotic process, ensuring that the ultrasound scan follows established medical protocols, moving systematically from transverse views to longitudinal assessments.
Retrieval-Augmented Generation (RAG)
A technique used to improve AI performance when medical data is scarce. Instead of relying only on its training data, the system retrieves relevant context from a database before generating an answer or planning an action, enhancing its accuracy and generalization.
Cross-modality Projector
A component that connects visual information (from the ultrasound image) and textual information (from the LLM). It aligns these two different types of data so the AI can effectively reason about the image while understanding its textual instructions.

Terminology used across episodes

This episode discusses

The paper

RAG-RUSS: A Retrieval-Augmented Robotic Ultrasound for Autonomous Carotid Examination · Read on arXiv

Dianye Huang, Ziping Cong, Nassir Navab, Zhongliang Jiang

Technical University of Munich · Munich Center for Machine Learning

Robotic ultrasound (US) has recently attracted increasing attention as a means to overcome the limitations of conventional US examinations, such as the strong operator dependence. However, the decision-making process of existing methods is often either rule-based or relies on end-to-end learning models that operate as black boxes. This has been seen as a main limit for clinical acceptance and raises safety concerns for widespread adoption in routine practice. To tackle this challenge, we introduce the RAG-RUSS, an interpretable framework capable of performing a full carotid examination in accordance with the clinical workflow while explicitly explaining both the current stage and the next planned action. Furthermore, given the scarcity of medical data, we incorporate retrieval-augmented generation to enhance generalization and reduce dependence on large-scale training datasets. The method was trained on data acquired from 28 volunteers, while an additional four volumetric scans recorded from previously unseen volunteers were reserved for testing. The results demonstrate that the method can explain the current scanning stage and autonomously plan probe motions to complete the carotid examination, encompassing both transverse and longitudinal planes.

DOI: 10.1109/ICRA57385.2026.11697339

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "RAG-RUSS: A Retrieval-Augmented Robotic Ultrasound for Autonomous Carotid Examination".

Dev: Here's the gist of the paper. Robotic ultrasound (US) has recently attracted increasing attention as a means to overcome the limitations of conventional US examinations, such as the strong operator dependence on page 1.

Rosa: To tackle this challenge, we introduce the RAG-RUSS, an interpretable framework capable of performing a full carotid examination in accordance with the clinical workflow while explicitly explaining both the current stage and the next planned action on page 1.

Dev: Furthermore, given the scarcity of medical data, we incorporate retrieval-augmented generation to enhance generalization and reduce dependence on large-scale training datasets on page 1.

Rosa: Clinical Workflow and Data Preparation The clinical workflow for a carotid artery examination involves a standard procedure where sonographers generally follow established protocols

9: ,

11: on page 2.

Paper summary: Dev: It’s interesting how they structure the entire process into eight sequential stages guided by specific anatomical cues on page three of this paper <ref:2603.01153#pg3>.

Rosa: That’s a lot of specific steps, from "Examine CCA proximal" to "Longitudinal scan completed," which shows they mapped the clinical workflow directly into the system's logic.

Dev: They built a dataset from thirty-two human volumetric scans and created three types of textual annotations for each one: the current scanning stage, an explanation of the key feature for that stage, and what API is next to finish that task on page four.

Rosa: That’s how they got their training data—pairing visual information with detailed text descriptions to teach the model.

Dev: The model itself uses a setup with a Large Language Model like Vicuna v1 point 5 7B and a vision foundation model called PubMedCLIPViT-B/thirty-two which gets frozen on page five.

Rosa: And they use a cross-modality projector, which is just a two-layer MLP, to align the visual and textual features together on page five <ref:2603.01153#pg3>.

Dev: The training involved three steps: first training the retrieval engine's ResMLP using metric learning with triplet loss, then pre-training that cross-modality projector, and finally instruction fine-tuning on page six <ref:2603.01153#pg3>.

Conclusion: Rosa: Looking at this paper RAG-RUSS: A Retrieval-Augmented Robotic Ultrasound for Autonomous Carotid Examination by Dianye Huang and colleagues, the main point is how they integrated retrieval augmentation with vision–language reasoning to achieve autonomous scanning.

Dev: It moves beyond just making a decision; it makes the system explain its reasoning, which is important for clinical acceptance because it addresses those safety concerns we talked about earlier.

Rosa: And by using retrieval-augmented in-context learning, they show that you don't need huge training sets to make the system adaptable and generalize better to new situations.

Dev: The results show that the RAG-RUSS@two configuration achieved the highest average accuracy of seventy-nine point one percent, which they claim is strong performance on anatomically well-defined stages on page seven.

Rosa: So, what this means for us is that we’ve got a framework here that can perform full-stage carotid scanning according to clinical protocols autonomously while providing explanations of the current image and the subsequent action.

Dev: It’s a step toward building trustworthy and acceptable robotic ultrasound systems for routine deployment, which is what the authors say on page eight <ref:2603.01153#pg3>.

Rosa: Exactly, so it’s about moving from a black box to something that can articulate its process, giving clinicians confidence in its performance during a complex procedure.

More episodes

← Home