Deep Learning for Semantic Segmentation of 3D Ultrasound Data

summary

The gist

Developing cost-efficient and reliable perception systems remains a central challenge for automated vehicles <ref:2601.13263#pg2> This work introduces a novel framework for learning-based 3D semantic…

In short

This work introduces a novel framework for learning-based 3D semantic segmentation using Calyo Pulse, a solid-state 3D ultrasound sensor. A 3D U-Net architecture was trained on spatial ultrasound data to segment volumes. Results showed robust performance, demonstrating that 3D ultrasound sensing is a promising complementary modality for reliable autonomous vehicle perception.

Key concepts

Calyo Pulse
This is a compact, solid-state 3D ultrasound sensor designed for volumetric perception in harsh environments. It uses a transceiver element and a broadband receiver array to capture raw Rx waveforms, 3D point clouds, and intensity projections without moving parts.
3D U-Net Architecture
This is a specific deep learning model used for volumetric segmentation. It consists of two encoder stages and two decoder stages designed to process the input intensity map (64x96x64x1) to produce a volumetric segmentation, rewarding overlap between prediction and ground truth.
CFAR Filtering
Constant False Alarm Rate (CFAR) filtering is a noise suppression technique applied to the raw RF signals. It helps reduce background noise in the 3D intensity maps, significantly improving signal quality for subsequent processing steps.
Ground Truth Generation
The ground truth masks were created by combining bounding box generation with rule-based cleaning. These masks were projected onto an occupancy grid and calibrated against the LiDAR frame to ensure accurate spatial alignment for training the segmentation model.

Terminology used across episodes

This episode discusses

The paper

Deep Learning for Semantic Segmentation of 3D Ultrasound Data · Read on arXiv

Chenyu Liu, Marco Cecotti, Harikrishnan Vijayakumar, Patrick Robinson, James Barson, Mihai Caleap

Cranfield University · Calyo Limited

Developing cost-efficient and reliable perception systems remains a central challenge for automated vehicles. LiDAR and camera-based systems dominate, yet they present trade-offs in cost, robustness and performance under adverse conditions. This work introduces a novel framework for learning-based 3D semantic segmentation using Calyo Pulse, a modular, solid-state 3D ultrasound sensor system for use in harsh and cluttered environments. A 3D U-Net architecture is introduced and trained on the spatial ultrasound data for volumetric segmentation. Results demonstrate robust segmentation performance from Calyo Pulse sensors, with potential for further improvement through larger datasets, refined ground truth, and weighted loss functions. Importantly, this study highlights 3D ultrasound sensing as a promising complementary modality for reliable autonomy.

DOI: 10.1007/978-3-032-23010-2_31

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Deep Learning for Semantic Segmentation of 3D Ultrasound Data".

Jane: Here's the gist of the paper. Developing cost-efficient and reliable perception systems remains a central challenge for automated vehicles <ref:2601.13263#pg2> This work introduces a novel framework for learning-based 3D semantic segmentation using Calyo Pulse, a modular, solid-state 3D ultrasound sensor system for use in harsh and cluttered environments <ref:2601.13263#pg6> A 3D U-Net architecture is introduced and trained on the spatial ultrasound data for volumetric segmentation <ref:2601.13263#pg8> Results demonstrate robust segmentation performance from Calyo Pulse sensors, with potential for further improvement through larger datasets, refined ground truth, and weighted loss functions Importantly, this study highlights 3D ultrasound sensing as a promising complementary modality for reliable autonomy Sensor Hardware Description Calyo Pulse is described as a compact in-air 3-D ultrasound sensor designed for volumetric perception <ref:2601.13263#pg6> The modular system employs a transceiver element (Tx), nominally centered at 40 kHz but configurable for other frequencies, and a broadband receiver (Rx) array with a wide spectral response (100 Hz - 80 kHz) <ref:2601.13263#pg6> This configuration enables flexible operation across different acoustic regimes and supports high-fidelity time-offlight and amplitude measurements <ref:2601.13263#pg6> The hardware is fully solid-state, with no moving parts, and engineered for ruggedised operation in automotive and industrial environments <ref:2601.13263#pg6> Power and data are provided through a USB-C interface (5 V, 0.15 A) with total power consumption under 1 W <ref:2601.13263#pg6> The device outputs raw Rx waveforms, 3D point clouds, and 2-D intensity projections accessible via the Calyo Sensus SDK <ref:2601.13263#pg6> Data Preprocessing Pipeline The processing pipeline for the ultrasound data involves several steps to enhance signal quality <ref:2601.13263#pg8> 1.

Tom: Raw RF signals were processed to generate 3D intensity maps <ref:2601.13263#pg8> 2.

Jane: The maps were first represented in spherical polar coordinates (r, θ, φ) and then converted into Cartesian space (x, y, z) for interpretability <ref:2601.13263#pg8> 3.

Tom: The processing pipeline also included Constant False Alarm Rate (CFAR) filtering to suppress noise 4.

Title and authors: Tom: So we're talking about this paper called Deep Learning for Semantic Segmentation of three dee Ultrasound Data today <ref:2601.13263#pg1,Deep Learning for Semantic Segmentation of 3D Ultrasound Data>. It looks like they're taking a compact ultrasound sensor and using it with a deep learning model to figure out what's in the three dee space around a vehicle <ref:2601.13263#pg1>.

Jane: That sounds really practical, Tom. So, what exactly is this ultrasound system they are using? Is it something we can actually put on a car or something similar?

Tom: It is called Calyo Pulse, and it’s described as a compact in-air three-D ultrasound sensor designed for volumetric perception <ref:2601.13263#pg1>. They built it to be fully solid-state, meaning no moving parts at all, which should make it pretty rugged for harsh environments like cars or factories.

Lu: What's interesting about the hardware description is that they detail the transceiver element and the broadband receiver array they use. It’s not just one fixed frequency; it can be configured for other frequencies, and the receiver covers a wide spectral response from one hundred Hertz to eighty kilohertz.

Meng: Eighty kilohertz is pretty wide, so that suggests they're aiming for high-fidelity time-offlight and amplitude measurements, which is important when you’re trying to see things clearly without direct light or in noisy settings. But how much power does this whole system actually use?

Tom: They say the power and data are provided through a USB-C interface at five volts with a total power consumption under one watt, which makes it very efficient for what they want it to do. They output raw Rx waveforms, three dee point clouds, and two dimensional intensity projections that you can access through their Calyo Sensus SDK <ref:2601.13263#pg1>.

Jane: So the hardware sounds solid and low-power. But the real magic here is how they use that data. The paper introduces a three dimensional U net architecture specifically trained on this spatial ultrasound data for volumetric segmentation.

Lu: That three dee U net architecture is what they use for the segmentation, and it has two encoder stages and two decoder stages, which is actually a reduction compared to some of the other four stage architectures that are common in three dee models <ref:2601.13263#pg1,3D U net architecture is>.

Tom: And what about the input data they feed into that model? It starts with raw RF signals from those ultrasound waves. They process those raw signals to generate three dimensional intensity maps first before converting them into spherical polar coordinates and then Cartesian space for interpretability.

Title and authors: Jane: Before you get to the segmentation, there’s a whole pipeline for cleaning up that data. They use constant false alarm rate filtering to suppress noise, and they apply linear range compensation which helps enhance the visibility of objects that are farther away.

Meng: That sounds like standard image processing cleanup, but I wonder if those steps are robust enough when you're dealing with real-world clutter, like a busy road. They also mention using bounding boxes and rule based cleaning to generate their ground truth for training.

Tom: The ground truth itself was created by projecting these bounding boxes onto an occupancy grid and using extrinsic calibration parameters to make sure the ultrasound coordinate system lines up with the LiDAR frame they used for calibration.

Lu: That alignment step is crucial because getting the sensor frames to match up correctly is a major hurdle in multimodal systems, and they addressed that by using those parameters. They also defined their loss function as one minus the mean of Dice, which really rewards good overlap between what the model predicts and what the ground truth actually shows.

Jane: So, if I’m driving, this paper suggests that this method can give us a volumetric segmentation map instead of just simple bounding boxes around objects. What does that mean for understanding three dee scenes <ref:2601.13263#pg1>?

Tom: It means we get a much better sense of the actual shape and volume of what's detected, even if it doesn't distinguish between two different types of objects that look similar. The paper shows robust segmentation performance, achieving a recall of sixty-two point six nine percent and an F1 score of sixty-seven point three zero percent on their test data.

Meng: Sixty-two percent recall is okay, but I’m thinking about real deployment. What happens when we move this to a fleet of vehicles? Does the system handle those cluttered images that obscure object boundaries well, like the authors mentioned in their early experiments?

Jane: They did mention challenges with cluttered detection images obscuring object boundaries, but they propose that advanced pre-processing and deep learning can mitigate those issues. They also note that the study shows this compact ultrasound sensor can serve as a viable complement to traditional LiDAR, radar, and camera systems.

Tom: Exactly, it’s positioned as a complement rather than a replacement for existing sensors. The paper says this approach opens up a new pathway for redundancy and safety in autonomous navigation when optical or radio frequency sensors might fail.

Title and authors: Lu: I think the most significant conceptual contribution is definitely highlighting three dee ultrasound sensing as a promising complementary modality alongside other sensing modalities <ref:2601.13263#pg1>. It's not just about getting a segmentation number; it's about having that extra layer of resilient three dee spatial information when things get tricky <ref:2601.13263#pg1>.

Jane: That’s what I wanted to hear, Lu. So, looking at the improvements they suggest in their work, what are the next steps they see for making this system even better?

Tom: They point to a few specific ways to improve it: using larger datasets, refining the ground truth annotations, and employing weighted loss functions during training. Those are straightforward ways to push the segmentation performance higher.

Meng: I'm thinking about the practical side of those improvements. Getting larger datasets is always expensive and time consuming; what’s the cost-benefit analysis there? And how much effort does it take to create refined ground truth annotations?

Jane: They also touched on enhancing input data quality by implementing constant false alarm rate filtering and applying linear range compensation, which they showed significantly reduced clutter. Plus, they suggest removing empty frames with no ground truth labels before training, which improves training efficiency.

Lu: And I see a point about generalization too; they suggest ensuring the ground truth includes reflections from non-vehicle objects like pedestrians or curbs. That addresses the limitation where the U net might treat valid ultrasound returns from those things as background noise.

Tom: That sounds like they are trying to make sure the model learns what's actually there, not just what it expects to see based on training examples. It’s about improving that class generalization across more real world scenarios.

Jane: And conceptually, they suggest moving beyond binary classification to multi class detection and tracking using this framework. That would allow the system to capture the full acoustic complexity of real-world scenes, not just a simple segmentation task.

Meng: If you move into multi class detection, does that mean more complex data management at scale? What’s the deployment profile for something that’s doing multi class segmentation in real time?

Tom: That would certainly increase the computational load on the processing pipeline, but if it gives us better scene understanding, it could be worth the trade-off for safety applications.

Lu: I think the framework itself is really valuable because it provides a reproducible method for processing ultrasound volumetric data with deep learning. It establishes a solid baseline methodology that others can build upon.

Title and authors: Jane: So, to wrap up on this paper called Deep Learning for Semantic Segmentation of three dee Ultrasound Data, the main thing is that they prove this compact ultrasound sensor works as a viable complement to LiDAR and camera systems for autonomous vehicles <ref:2601.13263#pg1,Deep Learning for Semantic Segmentation of 3D Ultrasound Data>.

Tom: Yeah, it’s not replacing those sensors; it’s adding a new way to get resilient three dee spatial information when optical and radio frequency sensors might fail <ref:2601.13263#pg1>. It's a solid addition to the toolkit.

Jane: For the listener who just wants the big picture, this work shows that using deep learning on ultrasound data can provide reliable three dee segmentation for autonomous systems in harsh environments, opening up new avenues for safety redundancy <ref:2601.13263#pg1>.

Lu: From my side, it sets a good methodological baseline. It provides a framework for how to take raw ultrasound data and get meaningful semantic maps using three dee U nets that are tuned specifically to the sensor's capabilities <ref:2601.13263#pg1>.

Meng: For deployment folks like me, the real question is how much overhead this adds compared to just running our existing LiDAR and radar stack, but it shows us what’s possible if we can integrate these sensors effectively.

Lalam: Based on my analysis of the literature, I’m quite sure this direction is sound because it directly addresses the cost and robustness trade-offs in current perception systems. It seems highly relevant for mobile robotics and industrial automation where those environments are dense.

Tom: So, that's our look at Deep Learning for Semantic Segmentation of three dee Ultrasound Data <ref:2601.13263#pg1,Deep Learning for Semantic Segmentation of 3D Ultrasound Data>. It’s a solid piece of work showing how we can use novel sensor data to build better perception systems.

Jane: Indeed, it’s a great reminder that complementary sensing modalities can significantly enhance the reliability of autonomous navigation when things get difficult out there.

Lu: We'll keep an eye on how this baseline affects future multimodal fusion research. It’s a solid foundation for more complex integration down the road.

Meng: And we’ll be watching those datasets and deployment challenges closely to see if they can move this from the lab into something that runs reliably in production vehicles.

Lalam: That's all for this paper on Deep Learning for Semantic Segmentation of three dee Ultrasound Data <ref:2601.13263#pg1,Deep Learning for Semantic Segmentation of 3D Ultrasound Data>. We’ve got some interesting stuff coming up next.

The paper's summary: Tom: So basically, they built this system for volumetric perception and then they trained a three dimensional U net on the data from that sensor to do semantic segmentation.

Jane: That’s right, Tom. The main point is showing that you can use this ultrasound data to create a detailed map of the three dee space where a car is driving.

Lu: What makes it interesting conceptually is that they aren't just drawing boxes around objects; they are trying to segment every single voxel in the volume with a class label, which means they’re trying to get the actual shape right.

Meng: So instead of just knowing "there's an object here," you get a full three dee picture of its boundaries. That sounds like it could really help in predicting how fast something is moving or what kind of obstacle it is.

Tom: Exactly, and they show that this segmentation performance is robust, hitting sixty-two point six nine percent recall and sixty-seven point three zero percent F1 score on their test data.

Jane: But the real takeaway for us listening today is that this isn't just a neat experiment; it suggests three dee ultrasound can be a really useful backup sensor when things get tough, like in bad weather or when other sensors fail.

Lu: They also pointed out how they improved the input data quality by using constant false alarm rate filtering and linear range compensation to clean up the raw signals before feeding them into the U net.

Tom: And that’s good because it shows you don't just throw raw data at a deep learning model; you have to preprocess it properly for things like noise suppression and making sure distant objects are visible.

Meng: From a deployment standpoint, this means we could potentially add this sensor to an existing LiDAR or radar stack to get that extra layer of three dee spatial awareness when those other sensors might be blinded by glare or heavy rain.

Jane: It’s about building redundancy in the system, which is huge for safety applications because you don't want reliance on just one type of sensing technology.

Lu: And they’re suggesting future work should focus on integrating this three dee ultrasound with existing sensors to make a truly multimodal autonomy system.

Tom: So, the paper sets up a solid baseline methodology for processing volumetric ultrasound data with deep learning and shows it can perform reasonably well in challenging environments.

Jane: And that opens the door for more complex applications, maybe even moving beyond simple segmentation to multi class detection, capturing more of the acoustic complexity of real-world scenes.

Lalam: I’m pretty sure this direction is sound because it directly addresses the cost and robustness trade-offs we see in current perception systems. It seems highly relevant for mobile robotics and industrial automation where those environments are dense.

Tom: So, we’ve seen how they trained a three dee U net on this sensor data to get segmentation results that show promising performance numbers.

Jane: And the big picture here is that three dee ultrasound isn't just a lab experiment; it’s a promising complementary modality for reliable autonomy.

Lu: We'll keep an eye on how this baseline affects future multimodal fusion research because this work provides a solid foundation for more complex integration down the road.

The paper's improvements: Tom: So, they pointed out several ways to make this whole system better, like using larger datasets or refining the ground truth annotations for more accurate results.

Jane: Right, and they also talked about improving the input data quality by implementing constant false alarm rate filtering and applying linear range compensation to clean up those raw signals.

Lu: They also suggested removing any empty frames from the training data before you start training, which they said improves training efficiency.

Meng: That sounds like good engineering practice for making sure the model learns what it actually needs to learn without wasting compute time on useless inputs.

Tom: And they mentioned that incorporating pre-processing steps like filtering empty frames can improve both recall and precision by about five percent, which is a concrete number we can actually work with.

Jane: But the most interesting part for me is how they suggested making sure the ground truth includes reflections from non-vehicle objects like pedestrians or curbs.

Lu: They did that because they noticed that the U net sometimes treated valid ultrasound returns from those things as just background noise, so including them helps the model learn better class generalization.

Meng: That addresses a real issue; if you only train it on vehicles, it won't recognize pedestrians or street signs when it sees them in traffic.

Tom: And they also suggested moving beyond binary classification to multi class detection and tracking using this framework, which would let the system capture more of the acoustic complexity of real-world scenes.

Jane: That would mean instead of just saying "this is an object," you could be detecting different types of objects in three dee space, which is a bigger step for understanding the scene.

Lu: That extension is suggested because it captures the full acoustic complexity, and it’s a natural next step after proving the basic segmentation works.

Meng: From a production standpoint, moving to multi class detection would definitely increase the computational load on our processing pipeline, but if it gives us better scene understanding, I guess we can look at that trade-off.

Tom: So, they're giving us a clear path forward: more data and better labeling for accuracy and robustness.

Jane: It really shows that this work isn't just a finished product; it’s a baseline model for future improvements in autonomous perception.

Lu: And the conceptual idea is moving towards integrating this ultrasound sensing with other modalities to achieve resilient multimodal autonomy.

Tom: So, to wrap up on these suggestions, they are focusing on data refinement and expanding the model's capability from simple segmentation to multi class detection.

Jane: And that sets us up perfectly for looking at how this ultrasound data might actually get fused with LiDAR or radar in future work.

Conclusion: Tom: So we've covered how they used a three dimensional U net on Calyo Pulse sensor data to get volumetric segmentation for autonomous vehicle perception.

Jane: Right, Tom, and the main point is that this compact ultrasound sensor can be a really useful backup when other sensors fail in harsh environments.

Lu: It establishes a solid baseline methodology for processing this kind of volumetric ultrasound data with deep learning architectures specifically tuned to the sensor's capabilities.

Meng: For deployment folks, it shows us what’s possible if we integrate these sensors effectively into existing stacks instead of just relying on one modality.

Lalam: I'm pretty sure this work is solid because it directly tackles the cost and robustness trade-offs we see in current perception systems for mobile robotics.

Tom: So, to recap, they proved that this method provides a reproducible way to use ultrasound volumetric data with deep learning to segment three dee space.

Jane: And the big implication is highlighting three dee ultrasound as a promising complementary modality for reliable autonomy.

Lu: It's setting up a good foundation for more complex integration down the road, especially when we think about fusing it with LiDAR or radar systems.

Meng: I’ll keep an eye on how this baseline affects deployment challenges and compute requirements if we try to scale this up.

Lalam: I’m confident that this paper provides a reliable framework for getting meaningful semantic maps from ultrasound data in tough conditions.

More episodes

← Home