Task-Aware Bitrate Thresholds for Codec-Robust Ultrasound Segmentation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.
Rosa: We're looking at the paper: "Task-Aware Bitrate Thresholds for Codec-Robust Ultrasound Segmentation".
Dev: Here's the gist of the paper. Remote robot surgery and tele-ultrasound increasingly rely on real-time streaming under strict bandwidth constraints, making lossy compression unavoidableSince the receiving end of these clinical pipelines is increasingly a machine rather than a human observer, codec selection must be guided by downstream task accuracy rather than perceptual fidelityThe critical question shifts from how compression affects visual quality to how compression affects the neural network that acts on the decoded imagesIn Video Coding for Machines (VCM), the neural network’s task accuracy is a more meaningful optimization target than classical quality metricsEvaluation must capture both region overlap, Dice Similarity Coefficient score, and boundary accuracy 95th Percentile Hausdorff Distance (HD95) In surgical navigation, a mask with inaccurate boundaries can misguide instrument placementThis paper presents a three-codec (H.264, HEVC, AV1) task-aware rate-distortion study for ultrasound segmentation with nnU-Net
14: Our main contributions are: Task-aware BD-Rate analysis across three codecs (H.264, HEVC, AV1), showing that H.264 needs 82% more bitrate than AV1 at matched DicePer-codec safe bitrate thresholds, reporting worst-case (5th-percentile) Dice alongside the mean, as a deployable criterion for the lowest acceptable bitrateA leave-one-codec-out (LOCO) protocol showing that robustness learned from two codecs generalizes to an unseen third, without codec-specific retrainingThis is the first cross-codec task-aware study for medical image segmentationThe remainder of this paper is organized as followsWe evaluate three codecs spanning successive coding generations: H.264
15: , the most widely deployed clinical standard; HEVC
16: , which roughly halves H.264’s bitrate at equal quality and is increasingly used in medical video
3: ; and AV1
17: , a royalty-free Alliance for Open Media (AOM) codec that further improves low-to-medium-bitrate efficiency, encoded with libsvtav1
18: All encoders run under FFmpeg constant-quality (CRF) controlWe use nnU-Net
14: as the backbone; its self-configuring pipeline minimizes manual tuning, so observed changes reflect compression rather than model configurationTo simulate bandwidth-limited streaming, each 3D volume is encoded as a grayscale video at a nominal 30 fps, with the Z-axis as timeWe compress the validation set with FFmpeg (version 8.0.1)
20: using H.264 (libx264), HEVC (libx265), and AV1 (libsvtav1), each at six Constant Rate Factor (CRF) settings C = C = C = C = C = set, yielding 18 configurationsFor a fair comparison at matched perceptual quality, we anchor on H.264 SSIM at CRF To expose worst-case, patient-level behavior that the mean would otherwise mask, we additionally report the 5th-percentile Dice at every operating pointCompression-aware fine-tuning (Fig.
Rosa: 2, red curves) substantially mitigates the H.264 low-bitrate cliff, shifting Dice at the lowest operating point from approximately 0.63 to 0.79 and yielding a more uniform HD95 profile across codecsBecause the fine-tuned curves are nearly flat (≈ 0.83 at every bitrate), this gain comes at a small cost at high bitrate, trading peak accuracy for low-bitrate robustnessWe can observe that averaged over the six CRF levels, each LOCO model generalizes to its held-out codec, staying within a few Dice points of the baselineThe improvement is largest on H.264, whose low-bitrate cliff is poorly handled by the baselineBecause each LOCO model generalizes to a codec unseen during finetuning, this transfer can eliminate the need to retrain for every deployment codecWe presented a task-aware rate-distortion study of prostate ultrasound segmentation across three codec families (H.264, HEVC, AV1) using Dice and HD95 as quality axesOur results show that AV1 is substantially more bitrate-efficient than H.264 and HEVC for preserving segmentation accuracyCompression-aware fine-tuning recovers segmentation accuracy at extreme compression, and the leave-one-codec-out protocol confirms that this robustness generalizes to unseen codecs without retrainingFuture work will extend this framework in two directionsFirst, we plan to evaluate VVC and optimize its configuration for medical segmentation tasksSecond, since compression sensitivity is likely modality-dependent, we aim to validate the proposed methodology on other imaging modalities such as CT and MRI, and across different anatomical targets Pruning redundant reasoning tokens The paper evaluates three widely deployed codecs H.264/AVC, HEVC, and AV1 on prostate ultrasound segmentation, reporting taskaware compression performance with region overlap (Dice) and boundary accuracy (HD95) as the quality metricsTo compare codecs fairly, we match their operating points at a common perceptual-quality (SSIM) levelWe establish, per codec, the lowest bitrate that preserves acceptable segmentation accuracy, reporting the 5th-percentile Dice alongside the mean to expose worst-case degradationAll three codecs preserve segmentation quality (mean Dice ≥ 0.84) down to approximately 0.07–0.20 bits per voxel (bpv)Below this point, AV1’s accuracy declines gradually, while H.264’s drops sharplyAs a result, H.264 needs 82% more bitrate than AV1 to reach the same region-overlap accuracy (Dice), and 104.7% more to reach the same boundary accuracy (HD95)H.264 requires 82.2% more bitrate than AV1 to achieve the same Dice, and 104.7% more for matched HD95Holding mean Dice ≥ 0.84, all three codecs reach a similar bitrate floor (H.264 0.35 Mbps, 0.071 bpv, at CRF 28, HEVC 0.37 Mbps, 0.076 bpv, at CRF 28, AV1 0.42 Mbps, 0.127 bpv, at CRF The decisive difference appears below this bitrate floorAV1 degrades slowly, holding Dice ≈ 0.84 down to 0.235 Mbps, 0.049 bpv, (CRF 43), whereas H.264 has already dropped to 0.80 at How the training signal is built The baseline nnU-Net is fine-tuned with codec augmentation across three codecs and six CRF levels (20% left uncompressed) During training, each sample is either left uncompressed with probability pclean = 0.20 (to prevent catastrophic forgetting), or encoded and decoded using a randomly sampled codec-CRF pair drawn uniformly from the three codecs and the six-point CRF gridAll models were trained using stochastic gradient descent (SGD) with momentum(µ = 0.99)
21: and an initial learning rate of 0.01, decayed according to a polynomial scheduleEach epoch comprised 250 iterationsThe compression-aware model was trained for 75 epochs and each leave-one-codecout (LOCO) model for 100 epochs, with all models initialized from the baseline checkpointInstead of training on a precomputed compressed corpus, compression was applied online to each sample during trainingTo test whether compression robustness transfers across codecs, we additionally fine-tune three leave-one-codec-out (LOCO) models with the same augmentation, each holding out one codec from the compression poolEach LOCO model is fine-tuned from the baseline checkpoint, with the held-out codec never used to compress its training samples Where the gains come from The task-aware BD-Rate analysis confirms this trend: H.264 requires 82% more bitrate than AV1 to achieve the same Dice, and 104.7% more for matched HD95Compression-aware fine-tuning (Fig.
Dev: 2, red curves) substantially mitigates the H.264 low-bitrate cliff, shifting Dice at the lowest operating point from approximately 0.63 to 0.79 and yielding a more uniform HD95 profile across codecsBecause the fine-tuned curves are nearly flat (≈ 0.83 at every bitrate), this gain comes at a small cost at high bitrate, trading peak accuracy for low-bitrate robustnessThe improvement is largest on H.264, whose low-bitrate cliff is poorly handled by the baselineThe contrast is sharpest at the H.264 CRF 43 point: the baseline collapses (Dice 0.626, HD95 12.7 mm), whereas all three LOCO models recover to Dice ≈ 0.78, including LOCO-A, which never saw H.264 during fine-tuning Evaluation Metrics and Protocols Segmentation quality is measured using two complementary metricsThe Dice Similarity Coefficient (DSC)
22: quantifies volumetric overlap between predicted and ground-truth masks, ranging from 0 (no overlap) to 1 (perfect agreement)The 95th-percentile Hausdorff Distance (HD95)
10: measures boundary accuracy by computing the 95th percentile of the symmetric surface distances in mm between predicted and ground-truth contours, making it sensitive to boundary errors that Dice alone cannot captureFor completeness, we additionally report PSNR and SSIM between the original and decoded volumesTo provide guidance for practical applications, we define per-codec safe bitrate thresholds as the minimum bitrate at which the mean Dice across all 299 test volumes remains above an acceptable floor (Dice ≥ 0.84), i.e.
Rosa: no more than 3 percentage points below the uncompressed baseline testset(Dice = 0.869)To expose worst-case, patient-level behavior that the mean would otherwise mask, we additionally report the 5th-percentile Dice at every operating point Pruning redundant reasoning tokens The paper evaluates three widely deployed codecs H.264/AVC, HEVC, and AV1 on prostate ultrasound segmentation, reporting taskaware compression performance with region overlap (Dice) and boundary accuracy (HD95) as the quality metricsTo compare codecs fairly, we match their operating points at a common perceptual-quality (SSIM) levelWe establish, per codec, the lowest bitrate that preserves acceptable segmentation accuracy, reporting the 5th-percentile Dice alongside the mean to expose worst-case degradationAll three codecs preserve segmentation quality (mean Dice ≥ 0.84) down to approximately 0.07–0.20 bits per voxel (bpv)Below this point, AV1’s accuracy declines gradually, while H.264’s drops sharplyAs a result, H.264 needs 82% more bitrate than AV1 to reach the same region-overlap accuracy (Dice), and 104.7% more to reach the same boundary accuracy (HD95)H.264 requires 82.2% more bitrate than AV1 to achieve the same Dice, and 104.7% more for matched HD95Holding mean Dice ≥ 0.84, all three codecs reach a similar bitrate floor (H.264 0.35 Mbps, 0.071 bpv, at CRF 28, HEVC 0.37 Mbps, 0.076 bpv, at CRF The decisive difference appears below this bitrate floorAV1 degrades slowly, holding Dice ≈ 0.84 down to 0.235 Mbps, 0.049 bpv, (CRF 43), whereas H.264 has already dropped to 0.80 at Pruning redundant reasoning tokens The paper evaluates three widely deployed codecs H.264/AVC, HEVC, and AV1 on prostate ultrasound segmentation, reporting taskaware compression performance with region overlap (Dice) and boundary accuracy (HD95) as the quality metricsTo compare codecs fairly, we match their operating points at a common perceptual-quality (SSIM) levelWe establish, per codec, the lowest bitrate that preserves acceptable segmentation accuracy, reporting the 5th-percentile Dice alongside the mean to expose worst-case degradationAll three codecs preserve segmentation quality (mean Dice ≥ 0.84) down to approximately 0.07–0.20 bits per voxel (bpv)Below this point, AV1’s accuracy declines gradually, while H.264’s drops sharplyAs a result, H.264 needs 82% more bitrate than AV1 to reach the same region-overlap accuracy (Dice), and 104.7% more to reach the same boundary accuracy (HD95)H.264 requires 82.2% more bitrate than AV1 to achieve the same Dice, and 104.7% more for matched HD95Holding mean Dice ≥ 0.84, all three codecs reach a similar bitrate floor (H.264 0.35 Mbps, 0.071 bpv, at CRF The decisive difference appears below this bitrate floorAV1 degrades slowly, holding Dice ≈ 0.84 down to 0.235 Mbps, 0.049 bpv, (CRF Evaluation Metrics and Protocols Segmentation quality is measured using two complementary metricsThe Dice Similarity Coefficient (DSC)
22: quantifies volumetric overlap between predicted and ground-truth masks, ranging from 0 (no overlap) to 1 (perfect agreement)The 95th-percentile Hausdorff Distance (HD95)
10: measures boundary accuracy by computing the 95th percentile of the symmetric surface distances in mm between predicted and ground-truth contours, making it sensitive to boundary errors that Dice alone cannot captureTo provide guidance for practical applications, we define per-codec safe bitrate thresholds as the minimum bitrate at which the mean Dice across all 299 test volumes remains above an acceptable floor (Dice ≥ 0.84), i.e.
Paper summary: Rosa: So we're looking at this paper titled "Task-Aware Bitrate Thresholds for Codec-Robust Ultrasound Segmentation." It’s about how we pick the right video codec when we’re streaming medical images over a limited bandwidth.
Dev: Yeah, it tackles the idea that you can't just optimize for how pretty the picture looks; you have to optimize for what a machine needs to do with that picture.
Rosa: Exactly. The thesis here is that since these systems are now running on machines, codec choice has to be based on downstream task accuracy rather than visual quality metrics like Perceptual Quality or SSIM.
Dev: So they aren't just looking at the video quality; they’re looking at how the compression affects the neural network itself.
Rosa: Right. They evaluate three major codecs: H-two hundred sixty-four HEVC, and AV1 on prostate ultrasound segmentation using nnU-Net as their backbone <ref:2611.06992#pg1,HEVC, and AV1 on prostate ultrasound segmentation>.
Dev: And they use Dice similarity and 95th percentile Hausdorff Distance as their quality metrics for that segmentation task <ref:2611.06992#pg3>.
Rosa: The main claim is a task aware rate distortion study showing that H-two hundred sixty-four needs eighty-two percent more bitrate than AV1 to achieve the same Dice score when matched at perceptual quality <ref:2611.06992#pg1>.
Dev: That’s a big gap. They also report safe bitrate thresholds for each codec, using the 5th percentile of Dice alongside the mean to show how bad things can get in the worst case <ref:2611.06992#pg1,Dice alongside the mean to>.
Rosa: And they found that compression-aware fine tuning substantially helps mitigate the low bitrate cliff that H-two hundred sixty-four has, shifting Dice at the lowest operating point from about zero point six three to about zero point seven nine and keeping the 95th percentile Hausdorff Distance profile more uniform across all codecs <ref:2611.06992#pg3>.
Dev: It sounds like they found a way to make H-two hundred sixty-four work much better under stress without needing a massive bitrate increase <ref:2611.06992#pg1>.
Rosa: That’s the core idea of their compression-aware fine tuning, which trades off peak accuracy at high bitrates for robustness at low bitrates.
Dev: But what about the transferability? The paper shows that using a leave one codec out protocol, or LOCO, means that a model robust to two codecs generalizes to an unseen third without needing retraining for every deployment codec.
Rosa: That’s pretty interesting because it suggests you don't have to spend all that time retraining models just because the streaming pipeline switches codecs in the field.
Dev: It moves the optimization away from specific codec tuning toward a more generalized robustness across different compression standards.
Rosa: So, they used H-two hundred sixty-four HEVC, and AV1 spanning successive coding generations to set up their comparison <ref:2611.06992#pg1>.
Paper summary: Dev: They used constant quality control for all encoders under FFmpeg constant quality rate factor control, which gives them a fair starting point for comparison.
Rosa: They also used nnU-Net as the backbone because its self configuring pipeline minimizes manual tuning, so whatever changes you see is really due to the compression itself and not model configuration.
Dev: And they simulated streaming by encoding each three dimensional volume as a grayscale video at thirty frames per second with the z axis representing time.
Rosa: So, for someone listening who doesn't do medical imaging, what does this mean? It means that in remote robot surgery or tele-ultrasound where bandwidth is tight, you can’t just pick the newest codec because it looks best; you have to pick one based on how well it keeps the segmentation task accurate when things get compressed.
Dev: Right. And they found that AV1 is substantially more bitrate efficient than both H-two hundred sixty-four and HEVC for preserving segmentation accuracy, according to their task aware rate distortion study <ref:2611.06992#pg1>.
Rosa: The paper shows that compression-aware fine tuning recovers segmentation accuracy at extreme compression, and the leave one codec out protocol confirms that this robustness generalizes to unseen codecs without retraining.
Dev: So the main implication is a way to select a codec based on how it maintains the required task accuracy under real-world streaming constraints, not just visual metrics.
Rosa: The paper evaluates three widely deployed codecs H-two hundred sixty-four HEVC, and AV1 on prostate ultrasound segmentation using Dice and HD95 as the quality metrics <ref:2611.06992#pg1,HEVC, and AV1 on prostate ultrasound segmentation>.
Dev: They compare them by matching their operating points at a common perceptual quality level defined by SSIM.
Rosa: They establish the lowest bitrate per codec that preserves acceptable segmentation accuracy, reporting the 5th percentile Dice alongside the mean to expose worst case degradation <ref:2611.06992#pg1,that preserves acceptable segmentation accuracy, reporting the 5th percentile Dice alongside the>.
Dev: All three codecs preserve segmentation quality with a mean Dice of about zero point eight four down to approximately zero point zero seven minus two tenths bits per voxel.
Rosa: Below that bit rate, AV1’s accuracy declines gradually, while H-two hundred sixty-four’s drops sharply <ref:2611.06992#pg1>.
Dev: As a result, H-two hundred sixty-four needs eighty-two percent more bitrate than AV1 to reach the same region overlap accuracy defined by the Dice score and one hundred four point seven percent more for matched boundary accuracy defined by the Hausdorff Distance ninety five <ref:2611.06992#pg1>.
Rosa: Holding that mean Dice above zero point eight four, all three codecs reach a similar bitrate floor at CRF twenty eight, HEVC at thirty seven megabits per second, and AV1 at forty two megabits per second.
Dev: The decisive difference appears below that bitrate floor, where AV1 degrades slowly while H-two hundred sixty-four has already dropped to zero point eight zero when you look at the lowest operating point <ref:2611.06992#pg1>.
Paper summary: Rosa: They also found that compression-aware fine tuning substantially mitigates the H-two hundred sixty-four low bitrate cliff, shifting Dice at the lowest operating point from about zero point six three to about zero point seven nine and yielding a more uniform Hausdorff Distance ninety five profile across all codecs <ref:2611.06992#pg1>.
Dev: That means if you tune your system with this fine tuning, you get a much flatter curve when you're squeezing the bits out of H-two hundred sixty-four <ref:2611.06992#pg1>.
Rosa: But they note that this gain comes at a small cost at high bitrates, trading peak accuracy for low bitrate robustness.
Dev: And the contrast is sharpest at the H-two hundred sixty-four CRF forty three point, where the baseline collapses to a Dice of zero point six two six and a Hausdorff Distance ninety five of twelve point seven millimeters <ref:2611.06992#pg1>.
Rosa: But all three leave one codec out models recover to about zero point seven eight, including LOCO A, which never saw H-two hundred sixty-four during fine tuning <ref:2611.06992#pg1>.
Dev: That transfer is huge because it eliminates the need to retrain for every deployment codec you might encounter in the field.
Rosa: The paper evaluates three widely deployed codecs H-two hundred sixty-four/AVC, HEVC, and AV1 on prostate ultrasound segmentation reporting task aware compression performance with region overlap Dice and boundary accuracy Hausdorff Distance ninety five as the quality metrics <ref:2611.06992#pg1,three widely deployed codecs H-264/AVC, HEVC, and AV1 on prostate>.
Dev: To compare them fairly, they match their operating points at a common perceptual quality level defined by SSIM.
Rosa: They establish per codec the lowest bitrate that preserves acceptable segmentation accuracy, reporting the 5th percentile Dice alongside the mean to expose worst case degradation <ref:2611.06992#pg1,establish per codec the lowest bitrate that preserves acceptable segmentation accuracy, reporting>.
Rosa: Below that bit rate, AV1’s accuracy declines gradually while H-two hundred sixty-four’s drops sharply <ref:2611.06992#pg1>.
Dev: So the main implication is that if you're deploying a system in a bandwidth limited environment, you need to look at how compression affects the neural network task rather than just how much visual fidelity is lost.
Rosa: This paper, "Task-Aware Bitrate Thresholds for Codec-Robust Ultrasound Segmentation," shows that AV1 is substantially more bitrate efficient than H-two hundred sixty-four and HEVC for preserving segmentation accuracy <ref:2611.06992#pg1,Task-Aware Bitrate Thresholds for Codec-Robust Ultrasound Segmentation>.
Dev: Compression aware fine tuning recovers segmentation accuracy at extreme compression, and the leave one codec out protocol confirms that this robustness generalizes to unseen codecs without retraining.
Rosa: They plan to evaluate Video Coding for Machines, or VCM, in future work by optimizing VVC for medical segmentation tasks.
Dev: And they also aim to validate this methodology on other imaging modalities like CT and MRI, across different anatomical targets.
Conclusion: Rosa: So we're talking about this paper called "Task-Aware Bitrate Thresholds for Codec-Robust Ultrasound Segmentation."
Dev: It’s about figuring out which video codec to use when you’re streaming medical images over a tight bandwidth, and it focuses on what the machine needs to actually do with those images.
Rosa: Yeah, the authors used three codecs: H-two hundred sixty-four HEVC, and AV1.
Dev: And they didn't just look at how pretty the picture looked; they measured quality using Dice similarity and ninety-fifth percentile Hausdorff Distance.
Rosa: The main finding is that H-two hundred sixty-four needs eighty-two percent more bitrate than AV1 to keep the same level of accuracy for those segmentation tasks.
Dev: They also set these safe bitrates, reporting the worst case, which is the fifth percentile of Dice score.
Rosa: And they showed that compression-aware fine tuning helps H-two hundred sixty-four handle low bitrates much better, boosting that Dice score from about zero point six three up to zero point seven nine at the lowest points.
Dev: That means you can get way more robustness when you're squeezing the bits out of H-two hundred sixty-four even if it costs a little bit of peak accuracy at higher bitrates.
Rosa: And they used a leave one codec out protocol, or LOCO, which shows that if a model is trained robustly across two codecs, it still works well on an unseen third without needing to be retrained for every single deployment codec.
Dev: That’s important because it means you can deploy models knowing they'll handle different streaming scenarios better than just one specific codec.
Rosa: It suggests that picking a codec shouldn't just be about the newest technology; it should be about how well the compression affects the actual task accuracy for the machine.
Dev: So, this research points toward a framework where you select codecs based on their impact on downstream performance metrics rather than just visual quality scores.
Rosa: It sets up a whole new way to think about codec selection in remote surgery and other real-time medical streaming scenarios.
Dev: Next, we're going to look at how the authors actually built those models and the training process that led to these results.
Brandenburgische Technische Universitat (BTU)
eess.IV
Submitted: 2026-11-04
Updated: 2026-11-04
Comments: It is already accepted at IEEE Visual Communications and Image Processing Conference
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 100/100
The gist: Remote robot surgery and tele-ultrasound increasingly rely on real-time streaming under strict bandwidth constraints…
Key concepts
- Task-Aware Rate-Distortion
- This approach optimizes video compression not just for visual quality, but specifically to maximize the accuracy of a downstream task, like medical image segmentation. Instead of just minimizing distortion, the goal is to minimize the impact on the neural network's performance on a specific medical task.
- Dice Similarity Coefficient (DSC)
- DSC measures how much two segmentation masks overlap. A score of 1 means perfect agreement between what the model predicted and what is in reality. It is a key metric for assessing how well the robot can identify and outline structures in an ultrasound image.
- 95th Percentile Hausdorff Distance (HD95)
- HD95 measures boundary accuracy by looking at the worst-case errors along the edges of the predicted mask. It calculates the distance to the ground truth for 95% of all points, making it crucial for surgical navigation where precise boundary placement is critical.
- Leave-One-Codec-Out (LOCO) Protocol
- This protocol tests model robustness by training models using augmented data from two codecs and then testing them on an unseen third codec. This demonstrates that the knowledge gained from training across different compression methods can generalize to new, unencountered compression formats without needing retraining.
Terminology
Summary
Remote robot surgery and tele-ultrasound increasingly rely on real-time streaming under strict bandwidth constraints, making lossy compression unavoidableSince the receiving end of these clinical pipelines is increasingly a machine rather than a human observer, codec selection must be guided by downstream task accuracy rather than perceptual fidelityThe critical question shifts from how compression affects visual quality to how compression affects the neural network that acts on the decoded imagesIn Video Coding for Machines (VCM), the neural network’s task accuracy is a more meaningful optimization target than classical quality metricsEvaluation must capture both region overlap, Dice Similarity Coefficient score, and boundary accuracy 95th Percentile Hausdorff Distance (HD95) In surgical navigation, a mask with inaccurate boundaries can misguide instrument placementThis paper presents a three-codec (H. p. 0264, HEVC, AV1) task-aware rate-distortion study for ultrasound segmentation with nnU-Net [14]Our main contributions are: Task-aware BD-Rate analysis across three codecs (H. p. 0264, HEVC, AV1), showing that H. p. 0264 needs 82% more bitrate than AV1 at matched DicePer-codec safe bitrate thresholds, reporting worst-case (5th-percentile) Dice alongside the mean, as a deployable criterion for the lowest acceptable bitrateA leave-one-codec-out (LOCO) protocol showing that robustness learned from two codecs generalizes to an unseen third, without codec-specific retrainingThis is the first cross-codec task-aware study for medical image segmentationThe remainder of this paper is organized as followsWe evaluate three codecs spanning successive coding generations: H. p. 0264 [15], the most widely deployed clinical standard; HEVC [16], which roughly halves H. p. 1264’s bitrate at equal quality and is increasingly used in medical video [3]; and AV1 [17], a royalty-free Alliance for Open Media (AOM) codec that further improves low-to-medium-bitrate efficiency, encoded with libsvtav1 [18]All encoders run under FFmpeg constant-quality (CRF) controlWe use nnU-Net [14] as the backbone; its self-configuring pipeline minimizes manual tuning, so observed changes reflect compression rather than model configurationTo simulate bandwidth-limited streaming, each 3D volume is encoded as a grayscale video at a nominal 30 fps, with the Z-axis as timeWe compress the validation set with FFmpeg (version 8. p. 10.1) [20] using H. p. 0264 (libx264), HEVC (libx265), and AV1 (libsvtav1), each at six Constant Rate Factor (CRF) settings C = C = C = C = C = set, yielding 18 configurationsFor a fair comparison at matched perceptual quality, we anchor on H. p. 1264 SSIM at CRF To expose worst-case, patient-level behavior that the mean would otherwise mask, we additionally report the 5th-percentile Dice at every operating pointCompression-aware fine-tuning (Fig. p. 2 2, red curves) substantially mitigates the H. p. 2 p. 2264 low-bitrate cliff, shifting Dice at the lowest operating point from approximately 0. p. 2 p. 263 to 0.79 and yielding a more uniform HD95 profile across codecsBecause the fine-tuned curves are nearly flat (≈ 0. p. 2 p. 283 at every bitrate), this gain comes at a small cost at high bitrate, trading peak accuracy for low-bitrate robustnessWe can observe that averaged over the six CRF levels, each LOCO model generalizes to its held-out codec, staying within a few Dice points of the baselineThe improvement is largest on H. p. 3264, whose low-bitrate cliff is poorly handled by the baselineBecause each LOCO model generalizes to a codec unseen during finetuning, this transfer can eliminate the need to retrain for every deployment codecWe presented a task-aware rate-distortion study of prostate ultrasound segmentation across three codec families (H. p. 3264, HEVC, AV1) using Dice and HD95 as quality axesOur results show that AV1 is substantially more bitrate-efficient than H. p. 3264 and HEVC for preserving segmentation accuracyCompression-aware fine-tuning recovers segmentation accuracy at extreme compression, and the leave-one-codec-out protocol confirms that this robustness generalizes to unseen codecs without retrainingFuture work will extend this framework in two directionsFirst, we plan to evaluate VVC and optimize its configuration for medical segmentation tasksSecond, since compression sensitivity is likely modality-dependent, we aim to validate the proposed methodology on other imaging modalities such as CT and MRI, and across different anatomical targets
Pruning redundant reasoning tokens
The paper evaluates three widely deployed codecs H.264/AVC, HEVC, and AV1 on prostate ultrasound segmentation, reporting taskaware compression performance with region overlap (Dice) and boundary accuracy (HD95) as the quality metricsTo compare codecs fairly, we match their operating points at a common perceptual-quality (SSIM) levelWe establish, per codec, the lowest bitrate that preserves acceptable segmentation accuracy, reporting the 5th-percentile Dice alongside the mean to expose worst-case degradationAll three codecs preserve segmentation quality (mean Dice ≥ 0. p. 0 p. 0 p. 0 p. 084) down to approximately 0. p. 0 p. 0 p. 0 p. 007–0.20 bits per voxel (bpv)Below this point, AV1’s accuracy declines gradually, while H. p. 0 p. 0 p. 0 p. 0264’s drops sharplyAs a result, H. p. 0 p. 0 p. 0 p. 0264 needs 82% more bitrate than AV1 to reach the same region-overlap accuracy (Dice), and 104. p. 0 p. 0 p. 0 p. 07% more to reach the same boundary accuracy (HD95)H. p. 0 p. 0 p. 0 p. 0264 requires 82. p. 2 p. 2 p. 2 p. 22% more bitrate than AV1 to achieve the same Dice, and 104. p. 2 p. 2 p. 2 p. 27% more for matched HD95Holding mean Dice ≥ 0. p. 0 p. 0 p. 0 p. 084, all three codecs reach a similar bitrate floor (H. p. 2 p. 2 p. 2264 0.35 Mbps, 0. p. 1 p. 1 p. 1071 bpv, at CRF 28, HEVC 0. p. 2 p. 237 Mbps, 0. p. 1 p. 1076 bpv, at CRF 28, AV1 0. p. 242 Mbps, 0. p. 1127 bpv, at CRF The decisive difference appears below this bitrate floorAV1 degrades slowly, holding Dice ≈ 0. p. 284 down to 0.235 Mbps, 0. p. 2 p. 2 p. 2049 bpv, (CRF 43), whereas H. p. 2 p. 2264 has already dropped to 0. p. 2 p. 280 at
How the training signal is built
The baseline nnU-Net is fine-tuned with codec augmentation across three codecs and six CRF levels (20% left uncompressed) During training, each sample is either left uncompressed with probability pclean = 0.20 (to prevent catastrophic forgetting), or encoded and decoded using a randomly sampled codec-CRF pair drawn uniformly from the three codecs and the six-point CRF gridAll models were trained using stochastic gradient descent (SGD) with momentum(µ = 0. p. 299) [21] and an initial learning rate of 0. p. 201, decayed according to a polynomial scheduleEach epoch comprised 250 iterationsThe compression-aware model was trained for 75 epochs and each leave-one-codecout (LOCO) model for 100 epochs, with all models initialized from the baseline checkpointInstead of training on a precomputed compressed corpus, compression was applied online to each sample during trainingTo test whether compression robustness transfers across codecs, we additionally fine-tune three leave-one-codec-out (LOCO) models with the same augmentation, each holding out one codec from the compression poolEach LOCO model is fine-tuned from the baseline checkpoint, with the held-out codec never used to compress its training samples
Where the gains come from
The task-aware BD-Rate analysis confirms this trend: H.264 requires 82% more bitrate than AV1 to achieve the same Dice, and 104. p. 27% more for matched HD95Compression-aware fine-tuning (Fig. p. 2 2, red curves) substantially mitigates the H.264 low-bitrate cliff, shifting Dice at the lowest operating point from approximately 0.63 to 0.79 and yielding a more uniform HD95 profile across codecsBecause the fine-tuned curves are nearly flat (≈ 0.83 at every bitrate), this gain comes at a small cost at high bitrate, trading peak accuracy for low-bitrate robustnessThe improvement is largest on H. p. 3264, whose low-bitrate cliff is poorly handled by the baselineThe contrast is sharpest at the H. p. 3264 CRF 43 point: the baseline collapses (Dice 0. p. 3626, HD95 12. p. 37 mm), whereas all three LOCO models recover to Dice ≈ 0. p. 378, including LOCO-A, which never saw H. p. 3264 during fine-tuning
Evaluation Metrics and Protocols
Segmentation quality is measured using two complementary metricsThe Dice Similarity Coefficient (DSC) [22] quantifies volumetric overlap between predicted and ground-truth masks, ranging from 0 (no overlap) to 1 (perfect agreement)The 95th-percentile Hausdorff Distance (HD95) [10] measures boundary accuracy by computing the 95th percentile of the symmetric surface distances in mm between predicted and ground-truth contours, making it sensitive to boundary errors that Dice alone cannot captureFor completeness, we additionally report PSNR and SSIM between the original and decoded volumesTo provide guidance for practical applications, we define per-codec safe bitrate thresholds as the minimum bitrate at which the mean Dice across all 299 test volumes remains above an acceptable floor (Dice ≥ 0.84), i.e. no more than 3 percentage points below the uncompressed baseline testset(Dice = 0. p. 2 p. 2869)To expose worst-case, patient-level behavior that the mean would otherwise mask, we additionally report the 5th-percentile Dice at every operating point
The paper evaluates three widely deployed codecs H.264/AVC, HEVC, and AV1 on prostate ultrasound segmentation, reporting taskaware compression performance with region overlap (Dice) and boundary accuracy (HD95) as the quality metricsTo compare codecs fairly, we match their operating points at a common perceptual-quality (SSIM) levelWe establish, per codec, the lowest bitrate that preserves acceptable segmentation accuracy, reporting the 5th-percentile Dice alongside the mean to expose worst-case degradationAll three codecs preserve segmentation quality (mean Dice ≥ 0.84) down to approximately 0.07–0.20 bits per voxel (bpv)Below this point, AV1’s accuracy declines gradually, while H.264’s drops sharplyAs a result, H.264 needs 82% more bitrate than AV1 to reach the same region-overlap accuracy (Dice), and 104.7% more to reach the same boundary accuracy (HD95)H.264 requires 82.2% more bitrate than AV1 to achieve the same Dice, and 104.7% more for matched HD95Holding mean Dice ≥ 0.84, all three codecs reach a similar bitrate floor (H.264 0.35 Mbps, 0.071 bpv, at CRF 28, HEVC 0.37 Mbps, 0.076 bpv, at CRF The decisive difference appears below this bitrate floorAV1 degrades slowly, holding Dice ≈ 0. p. 284 down to 0.235 Mbps, 0.049 bpv, (CRF 43), whereas H.264 has already dropped to 0.80 at
The paper evaluates three widely deployed codecs H.264/AVC, HEVC, and AV1 on prostate ultrasound segmentation, reporting taskaware compression performance with region overlap (Dice) and boundary accuracy (HD95) as the quality metricsTo compare codecs fairly, we match their operating points at a common perceptual-quality (SSIM) levelWe establish, per codec, the lowest bitrate that preserves acceptable segmentation accuracy, reporting the 5th-percentile Dice alongside the mean to expose worst-case degradationAll three codecs preserve segmentation quality (mean Dice ≥ 0.84) down to approximately 0.07–0.20 bits per voxel (bpv)Below this point, AV1’s accuracy declines gradually, while H.264’s drops sharplyAs a result, H.264 needs 82% more bitrate than AV1 to reach the same region-overlap accuracy (Dice), and 104.7% more to reach the same boundary accuracy (HD95)H.264 requires 82.2% more bitrate than AV1 to achieve the same Dice, and 104.7% more for matched HD95Holding mean Dice ≥ 0.84, all three codecs reach a similar bitrate floor (H.264 0.35 Mbps, 0.071 bpv, at CRF The decisive difference appears below this bitrate floorAV1 degrades slowly, holding Dice ≈ 0. p. 284 down to 0.235 Mbps, 0.049 bpv, (CRF
Segmentation quality is measured using two complementary metricsThe Dice Similarity Coefficient (DSC) [22] quantifies volumetric overlap between predicted and ground-truth masks, ranging from 0 (no overlap) to 1 (perfect agreement)The 95th-percentile Hausdorff Distance (HD95) [10] measures boundary accuracy by computing the 95th percentile of the symmetric surface distances in mm between predicted and ground-truth contours, making it sensitive to boundary errors that Dice alone cannot captureTo provide guidance for practical applications, we define per-codec safe bitrate thresholds as the minimum bitrate at which the mean Dice across all 299 test volumes remains above an acceptable floor (Dice ≥ 0.84), i.e. no more than 3 percentage points below the uncompressed baseline testset(Dice = 0.869)To expose worst-case, patient-level behavior that the mean would otherwise mask, we additionally report the 5th-percentile Dice at every operating point
The paper evaluates three widely deployed codecs H.264/AVC, HEVC, and AV1 on prostate ultrasound segmentation, reporting taskaware compression performance with region overlap (Dice) and boundary accuracy (HD95) as the quality metricsTo compare codecs fairly, we match their operating points at a common perceptual-quality (SSIM) levelWe establish, per codec, the lowest bitrate that preserves acceptable segmentation accuracy, reporting the 5th-percentile Dice alongside the mean to expose worst-case degradationAll three codecs preserve segmentation quality (mean Dice ≥ 0.84) down to approximately 0.07–0.20 bits per voxel (bpv)Below this point, AV1’s accuracy declines gradually, while H.264’s drops sharplyAs a result, H.264 needs 82% more bitrate than AV1 to reach the same region-overlap accuracy (Dice), and 104.7% more to reach the same boundary accuracy (HD95)H.264 requires 82.2% more bitrate than AV1 to achieve the same Dice, and 104.7% more for matched HD95Holding mean Dice ≥ 0.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Implement task-aware bitrate selection for remote ultrasound streaming by using per-codec safe bitrate thresholds, which report
worst-case (5th-percentile) Dice alongside the mean
to exposeworst-case degradation.
-
Enhance low-bitrate robustness by employing compression aware fine tuning, which shifts the Dice at the lowest operating point
from approximately 0.63 to 0.79 and yielding a more uniform HD95 profile across codecs.
-
Improve codec generalization by utilizing a leave-one-codec-out (LOCO) protocol, which shows that
robustness learned from two codecs generalizes to an unseen third, without codec-specific retraining.
-
Develop deployment guidance for clinical systems by using the Bjøntegaard Delta Rate (BD-Rate) analysis to determine that
H.264 needs 82% more bitrate than AV1 to achieve the same Dice
and establishing thresholds where "mean Dice across all 299 test volumes remains above an acceptable floor (Dice >= 0.84)." -
Enable automated, cross-codec deployment by allowing models to generalize across codecs unseen during fine-tuning, as the LOCO protocol confirms this
transfer can eliminate the need to retrain for every deployment codec.
Sources
Related papers
- HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
- Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction
- FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception
- A Disease-Centric Vision-Language Foundation Model for Precision Oncology in Kidney Cancer
- CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography
- RPL-UIE: Reference-Guided Prior Learning for Content-Preserving Underwater Image Enhancement