AI-Based On-Board Maritime Object Detection for Earth Observation Payload Data Reduction on Versal Embedded Hardware

arXiv:2610.12182 · cs.CV, cs.AI · Submitted 2026-10-08 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "AI-Based On-Board Maritime Object Detection for Earth Observation Payload Data Reduction on Versal Embedded Hardware".

Jane: The gist: Onboard vessel detection allows satellites to select what is downlinked, reducing data according to its content rather than coding every pixel, which is complementary to conventional onboard compression.

Tom: First, who's behind it and why it matters.

Title and authors: Tom: We’ve touched on what the paper is about and how they set up their experiment, but let's go over the title and who wrote it for a second. The paper is called "AI-Based On-Board Maritime Object Detection for Earth Observation Payload Data Reduction on Versal Embedded Hardware."

Jane: It really tells you exactly what this work aims to do: use AI on board to reduce data from Earth observation satellites, specifically focusing on maritime objects like vessels, and doing it all while running it on embedded hardware.

Lu: The authors are Thomas Goudemant, Aurélien Bobey, Omar Hlimi, and Marjorie Bellizzi. They’re bringing together computer vision research with the practical challenge of deploying models onto specialized chips like the Versal VC1902.

Meng: So it’s a cross-disciplinary effort, which I think is necessary for this kind of applied research. It bridges the gap between high-level AI concepts and actual satellite hardware limitations.

Tom: Right, and the implication here is that we’re moving beyond just compression techniques on board; we're talking about intelligent, content-aware data selection driven by real-time detection.

Jane: It means instead of saving space by compressing everything uniformly, you save space by only transmitting the parts that matter according to what the AI sees.

Lu: The whole paper follows three main axes: first the data and algorithm setup, second how they deploy it on hardware, and third the actual data reduction part with those different modes.

Meng: So when we talk about deployment, we're not just training a big model; we're dealing with quantization—taking that floating point model and making it run efficiently in eight-bit integers on the Versal DPU <ref:2610.12182#pg3>.

Tom: That’s where the engineering challenge gets really real. They have to make sure that this quantization doesn't totally kill the detection quality of the vessels they are trying to find.

Jane: It seems like they are very careful about that trade-off, which is something many researchers struggle with when moving from training to actual deployment in a resource-constrained environment.

Lu: This approach suggests that for satellite applications, efficiency on the hardware side has to be baked into the detection system from the start.

Meng: And that means the performance isn't just about how good the algorithm is on paper; it’s about how well it runs when it’s constrained by power and memory limits.

Tom: So, moving on from setting up what they did, let’s look at what they actually achieved in terms of their methodology.

The paper's summary: Jane: This is where we get into the meat of the method described in "AI-Based On-Board Maritime Object Detection for Earth Observation Payload Data Reduction on Versal Embedded Hardware." Essentially, they describe a processing chain that takes raw data through several steps to decide what to keep.

Tom: The flow starts with camera raw data, which gets corrected right there on board—things like restoring acquisition effects and co-registering the image against a reference uplinked from the ground.

Lu: After correction, because the scene is too big for one AI pass, they use a tiler to cut it into one thousand twenty-four by one thousand twenty-four pixel tiles <ref:2610.12182#pg2>. Then, each tile gets through that vessel detection stage using their YOLOX-S model, which outputs boxes and scores.

Meng: After all those detections are done across the tiles, they have a mosaicer that gathers those results into a scene-level output. Then comes the selection stage where they use it to decide what goes into mass memory or gets downlinked through one of their proposed modes.

Tom: The key takeaway here is that reducing data isn't just one algorithm; it’s this whole chain from acquisition to downlink, where the AI detection makes an informed decision about which data is valuable enough to transmit.

Jane: It’s moving away from just generic compression and towards using content awareness—if the AI sees nothing important in a tile, you don't bother sending that tile.

Lu: They also detail how they generate their training data, making sure to produce specific types of tiles: empty tiles where every grid tile with a vessel is kept, and then adding extra centered tiles around vessels or on vessels cut by the grid for better coverage of rare classes.

Meng: That careful data generation step is crucial. It ensures the detection model actually learns how to find those specific, potentially rare objects effectively.

Tom: So, to summarize: raw data correction, tiling, AI detection per tile, scene-level mosaicer results, and finally a selection stage based on those detections.

Jane: And the modes they test—from just sending metadata like box and class scores to sending image crops or even the whole scene with a degraded background.

Lu: It shows a structured way to approach data reduction where you can tune the process by changing which mode you use based on what’s available.

Tom: But this leads us into how they actually improve this system, because just having the chain isn't enough; it needs to be robust.

The paper's improvements: Jane: The authors suggest a few key improvements to make this onboard detection system even better, moving beyond just showing the basic chain in action. They focus on calibration and adaptation.

Tom: First, they talk about improving the detection fidelity by calibrating those quantized models on validation tiles before deployment. This means they’re not just deploying the model as is; they’re tuning it specifically for the real-world performance of that hardware setup.

Lu: The paper mentions this calibration strategy ensures that even when you quantize to eight-bit integers, the resulting loss in detection quality isn't a total collapse of performance <ref:2610.12182#pg3>.

Meng: That makes sense from an engineering standpoint. It acknowledges that quantization *will* cause some shift in the operating point, but by calibrating on validation tiles, they ensure it stays within acceptable bounds.

Tom: They also discuss implementing an adaptive downlink mode selection. Instead of picking one fixed mode, the system can dynamically choose between metadata only, crops around vessels, or tiles holding detections based on things like real-time scene density or link availability.

Jane: That’s smart because mission priorities change all the time. If you have a low-rate link, maybe you want to prioritize metadata only to save bandwidth, even if it means losing more vessel detail.

Lu: They show that by moving the threshold for these modes—say raising it from zero point one zero to zero point three zero—you can save about a fifth of the tile volume or crops, but you have to accept that you’re losing more vessels in the process, which is a direct trade-off.

Tom: So it’s not just about getting a high score; it’s about tuning those thresholds so the system balances energy savings against mission requirements like vessel detection accuracy.

Meng: That speaks to how practical these systems are. They aren't just theoretical models; they are built around making measurable decisions based on performance metrics that matter for the mission.

Jane: And finally, the paper suggests a significant step forward: integrating the raw data processing more deeply into the onboard chain. Moving from just AI detection per tile to incorporating raw data restoration and co-registration *before* it hits the AI stage.

Lu: That integration is what they call tying these steps together, judging the onboard reduction based on how errors from the deployed model affect each specific mode, not just looking at detection scores in isolation.

Tom: So this isn't just a better detector; it’s a smarter system where every part of the chain—correction, tiling, detection, selection—works together based on how well each step handles the errors of the preceding one.

Conclusion: Jane: So to wrap up "AI-Based On-Board Maritime Object Detection for Earth Observation Payload Data Reduction on Versal Embedded Hardware," this paper shows a structured pipeline for using AI to intelligently select data from satellites based on vessel detection.

Tom: The main contribution is tying all those steps together: the onboard reduction needs to be judged by how the errors of the deployed model affect each different downlink mode, not just by some raw detection scores.

Lu: The gain they’re aiming for is mainly in terms of energy savings, and that has to be carefully weighed against the transmission energy and storage that you’re avoiding.

Meng: It sounds like the next big step would be taking this concept and applying it to more complex scenarios, like open-sea scenes or coordinating these modes over links with different data rates.

Jane: And they suggest that the threshold for selection should actually adapt to the scene density—meaning you might adjust your settings depending on whether you’re looking at a dense harbor or an open area.

Tom: And they point out that future improvements could involve using larger networks, like YOLOX-L, or even open-vocabulary detectors to get better detection performance across all vessel types.

Lu: The paper concludes that the focus should be on this holistic approach: tying the reduction decisions directly to model errors and mode absorption.

Meng: For me, it means for practical deployment, we need to focus on making those calibration steps really robust so that the deployment shift is truly a shift in operating point rather than just losing information.

Jane: It’s a solid piece of work showing how you can combine embedded hardware constraints with sophisticated AI to achieve meaningful data reduction for Earth observation.

Tom: We’ve covered the setup, the method, and the improvements for this paper on "AI-Based On-Board Maritime Object Detection for Earth Observation Payload Data Reduction on Versal Embedded Hardware." That’s our time today.

Thomas Goudemant, Aurélien Bobey, Omar Hlimi, Marjorie Bellizzi

IRT Saint Exupéry

cs.CV, cs.AI

Submitted: 2026-10-08

Updated: 2026-10-09

Comments: 8 pages. Accepted at the 10th On-Board Payload Data Compression Workshop (OBPDC 2026), Barcelona, Spain, 12-14 October 2026

DOI: 10.5281/zenodo.23242463

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 81/100

The gist: The gist: Onboard vessel detection allows satellites to select what is downlinked, reducing data according to its content rather than coding every pixel, which is complementary to conventional

Key concepts

Onboard Vessel Detection: This is the AI step where the satellite camera images are analyzed to find ships and other maritime objects. Instead of sending all raw data, this detection identifies which parts of the image contain vessels, enabling intelligent selection for downlink.
Data Reduction Modes: The system tests four different ways to reduce data sent from space: sending only detection metadata (like a list of boxes), sending small cropped images around detected objects, sending tiles that contain detections, or sending the whole scene with a degraded background. The study evaluates how well each method handles errors from the detection AI.
Quantization and DPU Deployment: The vessel detection model is made smaller and more efficient by converting its calculations from standard floating-point numbers to 8-bit integers (int8). This optimized model is then deployed on a specific embedded processor (DPU) on the satellite, allowing it to run quickly with minimal quality loss.
Error Absorption Trade-off: The core finding is that data reduction effectiveness isn't just about detection scores. The paper shows that different modes of data reduction absorb errors from the imperfect AI model in distinct ways. For instance, crops lose every missed vessel, whereas tiles only lose a vessel if no other object in its tile is detected.

Terminology

Core — 5× in this paper · explained in 2 episodes
Quantization — 5× in this paper · explained in 12 episodes
Accuracy — 2× in this paper · explained in 1 episode
On-Board Payload Data Compression Workshop — 2× in this paper
Conference on Computer Vision and Pattern Recognition — 1× in this paper
Latent Representation — 1× in this paper · explained in 2 episodes
Latent Representations — 1× in this paper · explained in 3 episodes
Quantization-Aware Training — 1× in this paper · explained in 3 episodes

Summary

The gist: Onboard vessel detection allows satellites to select what is downlinked, reducing data according to its content rather than coding every pixel, which is complementary to conventional onboard compression.

How it works

The overall process involves a processing chain from image acquisition to downlink, which includes several steps like correction, tiling, AI detection, and selection (Fig. 1) The camera delivers raw data which are first corrected on board, e.g., restoration of acquisition effects [18] and co-registration against a compressed latent reference uplinked from the ground [20] The corrected scene, too large to be processed at once, is cut by a tiler into tiles (e.g., 1024 × 1024 pixels) Each tile then goes through the AI stage, here vessel detection, which outputs detection products (boxes, classes, scores) A mosaicer gathers the tile results into a scene-level result, and a selection stage uses it to decide what is written to the mass memory and downlinked

Data and Algorithm

The controlled dataset generation involves several steps to ensure measurable choices The input is 68 Maxar Standard (2A) pansharpened scenes of harbors, coastal areas with dense traffic and canals, annotated by an expert with 43 vessel classes (42 types and a generic “vessel” class) Each scene is converted to 8 bits with a per-scene 2–98 % percentile stretch and cut on a regular 1024 × 1024 grid, the same grid as the onboard tiler The generator controls four properties of the tiles it produces, including Empty tiles where every grid tile containing a vessel is kept, but only 30 % of the empty tiles are drawn at random so that the background does not dominate, and Class balance which adds extra tiles centered on vessels and on vessels cut by the grid to add views of rare classes.

Embedded Deployment

The detector is quantized and deployed on the DPU of a Versal VC1902 device, with a limited loss of detection quality and a processing time of a few seconds per scene The model is trained in floating point but the target computes with 8-bit integers (int8), requiring quantization before deployment The effect of quantization and deployment is measured by comparing the quantized model with the float one, on the test set and with the same metrics. The inference pipeline runs in three stages per batch: pre-processing (CPU), inference on the DPU, and post-processing (CPU)

Data Reduction by Onboard Filtering

The detections of the deployed model then decide what is downlinked through four modes compared in Table 1. These modes range from metadata only (box, class and score of each detection) to image crops around vessels, tiles holding detections, or the whole scene with a degraded background The paper quantifies this data reduction by estimating the trade-off between the vessels kept and the volume downlinked based on measured detection errors. For example, in dense harbor and coastal scenes, tiles keep 98 % of the vessels with 29 % of the scene volume, and crops 83 % with 3 %.

Experiments and Results

The detection performance after deployment shows that the AP drops from 86 to 81 %, and the F1 from 85 to 81 % (Table 3). The loss is largely a shift of the operating point rather than a loss of information, as lowering the threshold to 0.10 recovers most of the missed vessels, at the cost of more false alarms, which is the trade-off used for data reduction. The results show that with crops, every missed vessel is lost: 17 % of the vessels at threshold 0.10. With tiles, a missed vessel is lost only if no other vessel of its tile is detected, in our dense scenes this is rare. The main contribution is tying these steps together by judging onboard reduction from the errors of the deployed model and how each mode absorbs them.

Conclusion

The main contribution is to tie these steps together: onboard reduction must be judged from the errors of the deployed model and how each mode absorbs them, not from detection scores alone The gain is paid mainly in energy, to be weighed against the transmission energy and storage avoided. The main next step is raw data with detection integrated among the other bricks of the onboard chain, open-sea scenes and other applications. The modes could be orchestrated over links of different rates, and the threshold could adapt to the scene density. Larger networks (e.g., YOLOX-L) or open-vocabulary detectors could improve detection. The study concludes that onboard reduction must be judged from the errors of the deployed model and how each mode absorbs them, not from detection scores alone >.

Improvements for AI systems

textbfIncrease Detection Fidelity via Quantization Calibration Strategy: The improved AI system can maintain high detection accuracy even when deployed on resource-constrained hardware by calibrating quantized models on validation tiles to better reflect real-world performance, as described in the paper: The model is quantized by Vitis AI post-training quantization, calibrated on the validation tiles. This ensures that the deployment shift is a shift of the operating point rather than a loss of information.
textbfImplement Adaptive Downlink Mode Selection: The improved system can dynamically choose between downlink modes based on real-time scene characteristics, such as vessel density or link availability, by using the threshold moves each mode along its trade-off curve (Fig. 5): raising it to 0.30 saves about a fifth of the volume of tiles or crops, but increases the vessels lost. This allows for optimized data transmission based on mission priorities (The gain is paid mainly in energy, to be weighed against the transmission energy and storage avoided).
textbf Integrate Raw Data Processing into Onboard Chain: The improved system can transition from processing only tile-level detections to a complete onboard chain by integrating detection among the other bricks of the onboard chain (Fig. 1), specifically incorporating raw data restoration and co-registration before AI processing. This allows for future capabilities such as processing open-sea scenes and orchestrating modes over links of different rates.

Abstract

Very-high-resolution Earth-observation satellites acquire more data than they can store and downlink, while in maritime surveillance the vessels cover a tiny fraction of each scene. We study onboard vessel detection as a way to select what is downlinked, which reduces the data according to its content rather than coding every pixel; it is complementary to conventional onboard compression. The work follows three axes. (i) Data and algorithm: a controlled dataset is generated from 68 annotated Maxar scenes with 43 vessel classes, and a YOLOX-S detector is trained on it. (ii) Embedded deployment: the detector is quantized and deployed on the DPU of a Versal VC1902, with a limited loss of detection quality and a processing time of a few seconds per scene. (iii) Data reduction: we propose several downlink modes, from metadata only (box, class and score of each detection) to image crops around vessels, tiles holding detections, or the whole scene with a degraded background, and estimate from the measured detection errors the trade-off each offers between the vessels kept and the volume downlinked. On our dense harbor and coastal scenes, tiles keep 98% of the vessels with 29% of the scene volume, and crops 83% with 3%.