GeoDistill-Refine: Silhouette-First Geometry Distillation for Annotation-Free Spacecraft Segmentation

Core problem. On-orbit servicing, active debris removal, and rendezvous with non-cooperative targets require robust spacecraft foreground segmentation. Segmenting previously unseen spacecraft nevertheless remains difficult because illumination, background, apparent scale, and structural configuration vary substantially. Most existing approaches rely on manually annotated pixels, which are costly because real on-orbit imagery is scarce and spacecraft often contain low-contrast boundaries and slender appendages. Large foundation models offer supervision without manual masks, but their memory and computational requirements make direct on-board deployment impractical.

Proposed method. The paper presents GeoDistill-Refine, a two-stage framework that transfers offline SAM 3 pseudo-masks to a compact segmentation network. The approach is described as annotation-free specifically meaning student optimization without manual masks; validation annotations are still used for checkpoint selection and threshold calibration. The framework comprises:

1. Multi-prompt consensus pseudo-mask construction. A frozen SAM 3 teacher is queried with six fixed foreground prompts: spacecraft, satellite, space probe, space vehicle, spacecraft with rigid body and appendages, and artificial object in space with panels and antennas. Only predictions with score > 0 participate. The teacher pseudo-mask is obtained by an unweighted 50% consensus vote: pixelwise agreement is computed and the mask is formed where agreement ≥ 0.5. The authors note that a tie among the valid prompts is retained as foreground; for six valid prompts, at least three foreground votes are required. Prompt-specific masks, scores, and agreement maps are generated once and cached offline.

2. Silhouette-first two-stage student distillation. The student is a TinyUNet operating at 384×384 resolution with three encoder blocks of 24, 48, and 96 channels and a decoder restoring resolution through two upsampling levels with skip connections. The mask, SDF, and skeleton heads share decoded features; the area prediction is computed directly from the probability map. Only the mask branch is active in Stage I; the SDF and skeleton heads are initialized at the start of Stage II. Stage I trains the binary silhouette from random initialization for 20 epochs using balanced BCE plus Dice. Stage II is not reinitialized. Instead, low-weight geometric refinement begins from the Stage-I checkpoint, with the SDF serving as the principal continuous geometric target. Stage II runs for 10 epochs with loss coefficients fixed as λ SDF = 0.002, λ skel = 0.001, and λ area = 0.05.

3. Sample-level reliability gate. The gate combines three signals: mean foreground agreement within the voted foreground (c i), the valid-prompt ratio (v i = V i/6), and pseudo-mask area plausibility (comparing foreground area ratio a i against the training-set median a 0). The three factors are computed as clipped linear ramps for confidence and valid-prompt ratio and an exponential area-plausibility factor, then multiplied: g i = clip(g conf · g valid · g area, 0, 1). The gate g i is applied only to the SDF, skeleton, and area terms, not to the primary segmentation loss. The SDF loss weights boundary-near pixels more heavily via B i(x) = I[D i T(x) ≤ 0.55]; the skeleton term reuses the balanced BCE operator with g i broadcast as per-pixel weight; and the area term compares foreground proportions through a logarithmic ratio error. For a Landsat 8 example, the final weight is g i = 0.760, retaining substantial pseudo-geometric supervision when prompts agree and the area is plausible. Post-hoc analysis on KCP shows the suppressed group (g i = 0) has teacher Image IoU and Boundary F1 of only 0.2853 and 0.3258 versus 0.8539 and 0.9738 for the retained group, indicating that the gate identifies anomalous pseudo-labels rather than providing a calibrated continuous quality estimate.

Distance/skeleton/area targets. The SDF target is a normalized signed distance field: interior/exterior distance transforms are combined as d in − d out, normalized by the 95th-percentile absolute value (with fallback to 1), and clipped to [−1, 1]. The skeleton target consists of local maxima in the interior distance map after removing points whose distance does not exceed one pixel, with fallback to the pseudo-mask if no valid skeleton exists. The area target is the pseudo-foreground area ratio.

Results on unseen spacecraft (SpaceSense-Bench). Datasets are split by spacecraft identity with no target overlap across subsets: 270 training images of nine spacecraft, 60 validation images of three spacecraft, and three test groups (KCP, CVI, HJM), each with 60 images of three spacecraft. HJM is a pre-reserved lockbox set. Results are reported as means over three random seeds with 95% confidence intervals from 10,000 paired hierarchical bootstrap resamples. On the HJM lockbox set, Image IoU and Boundary F1 increase by 0.0456 and 0.1380, with corresponding 95% confidence intervals of [0.0107, 0.0835] and [0.0422, 0.2496] over Plain Student. Gains on the development sets are +0.0576 IoU/+0.1017 Boundary F1 on KCP and +0.0630/+0.1376 on CVI. Relative to the GABI-inspired pseudo-geometry baseline, GeoDistill-Refine gains 0.0302, 0.0330, and 0.0247 Image IoU across KCP/CVI/HJM, together with 0.0568, 0.0588, and 0.0605 Boundary F1. At a fixed threshold of 0.5 (no calibration), GeoDistill-Refine still improves Image IoU from 0.5125 ± 0.0560 to 0.5491 ± 0.0378 and Boundary F1 from 0.6863 ± 0.1313 to 0.7581 ± 0.0841.

External validation (SPEED+ and TANGO). These test in-domain distillation stability under different laboratory imaging conditions and consistency on an independent dataset, not unseen-spacecraft generalization. For these domains a pre-specified quality-adaptive extension is used, determined only from training pseudo-label statistics using a fixed boundary of 0.5. The proportions of pseudo-labels passing the confidence criterion are 0.724 (Lightbox), 0.544 (Sunlamp), and 0.308 (TANGO). Weak region anchoring is retained for the first two domains; for TANGO, the Stage-II region loss is disabled and the Stage-I checkpoint remains eligible for selection. Relative to the external reproduction baseline, GeoDistill-Refine is lower by 0.0052 and 0.0068 Image IoU on Lightbox and Sunlamp but improves Boundary F1 by 0.0226 and 0.0134 and precision by 0.0310 and 0.0089. On TANGO, Image IoU and Boundary F1 are lower by 0.0019 and 0.0064 while precision and recall increase by 0.0246 and 0.0028. These are descriptive differences between means and are not interpreted as statistically significant. Among methods using the same SAM 3 teacher and TinyUNet protocol, the quality-adaptive GeoDistill-Refine achieves the highest mean Image IoU, Boundary F1, and precision in all three external domains.

Ablations (KCP, seed 42). Joint training from scratch with the full geometry objective yields only 0.4509 Image IoU after 10 epochs; extending to the same 30-epoch budget produces 0.5645 IoU and 0.7876 Boundary F1, still below the two-stage result of 0.6056 and 0.8683. Training with only SDF from scratch gives 0.5214/0.7083 versus the warm-started 0.5979/0.8622, supporting that establishing the primary silhouette before introducing noisy pseudo-geometry is therefore more effective than joint optimization from random initialization. Component ablations identify gated SDF as the strongest reduced configuration; the full model adds 0.0077 Image IoU and 0.0061 Boundary F1 over gated-SDF-only, but area and skeleton show no monotonic standalone gain. Removing the gate lowers Image IoU from 0.6056 to 0.5917 and Boundary F1 from 0.8683 to 0.8588, supporting its role in limiting anomalous pseudo-geometry. The independent effects of skeleton and area auxiliaries remain unresolved in the single-seed ablation; this observation is also not generalized to other architectures.

Pseudo-mask quality. Multi-prompt majority voting on KCP raises pseudo-mask Image IoU and Boundary F1 by 0.1142 and 0.1338 relative to the validation-selected single prompt (spacecraft with rigid body and appendages), with the mean over six single prompts showing large dispersion (Image IoU 0.3726 ± 0.2625), demonstrating sensitivity to wording. Across datasets, the voted pseudo-masks achieve Image IoU/Boundary F1 of 0.6833/0.7794 (KCP), 0.7721/0.7600 (Lightbox), 0.6235/0.5892 (Sunlamp), and 0.3324/0.3768 (TANGO), showing regional and boundary noise need not vary together.

Efficiency and deployment. At inference, an RGB image passes only through the TinyUNet mask path... SAM 3, the prompt cache, the SDF head, and the skeleton head are absent from deployment; the area term has no prediction head to retain. The deployed student contains approximately 0.263 M parameters and 18.22 GFLOPs and requires about 1.1 ms per image on the RTX 4090, versus the teacher's 840.5 M parameters and 447.9 ms per image. Both the teacher and pseudo-geometry construction are confined to offline training.

Limitations and conclusions. The paper concludes that on the HJM lockbox set the design improves Image IoU and Boundary F1 by 0.0456 and 0.1380 over Plain Student; KCP ablations suggest that the silhouette-first warm start and gated SDF account for most of the observed refinement benefit under the current TinyUNet configuration. SPEED+ and TANGO "provide complementary evidence on difficult laboratory imagery, where the quality-adaptive configuration improves boundary quality or foreground precision but does not exceed the external reproduction baseline on every metric." The evidence is limited to binary segmentation, with manual masks used only for checkpoint selection, threshold calibration, and final evaluation, and only SpaceSense-Bench supports conclusions about unseen spacecraft identities. A qualitative failure case under severe glare shows both students inherit the teacher-derived systematic foreground omission, because pseudo-geometric refinement cannot recover structures absent from the pseudo-supervision.

Improvements to AI systems:

1. Use multi-prompt consensus pseudo-labeling instead of single-prompt teacher masks.

Query a frozen segmentation foundation model with multiple semantically valid prompts, keep only high-confidence predictions, and form a pixelwise majority-vote mask (≥50% agreement).

This raises pseudo-mask quality substantially: +0.1142 Image IoU and +0.1338 Boundary F1 over the best single prompt on KCP.

2. Distill a compact student in two stages: silhouette first, geometry second.

Train only the binary mask branch with balanced BCE + Dice for the first stage, then initialize auxiliary geometric heads from that checkpoint and continue with low loss weights.

This avoids noisy geometry from corrupting early representation learning and outperforms joint training from scratch (0.6056 vs 0.5645 Image IoU).

3. Gate unreliable pseudo-labels at the sample level.

Multiply a confidence ramp, valid-prompt-ratio ramp, and area-plausibility factor into a reliability gate, and apply it only to auxiliary geometric losses—not the main segmentation loss.

This suppresses anomalous pseudo-labels while preserving useful pseudo-geometry; removing the gate drops Image IoU from 0.6056 to 0.5917.

4. Add geometry-aware targets: signed distance field, skeleton, and foreground-area ratio.

Use a normalized SDF weighted toward boundary-near pixels, a skeleton from interior distance maxima, and a logarithmic foreground-area error. These improve boundary quality, especially for thin appendages and low-contrast edges.

5. Deploy only the tiny mask branch.

Keep the large teacher, prompt cache, SDF head, and skeleton head entirely offline. The deployed student has 0.263M parameters, 18.22 GFLOPs, and runs in 1.1 ms per image on an RTX 4090—suitable for on-board use.

What the improved AI system can do:

• Segment previously unseen spacecraft without any manually annotated training masks.

• Achieve significantly better boundary accuracy and foreground completeness on new spacecraft identities: +0.0456 Image IoU and +0.1380 Boundary F1 on a held-out lockbox set over a plain student trained with the same pseudo-masks.

• Improve segmentation on held-out development sets by +0.0576/+0.1017 and +0.0630/+0.1376 Image IoU/Boundary F1.

• Maintain reasonable performance without threshold calibration: at a fixed 0.5 threshold, Image IoU improves from 0.5125 to 0.5491 and Boundary F1 from 0.6863 to 0.7581.

• Distinguish reliable pseudo-labels from anomalous ones: samples suppressed by the gate have teacher Image IoU of only 0.2853 and Boundary F1 of 0.3258, while retained samples have 0.8539 and 0.9738.

• Operate in real time on resource-constrained hardware, making on-orbit/embedded deployment feasible.

• Transfer the same pipeline to other fine-grained foreground segmentation tasks where manual masks are scarce and boundary precision matters.