HyTBE: Hyperbolic Target-Background Expert Model for Cross-Domain Infrared Small Target Detection

This paper addresses the problem of cross-domain generalization in infrared small target detection (IRSTD). The authors note that although IRSTD has achieved substantial progress under domain-consistent evaluation, yet detector performance often degrades markedly when generalizing to unseen infrared domains. They argue that existing methods primarily improve detection by enhancing target responses and suppressing background interference, but when trained on only a limited set of source domains, their learned decision rules are inevitably established from a restricted range of source-domain target-background relation patterns.

The paper formulates the cross-domain failure as target-background relation shift: unseen domains may exhibit relation patterns that are not observed during training, thereby weakening the discriminative capability learned from the source domains. The authors explain that because infrared small targets contain few pixels and lack stable texture and semantic structures, "their detectability depends strongly on their relative contrast, saliency, morphology, and contextual difference from the surrounding background. Consequently, a target that is sufficiently distinctive against one background may become ambiguous against another. They conclude that a decision boundary learned from limited source-domain relation patterns may fail to reliably separate target and background features in an unseen domain," and define this phenomenon as target-background relation shift.

To address this problem, the authors propose HyTBE, a Hyperbolic Target-Background Expert model that expands source-domain relation patterns and adaptively adjusts visual representations using explicit relation cues. The method consists of three main components that form a progressive pipeline that diversifies target-background relations, explicitly represents them, and adapts visual features accordingly.

TBRI selectively perturbs either targets or backgrounds, broadening the observable relation patterns during training while maintaining valid supervision. It operates as follows: For each training sample, TBRI is activated with probability p = 0.5. Once activated, exactly one intervention operator is sampled from either the background or target branch.

• Background intervention diversifies the imaging context while preserving the target support, considering variations in global intensity response, background contrast, high-frequency components, and noise patterns. The target region and its annotation are retained, while the surrounding background is replaced.

• Target intervention preserves the surrounding background while jointly transforming target appearance and target support, considering variations in target saliency, brightness, morphology, and scale, together with random target sampling from the source domains. For random target sampling, an additional source-domain target is transformed and inserted into a valid background location.

HRM maps multi-scale visual cues into a Poincaré ball and characterizes each feature token through its relative distances to target and background anchors. Each relation token is mapped from the tangent space at the origin onto the Poincaré ball using the exponential map (with curvature c fixed to 1). Two opposite anchors are constructed, at = Expc0(ρ) and ab = Expc0(−ρ), representing target and background references. The relation score of each token is defined as si = dc(zi, ab) − dc(zi, at), where dc denotes geodesic distance. Thus, "si > 0 indicates a target relation, whereas si < 0 indicates a background relation."

A hyperbolic relation loss is defined: Lhyp = Lt + Lb, where Lt and Lb use a margin m to encourage target tokens to satisfy si ≥ m and background tokens to satisfy si ≤ −m, while preserving the diversity of relation patterns within each side.

HMA consists of Guide-Attention and a MoE adapter. Guide-Attention uses the average pooled visual feature as query, the HRM relation representation as key, and the visual feature as value, computing a relation-guided feature via a sigmoid-gated attention mechanism, so that HRM relation cues recalibrate the channel responses of each encoder feature. The guided feature is then fed to a scale-specific MoE adapter, where each expert applies a residual block with a channel gate. A router predicts input-dependent soft weights over the E experts, and expert corrections are aggregated by weighted fusion and added back to the guided feature with a learnable residual scale.

The model is trained with the total loss: L = LIoU + Lhyp + Llb + Lnc, supervising target segmentation, hyperbolic relation modeling, balanced expert routing, and expert diversity, respectively.

The authors evaluate HyTBE under a leave-one-domain-out protocol on three public datasets: NUAA-SIRST (427 images), NUDT-SIRST (1327 images), and IRSTD-1K (1000 images). In each setting, two datasets are used as source domains for training, while the remaining dataset is held out as an unseen target domain without target-domain fine-tuning. This yields three cross-domain settings: NUAA-SIRST + NUDT-SIRST → IRSTD-1K, NUAA-SIRST + IRSTD-1K → NUDT-SIRST, and NUDT-SIRST + IRSTD-1K → NUAA-SIRST. Metrics are mIoU, F-measure, Pd, and Fa. The model is trained from scratch for 200 epochs with batch size 4, Adam optimizer, and initial learning rate 0.001, on images resized to 256×256.

HyTBE is compared with ten representative methods (ALCNet, DNANet, UIUNet, MSHNet, SCTransNet, DRPCANet, PConv, MLPNet, PQGNet, and NS FPN). The results show that "HyTBE achieves the best mIoU and F-measure across all three cross-domain settings, outperforming the corresponding second-best methods by 0.48/0.42, 4.92/3.17, and 5.45/4.15 percentage points on IRSTD-1K, NUAA-SIRST, and NUDT-SIRST, respectively. It also obtains the highest Pd on NUAA-SIRST and NUDT-SIRST. Notably, HyTBE contains only 1.13M parameters with 8.42G FLOPs, demonstrating strong cross-domain performance with a compact parameter footprint and moderate computational cost."

Ablation experiments are conducted on the NUDT-SIRST + IRSTD-1K → NUAA-SIRST setting:

• Main components (Table 2): The baseline achieves 68.93% mIoU. Adding TBRI improves mIoU and F-measure by 2.93 and 2.03 percentage points, respectively. Adding HRM further increases mIoU to 74.48%, while the complete model with HMA reaches 78.58% mIoU, 88.00% F-measure, and 99.56% Pd, verifying the complementary contributions of relation diversification, relation modeling, and relation-guided feature adaptation.

• TBRI branches (Table 3): Target intervention and background intervention provide complementary relation patterns; combining both branches improves mIoU from 71.03% to 78.58%.

• Geometry (Table 4): Hyperbolic modeling consistently outperforms its Euclidean counterpart, particularly reducing Fa from 17.77 to 13.10.

• HMA design (Table 5): Guide-attention and MoE independently improve mIoU and F-measure; their combination achieves the best overall balance.

• Auxiliary objectives (Table 6): Lhyp alone improves mIoU and F-measure by 1.71 and 1.10 percentage points while reducing Fa from 26.20 to 9.51; using all three objectives (Lhyp, Llb, Lnc) achieves the best overall performance.

The supplementary material provides empirical evidence for target-background relation shift. Using descriptors of local saliency, target morphology, background clutter, and imaging style computed from the three datasets, the authors show clear cross-domain differences across all four relation factors via marginal distributions, joint PCA visualization (where the first two components explain 44.2% of variance), and pairwise standardized 1-Wasserstein distances, with the largest average discrepancy for NUAA–IRSTD (0.56), followed by NUDT–IRSTD (0.47) and NUAA–NUDT (0.34).

They further show that relation deviation is associated with source-model degradation: training SCTransNet on IRSTD-1K and NUDT-SIRST and evaluating on unseen NUAA-SIRST, the relation-deviation score is positively correlated with the miss rate (Spearman ρ = 0.25, image-cluster bootstrap 95% CI [0.05, 0.45]), and this correlation remains after controlling for normalized target area. The results indicate that test targets farther from the source-domain target-background relation patterns generally suffer higher detection miss rates.

The supplementary material also details the seven TBRI operators (imaging style, background complexity, saliency attenuation, morphology transformation, brightness enhancement, shrinkage, and target sampling) and hyperparameter analyses. The default configuration uses p = 0.50 for TBRI activation probability, ρ = 0.05 for anchor scale, and E = 4 experts: TBRI performs best at the moderate activation probability p = 0.50, The default scale ρ = 0.05 achieves the best result across all four metrics, and E = 4 achieves the highest mIoU, F-measure, and Pd.

The paper concludes that the three components of HyTBE are closely integrated to expand, represent, and adapt target-background relations for more transferable discrimination. The authors state: Experiments across three unseen target domains demonstrate the superiority of HyTBE, highlighting its potential for robust and practical IRSTD.

Improvements to AI systems based on HyTBE:

• Add target-background relation intervention during training to synthesize diverse background and target variations, enabling the model to observe a wider range of target-background relation patterns and avoid overfitting to source-domain-specific decision boundaries.

• Use hyperbolic relation modeling to represent feature tokens by their geodesic distances to explicit target and background anchors, capturing contrast/relation structure more effectively than Euclidean distance and improving discriminability for small, textureless targets.

• Integrate a hyperbolic-guided Mixture-of-Experts adapter that recalibrates visual features using relation cues and routes each input to specialized experts, allowing implicit adaptation to an unseen domain without target-domain fine-tuning.

• Add auxiliary losses for balanced expert routing and expert diversity, preventing expert collapse and promoting complementary, robust feature transformations.

• Keep the model lightweight by using only 1.13M parameters and 8.42G FLOPs, making the improved system suitable for practical deployment.

What the improved AI system can do:

• Detect infrared small targets in unseen infrared domains with state-of-the-art cross-domain generalization, e.g., training on two datasets and testing on a third without fine-tuning, improving mIoU by 0.48–5.45 percentage points and F-measure by 0.42–4.15 percentage points over the second-best methods.

• Maintain strong detection probability (Pd) and significantly reduce false alarms (Fa) under domain shift.

• Adaptively handle targets that are less salient, morphologically changed, or set against novel backgrounds by explicitly modeling relation shift rather than relying solely on target appearance.

• Preserve performance under target–background ambiguity, including cases where test targets lie farther from source-domain relation patterns and would typically cause higher miss rates.

• Serve as a compact, low-cost backbone for cross-domain small-target detection, suitable for edge or embedded infrared systems.

• Use the same relation-intervention and hyperbolic-adaptation principles to improve other perception tasks facing out-of-distribution contrast/context shift, such as medical lesion detection, maritime object detection, or low-contrast object segmentation.