XFeat Revisited: Reproducibility and Evaluation of a Lightweight Image Matcher

This paper presents a reproducibility study of XFeat, a lightweight local feature extractor and matcher designed for efficient image matching on resource-constrained hardware. The authors re-implement the architecture based on the paper and supplementary material, re-evaluate the released checkpoint, and conduct additional architectural ablations.

The study makes three main contributions: (1) a re-implementation guided by the paper and supplementary material, using official code only when the paper omits implementation-critical details; (2) re-evaluation of the released checkpoint and reproduction of original matching, homography estimation, and visual localization experiments, plus zero-shot out-of-distribution and cross-modal transfer testing; (3) testing whether XFeat benefits from its specific skip-connection and revisiting whether keypoint detection should remain separate from descriptor extraction.

The paper evaluates four claims from the original work:

• Claim 1: XFeat provides competitive matching accuracy at practical CPU inference speeds without hardware-specific optimizations

• Claim 2: Separating keypoint detection from descriptor extraction improves semi-dense matching performance

• Claim 3: XFeat benefits from a single skip-connection, preferable to alternative designs

• Claim 4: XFeat provides competitive results in downstream homography estimation and visual localization

The reproduction is based on the main XFeat paper, supplementary material, and official GitHub repository. The paper and supplement are treated as the primary specification, with released code used to resolve implementation details not described in either source. Training data combines MegaDepth with a 20,000-image subset of COCO 2017 in a 6:4 ratio, with all images resized to 800×600.

The model architecture consists of a six-block convolutional backbone with a skip-connection, a descriptor branch combining outputs from the final three backbone blocks, a reliability head, and a separate keypoint detection branch operating directly on the grayscale input. The paper notes that "the paper, supplementary material, and released code describe most of the model, but they disagree on several implementation details. For example, the backbone layout and fusion block described in the paper and supplement differ from the released model.py, while the implemented training losses do not fully match those presented in the paper."

The central speed-accuracy trade-off is recovered. On Apple M1 Pro, XFeat reaches 11.8 FPS and Acc@10° of 76.2, making it the fastest learned method while remaining competitive in pose accuracy. XFeat* increases Acc@10° to 86.2 at 6.3 FPS. The paper states: Irrespective of the platform, XFeat provides the highest throughput among the evaluated learned methods, while XFeat* improves pose accuracy at a lower but still comparatively high throughput.

For MegaDepth-1500, the reproduced sparse model consistently outperforms the re-evaluated checkpoint on AUC@10°, AUC@20°, and Acc@10°, with the reproduced model achieving AUC@10° of 57.9 compared to 57.1 for the re-evaluated checkpoint. The reproduced sparse model also yields a higher mean inlier ratio (0.65 vs 0.56) and more inliers (1079 vs 916). For XFeat*, reproduced and re-evaluated models remain closely aligned.

For ScanNet-1500, the reproduced sparse model closely matches original results and performs slightly better than the re-evaluated checkpoint. The difference is more pronounced for XFeat*, where the reproduced model increases AUC@10° from 34.7 (original) and 33.2 (re-evaluated) to 37.1.

The ablations test three implementation choices: whether keypoint detection should remain separate from descriptor extraction, whether XFeat benefits from its single skip-connection, and whether the loss formulation in the paper differs meaningfully from the released code.

Separate keypoint branch: Coupling the detector to the backbone modestly reduces sparse mode performance (AUC@10° from 57.9 to 55.4 on MegaDepth, from 30.8 to 25.6 on ScanNet). For XFeat*, the decrease is more substantial: AUC@10° drops from 65.7 to 53.8 on MegaDepth and from 37.1 to 31.1 on ScanNet. The paper concludes: These results support the use of a separate keypoint branch, particularly for semi-dense matching, but show that its benefit is less consistent than suggested by the original ablation.

Skip-connection: Removing the original skip-connection reduces performance on ScanNet but not MegaDepth for sparse XFeat. The paper states: "The small and inconsistent differences suggest that skip-connections do not play a major role in XFeat's performance. One possible explanation is that XFeat's shallow architecture enables efficient gradient flow without requiring skip-connections. Regarding the specific design, ResNet Skips obtain the highest mean performance on both datasets in semi-dense mode, leading to the conclusion: We therefore find limited evidence that the specific single skip-connection is essential or preferable to the alternatives tested."

Loss formulation: The results do not show a consistent advantage for either the paper-defined or code-defined objective. On MegaDepth, Code Loss improves pose accuracy metrics but reduces MIR and inliers. On ScanNet, Code Loss performs below default for sparse XFeat but slightly improves AUC@10° for XFeat*. The paper concludes: Since the direction of the effect changes across datasets, we do not find evidence that either loss formulation is preferable.

Homography estimation on HPatches: Results broadly reproduce XFeat's performance. For sparse XFeat, released and reproduced models perform similarly on illumination sequences, while reproduced checkpoints show lower and more variable performance under viewpoint changes. The reproduced XFeat* improves over the reproduced sparse model across all reported thresholds, with the best overall MHA@7 score of 88.0 on viewpoint sequences.

Visual localization on Aachen Day-Night: This is the only original downstream evaluation not recovered. Both the released checkpoint and reproduced model fall below published values at every threshold. The paper states: "This suggests that the gap comes from differences in the HLoc configuration or other missing evaluation details rather than from the reproduced model itself. We therefore consider the Aachen results only partially reproducible."

RUBIK benchmark: Both XFeat variants generalize less effectively than stronger baselines. Sparse XFeat reaches success rates of 14.5% (re-evaluated) and 15.1% (reproduced), while XFeat* reaches 14.9% and 16.4%. These remain well below methods paired with LightGlue, which achieve approximately 30-37%.

FIRE retinal registration: XFeat transfers reasonably well when image pairs retain sufficient overlap. The reproduced sparse model reaches AUC of 94.14% on easy, 75.93% on medium, and 46.19% on hard subsets. The paper notes: This suggests that XFeat can transfer from natural images to retinal data, but struggles with pairs that have reduced overlap and larger anatomical variation.

MTV thermal-visible: XFeat transfers poorly. The reproduced sparse model achieves matching precision of 9.01% and AUC@20° of 1.44%, substantially below LoFTR and QuadTreeAttention baselines. The paper states: When descriptor matching fails across modalities, adding more semi-dense correspondences mainly introduces additional noise and does not improve pose estimation.

SRIF remote sensing: XFeat performs below stronger matchers. The reproduced sparse model reaches a mean error of 90.7 pixels with 11.7% and 18.4% of points within 5 and 10 pixels. Performance is best on optical-optical pairs (S@10 of 40.17% for reproduced XFeat*), drops on optical-infrared (12.86%), and is lowest on optical-SAR (below 2%). The paper concludes: "This suggests that XFeat generalizes poorly to out-of-distribution aerial imagery and becomes unreliable when the sensing modality changes, particularly for optical-SAR pairs, making it unsuitable for zero-shot multimodal remote-sensing registration without further adaptation."

• Claim 1 (accuracy-efficiency trade-off): Supported. In the CPU benchmark, sparse XFeat is substantially faster than the other learned methods while maintaining competitive relative pose accuracy.

• Claim 2 (separate keypoint branch): Supported mainly for semi-dense matching. "The separate branch therefore appears important when detected locations require refinement and when semi-dense matching must retain and refine a larger set of less certain candidate points, although its benefit relative to sparse matching is less consistent across datasets and matching modes than originally claimed."

• Claim 3 (skip-connection): Limited support. The original skip-connection does not appear to provide a clear or consistent benefit over the alternatives, and is less influential than suggested in the original paper.

• Claim 4 (downstream tasks): Supported for homography estimation, only partially for visual localization.

The paper concludes: This reproducibility study confirms XFeat's central message that a lightweight local feature model can achieve a strong accuracy-efficiency trade-off without specialized hardware-specific optimizations. The separate keypoint branch is most beneficial for semi-dense matching, while the original skip-connection has a smaller and less consistent effect than suggested. Homography estimation is reproduced closely, whereas Aachen visual localization depends strongly on unspecified evaluation details. XFeat transfers reasonably under moderate domain shifts but degrades under severe appearance and modality changes.

Improvements to AI Systems:

1. Adaptive keypoint-detection routing: Implement a dynamic mechanism that switches between separate keypoint branches and coupled detectors based on task difficulty. The system can use a lightweight classifier to detect domain shift severity—using the separate branch for semi-dense matching under moderate shift, and falling back to coupled detection for sparse matching in low-resource or time-critical scenarios. This improves accuracy-efficiency trade-offs across heterogeneous inputs.

2. Skip-connection pruning with gradient-flow monitoring: Replace the fixed single skip-connection with an automated architecture search that monitors gradient flow during training. The system can add or remove skip-connections per layer based on measured gradient variance, avoiding unnecessary connections in shallow networks while retaining them where they demonstrably improve convergence. This reduces model complexity without sacrificing accuracy on datasets like ScanNet.

3. Loss-function selection via cross-validation: Build a meta-learner that selects between paper-defined and code-defined loss formulations based on dataset characteristics (e.g., overlap ratio, texture richness). The system can pre-train both loss variants on a small validation subset, then choose the one with better pose accuracy and inlier ratio for the full training. This prevents dataset-specific performance degradation, as seen in the inconsistent effects on MegaDepth vs. ScanNet.

4. Domain-shift-aware matching confidence scoring: Enhance the reliability head to output a calibrated confidence score that reflects expected matching quality under modality changes. The system can detect when inputs are cross-modal (e.g., thermal-visible or optical-SAR) and automatically down-weight unreliable correspondences, preventing noise injection into pose estimation. This addresses the observed failure where semi-dense correspondences added noise rather than signal in cross-modal transfer.

5. Zero-shot domain-adaptation layer: Add a lightweight feature normalization module that aligns feature distributions across modalities without retraining. The system can compute per-channel statistics from a small set of target-domain samples and apply affine transformations to the descriptor branch, improving transfer to retinal imaging or remote sensing while maintaining performance on natural images. This mitigates the degradation seen in optical-SAR pairs (S@10 below 2%).

6. Hierarchical matching with fallback strategies: Implement a cascade that first attempts sparse matching; if inlier ratio falls below a threshold (e.g., 0.3), it switches to semi-dense matching with the separate keypoint branch. If cross-modal cues are detected, it falls back to a classical feature (e.g., SIFT) or a domain-adapted variant. This ensures robust performance across the full spectrum of domain shifts, from easy (HPatches illumination) to hard (Aachen night-time or MTV thermal-visible).

What the Improved AI System Can Do:

• Achieve state-of-the-art speed-accuracy trade-offs on CPU (e.g., 11.8 FPS with 76.2% Acc@10°) while automatically adapting to dataset complexity.

• Maintain high pose accuracy (AUC@10° > 57.9 on MegaDepth) across diverse scenes without manual hyperparameter tuning for loss or architecture.

• Gracefully degrade under domain shift: retain >90% AUC on retinal data with moderate overlap, while avoiding catastrophic failure on cross-modal pairs by suppressing unreliable matches.

• Provide calibrated confidence scores that allow downstream systems (e.g., SLAM, visual localization) to know when to trust the matcher and when to invoke alternative methods.

• Reduce deployment complexity by pruning unnecessary skip-connections, yielding smaller models that are easier to deploy on resource-constrained hardware without accuracy loss on most benchmarks.

We present a reproducibility study of XFeat, a lightweight local feature extractor and matcher designed to identify corresponding points across images efficiently on resource-constrained hardware. We re-implement the architecture based on the paper and supplementary material, re-evaluate the authors' released checkpoint alongside our re-implementation, and conduct additional architectural ablations to examine design choices that were not fully justified in the original work. This distinction between re-evaluation and reproduction is important, as the paper, supplement, and public code differ in several implementation details, including the backbone layout, fusion block, and training losses. Empirically, our reproduced models closely match and, in some cases, outperform the re-evaluated original checkpoint on MegaDepth-1500 and ScanNet-1500, supporting the main claim that XFeat provides a strong accuracy-efficiency trade-off for standard image-matching benchmarks. Our ablations provide a more nuanced view of two architectural arguments from the original paper. In particular, the parallel keypoint branch is important for semi-dense matching, but its benefit is less pronounced than originally claimed, while the evidence for the specific placement of the single skip-connection remains inconclusive. Finally, we reproduce the original downstream evaluations and find close agreement for homography estimation, while Aachen visual localization remains below the reported results, even for the released checkpoint, suggesting sensitivity to underspecified evaluation details. We then extend the analysis to zero-shot out-of-distribution and cross-modal matching across retinal, thermal-visible, and multimodal remote-sensing imagery, where XFeat remains effective in some settings but degrades sharply under severe modality shifts.