Smart Fusion for Adaptive Visual-IMU Weighting in cuVSLAM Sparse Bundle Adjustment

In cuVSLAM, visual and inertial measurement unit (IMU) residuals are fused in sparse bundle adjustment (SBA) with a weighting fixed by pre-calibrated noise covariances, leaving no mechanism to adapt the visual-inertial balance at runtime. We introduce a scalar α at least1 that upscales the visual information matrix relative to the IMU preintegration constraint. Across the candidate set 1.0,2.0,5.0,10.0,20.0, the resulting trajectories form two empirical performance regimes. The IMU-TRUST regime contains α in1,2 near the calibrated baseline, while the VISION-TRUST regime contains α in10,20. The intermediate value α=5 is treated as a transition case. Differences within each regime are small relative to those observed between the regimes. This structure motivates reducing α-selection to a binary decision. We train a lightweight Visual Quality Estimator (VQE), implemented as a multilayer perceptron with 914 trainable parameters. It maps a 10-dimensional visual quality feature vector assembled from statistics already maintained by the tracker to a per-SBA regime prediction. The proposed method reduces mean RMSE APE by 27.7% and by 14.9% under leave-one-out cross-validation (LOOCV) on EuRoC MAV and TUM-VI Room, respectively.