FuseAdapt: Parameter-Efficient Multimodal Fusion for Semantic Segmentation with Missing Modalities
FuseAdapt is a parameter-efficient framework that fuses multi-modal data within a frozen vision foundation model's adaptation space. It uses lightweight, modality-specific branches and FFN low-rank residuals for cross-modal exchange. It also introduces Predictive Modality Modeling (PMM) to handle missing sensors during inference by predicting the missing modality's embedding based on available ones.