JOPP-3D: Joint Open Vocabulary Semantic Segmentation on Point Clouds and Panoramas
JOPP-3D creates a unified framework to understand scenes using both panoramic images and 3D point clouds via natural language queries. It converts panoramas into 3D representations, extracts semantic features from both modalities, and projects the resulting 3D understanding back onto the original panoramic image at full resolution. This allows for open-vocabulary scene segmentation.