Balancing Shared and Task-Specific Representations: A Hybrid Approach to Depth-Aware Video Panoptic Segmentation
Multiformer introduces a novel hybrid approach for depth-aware video panoptic segmentation by learning shared object representations across three tasks: segmentation, monocular depth estimation, and tracking. It uses task-specific branches within decoder blocks that fuse into a shared representation at block interfaces. Experiments show it achieves state-of-the-art results, significantly outperforming previous methods on all DVPS metrics.