From Pixels to Images: A Structural Survey of Deep Learning Paradigms in Remote Sensing Image Semantic Segmentation
cs.CV
Submitted: 2025-05-21
Updated: 2026-09-15
Comments: 35 pages, 10 figures, 6 tables
Journal ref: Q. Liu, T. Huang, J. Yang, and W. Xiang, "From Pixels to Images: A Structural Survey of Deep Learning Paradigms in Remote Sensing Image Semantic Segmentation," IEEE Access, vol.14, pp.125360-125394, 2026
DOI: 10.1109/ACCESS.2026.3721042
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Remote sensing images (RSIs) capture both natural and human-induced changes on the Earth's surface.
Terminology
Abstract
Remote sensing images (RSIs) capture both natural and human-induced changes on the Earth's surface. Semantic segmentation (SS) of RSIs enables the fine-grained interpretation of surface features, making it a critical task in RS analysis. With the increasing diversity and volume of RSIs collected by sensors on various platforms, traditional processing methods struggle to maintain efficiency and accuracy. In response, deep learning (DL) has emerged as a transformative approach, enabling substantial advances in remote sensing image semantic segmentation (RSISS). As researchers continue to explore end-to-end SS, DL-based RSISS has undergone a structural evolution from pixel-level and patch-based classification to tile-level and image-level segmentation. However, existing reviews often focus on individual components, such as supervision strategies or fusion stages, and lack a unified operational perspective aligned with segmentation granularity and the training/inference pipeline. This paper provides a comprehensive review by organizing DL-based RSISS into a pixel-patch-tile-image hierarchy, covering early pixel-based methods, prevailing patch-based and tile-based techniques, and emerging image-based approaches. Specifically, the survey analyzes four supervision strategies, eleven feature extraction strategies, and six information fusion strategies, revealing the field's progression from local to global feature extraction, from traditional DL architectures to foundation models, and from unimodal to multimodal segmentation. This review offers a holistic and structured understanding of DL-based RSISS, highlighting representative datasets, comparative insights, and open challenges related to data scale, model efficiency, domain robustness, and multimodal integration. Furthermore, to facilitate reproducible research, curated code collections are provided at: https://github.com/quanweiliu/RSISS.
Sources
- Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagery
- CM-UNet: Hybrid CNN-Mamba UNet for Remote Sensing Image Semantic Segmentation
- A Universal Knowledge Embedded Contrastive Learning Framework for Hyperspectral Image Classification
- S4DL: Shift-sensitive Spatial-Spectral Disentangling Learning for Hyperspectral Image Unsupervised Domain Adaptation
- HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model
- SpectralX: Parameter-efficient Domain Generalization for Spectral Remote Sensing Foundation Models
- CrossEarth: Geospatial Vision Foundation Model for Domain Generalizable Remote Sensing Semantic Segmentation
- Incomplete Multimodal Learning for Remote Sensing Data Fusion
- MoSAiC: Multi-Modal Multi-Label Supervision-Aware Contrastive Learning for Remote Sensing
- Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentation
- SegEarth-R1: Geospatial Pixel Reasoning via Large Language Model
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models