ScaleRSNet: Advancing Remote Sensing Image Segmentation With Multi-Scale Contextual Attention Mechanisms

Remote sensing image semantic segmentation faces critical challenges including extreme object scale variations, severe class imbalance, spectral similarity, and background noise. These challenges often lead to inaccurate boundary delineation and misclassification of small objects. To address these limitations, ScaleRSNet is proposed, a novel deep learning architecture designed to handle multi-scale objects through a hierarchical multi-scale feature learning framework. ScaleRSNet integrates a hierarchical encoder, multi-scale contextual aggregation, spatial feature transformation, and a cross-layer fusion decoder. The hierarchical encoder employs dilated convolutions with progressively increasing dilation rates to capture fine-grained details and broader context across multiple scales. Spatial attention mechanisms selectively enhance discriminative features and refine segmentation boundaries for small or fragmented objects. Experimental results demonstrate that ScaleRSNet achieves 69.62% mPA and 82.16% accuracy on the WHDLD dataset, representing improvements of 19.09% in mPA and 10.88% in accuracy compared to the baseline UNet. On the EarthVQA dataset, ScaleRSNet achieves 45.65% mPA and 60.02% accuracy, representing improvements of 20.15% in mPA and 22.24% in accuracy compared to UNet. Grad-CAM visualizations demonstrate the model’s ability to focus on relevant areas during inference, providing insights into its decision-making process. These results confirm that ScaleRSNet offers a robust solution for remote sensing image segmentation, delivering superior performance in diverse environments.