Local-Contextual Feature Fusion Network Based on Nonlinear Spiking Neural Model for Semantic Segmentation of Remote Sensing Images

Semantic segmentation of remote sensing (RS) image is crucial to tasks such as geographic research, land monitoring and environmental protection. In recent years, deep learning models built with convolutional neural networks (CNNs) and Transformer structures have proven effective in semantic segmentation task of RS images. With the increase in resolution of RS images in complex scenes and the complexity of high-resolution urban images, each object has rich textures and edges, and the distribution of objects is extremely irregular. To resolve above challenges, a semantic segmentation network for RS images is proposed, which employs an encoder-decoder structure, where four ResNet-18 blocks act as encoders, and four specially designed local-contextual Transformer blocks form the decoder. In order to effectively utilize local contextual features, a channel attention-feature fusion module using a novel nonlinear spiking neuron model is designed to assist the decoder in better feature recovery. The experimental results demonstrate that the proposed method is feasible and effective for semantic segmentation of RS images. Specifically, the suboptimal 86.42% and optimal 82.25% mIoU are achieved on Potsdam and Vaihingen datasets, respectively, and the best 52.4% and the near-optimal 65.3% mIoU on the LoveDA and UAVid datasets, respectively, for the proposed model.