Lightweight ORSI Salient Object Detection via Frequency and Mutual Assistance Attention

Lightweight Salient Object Detection in Optical Remote Sensing Image (ORSI-SOD) is expected to achieve a good balance between model complexity and detection accuracy. Existing lightweight ORSI-SOD methods usually adopt the MobileNet as the backbone, which greatly reduces the model complexity, but also restricts the detection accuracy. In this paper, we propose a novel lightweightFrequency andMutualAssistance AttentionNetwork,i.e., FreMaNet, with a lightweight transformer backbone for ORSI-SOD. Our FreMaNet is built on the strategy of intra-level modeling and inter-level assistance. Frequency-domain Self-Attention (FreSA) and Mutual Assistance Channel Attention (MaCA) are responsible for intra-level modeling and inter-level assistance, respectively. Specifically, FreSA is arranged behind the backbone to further model global relationships within each level of features (i.e., intra-level features). Different from the vanilla self-attention, FreSA achieves global relationship modeling through multiplication in the frequency domain, resulting in less computational load. Then, different levels of features (i.e., inter-level features) assist and interact with each other in MaCA. MaCA first performs a simple fusion on the features of two adjacent levels, and then adopts parallel self-channel attention and assistance channel attention to adaptively achieve mutual assistance of features at different levels. With the cooperation of the above components and an efficient saliency decoder, our FreMaNet has only 4.91M parameters and 4.52G floating point operations for a 352×352 input. Extensive experiments on three datasets demonstrate that our lightweight FreMaNet achieves competitive performance compared to lightweight and normal-size ORSI-SOD methods. The code and results of our method are available at https://github.com/MathLee/FreMaNet.