Improving text–image alignment for referring remote sensing image segmentation guided by vision foundation model

The task of referring remote sensing image segmentation (RRSIS) aims to predict segmentation masks of target objects based on text descriptions. Effective alignment between textual and visual modalities is crucial for achieving high performance. However, most existing RRSIS methods overlook the inherent domain gap between the two modalities and naively fuse visual and text features. Moreover, they lack guidance from domain-specific priors during alignment, often leading to inaccurate segmentation results. In this paper, we propose a novel method for RRSIS, namely TIANet. This method combines implicit and explicit alignment approaches to achieve fine-grained image-text alignment. Furthermore, to compensate for the lack of domain priors during alignment, we leverage the Remote Sensing Vision Foundation Model (RS-VFM) to guide both alignment strategies. On one hand, the Prior-Guided Visual-Text Feature Alignment Module (PVTAM) leverages domain priors to fuse textual and visual features in a stage-wise manner, thereby achieving cross-modal implicit alignment at the feature level. On the other hand, the Dynamic Bidirectional Alignment and Prior Distillation (DBAPD) loss is adopted to implement explicit cross-modal alignment during training. Furthermore, to enhance the perception ability of small targets, the Kernel-Adaptive Small Target Enhancement Module (KASTEM) is used in the decoder stage. Extensive experiments on the RefSegRS and RRSIS-D datasets show that our TIANet achieves excellent performance compared with other advanced methods, with a 1.22% mIoU improvement over existing state-of-the-art approaches on the RefSegRS dataset.