MovSeg: Efficient Adaptation of Vision–Language Models for Multispectral Open- Vocabulary Segmentation

Open-vocabulary segmentation (OVS) allows models to segment any categories based on text prompts, overcoming the limitations of traditional closed-set methods. Open-vocabulary segmentation (OVS) in remote sensing is limited by its reliance on standard RGB images. This prevents models from using valuable information from other spectral bands, such as Near-Infrared (NIR). This paper proposes MovSeg, a framework for multispectral open-vocabulary segmentation. Our method efficiently integrates 4-band (RGB+NIR) data into a pre-trained vision-language model. MovSeg introduces two components: a Multispectral Input Adaptation (MIA) module and a Spatial-Channel Adaptive Tuning (SCAT) strategy. The MIA module adapts the input layer to process 4-band data. It uses a weight-copying strategy for initialization and a WindowECA mechanism to re-weight spectral channels. The SCAT strategy is a parameter-efficient strategy that uses a hybrid of LoRA and Adapters to fine-tune deep features with low computational cost. Experiments on three multispectral datasets show that MovSeg outperforms existing general-domain and remote sensing OVS methods. The model achieves significant gains on NIR-sensitive classes, confirming its ability to exploit the extra spectral data. The codes will be coming soon.