RSV-LoRA: Enhancing Object Segmentation in Remote Sensing Videos via Fine-Tuning With LoRA-Based Mixture of Experts

With the widespread application of high-resolution remote sensing videos (RSV), object segmentation in RSV plays an irreplaceable role in various fields such as surface target recognition. However, existing methods struggle with this task due to the small object size, high object density, and significant scale variation among objects. To address these challenges, we propose RSV-LoRA, a novel approach that fine-tunes Segment Anything Model (SAM) to effectively capture and segment objects in remote sensing videos. First, RSV-LoRA integrates a Mixture of Experts into the LoRA framework, alleviating the limitations of fixed network structures in recognizing objects at varying scales. Subsequently, RSV-LoRA employs multi-scale dilated convolutions as experts while maintaining the inherent efficiency of low-rank adaptation. This introduces enhanced local feature extraction capability, thereby improving the recognition performance for objects in RSV. Moreover, RSV-LoRA is efficient in terms of training as only the RSV-LoRA layers are trained while the weights of SAM remain frozen, thereby reducing the number of trainable parameters. Owing to such training characteristics, RSV-LoRA is plug-and-play, allowing its layers to be adapted to various tasks and objects. Extensive experiments conducted on the SAT-MTB, iSAID, and VHR-10 datasets demonstrate the effectiveness of RSV-LoRA.