MultiSight: A Vision-Language Model for Collaborative Understanding of Multiview Remote Sensing Images

Traditional remote sensing (RS) interpretation paradigms rely on single-view imagery, but limited perspective often leads to suboptimal performance. With advances in Unmanned aerial vehicles (UAV) technology, acquiring multi-view images has become more accessible, making it feasible to leverage cross-view complementarity for enhanced interpretation. To this end, we propose MultiSight, a vision-language model (VLM) designed for collaborative understanding of multi-view RS images. MultiSight integrates features from multiple views and incorporates task-specific textual prompts through an Adaptive Multi-Image and Text Encoding Fusion (AMITEF) module, enabling joint reasoning across diverse inputs. Additionally, we introduce a Multi-View Attention (MVA) module that effectively captures inter-view feature correlations, enhancing the model's ability to perform consistent and coherent interpretation across varying perspectives. We evaluate our model on three downstream tasks that heavily rely on multi-view information: fine-grained vehicle recognition, fine-grained ship recognition, and building localization. For each task, we construct corresponding multi-view datasets: MultiSight-Vehicle, MultiSight-Ship, and MultiSight-Building to thoroughly assess the effectiveness of MultiSight. Experimental results show that MultiSight outperforms existing single-image and multi-image VLMs, highlighting its robust capabilities in multi-view remote sensing interpretation and semantic reasoning. Our dataset, along with the data construction scripts and model evaluation code, will be publicly released athttps://github.com/lostwolves/MultiSight.