MSAM: Multi-Semantic Adaptive Mining for Cross-Modal Drone Video-Text Retrieval

With the advancement of drone technology, the volume of video data increases rapidly, creating an urgent need for efficient semantic retrieval. This work presents one of the early studies on the drone video-text retrieval (DVTR) task. Drone videos feature overhead perspectives, strong structural homogeneity, and diverse semantic expressions of target combinations, which challenge existing cross-modal methods designed for ground-level views in effectively modeling their characteristics. Therefore, dedicated retrieval mechanisms tailored for drone scenarios are necessary. To address this issue, we propose a novel approach called Multi-Semantic Adaptive Mining (MSAM). MSAM introduces a multi-semantic adaptive learning mechanism that models dynamic changes across frames and extracts region-level semantic information. This design improves fine-grained cross-modal alignment. This method relies on fine-grained interactions between textual tokens and drone video frames. It integrates an adaptive semantic construction module, a distribution-driven semantic learning term, and a diversity-aware semantic constraint to enhance cross-modal interactions and improve the robustness of feature representations. To reduce the interference of complex backgrounds in drone videos, we introduce a cross-modal interactive feature fusion pooling mechanism that focuses on feature extraction and matching in target regions, minimizing noise effects. Extensive experiments on two self-constructed drone video-text datasets show that MSAM outperforms other existing methods in the DVTR task. The code and data are accessible at https://github.com/huangjh98/MSAM.