Enhancing UAV Geo-Location With Multimodal Transformer Networks: The MMGLT Approach

Heterogeneous data from diverse sensor platforms can complement each other, providing more comprehensive and richer information for the analysis of complex scenarios. To effectively utilise this information for UAV geo-location tasks, accurate feature extraction and fusion methods are crucial. To address this challenge, we propose the Multi-Modal Geo Location Transformer Network (MMGLT), leveraging attention mechanisms to fuse cross-modal features. Unlike traditional dual modal fusion frameworks integrating only drone downward-view and satellite images, MMGLT innovatively uses a triple-input structure to process RGB images, LiDAR-derived auxiliary data, and satellite images. It also designs a tailored cross-attention mechanism to link visual semantics with remote sensing features f inely, enabling more robust multi-source feature learning. Subsequently, MMGLT employs a Global Attention Upsampling Head that refines features layer by layer via attention and adaptive up sampling; by combining dynamic feature re-allocation with multi scale pyramid fusion, it mitigates feature degradation and boosts representational power. With the aim of facilitating subsequent applications, we have successfully deployed the MMGLT model on the NVIDIA Jetson Orin NX compact computer. To evaluate the performance of MMGLT in the task, this paper constructs the CrossView-Van dataset. To the best of our knowledge, this is the first end-to-end geo-location dataset that integrates RGB images, high-precision remote sensing feature data, and satellite images. On this new dataset, MMGLT significantly outperforms existing single-modal methods, and achieves improvement in low-texture forest regions where traditional approaches underperform.