BLMRS: A Balanced Learning Method for Multimodal Remote Sensing Image Classification

Multimodal remote sensing image classification provides essential support for fine-grained understanding of the Earth’s surface by fusing complementary information. Yet, multimodal fusion often suffers from a modal imbalance problem, where applying a uniform optimization objective leads to the under-optimization of certain modalities, posing a bottleneck to realizing the full benefits of fusion. Existing balanced learning strategies attempt to solve this problem by emphasizing the training of encoders for weaker modalities. However, in remote sensing scenarios, they overlook the domain-specific reality that certain weaker modalities inherently lack discriminative information; for example, the LiDAR-derived Digital Surface Model (DSM), which does not provide the necessary detail to effectively distinguish between land cover types. Indiscriminately emphasizing the learning of such modalities may introduce noise and impair the model’s overall performance. To address this, this work proposes a Balanced Learning Method for Multimodal Remote Sensing image classification (BLMRS). The method takes unimodal training in a non-competitive environment as a reference, and uses its output as the guidance signals to dynamically adjust the optimization preference in multimodal learning. This allows adaptive optimization for weaker modalities, avoiding overemphasis when their discriminative information is limited. To further avoid conflicts among guidance signals from different unimodal sources, we measure and filter out components inconsistent with the weaker modality’s guidance, ensuring the training focus remains on it. BLMRS is architecture-agnostic and can be seamlessly integrated into existing classification backbones. Extensive experiments on four public benchmarks and six backbone architectures demonstrate that our method consistently offers performance gains. It remains robust even when a modality’s discriminative information is degraded.