Reliability-Aware Cross-Modal Fusion of Sentinel-1 SAR and Sentinel-2 Optical Imagery for Robust Patch-Level Land Cover Scene Classification

Read full text
Authors: Srinivas Vulli, Ming Liu, Zhenyu Huang
Cited by: 1
References: 16

Abstract

Fusing Sentinel-1 Synthetic Aperture Radar (SAR) and Sentinel-2 optical imagery is challenging when one modality is degraded. This study proposes a reliability-aware cross-modal fusion framework for patch-level land cover scene classification, combining modality-specific encoders, per-modality classification heads, and a noise-aware reliability gate that estimates adaptive fusion weights from feature representations, prediction confidence, and optical quality indicators. Sentinel-2 inputs were synthetically degraded under five controlled scenarios-Gaussian noise, heavy Gaussian noise, channel drop, cloud-like masking, and mixed degradation and six approaches were compared: SAR-only, optical-only, early fusion, feature-level fusion, logit-level fusion, and the proposed method. Experiments on a four-class SEN12MS-derived dataset (16,000 paired patches) were run under deterministic settings and reported over five random seeds. The proposed model achieved the highest or statistically comparable mean accuracy under clean, channel-drop, and cloud-mask conditions and a clear advantage under Gaussian noise (0.922 ± 0.051, versus 0.651 ± 0.201 for logit-level fusion and 0.293 ± 0.096 for optical-only) and remained the highest or statistically comparable method on clean inputs. Heavy noise remained the most challenging and seed-sensitive condition (0.604 ± 0.236 across seeds), although the representative run retained 86.3% accuracy, and a scene-held-out evaluation showed that the noise-robustness advantage persists under spatially independent testing. These results position reliability-aware fusion as a robustness-oriented strategy supported by multi-seed statistics and explicit failure-mode analysis.

Fields of Study & Subjects

Remote Sensing in Agriculture Remote-Sensing Image Classification Synthetic Aperture Radar (SAR) Applications and Techniques

Publication details

Journal / source
Remote Sensing
Publisher
MDPI AG
Volume
18
Issue
15
Pages
2484–2484
Publication date
30 July 2026
Language
EN