TAMC-CD: A terrain-aware multimodal change detection framework with self-supervised spatio-temporal transformer

Detecting land cover change in mountainous areas is difficult due to rugged terrain, changes associated with season, and an absence of labeled data. In this work, we propose a new land cover change detection framework called Terrain-Aware Multimodal Change Detection (TAMC-CD) that combines Sentinel-1 Synthetic Aperture Radar, Sentinel-2 optical, and SRTM DEM data that can provide change detection of land cover through three years (2018, 2021, 2024) in the Almora district Uttarakhand, India. Our approach incorporates a dual encoder multimodal network, a custom Swin-Transformer-based spatiotemporal transformer, and a terrain-aware graph neural network to jointly leverage structural, topographical, and spectral information. The self-supervised contrastive pretraining approach makes it possible to learn representations from unlabeled samples, and uncertainty-guided active learning can mitigate labeling by focusing on the informative regions. We demonstrate that TAMC-CD outperforms state-of-the-art methods achieving an overall accuracy measured at 92.8%, F1 measured at 0.90, and Kappa at 0.88. The model performs well in mitigating misclassification due to terrain, especially in shadow and steep regions, as well as providing expected calibration error (ECE = 0.042) that can assist human decision making. Deforestation (i.e. forests to farms = 3.4% of transitions) and urbanization (i.e. farms to cities = 2.8% of transitions) represent two of the most important transitions. These point to persistent human pressure on Himalayan ecosystems. The results also suggest that the TAMC-CD framework is a scalable, explainable, and label-efficient way of detecting multimodal variation across difficult hilly landscapes, which can inform sustainable land use practices and regional planning in areas of limited data.