TCF-VQGAN: Two-Stage Codebook Fusion Vector-Quantized GAN for Multimodal Remote Sensing Image Cloud Removal

With the advancement of remote sensing technology, image acquisition has become more convenient and the amount of information captured has significantly increased, playing a vital role in numerous fields. However, cloud cover often results in missing image data, severely affecting data usability. In recent years, although deep learning methods have made progress in cloud removal tasks, the complexity of modeling multispectral band relationships and the scarcity of paired data remain major challenges. To address this, this paper proposes a two-stage codebook fusion vector-quantized generative adversarial network (TCF-VQ GAN) and a training framework. The first stage employs synthetic aperture radar (SAR), MODIS, and cloud-free data for unsupervised training; the second stage performs fusion fine-tuning using SAR and MODIS on paired cloudy/cloud-free data. The model incorporates a space-channel jointed gated convolution (SCGC) module to model irregular cloud cover and combines channel attention for band selection, while a dynamically weighted wavelet alignment loss function (DW2A) is designed to enhance multiscale feature representation. Experiments on the SEN12MS-CR and SMILE-CR datasets demonstrate that the proposed method outperforms existing methods across all metrics: on SEN12MS-CR, PSNR is 31.0397 and SAM is 4.7243; they are 33.5191 and 2.1663, respectively, on SMILE-CR. Furthermore, under fixed paired data conditions, simply adding auxiliary and cloud-free data further improves performance, validating the method’s effectiveness in data-scarce scenarios.