Disturbance-Robust Remote Sensing Change Captioning with Self-supervised Multifrequency Representation

Remote sensing change captioning aims at revealing the changing regions between multitemporal images with human-interpretable text. However, the acquisition process of the remote sensing images often encounters unavoidable influences (e.g., rotation or scale variation), which may lead to catastrophic semantic reversal in text generation tasks that heavily rely on global context understanding. In this paper, we explore a self-supervised multifrequency representation for bitemporal remote sensing change captioning, enhancing the representation capability in the scenario with rotation disturbance. The input samples are first disentangled by frequency dimension to obtain the foreground and background features by a pretrained multimodal encoder. Afterward, cross-temporal attention is performed to capture the foreground changed patterns. Meanwhile, a self-supervised correlation constraint is designed for low-frequency background tokens to establish implicit alignment under rotated scenarios. Finally, the transformer-based decoder is utilized to project the differential visual tokens into textual domains to generate the output captions. Extensive experiments on the several created datasets with rotation disturbance demonstrate the effectiveness of our proposed method.