Correspondence Calibrating and Dynamic Consistency Learning for Noisy Cross-Modal Retrieval

Cross-modal retrieval has drawn an increasing amount of attention due to its effective ability for searching semantic relative data points with different modalities. In spite of some progress obtained, such methods often require the data pair maintaining the correct cross-modal correspondence in the training process, which is impractical in real application. To tackle this issue, we propose a Correspondence Calibrating and Dynamic Consistency Learning Network (CCDCL), aiming at optimizing the correspondence of positive samples and deeply investigating the consistency of negative samples. Specifically, to effectively alleviate the false positive issues, co-teaching paradigm is introduced to optimize the correspondence of positive samples by calibrating their confidence scores. To address the false negative sample problem, we propose a Vision-Language Semantic Collaborative Dynamic Margin Adaptation (VLC-DMA) method, which integrates unimodal similarities with calibrated confidence to produce the cross-modal semantic similarities, and consequently employs a dynamic margin function for accurately discriminating false and true negative samples. Experimental results demonstrate that the proposed method effectively improves cross-modal retrieval performance across multiple image-text matching datasets. The code will be available on GitHub upon acceptance of this paper.