DFCap: Zero-Shot Image Captioning With Dynamic Fusion Module

Image captioning is a task that generates natural language descriptions for visual content and connects research in computer vision with natural language processing.While supervised methods have achieved significant success, they rely heavily on large-scale annotated datasets, limiting their applicability in low-resource scenarios. Zero-shot image captioning has emerged as a promising alternative by leveraging pre-trained vision-language models and text-only training to generalize to unseen images. However, existing approaches often suffer from the modality gap between visual and textual representations, leading to inaccurate or hallucinated content. One possible solution is to introduce retrieval-based strategies that incorporate external textual knowledge, but their performance is highly sensitive to the quality of retrieved samples. To address these challenges, we propose DFCap, a novel zero-shot image captioning framework with a dynamic fusion module. This module adaptively integrates visual features, textual features, and external retrieved knowledge, effectively reducing retrieved noise and enhancing caption quality. The proposed model achieved a BLEU-4 score of 30.4, METEOR score of 27.0, a CIDEr score of 108.0, and a SPICE score of 20.7 on MSCOCO. On Flickr30k, it achieved a BLEU-4 score of 23.2, a METEOR score of 22.1, a CIDEr score of 57.3, and a SPICE score of 15.5. Extensive experiments on MSCOCO and Flickr30k demonstrate that DFCap achieves competitive performance, validating its effectiveness and robustness in generating highquality captions without task-specific training.