Multimodal regularization network with visual-text prompt for few-shot image classification

Few-shot image classification aims to achieve accurate categorization with limited labeled examples, leveraging auxiliary knowledge to compensate for the scarcity of supervision information. However, current few-shot learning approaches often exhibit limitations in effectively harnessing the synergistic potential of multimodal cues. Furthermore, existing methods frequently fail to model the inherent complementarity and regularization relation between visual and linguistic modalities. Based on these considerations, we propose a framework termed multimodal regularization network with visual-text prompts (MRN-VTP) for few-shot image classification. Specifically, MRN-VTP introduces a visual prompt and frequency-domain combination strategy to capture more diverse feature representations from a few samples. This strategy effectively integrates visual prompt information with frequency-domain features, thereby enhancing the model’s comprehension of both global features and local details. To further explore cross-modal interactions, we design a multimodal regularization module, which imposes consistency constraints between visual and textual modality features to ensure synergistic multimodal collaboration. Extensive experiments conducted on multiple benchmark datasets demonstrate the superior efficacy and robustness of the proposed MRN-VTP method compared with state-of-the-art approaches.