Multimodal Fake News Detection in Bilingual Media: A Vision-Language Fusion Approach Using Task-Specific CNNs and Multilingual Transformers

Misinformation on social media increasingly exploits visually persuasive thumbnail-style content, yet multimodal fake news detection remains underexplored for low-resource languages. This paper presents a bilingual multimodal framework for fake news detection using textual and visual information extracted from Bangla and English social media thumbnails collected from YouTube, Facebook, and Instagram. A manually annotated dataset of 19,890 thumbnail images with embedded textual content is constructed. Multilingual transformer models are fine-tuned using a robust text preprocessing pipeline, while a custom convolutional neural network is designed for visual feature extraction. Unimodal predictions are combined using a late fusion strategy to enhance robustness. Experimental results demonstrate that the XLM-RoBERTa-base model outperforms other multilingual transformer-based models on OCR-extracted text, achieving the highest macro F1-score of 97.20. The Visual Misinformation Detection Convolutional Neural Network (VMD-CNN) model achieves 96.05% accuracy on visual content, confirming that textual cues dominate while visual features provide complementary signals. Late fusion of the best-performing unimodal models further improves overall performance, reaching 97.15% test accuracy, highlighting the effectiveness of decisionlevel fusion in integrating heterogeneous modalities. This framework provides a practical solution for detecting misleading social media content, particularly in bilingual low-resource settings, and offers a foundation for future research in multilingual multimodal fake news detection and automated content moderation.