Multimodal misinformation detection across diverse languages using RAG and LLMs

Read full text
Authors: Sheetal Harris, Vinh Thong Ta, Marcello Trovati, Ghada Nakhla, Faiza Latif, Ioannis Korkontzelos
Cited by: 4
References: 49

Abstract

The rapid spread of multimodal fake news (FN) on Online Social Networks (OSNs) threatens digital information ecosystems, particularly in low-resource languages. Existing multimodal fake news detection (FND) methods are largely limited to high-resource settings, restricting their global applicability. We propose an M&M-RAG, a Multilingual & Multimodal Retrieval-Augmented Generation framework, that leverages Large Vision-Language Models (LVLMs) and Large Language Models (LLMs) to verify news claims across English, Chinese and Urdu. M&M-RAG integrates real-time multilingual evidence retrieval, language-aware prompting, and cross-modal reasoning for fact verification. We further propose Multi-Ax-to-Grind Urdu, the first large-scale, multi-domain multimodal benchmark for FND in Urdu. Experiments on typologically diverse monolingual multimodal datasets demonstrate that M&M-RAG achieves state-of-the-art (SOTA) performance, with 94.6% accuracy and 94.2% F1 score, surpassing models such as SpotFake, MPFN, MMCFND, and Semi-FND. The proposed framework remains robust in zero-shot and cross-lingual scenarios under frozen-model inference without task-specific fine-tuning. The results underscore the scalability and interpretability of LVLM-based approaches for combating multimodal misinformation, particularly in under-represented and typologically diverse languages.

Fields of Study & Subjects

Misinformation and Its Impacts Big Data and Digital Economy Spam and Phishing Detection

Publication details

Journal / source
Journal of Intelligent Information Systems
Publisher
Springer Science and Business Media LLC
Volume
64
Issue
4
Pages
1719–1746
Publication date
9 April 2026
Language
EN