Why Text Alone Is Not Enough: An Empirical Analysis of Fake News Detection

Fake news detection has received significant attention with the increasing spread of misinformation on social media platforms. Many existing studies report promising results using text-based machine learning and transformer models. In this paper, we conduct an empirical study to evaluate the effectiveness of text-only fake news detection. We experiment with traditional machine learning models using TF-IDF features as well as a finetuned DistilBERT transformer model on a balanced fake news dataset. Despite extensive preprocessing and model tuning, both classical and transformer-based approaches fail to outperform random baselines, with the transformer model frequently collapsing to single-class predictions. Our analysis reveals that textual content alone often lacks sufficient discriminative information to determine factual correctness, leading to label-text misalignment. These findings highlight a fundamental limitation of text-only fake news detection and suggest that additional signals such as metadata, source credibility, or external knowledge are necessary for reliable misinformation detection.