TransFuse: An Interpretable Transformer Ensemble Framework for Bangla Smishing Detection

The proliferation of mobile messaging has led to a surge in malicious activities like smishing, posing a significant threat to users, particularly in low-resource languages such as Bangla. While deep learning models have shown promise, their “black box” nature often hinders adoption in security-critical applications where trust is paramount. To address this, we introduce TransFuse, a robust and interpretable framework that employs a cross-validated ensemble of two state-of-the-art (SOTA) multilingual transformers, XLM-RoBERTa and mDeBERTa-v3, fine-tuned using a novel Layer-wise Learning Rate Decay (LLRD) strategy. On the publicly available BangalaBarta dataset, TransFuse achieves a new SOTA accuracy of 99.52%, demonstrating superior capability in classifying smishing, promotional, and normal SMS. Critically, we move beyond predictive accuracy by leveraging SHAP (SHapley Additive exPlanations) to provide faithful interpretations of the ensemble's decisions. This analysis confirms that our model's predictions are driven by semantically meaningful keywords, enhancing its trustworthiness. TransFuse represents a significant step towards building highly accurate and transparent AI systems for combating digital threats, offering a practical solution for securing mobile communications in underresourced linguistic environments.