A dual attention transformer modelling for explainable mental health analysis in academic environments using TaBERT

Mental health plays a crucial role in shaping students' academic performance and practical outcomes, yet traditional analytical methods often fall short in capturing the complex, multifactorial nature of psychological well-being. In recent years, the latest techniques are increasingly exploring eXplainable AI (XAI) to enhance model transparency and interpretability in mental health research. In this study, we introduce a transformer-based model, Tabular BERT (TaBERT), designed to comprehensively integrate and analyze contextual, psychological, personal, and social features for mental health prediction. By integrating deep contextual embeddings, bidirectional attention, and a novel memorization mechanism, TaBERT excels in capturing intricate feature interactions that conventional machine learning and ensemble learning methods may overlook. Comparative experiments confirm the superiority of the proposed model with highest accuracy of 96%. Empirical analyses were further strengthened through advanced feature ranking techniques, including information gain, gain ratio, and entropy, revealing that mental health-related features provide the most significant information value of 0.129 for prediction. To enhance interpretability and trust, we applied explainable AI techniques, including Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP) offering detailed global and local explanations of feature contributions. Finally, comprehensive statistical tests based on p-values provided additional support for the robustness and significance of the findings.