Comparative study of transformer-based models for cardiovascular disease risk stratification with tabular biomarker data

Cardiovascular disease (CVD) remains one of the leading causes of global mortality and disability. Advances in computational modeling and artificial intelligence have enhanced CVD risk prediction by integrating multivariate clinical and biochemical features. Recent developments in deep learning, especially transformer-based models for tabular data, have demonstrated superior capabilities in capturing nonlinear and high-dimensional biomarker interactions. This study proposes a predictive framework that uses statistical and clinical biomarker data to assess CVD risk. Traditional machine learning models (logistic regression, Gaussian Naïve Bayes, linear discriminant analysis, AdaBoost, and XGBoost) were compared with deep learning models (gated recurrent unit [GRU] and long short-term memory [LSTM]) and transformer-based models—self-attention and intersample attention transformer (SAINT), feature tokenizer (FT), and tab transformer. Experiments were conducted to evaluate the impact of data augmentation, analyze learning behavior through loss and accuracy curves, and assess a fusion approach combining tab transformer with recurrent networks. Model performance was evaluated using accuracy and receiver operating characteristic analysis. Transformer-based models consistently outperformed conventional machine learning and deep learning methods. SAINT and FT achieved an area under the curve (AUC) of 0.8875 and 0.9489, respectively. The tab transformer demonstrated the highest performance with an AUC of 0.9728. The fusion of the tab transformer with GRU and LSTM further enhanced predictive precision, improving representation learning and generalization for CVD risk prediction. The proposed transformer-based framework offers a robust, scalable, and interpretable solution for accurate CVD risk assessment. Its superior predictive capability highlights the potential for integration into clinical decision-support systems for early diagnosis and patient management.