A Bayesian-Optimized XGBoost Approach for Money Laundering Risk Prediction in Financial Transactions

The rapid expansion of global commerce has escalated the complexity of money laundering schemes, making the detection of illicit transfers an urgent but highly challenging research problem. In operational anti-money laundering (AML) systems, the extreme rarity of illicit transactions often overwhelms compliance teams with false positives, leading to severe “alert fatigue.” To address this critical bottleneck, this paper introduces an enhanced, probability-driven risk-prioritization framework utilizing an XGBoost classifier integrated with Bayesian Optimization (BO-XGBoost). By optimizing directly for the Area Under the Precision–Recall Curve (PR-AUC), the model is specifically tailored to rank high-risk anomalies under severe class imbalance. We validate the proposed approach on a rigorously resampled transaction dataset simulating a realistic 5% laundering rate. The BO-XGBoost model demonstrates exceptional prioritization capability, achieving an ROC-AUC of 0.9686 and a PR-AUC of 0.7253. Most notably, it attains a near-perfect Precision@1%, meaning the top 1% of flagged transactions are 100% true illicit activities, entirely eliminating false positives at the highest priority tier. Comparative and SHAP-based interpretability analyses confirm that BO-XGBoost easily outperforms sequence-heavy deep learning baselines. Crucially, it matches computationally expensive stacking ensembles in peak predictive precision while significantly surpassing them in operational efficiency, indicating its immense promise for resource-optimized, real-world compliance screening.