Interpretable and Explainable AI for Predicting Systems Thinking Skills

Rapid technological advancements in recent years have led to escalating complexity in the modern systems environment. As a result, many contemporary challenges may exceed engineers’ ability to operate effectively within today’s highly complex systems landscape, where systems thinking skills are particularly important. This study aimed at applying Interpretable Machine Learning (ML) algorithms to assess individual systems thinking (ST) ability in light of seven essential ST dimensions, including Level of - Complexity (CMP), Independence (IND), Interaction (INT), Change (CNG), Uncertainty (UNCR), Flexibility (FLEX), and Systems Worldview (SYSW). Consequently, several ML algorithms, including Logistic Regression (LR), AdaBoost, Artificial Neural Network (ANN), Random Forest (RF), and K-Nearest Neighbors (KNN) were trained and evaluated to classify participants’ ST levels across these dimensions. In addition, all the models underwent hyperparameter tuning to improve predicting capabilities, with an optimal model selected based on comparison analysis that can better forecast an individual’s ST skills. Further, a set of explainable ML analysis (XAI) tools, such as Brier Score, Youden Index, LIME, SHAP force, and summary plots, has been generated to offer further insights into the underlying ML models. The comparative analysis indicates that LR outperformed other models with respect to all four performance metrics (Accuracy, Precision, Recall, F1 score). The study’s model can assist industries in adopting systems thinking principles for both hiring candidates and workforce development. Moreover, integration of XAI enhances the validity of the analysis and enables more intelligent decision-making.