Evaluation of Surgical Skills Using Machine Learning and Interpretation of Results with Explainable AI in Practical Laparoscopic Surgery Training

To facilitate efficient laparoscopic surgical education, a system was developed that utilizes machine learning to classify surgical skill levels—novice, intermediate, and expert—based on the motion dynamics of surgical instruments. This system not only categorizes surgical proficiency but also incorporates SHAP, an Explainable AI technique, to provide insights into the rationale behind each classification result. For the machine learning dataset, the movements of four surgical instruments were recorded using a motion capture (mocap) system during total nephrectomy training sessions conducted on 46 cadaveric specimens prepared for laparoscopic surgery. The entire nephrectomy procedure was divided into three distinct processes: colon mobilization (Process 1), renal vascular dissection (Process 2), and incision and removal of the remaining tissues (Process 3). Surgical skill analysis was performed separately for each phase. Surgeons were categorized into three groups based on their prior experience with laparoscopic procedures: novices (0–9 cases), intermediates (10–49 cases), and experts (50 or more cases). A total of 111 features were extracted from the instrument motion data for each phase, and comparative analyses were conducted across the three groups. Multiple machine learning approaches—including Support Vector Machine (SVM), Principal Component Analysis followed by SVM (PCA-SVM), and Random Forest—were employed to develop models for classifying surgeons into three distinct skill levels. The classification performance of these models was subsequently validated. The results revealed that features related to efficiency and speed significantly contributed to differences in surgical skill levels. The developed system enables quantitative comparison and visualization of specific instrument characteristics. This system contributes to intelligent integration of surgical education and Explainable AI, providing actionable feedback for skill improvement.