Artificial Intelligence vs. Human Readers in Contrast-Enhanced Harmonic Imaging Endoscopic Ultrasound Interpretation of Solid Pancreatic Masses: A Multicenter Interobserver Study

Background/Objectives: Contrast-enhanced harmonic imaging endoscopic ultrasound (CHI-EUS) is a valuable tool for characterizing solid pancreatic tumors. However, interobserver variability remains a significant limitation in clinical interpretation. Artificial intelligence (AI) may offer objective, reproducible assessments, potentially enhancing diagnostic performance. This study compared the diagnostic accuracy and interobserver agreement of nine physicians with varying CHI-EUS experience levels vs. a dedicated AI system and a general-purpose large language model (ChatGPT) on the same 118 histologically confirmed cases. Methods: We conducted a prospective, multicenter, observer-blinded study involving 118 CHI-EUS video cases of histologically confirmed (EUS-FNB) focal pancreatic masses from three tertiary care centers in Romania. Nine readers were stratified into three groups: trainees (10 years). All readers and two AI models received standardized, anonymized 2 min CHI-EUS video clips. A dedicated AI system used a convolutional neural network (CNN) for lesion segmentation and time–intensity curve (TIC) extraction, followed by a feedforward neural network (FNN) for classification. ChatGPT was separately evaluated on the same videos. Diagnostic metrics (accuracy, sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV], and AUROC) were calculated. Interobserver agreement was assessed using Fleiss’ and Cohen’s kappa statistics. Results: The dedicated AI system achieved an overall accuracy of 95.8% (sensitivity 96.6%; specificity 94.1%) in diagnosing pancreatic adenocarcinoma. Expert readers had a mean accuracy of 78.8% (sensitivity 86%, specificity 61%, and AUROC 0.74), intermediates 80.8% (sensitivity 83%, specificity 75%, and AUROC 0.84), and trainees had a mean accuracy of 67.2% (sensitivity 70%, specificity 60%, and AUROC 0.67). For the most-likely-diagnosis parameter, interobserver agreement was similar between intermediates (Fleiss’ κ = 0.407) and experts (κ = 0.389), while trainees showed lower agreement (κ = 0.203). ChatGPT correctly classified only 14.1% of PDAC cases. Conclusions: A specialized AI model for CHI-EUS video analysis can achieve expert-level performance and reduce diagnostic variability across experience levels. Integration of dedicated AI systems into CHI-EUS interpretation may enhance accuracy and serve as a valuable decision support tool in clinical and training settings.