Assessing AI Accuracy in Diagnosing Pyoderma Gangrenosum

Introduction Pyoderma gangrenosum (PG), often considered a diagnosis of exclusion, can be challenging to diagnose. A new deep convolutional neural network (CNN) outperformed dermatologists (sensitivity 97% vs. 73%, respectively) in diagnosing PG versus venous ulcer (VU) using 422 clinical wound photographs in a training setting. Our purpose was to test its performance using real-world photographs. Methods Clinical photos from 58 cases of suspected PG at our institution were obtained retrospectively, with 51 having confirmed PG diagnoses by board-certified dermatologists. The CNN predicted the probability of PG for each photo using 50% as the decision threshold. For patients with more than one photo, probabilities were averaged to generate one per patient. Results The CNN correctly identified 31 of 51 PG cases and 5 of 7 non-PG cases, yielding a sensitivity and specificity of 60.8% and 71.4%, respectively. Discussion Delayed or missed diagnoses of PG can lead to poor patient outcomes due to distinct treatment pathways for PG versus other causes of ulceration. Therefore, a reliable and prompt diagnostic tool is needed. Based on data from our prior retrospective study evaluating PG rating tools, the CNN outperforms the Delphi and Su methods but not PARACELSUS (sensitivities 12%, 25%, and 100%, respectively). Limitations of this study include a lack of standardized image quality, and the CNN’s ability to distinguish only between PG and VU. The CNN may hold value in validating practitioners’ diagnoses but further refinement and training using diverse, real-world datasets are necessary to enhance its diagnostic, clinical utility.