Comparing Interactive Machine Learning and Unassisted Segmentation of Geographic Atrophy From Fundus Autofluorescence

Purpose: To evaluate the benefit of using OPTIMEyes, an Interactive Machine Learning (IML) platform for annotation of ophthalmology images, along three axes: efficiency, uniformity, and noninferiority when compared to unassisted segmentation of geographic atrophy (GA) from fundus autofluorescence. Methods: We enrolled 10 annotators and collected 110 fundus autofluorescence images from patients with GA. We obtained ground truth GA segmentations from an expert retinal specialist and used 100 images to train the OPTIMEyes Artificial Intelligence (AI) segmentation model and 10 images to compare unassisted segmentation with AI-assisted segmentation. During data analysis, we excluded two annotators due to highly inconsistent segmentations. We measured annotation time in seconds and compared segmentations using the DICE score. Results: We showed that AI assistance shortened average annotation time by 96 seconds when compared to unassisted segmentation and helped produce more similar segmentations across annotators while maintaining similar DICE score between the annotators' segmentations and the expert's (mean DICE difference 0.02). Additionally, challenging images (i.e., unassisted DICE score < 0.3) showed the highest improvements in segmentation quality (mean DICE improvement: 0.38). Conclusions: We showed how OPTIMEyes can improve segmentation of GA from fundus autofluorescence by shortening annotation time and improving agreement across annotators without sacrificing segmentation quality. Translational Relevance: OPTIMEyes can be integrated in clinician's workflows to provide automated segmentations that can be easily manually improved, increasing the task's effectiveness and efficiency. In addition, it enables the efficient collection of high-quality manually labeled large imaging datasets that can be of great interest to the research community.