Reference Standard for Validation of Age-Related Macular Degeneration Screening Algorithms

PURPOSE: Artificial intelligence (AI)-based screening models hold promise for identifying individuals with undiagnosed age-related macular degeneration (AMD) in nonspecialist settings. A standardized reference framework for image labeling is needed to enable consistent training, validation, and deployment of AI-based screening algorithms. The goal of the present study was to establish expert consensus on an image-based reference standard for labeling AMD. DESIGN: Modified Delphi consensus study. PARTICIPANTS: Fellowship-trained retina specialists, ophthalmologists, AI specialists, and imaging specialists. METHODS: A prespecified Delphi process was conducted using structured surveys. Over 2 rounds, panelists assessed opinions on existing reference standards, including the Age-Related Eye Disease Study scale and Beckman scale, as well as imaging methods such as color, OCT, and autofluorescence. The surveys also evaluated imaging features of AMD, including drusen, pseudodrusen, and pigment changes, as well as referral criteria. Consensus was defined using a 9-point Likert scale, with predefined statistical thresholds for agreement. MAIN OUTCOME MEASURES: Agreement on key elements of a reference standard. RESULTS: Consensus was reached on adopting the Beckman classification as the level 1 reference standard (median score, 8; agreement). OCT use for identifying key AMD features, including drusen, geographic atrophy (GA), and choroidal neovascularization, also reached consensus (median scores, 8.5-9; agreement). Pigment change detection did not reach consensus (median, 7.5; uncertain), and screening age thresholds showed nonconsensus (median, 8; uncertain). Referral thresholds reached consensus, including urgent referral for neovascular AMD and nonurgent referral for GA and intermediate AMD (median, 9; agreement). CONCLUSIONS: This study defined a consensus-based reference standard for labeling AMD from images for AI-based screening. These recommendations are intended to support consistent AI model development and evaluation, while remaining distinct from clinical practice guidelines. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.