Multi‐site performance of the telemedicine retinopathy of prematurity severity score (tROP‐SS)

OBJECTIVE: This study aims to evaluate the performance of the telemedicine retinopathy of prematurity severity score (tROP-SS) across all 14 NICUs in the TELEROP cohort against the modified ROP activity score (mROP-ActS). DESIGN: Retrospective cohort study. SUBJECTS: The study included 2037 neonates who underwent ROP screening in 14 NICUs via the TELEROP telemedicine network from November 2017 to January 2025. METHODS: We compared the robustness of tROP-SS and mROP-Act systems across all exams. MAIN OUTCOME MEASURES: The primary outcomes studied were the ability to return a score in cases responding to treatment, correlation between the two scores and the ability to assess disease directionality. RESULTS: The study analysed 192 704 photos from 11 368 ROP exams on 2037 patients. tROP-SS generated scores for 100% of exams, demonstrating significant improvements over mROP-ActS, which scored only 92.3% of all exams (p = 0.0136). All patients who met treatment criteria received treatment in both eyes. Among high-risk and treatment warranted (TW) patients, 100.0% exams had a valid tROP-SS exam compared with only 72.73% and 45.27% with a valid mROP-ActS. Only 20.9% high-risk or TW patients had a valid mROP-ActS across all their exams. Incremental increases in tROP-SS were more strongly related to worsening of disease: OR (95% CI) - 3.37 (2.51-4.53) versus 1.57 (1.46-1.69), both p < 0.0001. tROP-SS also demonstrated improved granularity and accuracy in categorizing patients that would ultimately require treatment (AUC: 0.991 vs. 0.825). CONCLUSIONS: tROP-SS was retrospectively evaluated across 14 NICUs, proving to be a more robust scoring system compared to mROP-ActS. The findings support continued use of tROP-SS, supplemented by standard screening protocols to ensure no cases of treatment-warranted ROP are missed. Future research may focus on integrating additional parameters to enhance predictive models to improve early ROP detection.