Trustworthiness and Explainability of Deep Learning for Diabetic Retinopathy Screening: Calibration and Clinical Utility Analysis

The implementation of deep learning models for diabetic retinopathy (DR) screening necessitates not only superior predictive accuracy but also dependable probability assessments, transparent decision processes, and verifiable clinical efficacy. Despite the increasing volume of work indicating high classification accuracy, the reliability of these models in practical screening environments remains little investigated. This paper introduces a thorough post-hoc evaluation approach for evaluating the reliability of a pretrained deep learning model utilized in diabetic retinopathy screening. A ResNet-50 model, trained on the APTOS 2019 dataset, was assessed for binary classification of referable versus non-referable diabetic retinopathy without any model retraining. In addition to standard performance measurements, probability calibration was evaluated through predicted calibration error and reliability diagrams, as well as post-hoc temperature scaling. Model explainability was evaluated by Grad-CAM visualizations, while clinical utility was tested using decision curve analysis at different referral levels. The model had robust discriminative performance, attaining an area under the receiver operating characteristic curve of 0.907, although it displayed considerable probability miscalibration. The analysis of explainability revealed that precise predictions mostly focused on therapeutically relevant retinal regions, while high-confidence incorrect predictions highlighted potential risks in autonomous applications. Decision curve analysis demonstrated a positive net clinical benefit across a wide range of parameters. These findings highlight that accuracy alone is inadequate for clinical preparedness and stress the need for a comprehensive assessment of trustworthiness for the safe implementation of deep learning models in diabetic retinopathy screening.