Comparison of Deep Learning Tools for Optic Nerve Axon Quantification Finds Limited Generalizability Upon Independent Validation

Machine learning approaches for automated quantification of optic nerve histology have emerged as tools for the objective assessment of axonal injury in experimental glaucoma models. However, their generalizability to independent datasets remains unclear. Guided by a scoping review following PRISMA-ScR guidelines, this study evaluated the performance of publicly available models on novel datasets. PubMed, EMBASE, Scopus, and Cochrane CENTRAL were searched (2000-2025). Two reviewers screened studies and extracted model characteristics and performance metrics. Three models (AxoNet, AxonDeepSeg, and AxoNet 2.0) were independently validated on a rat optic nerve dataset (44 images; 6941 axons) and a mouse optic nerve dataset (74 full cross-sections). From 2036 records, four manuscripts describing three deep learning models met the inclusion criteria, with reported correlation coefficients of 0.959-0.99 between model predictions and reference counts. On the rat dataset, the performance correlation declined (r = 0.831-0.907), precision remained high (>0.94), but recall was low (0.18-0.27), with Dice coefficients of 0.29-0.40. On the mouse dataset, correlations decreased further (r = 0.57-0.74) and model rankings differed, reflecting the domain shift and scale-dependent effects. These findings demonstrate strong within-study performance but reduced generalizability to independent datasets, highlighting the need for standardized validation datasets and multi-center testing.