Comparative Evaluation of Deep-Reasoning Large Language Models for Ophthalmic Emergencies

Purpose: To evaluate contemporary deep-reasoning large language models (LLMs) for early assessment of ophthalmic emergencies using sequential, workflow-mimicking information levels. Design: Cross-sectional, vignette-based, head-to-head comparative evaluation. Subjects: Thirty-four deidentified emergency ophthalmology teaching cases curated from a publicly accessible repository. Methods: Each case was reconstructed into 3 sequential information levels (level 1 [L1]: history; level 2: basic examination; level 3 [L3]: specialist examination). Six LLMs (Doubao, DeepSeek, Kimi-2, ChatGPT-5, Gemini-3, and Grok-4), operating in deep-reasoning mode, generated outputs that were independently scored by 2 ophthalmologists. Diagnoses were graded as fully correct, partially correct, or incorrect; triage category (typical vs. atypical emergency) was rated as correct or incorrect. Ancillary test recommendations were mapped to a prespecified 10-category taxonomy and classified as undertesting, exact match, or overtesting. A four-level composite outcome integrated diagnostic correctness, triage accuracy, and testing. Main Outcome Measures: Diagnostic correctness (fully correct, partially correct, incorrect), triage-category accuracy, ancillary test recommendation patterns, and composite outcome (ideal, safe but overtesting, potentially dangerous, intermediate). Results: < 0.001 across models). In generalized estimating equation pairwise comparisons, ChatGPT-5 showed higher odds of a fully correct diagnosis than DeepSeek (odds ratio [OR], 3.54; 95% confidence interval [CI], 1.49-8.43) and Gemini-3 (OR, 2.24; 95% CI, 1.31-3.83), and lower odds of potentially dangerous composite outcomes than DeepSeek (OR, 0.28; 95% CI, 0.10-0.74) and Gemini-3 (OR, 0.31; 95% CI, 0.11-0.89). Conclusions: Deep-reasoning LLMs demonstrated high triage category accuracy and moderate diagnostic performance for ophthalmic emergencies, with diagnostic correctness improving at higher information levels. However, ancillary testing patterns varied substantially, and ideal composite safety profiles were uncommon, supporting cautious, supervised deployment with explicit guardrails governing workup recommendations. Financial Disclosures: The author has no/the authors have no proprietary or commercial interest in any materials discussed in this article.