Comparative performance of large language models in answering cornea and cataract surgery questions for resident training

Read full text Summarise with VIRAT
Authors: Seonghwan Kim, Man Ji, Chung Young Kim, Young In Yun, Gahye Lee, Chang Ho Yoon, Mee Kum Kim
Cited by: 0
References: 24

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

BACKGROUND: The application of large language models (LLMs) in the medical field has gained increasing popularity; however, their effectiveness in ophthalmology remains uncertain. This study aimed to evaluate the accuracy of responses generated by various deep learning-based LLMs to questions on cataract and corneal diseases and surgeries, and to verify their educational effectiveness by comparing the performances of LLMs with those provided by ophthalmology fellows and residents. METHODS: Eighty-one multiple-choice questions on corneal diseases and cataract surgeries were developed based on the standard format of the Korean ophthalmology board examination and categorized into three subtypes: recall-type (n = 27), interpretation-type (n = 27), and problem-solving-type (n = 27). The accuracy and appropriateness of commonly used LLMs (ChatGPT-4o, ChatGPT-5, Gemini 3.0 Pro, and Claude Sonnet 4.5) were evaluated and compared with one another, and against the performances of three ophthalmology residents and three corneal fellows. RESULTS: Among the four LLMs, ChatGPT-5 demonstrated the highest overall accuracy (75/81; 92.59%), followed by Gemini 3.0 Pro (73/81; 90.12%) and Claude Sonnet 4.5 (73/81; 90.12%), all of which outperformed ChatGPT-4o (70/81; 86.42%) as well as ophthalmology fellows (86.42 ± 1.23%) and residents (82.30 ± 8.22%). For recall-type questions, ChatGPT-5 achieved the highest accuracy (92.59%), and the other three LLMs (85.19%) outperformed both fellows (82.72 ± 4.28%) and residents (71.60 ± 17.50%). In interpretation-type questions, ChatGPT-5, Gemini 3.0 Pro, and Claude Sonnet 4.5 achieved perfect scores (100%), while ChatGPT-4o (92.59%) performed comparably to fellows (92.59 ± 7.41%) and better than residents (87.65 ± 4.28%). However, in problem-solving-type questions, all LLMs scored relatively lower (ChatGPT-5: 85.19%; others: 81.48%) than the mean performance of fellows (87.65 ± 11.91%). CONCLUSIONS: Among the LLMs, ChatGPT-5 demonstrated the highest overall performance; however, no statistically significant differences were observed between the models. Compared with ophthalmology trainees, ChatGPT-5, Gemini 3.0 Pro, and Claude Sonnet 4.5 achieved significantly higher overall scores than the mean trainee performance. Although LLMs showed good performance in recall- and interpretation-type questions, their relatively lower accuracy in problem-solving-type questions suggests that further advances in LLMs are necessary for their reliable use as educational tools in ophthalmology.

Fields of Study & Subjects

Artificial Intelligence in Healthcare and Education Ophthalmology and Visual Health Research Artificial Intelligence in Education

Publication details

Journal / source
BMC Ophthalmology
Volume
26
Issue
1
Publication date
28 March 2026
Language
EN