Performance of ChatGPT-5 in producing educational questions for ophthalmology

Purpose Most ophthalmology large language model (LLM) studies evaluate models as examinees rather than as generators of educational content. We assessed whether ChatGPT-5 can generate multiple-choice questions (MCQs) and explanations suitable for junior ophthalmology trainees, and whether the material meets basic standards of validity and safety when reviewed by clinicians. Design Cross-sectional content validation study. Subjects Eight ophthalmology clinicians (three attendings and five senior residents) served as expert raters. No human patient subjects were involved. Methods ChatGPT-5 generated fifteen ophthalmology MCQs (four options each) with brief explanations across multiple subspecialties. Raters, blinded to AI authorship, evaluated each question on four-point scales for accuracy, clarity, relevance, difficulty appropriateness, and safety. Explanations were rated for reasoning quality, clarity, clinical safety, and appropriateness. Item-level and scale-level Content Validity Indices (I-CVI, S-CVI/Ave) were calculated. Inter-rater agreement was assessed using Kendall’s coefficient of concordance (W), and differences between domains were tested using nonparametric methods. Results Inter-rater agreement for questions was moderate (Kendall’s W = 0.28–0.52) and higher for explanations ( W = 0.53–0.71). Relevance received the highest ratings, whereas accuracy was relatively lower. Explanation ratings were consistently high across domains, supporting their perceived educational value and clinical safety. Conclusions ChatGPT-5 can generate ophthalmology MCQs and explanations that clinicians judge as relevant, clear, and appropriate for junior learners. However, expert review remains necessary, particularly to verify accuracy, before incorporation into educational materials.