Utility of ChatGPT-4o in Creating Patient Handouts in Ophthalmology: A Comparison With American Academy of Ophthalmology Educational Materials

Purpose: This study compares the readability and quality of ChatGPT-4o-generated and American Academy of Ophthalmology (AAO) patient handouts in English and Spanish, as AAO materials are expert developed but not publicly accessible. Methods: Ten AAO handouts on common ocular conditions were obtained in English and Spanish. ChatGPT-4o was queried to create handouts at an 8th-grade reading level of comparable length. English readability was assessed with Simple Measure of Gobbledygook (SMOG) and Flesch-Kincaid Reading Ease, and Spanish readability was assessed with five validated metrics, including the Szigriszt-Pazos Perspicuity Index (SPPI) and the Flesch-Szigriszt Index (INFLESZ). Grade levels were measured by the Flesch-Kincaid Grade Level and Crawford-Nivel-de-Grado. Three ophthalmologist graders per language assessed content with the Quality of Generated Language Outputs for Patients (QGLOP), indicated their preferred handout, and attempted to identify its source. Results: ChatGPT-4o handouts had English readability similar to that of AAO (SMOG 7.86 ± 0.53 vs. 7.49 ± 0.76; P = 0.35) but significantly lower Spanish SPPI scores (60.26 ± 4.47 vs. 64.82 ± 2.97; P = 0.04). Grade levels were comparable. English QGLOP scores were similar (14.73 vs. 14.87; P = 0.97), but Spanish ChatGPT handouts scored higher in all categories (15.83 vs. 14.23; P < 0.001). ChatGPT-4o handouts were correctly identified by 17 of 30 in English (56.7%) and by 3 of 30 in Spanish (10.0%). They were favored 23 out of 30 times in English (76.7%) and 29 out of 30 times in Spanish (96.7%). Conclusions: ChatGPT-generated materials matched or exceeded AAO handouts in readability and content quality, with Spanish versions being most preferred. Translational Relevance: Integrating artificial intelligence-generated patient education materials into ophthalmic care can enhance health literacy for multilingual patient populations.