Systematic review and meta-analysis of the readability of large language models for patient education on ophthalmological conditions

OBJECTIVE: Ophthalmological patient education materials (PEMs) are written at a reading level of grades 10.4-12.6, higher than the recommended 8th-grade level. This study quantitatively assesses the readability of large language model (LLM)-generated chatbot responses to patient queries, de novo PEM creation, and simplification of existing PEMs. METHODS: This systematic review and meta-analysis article was registered in PROSPERO (CRD420251123401). We searched Ovid MEDLINE, Embase, CINAHL, Web of Science, and Scopus (August 11, 2025). The primary outcome was readability, measured by the Flesch-Kincaid Grade Level (FKGL). Random-effects meta-analysis compared ChatGPT-generated PEM readability with existing control PEMs (i.e., clinician- or organization-authored websites, brochures, and clinical guidelines). A binomial test evaluated the proportion of studies achieving an 8th-grade reading level. Joanna Briggs Institute and Grading of Recommendations Assessment, Development, and Evaluation tools assessed risk of bias and quality. RESULTS: = 97.08%). Seventeen studies (29.82%) included LLM(s) at or below 8th-grade reading level. Grade-level prompting significantly improved LLM-generated PEM readability. Risk of bias was low to moderate. Reasons for downgrading were inconsistency and imprecision. CONCLUSIONS: In the context of high heterogeneity and low certainty, it appears that LLMs do not produce ophthalmology PEMs that are more readable than existing materials.