Evaluation of large language models in generating and optimizing educational materials for neonatal home oxygen therapy
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Background Neonatal Home Oxygen Therapy (NHOT) is a critical treatment for premature infants with Bronchopulmonary Dysplasia (BPD). However, existing health education materials are generally difficult to read, particularly for grandparent caregivers with lower educational backgrounds. This study aimed to systematically evaluate the capacity of six major Large Language Models (LLMs) to generate and optimize NHOT health education materials. Methods Six LLMs were included: ChatGPT-5.1, Claude 4.5 Sonnet, Gemini 2.5 Pro, Grok-4.1, Qwen-3-Max, and DeepSeek-V3.2. Each model generated 20 texts under three prompting strategies—baseline (Prompt A), simplification (Prompt B), and rewriting (Prompt C)—yielding 360 texts in total. Twenty WeChat public health articles served as the human-authored baseline. Subjective evaluation employed C-DISCERN, C-PEMAT (understandability and actionability), and a medical accuracy Likert scale, supplemented by objective linguistic analysis using the Alpha Readability Chinese (ARC) tool. Results All models demonstrated superior medical accuracy compared to the human baseline (Likert median 1.0, against 2.0 for the original articles). Under baseline conditions, Qwen achieved the highest content quality (C-DISCERN median 57.0), while Claude attained perfect actionability scores. The simplification prompt (Prompt B) significantly reduced C-DISCERN scores across all models (all p < 0.001) without meaningfully improving understandability or actionability. In the rewriting task (Prompt C), all models significantly enhanced the understandability of original texts ( p < 0.01), with Grok and Qwen additionally improving content quality and actionability. Linguistic analysis revealed that prompt optimization improved semantic accuracy and reduced semantic noise, but at the cost of decreased lexical richness. Conclusion LLMs demonstrate significant potential for optimizing existing health education materials, performing more reliably in rewriting mode than in de novo generation. Simplistic “plain language” instructions risk compromising content quality, highlighting the need for carefully designed prompts that balance accuracy, clarity, and completeness. All AI-generated materials require rigorous review by qualified clinical professionals prior to distribution.
Fields of Study & Subjects
Artificial Intelligence in Healthcare and Education
Text Readability and Simplification
Health Literacy and Information Accessibility
Publication details
- Journal / source
- Frontiers in Artificial Intelligence
- Volume
- 9
- Pages
- 1770564–1770564
- Publication date
- 27 March 2026
- Language
- EN