MediCERN: An LLM-Powered Platform for Quality Evaluation of Medical Videos

YouTube has become a vital platform for disseminating medical information and guiding health decision-making. However, the literature confirms the prevalence of inaccurate, biased, or misleading content on this platform and underscores the need for robust quality assessment. This paper presents MediCERN, a platform that leverages a large language model (LLM) to evaluate the quality of medical videos using DISCERN, a widely accepted instrument for assessing health information reliability. Users can access MediCERN through a web interface to search for medical videos. The platform forwards each search query to YouTube and retrieves a list of relevant videos. MediCERN then checks whether any of these videos have been previously evaluated. Evaluated videos are presented to users ranked by their quality scores, while unevaluated videos are displayed without a score; however, users can request on-demand evaluation. The system processes these requests and provides scores within a few seconds. Sample analyses demonstrate substantial agreement between the model’s and experts’ ratings, with a Brennan–Prediger Kappa of 0.87. We further demonstrate the platform’s utility by analyzing over 5,000 videos. The results indicate that quality scores followed a Gaussian-like distribution, with most videos exhibiting medium quality. We also assessed the platform’s efficiency in terms of cost, time, and storage. For instance, the current prototype can evaluate one hour of video content in approximately 0.6 minutes at a cost of $0.0026 USD. These findings demonstrate that MediCERN offers a scalable and efficient solution for promoting access to high-quality medical content on YouTube.