Artificial Intelligence-Based Risk Identification in Supervision Reports of the Ministry of Health

Abstract This study explores the potential of Artificial Intelligence (AI) in identifying and categorizing risks from unstructured open text, using advanced Natural Language Processing (NLP) architectures such as Dicta and HeBERT. The research aimed to develop a methodology for analyzing supervision reports from the healthcare sector, enabling risk detection and classification into predefined categories. The findings demonstrate that AI-based models can effectively identify risks and classify them with a high degree of accuracy, with Dicta outperforming HeBERT in all evaluated metrics, particularly in recall and F1 score. The study highlights the critical role of semantic features and keywords in risk identification. It also addresses challenges associated with ambiguous sentences and overlapping categories, emphasizing the need for future research to develop multicategory classification algorithms. While Dicta showed superior performance in identifying key categories such as “Infrastructure, Equipment, and Logistics” and “Medical Services and Quality of Care,” HeBERT exhibited limitations in distinguishing midrange categories, resulting in higher error rates. The findings suggest practical applications for regulatory bodies, such as optimizing resource allocation, enhancing decision-making through data-driven insights, and improving transparency and service quality. Despite its promising results, the study acknowledges limitations, including the reliance on a single corpus of healthcare supervision reports and the constrained sample size. Future research should expand the corpus and explore AI techniques for less structured texts. This research provides a foundational framework for applying AI to risk detection in healthcare and other domains, offering valuable insights for improving supervision, monitoring, and service delivery.