SUS LLM Hallucination Dataset: A curated benchmark for evaluating hallucinations in large language models in a regulated public health system

Read full text Summarise with VIRAT
Authors: Christiano Talhate
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This dataset provides a curated benchmark for evaluating hallucinations in large language models (LLMs) in a highly regulated public health system. The dataset contains 90 structured questions covering key domains of Brazil’s Unified Health System (SUS), including coverage, care pathways, eligibility criteria, and patient rights. Each item is annotated with a taxonomy of hallucination types (T1–T6), risk levels, and associated normative sources, enabling systematic evaluation of factuality, compliance, and safety in LLM-generated responses. The dataset is designed for use in:- Benchmarking LLM performance in regulated environments- Evaluating retrieval-augmented generation (RAG) systems- Research in AI safety, governance, and hallucination mitigation- Public sector AI evaluation and policy-oriented research This resource emphasizes high-risk, regulation-sensitive scenarios, where hallucinations may have real-world consequences. All data are provided in structured formats with accompanying documentation to support reproducibility and reuse.

Fields of Study & Subjects

Artificial Intelligence in Healthcare and Education Topic Modeling Natural Language Processing Techniques

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
20 March 2026
Language
EN
Document type
Dataset