HITL-RAG FAITHFULNESS EVALUATION DATASET FOR CULTURAL TOURISM INFORMATION

This dataset supports accuracy evaluation for Retrieval-Augmented Generation (RAG) systems applied to cultural tourism information, with a specific focus on Human-in-the-Loop (HITL) governance. Designed as an evaluation artifact rather than a performance benchmark, the dataset emphasizes traceability, foundational accuracy, and decision-making transparency. It comprises structured query-response evaluations, including retrieved text snippets, candidate answers generated by RAG-based language models, reference annotations from LLM-based evaluators, automated similarity metrics, and final human evaluation decisions. Accuracy is assessed against curated cultural knowledge snippets, while system-level fallback mechanisms and failure handling are explicitly distinguished from content-level fundamentals. A structured HITL error taxonomy (E0–E10) is applied to differentiate between accurate responses, controlled refusals, content errors, and system failures. Repeated queries are intentionally included to observe consistency across evaluation cycles, enabling analysis of generative variation while maintaining the stability of classification results based on human assessment. Version management follows a snapshot-based policy to ensure reproducibility and cross-version comparability. This release (Version 1.3) contains no substantial changes other than improved image quality and the addition of metadata tables. The inclusion of technical specifications for the LLM models used represents a revision not previously documented. Version 1.3 is confidential and will only become publicly available following publication in a relevant journal.