Hallucination-aware Retrieval Augmented Generation with Evidence-based Semantic Clustering

View at publisher Summarise with VIRAT
Authors: Cunxin Jin, Yao Lu, Jiaqi Nie, Shanqing Yu
Cited by: 0
References: 12

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

The rapid advancement of large language models (LLMs) has significantly improved open-domain question answering. However, their vulnerability to factual inconsistencies, or hallucinations, hinders their use in high-stakes domains. To address this, our approach employs a two-stage generation strategy. First, a seed answer is generated with low-temperature sampling to retrieve and inject the most relevant evidence into the prompt. This evidence anchoring stage achieves high precision by calculating the semantic similarity of retrieved sentences and selecting only the single most relevant. Second, multiple diverse answers are generated with high-temperature sampling to expose model uncertainty. We quantify this uncertainty by clustering the candidates and calculating a consistency ratio; if the ratio falls below a threshold, the system abstains from answering. This consistency ratio threshold is optimally set at 0.5 based on ablation experiments. Evaluations on SQuAD 2.0 and SelfRC show our framework significantly outperforms baselines like standard Retrieval-Augmented Generation(RAG) and Chain-of-thought(CoT), achieving up to an 8.8% accuracy improvement on SQuAD 2.0. These results demonstrate that combining evidence anchoring with uncertainty-guided refusal yields more reliable, auditable, and deployment-ready Question Answering (QA) systems. Consequently, our framework provides a robust and generalizable methodology for advancing the safety and trustworthiness of large language models in practical applications.

Fields of Study & Subjects

Topic Modeling Multimodal Machine Learning Applications Information Retrieval and Search Behavior

Publication details

Journal / source
International Conference on Artificial Intelligence and Social Network Systems (AISNS)
Pages
1–6
Publication date
9 January 2026
Language
EN