Trustworthy of large language models

Summarise with VIRAT
Authors: Shucheng Gu
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This thesis investigates hallucination mitigation in question answering grounded in given knowledge and formulates the problem as one of faithfulness to the provided knowledge: when the knowledge is sufficient to support an answer, the model should answer based on that knowledge; when it is insufficient to support a verifiable answer, the model should refuse rather than guess. To address this objective, the thesis proposes a protocol-based supervised fine-tuning (SFT) approach that explicitly models answer behavior as a set of interdependent structured subtasks, including answerability judgment, evidence extraction, minimal rationale generation, short-answer generation, and discrete confidence self-assessment. On this basis, it constructs a complete workflow spanning data construction, teacher-supervision generation, sample filtering, SFT training, and unified evaluation. In the experiments, this thesis constructs answerable and derived unanswerable instances from the HotpotQA distractor subset, uses a strong teacher model to produce protocolized supervision, and applies LoRA-based SFT to Qwen3-4B. The best SFT checkpoint achieves 88.2%, 94.6%, and 92.6% on answer correctness, answerability accuracy, and semantic faithfulness, respectively, representing gains of 9.2, 7.2, and 11.6 percentage points over unconstrained natural-language answering by the base model. Further analyses show that these gains arise not primarily from surface formatting, but from a more stable refusal boundary, stronger evidence copying behavior, and more reliable evidence-grounded reasoning. Comparisons with auxiliary baselines further indicate that a protocol internalized through SFT yields substantial gains over obtaining protocol-structured outputs through detailed prompting and example demonstrations alone, and also offers a modest advantage over generic thinking on this task. These results suggest that, for QA grounded in given knowledge, protocolizing answer behavior and internalizing high-quality structured supervision into model parameters can reduce unsupported answering and improve behavioral verifiability.

Fields of Study & Subjects

Topic Modeling Natural Language Processing Techniques Multimodal Machine Learning Applications

Publication details

Journal / source
DR-NTU (Nanyang Technological University)
Publication date
1 January 2026
Language
EN