Trustworthy of large language models
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
This thesis investigates hallucination mitigation in question answering grounded in given knowledge and formulates the problem as one of faithfulness to the provided knowledge: when the knowledge is sufficient to support an answer, the model should answer based on that knowledge; when it is insufficient to support a verifiable answer, the model should refuse rather than guess. To address this objective, the thesis proposes a protocol-based supervised fine-tuning (SFT) approach that explicitly models answer behavior as a set of interdependent structured subtasks, including answerability judgment, evidence extraction, minimal rationale generation, short-answer generation, and discrete confidence self-assessment. On this basis, it constructs a complete workflow spanning data construction, teacher-supervision generation, sample filtering, SFT training, and unified evaluation. In the experiments, this thesis constructs answerable and derived unanswerable instances from the HotpotQA distractor subset, uses a strong teacher model to produce protocolized supervision, and applies LoRA-based SFT to Qwen3-4B. The best SFT checkpoint achieves 88.2%, 94.6%, and 92.6% on answer correctness, answerability accuracy, and semantic faithfulness, respectively, representing gains of 9.2, 7.2, and 11.6 percentage points over unconstrained natural-language answering by the base model. Further analyses show that these gains arise not primarily from surface formatting, but from a more stable refusal boundary, stronger evidence copying behavior, and more reliable evidence-grounded reasoning. Comparisons with auxiliary baselines further indicate that a protocol internalized through SFT yields substantial gains over obtaining protocol-structured outputs through detailed prompting and example demonstrations alone, and also offers a modest advantage over generic thinking on this task. These results suggest that, for QA grounded in given knowledge, protocolizing answer behavior and internalizing high-quality structured supervision into model parameters can reduce unsupported answering and improve behavioral verifiability.
Fields of Study & Subjects
Topic Modeling
Natural Language Processing Techniques
Multimodal Machine Learning Applications
Publication details
- Journal / source
- DR-NTU (Nanyang Technological University)
- Publication date
- 1 January 2026
- Language
- EN