Self-consistency phenomenon in large language model and its application in knowledge augmentation tasks
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Recent studies have revealed significant “source bias” in retrieval and large language models (LLMs), where these models exhibit a preference for model-generated content over authentic human-written data. Humans similarly demonstrate “self-consistency” phenomena, characterized by a tendency to maintain coherent self-concepts and favor existing knowledge over external facts. The study investigated whether LLMs exhibit human-like cognitive biases. It was conducted from two perspectives: an explicit approach (content directly generated by theLLMs) and an implicit approach (using prompts to induce the models to perceive the content as self-generated). Confidence was evaluatedusing both self-assessment and consistency methods. Systematic experiments were carried out on several models, including GPT-4o, DeepSeek-R1, Llama2 (7B70B), and Qwen2 (7B72B). The results demonstrated that under implicit conditions,LLMs exhibit pronounced self-consistency phenomena, displaying significantly higher confidence in self-attributed content. Leveraging this characteristic yields consistent performance gains: the proposed role-based prompting strategy improved accuracy on TriviaQA, NQ, HotpotQA, FEVER, and ZsRE, and maintained a clear advantage under high levels of noisy retrieved documents, indicating strong robustness and generalization. Finally, our mechanistic analysis revealed that instruction fine-tuning and reinforcement learning from human feedback were the primary factors contributing to such biases in models, explaining how self-consistency emerges during training.
Fields of Study & Subjects
Topic Modeling
Information Retrieval and Search Behavior
Natural Language Processing Techniques
Publication details
- Journal / source
- DOAJ (DOAJ: Directory of Open Access Journals)
- Publication date
- 1 January 2026
- Language
- ZH