Confidence Calibration of Large Language Models (LLMC)

Large Language Models (LMs) have shown remarkable capabilities in text generation and question answering tasks. Their ability to generate plausible responses to user queries has resulted in their widespread adoption by those in academia, industry, and society at large. However, their propensity to hallucinate facts poses risks to widespread adoption. A key form of this hallucination is overconfidence. That is, LMs often assign high confidence to incorrect or unverifiable outputs, misleading users into trusting false information. This misalignment between a model's confidence and its actual correctness undermines reliability, especially in high-stakes applications such as education, healthcare, and legal reasoning. In this work, we examine the calibration of LMs across a range of task difficulties and domains, assessing whether their stated confidence accurately reflects their true performance. We further explore the conditions under which models are most prone to overconfidence and propose evaluation protocols for systematically identifying and mitigating confidence-based hallucinations. Our findings offer insights into improving model interpretability and trustworthiness by aligning internal certainty with empirical accuracy.