Assessing Factual Knowledge in Large Language Models Using Prime Recall

View at publisher
Authors: Davut Çulha
Cited by: 0
References: 12

Abstract

Assessing how Large Language Models (LLMs) acquire, retain, and retrieve factual information is crucial for understanding their reliability and knowledge depth. Existing benchmarks often blur the distinction between factual recall and reasoning, making it challenging to measure factual retention in isolation. This study introduces a novel method for assessing the factual knowledge capacity of LLMs using prime recall through prompt engineering. Prime numbers serve as an ideal testing domain because they exhibit no predictable patterns, requiring identification based solely on memorized factual knowledge rather than inferential reasoning. The proposed approach systematically queries LLMs with structured prompts and employs an iterative algorithm that progressively increases the numerical complexity of tested primes until model accuracy falls below a defined threshold, thereby revealing the upper boundary of factual retention. The assessment was performed on models from the Gemini, GPT, and Claude families. Results indicate that newer and larger models demonstrate stronger factual consistency and maintain accuracy across higher number positions before degradation occurs. The observed performance hierarchy underscores the influence of model scale and training diversity on factual retention. Furthermore, the findings suggest that numerical factuality offers an effective diagnostic perspective for analyzing how LLMs store, retrieve, and generalize factual information. The results confirm that prime recall provides a lightweight, interpretable, and reproducible method for quantifying factual depth in LLMs.

Fields of Study & Subjects

Topic Modeling Natural Language Processing Techniques Information Retrieval and Search Behavior

Publication details

Journal / source
International Informatics and Software Engineering Conference (IISEC)
Pages
256–261
Publication date
5 February 2026
Language
EN