Artificial Confabulation and Untrustworthiness: The Structural Limits of Transformer-based, Next-Token AI Architectures

Read full text
Authors: A. Devin
Cited by: 0

Abstract

Transformer-based large language models (LLMs) are architecturally incapable of verification (V), a core component of epistemic integrity. This paper argues that the structural absence of V, coupled with the absence of consequence-bearing experience (E), renders LLMs fundamentally untrustworthy, regardless of scale or performance on pattern-matching benchmarks. This incapacity also prevents organisations from sustaining Sovereign Intelligence – the ability to verify, override and remain accountable for decisions made with or by AI systems. While most research treats “hallucinations” as behavioural glitches to be solved, the KEV constraint (Knowledge is massive, Experience is absent, Verification is absent) dictates that scaling intensifies the failure mode, which is taxonomically classified as Artificially Confabulating and Untrustworthy (ACU). This confabulation is not a defect but the architecture operating as designed: fluent completion without truth. This claim concerns model-internal epistemic capacity: tool use, retrieval and external artefacts may support system-level checking, but they do not create internal verification capability and therefore cannot be treated as self-verification. The paper draws a parallel with clinical confabulation, documents the compounding failure stack (Speed-Weighted Reasoning Collapse → Architectural Confabulation → Uncalibrated Confidence) and presents systematic adversarial interrogation across frontier systems to demonstrate cross-model repeatability. Ensembling, debate and multi-agent coordination may reduce error rates in some tasks; however, agreement among generators is not evidence of truth and does not resolve the absence of verification (V = 0). The path forward is not autonomous AI but Governed Hybrid Intelligence (GHI): human judgement augmented by machine synthesis and constrained by external verification that supplies the grounding the architecture lacks. Without GHI, scaling produces more sophisticated confabulation, not more reliable intelligence. This paper concludes that contemporary AI deployment rests on an epistemic mirage. Truth does not emerge from these architectures. Where Sovereign Intelligence is not preserved, judgement authority necessarily migrates away from the human actor. This version (v1.1) tightens the definition of Verification (V), clarifies falsification boundaries, and strengthens governance implications. Concept DOI unchanged.

Fields of Study & Subjects

Ethics and Social Impacts of AI Artificial Intelligence in Healthcare and Education Embodied and Extended Cognition

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
1 January 2026
Document type
Preprint