Artificial Confabulation and Untrustworthiness: The Structural Limits of Transformer-Based, Next-Token AI Architectures
Abstract
Transformer-based large language models (LLMs) are structurally incapable of verification – defined here as architectural veto authority to halt or refuse output under conditions of irreducible ignorance (V) a core component of epistemic integrity. This paper argues that the architectural absence of V, coupled with the absence of consequence-bearing Experience (E), renders LLMs fundamentally untrustworthy, regardless of scale and performance on pattern-matching benchmarks. This incapacity also prevents organisations from sustaining Sovereign Intelligence, the ability to verify, override and remain accountable for decisions made with or by AI systems. While most research treats "hallucinations" as behavioural glitches to be solved, the KEV constraint (Knowledge is massive, Experience is absent, Verification is absent) dictates that scaling only intensifies the failure mode, which is taxonomically classified as Artificially Confabulating and Untrustworthy (ACU). This confabulation is not a defect but the architecture operating as designed: fluent completion without truth. This claim concerns model-internal epistemic capacity: tool use, retrieval and external artefacts can support system-level checking, but they do not create internal verification capability and therefore cannot be treated as self-verification. The paper draws a parallel with clinical confabulation, documents the compounding failure stack (Speed-Weighted Reasoning Collapse → Architectural Confabulation → Uncalibrated Confidence) and presents systematic adversarial interrogation across frontier systems to demonstrate cross-model repeatability. Ensembling, debate and multi-agent coordination may reduce error rates in some tasks; however, agreement among generators is not evidence of truth and does not resolve the absence of verification (V = 0). The path forward is not autonomous AI but Governed Hybrid Intelligence (GHI): human judgement augmented by machine synthesis, constrained by external verification that supplies the grounding the architecture lacks. Without GHI, scaling produces more sophisticated confabulation, not more reliable intelligence. This paper concludes that contemporary AI deployment rests on an epistemic mirage. Truth does not emerge from these architectures. Where Sovereign Intelligence is not preserved, judgement authority necessarily migrates away from the human actor. Version note (v2.0): This published version incorporates minor clarifications and an expanded falsification appendix addressing common architectural misinterpretations, including claims related to recursion, self-questioning and memory.The core thesis, definitions and conclusions are unchanged.
Fields of Study & Subjects
Artificial Intelligence in Healthcare and Education
Ethics and Social Impacts of AI
Explainable Artificial Intelligence (XAI)
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 9 January 2026