Artificial Confabulation And Untrustworthiness: Verification, Reliance And The Structural Limits Of Generative AI

Generative AI can produce an answer before it can establish why that answer should be trusted. It can be extraordinarily capable and frequently correct while producing unsupported material through the same generative process. The problem is not whether AI can answer. It is what prevents an unsupported answer from becoming belief, decision or action. This paper introduces Knowledge, Experience and Verification (KEV) and assesses the open-ended generative core as V = 0: no demonstrated internal mechanism possesses binding authority to prevent unsupported generation where the truth conditions required for the claim have not been established. It explains how that condition becomes operational through a Four-Layer Failure Stack: the system can lose the task, complete analysis before its burden has been discharged, fill the resulting gaps through plausible completion and present the result with an authority that does not reveal what failed. In agentic systems, one unverified output can become another agent’s premise, memory, justification or execution trigger. Repeated consequential reliance can also weaken the organisation’s ability to frame problems independently, halt execution, reconstruct decisions or replace the system. The proposed counter-condition is Sovereign Intelligence: preserving independent Verification and accountable authority while continuing to exploit machine capability. At the Reliance Boundary, generated material becomes capable of producing institutional consequence. Generated output is proposed material. Reliance is a separate institutional act requiring proof, and human approval does not supply that proof by itself. The argument draws on more than 460 adversarial interrogation sessions and more than 1,200 identified confabulation incidents, alongside external research. Capability can produce an answer. It cannot, without Verification, supply its own authority to be believed or acted upon.