Geometric Taxonomy of LLMs Hallucinations
Abstract
The term "hallucination" in large language models conflates distinct phenomena with different geometric signatures in embedding space. We propose a taxonomy identifying three types: unfaithfulness(failuretoengagewithprovidedcontext), confabulation(inventionofsemanticallyforeign content), and factual error (incorrect claims within correct conceptual frames). We observe a striking asymmetry. On standard benchmarks where hallucinations are LLM-generated,detection is domain-local: AUROC 0.76–0.99 within domains, but 0.50 (chance level) across domains. Discriminative directions are approximately orthogonal between domains (mean cosine similarity -0.07). On human-crafted confabulations — invented institutions, redefined terminology, fabricated mechanisms — a single global direction achieves 0.96 AUROC with 3.8%cross-domain degradation. We interpret this divergence as follows: benchmarks capture generation artifacts (stylistic signatures of prompted fabrication), while human-crafted confabulations capture genuine topical drift. The geometric structure differs because the underlying phenomena differ. Type III errors show 0.478 AUROC — indistinguishable from chance. This reflects a theoretical constraint: embeddings encode distributional co-occurrence, not correspondence to external reality. Statements with identical contextual patterns occupy similar embedding regions regardless of truth value. The contribution is a geometric taxonomy clarifying the scope of embedding-baseddetection: TypesIandIIaredetectable; TypeIIIrequiresexternalverification mechanisms.
Fields of Study & Subjects
Topic Modeling
Misinformation and Its Impacts
Adversarial Robustness in Machine Learning
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 29 January 2026
- Document type
- Preprint