Against Smooth Attention: Why Language Models Hallucinate, What NMDA Receptors Know About It, and Three Things to Try (v2, with erratum)
Abstract
Version 2 adds a one-page erratum to the preprint of 29 March 2026. The text of version 1 is unchanged and is retained as published; the erratum states what is corrected and what is withdrawn. Corrections. Three citation-to-claim defects are corrected. The structural-hallucination sentence of Section 1 was cited to Vaswani et al. 2017, which makes no claim about hallucination; the citation is corrected to Kalai and Vempala 2024 (calibration) and Kalai, Nachum, Vempala and Zhang 2026 (evaluation incentives). The methamphetamine pathway of Sections 1 and 2.4 was cited to two sources that concern ketamine and alcohol withdrawal and not methamphetamine; the claim of three independent pharmacological pathways is reduced to the two that are documented. The five-step symptom cascade of Section 2.4 had no source for its first stage, which is withdrawn, and omitted the 100-patient staging series (Dalmau et al. 2008, Lancet Neurology 7(12):1091-1098) and the recovery cohort (Titulaer et al. 2013, Lancet Neurology 12(2):157-165), both of which are added. The bibliography key dehaene2011 is corrected to dehaene2014. Withdrawals. Section 6, the plasma-derived predictions (optimal ramp functions, the Troyon analogue, disruption avoidance, and the scaling-law phase transition), is withdrawn on the ground already stated in the paper's own Section 7.3. The novelty claim of Section 7.6 against existing sparse attention is withdrawn. Status of the central claim. The mechanistic claim that smooth attention is the structural origin of hallucination and that a learnable hard threshold in the attention kernel is the fix is not upheld by subsequent work of the author. Causal masking is already an exact zero; softmax-free attention with exact zeros is built and published (Leroux et al. 2025, Nature Computational Science 5(9):813-824); a hard threshold after a multiplicative gate turns a gain error into an offset error; per-head threshold spread produces input-independent selection. The successor work reframes the gap as a selection gap: the shared coordinate is present, a commitment step with memory and a persistent addressable label are absent. It is published as doi:10.5281/zenodo.22820231. Prior art added: Martins and Astudillo 2016 (sparsemax), Correia, Niculae and Martins 2019 (per-head learnable sparsity), Zhang, Titov and Sennrich 2021 (rectified attention scores), Leroux et al. 2025, Kalai and Vempala 2024, Kalai et al. 2026. Every reference added by the erratum carries a DOI verified against Crossref, the sparsemax entry against DataCite. Files: the erratum (PDF and LaTeX source), the unchanged version-1 source, and the changelog.
Fields of Study & Subjects
Cold Fusion and Nuclear Reactions
Functional Brain Connectivity Studies
Advanced Memory and Neural Computing
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 17 September 2026
- Language
- EN