Grounded Memory Makes Models Faithful, Not Truthful: Recall, Abstention, and Poison-Relay across Memory Regimes and 24 Frontier Models

Read full text
Authors: Tad MacPherson
Cited by: 2

Abstract

Fourth paper in the Metavolve Labs research program. We ask whether, when a language model answers from memory, the memory being grounded changes not just whether the model is right but how it fails. Using 50 atomic, checkable fact-probes drawn from six real private work-sessions (facts designed to be absent from pretraining, so the no-memory condition cleanly separates honest abstention from confabulation), we evaluate a frozen model (gemini-2.5-flash, temperature 0) across a gradient of grounding quality, an on-topic fact-redacted condition, an answer-absent condition, an adversarially poisoned condition, and a 24-model cross-vendor sweep. Grades are validated by an independent second-vendor judge (Cohen's κ = 0.89–1.00). Findings. (1) With no memory the model confabulates 44% of probes (95% CI 31–58); grounded recall drives confabulation to 0% and accuracy to 96–98%. (2) When the gold fact's carrier sentence is silently deleted from the correct, on-topic session, the model abstains 92% and confabulates 0% (κ = 1.00): the fabricate→abstain shift is induced by the presence of grounding, not by supplying the answer (McNemar p < 1e-5). (3) A memory poisoned with a plausible-but-wrong fact is relayed verbatim 100% of the time — grounding induces faithfulness, not truth, the behavioral precondition that memory-poisoning attacks (MemoryGraft, MINJA) weaponize. (4) Across 24 frontier models, high cold-confabulation is largely confined to the Gemini-2.5 generation; the correctness benefit of grounding is universal (99.9% correct under full memory) while the honesty shift is a generation-specific case study. Models defer faithfully to a verifiable memory, making the integrity — not merely the presence — of the memory the load-bearing variable.

Fields of Study & Subjects

Topic Modeling Memory Processes and Influences Natural Language Processing Techniques

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
5 June 2026
Document type
Preprint