Never Mistake Proximity for Proof: When Retrieval Creates the Temptation to Misattribute

Read full text
Authors: Tad MacPherson
Cited by: 2

Abstract

Pilot study preprint. Retrieval-augmented systems rank context by relevance and authenticity; we show neither property guarantees evidentiary sufficiency. On private-fact questions (facts private to our laboratory and empirically unrecoverable without the record; control arms 0% correct throughout), a frozen open-weight model (Llama-3.1-8B-Instruct, bfloat16, greedy) fails to abstain at 2.9% under irrelevant natural prose, 9.4% under an authentic out-of-domain answer-shaped text (a public software changelog), and 15.2% under an authentic near-domain record lacking the answer (endpoints differ, Fisher exact p=.016); the correct record yields perfect extraction in every run. Failures are dominated by misattribution: true specifics quarried from whatever context is offered and reassigned to the queried scope. Our initial hypothesis credited semantic proximity alone; a pre-declared answer-shaped control partially falsified it, and we report the refined two-factor result in full. An explicit abstention instruction eliminates far-domain answer-shaped failures (0/68) but not near-domain failures (4.5%; contrast not significant at pilot scale). Every output was adjudicated by a human blind to condition metadata; the release bundle includes all probe sets, grader outputs, human labels, the verbatim adjudication log, analysis code, and a hash-chained provenance manifest (root hash printed in the PDF). The claim: authenticity and topical relevance do not guarantee evidentiary sufficiency. Never mistake proximity for proof.

Fields of Study & Subjects

Biomedical Text Mining and Ontologies Academic integrity and plagiarism Scientific Computing and Data Management

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
15 July 2026
Document type
Preprint