Corrupted Provenance Degrades Attribution Continuously, Not Catastrophically

Read full text
Authors: CHRISTOPHER SWEENEY
Cited by: 6

Abstract

A natural hypothesis about provenance metadata is that corruption is tolerable up to a point and then catastrophic — that there exists a critical ratio of false-to-true attribution beyond which a model's ability to attribute content collapses. This paper tests that hypothesis directly and finds no such threshold. Twenty GPT-2 small (124M) models were fine-tuned on the same 40,015-sentence Wikipedia corpus with source tags corrupted at nine rates from 0% to 100%, then evaluated on source recovery against held-out text drawn from articles never seen during fine-tuning. For each rate, a random fraction of sentences has its source tag replaced with another sentence's title; the replacement is a derangement within the corrupted subset, so no sentence designated as corrupted accidentally keeps its own source, and the overall multiset of titles is preserved exactly. THE RESULT Degradation is smooth, monotone, and strongly convex. Segment slopes accelerate at every one of eight steps — from -0.029 to -3.800 accuracy points per percentage point of corruption, a 131-fold acceleration — with no discontinuity anywhere. corruption: 0% 25% 50% 75% 85% 90% 95% 99% 100% recovery: 75.38 74.65 72.70 68.70 64.17 60.50 52.90 42.65 38.85 Two candidate explanations were available and both are refuted. A critical threshold requires a discontinuity; there is none at any of eight sampled intervals. A boundary effect at 100% — the possibility that a model exploits any nonzero clean signal, so that only the total absence of signal destroys the capability — predicts that the final segment should dominate. It is instead the mildest acceleration in the sequence, at 1.48 times the preceding step. The rates most load-bearing for the shape claim, 85% and 95%, were each replicated across three independent corruption draws and three training seeds. Standard deviations across replicated arms are 0.47 to 0.87 accuracy points against a total curve range of 36.5 points. WHAT IT IS GOOD FOR The practical content is asymmetric. Low corruption is nearly free: half the training corpus can carry false source tags for a cost of 2.7 accuracy points. High corruption is ruinous per unit: the final percentage point costs 3.8 points on its own, more than the first fifty percentage points combined. There is no threshold to engineer against, because there isn't one — what the curve licenses instead is a marginal-value argument, in which effort spent improving provenance quality has increasing returns as quality rises. A METHODOLOGICAL RESULT The first five rates sampled were 0, 25, 50, 75 and 100%. On that grid the same underlying phenomenon places 80.3% of the total decline in the final interval, with a last step 7.3 times the one before it. It looks exactly like a phase transition, and nothing internal to the five-point data exposes the error: the curve is monotone, the effect is large, the endpoints are replicated across three seeds each, and the collapse falls precisely where a capacity-limit account predicts. The entire appearance is produced by the choice of sampling grid. Adding four points inside that final interval reduced the last step ratio from 7.3x to 1.48x and eliminated the threshold completely. The generalisable point: sparse sampling manufactures thresholds. Any monotone convex curve, sampled on a grid whose final interval spans the region of greatest curvature, will present as a phase transition. Claims of critical thresholds require sampling density chosen after the region of curvature is located, and the check is cheap — subdivide the interval containing the apparent discontinuity and see whether it survives. Here that cost four additional training runs and under two hours. LIMITATIONS The low-corruption region is not resolved: the largest within-arm standard deviation (0.87 points) exceeds the smallest step between adjacent rates (0.73 points, between 0% and 25%), so the three lowest points cannot be confidently ordered relative to one another. Four of nine rates are single-seed; the two chosen for replication were those on which the shape claim most depends. Recovery is measured closed-set, with the true source always among the 50 candidates, so these are not open-world attribution rates; the companion work measures the open-set case and finds it substantially harder (AUC 0.785, falling to 0.632 without lexical overlap), and whether the convex shape reported here holds under an open-set metric is untested. Most attribution ability measured here is inherited from pretraining rather than taught — a model trained with no tags at all recovers sources at 64.7%. Single model scale, single corpus, single architecture; whether the position of the steep region transfers is untested. This work is a companion to "Provenance Conditioning Generalizes to Unseen Sources" (10.5281/zenodo.21782935), which established that source attribution is information-bearing, that the effect generalizes to sources unseen in training, and that training on false attribution costs roughly 2.4 times what correct attribution contributes. The present paper measures the interior of that endpoint comparison. Code and data-generation scripts are included for full reproduction. VERSION HISTORY Version 2 (this version) — no change to the experiment or its results. Corrects the deposit: version 1 mistakenly attached the companion paper's PDF alongside this one, carried a superseded build of this paper, and omitted the open-set evaluation script and results. The limitations section now cites the companion work's open-set measurement rather than leaving the closed-set caveat unresolved. Version 1 — the nine-point dose-response curve and the sampling-artifact result, as reported here.

Fields of Study & Subjects

Scientific Computing and Data Management Machine Learning in Materials Science Topic Modeling

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
4 August 2026
Document type
Preprint