Identifier Stability Is Not Referent Stability: Failures of Attribution in Behavioural AI Evaluation
Abstract
Reports five cases, discovered within one evaluation estate over five months, where an evaluation identifier continued to resolve while its referent changed. A model name kept resolving after the served model was replaced. Probe identifiers shared between two benchmark versions carried different definitions. One evaluation appeared as three database records. A compliance detector classified refusals as acceptances. The paper's own drafting attributed one date's evidence to another. Each failure has the same structure: the identifier remained syntactically valid while what it denoted changed silently. The paper specifies the minimal attestation record required to make evaluation results attributable after the fact, and documents that routine response truncation destroyed the evidence needed to audit detector classifications retrospectively.
Fields of Study & Subjects
Ethics and Social Impacts of AI
Explainable Artificial Intelligence (XAI)
Adversarial Robustness in Machine Learning
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 29 July 2026
- Document type
- Preprint