Thirteen Descriptive Corrections and Evidence-Grade Re-Adjudication: AI Distillation Applications v15 — Applied Distillation Science
Abstract
This work is part of the Applied Distillation Science (蒸留応用学) series on AI distillation with local language models. Don't take our word for it — the per-draft supplement, the pre-registrations,and (from this version) the frozen scorer and all analysis scripts are here.Download and check. v15 makes two kinds of changes, and separates them explicitly. (1) Descriptive corrections: 13 statements (3 known since v13, 10 more foundby a systematic coverage audit of ~1,013 numeric claims; the audit report andits denominator method ship with this version). No frozen measurement valueand no STUDY7/8 scientific adjudication was changed by any of them. Notably,the audit found that our own scope scan's claim that v12/v13 were "clean" wasitself wrong (2 same-type errors found there), and that an audit document —not just prose summarizing it — can over-generalize its own scope. (2) Evidence-grade re-adjudication: our standard's condition 6 requires thatthe frozen scorer and raw outputs be publicly retrievable at adjudicationtime. They were not. Applying the strict reading, all four legacy procedural-Afindings are set to legacy_final_grade = UNRESOLVED_UNDER_V1 withA_eligibility = WITHHELD (reason: CONDITION_6_PUBLIC_RETRIEVABILITY_FAILURE).The procedural A count at v15 is therefore 0. This is a grading decision onthe public-reproducibility axis, not a change to any measurement. What we do about it: the frozen scorer (q_metric_v1.py) and all 35 analysisscripts are deposited with this version. Raw generated outputs are NOTdeposited yet: they contain all 26 held-out topics, and releasing them wouldpermanently burn the evaluation set that every baseline in this series restson. Instead, a series-end deposit trigger is frozen in the record: when norunning, pre-registered, or queued study still references the holdout set (orbaselines measured on it), the raw outputs, scorer, scripts, and SHA manifestare deposited as one bundle, and the 26 topics are never reused as cleanholdout afterwards. Depositing later does not retroactively revive the old Agrades — that failure is recorded as historical fact. The v14 record remains preserved and is not overwritten. File layout: 43 files are uploaded individually (8 documents, 35 analysisscripts) plus UPLOAD_SET_SHA256.txt listing the SHA-256 of each.v15_upload_set.zip is a convenience bundle of the same 43 files — identicalcontent, verifiable against the same SHA list. This version restores the full 43-file deposit plus SHA manifest and zip bundle alongside 00_README_EN.md (English overview, sorted first). The immediately preceding version contained only the README due to an upload error; measurements and files are otherwise identical to v15. Previously titled: "Distillation Applications v15: thirteen descriptive corrections, evidence-grade re-adjudication (procedural A = 0), scorer and analysis scripts deposited"
Fields of Study & Subjects
Benford’s Law and Fraud Detection
Taxation and Compliance Studies
Crime
Illicit Activities
and Governance
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 11 August 2026
- Language
- JA
- Document type
- Preprint