Identifiability Limits of Boundary-Relative Audit Quantities Under Noisy Verification

Read full text
Authors: Asaad Riaz
Cited by: 0

Abstract

This paper analyzes the fundamental identifiability limits of deployment-level audit metrics defined relative to external verification boundaries when audit judgments are noisy. Many AI audits report quantities such as uncertifiable input mass, correctness rates, or semantic divergence based on finite verification procedures. These quantities are boundary-relative by design: they depend on what evidence is admissible under a declared audit protocol, not on internal model signals or oracle access to ground truth. When audit outcomes are imperfect, as they are in practice, this creates a nontrivial inference problem. The paper establishes that: Without explicit bounds on audit noise, boundary-relative audit quantities are not identifiable from audit data, even with infinite samples. With declared audit-noise caps, these quantities are partially identifiable, and the strongest possible conclusions take the form of tight identification intervals rather than point estimates. Finite-sample confidence intervals can be constructed that correctly separate sampling uncertainty from an irreducible audit-imposed resolution limit. The same identifiability structure applies across different audit tasks, including certifiability audits and correctness audits. A new cross-audit grounding-consistency condition links independently audited quantities under separate noise guarantees, providing a falsifiable diagnostic for incoherent or non-credible audit claims. The mathematical results are closely related to classical misclassification and partial-identification theory. The contribution lies in establishing their operational and compliance-level consequences for AI auditing, clarifying which deployment-level claims are supported by audit data under explicit assumptions and which are not.

Fields of Study & Subjects

Ethics and Social Impacts of AI Explainable Artificial Intelligence (XAI) Safety Systems Engineering in Autonomy

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
10 January 2026
Language
ENC
Document type
Preprint