Why Selective Verification Poses Structural Risks in Open Domains and How Scalable Evaluation May Mitigate Them

Read full text
Authors: Andy E Williams
Cited by: 0

Abstract

Selective human verification—allocating scarce judgment to high-risk regions—is defensible in closed domains with stable priors and bounded hypothesis spaces. We show that in open domains—characterized by sustained conceptual novelty, absent priors, and long deployment horizons—the judgment-allocation rule itself becomes underdetermined, inducing allocation ambiguity: the systematic accumulation of unevaluated regions in behavioral and conceptual space. We formalize this failure mode under a stylized functional abstraction of prevailing evaluation pipelines, the Centralized Opaque Propagation Model (COP-Model), in which alignment-relevant knowledge is discovered locally but integrated centrally. Under sustained novelty, discovery rates exceed integration capacity, inducing an effectively bushy evaluation knowledge graph. Within this model and regime, we establish a formal separation result—absent in prior literature under these jointly specified conditions—showing that all bounded local-selective evaluation methods suffer asymptotic coherence collapse due to invalid inference casts from local evidence to global guarantees. We then introduce an alternative regime: compiling local assessments into a strongly typed axiom set and applying a \emph{global logical coherence operator} that enforces admissibility constraints at the system level. Under the same functional assumptions, this regime exhibits a critical-mass phase transition to bounded false negatives as evaluative capacity scales, detecting inadmissibility where local methods provably fail. If the COP-Model remains a representative abstraction of real-world evaluation architectures, these results identify a structural limitation of judgment-limiting approaches and motivate a shift toward judgment scaling via interpretable conceptual spaces and genuinely decentralized collective intelligence infrastructure. Proofs and simulations are

Fields of Study & Subjects

Ethics and Social Impacts of AI Explainable Artificial Intelligence (XAI) Evaluation and Performance Assessment

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
4 February 2026
Document type
Preprint