Why Selective Verification Poses Structural Risks in Open Domains and How Scalable Evaluation May Mitigate Them
Abstract
Selective human verification—allocating scarce judgment to high-risk regions—is defensible in closed domains with stable priors and bounded hypothesis spaces. We show that in open domains—characterized by sustained conceptual novelty, absent priors, and long deployment horizons—the judgment-allocation rule itself becomes underdetermined, inducing allocation ambiguity: the systematic accumulation of unevaluated regions in behavioral and conceptual space. We formalize this failure mode under a stylized functional abstraction of prevailing evaluation pipelines, the Centralized Opaque Propagation Model (COP-Model), in which alignment-relevant knowledge is discovered locally but integrated centrally. Under sustained novelty, discovery rates exceed integration capacity, inducing an effectively bushy evaluation knowledge graph. Within this model and regime, we establish a formal separation result—absent in prior literature under these jointly specified conditions—showing that all bounded local-selective evaluation methods suffer asymptotic coherence collapse due to invalid inference casts from local evidence to global guarantees. We then introduce an alternative regime: compiling local assessments into a strongly typed axiom set and applying a \emph{global logical coherence operator} that enforces admissibility constraints at the system level. Under the same functional assumptions, this regime exhibits a critical-mass phase transition to bounded false negatives as evaluative capacity scales, detecting inadmissibility where local methods provably fail. If the COP-Model remains a representative abstraction of real-world evaluation architectures, these results identify a structural limitation of judgment-limiting approaches and motivate a shift toward judgment scaling via interpretable conceptual spaces and genuinely decentralized collective intelligence infrastructure. Proofs and simulations are
Fields of Study & Subjects
Ethics and Social Impacts of AI
Explainable Artificial Intelligence (XAI)
Evaluation and Performance Assessment
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 4 February 2026
- Document type
- Preprint