Generativity, Composition, and the Propagation Problem: A Structural Limit on Risk Evaluation in Open Domains

Read full text
Authors: Andy E Williams
Cited by: 0

Abstract

We establish a structural limit on risk evaluation in open domains. When a domain generates novelty through composition, any model adequate to represent future risks must be compositionally closed. Such models generate hypotheses faster than finite-capacity evaluators can test them. We prove that under these conditions, external validation faces an unbounded hypothesis space with no termination condition, while internal validation---conditional adoption followed by coherence testing---faces a finite test battery. The decision-theoretic consequence is that refusing to test compositionally closed models in domains with existential stakes introduces unbounded regret. This is not speculation about specific risks; it is a theorem about the relationship between model structure and evaluation capacity. We identify a class of hypotheses---those concerning compositionally closed frameworks for AI alignment and related domains---that are well-specified, potentially existential, and untested. The propagation problem predicts this non-testing; the non-testing constitutes evidence of a structural blind spot in evaluation capacity.

Fields of Study & Subjects

Ethics and Social Impacts of AI Diverse Interdisciplinary Research Studies Safety Systems Engineering in Autonomy

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
30 January 2026