Generativity, Composition, and the Propagation Problem: A Structural Limit on Risk Evaluation in Open Domains
Abstract
We establish a structural limit on risk evaluation in open domains. When a domain generates novelty through composition, any model adequate to represent future risks must be compositionally closed. Such models generate hypotheses faster than finite-capacity evaluators can test them. We prove that under these conditions, external validation faces an unbounded hypothesis space with no termination condition, while internal validation---conditional adoption followed by coherence testing---faces a finite test battery. The decision-theoretic consequence is that refusing to test compositionally closed models in domains with existential stakes introduces unbounded regret. This is not speculation about specific risks; it is a theorem about the relationship between model structure and evaluation capacity. We identify a class of hypotheses---those concerning compositionally closed frameworks for AI alignment and related domains---that are well-specified, potentially existential, and untested. The propagation problem predicts this non-testing; the non-testing constitutes evidence of a structural blind spot in evaluation capacity.
Fields of Study & Subjects
Ethics and Social Impacts of AI
Diverse Interdisciplinary Research Studies
Safety Systems Engineering in Autonomy
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 30 January 2026