The Unfaithful Cut: Why Some Intervention Failures Are Undetectable from Observation Alone

Predictive models are routinely evaluated by their ability to reproduce observed data. However, many models that perform well under observational evaluation fail when used for intervention or control. This paper identifies and formalizes a fundamental source of this discrepancy. We introduce the unfaithful cut: a representational reduction that is closed under prediction but not closed under intervention. Such reductions collapse distinct system histories into identical interface states, preserving observational accuracy while erasing distinctions that later become operationally relevant under admissible interventions. In the presence of an unfaithful cut, intervention outcomes are fundamentally underdetermined by observational data alone, even with infinite data and perfect predictive accuracy. The contribution of this work is diagnostic and epistemic, not algorithmic. The unfaithful cut does not identify failure modes beyond those recognizable as causal insufficiency; rather, it provides a representational perspective that makes explicit how such insufficiency arises from interface design choices in learned systems. This perspective is particularly relevant in reinforcement learning, robotics, and control, where causal structure is often implicit, underspecified, or impractical to model explicitly. The paper: Formalizes prediction closure and intervention closure as structural properties of representational reductions Shows why memory-augmented or non-Markovian models may restore predictive performance without restoring interventional reliability Demonstrates, through worked examples and a minimal offline-RL illustration, how intervention failures can remain invisible under logged data distributions Clarifies why certain intervention failures are provably undetectable without acting on the system The framework does not propose new learning algorithms, causal discovery methods, or guarantees of tractable detection. Instead, it delineates a necessary boundary condition for trust under intervention: if a representation induces an unfaithful cut relative to the deployment context, reliable control is impossible in principle, regardless of training data or optimization. This work provides a unifying conceptual lens on well-known failures across sim-to-real transfer, offline reinforcement learning, and causal modeling, and offers a principled explanation for why active intervention or experimentation is sometimes unavoidable for safe deployment.