The Approximation-of-Approximation Problem

The more capable artificial intelligence systems become, the more they are trained on their own outputs—and the more they risk losing contact with the reality they were built to model. For decades, machine learning operated under an implicit contract: models approximate an external reference through data, and errors remain correctable because that reference persists. A structural shift has recently occurred. Increasingly, learning systems are trained not on reality or human-generated symbols, but on the outputs of other learning systems. Because these systems are themselves approximators, this introduces a recursive loop in which approximation becomes self-referential. This paper names and analyzes this emerging failure mode—the Approximation-of-Approximation Problem (hereafter A²). We argue that this problem is distinct from hallucination, statistical model collapse, or data bias, and that it cannot be resolved through scale alone. The paper contributes a diagnostic vocabulary for an emerging training regime whose epistemic implications are not yet fully articulated, and specifies the formal requirements any resolution must satisfy.