Measuring What Retrieval Misses: Probe-Calibrated Recall and Absence Estimation Without Assessing the Unretrieved Segment

Deployed retrieval systems routinely need two numbers they cannot currently obtain: an estimate of the relevant material they failed to retrieve, and a calibrated statement of what a run of failed searches does — and does not — imply about absence. The measurement tools that exist each require a precondition that deployment does not meet: sample-based recall estimation requires human assessment of documents drawn from the unretrieved segment; pooling-based evaluation requires a fixed judged pool; learned truncation requires labeled training data; conformal risk control requires a labeled calibration set. This note describes a measurement protocol that replaces these preconditions with one the target field can actually satisfy: ground truth by construction. Queries generated from documents yield known-relevant probes; planted decoy documents yield a measured false-positive rate; an occupancy-class estimator with a document-level covariate turns repeated, varied searches into detection probabilities and posterior estimates of missed relevant mass with prediction intervals; a mandatory adequacy statistic, computable only because probes exist, indicates when the covariate model is misspecified — the protocol's own validity conditions are part of its output. The construction composes four measurement traditions (spike-in standards, target–decoy, site-occupancy estimation, empirical null calibration). In a pre-registered simulation the estimator attains small bias and stated interval coverage where its assumptions hold, degrades by approximately the hidden mass where they are violated, and flags the violation. Two pre-registered transport tests on a public collection (v2 §4b; v3 §4c) then measure the protocol's remaining exposed assumption — that probe-calibrated detection probabilities transfer to real queries — and report it broken in both error modes tested, for a lexical and for a dense retriever, loudly rather than silently. Two findings reframe the assumption rather than merely failing it: two independent judgment sources for "real" relevance disagreed with each other more than the probes disagreed with either; and the probe-to-real gap is larger, not smaller, for the dense retriever, so probe difficulty is a property of the pair (probe construction, retriever) and not of the probes. Absence calibration is therefore stated relative to a named query population, a named judgment provenance and a named retriever.