Auditing Conformal Prediction under Distribution Shift: A Detectability Boundary, Exact Label-Budget Design, and Repair

Abstract Conformal prediction quantifies uncertainty without parametric outcome assumptions, but standard coverage relies on exchangeability. Importance weighting can restore validity under covariate shift when density ratios are known. In deployment, practitioners often have labeled source data and only unlabeled target covariates, raising a basic question: can those data determine whether coverage remains credible? We introduce DriftGuard, a two-stage audit. Its label-free stage reports domain discriminability, effective sample size of estimated density ratios, weight stabilization, and visible abstention. We establish exact and approximate detectability bounds, a minimax testing consequence, and a sharp $[0,1]$ identified set for target coverage under unrestricted conditional shift. DriftGuard-L crosses this boundary with a probability sample of target labels, using an exact one-sided coverage test, prospective label-budget calculation, and finite-sample conformal recalibration. Under strong Gaussian covariate shift, ordinary conformal coverage fell to 0.789, while oracle and cross-fitted weighted conformal prediction attained 0.926 and 0.923. A published doubly robust comparator was robust when either nuisance was oracle, whereas both-misspecified coverage fell to 0.812. In an untouched chronological deployment of 8,734 hourly Bike Sharing observations, source conformal, estimated weighting, and estimated doubly robust calibration undercovered (0.568--0.594); 2,176 delayed target labels raised subsequent coverage from 0.558 to 0.884. Under pure concept shift, an overlap score remained near 0.96 while coverage fell from 0.90 to 0.40. Exact label budgets quantified power, and target recalibration restored synthetic evaluation coverage. The framework separates overlap diagnosis, visible abstention, target-label falsification, and repair.