Demystify doubly-robust estimation: the role of overlap

Abstract Objectives Doubly-robust (DR) estimators are widely used for evaluating causal effects in observational studies and are often perceived as more desirable than propensity score weighting, such as inverse probability weighting (IPW), or outcome modeling alone, because they provide extra protection against model misspecification. However, double robustness is an asymptotic property that may not hold in finite samples. We investigate how the finite sample performance of DR estimators depends on the degree of covariate overlap between comparison groups. Methods Using analytical illustrations and extensive numerical studies with synthetic and real-world data, we examine the bias and variance of two representative doubly-robust estimators, the canonical DR (augmented IPW) estimator and the targeted maximum likelihood estimator (TMLE), relative to IPW and outcome regression estimators under different degrees of covariate overlap and model specifications. Results We find that: (i) specification of the outcome model has a stronger influence on DR estimates than specification of the propensity score model, and this dominance increases as overlap decreases; (ii) with poor overlap, DR estimators generally amplify the hazards of extreme weights (large bias and/or variance) regardless of model specifications, and can be inferior to IPW and outcome regression estimators. Conclusions As a practical guide, we recommend always first checking the degree of overlap in applications. In the case of poor overlap, analysts should consider shifting the target estimand from the full study population to a subpopulation with adequate overlap via, e.g., propensity score trimming or overlap weighting, or conducting subgroup analysis.