An automatic finite-sample robustness metric: when can dropping a little data change conclusions? Part II: theory and intuition

In Part I, we propose a method to assess the sensitivity of applied conclusions to the removal of a small fraction of the sample; we call our metric the approximate maximum influence perturbation (AMIP). In this article, we support the intuition and accuracy of our proposed AMIP sensitivity with theory. For intuition, we illustrate that AMIP sensitivity is driven by a signal-to-noise ratio in the inference problem and is distinct from common existing forms of sensitivity. In particular, we show that AMIP sensitivity is not reflected in standard errors, does not disappear asymptotically, is not due to misspecification and differs from gross-error robustness. For accuracy, we provide finite-sample error bounds on the performance of AMIP as an approximation of the worst-case removal of a small fraction of data. Our results suggest that AMIP is complementary to existing robustness checks and we recommend its use as part of a suite of checks a user should run on their data analysis. This article is part of the theme issue 'Statistical workflow'.