Effect of percent non-detects on estimation bias in censored distributions

Uniqueness of the problem surrounding non-detects has been a concern alike to researchers and statisticians dealing with summary statistics while analyzing censored data. To incorporate non-detects in the estimation process, a simple substitution by the MDL (method detection limit) and the maximum likelihood estimation method are routinely implemented as standard methods by the US EPA laboratories. In situations, where numerical standards are set at or near the MDL by regulatory agencies, it is not only prudent but also important to closely investigate both the variability in test measurements and the estimation bias. It is because an inference based on the biased estimates could entail significant liabilities. The variability is understood to be not only inevitable but also an inherent and integral part of any chemical analysis or test. In situations where regulatory agencies fail to account for the inherently present variability of test measurements, there is a need for regulated facilities to seek remedial action merely as a consequence of inadequate statistical procedure. The paper utilizes the mathematical approach to derive the bias function for substitution method and furthermore bias curve are developed to investigate the censored samples from a variety of probability distributions such as the normal, lognormal, gamma, and Gumbel distributions. The bias functions and bias curves are also verified by using the Monte Carlo simulations. To assess the influence of percent non-detects on the estimation bias, the censoring rate was used as an independent variable. Results show that half detection limit method yields very reasonable estimates of the mean and standard deviation for a censoring rate of 48% and 32% respectively for samples from censored Normal distribution. For a censoring rate of 73% and 61% respectively, the method yields reasonable estimates of the mean and standard deviation for samples from the Log Normal distribution. For the gamma distribution, the limits of censoring rate are found to be 84% and 62% respectively for the mean and standard deviation. The half-detection limit was not found to be suitable for the Gumbel distribution because a significant bias was found to exist even at a considerable low censoring rate.