Error in Data and Making Data Clean

Error, in relation to data, means any observation that deviates from a true state. Statistical errors include sampling errors, which are related to the size and composition of the dataset, and non-sampling errors, which may be due to instrumentation, miscalculation, and/or the conscious or unconscious bias of the researcher. While it might be nice to think that errors can be eliminated entirely, anyone who has ever tried to collect even the most simplistic and straightforward data knows that this is not the case. Rather than eliminate errors, we must try to reduce them as much as possible and then understand and account for what remains. In this chapter, we look at several strategies for reducing errors, such as identifying and eliminating outliers in our dataset, and how the remaining error can be quantified by estimating inter- and intraobserver error. This chapter also looks at what to do about missing data, including how to use imputation to replace missing values. The end goal of these tasks is a “clean” dataset ready for analysis in Chapter 5 .