Data Quality Awareness: A Journey from Traditional Data Management to Data Science Systems

In this paper, we present a comprehensive review of the evolution of data quality awareness from traditional data management systems to modern data-driven ML systems, which are integral to data science. We synthesize the existing literature, highlighting the quality challenges and techniques that have evolved from traditional data management to data science, including Big Data and ML fields. As data science systems support a wide range of activities, our focus in this paper lies specifically in the analytics aspect driven by ML. We use the cause-and-effect connection between the quality challenges of ML and those of Big Data to allow a more thorough understanding of emerging DQ challenges and the related quality awareness techniques in data science systems. To the best of our knowledge, our paper is the first to provide a review of DQ awareness spanning traditional and emergent data science systems.