High-quality versus low-quality unit tests: a controlled experiment on developer performance

Abstract This paper presents the results of an empirical study investigating the impact of unit test quality on the development time, the ability to solve all tasks, the perceived software understanding, and the presence of adequate feedback. For this purpose, an application was developed with two test suites, one containing high-quality tests and the other low-quality tests. The study was performed on 53 graduate students of computer science, who were randomly divided into three groups: students with high-quality tests (Q), students with low-quality tests (L), and students with no tests (N). The goal was to observe the differences between the groups when solving different tasks. Based on the results obtained, it can be said that a significant time difference could be discovered between the groups Q and L, as the Q group completed the individual tasks in a shorter time. In addition, a connection between the presence of quality tests and having adequate feedback has been found, though there was no significant difference in perceived code understanding or ability to solve all tasks.