Visibility of Domain Elements in the Elicitation Process Interviews: A Family of Empirical Studies

Various factors determine analyst effectiveness during elicitation. While the literature suggests that elicitation technique and time are influential factors, other attributes could also play a role.Aim: Determine aspects that may have an influence on analysts’ ability to identify certain elements of the problem domain.Methodology: We conducted 14 quasi-experiments, inquiring 134 subjects about two problem domains. For each problem domain, we calculated whether the experimental subjects identified the problem domain elements (concepts, processes, and requirements), i.e., the degree to which these domain elements were visible.Results: Domain element visibility does not appear to be related to either analyst experience or analyst-client interaction. Domain element visibility varies depending on how analysts express the information they have elicited. When analysts are directly asked about the knowledge they acquired during elicitation, visibility increases substantially compared to when they describe the information in a written report.Conclusions: Further research is required to replicate our results. However, the finding that analysts have difficulty reporting the information they have acquired is useful for identifying alternatives for improving the documentation of elicitation results. We found evidence that other issues, like domain complexity, the relative importance of different elements within the domain, and the interview script, also seem influential.