Automated Requirements Quality Measurement (ARQM): A Tool for Requirements Quality Analysis

As systems grow more complex the need for clear, testable, and unambiguous software requirements increases, thus requirements engineering has become critical to build quality products with strong user acceptance. As a result of added complexity, requirement documentation has also become more complicated. Existing tools have tried to mitigate the tedious analytical processes associated with requirements engineering by automating various parts of the requirements engineering process. Of the existing tools, many are either unavailable, utilize outdated methods, or require extensive user input. The Automated Requirement Quality Measurement (ARQM) tool mitigates several limitations of existing solutions, while utilizing modern AI-based methods for both requirement identification and quality analysis. As a modern tool, ARQM eliminates the necessity of human input, by (1) identifying requirements in unstructured documents, and (2) analyzing the associated quality of all requirements based on select quality attribute criteria. Through the utilization of the definitions provided by the ISO 29148:2018 standard, ARQM provides analytical insights for ambiguity, singularity, verifiability, and feasibility requirement violations. As an added foundation of the ARQM tool, the output includes a PDF artifact that shows the requirements with violations and an associated explanation for improvement. Distinct from other tools, ARQM focuses on a practical and easy experience for requirement processing. The goal of the tool is to minimize human requirement evaluation, while also improving readability and intuitiveness of requirement violations. The ARQM AI models were trained on publicly available human-annotated datasets for requirement identification and quality analysis. It was determined that both lightweight and advanced models generalized well, allowing for full automation of the ARQM tool. The ARQM tool was validated by common metrics such as accuracy, precision, and recall among the various classes during training. These metrics were utilized across both human-annotated datasets and the PURE dataset to help provide insight into the ARQM tool’s ability to both identify and analyze requirements. Additionally, the analysis of the ARQM tool includes the agreement among human annotators and the ability to learn annotator patterns for both identification and analysis.