Optimizing Requirements Coverage in Software Testing Through Large Language Model-Based Scoring

Ensuring high-quality requirements remains a critical challenge in software engineering, often leading to project delays and failures due to manual effort, inconsistencies, and ambiguities. This work addresses this gap by presenting an LLM (Large Language Model)-driven designed to automate requirement quality assessment. Central to our method, a Mistral-7B-Instruct model is augmented by a Retrieval Augmented Generation (RAG) system that dynamically incorporates predefined assessment criteria and technical documentation to guide prompts. An automated scoring system then precisely quantifies quality levels and provides actionable suggestions for enhancement, focusing on the assessment of requirement elements. Validation across two large-scale projects demonstrated the system's effectiveness, notably revealing that only 30% (3/10) and 11.8% (2/17) of analyzed Epics, respectively, met satisfactory quality thresholds. This emphasizes the critical need for automated quality gates to prevent the progression of poorly defined requirements, thereby strengthening requirement integrity, minimizing costly errors, increasing customer satisfaction and expediting development to foster more dependable and efficient software product delivery.