NLP-Based Methods for Conflict Identification in Software Requirement Engineering

Recent advancements in Natural Language Processing (NLP), specifically Large Language Models (LLMs), have demonstrated their transformative power in various domains, including software engineering. This thesis focuses on addressing the critical problem of conflict and duplicate requirement identification in requirement engineering, leveraging the potential of structured and non-structured data in software engineering for automation. For this purpose, first, a diverse set of requirements are annotated for entity recognition tasks, enabling the creation of machine learning and transformer-based models which provides 95.8% accuracy in identifying software-specific entities from requirement texts present in DOORS dataset. Second, building upon this foundation, a two-phase algorithm is developed for conflict detection in requirement documents. The algorithm utilizes supervised learning on requirement data to identify potential conflicts and subsequently validates them semantically using entity extraction techniques. Third, to further refine the conflict and duplicate identification process, the problem is formulated as a sentence pair classification task. Transfer learning techniques, such as sequential transfer learning and cross-domain transfer learning, are employed, and a novel architecture called Software-Requirement Bi-directional Encoder Representation Transformer (SR-BERT) is proposed for requirement pair classification. SR-BERT achieves 95.3% F1-score in determining conflicts, duplicate, and neutral requirement pairs. Additionally, misclassifications flagged by cross-domain transfer learning are reevaluated using Actor-Action (AA) extraction, Part of Speech (POS) tagging, and Semantic Role Labeling (SRL) techniques. The requirement pair classification approach demonstrates great potential in conflict detection tasks, and its performance is enhanced by integrating bi-directional encoders with a contrastive learning framework and cross-encoders. Fourth, inspired by recent advancements in prompt-based learning, a Prompt-based Text-to-Text Models for Software Requirement Pairs (PT2-SRP) framework is also developed, leveraging specially crafted prompts for requirement pair datasets and training encoder-decoder models to identify conflicts in requirement pairs. Lastly, the impact of Data Augmentation (DA) on requirement conflict identification is examined. Specif ically, during the curation of datasets for requirement pair classification task, a scarcity of conflict requirement pairs is noted. To overcome this limitation, novel data augmentation techniques are devised, which ensure the generation of meaningful data instances while preserving the class labels in requirement pair datasets.