Investigating Requirements-Based Test Case Generation With LLMs in Aerospace Applications

The rapid evolution of Large Language Models (LLM) has opened the possibility of developing new tools to assist engineers in their software development and verification activities. This paper evaluates the assistance provided by LLMs in test case identification in the aerospace industry. Three different paradigms were considered for the evaluation: Retrieval Augmented Generation (RAG), prompt, and agents. The findings show that these tools can reduce the time to identify the initial set of test cases for a given requirement. However, the evaluation also revealed recurring sources of errors, including incorrect mathematical calculations, unnecessary tests, and imprecise boundary-value derivation that compromise the test quality. The paper summarizes these observations and compares the results obtained with the different application paradigms.