ASTH agent: Aviation software testing hierarchy agent for test case generation

Software testing for aviation simulation systems is essential in aerospace design, requiring a high level of domain expertise and precision. Traditional rule-based and pipeline-driven Natural Language Processing (NLP) approaches for test case generation heavily rely on structured testing documentation, rendering them inadequate for comprehending and processing the complex, unstructured descriptions commonly found in aviation software testing rules. Likewise, conventional formal modeling methods fail to model testing processes characterized by intricate logic and long workflows. This study presents ASTH agent, a multi-agent framework based on Large Language Models (LLMs), for the automatic generation of test cases from aviation software testing documents. It first constructs a knowledge base of aviation software testing by means of adaptive parsing of multimodal documents, after which an agent autonomously extracts and refines standardized testing rule sets through iterative steps. To accurately model the testing process, a domain-specific hierarchical multi-agent architecture is proposed, which autonomously decomposes complex tasks into executable actions, enhanced by dynamic hierarchical planning adjustment that revises planning based on execution feedback to improve robustness. Moreover, the framework incorporates continual learning from historical executions to enhance its cross-scenario adaptability. The experimental results indicate that the framework effectively supports task planning and action execution for the generation of aviation software test cases, and shows strong adaptability to different scenarios. Notably, the accuracy of the generated test cases and their fault detection capability significantly surpass those of traditional method, demonstrating that this framework has the capacity to substantially reduce the workload of the testers.