Automated Scenario Generation to Maximize Coverage of an Operational Design Domain for AI-Based Aviation Systems

AI is set to enhance the safety and functionality of aviation systems in the near future. With this, the assurance of AI systems becomes an important topic. It has gained momentum in recent times, with the development of numerous guidelines and standards, such as ARP 6983 and the EASA concept paper. The guidelines recommend an Operation Design Domain (ODD) to define the system's boundaries and scenarios, to specify the selection and transition of parameters within these boundaries for training and testing the system. Selecting the proper parameter values across an almost infinite parameter space is challenging. These values in the parameter space should be limited to reduce time and complexity, and large enough to cover the ODD and represent the actual parameters that a system would encounter. Previously, we showcased an operational domain modeling approach for modeling ODD and scenarios, as well as a mechanism for estimating the coverage of these scenarios against the ODD. A significant limitation was the manual effort required to model scenarios to achieve high coverage levels. This paper presents a mechanism to automate scenario generation, enabling high coverage. It also includes an enhanced ODD that incorporates dependencies and distributions to optimize automation. This research will provide a significant step towards ensuring the reliability of AI systems and optimizing costs, effort, and performance for these processes.