Integrating Real-Time Wind Hazard Prediction and Visual Augmentation in Full-Flight Simulators to Assess the Pilot's Workload

Urban Air Mobility (UAM) operations frequently encounter unique low-altitude wind hazards such as gusts, wind shears, vorticities, venturi effects, up- and downdrafts, turbulences, and self-induced downwash. These micro-scale atmospheric hazards change rapidly, particularly due to building-induced airflow, and are difficult to detect, emphasizing the importance of pilot awareness for UAM safety. To address this challenge, this study introduces a novel approach that, for the first time, integrates real-time wind field forecasts, using a Factorized Fourier Neural Operator (F-FNO), into an H145 Level D full-flight simulator (FFS) to enable the simulation of highly realistic atmospheric conditions. For each predicted wind field, hazardous wind phenomena are classified as adaptive keep-out geofences using an enhanced $\alpha$-shape algorithm. These keep-out geofences are depicted in real time through a JVC augmented reality (AR) headset as well as through an additional display inside the cockpit, providing pilots with a new and immersive method for wind field visualization to support navigation and situational awareness. Workload and performance were evaluated in specific, reproducible flight test scenarios conducted by two professional pilots, each with more than 3,000 hours of flight experience. Special emphasis was placed on the realism of wind effects in the lee of buildings during approach and landing in urban environments, as well as on the acceptance, applicability, and usability of innovative visualization techniques for UAM pilots. The provided feedback indicated that the detailed and realistic simulated wind and flow fields, together with the flight model’s response, provide a clear training advantage and outperforms all previously experienced simulators. This work demonstrates the added value of realistic wind effects in flight simulators for pilot training and educational purposes, as well as the benefits of visualized hazardous wind regions for pre-flight briefing, approach and landing procedures. Consequently, the future objective is to enable UAM pilots to train even more effectively using novel visualization approaches such as AR, virtual reality (VR), mixed reality (MR), or flight instrument adjustment, thereby enhancing pilot behavior in critical situations and increasing safety in upcoming real-world operations.