VHF-Sim2Real: Generative Noise Injection for Improved Automatic Speech Recognition in Air Traffic Control

Robust Automatic Speech Recognition (ASR) is a prerequisite for automated support tools in Air Traffic Control (ATC), yet publicly available labeled corpora of real Very High Frequency (VHF) radio traffic are extremely limited. The open portion of the ATCO2 corpus provides only about one hour of manually transcribed VHF speech, whereas the widely used ATCOSIM corpus contains roughly ten hours of clean, simulated controller speech recorded in a laboratory. After fine‑tuning on the combination of ATCOSIM and the labeled ATCO2 subset, large open ASR models such as Whisper can reach a Word Error Rate (WER) of about 1.2% on ATCOSIM, yet WER on real ATCO2 speech remains around 13–15% as reported by van Doorn et al. and confirmed by our reproduction. This behavior highlights both the acoustic mismatch between simulated and real VHF channels and the limited amount of labeled operational data. We investigate whether generative noise modeling can reduce this mismatch without introducing additional labeled ATC speech. Our VHF‑Sim2Real pipeline trains a spectrogram‑based Generative Adversarial Network (GAN) on unlabeled VHF recordings from TartanAviation and ATCO2 to learn VHF‑like interference patterns and spectral distortions, then uses the trained generator to inject this noise on the fly into ATCOSIM utterances during Whisper fine‑tuning. The fine‑tuning stage mixes these noisy ATCOSIM signals with about one hour of labeled ATCO2 speech, so that Whisper is exposed to ATC phraseology under realistic VHF degradations while still benefiting from accurate transcripts. On the evaluation split of the public ATCO2‑test‑set‑1h subset, this procedure reduces WER from 14.32% for our reproduction of the Whisper‑ATC baseline to 3.58%, a 75.0% relative improvement obtained with the same amount of labeled ATC data, and suggests that, in this benchmark setting, unpaired generative noise augmentation can substantially reduce the sim‑to‑real performance gap.