Robust End-to-End FSO Transmission With Joint Coding Modulation and BiLSTM-Based Channel Modeling Under Atmospheric Turbulence

Free space optical (FSO) communication is a promising solution for next-generation communication networks. Atmospheric turbulence, however, severely degrades its performance. We thus propose a novel turbulence-robust end-to-end FSO communication system (TRFSO) that performs joint training across the entire transmission process from source to transmission channel to destination. To introduce physical FSO channel impairments into the training loop, we developed a bidirectional long short-term memory (BiLSTM)-based FSO channel model. Through end-to-end joint training, the system achieved high quality and robust transmission under dynamic atmospheric turbulence. Trained on experimental data collected over a physical FSO link under varying turbulence conditions, the proposed model accurately reproduced real-world channel distortions, achieving a minimum Kullback–Leibler (KL) divergence of 0.0019 nats in amplitude distribution matching. Our experimental results revealed that TRFSO significantly outperformed conventional separate coding modulation schemes in FSO links. Moreover, under strong turbulence, TRFSO achieved a 3.5 dB gain in average multi-scale structural similarity (MS-SSIM) compared with the same network architecture trained without the channel model.