Robust OAM-SK FSO demodulation under strong turbulence via Vision Transformer and Polar code integration

This paper proposes a joint demodulation and decoding scheme for orbital angular momentum shift keying free-space optical (OAM-SK-FSO) communication systems, termed ViT-Polar, which deeply integrates a Vision Transformer (ViT) with Polar coding to enhance reliability over turbulent channels. The proposed framework organically unifies channel coding with modulation and, correspondingly, demodulation with decoding. At the transmitter, the information bits are encoded using Polar codes and then modulated into 16-ary OAM Laguerre–Gaussian (LG) beams via Gray mapping. At the receiver, a ViT is employed to extract discriminative features and identify OAM modes from the received optical-field images, producing symbol-level posterior probabilities. These probabilities are further converted into bit-wise soft information, i.e., log-likelihood ratios (LLRs), which are fed into a Polar decoder for soft-decision decoding. The system supports four Polar decoding algorithms — SC, SCL, BP, and SCAN — and substantially improves end-to-end reliability through deep fusion of probabilistic soft information. Numerical simulations demonstrate that, under strong turbulence and long-distance transmission conditions, the proposed ViT-Polar scheme achieves significantly better bit error rate (BER) performance than conventional CNN-based adaptive demodulation implemented in this study and separated demodulation–decoding approaches. In particular, the ViT-SCL configuration delivers the best BER performance and the highest decoding stability among the considered decoders, benefiting from ViT’s global attention mechanism for high-accuracy OAM mode classification and the effective exploitation of soft information by the SCL algorithm. This study provides a highly reliable and intelligent end-to-end communication paradigm for OAM-SK-FSO systems, offering valuable theoretical insights and technical support for the design of future high-speed optical communication systems.