Deep learning based automatic modulation classification

The objective of this thesis is to exploit recent advances in deep learning (DL)-based techniques for automatic modulation classification (AMC) for both wireless communication and radar systems. Firstly, we propose a dual-component modulation recognition framework, which incorporates the residual Swin transformer denoise network (ResSwinT), Swin transformer feature extraction network (SwinT), residual-attention (RA) modulation recognition head, and signal-to-noise ratio (SNR) level classifier and achieves robust recognition performance against various SNR conditions with tolerable complexity and accuracy trade-off. The proposed framework can identify randomly combined dual-component radar signals from twelve modulation formats, while enhancing the utilization of SwinT features and reducing unnecessary computation of the ResSwinT. Simulation results show that the proposed scheme can obtain an exact match ratio (EMR) exceeding 97% at SNR> -6dB. At the low SNR condition (-12dB), the ResSwinT can obtain about an EMR gain of 20%, and the overall framework can achieve an EMR exceeding 80%. Following that, to cope with the evolving communication systems, we develop an orthogonal frequency division multiplexing (OFDM) modulation classification framework, which incorporates the initial blind in-phase and quadrature (IQ) imbalance and carrier frequency offset (CFO) estimation and compensation modules and the cross-selective kernel network (Cross-SKNet). The proposed framework can identify four different OFDM modulation formats and enhance overall classification performance by IQ imbalance and CFO compensation with a fraction of additional computation. Simulation results demonstrate that the proposed Cross-SKNet outperforms the state-of-the-art method across a range of SNR levels and exhibits superior generalization capabilities at low SNR conditions. Based on the simulation results of the generated dataset, the Cross-SKNet achieves a classification accuracy of over 95% for SNR levels above 8dB. Moreover, the incorporation of IQ imbalance and CFO compensation modules enhances the performance of the Cross-SKNet, which can achieve an accuracy gain of 8% at the low SNR condition at -10dB. Given the increasing focus on AMC for MIMO-OFDM systems, we propose a novel DL architecture named 4D2DConvNet, which integrates multi-branch shallow two-dimensional convolution (2DConv) layers with four-dimensional convolution (4DConv) layers to effectively extract both channel-specific OFDM symbol features and cross-channel correlations. Simulation results demonstrate that the proposed 4D2DConvNet achieves robust performance, attaining a classification accuracy exceeding 95% at a SNR of 8dB over fifth-generation (5G) frequency-selective fading channels, under various delay profiles and Doppler shift configurations. To tackle the practical issue of limited labeled data, we further propose a self-supervised learning framework, named cross-view prediction with automatic augmentation (CPAA). Unlike existing supervised methods requiring substantial labeled data, the CPAA leverages massive unlabeled samples and minimal labeled data. The proposed framework is capable of identifying five different OFDM modulation formats and enhancing performance through automatic data augmentation, which effectively retains data identity and generates diverse views. Simulation results indicate that CPAA can achieve robust classification performance with massive unlabeled samples and extremely limited labeled samples. Moreover, the proposed framework outperforms state-of-the-art methods across a range of SNR levels and demonstrates better generalization capability under low SNR conditions. Based on the simulation results, the CPAA framework achieves a classification accuracy of over 85% for SNR levels above 8dB, using only one labeled sample per SNR for each modulation scheme. Moreover, the automatic data augmentation module enhances the performance of self-supervised learning when limited labeled data are available, yielding an approximate average performance gain of 48.78% and demonstrating good transferability on the existing self-supervised learning algorithms. As transformers are widely adopted for time series analysis, we propose the temporal-spatial convolutional transformer (TSConvT), which utilizes asymmetric convolutional (AsyConv) projection and separated temporal and spatial attention mechanisms to enhance the intra-symbol and inter-symbol feature extraction. The results show that the TSConvT network demonstrates superior classification performance compared to state-of-the-art methods for OFDM modulation classification, which achieves an average test accuracy of 84.05% across SNR levels ranging from -10dB to 20dB. Furthermore, the proposed model demonstrates effective generalization capability under varying communication channel scenarios. This thesis demonstrates that DL–based AMC methods can achieve high accuracy and robustness under low SNR and fading conditions. These DL techniques contribute to more efficient and resilient wireless communication systems by enabling real-time, accurate modulation recognition across diverse operating conditions.