Joint Multimodal Representation and Deep Fusion for Pattern Recognition of Radar Pulse Stream

Radar pulse stream data exhibit characteristics that include multidimensionality, diverse modulations, discrete, and uneven distributions, which pose significant challenges for feature representation and pattern recognition using deep learning technologies. To address these challenges, this paper proposes a joint multimodal representation and deep fusion framework. For this purpose, we first formulate the dynamic pulse stream using a graph model and construct a set of pulse states based on graph connectivity. After that, the multimodal representation of the radar pulse stream is achieved through two main components. The first component employs the vector embedding strategy to represent the distribution probabilities of pulse states as a one-dimensional (1D) feature vector. The second component introduces the Markov transition field to represent the transition probabilities among the pulse states as a two-dimensional (2D) feature map, thus achieving a unified representation of radar pulse stream data from a probabilistic perspective. Finally, we construct a deep fusion network that simultaneously employs 1D and 2D convolution branches to process different modal features, and further integrate them as a whole for recognition purpose within a multimodal interaction framework. Experimental results show that the proposed method can achieve a recognition accuracy level of 95% even under the condition of 70% pulse missing ratio (nearly 3.0% accuracy improvements over state-of-the-art results).