UniCrossGait: Unified Cross-Modal Gait Recognition Based on Knowledge Distillation

Gait recognition provides a crucial biometric identification technique for intelligent security since it is long-range and nonintrusive. Previous methods have generally focused on gait recognition within a single modality, but with the constant advancement of multimedia sensors, there is a growing requirement for the development of multimodal and cross-modal gait recognition. The various modalities are aligned directly in a hard manner by current cross-modal approaches, which could result in class collapse caused by imbalanced learning and make learning discriminative representations difficult for the weaker modality. To overcome these limitations, we propose a unified cross-modal gait recognition framework, UniCrossGait. Specifically, we employ the knowledge distillation paradigm to transfer the knowledge of a fused multimodal teacher network, Exist Methods UniCrossGait(Ours) Figure 1: Illustration of comparison between exist methods and which serves as a shared intermediate representation, to unimodal student networks, which enables them to learn modality-invariant representations. To perform knowledge distillation effectively, we design the Direction-level Feature Imitation loss and Intra&Inter sample Correlation loss. Furthermore, we construct the Warm up operator and Distillation Balancing operator to prevent the network from falling into suboptimal solutions due to pure imitation and imbalanced distillation. Our method achieves state of-the-art (SOTA) results on cross-modal gait recognition datasets, and extensive ablation experiments demonstrate the effectiveness of the proposed paradigm and modules. Our code is available at https://github.com/O-VIGIA/UniCrossGait.