Multimodal deep learning framework for robust individual identification of Holstein cattle using gait dynamics and coat pattern features

Accurate individual cattle identification is critical for precision livestock farming. Some of the challenges associated with identification in the outdoor farm environment include factors such as variable illumination, mud contamination, and frequent occlusions, which often degrade the quality of visual features, limiting the reliability of existing identification methods. Given the lack of specificity to identify and track uniform-colored cattle in outdoor environments using existing methods, this study presents a deep learning framework that integrates gait dynamics and coat pattern features to reliably identify Holstein cattle. Using ground-based video data capturing 54 cows, developed was the framework that employs structural similarity index −based gait cycle segmentation, a CNN-BiLSTM network with attention (FCANet) for gait feature extraction, and an AlexNet-based model for occlusion mitigated pattern recognition. A dynamic decision-level fusion strategy (gait-pattern weighting: 4:6 for patterned and 7:3 for patternless cows) enhances cattle identification accuracy. The framework achieved 96.76% Rank-1 accuracy for patterned cows and 92.15% for patternless ones, with dynamic fusion improving performance by 2.24%. Framework maintained 88.12% mAP for lame cattle, though health transitions affected feature consistency. Error analysis highlighted challenges from cattle head turning behaviour and environmental factors like structural shadows and mud on the cattle. Overall, the study developed a multimodal approach to address key limitations of unimodal biometrics, offering a practical solution for cow identification in heterogeneous cattle populations under farm conditions.