FHBSight: multimodal UAV-Based early detection of wheat fusarium head blight using sensitive vegetation indices and RGB phenotypic features

Fusarium head blight (FHB) is a destructive wheat disease that threatens grain yield, quality, and food safety. Early-stage FHB detection is critical for timely field management but remains challenging in UAV imagery because infected regions are often small, sparsely distributed, and weakly contrasted. In this study, early-stage FHB refers to the initial phase of visible symptom development, typically occurring approximately 8–14 days after infection onset, and characterized by slight spikelet discoloration and limited infection within wheat ears. Physiological stress responses associated with this stage may also precede clearly visible symptoms, providing a potential basis for vegetation-index-assisted (VI-assisted) detection. To address these challenges, this study proposes FHBSight, a UAV-based multimodal semantic segmentation framework that integrates RGB-derived phenotypic information with vegetation-index-derived (VI-derived) physiological cues for FHB detection. FHBSight adopts a dual-branch SeMask-FPN backbone to separately encode RGB spatial features and VI-based stress-sensitive representations. An Adaptive Cross-Modal Attention module dynamically reweights cross-modal features to reduce redundancy and enhance fine-grained lesion representation. A Joint Loss combining cross-entropy and Dice Loss is designed to improve sensitivity to small and early-stage lesions under severe lesion-background class imbalance. A UAV multispectral dataset containing 5,059 RGB images and 19 VI datasets was constructed for model evaluation. Experimental results showed that the anthocyanin reflectance index, normalized difference vegetation index, and green difference vegetation index provided relatively sensitive cues for FHB detection. Using these selected VIs, FHBSight achieved an mIoU of 73.01%, an mAcc of 86.17% and an mDice of 83.21%. Compared with the SegNeXt, FHBSight improved mIoU and mAcc by 1.25% and 6.34%, respectively. These results indicate that combining RGB imagery with compact VI-derived physiological information can improve early FHB detection under complex field conditions. The proposed framework provides a practical approach for UAV-based wheat disease detection and field-scale precision management.