Estimating the Yield of Mature Rice With UAV-Based Multimodal Images Fusion and Five-Points Sampling

The accurate estimation of mature rice yield using Unmanned Aerial Vehicle (UAV)-based multispectral and RGB images is crucial for precision agriculture. However, existing methods face challenges from grain fullness, complex soil backgrounds and weeds. This research proposes a mature rice yield estimation approach that consists of Fusion Yield Network (FusYieldNet) model and five-points sampling method. To estimate the yield of small plot with fixed size, FusYieldNet model is designed with RGB, multispectral, and fusion branches for feature extraction and fusion. Efficient Multi-scale Convolution Attention Module (EMCAM) is introduced in three branches, and Parallel Path Attention (PPA) in the RGB branch to capture multi-scale features. Furthermore, in the RGB branch, a novel Dilated Deformable Dense Block (DDDB) is developed by enabling convolutions in the module DenseBlock dilated and deformable to extract features from changeable-size and deformable image regions. With the loss of three branches, the gradient blending technique is employed to mitigate the risk of overfitting and improve the model's generalization capability. Afterward, the field yield is estimated as the average yield of five sampled small plots with five-points sampling method. Experimental results demonstrate that FusYieldNet achieves a mean absolute error of 18.03 kg, a relative error of 1.74%, and a co-efficient of determination (R²) of 0.9927, outperforming all unimodal and competing multimodal fusion approaches. The proposed five-points sampling method enables in-field rice yield estimation with approximately 3% relative error and 2-second computational efficiency, effectively meeting practical application requirements.