A Crop-Wise Hybrid Ensemble Learning Framework for Accurate and Explainable Crop Yield Prediction

Yield forecasting in agriculture is important for agricultural operations and ensuring food security. The proposed work intends to combine a comprehensive approach towards agricultural yield prediction for multiple crops using a LightGBM model, considering the performance of four different hybrid models, namely Super Ensemble (LightGBM, XGBoost, Random Forest), LightGBM-CatBoost, LightGBM residual improvement via neural networks, as well as a weighted ensemble method. The proposed work carried out experiments on a cleared agricultural dataset, which considered the six primary agricultural products, barley, cotton, maize, rice, soybeans, and wheat, for evaluation purposes based on the measures RMSE, MAE, and$\mathrm{R}^{2}$. The baseline LightGBM model was highly accurate for detecting all crops, yielding an RMSE measure of 0.49 and$\mathrm{R}^{2}$scores exceeding 0.91. The LightGBM + CatBoost ensemble consistently surpassed the other hybrid models by generating the most accurate forecasts and providing the most information for all crops. Actual versus predicted yield plots confirmed that the model was well-calibrated and had very little bias. Permutation feature importance and SHAP analysis are two explainable AI techniques that were used to make sure that the AI was clear and easy to understand. Rainfall, fertilizer application, and irrigation were identified as the predominant factors influencing crop yield, with SHAP dependence plots illustrating distinct, agronomically significant correlations. The suggested LightGBM-CatBoost ensemble is a strong, accurate, and easy-to-understand way to predict yields for multiple crops. It can also be a useful tool for making decisions in agricultural management.