Predicting soil health properties across different agricultural land use systems using mid-infrared spectroscopy

Mid-infrared (MIR) - diffuse reflectance spectroscopy (DRS) combined with chemometrics offers a rapid, cost-effective approach for predicting soil properties, yet accuracy varies across soil attributes and validation methodologies. This study calibrated and validated partial least squares regression (PLSR) models for 30 soil health properties using MIR spectra (4000–600 cm−1) from 829 samples representing eight cropping systems in Nova Scotia, Canada. Models were assessed via two statistical techniques: an 80–20 random holdback split, and a leave group out cross validation (LGOCV) based on cropping system. Results demonstrated strong predictive performance for soil pH, total organic carbon, total nitrogen, aluminum, water stable aggregates, sand, and silt under both validation methods (RPIQ 2.2–3.7). Conversely, clay (averaged 12%) proved challenging to model. Labile nutrient fractions (ammonium, nitrate, total soluble nitrogen) and Mehlich-3 extractable nutrients (P2O5, K2O, sulfur, boron, copper, zinc) exhibited limited predictability (RPIQ < 2). Across all properties, validation method influenced accuracy, with the 80–20 split averaging 18.1% higher RPIQ for properties that outperformed LGOCV. The largest declines under LGOCV were for active carbon, respiration, ACE protein, cation exchange capacity, total base saturation, and available water content. Total organic carbon, total nitrogen, organic matter, sand, silt, and pH maintained strong predictive accuracy (RPIQ = 2.2–3.7) despite modest declines under LGOCV, while calcium, aluminum, and clay performed slightly better under LGOCV (RPIQ = 2.0–2.8). These findings highlight the risk of overestimating performance when cropping system heterogeneity is ignored and the value of group-based validation for assessing model robustness.