Deviation measures for model validation: properties and behavior in the presence of bias

This study examined 14 deviation measures (DMs) used in model validation, assessing their mathematical properties and behavior in the presence of bias. Three data sets were used: without bias (WB), constant bias (CB), and proportional bias (PB). The DMs were evaluated based on a 0.5-unit threshold from the optimal value. The key findings were: In the WB model, six DMs (MAE, MSPE, RMSE, RRMSE, RMS, PMARE) performed contrary to expectations; With CB, six DMs (CD, MAE, ME, MSPE, RMSE, MSE) failed to detect bias; With PB, five DMs (CD, C, r, R2, dr) didn't detect bias; MEF and ELM identified both bias types; And PMARE was not comparable due to its unique classification approach. The study recommends using graphical and inferential methods alongside multiple effective DMs to identify model inconsistencies. It emphasizes considering each DM's mathematical properties for correct application. Subjectivity plays a role in interpreting DMs when assessing model appropriateness. This research provides insights into the strengths and limitations of various DMs, aiding researchers in selecting appropriate measures for model validation.