Large-sample hydrologic models poorly simulate interannual variability in seasonal catchments, despite high Nash-Sutcliffe and Kling-Gupta Efficiencies

Variability in river flow can be understood as the sum of irregular, seasonal and interannual variance components. Skillful simulations of irregular events are needed to accurately predict short-duration events such as floods, while skillful simulation of interannual variance is required to accurately predict long-term change and long-duration droughts. However, popular performance metrics such as the Nash-Sutcliffe Efficiency (NSE) and Kling-Gupta Efficiency (KGE) do not distinguish these three variance components. We analyse streamflow simulations from 18 process-based, machine learning, and hybrid hydrologic models from around the globe (22,089 simulated time series in total) to investigate how well large-sample hydrologic models represent each variance component. We find that in highly seasonal (tropical, alpine, and polar) catchments these models achieve very high NSE and KGE values but produce worse-than-average simulations of interannual and irregular variance. Year-to-year variability in streamflow extremes and monthly mean flows is consistently more poorly simulated in highly seasonal catchments than in less-seasonal catchments. This suggests that these hydrologic models have limited skill in predicting long-term responses to climate change in alpine, polar, and tropical regions, which are some of the most vulnerable regimes regarding climate change. There is a need to rethink the value of efficiency scores such as NSE and KGE in large-domain model evaluation, and to complement such approaches with more detailed and more process-based investigations of model performance.