Forecasting earthquakes by Machine Learning techniques: lights and shadows

Abstract Machine Learning techniques have gained traction in earthquake forecasting, with recent studies reporting exceptionally high accuracy rates. However, concerns regarding the reliability of these models in real-world scenarios persist. This study provides a critical reappraisal of a recently proposed Machine Learning workflow that originally reported >97% accuracy in forecasting seismic events. We replicated the methodology using a new dataset from Tokyo, Japan, to assess its generalizability and robustness. While standard random train-test splits reproduced the originally reported high accuracy (>99%) in this distinct tectonic setting, subsequent analysis revealed this performance to be an artifact of data leakage. When subjected to rigorous time-based validation and walk-forward testing, the model’s predictive performance dropped drastically to near-baseline levels (24%). Furthermore, direct cross-location testing yielded results indistinguishable from random chance (16%), confirming that the model relies on local artifacts rather than physical precursors. Our findings suggest that the promising results often found in the literature may stem from improper handling of seismic data characteristics. We conclude that the advance of ML-based earthquake forecasting requires a fundamental shift towards stricter methodological standards, prioritizing realistic validation protocols and domain-specific rigor over optimistic performance metrics.