How to Use Score-Based Diffusion in Earth System Science: A Satellite Nowcasting Example

Abstract Machine learning (ML) is used for many Earth science applications; however, traditional ML methods trained with squared errors often create blurry forecasts. Diffusion models are an emerging generative ML technique with the ability to produce sharper, more realistic images by learning the underlying data distribution. Diffusion models are becoming more prevalent, yet adapting them for Earth science applications can be challenging because most articles focus on theoretical aspects of the approach, rather than making the method widely accessible. This work illustrates score-based diffusion models with a well-known problem in atmospheric science: cloud nowcasting (0–3-h forecast). After discussing the background and intuition of score-based diffusion models using examples from geostationary satellite infrared imagery, we experiment with three types of diffusion models: a standard score-based diffusion model (Diff), a residual correction diffusion model (CorrDiff), and a latent diffusion model (LDM). Our results show that the diffusion models not only advect existing clouds but also generate and decay clouds, including convective initiation. A case study qualitatively shows the preservation of high-resolution features longer into the forecast than a conventional U-Net. The best of the three diffusion models tested was the CorrDiff approach, outperforming all other diffusion models, the conventional U-Net, and persistence. The diffusion models also enable out-of-the-box ensemble generation with skillful calibration. By explaining and exploring diffusion models for a common problem and ending with lessons learned from adapting diffusion models for our task, this work provides a starting point for the community to utilize diffusion models for a variety of Earth science applications. Significance Statement Machine learning is an invaluable tool for Earth science applications, but they often result in blurry images, predictions, or forecasts. Diffusion models are an emerging technique to enable more realistic looking images. This work intuitively explains the diffusion modeling process and explores diffusion models for forecasting satellite imagery. Our results show skillful performance by diffusion models, outperforming traditional machine learning techniques. We discuss lessons learned from applying different diffusion models and trade-offs between performance and computing requirements that are important considerations for deploying these models. The main goal is to provide basic intuition for diffusion modeling and to pass along our hard-earned tips to help accelerate others toward using diffusion models for their own Earth and Environmental Science research tasks.