Deep Fusion of Clear-Sky Physics and a Multimodal Transformer for Solar Irradiance Forecasting

Accurate ultra-short-term prediction of solar irradiance is crucial for ensuring the stable operation and dispatch optimization of photovoltaic (PV) power generation systems. Existing deep learning-based prediction methods primarily rely on pure data-driven approaches, which often suffer from insufficient physical consistency under complex meteorological conditions, leading to predictions that deviate from radiative physical principles. Some studies employ physical models to correct deep learning predictions post-hoc, yet physical laws are not deeply embedded during the inference process. To address this issue, this paper proposes a multimodal Transformer methods framework incorporating clear-sky physical constraints. It integrates ground-based cloud image sequences, meteorological observation elements, and clear-sky irradiance models to achieve joint modeling and cross-modal information interaction among image, meteorological, and physical modalities. The model backbone employs an improved Transformer architecture, introducing a cross-modal attention mechanism and a gated fusion module to achieve multi-level feature refinement and modal adaptive fusion, thereby embedding physical laws during inference. Experimental results demonstrate that the model excels in ultra-short-term solar irradiance prediction tasks, significantly reducing prediction errors and maintaining high robustness and accuracy under rapidly changing weather conditions.