An Explainable Transformer-Based Mixture of Experts for Heat Load Forecasting

Accurate heat load forecasting is essential for predictive control, demand response, and operational planning in buildings and district heating systems. However, heat load patterns differ strongly across buildings due to variations in function, size, solar exposure, thermal efficiency, occupancy behavior, and HVAC technologies. These differences challenge single model predictors, especially when no metadata beyond measured consumption is available. We propose an explainable metadata-free Mixture of Experts (MoE) framework that clusters buildings by their heat load characteristics and trains specialized experts for distinct heat load regimes. Each expert is implemented as an Informer-based transformer trained on one cluster, while a gating network learns adaptive weights to combine their forecasts at inference. These time-varying weights, together with expert attention patterns and feature importance, provide model-intrinsic explanations of how different heat load regimes are learned and represented. The proposed framework is benchmarked against four baselines (Informer, CNN-LSTM, LSTM, and PatchTST) across three datasets covering both synthetic and real-world conditions: Demandlib (synthetic archetypes), Green Fusion (multi-family residential), and Hotels (commercial buildings). Averaged over three input windows and three prediction horizons, MoE reduces NRMSE by 24%, 13%, and 10% on these datasets, respectively. The results demonstrate that clustering-based expert specialization significantly improves forecasting accuracy and robustness compared to a single model across diverse building contexts.