An explainable hybrid transformer–BiLSTM framework for multivariate household energy demand forecasting with weather integration

Abstract Purpose Accurate and explainable household load forecasting is critical for demand-side management, tariff-aware scheduling, and reliable smart grid operation. This study introduces a leakage-controlled multi-horizon forecasting pipeline that integrates predictive accuracy with statistical validation, interpretability, robustness, and operational relevance. Methods We model multivariate household demand using an hourly smart-meter dataset spanning 14 months (Nov 2022-Jan 2024; N=10,234 time steps), incorporating aligned local weather covariates. A hybrid Transformer-BiLSTM is trained using a multi-output configuration to predict 24-hour and 168-hour trajectories. Hyperparameters are optimized via Bayesian optimization (Optuna) employing chronological train/validation/test splits and a rolling-origin evaluation protocol. Performance is assessed using MAE, RMSE, and MAPE, while pairwise forecast differences are validated using the Diebold–Mariano procedure. Model explanations are generated through SHAP and attention analyses, further complemented by robustness testing (noise and feature dropout) and inference-efficiency measurements. Results At the 24-hour horizon, the hybrid model achieves MAE and RMSE values of 0.0539/0.0701 (MAPE 29.7%), yielding performance comparable to N-BEATS (MAE/RMSE 0.0531/0.0684). For the 168-hour horizon, the model attains superior performance among evaluated baselines (MAE/RMSE/MAPE 0.0566/0.0746/30.1%) and demonstrates statistically significant improvements over the Transformer, BiLSTM, and TCN models (Diebold-Mariano, $$p