Causally Aligned Multiagent Reinforcement Learning for Coordinated Control of Heterogeneous Home Energy Devices

Deep reinforcement learning (DRL) has emerged as a promising paradigm for home energy management systems (HEMS) due to its model-free nature and ability to handle complex dynamics. However, existing DRL-based approaches typically employ a unified reward function that aggregates multiple objectives into a single scalar, failing to account for the heterogeneous roles of controllable energy devices (CEDs) and their distinct causal relationships with control objectives. This leads to reward misattribution, where CEDs with simpler constraints dominate the optimization process while critical components such as battery storage remain underutilized. To address this challenge, we propose a causally aligned multi-agent reinforcement learning (MARL) framework that explicitly models CED-objective causal pathways using a structural causal model (SCM). A causal surgery procedure decomposes shared objectives into CED-specific variants, enabling individualized reward signals aligned with each CED’s causal responsibility. The proposed HR-MASAC algorithm features a multi-head centralized critic for learning vectorized Q-values and agent-specific entropy coefficients for heterogeneous exploration. Experiments across diverse home scenarios demonstrate that our method achieves 40.1% cost reduction and 57.1% comfort improvement over unified-reward baselines, with robust performance under sensor noise and household heterogeneity.