Reinforcement Learning-Based Plug-In Hybrid Electric Vehicle Energy Management Considering Vehicle Mass Uncertainty

Current reinforcement learning-based hybrid electric vehicle energy management strategies (EMS) assume constant vehicle mass, failing to account for vehicle mass variations. Vehicle mass discrepancies between the control-oriented mathematical model and the actual vehicle can result in control performance degradation. This paper first establishes a control oriented model (COM) for a series-parallel plug-in hybrid power system. Based on the COM, we design a new EMS that incorporates dynamics randomization during reinforcement learning (RL) training to enhance adaptability to vehicle mass uncertainty. Particularly, dynamics randomization dictates that the vehicle mass is randomized for different episodes during RL training. Simulation results indicate that when the vehicle mass used for RL training and testing is the same, the RL EMS without dynamics randomization has an energy cost gap of just 1.10% compared to the dynamic programming (DP) result. However, when the vehicle mass used for training and testing differs, the energy cost gap reaches a maximum of 14.17%. In contrast, the RL EMS with dynamics randomization maintains an energy cost gap from DP within 5% across all vehicle masses. Finally, high-fidelity model (HFM) testing reveals that dynamics randomization mitigates the severe energy management performance decline duel to modeling errors.