Proximal Policy Optimization With Predictive and Memory-Aware Battery Management for BEVs Using Hybrid Energy Storage and Battery Health-Oriented Rewards

The increasing use of electric vehicles (EVs) calls for the development of more sophisticated battery management strategies (BMSs) in order to maximize performance and lengthen the lifespan of components. Battery electric vehicles (BEVs) that make use of a single energy storage system (ESS) are another type of electric vehicle architecture that necessitates rigorous energy management in order to guarantee both efficiency and longevity. In order to improve the efficiency with which energy is utilized, hybrid energy storage systems (HESS) have been developed. These kinds of ESS combine super capacitors with batteries (SCs). Despite the fact that HESS designs provide enhanced power handling and a longer driving range, they also add an additional layer of complexity to the design of BMSs. In this research, an intelligent BMS for BEVs that are equipped with a battery-supercapacitor as HESS is proposed. The proposed framework integrates a Proximal Policy Optimization (PPO) reinforcement learning algorithm with a Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) predictor to forecast future driving demands while considering historical patterns and the battery's state of health (SOH). For the purpose of optimizing energy distribution and preserving battery health, the BMS makes use of multi-dimensional observations, such as the state of charge (SOC) of the battery, the SOH, the voltage of the SC, the speed of the vehicle, and the distance traveled. The proposed strategy demonstrates superior performance in both energy efficiency and battery health preservation when compared with state-of-the-art RL-based methods. Specifically, it reduces battery capacity fading by 2.38% and 1.12% relative to PPO and SAC without memory-aware mechanisms, as well as SAC with memory-aware features, while simultaneously achieving a higher final SOC than all benchmark counterparts.