Twin delayed deep deterministic policy gradient with long short-term memory for energy management in electric vehicles with battery–supercapacitor storage: A reinforcement learning benchmark study

As electric vehicles (EVs) evolve to meet increasingly demanding performance and efficiency targets, hybrid energy storage systems, combining batteries with supercapacitors, are emerging as a promising solution to enhance energy responsiveness, extend battery life, and improve overall system stability. This study introduces a memory-augmented Twin Delayed Deep Deterministic Policy Gradient controller with Long Short-Term Memory (TD3-LSTM) for real-time power-split management in multisource EVs. The agent is trained on the FTP-75 driving cycle, and is evaluated on FTP-75, UDDS, and HWFET to assess its generalization. Distinct from prior feed-forward deep-reinforcement-learning energy-management strategies, TD3-LSTM incorporates recurrent sequence awareness, enabling the policy to anticipate imminent load variations rather than reacting exclusively to instantaneous states. Performance is benchmarked against vanilla TD3, Soft Actor-Critic (SAC), Proximal Policy Optimization (PPO), and Q-learning, and further validated through an ablation study with capacity-matched and frozen-recurrence variants to disentangle temporal modeling from parameter count. Evaluation using specialized metrics, the Steady-State Battery Dominance Index (SSBDI), the Battery Thermal Stress Index (BTSI), DC bus voltage fluctuation, and the Cumulative Ampere-hour Throughput (CAT) shows that TD3-LSTM achieves the smoothest power allocation and the most balanced battery–supercapacitor cooperation, reducing BTSI by up to 61%, SSBDI by up to 24%, DC bus voltage ripple by up to 56%, and CAT depletion by up to 31% relative to PPO, while yielding a 2.4% higher evaluation return than SAC. These findings demonstrate TD3-LSTM's potential as a practical, energy-aware controller for next-generation EVs.