A Comparative Study of Energy Management Strategies for Battery-Ultracapacitor Electric Vehicles Based on Different Deep Reinforcement Learning Methods

An efficient energy management strategy (EMS) is crucial for the energy-saving and emission-reduction effects of electric vehicles. Research on deep reinforcement learning (DRL)-driven energy management systems (EMSs) has made significant strides in the global automotive industry. However, most scholars study only the impact of a single DRL algorithm on EMS performance, ignoring the potential improvement in optimization objectives that different DRL algorithms can offer under the same benchmark. This paper focuses on the control strategy of hybrid energy storage systems (HESSs) comprising lithium-ion batteries and ultracapacitors. Firstly, an equivalent model of the HESS is established based on dynamic experiments. Secondly, a regulated decision-making framework is constructed by uniformly setting the action space, state space, reward function, and hyperparameters of the agent for different DRL algorithms. To compare the control performances of the HESS under various EMSs, the regulation properties are analyzed with the standard driving cycle condition. Finally, the simulation results indicate that the EMS powered by a deep Q network (DQN) markedly diminishes the detrimental impact of peak current on the battery. Furthermore, the EMS based on a deep deterministic policy gradient (DDPG) reduces energy loss by 28.3%, and the economic efficiency of the EMS based on dynamic programming (DP) is improved to 0.7%.