Energy Management Strategy in Consideration of Battery Health for PHEV via Stochastic Control and Particle Swarm Optimization Algorithm

Abstract

This paper presents an energy management strategy for plug-in hybrid electric vehicles (PHEVs) that not only tries to minimize the energy consumption, but also considers the battery health. First, a battery model that can be applied to energy management optimization is given. In this model, battery health damage can be estimated in the different states of charge (SOC) and temperature of the battery pack. Then, because of the inevitability that limiting the battery health degradation will increase energy consumption, a Pareto energy management optimization problem is formed. This multi-objective optimal control problem is solved numerically by using stochastic dynamic programming (SDP) and particle swarm optimization (PSO) for satisfying the vehicle power demand and considering the tradeoff between energy consumption and battery health at the same time. The optimization solution is obtained offline by utilizing real historical traffic data and formed as mappings on the system operating states so as to implement online in the actual driving conditions. Finally, the simulation results carried out on the GT-SUITE-based PHEV test platform are illustrated to demonstrate that the proposed multi-objective optimal control strategy would effectively yield benefits.

1. Introduction

As the energy crisis and climate deterioration become increasingly serious problems, PHEVs have been regarded as one of the research focuses for how to reduce petroleum demand and exhaust emission. In order to improve the fuel economy of the vehicle, a wide range of optimal control methods has been proposed to solve the energy management problem of PHEVs. Examples include rule-based control [1], deterministic dynamic programming [2], stochastic dynamic programming [3], the equivalent consumption minimization strategy (ECMS) [4], Pontryagin’s minimum principle [5] and model predictive control [6]. Furthermore, many intelligent optimization algorithms have been used for solving the energy management control problem recently [7,8]. The control objectives of these optimal strategies for general hybrid electric vehicles (HEVs) are usually the performance of fuel and electricity consumption [1,4], drivability [9] and emissions [7,10].

For PHEVs, recent studies have shown that the cost of the whole vehicle is highly sensitive to battery health. Compared with the conventional HEVs, the characteristics of PHEVs are the wide range of SOC changes and repeated charging/discharging of the battery. A large depth of discharge (DOD) and frequent use can accelerate the battery aging [11]. It is generally known that the batteries with high capacity are usually expensive, and the costs of batteries occupy a large proportion of the total costs of vehicles. Because the commuter vehicle normally has to work for a long time, often more than 10 years, and the premature failure of battery will add an additional expenditure, the battery replacement cost is important to the vehicle economy. Accordingly, the equivalent cost of battery health degeneration must be considered in the long-term operation of the vehicle [12]. Moreover, when the battery life is attenuated to a certain extent, the performance of the battery will be changed. This could lead to different results in the energy management and even may cause the pre-designed controller to fail to achieve the desired target [13]. This indicates that if one wants to achieve economic feasibility for PHEVs, battery health cannot be ignored. Therefore, battery health-conscious energy management in PHEV is becoming a new hotspot. A life evaluation model of battery under different driving conditions is given by Lunz et al. [14], in which the influence of different SOC on the battery’s capacity decline also has been studied. An optimization-oriented model integrating vehicle simulation, battery degradation dates and driving cycle data is developed by Shiau et al. [15] for minimizing battery cycle life cost, fuel consumption and exhaust emissions. Additionally, in order to reduce the total daily consumption of PHEVs, a new convex programming is proposed by Hu et al. [16] to optimize charging, on-road power management and battery degradation mitigation. In [17] the state of battery health (SOH) is also taken as a state variable in the control problem, and two separate controllers, SOC controller and SOH controller, are designed to maintain SOC and SOH in the desired states. A hybrid energy storage system with an ultracapacitor is usually considered to prolong battery life. To extend the life of the battery, Akar et al. [18] proposed an energy management control strategy based on fuzzy logic rules to control the hybrid energy storage system, which is equipped with multi-input converters. Additionally, with the consideration of the effect of temperature on battery health, the thermal management of the battery is taken into account in the energy management control strategy in [19]. Reducing the damage to the health of the battery can be achieved by controlling the battery temperature during the operation of the vehicle. Additionally, in order to predict the dynamics of the battery cell more accurately, reduced electrochemical battery models are proposed. Based on these models, the optimization control strategies with the consideration of battery health are developed. The several representative methods are as follows: the charge pattern optimization for energy cost and battery longevity by Bashash et al. [20] and the energy management with consideration of battery health on stochastic drive cycles by Moura et al. [21]. A weighted ampere-hour throughput model based on the severity factor is built by Onori et al. [22] to estimate the battery life, which is used in PHEV. On the basis of this model, an optimal control-based energy management strategy is presented by Tang et al. [23] to minimize fuel consumption and the wear of the battery.

It is obvious that limiting the lifetime degradation of the battery will result in a negative impact on reducing the energy consumption, which means there is a balance between energy economy and battery health. In this paper, we mainly concentrate on the distribution of power between the engine and electric machines for a commuter plug-in hybrid vehicle with the consideration of energy consumption and battery health. Firstly, a semi-empirical model of battery health is given. This model is obtained by a curve fitting tool from experiential data. Next, the energy management problem is solved offline by SDP and PSO based on the real historical traffic data. The Pareto optimal set will be obtained while solving the multi-objective optimal problem with different weight coefficients between energy consumption and battery health via SDP. Then, PSO is used to find the weight coefficient, which is the best compromise between the two objectives to achieve the economy and find out the best control strategy ensuring the vehicle optimal economic performance. Furthermore, it should be noted that the obtained offline optimal control strategy including the weight coefficient and the control policy of the power-demand-split is not static constant, but a state-feedback type, namely mappings of battery SOC, power demand and road information, and then, in the online implementation, the control strategy is generated by quickly looking up these mappings according to the current system operating states.

The remainder of this paper is organized as follows. In Section 2, the PHEV model is described, which includes the powertrain model, battery model and drive cycle model. In Section 3, the optimal control problem is formulated and is solved. The simulation results and simulation analysis are given in Section 4. Finally, Section 5 summarizes the conclusions of this paper.

2. Model Description

A power-split PHEV with a planetary gear set is taken as the research object. The configuration of the PHEV system is illustrated in Figure 1. It mainly consists of a planetary gear set, an internal combustion engine (ICE), a motor, a generator, a battery pack and a fuel tank.

The basic specifications for this PHEV are provided in

Table 1. PHEV model specifications.

Component	Parameter (Symbol)	Specification
Vehicle	HEV configuration	Power-split
	Gross vehicle weight (M)	1460 kg
	Frontal area (A)	3.8 ${\mathrm{m}}^2$
	Tire radius ($R_{tire}$)	0.2982 m
IC Engine	Type	Gasoline Inline 4-cylinder
	Displacement	1.497 L
	Max power	51 kW
Electric Machine	Type	Permanent Magnet AC
	Motor max power	50 kW
	Generator max power	30 kW
Powertrain	Final differential gear ratio	4.113
	Sun gear teeth number ($R_s$)	30
	Ring gear teeth number ($R_r$)	78
Battery Pack	Max charge capacity ($Q_{batt}$)	6.5 Ah
	Number of cells	168

[24].

2.1. Powertrain Model

The planetary gear set is the core component of the PHEV powertrain. The generator, engine and motor connect to the sun gear, carrier gear and ring gear, respectively. Let $\omega$ and T denote the speed and torque of the gears with the subscripts s, c and r denoting sun gear, carrier gear and ring gear. For the planetary gear, the steady-state speed and torque relationship can be described as the following equations with an assumption that the connections in the powertrain are rigid and there is no friction loss.

$$(R_r+R_s){\omega }_c=R_r{\omega }_r+R_s{\omega }_s$$

(1)

$$T_c=\frac{R_s+R_r}{R_r}{T}_r=\frac{R_s+R_r}{R_s}{T}_s$$

(2)

where $R_s$ and $R_r$ are the number of sun gears and ring gears, respectively.

Then, as obtained from Euler’s laws, the dynamics with respect to the generator speed ${\omega }_g$, the engine speed ${\omega }_e$ and the motor speed ${\omega }_m$ can be described, respectively:

$$\left\{\begin{array}{c}{J}_g{\dot{\omega }}_g=T_s+T_g\hfill \\ J_e{\dot{\omega }}_e=T_e-T_c\hfill \\ J_m{\dot{\omega }}_m=T_m+T_r-\frac{T_{trac}}{g_f}\hfill \end{array}\right.$$

(3)

where $J_g$, $J_e$ and $J_m$ denote the inertia of the generator, engine and motor, respectively. $T_g$, $T_e$ and $T_m$ denote their torques. $T_{trac}$ denotes the traction torque that the driver requires, and $g_f$ denotes the final gear ratio. With the assuming that the connecting shafts are rigid, the following speed relationships hold:

$${\omega }_c={\omega }_e,{\omega }_r={\omega }_m,{\omega }_s={\omega }_g$$

(4)

Moreover, the motor speed and the vehicle velocity have the relationship:

$${\omega }_m=\frac{g_f}{R_{tire}}{v}_s$$

(5)

where $R_{tire}$ is the tire radius and $v_s$ is the vehicle speed.

When the road slope is neglected, the road load of the vehicle mainly includes aerodynamic drag and rolling friction force. Hence, the dynamics of the vehicle velocity can be represented by:

$$M{\dot{v}}_s=\frac{{\eta }_f{T}_{trac}-T_{br}}{R_{tire}}-{\mu }_{r}Mg-\frac{1}{2}\rho A{C}_d{v}_s^2$$

(6)

where $T_{br}$ is the brake torque. ${\eta }_f$ is the transmission efficiency of the differential gear. ${\mu }_r$ is the coefficient of rolling resistance. $\rho$ is air density. A is the frontal area, $C_d$ is the drag coefficient. g is gravity acceleration.

Furthermore, as the control objectives include fuel economy, the fuel mass flow rate ${\dot{m}}_f$ can be used to measure the fuel consumption, which is a function of engine torque and engine speed, described as:

$${\dot{m}}_f=BSFC({\omega }_e,T_e)·T_e·{\omega }_e·{10}^{-5}/36$$

(7)

where BSFC is the break-specific fuel consumption, which is given by a steady state map of the engine speed and torque as shown in Figure 2. Additionally, the map data are from the JSAE-SICEbenchmark problem [24].

2.2. Battery Model

Generally, energy management problems regard that the battery energy from braking is free and the excess engine power can be transformed to electricity without extra cost. However, in fact, they both have the cost of aging the battery. Many factors are recognized to damage the battery health, such as high or low SOC, large C-rate and high temperature. In order to serve the control problem in this paper, the schematic of a control-oriented battery model is given in Figure 3 [25]. The model is composed of the electrical model, the thermal model and the aging model. The model is used to predict SOC, battery temperature and capacity loss in response to the operating conditions, such as the battery current and ambient temperature.

The battery electric model is idealized as a zero order equivalent circuit. As shown in Figure 4, the model includes a voltage source and an internal resistance, which are in series, and both $V_{OC}$ and $R_{batt}$ are functions related to SOC. The battery current is given as:

$$I_{batt}=\frac{V_{oc}-\sqrt{V_{oc}^2-{4R}_{batt}{P}_{batt}}}{{2R}_{batt}}$$

(8)

where $P_{batt}$ denotes the battery power, which is determined by:

$$P_{batt}={\eta }_m^{k_m}{T}_m{\omega }_m+{\eta }_g^{k_g}{T}_g{\omega }_g$$

(9)

where ${\eta }_m$ and ${\eta }_g$ denote the efficiency of the motor and generator, respectively. $k_m,k_g=1$ means discharging state, and $k_m,k_g=-1$ means charging state, which can be given by:

$$k_i=\left\{\begin{array}{cc}-1,\hfill & T_i{\omega }_i>0\hfill \\ 1,\hfill & T_i{\omega }_i\le 0\hfill \end{array}\right.,i=m,g$$

The SOC and current have an integral relationship, so the dynamics of SOC can be calculated as:

$$\dot{SOC}=-\frac{V_{OC}-\sqrt{V_{OC}^2-{4P}_{batt}{R}_{batt}}}{{2Q}_{batt}{R}_{batt}}$$

(10)

where $Q_{batt}$ is the max charge capacity of the battery, which will continuously attenuate in the battery life.

With the assumption that each cell inside the battery pack is equivalent, the dynamics of battery temperature $T_{batt}$ can be expressed as the following equations [25]:

$${\dot{T}}_{batt}=\frac{{\dot{Q}}_{gen}-{\dot{Q}}_{loss}}{m_{batt}·c_{batt}}$$

(11)

where $m_{batt}$ is battery mass and $c_{batt}$ is battery specific heat. ${\dot{Q}}_{gen}$ and ${\dot{Q}}_{loss}$ denote the rate of thermal energy generate and loss, respectively, which follow the equations:

$${\dot{Q}}_{gen}=I_{batt}^2{R}_{batt}$$

(12)

$${\dot{Q}}_{loss}=h·S_{batt}(T_{batt}-T_{amb})$$

(13)

where h is the heat transfer coefficient, $S_{batt}$ is the battery surface area and $T_{amb}$ is the ambient temperature.

As the research objective for optimal control in this paper includes the battery health, the model that can quantify the battery health in the cost function is crucial. The commonly-used health models of lithium-ion batteries can be classified into two categories, electrochemical models [21] and empirical models [17]. Electrochemical models that provide detailed performance of the battery are helpful to understand the principle of battery aging, but the computational loads of these models are too heavy because of the intrinsic complexity requiring chemical and physical knowledge [26], such as active materials and electrolytes, etc. Such a large calculation is not suitable for the management design of this paper, even for offline optimization, because it is necessary to calculate the cost function with the battery health model many times in the iteration. Besides greatly increasing the time of obtaining the optimal policy in the optimal process, the larger calculation also increases the difficulty of adjustment and improvement for the management strategy. Thus, in this paper, semi-empirical models are employed considering the good balance between accuracy and simplicity. In fact, the empirical and semi-empirical models that can be applied easily to solve the control problem have been proposed in [25]. Generally, the empirical and semi-empirical models are curve fitted by the battery aging experimental data with the consideration of a simplified physical principle. The accuracy of such modeling is a little lower than the chemical models, but the computation is much less; and the battery health can be evaluated by the fitting function or look-up table to solve and improve the optimal control strategy without sacrificing control performance. The end of life is often defined as the battery capacity can only reach 80% of its initial value after a full charge. The battery health can be evaluated with the accepted assumption that under nominal conditions, the total ampere-hour flow in/out is a determined value during the whole life of a battery [22,27]. Accordingly, the battery life can be expressed as:

$$L={\int }_0^{t_f}\left|I_{nom}\right|dt$$

(14)

where $I_{nom}$ is the nominal current and $t_f$ denotes the end of life.

Due to the consideration of different currents, SOC and temperature will result into different aging effects under actual conditions. The work in [22] developed the weighted Ah-throughput model, which uses the concept of the severity factor for expressing the different damage multipliers of the battery health. Based on this model, the effective Ah-throughput can be defined as:

$${Ah}_{eff}={\int }_{t_0}^{t_f}\sigma (·)·\left|I_{batt}\right|dt$$

(15)

where $\sigma (·)$ is the severity factor under different conditions, namely the ratio of the damage degree of the actual condition to the nominal condition.

Then, it can be considered that when ${Ah}_{eff}=L$, the battery reaches its end of life. In other words, according to the hypothesis mentioned above that the total Ah-throughput is a certain value during its whole life, it may be regarded that the capacity loss is as $Q_{loss}\%={Ah}_{eff}/L$. Therefore, minimizing the battery health damage is equivalent to minimizing the effective Ah-throughput.

In this paper, the severity factor is estimated by the experimental data in [28,29]. Specifically, the severity factor is fitted as the following expression based on the experimental data:

$$\sigma (SOC,T_{batt})=f(SOC)\frac{e^{0.05T_{batt}}}{e^{0.05\times 25}}$$

(16)

with

$$f(SOC)=8.0401\times {SOC}^3-4.28741\times {SOC}^2-1.3087\times SOC+1.7263$$

(17)

According to the data, it is not hard to find that both high and low SOC cause a lot of damage to the battery health. Additionally, the operating temperature also has a large effect on battery aging. Every time the temperature of the battery rises 10 °C, the attenuation rate of battery health will increase about 1.5–2-times. Additionally, it should be noted that the nominal cycle in this paper is regarded as the C-rate being 1 (1/h), temperature being 25 (°C) and SOC being 0.5. According to (16), the severity factors of different SOC and temperatures are shown in Figure 5a.

However, if the battery life attenuation is directly measured by temperature and SOC is optimal, two state variables, $T_{batt}$ and $SOC$, need to be used in the SDP algorithm, which will be introduced in the next section. Usually, the SDP algorithm has a large calculation and long simulation time, and it is obvious that the use of two states will enormously increase the calculation. From Equations (11)–(13), it can be discovered that the change of temperature is largely influenced by current. Moreover, current can be introduced into the optimal problem without increasing the state because current is related to SOC. Hence, in order to reduce the number of state variables in the optimization algorithm, we translate the severity factor on temperature to the C-rate. By observing and analyzing the data provided by [29], a relationship between temperature change and C-rate can be obtained. Moreover, Equations (11)–(13) also indicate that temperature change is proportional to current. Therefore, combined with Equation (16), the function of the severity factor of different SOC and C-rates can be obtained as the following equation, and the fitting result is shown in Figure 5b.

$$\sigma (SOC,C-rate)=f(SOC)\frac{e^{0.05((0.507C-rat{e}^2+0.2906)\times 2+25)}}{e^{0.05\times 25}}$$

(18)

2.3. Driving Cycle Model

The commuter vehicle does not have a fixed route; however, a regular route can mostly be determined after a long run. That means the probability map of average speed for different distances can be obtained with the historical vehicle speeds by utilizing mathematical estimation algorithms, such as the maximum likelihood estimation. The process follows the steps [3]:

(1)

Collect the data of the regular driving speed.

(2)

Calculate the average speed according to a certain distance to obtain the average speed profile vs. the distance.

(3)

Divide the total distance into a number of segments.

(4)

Determine the probability distribution of the average speeds for each segment.

The probability distribution of the driving route used is shown in Figure 6. It can be seen that the whole distance is divided into eight segments with different distance lengths. The details of the drive cycle model are illustrated in [3].

3. Optimal Control Problem Formulation and Its Solution

According to the objectives and the models described previously, the control-oriented model can be represented by:

$$x(k+1)=f(x(k),u(k),w(k))$$

(19)

where x is the state variable, the battery SOC, $x(k)=SOC(k)$. u is the control variable, the generator speed and the motor torque $u(k)={[{\omega }_g(k)T_m(k)]}^T$. w is the external disturbance, the average vehicle speed $w(k)=\overline{v}(k)$, which has identical statistics in each segment. Additionally, $x(k)\in S,u(k)\in C,w(k)\in D$, where $S,C,D$ are finite sets and $S=\{1,2,\cdots ,s\}$.

Moreover, with the consideration of the feature of stochastic dynamic programming, the battery dynamics equation can be written in discrete form as Equation (20). Additionally, it should be noted that the discretization is based on a small distance $\Delta L$.

$$SOC(k+1)=SOC(k)+\frac{-V_{oc}+\sqrt{V_{oc}^2-4R_{batt}{P}_{batt}}}{2Q_{batt}{R}_{batt}}\frac{\Delta L}{{\overline{v}}_k}$$

(20)

Due to the two control objectives, minimizing energy consumption and minimizing battery health damage, the cost function can be formed as:

$$J=\underset{N\to \infty }{\min }\underset{{\overline{v}}_k}{E}\left\{\sum _{k=0}^{N-1}{\alpha }^{k}g(x_k,u_k,w_k)\right\}$$

(21)

where $\alpha$ is a discount factor whose value is between zero and one. Additionally, the cost functional g consists of two parts, energy consumption cost $g_1$ and the effective Ah-throughput $g_2$ described as Equation (22). Energy consumption includes the fuel cost and the electric cost, which are traded off by $\beta$.

$$g(k)=(1-\theta )\frac{[{\dot{m}}_{fuel}(k)+\beta ·P_{elec}(k)]·\Delta L/{\overline{v}}_k}{\Omega }+\theta \frac{\sigma (k)·\left|I_{batt}(k)\right|·\Delta L/{\overline{v}}_k}{\Lambda }$$

(22)

where $\beta$ is the coefficient that translates the electric cost to fuel cost at the same price. $\theta$ is the weight that combines two objectives. $\Omega$ represents the target energy consumption when only the energy consumption is considered. To explain in detail, $\Omega$ is the optimal value, which is solved by the SDP algorithm with no consideration of battery health, in other words, the weight value $\theta$ is zero. That is to say, $\Omega$ is the best value that can be reached in only energy consumption optimization, so it can be used for normalization. Additionally, in the same way, $\Lambda$ is the target Ah-throughput with only the consideration of battery health. ${\dot{m}}_{fuel}$ is the fuel flow rate, which is shown in Equation (7). Additionally, owing to the characteristic that the PHEV can be charged through an external power grid, electricity economy should also be taken into account in the control objectives besides the fuel economy. Electricity economy can be evaluated by the instantaneous power $P_{elec}$, which can be described as:

$$P_{elec}=V_{oc}{I}_{batt}\dot{SOC}$$

(23)

In addition, the energy management satisfies the constraints on the states and control actions, which correspond to physical limits.

$$\left\{\begin{array}{c}{SOC}_{\min }\le SOC\le {SOC}_{\max }\hfill \\ {\omega }_{e\_\min }\le {\omega }_e\le {\omega }_{e\_\max }\hfill \\ {\omega }_{m\_\min }\le {\omega }_m\le {\omega }_{m\_\max }\hfill \\ {\omega }_{g\_\min }\le {\omega }_g\le {\omega }_{g\_\max }\hfill \\ T_{e\_\min }({\omega }_e)\le T_e\le T_{e\_\max }({\omega }_e)\hfill \\ T_{m\_\min }({\omega }_m)\le T_m\le T_{m\_\max }({\omega }_m)\hfill \\ T_{g\_\min }({\omega }_g)\le T_g\le T_{g\_\max }({\omega }_g)\hfill \end{array}\right.$$

(24)

Furthermore, the policy iterations of the SDP and PSO algorithms are used for the optimal solution in this paper. When solving this multi-objective optimal problem via SDP, the Pareto set will be obtained by sweeping different weights between energy consumption and battery health. Additionally, in order to achieve the best economy, PSO is used for finding the weight coefficient that can be the best compromise between these two objectives. Specifically, the multi-objective optimization solution is obtained offline by the dynamic loop nest optimization of PSO and SDP based on the driving cycle model established in Section 2.3 utilizing the real historical traffic data. Because the established driving cycle model of the commuter PHEV has statistical characteristics, the whole commuting route is divided into eight segments according to the same probabilistic characteristics. Consequently, the energy management strategy of the whole route is composed of different management control strategies of eight different segments. The optimal management control strategy for each segment, including the weight coefficient and the control policy, is obtained by PSO and SDP based on the driving cycle model corresponding to the segment. In each segment, SDP optimizes the control policy, and PSO optimizes the weight coefficient of the objective function, which is important for calculating the optimal management control strategy because the results of the SDP optimization will be different with different weight coefficients $\theta$. When the weight coefficient $\theta =0$, the result of SDP is the optimal management strategy considering only fuel consumption. In the same way, when $\theta =1$, only battery life is considered. These two results are contradictory, so in order to achieve the optimal result, the PSO algorithm is used to find the appropriate $\theta$ to balance. The flowchart showing the entire optimization process with SDP and PSO is shown in Figure 7, in which the specific SDP and PSO algorithms first are presented as follows.

The SDP problem solved by policy iteration is framed as a stochastic optimal problem with infinite horizon (21) and (22). In order to make the energy management problem suitable for computer calculations, the discretization of the state variable, control inputs and cost function are given in Equations (19)–(22). The policy iteration algorithm consists of two main steps, policy evaluation and policy improvement. The two steps are successive and repeated iteratively until they converge to the best policy. The policy iteration is given in detail as follows.

Firstly, an initial policy ${\mu }^0=\{{\mu }^0(1),{\mu }^0(2),\cdots ,{\mu }^0(s)\}$ is guessed.

In the policy evaluation step, the cost function $J_{{\mu }^k}$ of the given control policy ${\mu }^k$ is calculated by:

$$J_{{\mu }^k}=g_{{\mu }^k}{(I-\alpha P_{{\mu }^k})}^{-1}$$

(25)

where I is the identity matrix. $P_{{\mu }^k}$ and $g_{{\mu }^k}$ are the probability and cost for the given policy ${\mu }^k$, which is defined by the following forms, respectively.

$$P_{{\mu }^k}=\left[\begin{array}{ccc}{p}_{11}({\mu }^k(1))& \cdots & p_{1s}({\mu }^k(1))\\ ⋮& ⋮& ⋮\\ p_{s1}({\mu }^k(s))& \cdots & p_{ss}({\mu }^k(s))\end{array}\right],g_{{\mu }^k}=\left[\begin{array}{c}g(1,{\mu }^k(1))\\ ⋮\\ g(s,{\mu }^k(s))\end{array}\right]$$

(26)

where $p_{ij}(·)$ is the one-step transition probability from state i to the next state j and $g_i(i,·)$ is the cost of each stage, which can be calculated by the following equations.

$$p_{ij}(u(i))=P\{W_{ij}(u(i))|i,u(i)\},i=1,\dots ,s$$

$$W_{ij}(u(i))=\{w\in D|f(i,u(i),w)=j\}$$

$$g(i,u(i))=\sum _{j=1}^s{p}_{ij}(u(i))\tilde{g}(i,u(i),j)$$

$$\tilde{g}(i,u(i),j)=\underset{w\in W_{ij}(u(i))}{E}\{g(i,u(i),w)\}$$

where D is the finite set of the stochastic disturbances with identical statistics. $W_{ij}(u(i))$ represents the set of random disturbances, which can move the state from i to j by the control policy $u(i)$. Additionally, $g(i,u(i),w)$ is the cost of moving the state from i to j by $u(i)$.

Then, proceeding to the next step, the policy improvement step, a new control policy ${\mu }^{k+1}$ for the stochastic control problem is obtained, which satisfies the following equations.

$$T_{{\mu }^{k+1}}{J}_{{\mu }^k}(i)=T{J}_{{\mu }^k}(i)$$

(27)

where:

$$\left\{\begin{array}{c}{T}_{{\mu }^{k+1}}{J}_{{\mu }^k}(i)=g(i,{\mu }^{k+1}(i))+\alpha {\displaystyle \sum _{j=1}^s}{p}_{ij}({\mu }^{k+1}(i))J_{{\mu }^k}(j)\hfill \\ T{J}_{{\mu }^k}(i)=\underset{u\in C}{\min }\left[g(i,u)+\alpha {\displaystyle \sum _{j=1}^s}{p}_{ij}(u)J_{u^k}(j)\right]\hfill \end{array}\right.,i=1,\dots ,s$$

In this paper, the PSO algorithm with the fitness function defined as (21) and (22) is used to choose the best tradeoff coefficient between two objectives. In the PSO algorithm, a group of random particles first is initialized. Then, the optimal result is searched iteratively, and in each iteration, particles update themselves constantly by tracking two extreme values, the individual extremum ($g_{ibest}$) and the global extremum ($g_{gbest}$). More concretely, through solving the SDP algorithm with the weight value $\theta$, which is represented by the current particle individual extremum, the best performance, $g_{ibest}$, is calculated by Equation (22). Obviously, $g_{gbest}$ is the minimum value of all $g_{ibest}$. The update operation is the core of the PSO algorithm, and the principles of velocity $v_i$ update and position $x_i$ update are presented as:

$$v_i^{(t+1)}=\omega v_i^{(t)}+c_1{r}_1(g_{gbest}-x_i^{(t)})+c_2{r}_2(g_{ibest}-x_i^{(t)})$$

(28)

with:

$$\left\{\begin{array}{ccc}{v}_i^{(t+1)}=v_{\max },\hfill & \mathrm{if}\hfill & v_i^{(t+1)}>v_{\max }\hfill \\ v_i^{(t+1)}=v_{\min },\hfill & \mathrm{if}\hfill & v_i^{(t+1)}<v_{\min }\hfill \end{array}\right.$$

$$x_i^{(t+1)}=x_i^{(t)}+v_i^{(t+1)}$$

(29)

with:

$$\left\{\begin{array}{ccc}{x}_i^{(t+1)}=x_{\max },\hfill & \mathrm{if}\hfill & x_i^{(t+1)}>x_{\max }\hfill \\ x_i^{(t+1)}=x_{\min },\hfill & \mathrm{if}\hfill & x_i^{(t+1)}<x_{\min }\hfill \end{array}\right.$$

where t represents the iteration step. $v_i^{(t)}$ and $x_i^{(t)}$ represent the velocity and position of the i-th particle in the t-th step. $c_1,c_2$ are the acceleration factors, which usually are chosen as positive constants, and $r_1,r_2$ are random numbers between zero and one. $v_{\max }$, $v_{\min }$ and $x_{\max }$, $x_{\min }$ are the maximum and the minimum of $v_i^{(t)}$ and $x_i^{(t)}$. As the particles represent the weight coefficients in Equation (22), the following settings are given: $v_{\max }=0.1$, $v_{\min }=-0.1$, $x_{\max }=1$, $x_{\min }=0$. It should be noted that $\omega$ is the inertia weight, which will be explained below.

In order to balance the global search ability and local improvement ability of particle swarm optimization, a dynamic inertia weight as Equation (30) is used in this paper. When the fitness value is greater than the average fitness of all particles, the maximum inertia weight is selected to strengthen the global search capability. Once the fitness value is no longer large, the nonlinear dynamic inertia weight is selected according to the current fitness and the average fitness.

$$\omega (t)=\left\{\begin{array}{ccc}{\omega }_{\max }-({\omega }_{\max }-{\omega }_{\min })\frac{f-f_{\min }}{f_{avg}-f_{\min }},\hfill & \mathrm{if}\hfill & f\le f_{avg}\hfill \\ {\omega }_{\max },\hfill & \mathrm{if}\hfill & f>f_{avg}\hfill \end{array}\right.$$

(30)

where ${\omega }_{\max }$ and ${\omega }_{\min }$ represent the maximum and the minimum of $\omega$, respectively. Typically, ${\omega }_{max}=0.9$ and ${\omega }_{\min }=0.4$. f is the current value of the particle. $f_{avg}$ is the average value, and $f_{min}$ is the minimum value of the current iteration step.

From Figure 7, obviously, $g_{{\mu }^k}$ is very important in the SDP iteration to obtain the optimal strategy, and Equation (22) is used to calculate $g_{{\mu }^k}$. The particles that represent the different weights are updated by PSO, and SDP problems are solved with these particles, respectively. The optimal control policy ${pol}_{ibest}$ and optimal cost $g_{ibest}$ are obtained, and $g_{ibest}$ is also the individual extremum of the i-th particle. A global extremum can be found by updating the particles to find the optimal compromise weight coefficient in each segment; the corresponding optimal policy solved by SDP is the final solution expected, the power-split. Accordingly, the weight coefficient $\theta$ optimized offline by PSO is a parameter in each segment, but in the whole route, it is a mapping on the different road traffic segment. The final control policy optimized offline by SDP is a static state-feedback type on battery SOC in each segment, but in the whole route, it is a 2D-mapping on SOC and different road traffic segments. That is to say, the obtained energy management strategy in the whole route is optimized offline by PSO and SDP and is formed as mappings on the different road information, battery SOC and power demand based on the real historical road traffic data. These mappings will be embedded in the energy management control system of PHEV in actual operation, and the management control strategy is generated online by looking up the mappings according to the current system states such as SOC, power demand and road information.

4. Simulation Results

The GT-SUITE-based PHEV simulator is used to verify the effectiveness of the presented energy management strategy achieving the control objectives. The PHEV model described in Section 2 is a control-oriented model, but the model in the GT-SUITE PHEV simulator is further detailed in order to simulate the real vehicle powertrain more realistically. The PHEV system with supervisory controller in the simulation is constructed as shown in Figure 8. Additionally, the parameters and specifications of model components are the same as in Table 1 [24].

PHEVs, which are different from traditional HEVs, have a large SOC range when they work. In this simulation, the battery SOC range in the simulation is chosen as 0.3–0.9. Hence, the initial SOC of the vehicle each day is 0.9. In order to ensure the physical limits, in the real-time simulation of the proposed energy management strategy, the reasonable rules are joined into the supervisory controller besides the ${\omega }_g,T_m$ map, which are calculated by the SDP algorithm. Therefore, the supervisory controller consists of two parts: one is the control strategy calculated by the off-line algorithm, and the other is the reasonable rules. The control strategy is given by static maps ($SOC,w_g$) and ($SOC,T_m$). Through the combination of the two parts, the required torques $T_{er},T_{mr},T_{gr}$ are obtained.

The driver model and PHEV model are provided by GT-SUITE. The ‘ControllerHEVehicle’ component, which can calculate the necessary tractive power for a target vehicle speed, is used to calculate the driving power required at the vehicle to match the driving cycle. The PHEV model is also given by [24] in GT-SUITE. It is should be noted that the original model in [24] is HEV. Therefore, in order to adapt to our research, some parameters have been modified. The major modifications are changing the initial SOC of the battery and the threshold SOC of regenerative braking.

The cost function evaluator, as a module for evaluating the performance of our supervisory controller, has two parts. One part is the energy consumption evaluator, which calculates the fuel consumption by Equation (7) and electric consumption by Equation (23). The other part is the battery health evaluator. For the sake of battery life evaluation, the semi-empirical battery life model is used. As introduced in Section 2, the approximate severity factor map of SOC and C-rate is used to estimate the battery health in the off-line SDP algorithm in order to reduce the number of states. However, in the real-time simulation, we employ the more accurate severity factor map obtained from fitting the battery life experimental data to different temperatures and SOC, which is shown in Figure 5. Therefore, SOC and $T_{batt}$ need to be obtained in the real-time simulation. SOC can be obtained directly by the PHEV model on GT-SUITE, and $T_{batt}$ can be calculated by Equations (11)–(13) with the value of $I_{batt}$ provided by GT-SUITE. Then, we use Equation (15) to evaluate the decay of battery life quantitatively.

The drive cycle model provides the driver’s target speed and gives the target speed to the driver, which is simulated by GT-SUITE. The actual traffic routes [3,24] of 15 days introduced in Section 2 are used in the simulation, and the route from home to office is divided into eight segments according to the stochastic consistent characteristics. The PHEV is considered to be charged once a day when parked at home. Therefore, the round trip from home to the workplace is chosen as the whole route of each day in the real-time simulation. It should be noted that the segments are divided according to the distance length from home. Therefore, the sectional strategies of the journey to work select 1∼8 segments in turn and select the 8∼1 segments strategies from work to home.

In the research, the simulation is divided into two parts. The first part is to calculate off-line the optimal control strategy by the SDP and PSO algorithm. Subsequently, the real-time simulation is carried out based on the strategy obtained on the GT-SUITE and MATLAB platform. Specifically, each segment optimal control policy can be obtained by the SDP and PSO algorithms shown in Figure 7. According to the eight statistical probability characteristics of commuting routes, the eight static maps ($SOC,w_g$) and ($SOC,T_m$) obtained for the whole route are shown in Figure 9. In the implementation, for different segments and SOC, the corresponding control variables $w_g$ and $T_m$ can be obtained by looking up these maps according to the current system states. In addition, since these maps are obtained based on the different probability characteristics of each segment in the whole route, the generated real-time management control strategy by these mappings is feasible with respect to the real road traffic information in the actual operation. At the same time, the current management control strategy can be obtained merely by looking up maps, so there is no computational burden during real-time operation.

When solving the SDP algorithm with given fixed weight $\theta$ in Equation (22), a Pareto set will be obtained by sweeping weight $\theta$ from 0 to 1. However, in our research, PSO is used for finding the weight coefficient that would be the best compromise between energy consumption and battery health. An optimal policy can be gained with eight different weight $\theta$ for eight segments. The solutions are shown in Figure 10. It explains that the optimization performance of using different weights of different segments is better than using the same weight of the whole route.

Figure 11 shows the result with the consideration of both energy consumption and battery life for the second week Monday speed profiled to demonstrate that the proposed energy management strategy can satisfy the vehicle power demand for drivability. Because of the planetary gear structure, the PHEV has several operating modes. The PHEV can be driven only by the motor when the speed is low and the battery capacity is adequate. Additionally, when the demanded power is great, customarily, the PHEV is driven at a high speed or in a climbing condition, and the vehicle is propelled jointly by the engine and motor. In that mode, the engine can work on the optimal operation line and recycle braking power to charge the battery.

Moreover, the simulation results of 15 days are listed in

Table 2. Simulation results.

Day	Only Energy Considered		Both Considered		Only Battery Considered
	Consumption	Battery ${\mathbf{Q}}_{\mathbf{loss}}$	Consumption	Battery ${\mathbf{Q}}_{\mathbf{loss}}$	Consumption	Battery ${\mathbf{Q}}_{\mathbf{loss}}$
	(L/100 km)	(%/100 km)	(L/100 km)	(%/100 km)	(L/100 km)	(%/100 km)
Monday 1	2.569	6.863 × ${10}^{-4}$	2.676	6.178 × ${10}^{-4}$	3.058	6.118 × ${10}^{-4}$
Tuesday 1	2.955	6.997 × ${10}^{-4}$	3.097	6.359 × ${10}^{-4}$	3.269	6.513 × ${10}^{-4}$
Wednesday 1	2.462	6.989 × ${10}^{-4}$	2.530	6.526 × ${10}^{-4}$	2.534	6.471 × ${10}^{-4}$
Thursday 1	2.827	7.627 × ${10}^{-4}$	2.913	6.842 × ${10}^{-4}$	3.131	6.678$\times {10}^{-4}$
Friday 1	2.523	7.734 × ${10}^{-4}$	2.490	7.098 × ${10}^{-4}$	2.542	6.900 × ${10}^{-4}$
Monday 2	2.525	6.785 × ${10}^{-4}$	2.614	6.185 × ${10}^{-4}$	3.150	6.081 × ${10}^{-4}$
Tuesday 2	3.142	7.611 × ${10}^{-4}$	3.311	7.343 × ${10}^{-4}$	3.454	7.253 × ${10}^{-4}$
Wednesday 2	2.686	7.568 × ${10}^{-4}$	2.745	6.831 × ${10}^{-4}$	2.787	6.786 × ${10}^{-4}$
Thursday 2	2.907	7.237 × ${10}^{-4}$	3.029	6.579 × ${10}^{-4}$	3.483	6.421 × ${10}^{-4}$
Friday 2	2.361	7.150 × ${10}^{-4}$	2.450	6.712 × ${10}^{-4}$	2.499	6.632 × ${10}^{-4}$
Monday 3	2.824	7.314 × ${10}^{-4}$	2.910	6.551 × ${10}^{-4}$	3.436	6.492 × ${10}^{-4}$
Tuesday 3	2.746	6.925 × ${10}^{-4}$	2.843	6.678 × ${10}^{-4}$	2.929	6.663 × ${10}^{-4}$
Wednesday 3	2.262	6.597 × ${10}^{-4}$	2.294	6.343 × ${10}^{-4}$	2.290	6.294 × ${10}^{-4}$
Thursday 3	2.922	7.266 × ${10}^{-4}$	2.937	6.590 × ${10}^{-4}$	2.970	6.481 × ${10}^{-4}$
Friday 3	2.946	7.342 × ${10}^{-4}$	3.073	7.043 × ${10}^{-4}$	3.139	6.939 × ${10}^{-4}$
average	2.711	7.200 × ${10}^{-4}$	2.794	6.657 × ${10}^{-4}$	2.978	6.581 × ${10}^{-4}$

for three strategies, only energy consumption considered, only battery life considered and both energy consumption and battery life considered. More concretely, three different weight values $\theta$ in Equation (22) are chosen in order to demonstrate that the proposed strategy can largely reduce the battery life degradation without increasing fuel consumption too much. When the battery life is the only object to consider, $\theta =1$. Additionally, when energy consumption is the only object, $\theta =0$. Calculated by the particle swarm algorithm, eight different weight values $\theta$ for eight segments are used in the case that both energy consumption and battery life are considered. The results shown in Table 2 also demonstrate for the Pareto chart explanation of Figure 10 that the optimization performance of using the optimized different weights of different segments is the best compromise.

In Figure 11, it can be seen that the PHEV can work in pure electric mode for a longtime. Therefore, it has better fuel performance than the fuel vehicle and the traditional HEV. The battery capacity and the motor maximum power of the PHEV model used in this paper are much smaller. However, it is also shown that the one-hundred-kilometer fuel consumption is less than 3 L in Table 2. The lesser average fuel consumption can reflect that PHEV is more dependent on battery performance than the traditional HEV. Hence, the considerations of battery health are essential.

According to the data in Table 2, it is shown intuitively that fuel consumption inevitably increases as the battery health deterioration decreases. If only considering the battery health, the battery ${\mathrm{Q}}_{\mathrm{loss}}$ is reduced by 8.5%, but the energy consumption is increased by 9.8% compared to energy consumption alone considered. This result has led to sacrificing greater energy consumption while improving battery health, which is clearly not conducive to the improved economic performance of the vehicle. However, the method proposed in this paper, which uses SDP and PSO calculating different weight values for different segments, can be a good solution to this problem. It can be seen that with the consideration of both energy consumption and battery life, battery life loss has been eased without too much energy consumption.

In this case, the simulation results show that the battery ${\mathrm{Q}}_{\mathrm{loss}}$ is reduced by 7.6%, while the energy consumption is only increased by 3.2%. Obviously, the proposed approach can increase the useful life of the vehicle’s battery, at the same time, as much as possible to reduce the energy consumption. Although the energy consumption still increases, it may increase the useful life of PHEV by nearly two years before the battery is damaged and then improve the whole vehicle’s economic performance.

Figure 12 shows the comparison of the three cases in Table 2 more clearly. The second week Thursday speed profile is taken as an example. In the three cases, SOC starts from 0.9 and gradually declines with a general trend. At the end of the day, SOC reaches a lower value to make full use of the stored energy in the batteries, which can be charged through the grid. Figure 5 in Section 2 shows that when SOC remains near 0.5, the speed of battery health decay is slowest. Thus, the energy management control strategies that consider battery life keep the battery SOC in a relatively better condition for a long time. At the same time, the temperature of the battery is also a major factor that affects battery life, as was introduced in Section 2. Additionally, it is easy to identify that the battery temperature is controlled so as not to be too high with the consideration of battery health by observing the change of temperature in Figure 12c. As the results shown in Figure 12d,e, it can be seen that the consideration of battery life will increase the fuel consumption. However, in order to achieve a better performance with respect to the total cost, the policy of considering both energy consumption and the battery can reduce the battery degradation with a much smaller energy consumption increase.

5. Conclusions

This paper solves the energy management optimization problem for PHEVs with the consideration of the battery health, besides the energy consumption. The employed battery lifetime model is a semi-empirical model in which the effects of battery temperature and SOC are considered. It is obvious that there is a contradiction between limiting the battery health degradation and reducing fuel consumption. Therefore, the energy management strategy considering battery life can be converted into a multi-objective optimization problem. Additionally, stochastic dynamic programming (SDP) and particle swarm optimization (PSO) are used for solving this multi-objective optimal control, which balanced energy consumption and battery health. Furthermore, the multi-objective optimization solution, including the compromise weight coefficient and the power-split control policy, is obtained offline by the dynamic loop nest optimization of PSO and SDP according to the statistical characteristics of each segment in the whole commuting route based on the real historical traffic data, and the energy management strategy of the whole route is formed as mappings on different road segments and battery SOC. In the implementation, the actual management control strategy is generated online by looking up these mappings according to the current system states of SOC, power demand and road information without computational burden. Meanwhile, the generated real-time management control strategy by these mappings is feasible with respect to the real road traffic information in the actual operation since these maps are obtained based on the different probability characteristics of each segment in the whole route, which is also validated by the real-time simulation in the GT-SUITE-based PHEV test platform. In addition, from the simulation verification, it can be seen that reduce battery life loss will inevitably lead to fuel consumption increase; however, the multi-objective energy management strategy proposed in this paper can be used to achieve the target of reducing the battery life degradation greatly without increasing the fuel consumption much.