# SDP Policy Iteration-Based Energy Management Strategy Using Traffic Information for Commuter Hybrid Electric Vehicles

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Problem Description

#### 2.1. Powertrain Model

- (1)
- Motor alone propels the vehicle. The motor can be powered by either the battery or the generator that transforms the mechanical power generated by the engine into the electrical power. i.e., the driver propelling power demand P
_{trac,dem}, and battery discharge power P_{batt,dis}can be written as:P_{trac,dem}= P_{m}= T_{m}ω_{m}P_{batt,dis}= P_{m}/ η_{m}_{m}, T_{m}, ω_{m}are the motor power, torque and speed, respectively. η_{m}is the motor efficiency, which is generally a function of the motor torque and speed. - (2)
- Engine alone propels the vehicle, in which the mechanical power generated by the engine is transmitted to the vehicle from the carrier gear directly to the ring gear connected to the final drive. Meanwhile, the excess engine power can be transformed to the electrical form through the generator and then pumped into the battery. i.e., the driver propelling power demand P
_{trac,dem}, and battery charge power P_{batt,ch}can be written as:P_{trac,dem}= T_{r}ω_{r}= T_{r}ω_{m}P_{batt,ch}= η_{g}P_{g}= η_{g}T_{g}ω_{g}_{g}, T_{g}, ω_{g}are the generator power, torque and speed, respectively. η_{g}is the generator efficiency, generally, which also is a function of its torque and speed. T_{r}, ω_{r}are the ring gear torque and speed, respectively. - (3)
- Engine and motor jointly propel the vehicle, i.e., the demand power of the vehicle is provided by both engine and motor. However, the motor may be powered by the generator, besides by the battery. i.e., the driver propelling power demand P
_{trac,dem}, and battery power P_{batt}can be written as:P_{trac,dem}= T_{r}ω_{r}+ T_{m}ω_{m}= (T_{r}+ T_{m})ω_{m}P_{batt}= η_{g}P_{g}+ P_{m }/ η_{m}= η_{g}T_{g}ω_{g}+ T_{m}ω_{m}/ η_{m} - (4)
- The vehicle experiences braking. Here we only consider when the demanded braking power is less than the maximum regenerative braking power that the motor can supply. Then, the motor is controlled to function as a generator to produce a braking power that equals the commanded braking power. i.e., the driver braking power demand P
_{brak,dem}, and battery charge power P_{batt,ch}can be written as:P_{brak,dem}= T_{m}ω_{m}P_{batt,ch}= η_{m}P_{m}= η_{m}T_{m}ω_{m}

_{dem}and the battery power P

_{batt}can be represented as:

_{dem}= T

_{e}ω

_{e}+ T

_{g}ω

_{g}+T

_{m}ω

_{m}

_{batt}> 0 indicates the battery is discharging and P

_{batt}< 0 means charging state. P

_{m}< 0, P

_{g}< 0 represent generating states and P

_{m}> 0, P

_{g}> 0 represent motoring states, and:

_{s}, ring gear ω

_{r}, and carrier gear ω

_{c}, have the relationship:

_{r}+ R

_{s})ω

_{c}= R

_{r}ω

_{r}+ R

_{s}ω

_{s}

_{r}, R

_{s}are the radii (or number of teeth) of the ring gear and sun gear respectively.

_{s}, ring gear T

_{r}, and carrier gear T

_{c}have the relationship:

_{g}, J

_{e}, J

_{m}, are the inertia of the generator, the engine, and the motor, respectively. T

_{trac}is the torque on the axle of the differential gear, and g

_{f}is the final differential gear ratio.

_{tire}is the radius of the tire.

_{f}is the transmission efficiency of differential gear; T

_{br}the friction brake torque; μ

_{r}coefficient of rolling resistance; ρ air density; A frontal area of vehicle; C

_{d}drag coefficient; and θ road angle (grading).

_{f}(g/s), which is defined as follows:

_{f}= BSFC × P

_{e}× 10

^{−5}/ 36

_{elec}= V

_{oc}I

_{batt}= −V

_{oc}I

_{batt}SȯC

_{oc}, I

_{batt}, Q

_{batt}, SoC are battery open-circuit voltage, cuurent, maximum charge capacity, and state of charge, respectively. The dynamics of battery SoC can be represented by:

_{b}is battery internal resistance. Both V

_{oc}and R

_{b}are functions on battery SoC, which can be obtained through curve-fitting a predefined map.

#### 2.2. Traffic Information

**Figure 3.**Traffic speed information on Monday, Wednesday, Friday in one week. (

**a**) The instantaneous speed vs. time and vs. distance; (

**b**) the average speed vs. distance.

**Figure 4.**Traffic speed information on Mondays in three weeks. (

**a**) The instantaneous speed vs. time and vs. distance; (

**b**) the average speed vs. distance.

#### 2.3. Problem Formulation

- (1)
- The statistical characteristic of the traffic speed profile information from the collected data is modeled for achieving and generating scenarios in the stochastic approach for energy management.
- (2)
- The design objective of the energy management is achieved, i.e., the statistical characteristic, rather than the reference driving cycle, is utilized to design energy management strategy so as to guarantee the optimization for fuel-electricity consumption, the battery state-of-charge maintained within some specified limits, and the power demand for drivability of hybrid electric vehicles irrespective of real traffic flow and driving conditions.
- (3)
- The compromises, between the benefit of reduced trip information and computing requirement, and the expense of reduced performance, are considered in the design of the control policy.

## 3. Energy Management Based on SDP

#### 3.1. Traffic Information Modeling

- (1)
- To calculate the average speed according to a distance of 200 m to obtain the average speed profile vs. the distance for the adequate speed profile sampled, which on Monday, Wednesday and Friday in one week has been shown in Figure 3.
- (2)
- To divide the total distance into a number of segments, according to the similar statistics of the averaged speed vs. distance observed in a certain segment for all sampled day data. In this study, the total distance is 14 km and divided into eight segments, in which the distance of the each segment is:L = [0–2 km, 2–2.8 km, 2.8–4.4 km, 4.4–6 km, 6–7.2 km, 7.2–8 km, 8–11.2 km, 11.2–14 km]
- (3)
- To determine the probability distribution of the average speeds for each segment. Each segment has a different probability distribution of the average speed profile, but, it can be assumed that its probability distribution is invariant in the segment and fitted to the normal distribution.

_{j}is the j-th sampled data of the total sampled data n in the i-th segment.

_{1}(v) in the first segment. Similarly, the probability distribution Prob

_{j}(v), j = 1,2,···,8 in each segment of the driving route can be obtained, which is shown in Figure 8.

#### 3.2. Stochastic Process Model and Optimization Problem

_{m}and the generator speed ω

_{g}are control inputs, and the average vehicle speed in term of distance as stochastic disturbance. Meanwhile, for obtaining control laws T

_{m}and ω

_{g}by the discrete stochastic dynamic programming, the battery dynamics Equation (20) is rewritten in the following discrete form:

_{k}

_{+1}= f(SoC

_{k},ω

_{g,k},T

_{m,k},v

_{k}) = SoC

_{k}+ ΔSoC

_{k}

_{batt}(ω

_{g,k},T

_{m,k},v

_{k}) is described as:

_{dem}is also the average power, which can be determined by:

_{k}= SoC

_{k}, the control input is u

_{k}= (ω

_{g,k},T

_{m,k}) and the stochastic disturbance is w

_{k}= v

_{k}. And x

_{k}∈ S, u

_{k}∈ C, v

_{k}∈ D,S,C,D are finite sets and S = {1, 2, ···, s}. u

_{k}is constrained to take values in a given nonempty subset U(x

_{k}) of C, i.e., u

_{k}∈ U(x

_{k}), ∀x

_{k}∈ S. The random disturbances w

_{k}has identical statistics [20]. Furthermore, the stochastic optimization problem is formulated as follows:

_{min}≤ SoC ≤ SoC

_{max}, ω

_{e}

_{min}≤ ω

_{e}≤ ω

_{e}

_{max}, ω

_{g}

_{min}≤ ω

_{g}≤ ω

_{g}

_{max},

T

_{e}

_{min}≤ T

_{e}≤ T

_{e}

_{max}, T

_{m}

_{min}≤ T

_{m}≤T

_{m}

_{max}, P

_{e}≤ P

_{erated}, P

_{m}≤ P

_{mrated}, P

_{g}≤ P

_{grated}

_{fuel,comp}+ g

_{elec,comp}, which are expressed as:

_{fuel,comp}(ω

_{g,k},v

_{k})=ṁ

_{f,k}· H

_{l}· ΔL/v

_{k}= BSFC

_{k}· P

_{e,k}· H

_{l}· 10

^{-3}· ΔL/v

_{k}

_{elec,comp}(SoC

_{k},ω

_{g,k},T

_{m,k},v

_{k})= γ · P

_{elec,k}· ΔL/v

_{k}· 3.6 · 10

^{−3}

_{k}is utilized in term of the following formula, which is a map fitting by the least-square algorithm:

_{k},ω

_{g,k},T

_{m,k}represent the average measure of the vehicle speed, generator speed and motor torque at the end of the interval ΔL.

#### 3.3. Control Policy Iteration of SDP for Optimal Solution

_{g}) and (SoC, T

_{m}) are shown in Figure 10, notice that the accuracy is limited by the grid size on each state.

_{g}and the motor torque T

_{m}, the demand torques T

_{er}, T

_{gr}, T

_{mr}for the power sources can be obtained. Since engine can be controlled at its optimal operating area as long as it is operating, which indicates the engine torque T

_{e}can be a function on the engine speed ω

_{e}through the curve-fitting the optimal torque operating line. Consequently, combining the determined ω

_{g}and the vehicle speed can derive the engine speed ω

_{e}, and then the demand torque for engine T

_{er}is obtained. Meanwhile, combining the determined T

_{mr}and the traction power demand can derive the demand torque for generator T

_{gr}.

## 4. Simulation Validation and Observation

Notation | Meaning | Value (Unit) |
---|---|---|

M | Vehicle mass | 1460 (kg) |

ρ | Air density | 1.293 (kg/m^{3}) |

C_{d} | Air drag coefficient | 0.33 |

A | Frontal area of vehicle | 3.8 (m^{2}) |

μ | Coefficent of rolling resistance | 0.015 |

g_{f} | Final differential gear ratio | 4.113 |

R_{tire} | Radius of the tire | 0.2982 (m) |

J_{e} | Inertia of the engine crankshaft | 0.16 (kg·m^{2}) |

J_{m} | Inertia of the motor | 0.035 (kg·m^{2}) |

J_{g} | Inertia of the generator | 0.0265 (kg·m^{2}) |

ε | planetary gear ratio | 0.3846 |

η_{m} | Efficiency of EM as motor | 0.8301 |

η_{g} | Efficiency of EM as generator | 0.876 |

Q_{max} | Battery maximum charge capacity | 6.5 (Ah) |

_{g}

^{*}, T

_{m}

^{*}from SDP-based management strategy, the component transforming the control policy command ω

_{g}

^{*}, T

_{m}

^{*}into the torque demands T

_{er}, T

_{gr}, T

_{mr}distributed to the power sources, and some logical strategies in terms of the limitations for thermal and/or mechanical conditions as well as a whole operation range of boundary conditions. The schematic diagram of the torque-split controller is shown in Figure 13.

- (1)
- For sensory evaluation, the designed control policy shown in Figure 10, i.e., each segment in whole route has itself policy C
_{i}→ L_{i}, i = 1, 2, ···, 8, is executed in the HEV powertrain control system as shown Figure 11, to show the performance on the fuel-electricity consumption, charge sustenance, and drivability. Where the policy C_{i}→ L_{i}, i = 1, 2, ···, 8, is called full-policy. - (2)
- For comparative evaluation, a fixed single policy, such as C
_{2}is used for the whole route, i.e., the whole route only adopts the policy C_{2}corresponding to the second segment L_{2}, executed in the HEV powertrain control system, to show the comparative performance on the fuel-electricity between the full-policy and the fixed policy C_{i}, i = 1, 2, ···, 8. - (3)
- Similarly, a comparative evaluation is given by the results of driving speed profile only for a certain segment in the whole route. The comparison is made between the single policy corresponding to this segment and other single policies. For example, in terms of the driving speed profile in the segment L
_{3}, comparison is given between the result of executing C_{3}and that of executing C_{7}.

**Figure 14.**Result with the full-policy for the third Monday speed profile. (

**a**) Vehicle speed, SoC and fuel-electricity consumption; (

**b**) batter powery, demand power and engine, generator and motor power outputs; (

**c**) engine, generator, motor speeds and their torques; (

**d**) engine operating point densities.

**Figure 15.**Result with the full-policy for the first Wednesday speed profile. (

**a**) Vehicle speed, SoC and fuel-electricity consumption; (

**b**) Batter power, demand power and engine, generator and motor power outputs; (

**c**) Engine, Generator, motor speeds and their torques; (

**d**) Engine operating point densities.

_{i}, i = 1, 2, ···, 8 lie in the third column to the 10th column, respectively.

[km/L] | Full-poli | S1-poli | S2-poli | S3-poli | S4-poli | S5-poli | S6-poli | S7-poli | S8-poli |
---|---|---|---|---|---|---|---|---|---|

Mon1 | 30.9056 | 30.9056 | 30.8669 | 30.9056 | 30.9056 | 30.9056 | 30.9056 | 30.9056 | 30.9056 |

Mon2 | 28.3881 | 28.3881 | 27.5434 | 28.3881 | 28.3881 | 28.3881 | 28.3881 | 28.3881 | 28.3881 |

Mon3 | 33.7096 | 33.7096 | 33.7096 | 33.7096 | 33.7096 | 33.7096 | 33.7096 | 33.7096 | 33.7096 |

Tue1 | 25.5455 | 25.5455 | 25.4511 | 25.5455 | 25.5455 | 25.5455 | 25.5455 | 25.5455 | 25.5455 |

Tue2 | 19.8163 | 19.8163 | 16.2463 | 19.8163 | 19.2507 | 19.2163 | 19.8163 | 19.0881 | 19.8163 |

Tue3 | 20.7915 | 16.2526 | 15.2868 | 16.2526 | 21.0331 | 14.5505 | 16.2526 | 18.6054 | 20.2932 |

Wed1 | 22.1123 | 22.1123 | 19.0863 | 22.1123 | 22.1123 | 22.1123 | 22.1123 | 22.1123 | 22.1123 |

Wed2 | 28.8094 | 28.8094 | 28.6970 | 28.8094 | 28.8094 | 28.8094 | 28.8094 | 28.8094 | 28.8094 |

Wed3 | 26.7733 | 26.7733 | 26.7733 | 26.7733 | 26.7733 | 26.7827 | 26.7733 | 26.7827 | 26.7733 |

Thur1 | 26.6242 | 26.6242 | 26.3172 | 26.6242 | 26.6242 | 26.6242 | 26.6242 | 26.6242 | 26.6242 |

Thur2 | 22.3535 | 22.3535 | 19.2765 | 22.3535 | 22.3229 | 22.3207 | 22.3535 | 22.3124 | 22.3535 |

Thur3 | 25.5654 | 25.5654 | 25.5654 | 25.5654 | 25.5654 | 25.5654 | 25.5654 | 25.5654 | 25.5654 |

Fri1 | 30.4161 | 30.4161 | 17.5464 | 30.4161 | 30.4161 | 30.4161 | 30.4161 | 30.4161 | 30.4161 |

Fri2 | 23.3546 | 23.3546 | 20.8797 | 23.3552 | 23.2168 | 23.2069 | 23.3552 | 23.1733 | 23.3546 |

Fri3 | 25.5654 | 25.5654 | 25.5654 | 25.5654 | 25.5654 | 25.5654 | 25.5654 | 25.5654 | 25.5654 |

Averg | 26.0487 | 25.7462 | 23.9208 | 25.7462 | 26.0159 | 25.5892 | 25.7462 | 25.8401 | 26.0155 |

_{i}→ L

_{i}can guarantee better fuel economy in an average sense than the control strategy with a fixed single policy in the whole route. It should be noted that the so-called summing-up in an average sense results not from the averaged value meanings, but rather the essential characteristic of the stochastic optimization. On the other hand, there is no doubt that the control policy of the stochastic optimization problem is dependent of the adequate statistical analysis. Thus, it is presumable that the appearance of not always optimal results from the question of whether the collected data is adequate enough for statistical analysis.

**Figure 16.**Comparison of the full-policy and a fixing single policy-1, single policy-5 in the third Tuesday speed profile. (

**a**) Vehicle speed, SoC and fuel-electricity consumption; (

**b**) engine operating point densities.

**Figure 17.**Comparison of the full-policy and a fixing single policy-2 in the first Friday speed profile. (

**a**) Vehicle speed, SoC and fuel-electricity consumption; (

**b**) engine operating point densities.

_{3}corresponding to the third segment and the single policy-7 C

_{7}corresponding to the seventh segment are chosen as examples to give the comparative validations. The comparison results of two single policies in the corresponding segment and the opposite side segment are shown in Figure 18 and Figure 19, respectively, which illustrate the effectiveness of the control policy in the corresponding segments.

**Figure 18.**Comparison of the policy-3 and the policy-7 in the third segment driving data. (

**a**) Vehicle speed, SoC and fuel-electricity consumption; (

**b**) engine operating point densities.

**Figure 19.**Comparison of the policy-3 and the policy-7 in the seventh segment driving data. (

**a**) Vehicle speed, SoC and fuel-electricity consumption; (

**b**) engine operating point densities.

## 5. Conclusions

## Author Contributions

## Conflicts of Interest

## References

- Lin, C.C.; Peng, H.; Grizzle, J.W.; Kang, J.M. Power management strategy for a parallel hybrid electric truck. IEEE Trans. Control Syst. Technol.
**2003**, 11, 839–849. [Google Scholar] [CrossRef] - Liu, J.; Peng, H.; Filipi, Z. Modeling and Analysis of the Toyota Hybrid System. In Proceedings of the International Conference on Advanced Intelligent Mechatronics, Monterey, CA, USA, 24–28 July 2005.
- Liu, J.; Peng, H. Modeling and control of a power-split hybrid vehicle. IEEE Trans. Control Syst. Technol.
**2008**, 16, 1242–1251. [Google Scholar] [CrossRef] - Gong, Q.; Li, Y.; Peng, Z.R. Trip-based optimal power management of plug-in hybrid electric vehicles. IEEE Trans. Veh. Technol.
**2008**, 57, 3393–3401. [Google Scholar] [CrossRef] - Zhang, C.; Vahidi, A. Route preview in energy management of plug-in hybrid vehicles. IEEE Trans. Control Syst. Technol.
**2012**, 20, 546–553. [Google Scholar] [CrossRef] - Johannesson, L.; Asbogard, M.; Egardt, B. Assessing the potential of predictive control for hybrid vehicle powertrains using stochastic dynamic programming. IEEE Trans. Intell. Transport. Syst.
**2007**, 8, 71–83. [Google Scholar] [CrossRef] - Moura, S.J.; Fathy, H.K.; Callaway, D.S.; Stein, J.L. A stochastic optimal control approach for power management in plug-in hybrid electric vehicles. IEEE Trans. Control Syst. Technol.
**2011**, 19, 545–555. [Google Scholar] [CrossRef] - Opila, D.F.; Wang, X.; McGee, R.; Gillespie, R.B.; Cook, J.A.; Grizzle, J.W. An energy management controller to optimally trade off fuel economy and drivability for hybrid vehicles. IEEE Trans. Control Syst. Technol.
**2012**, 20, 1490–1505. [Google Scholar] [CrossRef] - Sciarretta, A.; Back, M.; Guzzella, L. Optimal control of parallel hybrid electric vehicles. IEEE Trans. Control Syst. Technol.
**2004**, 12, 352–363. [Google Scholar] [CrossRef] - Pisu, P.; Rizzoni, G. A comparative study of supervisory control strategies for hybrid electric vehicles. IEEE Trans. Control Syst. Technol.
**2007**, 15, 506–518. [Google Scholar] [CrossRef] - Wang, X.; He, H.; Sun, F.; Sun, X.; Tang, H. Comparative study on different energy management strategies for plug-in hybrid electric vehicles. Energies
**2013**, 6, 5656–5675. [Google Scholar] [CrossRef] - Kim, N.; Cha, S.; Peng, H. Optimal control of hybrid electric vehicles based on Pontryagin’s minimum principle. IEEE Trans. Control Syst. Technol.
**2011**, 19, 1279–1287. [Google Scholar] [CrossRef] - Kim, N.; Cha, S.; Peng, H. Optimal equivalent fuel consumption for hybrid electric vehicles. IEEE Trans. Control Syst. Technol.
**2012**, 20, 817–825. [Google Scholar] [CrossRef] - Zou, Y.; Liu, T.; Sun, F.; Peng, H. Comparative study of dynamic programming and Pontryagin’S minimum principle on energy management for a parallel hybrid electric vehicle. Energies
**2013**, 6, 2305–2318. [Google Scholar] [CrossRef] - Borhan, H.; Vahidi, A.; Phillips, A.M.; Kuang, M.L.; Kolmanovsky, I.V.; Cairano, S. MPC-based energy management of a power-split hybrid electric vehicle. IEEE Trans. Control Syst. Technol.
**2012**, 20, 593–603. [Google Scholar] [CrossRef] - Ericsson, E. Independent driving pattern factors and their influence on fuel-use and exhaust emission factors. Transp. Res. Part D
**2001**, 6, 325–341. [Google Scholar] [CrossRef] - Langari, R.; Won, J.S. Intelligent energy management agent for a parallel hybrid vehicle—Part I: system architecture and design of the driving situation identification process. IEEE Trans. Veh. Technol.
**2005**, 54, 925–934. [Google Scholar] [CrossRef] - Won, J.S.; Langari, R. Intelligent energy management agent for a parallel hybrid vehicle—Part II: torque distribution, charge sustenance strategies, and performance results. IEEE Trans. Veh. Technol.
**2005**, 54, 935–953. [Google Scholar] [CrossRef] - Yasui, Y. JSAE-SICE Benchmark Problem 2: Fuel Consumption Optimization of Commuter Vehicle Using Hybrid Powertrain. In Proceedings of the 10th World Congress on Intelligent Control and Automation, Beijing, China, 6–8 July 2012.
- Bertsekas, D. Dynamic Programming and Optimal Control; Athena Scientific: Belmont, MA, USA, 1995; Volume II. [Google Scholar]

© 2014 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

## Share and Cite

**MDPI and ACS Style**

Jiao, X.; Shen, T.
SDP Policy Iteration-Based Energy Management Strategy Using Traffic Information for Commuter Hybrid Electric Vehicles. *Energies* **2014**, *7*, 4648-4675.
https://doi.org/10.3390/en7074648

**AMA Style**

Jiao X, Shen T.
SDP Policy Iteration-Based Energy Management Strategy Using Traffic Information for Commuter Hybrid Electric Vehicles. *Energies*. 2014; 7(7):4648-4675.
https://doi.org/10.3390/en7074648

**Chicago/Turabian Style**

Jiao, Xiaohong, and Tielong Shen.
2014. "SDP Policy Iteration-Based Energy Management Strategy Using Traffic Information for Commuter Hybrid Electric Vehicles" *Energies* 7, no. 7: 4648-4675.
https://doi.org/10.3390/en7074648