## 1. Introduction

Hybrid electric vehicles (HEVs) have been considered as an inevitable choice of vehicle industry from the traditional type to the pure electric vehicle for energy shortage and air pollution. In all hybridization techniques, energy management strategy (EMS) that defines the contribution of the two power sources: an internal combustion engine (ICE) and an electric battery in fulfilling a given power demand play an important part in fuel consumption improvement of an HEV.

Extensive literature has dealt with the EMS design [

1,

2]. In the automotive industry, the rule-based control strategies, from thermostat strategy, power follower control strategy, and engine optimal working point strategy to fuzzy rule based techniques (e.g., conventional fuzzy strategy, adaptive fuzzy logic strategy and fuzzy Q-Learning strategy), were developed on the basis of experimental trials and engineering experiences for their effectiveness in real time control [

3,

4,

5,

6,

7,

8,

9,

10]. However, the nature of results limited to specific vehicle design and not coping with multiple objects limits the application of these rule-based control strategies in HEVs.

In academic circles, the optimal control strategy is more popular [

11]. In fact, this kind of methods usually finds the minimum fuel consumption by minimizing a cost function over a fixed driving situation. Some machine learning techniques, such as game theory, neural network, particle swarm optimization, genetic algorithm, and reinforcement learning, are used in the optimization [

10,

12,

13,

14,

15]. Meanwhile, both dynamic programming (DP) and Pontryagin’s Minimum Principle (PMP) are famous optimal control theories in EMS design [

16,

17]. Dynamic programming (DP), a basis of comparison for evaluating the quality of other control strategies, is used to analyze the control actions and extract implementable rules in vehicle controls (e.g., switching of the drive train operating modes and the power split control), and then to design and improve other algorithms, such as rule-based algorithm [

12,

16,

18,

19,

20]. Although the computation time improved significantly with the help of the high performance multi-core processor and some computational techniques [

21], the major disadvantage of the DP approach, compared with other methods, is the heavy computational cost and requiring all the driving cycle information. In recent years, equivalent consumption minimization strategy (ECMS), based on the assumptions that the consumed electric energy could be converted to the equivalent fuel consumption, ensures that the equivalent fuel consumption is minimized at each moment [

22,

23]. However, the equivalent factor could not be concluded fast over complex driving conditions and the method still fails to be directly implemented in real-world scenarios for acquiring knowledge of future routines. The model predictive control method, characterized with optimization in receding horizon, feedback nature, and forecasting model, has been applied to EMS for not requiring detail drive schedule information in advance [

24,

25,

26,

27,

28,

29,

30,

31].

Meanwhile, since Lin et al. firstly introduced the stochastic dynamic programming (SDP) in the infinite-horizon form to the EMS of a parallel HEV in 2004 [

32], present literature highlights SDP in all kinds of HEVs (e.g., ICE HEVs, fuel cell HEVs, Plug-in HEVs, and hybrid electric bus) with different drivetrains for most calculation can be carried out offline and in potential real application, especially with the development of high performance computing equipment [

33,

34]. SDP EMS researches based on more perfect vehicle model have been concluded with simulation, hardware, and on-road testing [

34,

35,

36]. In the optimization, cost functions mainly focus on fuel economy, including electricity consumption, emissions, drivability (e.g., gear changes, braking, and engine start-stop), component degradation (e.g., state of charge,

SOC, to an expected final or nominal value, fuel cell degradation for reduction of transient loading), and electrical powertrain stress (e.g., square of battery charge) [

33,

35,

37,

38]. The method also highly relies on the discount factor, however, which is still an open research problem [

39]. The SDP aims to use the DP method to solve a statistical model of future driving conditions (e.g., slope degree, speed limits, traffic flow information, and vehicle load) for a vehicle, which is usually presented by Markov chains. The Markov chains could be modeled as one-state and multi-step-state according to both standard drive schedule and historical real-world driving data (e.g., based on fixed routine with road-segment discrete mode) [

35,

36,

40]. As a special case, shortest path SDP (SP-SDP) has been proposed, which usually does not have a discount factor and considers each cycle ending with an absorbing terminal state (e.g., vehicle stop) [

35,

38,

41,

42].

The main focus in this study is to develop a near optimal EMS with real-time application potential for a pre-transmission single-shaft torque-coupling parallel HEV that will improve the fuel economy and reduce emissions without the prior knowledge of future traffic situations and without deteriorating the vehicle performances. For the parallel HEV discussed in this paper, when clutch is engaged, engine speed is equal to the motor speed; whereas, engine speed is zero when clutch is set free. We then define the EMS with torque-split between the engine and motor. Therefore, a strategy that is based on splitting the demanded propelling torque is investigated. Unlike other methods that treat the demanded torque from driver as a priori information, it is modeled as a one-state Markov process to represent the uncertainty of congestion degree, road type, and so on. The Markov chains are modeled on an American Urban Dynamometer Driving Schedule, a Japanese 1015 drive cycle, and a self-defined one. A stochastic dynamic programming approach is then applied to the EMS problem to alleviate the cycle-sensitivity of the optimal control law.

The remainder of this paper is organized as follows. The parallel HEV model and related EMS problem formulation is described in

Section 2. In

Section 3, we mainly focus on the stochastic modeling of the driver torque demand, the SDP control strategy, and its specific implementation steps. To test different performances of this EMS, a simulation platform based on ADVISOR2002 was built. Simulation results and comparison with a rule based strategy and a DP based strategy over three different driving cycles are presented in

Section 4. Finally, the conclusions are given in

Section 5.

## 4. Performance Evaluation

Simulation experiments were done with ADVISOR2002 (NREL’s Advanced Vehicles Simulator) on a Pentium IV computer with Intel Core 2 Duo 3.0 GHZ CPU and 2G memory to evaluate performance of the SDP approach with a time consumption of 180 senconds. ADVISOR is a set of models, data, and script text files for use with MATLAB and Simulink. The modeling parameters of the parallel HEV are shown in

Table 2. We defined a torque-split-ratio

TSR =

T_{e}/

T_{dem} to quantify the positive power flows in the powertrain and to analyze the EMS performance. Four operation modes were also defined when the required torque is in the positive manner: pure electric traction mode (

TSR = 0), hybrid traction mode (0 <

TSR < 1), pure engine traction mode (

TSR = 1) and battery charging mode (

TSR > 1).

#### 4.1. Simulation Results under the Urban Dynamometer Driving Schedule (UDDS)

We firstly did simulations under the standard driving cycle UDDS (Urban Dynamometer Driving Schedule) to test the SDP EMS performances.

The

TSR maps of SDP EMS at driving wheel angular speed of 8 rad/s, 20 rad/s, 40 rad/s, 60 rad/s, 80 rad/s, and 100 rad/s, are shown in

Figure 4a–f respectively. From

Figure 4, we can see that when

SOC > 0.7 and

T_{dem} < 180 N·m, a relative small torque required,

TSR = 0 and the vehicle is in a low speed. Thus, the drive train is in the electric-alone traction mode; when

SOC > 0.7 and

T_{dem} ≥ 180 N·m, 0 <

TSR < 1 and the vehicle is in the hybrid traction mode; when 0.5 ≤

SOC ≤ 0.7, the

SOC has reached its bottom line and the drive train is in engine-alone traction mode; when

SOC < 0.5, the constraint on

SOC is not met,

TSR > 1 and the engine will charge the battery no matter what vechile speed (from 20 rad/s to 100 rad/s). We can also see that from

Figure 4b–f), when the same

T_{dem}, no matter vehicle speed variation,

TSR will increase with the

SOC decreasing. In another word, given a specific

T_{dem}, power though the engine and fuel consumption will increase with the decreasing of

SOC to meet the battery constraints. When

T_{dem} < 0, EMS could be handled with a simple way: the motor will be in regenerative braking mode and recover as much regeneration energy as possible within constraints imposed by the motor and the battery. The mechanical brake device will supply whatever is left over.

The SDP EMS simulation results with time going on are shown in

Figure 5. We can see from

Figure 5 that SDP EMS tends to keep

SOC within the range of 50%–65%, which guarantees efficient battery operation and prevents battery depletion. We can also see the range of

SOC leaves enough capacity to handle an extended period of the battery discharge and enough capacity to absorb a long period of charging. That means the battery is maintained near a balance point to ensure charge-sustaining. The simulation results show the reliability and viability of SDP EMS.

Figure 6 shows the torque distribution trajectories from 160–300 s, which further explains the benefits of SDP EMS in improving fuel economy. It could be seen that the engine provides the cruising torque demand while the battery pack through motor helps meet the peak torque demand. The output torque profile of an engine, between 40 N∙m and 60 N∙m, has a large constant region but little peaking, which satisfies the quasi-static model of the engine.

Figure 7a,b respectively depict the torque-speed operating points of engine and motor using the rule-based EMS under UDDS.

Figure 8a,b report those using the SDP EMS.

Figure 7 and

Figure 8 show that the engine operating points of SDP distribute in the higher efficiency region than that of the rule-based method. Hence, fuel consumption of SDP will be lower than that of the rule-based method. The SDP approach helps improving fuel economy and alleviating the emissions. We could see from

Figure 8a that most of the engine operating points are in 35.6%–38%, and the engine torque is in the domain of 40 N∙m and 60 N∙m. When higher torque is needed, the engine of SDP will give a good performance. In the rule-based EMS, the motor operating points are mainly concentrated in low efficiency region; in the SDP strategy, the motor operating points are located more in high efficiency operation region, which shows that the SDP method enjoys higher motor performance than the rule-based EMS.

#### 4.2. Comparisons of Simulation Results under Different Driving Cycles

To test the robustness of the SDP controller, two driving cycles—1015 and a new one—were also engaged in simulations. The new driving cycle, defined by ourselves, consisted of some repetitions of three urban schedules of different natures (e.g., UDDS + 1015 + WVUCITY). The 1015 driving cycle used here is the one in ADVISOR2002, depicting the Japanese 1015 mode driving cycle, which represents an urban cycle with road of zero or near zero grades [

45]. Meanwhile, in the new driving cycle simulation, the transition probability matrix was built for each driving cycle individually.

Moreover, a rule-based approach, the Parallel Electric Assist Control Strategy (PEACS), and DP approach simulations were also conducted and compared with the SDP simulation to evaluate the SDP performance though four aspects: fuel economy, engine efficiency, motor efficiency, and generating efficiency. The PEACS is a heuristic strategy defined in the ADVISOR document with five different operating modes. When vehicle speed below the minimum speed is set in advance, the motor propels the vehicle alone. When the demand torque is bigger than the maximum output torque of the engine, the motor provides an auxiliary torque. In the regenerative braking mode, the braking torque drives the motor for battery charging. Given the rotate speed and demand torque, an engine with low efficiency will be off and the motor will drive the vehicle alone. With a low SOC, the engine will drive the motor for battery charging. The cost function expressions and parameter-selection in PEACS and the DP approach are the same with the SDP strategy discussed above.

Simulation results from UDDS, 1015 and the composite driving cycle are reported in

Table 3,

Table 4 and

Table 5. We could conclude that the SDP strategy achieves obviously better results in both fuel consumption and components efficiency (e.g., engine efficiency, motoring efficiency, and generating efficiency) compared with the rule-based control strategy, PEACS. Simulation results of SDP and the global optimum results using DP show little difference within a few percent.

## 5. Conclusions

EMS design of HEVs is a challenging problem due to its complex structure and uncertain driving conditions. A stochastic dynamic program (SDP) is adopted to solve the EMS problem of a pre-transmission single-shaft torque-coupling parallel HEV. The special configuration enjoys an effective motor assist operation. In SDP, the required torque from the driver is modeled as a one-state Markov process to represent the uncertainty of future driving situations. ADVISOR2002 simulation results under three different driving cycles: UDDS, 1015, and a self-defining one (UDDS + 1015 + WVUCITY), indicate that this special SDP EMS achieves little performance than DP method. The engine efficiency and motor efficiency are greatly improved compared with a traditional rule-based strategy, PEACS. Therefore, we can conclude that the SDP approach has the potential for an off-line real-time on board control application to use the host computer-lower machine structure. The host computer is responsible for the establishment of the transfer probability matrix and the problem solution. Moreover, the slave computer, or the embedded system, is responsible for data collection and regular updating of the energy management strategy.

The SDP in this paper is a near-optimal control strategy only considering the fuel economy, gear-shift, and SOC sustaining. Our future work will focus on the aspects that could be tradeoff with the fuel economy—e.g., PM emissions, engine noise characteristics, other battery safety indicators and so forth.