Real-time Energy Management of Parallel Hybrid Electric Vehicles Using Linear Quadratic Regulation

: Optimization-based methods are of interest for developing energy management strategies due to their high performance for hybrid electric vehicles. However, these methods are often complicated and may require strong computational efforts, which can prevent them from real-world applications. This paper proposes a novel real-time optimization-based torque distribution strategy for a parallel hybrid truck. The strategy aims to minimize the engine fuel consumption while ensuring battery charge-sustaining by using linear quadratic regulation in a closed-loop control scheme. Furthermore, by reformulating the problem, the obtained strategy does not require the information of the engine efficiency map like the previous works in literature. The obtained strategy is simple, straightforward, and therefore easy to be implemented in real-time platforms. The proposed method is evaluated via simulation by comparison to dynamic programming as a benchmark. Furthermore, the real-time ability of the proposed strategy is experimentally validated by using power hardware-in-the-loop Author Contributions: Conceptualization and methodology B.-H.N., J.P.F.T., and A.B.; software, B.-H.N.; validation, B.-H.N., J.P.F.T., R.G., and A.B.; formal analysis, B.-H.N.; investigation, B.-H.N. and R.G.; resources, J.P.F.T. and A.B.; data curation, B.-H.N.; writing—original draft preparation, B.-H.N.; writing—review and editing, B.-H.N., J.P.F.T., R.G., and A.B.; visualization, B.H.N.; supervision, J.P.F.T. and A.B.; project administration, J.P.F.T. and A.B.; funding acquisition, J.P.F.T. and A.B. All authors agreed of the manuscript.


Introduction
The world is facing critical issues of environmental pollution and scarcity of fossil fuel resources. Meanwhile, transportation systems play an important role in environmental care and fossil fuel consumption [1]. Hybrid electric vehicles (HEVs) are among the current promising solutions for these problems [2][3][4]. HEVs can be classified as series, parallel, and series-parallel configurations [5], in which parallel HEVs have their engines directly connected to the drivetrains. Both internal combustion engine (ICE) and electrical machines produce torque to propel the vehicle. This torque assistance ability makes parallel HEVs appropriate for heavy-duty vehicles such as trucks [6][7][8]. Since the mechanical power can be shared between the engine and the machine, the parallel configuration does not require bulky machines as with series HEVs. Moreover, the power coupling in the parallel configuration can be realized by simple mechanical devices, e.g., a belt. This sort of HEVs do not need complex and large components like its series-parallel counterpart.
However, parallel HEVs suffer from some disadvantages. Firstly, the electrical drive must ensure most of the transient phases; hence, high peak currents are imposed to the batteries. That can notably degrade the batteries [9,10]. To reduce the aging stress on batteries, supercapacitors (SCs) can simulations, or additional adaptation such as adaptive Pontryagin's minimum principle (A-PMP, also called -control) like in [15,26].
The effectiveness of the new EMS is examined by comparison to an off-line optimal benchmark achieved by using DP [27] in term of fuel consumption, as used by many relevant works [15,28,29]. Furthermore, the real-time ability of the proposed strategy is demonstrated via experiment by using reduced-scale power hardware-in-the-loop (HIL) simulation [30].
Section 2 presents the modeling of the studied system for developing EMS. The proposed strategy is developed in Section 3. The comparative evaluation via simulation is given in Section 4. Section 5 addresses the experimental validation of the novel EMS.

Modeling of the Studied System
The studied system is a parallel hybrid delivery truck ( Figure 1). The electrical drive is mechanically coupled with the ICE via a belt. The engine and the machine propel the truck via a gearbox. The studied system is modeled by the equations given in Table 1 and graphically organized by using EMR (Figure 2). The batteries are modeled by Equation (1) in Table 1. The open-circuit voltage bat OC and the equivalent series resistance (ESR) bat are the non-linear functions of the SoC. These functions are given by look-up tables. The electrical drive is modeled by Equation (2) in Table 1. Here, a static model is used by assuming that its torque ED perfectly follows its reference ED ref [31]. The efficiency is given by its average values in traction and regenerative modes.    The static model of the ICE is addressed by Equation (3) in Table 1 with the assumed perfect torque control. The fuel consumption rate fuel , in g/s, is given by a fuel consumption map of the engine torque and speed (look-up table). Here, it should be noted that the fuel consumption map is only used for modeling in order to practically examine the control and strategy. It is not required in the EMS development step using the novel method proposed in this paper, which will be presented in the next section. The belt is modeled by Equation (4) in Table 1. The engine speed Ω ICE and the drive speed Ω ED are fixed due to the constant value of the belt ratio belt . The belt ensures the distribution of the mechanical power between the engine and the machine.
The gearbox (with clutch) is modeled by Equation (5) in Table 1. The clutch engagement is associated with the gearshift. The gear ratio gear is shifted discontinuously. The transmission and the wheel are given by Equation (6) in Table 1 with a global ratio tran . Equation (7) in Table 1 expresses the mechanical brake model. Here, the assumption is no response delay of the braking force. The chassis and the braking coupling are modeled by Equations (8) and (9) in Table 1, respectively. The vehicle mass veh includes goods and driver. The environment, including the air and the road, is modeled by Equation (10) in Table 1 with three resistive force components, in which is the gravitational acceleration (m/s 2 ), the air drag coefficient, x the aerodynamic standard (m 2 ), and the slope angle (rad).

Local Control of the System
EMR follows the principle of inversion which allows "automatic" deduction of the control scheme from the model organization. The control is deduced by functionally inverting the model representation [22]. If the element contains no dynamical delay, i.e., there are only algebraic operators, it should be directly inverted. The direct inversion is realized by algebraic manipulations. If there is dynamical delay, i.e., the element contains differential equations reorganized by the integral causality, it should be indirectly inverted by using a closed-loop controller.
The control of the hybrid traction subsystem with equation given in Table 2 is deduced as follows. The vehicle dynamics (Equation (9) in Table 1) is indirectly inverted by a closed-loop velocity controller (Equation (11) in Table 2) (crossed parallelogram). Here, the IP control structure is employed to cancel the zero of the closed-loop transfer function. Then, the braking coupling relationship (Equation (8) in Table 1) is inverted by Equation (12) in Table 2 (overlapped parallelogram) as a braking force distribution. The ratio br is imposed from a braking strategy. Next, Equations (13) and (14) in Table 2 (parallelograms) are the direct inversion of Equations (6) and (5) in Table 1

Model Reduction for Energy Management Strategy
To develop EMS, the system model should be reduced. Firstly, the full dynamical model is complex, which is often a multi-variable high-order model. That can make the EMS development complicated and hard to be realized. Hierarchical schemes are therefore often of interest to organize the strategies of complex system, such as [32]. Second, the dynamics at the higher level (strategy) are slower than the ones at the lower level (control) [33,34]. Addressing all the fast dynamics can cause huge computation for the EMS, which could be difficult for real-time implementation. Finally, when the local control is properly developed, the controlled variables can be considered as perfect response to the references.
This work focuses on developing the torque distribution strategy. In this study, the gearshift and braking strategies are developed by using look-up tables deduced from DP for ensuring fair comparisons. Hence, the drivetrain subsystem model can be represented as an equivalent mechanical source that imposes the belt speed Ω belt to the system. A reduced EMR ( Figure 3) is obtained for the development of the torque distribution strategy.

Batteries
Torque distribution (4) Figure 3. Reduced EMR for torque distribution strategy development.
In EMR, the source element (oval) is the terminal of the considered model. By using a source element to represent the drivetrain subsystem, the considered model for EMS development is reduced to (1)-(4) (see Figure 3).
The SoC of the batteries is the state variable to be controlled at the strategy level. Thus, the batteries are depicted as an accumulation element (crossed rectangle), which represents the considered dynamics of the studied model.
The ICE torque reference ICE ref is imposed by the strategy block; thus, it is the control variable. The belt torque reference belt ref is imposed by the drivetrain subsystem as the disturbance to be compensated.
A mathematical model can then be deduced from the reduced EMR of the system. By neglecting the efficiencies, the reduced mathematical model of the system can be written as follows: where Ω ICE and bat are measurable disturbances; belt ref indirectly determined by the driver request; belt and bat the given parameter; bat the state variable; and ICE ref the control variable. This is a linear model which is convenient for applying optimal control.

Original Problem Statement
The energy management problem of the parallel hybrid truck can be stated as follows. Find an optimal control law ICE ref for the model (16) to minimize the cost function : Applying directly the optimal control theory to solve this problem leads to a trivial solution in which ICE ref = 0 [35]. This solution can be intuitively explained as: the best way to save fuel is not to use the ICE. However, it is not an expected solution because the batteries will be fully discharged very quickly.
To overcome this issue, two approaches are often used in literature. The first one is to add the final constrain of the batteries SoC: Then, optimal control techniques such as DP [36,37] or PMP [35,38] are applied. This approach, however, is suitable only for off-line strategies when the driving cycles are known in advance. In fact, there is no way to ensure this final state constraint in real-time since the final time is unknown. To develop real-time EMSs, -control [26,39] is often applied to adapt the PMP-based optimal solution to real-time operations. The strategies, therefore, become sub-optimal.
The second approach is to charge the cost of SoC variation by adding to the cost function a penalty such as [25]: where and are weighting factors. The problem is then solved by using MPC [25] or PMP [40] with -control scheme [26].
There is a common drawback of these approaches that the fuel consumption rate fuel ICE , Ω ICE must be considered to develop the strategies. For numerical methods such as DP, a look-up table of the fuel consumption rate is directly used. For analytical methods such as PMP, the fuel consumption map is approximated by polynomial functions of which derivatives can be analytically calculated, such as [41]. This is because the PMP method requires analytical expression of the partial derivative. In both cases, a fuel consumption map of ICE is required. This is a drawback in real-world applications since such data are not often available.
Strategy development using these approaches is also often complicated. Moreover, methods such as MPC require strong computational efforts that leads to difficulties to implement them in realtime.
As a consequence, the problem could be reformulated for simplification without requirement of any data map and low computational efforts.

Problem Reformulation
This study proposes an alternative approach, firstly by reformulating the problem. It is known that the fuel consumption rate is mainly proportional to the ICE power [26,42]. In other words, for a given speed, the higher the engine torque is, the higher the fuel consumption rate fuel is. The data of the engine (Figure 4) show an almost linear behavior of the fuel consumption rate as a function of the torque and the speed. Hence, it can be considered that minimizing the engine torque will minimize the fuel consumption. That means instead of using the original cost function (Equation (17)), it can be: Here, Equation (20) can be considered as an approximation of the cost (Equation (17)). This approximation offers the following advantages: (i) it does not require any data set of fuel consumption rate of the engine; and (ii) it allows simple analytical calculations that are useful for developing real-time strategy.
Additionally, as discussed above, a term of SoC variation should be used with penalty in order to ensure the charge sustaining of the energy storage as follows: where is an equivalent conversion factor to convert the SoC variation to the engine torque. Since the reduced model (Equation (16)) is linear while the cost function (Equation (21)) is in a quadratic form of the state and control variables, the LQR method can be employed [43]. That can deduce a simple analytical control law that is suitable for on-board real-time implementation.

Linear Quadratic Regulation (LQR)
The LQR control law is developed for the linear system in the following form: where generally denotes state variables vector; the control variables vector; the dynamical matrix; the control matrix; with the cost function in the form given by: where and are weighting matrices. In real-world real-time applications, the final time of the driving cycle is unknown. Thus, the formulation with an infinite time horizon is suitable.
By applying PMP to this linear system and quadratic cost function, the control law is obtained as follows: where is the solution of the algebraic Riccati equation:

Proposed LQR-based Strategy
Applying the reduced mathematical model (Equation (16)) and the reformulated cost function (Equation (21)) to their general forms (Equations (22) and (23)), respectively, the system coefficients are defined by: Replacing Equation (26) in Equation (25), it leads to: By replacing Equation (27) in the general form (Equation (24)) of the control law, the ICE torque reference can be deduced as follows: It is seen that, eventually, the LQR-based control law has a form of a proportional (P) controller of the batteries SoC (see Appendix).

Weighting Factor Determination
The square-root of the weighting factor can be determined by using the well-known poleplacement technique for the P controller. The reduced model (16) The closed-loop system is a first-order transfer function, in which its response time (to 95% of the step reference) is three times of the time constant, as given by: where res is the response time of the closed-loop system. Hence, the square-root of the weighting factor can be determined as: Eventually, by replacing Equation (32) in Equation (28), the LQR-based torque distribution strategy for the studied parallel hybrid truck is obtained as follows: where the response time res is the only parameter needed to be defined by the strategy developer. In fact, it is a trade-off between the fuel saving (longer res ) and charge sustaining of the batteries (shorter res ). In order to guarantee the charge-sustaining condition of the energy management problem for parallel HEVs (see Equation (18)), the battery SoC reference bat ref should be set as bat init . The proposed real-time LQR-based strategy is implemented as illustrated in Figure 5. The limitations of the electrical drive torque and the batteries SoC are treated by using the conventional switching method. When the system reaches its limitations, the ICE must provide all the demanded traction power. The strategy is simple and straightforward without requiring complex data such as an engine fuel consumption map. Thus, it is suitable for on-board real-time implementation for realworld applications.

Simulation Conditions
The studied vehicle is based on a parallel hybrid truck designed in [44] with main parameters given in Table 3. A traditional six-level gearbox is employed. A 205-kW diesel engine is coupled with a 58-kW permanent magnet synchronous machine (PMSM) drive via a belt with the 1:1 ratio. The electrical drive is supplied by a 300-V 62-Ah batteries pack. The PMSM electrical drive is modeled by using nominal efficiency in traction and regenerative modes which are given in [44]. It could be of interest if an efficiency map is available that may compute the power consumption of the drive with higher accuracy. However, the use of the electrical drive efficiency data only affects the comparison between different traction models rather than between the different EMSs which is the objective of this paper. Hence, the nominal efficiency parameters are sufficient for this study.
This study examines the system with two standard driving cycles: the Urban Dynamometer Driving Schedule (UDDS) and the New European Driving Cycle (NEDC). UDDS was developed for testing heavy-duty vehicles, which is the case of the studied hybrid truck. Meanwhile, NEDC is for passenger cars and light-duty trucks. However, in this study, NEDC was employed, due to its simple profile, to better analyze the results. DP was employed to ensure the global optimal solution as a benchmark.

Results and Discussions
The objective of the torque distribution strategies is to minimize the engine fuel consumption. Hence, with different driving cycles, fuel consumption per 100 km is the criteria to evaluate and compare the different EMSs. Figure 6 shows a comparative evaluation of the proposed LQR-based strategy with the DP-based optimal solution and the conventional ICE truck. The hybrid truck can save up to 7.8% the fuel consumption with DP in the case of driving with UDDS cycle. It should be noted that DP is the off-line benchmark and only gives the theoretical optimal result, whereas the proposed real-time strategy saves 6.3% without knowing the driving cycle in advance. The fuel saving is 7.9% (DP) and 5.6% (proposed EMS) with NEDC. To have better understanding of the system behavior with the proposed strategy, the results with NEDC are presented here in detail. This driving cycle is simple; thus, the behavior is easier to analyze and understand. The examined NEDC contains four repeated urban cycles and a highway part (Figure 7). The controller acts to force the vehicle velocity to respond well to its reference without any overshoot. The torque distribution results are given in Figure 8. During every urban cycle, the ICE torque and the electrical drive torque perform in the same patterns. That confirms the consistency of the proposed EMS. The strategy let the machine support the engine as much as possible during the accelerations, then the ICE produce power to recharge the batteries. The electrical drive torque ED is kept within the torque constraints due to the drive power limitation. The electrical drive works to support the ICE to reduce the engine torque production as much as possible. When the machine reaches its torque limitation, the engine has to compensate for the remaining part to follow the torque reference needed to drive the vehicle. It should be noted that the electrical drive torque limitations reduce when its speed is higher than the machine nominal speed. Furthermore, it is seen that the drive torque is balanced in traction and regenerative braking mode. It indirectly shows the charge sustaining of the battery, which is often a requirement of the non-plug-in HEVs.  The batteries charge sustaining can be observed via the SoC evolutions, in a comparison with the result from DP (Figure 9). DP strategy can anticipate that there is a huge amount of regenerative energy at the end of the driving. Thus, it allows the batteries SoC to continuously reduce after each urban cycle. Meanwhile the real-time EMS does not "know" the driving condition in advance. Hence, it "tries" to ensure the charge sustaining after every urban one. The proposed LQR-based EMS perform well in both fuel saving and batteries charge sustaining. Yet, the DP strategy can save more fuel by a priori knowing the cycle. It can be seen that the batteries SoC variation is kept within a narrow range of about 2%. This is because the batteries are often over-sized for non-plug-in HEVs in order to ensure the chargesustaining condition. It is noteworthy that if the batteries are more deeply discharged, there will be a need of more energy to recharge them to the initial SoC. Working around this SoC range is optimal for this vehicle, as proven by the off-line optimal solution deduced by DP. The agreement between DP and the LQR-based strategy in term of batteries SoC range confirms the close-to-optimal performance of the proposed method. The batteries can be downsized by using advanced design methods; however, the sizing problem is not considered here because this paper focuses on EMS development.

Experimental Setup
The experiments are carried out to validate the proposed real-time torque distribution strategy by using reduced-scale power HIL simulation [30]. Figure 10 illustrates the experimental setup configuration and Figure 11 shows the test bench in the laboratory.  Figure 2 Equations ( Figure 11. Experimental test bench. The electrical drive is realized by a three-phase wound rotor induction machine (IM) and a voltage-source inverter. The DC bus of the inverter is connected in parallel with the DC bus capacitor of the battery emulator. The IM is connected in delta-connection scheme to increase the speed range since the emulated batteries voltage bat HIL is much lower than the nominal DC bus voltage of the machine. The traction emulation is realized by using a DC machine, of which the armature is connected to a chopper to control the armature current DCM . That DC machine drive is supplied by a voltage source composed by a SCs pack, an inductor, a chopper, and a DC bus capacitor. The SCs pack charges and discharges to consume and to provide the emulated traction power, respectively. The traction DC bus voltage dc trac is controlled to be constant. The electrical drive and the traction emulator are mechanically connected via a single shaft with a speed sensor. The batteries emulator is realized in the same way as described in [15], in which a controllable voltage source is composed by using a bidirectional DC/DC converter and SCs are employed to emulate the batteries.
The model and control program are implemented in a dSPACE controller board. As there is no full-scale machine available in the laboratory, a reduced-scale HIL testing is achieved. The mechanical emulation of the traction subsystem is realized in a similar way to [45]. The only difference in this work is that a SC pack, connected via a DC/DC converter, is employed to exchange the emulated traction energy. Via the power adaptation blocks, the full-scale traction model imposes the drive speed and torque references to the system. The DC machine is controlled by a speed controller to follow the speed reference, whereas the IM is controlled by a torque controller to follow the torque reference. The emulated torque is estimated from the measured currents of the IM. This torque is then imposed to the full-scale traction model via a power adaptation block.

Results and Discussions
During the driving cycle, the gearbox is shifted by a DP-based look-up table, which forms the electrical drive speed, and therefore also the ICE speed, as shown in Figure 12. The rotational speed is scaled with a ratio of 1.8 due to the limitations of the electrical machines used for the experiments. The lower boundary is the ICE idle speed. The left axis indicates the experimental results, while the right axis shows the simulation ones. It can be seen that the experimental speed matches well with the simulation. These results confirm that the emulator emulates well the behaviors of the traction subsystem. Some small differences appear when the engine is shifted too fast. This is because the DPbased gearshift strategy is developed with the assumption that the gearbox can be shifted without any delay. The results of electrical drive torque with the proposed LQR-based torque distribution strategy are plotted in Figure 13. The torque is scaled 80 times by considering the power limitation of the experimental IM drive. The experimental and simulated torques are kept within the boundaries of the electrical drive torque limitations. Here, it should be noted that the electrical drive torque limitation varies with time. It is reduced when the machine speed is higher than its rated value. The software program of the experiments is implemented in the dSPACE DS1005 card with a sampling time of 0.2 ms. In fact, most of the computational resource of the card is devoted for the complex control schemes of the batteries and traction emulators and for the full-scale model of the traction subsystem. Very little computational effort is required to perform the LQR-based strategy, which composed of a simple Equation (33). That demonstrates the real-time capability of the proposed EMS which is the main objective of the experiments despite the limitation of the experimental set-up in terms of power.

Conclusions
In this paper, a novel real-time energy management strategy has been proposed and validated for a parallel hybrid truck. Firstly, the truck has been modeled and controlled considering its full dynamical behaviors. To overcome the complexity of the system, the model, represented by using EMR, has been reduced to obtain a suitable mathematical model. Then, optimal control theory has been applied to this reduced mathematical model to deduce the real-time EMS.
To avoid the requirement of using ICE fuel consumption data, which is hard to achieve in realworld applications, the cost function has been reformulated. Furthermore, the variation of the batteries SoC has been added to the function as a penalty. Eventually, the cost is in form of a quadratic function of the control and the state variables. Since the reduced model is linear, the LQR method has been applied to obtain the optimization-based control law of the ICE torque reference. The obtained strategy is a closed-loop control scheme of the batteries SoC, which is suitable to be implemented in real-time. The development procedure and the strategy implementation require only basic system parameters such as the batteries capacity and the belt ratio. The proposed EMS is, therefore, realistic for real-world applications.
The new strategy has been compared to the DP-based off-line optimal solution and the case of the conventional ICE-only truck. Simulation results have verified the performances of the LQR-based strategy. Moreover, reduced-scale power HIL simulation has been used to validate the proposed EMS in experiments. It has been shown that the LQR-based strategy works properly in real-time.