# Energy Management of P2 Hybrid Electric Vehicle Based on Event-Triggered Nonlinear Model Predictive Control and Deep Q Network

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

## 3. System Dynamics and Prediction Model

## 4. MPC Event-Triggered and Weight Adaptation Mechanism

#### 4.1. Training of Event-Triggered Mechanism

#### 4.2. Training for Adaptation of Weights of MPC’s Cost Function

## 5. Results and Discussion

## 6. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Ehsani, M.; Gao, Y.; Emadi, A. (Eds.) Modern Electric, Hybrid Electric, and Fuel Cell Vehicles: Fundamentals, Theory, and Design, 2nd ed.; CRC Press: Boca Raton, FL, USA, 2017; ISBN 9781315219400. [Google Scholar]
- Emadi, A. Advanced Electric Drive Vehicles; CRC Press: Boca Raton, FL, USA, 2014. [Google Scholar]
- Yang, Y.; Hu, X.; Pei, H.; Peng, Z. Comparison of power-split and parallel hybrid powertrain architectures with a single electric machine: Dynamic programming approach. Appl. Energy
**2016**, 168, 683–690. [Google Scholar] [CrossRef] - Zeng, Q.; Fang, Y. Structural synthesis and analysis of serial–parallel hybrid mechanisms with spatial multi-loop kinematic chains. Mech. Mach. Theory
**2012**, 49, 198–215. [Google Scholar] [CrossRef] - Yu, P.; Li, M.; Wang, Y.; Chen, Z. Fuel cell hybrid electric vehicles: A review of topologies and energy management strategies. World Electr. Veh. J.
**2022**, 13, 172. [Google Scholar] [CrossRef] - Pattipati, B.; Sankavaram, C.; Pattipati, K. System identification and estimation framework for pivotal automotive battery management system characteristics. IEEE Trans. Syst. Man, Cybern. Part Appl. Rev.
**2011**, 41, 869–884. [Google Scholar] [CrossRef] - Samadani, E.; Fraser, R.; Fowler, M. Evaluation of Air Conditioning Impact on the Electric Vehicle Range and Li-ion Battery Life; SAE Technical Paper No. 2014-01-1853; SAE International: Warrendale, PA, USA, 2014. [Google Scholar]
- Nitta, N.; Wu, F.; Lee, J.T.; Yushin, G. Li-ion battery materials: Present and future. Mater. Today
**2015**, 18, 252–264. [Google Scholar] [CrossRef] - Wikner, E.; Björklund, E.; Fridner, J.; Brandell, D.; Thiringer, T. How the utilised SOC window in commercial Li-ion pouch cells influence battery ageing. J. Power Sources Adv.
**2021**, 8, 100054. [Google Scholar] [CrossRef] - Xie, J.; Ma, J.; Bai, K. Enhanced coulomb counting method for state-of-charge estimation of lithium-ion batteries based on peukert’s law and coulombic efficiency. J. Power Electron.
**2018**, 18, 910–922. [Google Scholar] - Hong, J.; Wang, Z.; Chen, W.; Wang, L.; Lin, P.; Qu, C. Online accurate state of health estimation for battery systems on real-world electric vehicles with variable driving conditions considered. J. Clean. Prod.
**2021**, 294, 125814. [Google Scholar] [CrossRef] - Noura, N.; Boulon, L.; Jemeï, S. A review of battery state of health estimation methods: Hybrid electric vehicle challenges. World Electr. Veh. J.
**2020**, 11, 66. [Google Scholar] [CrossRef] - Maures, M.; Zhang, Y.; Martin, C.; Delétage, J.Y.; Vinassa, J.M.; Briat, O. Impact of temperature on calendar ageing of Lithium-ion battery using incremental capacity analysis. Microelectron. Reliab.
**2019**, 100, 113364. [Google Scholar] [CrossRef] - Singh, K.V.; Bansal, H.O.; Singh, D. Fuzzy logic and Elman neural network tuned energy management strategies for a power-split HEVs. Energy
**2021**, 225, 120152. [Google Scholar] [CrossRef] - Rahmeh, H.; Bonfitto, A.; Ruzimov, S. Fuzzy logic vs equivalent consumption minimization strategy for energy management in P2 hybrid electric vehicles. In Proceedings of the International Design Engineering Technical Conferences and Computers and Information in Engineering, Online, 17–19 August 2020. [Google Scholar] [CrossRef]
- Fu, X.; Wang, B.; Yang, J.; Liu, S.; Gao, H.; He, B.Q.; Zhao, H. A Rule-Based Energy Management Strategy for a Light-Duty Commercial P2 Hybrid Electric Vehicle Optimized by Dynamic Programming; SAE Technical Paper No. 2021-01-0722; SAE International: Warrendale, PA, USA, 2021. [Google Scholar] [CrossRef]
- Zhang, B.; Xu, F.; Shen, T. A Real-Time Energy Management Strategy for Parallel HEVs with MPC. In Proceedings of the IEEE Vehicle Power and Propulsion Conference (VPPC), Hanoi, Vietnam, 14–17 October 2019; pp. 1–5. [Google Scholar]
- Ghosh, S.; Biswas, D.; Mitra, D.; Sengupta, S.; Mukhopadhyay, S. Effect of soc uncertainty on mpc based energy management strategy for hevs. In Proceedings of the IEEE 17th India Council International Conference (INDICON), New Delhi, India, 10–13 December 2020; pp. 1–6. [Google Scholar]
- Ezemobi, E.; Yakhshilikova, G.; Ruzimov, S.; Castellanos, L.M.; Tonoli, A. Adaptive Model Predictive Control Including Battery Thermal Limitations for Fuel Consumption Reduction in P2 Hybrid Electric Vehicles. World Electr. Veh. J.
**2022**, 13, 33. [Google Scholar] [CrossRef] - Sotoudeh, S.M.; HomChaudhuri, B. A robust MPC-based hierarchical control strategy for energy management of hybrid electric vehicles in presence of uncertainty. In Proceedings of the American Control Conference (ACC), Denver, CO, USA, 1–3 July 2020; pp. 3065–3070. [Google Scholar]
- Zhang, Y.; Chu, L.; Ding, Y.; Xu, N.; Guo, C.; Fu, Z.; Xu, L.; Tang, X.; Liu, Y. A hierarchical energy management strategy based on model predictive control for plug-in hybrid electric vehicles. IEEE Access
**2019**, 7, 81612–81629. [Google Scholar] [CrossRef] - Chen, Z.; Hu, H.; Wu, Y.; Zhang, Y.; Li, G.; Liu, Y. Stochastic model predictive control for energy management of power-split plug-in hybrid electric vehicles based on reinforcement learning. Energy
**2020**, 211, 118931. [Google Scholar] [CrossRef] - Dimarogonas, D.V.; Frazzoli, E.; Johansson, K.H. Distributed event-triggered control for multi-agent systems. IEEE Trans. Autom. Control.
**2011**, 57, 1291–1297. [Google Scholar] [CrossRef] - Fan, Y.; Feng, G.; Wang, Y.; Song, C. Distributed event-triggered control of multi-agent systems with combinational measurements. Automatica
**2013**, 49, 671–675. [Google Scholar] [CrossRef] - Fan, Y.; Liu, L.; Feng, G.; Wang, Y. Self-triggered consensus for multi-agent systems with Zeno-free triggers. IEEE Trans. Autom. Control.
**2015**, 60, 2779–2784. [Google Scholar] [CrossRef] - Seyboth, G.S.; Dimarogonas, D.V.; Johansson, K.H. Event-based broadcasting for multi-agent average consensus. Automatica
**2013**, 49, 245–252. [Google Scholar] [CrossRef] - Sun, Z.; Xia, Y.; Dai, L. Campoy, Tracking of unicycle robots using event-based MPC with adaptive prediction horizon. Automatica
**2019**, 25, 739–749. [Google Scholar] - Yang, H.; Zhao, H.; Xia, Y.; Zhang, J. Event-triggered active MPC for nonlinear multiagent systems with packet losses. IEEE Trans. Cybern.
**2019**, 51, 3093–3102. [Google Scholar] [CrossRef] - Zhan, J.; Li, X. Self-triggered consensus of multi-agent systems via model predictive control. IFAC-PapersOnLine
**2016**, 49, 19–24. [Google Scholar] [CrossRef] - Chen, J. Event-Triggered Model Predictive Control for Autonomous Vehicle with Rear Steering; SAE Technical Paper No. 2022-01-0877; SAE International: Warrendale, PA, USA, 2022. [Google Scholar]
- Zhou, Z.; Rother, C.; Chen, J. Event-Triggered Model Predictive Control for Autonomous Vehicle Path Tracking: Validation Using CARLA Simulator. IEEE Trans. Intell. Veh.
**2023**, 1–9. [Google Scholar] [CrossRef] - Chen, J.; Yi, Z. Comparison of event-triggered model predictive control for autonomous vehicle path tracking. In Proceedings of the IEEE Conference on Control Technology and Applications (CCTA), San Diego, CA, USA, 9–11 August 2021; pp. 808–813. [Google Scholar]
- Han, X.; He, H.; Wu, J.; Peng, J.; Li, Y. Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle. Appl. Energy
**2019**, 254, 113708. [Google Scholar] [CrossRef] - Fang, Y.; Song, C.; Xia, B.; Song, Q. An energy management strategy for hybrid electric bus based on reinforcement learning. In Proceedings of the 27th Chinese Control and Decision Conference (2015 CCDC), Qingdao, China, 23–25 May 2015; pp. 4973–4977. [Google Scholar]
- Lin, Y.; McPhee, J.; Azad, N.L. Comparison of deep reinforcement learning and model predictive control for adaptive cruise control. IEEE Trans. Intell. Veh.
**2020**, 6, 221–231. [Google Scholar] [CrossRef] - Gardezi, M.S.M.; Hasan, A. Machine learning based adaptive prediction horizon in finite control set model predictive control. IEEE Access
**2018**, 6, 32392–32400. [Google Scholar] [CrossRef] - Beckenbach, L.; Osinenko, P.; Streif, S. Addressing infinite-horizon optimization in MPC via Q-learning. IFAC-PapersOnLine
**2018**, 51, 60–65. [Google Scholar] [CrossRef] - Mohammadi, A.; Asadi, H.; Mohamed, S.; Nelson, K.; Nahavandi, S. Multiobjective and interactive genetic algorithms for weight tuning of a model predictive control-based motion cueing algorithm. IEEE Trans. Cybern.
**2018**, 49, 3471–3481. [Google Scholar] [CrossRef] - Zhao, Y.; Wang, L.P.; Sougrati, M.T.; Feng, Z.; Leconte, Y.; Fisher, A.; Srinivasan, M.; Xu, Z. A review on design strategies for carbon based metal oxides and sulfides nanocomposites for high performance Li and Na ion battery anodes. Adv. Energy Mater.
**2017**, 7, 1601424. [Google Scholar] [CrossRef] - BU-205: Types of Lithium-Ion. Available online: https://batteryuniversity.com/article/bu-205-types-of-lithium-ion (accessed on 12 June 2021).
- ISO2631-5; Mechanical Vibration and Shock—Evaluation of Human Exposure to Whole-Body Vibration. Part 5: Method for Evaluation Of Vibration Containing Multiple Shocks. ISO: Geneva, Switzerland, 2004.
- Jossen, A. Fundamentals of battery dynamics. J. Power Sources
**2006**, 154, 530–538. [Google Scholar] [CrossRef] - Rahmoun, A.; Biechl, H. Modelling of Li-ion batteries using equivalent circuit diagrams. Prz. Elektrotechniczny
**2012**, 88, 152–156. [Google Scholar] - Wang, W.; Wei, X.; Choi, D.; Lu, X.; Yang, G.; Sun, C. Electrochemical cells for medium-and large-scale energy storage: Fundamentals. In Advances in Batteries for Medium and Large-Scale Energy Storage; Woodhead Publishing: Cambridge, UK, 2015; pp. 3–28. [Google Scholar]

**Figure 3.**Motor and engine characteristics: (

**a**) engine efficiency and physical limits; (

**b**) motor efficiency and physical limits.

**Figure 5.**Open-circuit voltage C rates: (

**a**) OCV charging currents with different C rates; (

**b**) OCV discharging currents with different C rates.

**Figure 6.**Charging and discharging current limits: (

**a**) charging current limits for SOC and pack temperature; (

**b**) charging current limits for SOC and pack temperature.

**Figure 8.**Cell temperature and number of cycle effect for capacity retention and internal resistance: (

**a**) 45${}^{\circ}$; (

**b**) 25${}^{\circ}$; (

**c**) −10${}^{\circ}$.

**Figure 13.**Gear shifting strategy: (

**a**) vehicle velocity; (

**b**) accelerator pedal position; (

**c**) transmission gear.

**Figure 15.**SOC and voltage results: (

**a**) SOC results with different initial SOC values; (

**b**) voltage results with different initial SOC values.

**Figure 17.**Torque results with different initial SOC values: (

**a**) motor torque results with different initial SOC values; (

**b**) engine torque results with different initial SOC values.

**Figure 23.**Engine operational points results: (

**a**) engine operational points without DQN; (

**b**) engine operational points with DQN.

**Figure 24.**Motor operational point results: (

**a**) motor operational points without DQN; (

**b**) motor operational points with DQN.

Specification | |
---|---|

Motor Type | IPMSM |

10 s Peak Power | 32 kW |

Maximum Torque | 170 Nm |

Maximum Speed | 3750 min${}^{-1}$ |

System Nominal Voltage | 240V |

Motor Max. Efficiency | 95% |

Connected Inverter Efficiency | 97% |

Inverter | Electric Motor | Powertrain |
---|---|---|

97% | 95% | 92.150% |

Component | 10 s Peak Discharging Power | 10 s Peak Charging Power |
---|---|---|

Electric Motor | 32 kW | 32 kW |

Battery | 34.726 kW | 29.488 kW |

Criteria | Value | Unit |
---|---|---|

Target Range | 6 | km |

Consumption | 25 | kWh/100 km |

Consumption | 250 | Wh/km |

Usable Target Energy | 1500 | Wh |

**Table 5.**Li-On battery chemistries [40].

Criteria | LCO | LMO | NMC | LFP | NCA | LTO |
---|---|---|---|---|---|---|

Nominal Voltage (V) | 3.6 | 3.7 | 3.7 | 3.2 | 3.6 | 2.4 |

Specific Energy (Wh/kg) | 150–200 | 100–150 | 150–220 | 90–120 | 200–260 | 50–80 |

Cycle Life | 500–1000 | 300–700 | 1000–2000 | 2000–2500 | 500 | 3000–7000 |

Thermal Runaway (°C) | 150 | 250 | 210 | 270 | 150 | Safest |

Criteria | 65s1p | 65s2p |
---|---|---|

Battery Nominal Voltage | 240.5 V | 240.5 V |

Usable Target Energy | 1.5 kWh | 1.5 kWh |

Installed Target Energy | 3 kWh | 3 kWh |

Used DOD | 50% | 50% |

Cell Usable Energy Capacity (Ah) | 6.237 Ah | 3.118 Ah |

Cell Installed Energy Capacity (Ah) | 12.474 Ah | 6.237 Ah |

Specification | Battery | System Requirement |
---|---|---|

Configuration | 65s2p | - |

Cell Nominal Voltage (V) | 3.71 | 3.7 |

Pack Nominal Voltage (V) | 241.15 | 240 |

Cell Usable Energy (Ah) | 6.73 | 6.237 |

Cell Installed Energy (Ah) | 13.46 | 12.474 |

SoC Window | 30–80% | 30–80% |

Used SOC | 50% | 50% |

Usable Energy (kWh) | 1.6229 | 1.5 |

Installed Energy (kWh) | 3.245 | 3 |

Variable | Description |
---|---|

F${}_{TotR}$ | Total resistance force |

F${}_{AirR}$ | Aerodynamic resistance force |

F${}_{GradR}$ | Grade resistance force |

F${}_{RollR}$ | Rolling resistance force |

F${}_{Req}$ | Force requested by driver |

F${}_{Net}$ | Net force |

${\tau}_{Eng}$ | Engine-based torque |

${\tau}_{Mot}$ | Motor-based torque |

$\rho $ | Air density |

c${}_{d}$ | Drag coefficient |

A${}_{f}$ | Vehicle frontal areal |

V | Vehicle velocity |

m | Vehicle mass |

g | Gravity |

$\theta $ | Road angle |

C${}_{r}$ | Rolling resistance coefficient |

N${}_{Gear}$ | Gear ratio |

N${}_{FDR}$ | Final drive ratio |

r | Wheel radius |

**Table 9.**ISO 2631-5 acceleration limits [41].

Comfort Levels | Acceleration Limits |
---|---|

Not uncomfortable | 0.315 m/s${}^{2}$ |

A little uncomfortable | 0.63 m/s${}^{2}$ |

Fairly uncomfortable | 1.0 m/s${}^{2}$ |

Uncomfortable | 1.6 m/s${}^{2}$ |

Very uncomfortable | 2.5 m/s${}^{2}$ |

Criteria | Structure with Two MPCs Connected in Series | Structure with Two Event-Triggered MPCs with DQNs Connected in Series |
---|---|---|

Triggering Number | 30,000 | 14,396 |

Execution Time (s) | 13.881 s | 7.815 s |

Engine Efficiency (%) | 19.26% | 22.12% |

Motor Efficiency (%) | 78.95% | 82.56% |

Average Velocity Error Per Sample (km/h) | 0.0342 km/h | 0.0518 km/h |

Criteria | Time (s) | Calls | Time/Calls | Self-Time (s) |
---|---|---|---|---|

Structure with Two MPCs Connected in Series | 15.321 s | 30,000 | 0.0005107 | 13.881 s |

Structure with Two Event-Triggered MPCs with DQNs Connected in Series | 8.982 s | 14,396 | 0.0005087 | 7.815 s |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Haspolat, C.; Yalcin, Y.
Energy Management of P2 Hybrid Electric Vehicle Based on Event-Triggered Nonlinear Model Predictive Control and Deep Q Network. *World Electr. Veh. J.* **2023**, *14*, 135.
https://doi.org/10.3390/wevj14060135

**AMA Style**

Haspolat C, Yalcin Y.
Energy Management of P2 Hybrid Electric Vehicle Based on Event-Triggered Nonlinear Model Predictive Control and Deep Q Network. *World Electric Vehicle Journal*. 2023; 14(6):135.
https://doi.org/10.3390/wevj14060135

**Chicago/Turabian Style**

Haspolat, Cuneyt, and Yaprak Yalcin.
2023. "Energy Management of P2 Hybrid Electric Vehicle Based on Event-Triggered Nonlinear Model Predictive Control and Deep Q Network" *World Electric Vehicle Journal* 14, no. 6: 135.
https://doi.org/10.3390/wevj14060135