Research on Deep Q-Network Hybridization with Extended Kalman Filter in Maneuvering Decision of Unmanned Combat Aerial Vehicles

Ruan, Juntao; Qin, Yi; Wang, Fei; Huang, Jianjun; Wang, Fujie; Guo, Fang; Hu, Yaohua

doi:10.3390/math12020261

Open AccessArticle

Research on Deep Q-Network Hybridization with Extended Kalman Filter in Maneuvering Decision of Unmanned Combat Aerial Vehicles

by

Juntao Ruan

^1,2,

Yi Qin

^1,*,

Fei Wang

³,

Jianjun Huang

²,

Fujie Wang

¹,

Fang Guo

¹ and

Yaohua Hu

¹

School of Electrical Engineering and Intelligentization, Dongguan University of Technology, Dongguan 523000, China

²

College of Electronics and Information Engineering, Shenzhen University, Shenzhen 518060, China

³

School of Electronic and Information Engineering, Harbin Institute of Technology, Shenzhen 518060, China

^*

Author to whom correspondence should be addressed.

Mathematics 2024, 12(2), 261; https://doi.org/10.3390/math12020261

Submission received: 30 November 2023 / Revised: 5 January 2024 / Accepted: 8 January 2024 / Published: 12 January 2024

(This article belongs to the Special Issue Deep Learning and Adaptive Control, 2nd Edition)

Download

Browse Figures

Versions Notes

Abstract

To adapt to the development trend of intelligent air combat, it is necessary to research the autonomous generation of maneuvering decisions for unmanned combat aerial vehicles (UCAV). This paper presents a maneuver decision-making method for UCAV based on a hybridization of deep Q-network (DQN) and extended Kalman filtering (EKF). Firstly, a three-dimensional air combat simulation environment is constructed, and a flight motion model of UCAV is designed to meet the requirements of the simulation environment. Secondly, we evaluate the current situation of UCAV based on their state variables in air combat, for further network learning and training to obtain the optimal maneuver strategy. Finally, based on the DQN, the system state equation is constructed using the uncertain parameter values of the current network, and the observation equation of the system is constructed using the parameters of the target network. The optimal parameter estimation value of the DQN is obtained by iteratively updating the solution through EKF. Simulation experiments have shown that this autonomous maneuver decision-making method hybridizing DQN with EKF is effective and reliable, as it can eliminate the opponent and preserve its side.

Keywords: maneuvering decision; DQN; EKF; intelligent air combat; UCAV

Share and Cite

MDPI and ACS Style

Ruan, J.; Qin, Y.; Wang, F.; Huang, J.; Wang, F.; Guo, F.; Hu, Y. Research on Deep Q-Network Hybridization with Extended Kalman Filter in Maneuvering Decision of Unmanned Combat Aerial Vehicles. Mathematics 2024, 12, 261. https://doi.org/10.3390/math12020261

AMA Style

Ruan J, Qin Y, Wang F, Huang J, Wang F, Guo F, Hu Y. Research on Deep Q-Network Hybridization with Extended Kalman Filter in Maneuvering Decision of Unmanned Combat Aerial Vehicles. Mathematics. 2024; 12(2):261. https://doi.org/10.3390/math12020261

Chicago/Turabian Style

Ruan, Juntao, Yi Qin, Fei Wang, Jianjun Huang, Fujie Wang, Fang Guo, and Yaohua Hu. 2024. "Research on Deep Q-Network Hybridization with Extended Kalman Filter in Maneuvering Decision of Unmanned Combat Aerial Vehicles" Mathematics 12, no. 2: 261. https://doi.org/10.3390/math12020261

APA Style

Ruan, J., Qin, Y., Wang, F., Huang, J., Wang, F., Guo, F., & Hu, Y. (2024). Research on Deep Q-Network Hybridization with Extended Kalman Filter in Maneuvering Decision of Unmanned Combat Aerial Vehicles. Mathematics, 12(2), 261. https://doi.org/10.3390/math12020261

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

Research on Deep Q-Network Hybridization with Extended Kalman Filter in Maneuvering Decision of Unmanned Combat Aerial Vehicles

Abstract

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI