Research on Autonomous Manoeuvre Decision Making in Within-Visual-Range Aerial Two-Player Zero-Sum Games Based on Deep Reinforcement Learning

Lu, Bo; Ru, Le; Hu, Shiguang; Wang, Wenfei; Xi, Hailong; Zhao, Xiaolin

doi:10.3390/math12142160

Open AccessArticle

Research on Autonomous Manoeuvre Decision Making in Within-Visual-Range Aerial Two-Player Zero-Sum Games Based on Deep Reinforcement Learning

by

Bo Lu

^1,2

,

Le Ru

^1,2,*,

Shiguang Hu

^1,2

,

Wenfei Wang

^1,2,

Hailong Xi

^1,2 and

Xiaolin Zhao

^1,2

¹

Equipment Management and UAV Engineering College, Air Force Engineering University, Xi’an 710051, China

²

National Key Lab of Unmanned Aerial Vehicle Technology, Air Force Engineering University, Xi’an 710051, China

^*

Author to whom correspondence should be addressed.

Mathematics 2024, 12(14), 2160; https://doi.org/10.3390/math12142160

Submission received: 17 June 2024 / Revised: 4 July 2024 / Accepted: 7 July 2024 / Published: 10 July 2024

(This article belongs to the Special Issue Navigating Complexity: Advanced Optimization Techniques for Machine Learning)

Download

Browse Figures

Versions Notes

Abstract

In recent years, with the accelerated development of technology towards automation and intelligence, autonomous decision-making capabilities in unmanned systems are poised to play a crucial role in contemporary aerial two-player zero-sum games (TZSGs). Deep reinforcement learning (DRL) methods enable agents to make autonomous manoeuvring decisions. This paper focuses on current mainstream DRL algorithms based on fundamental tactical manoeuvres, selecting a typical aerial TZSG scenario—within visual range (WVR) combat. We model the key elements influencing the game using a Markov decision process (MDP) and demonstrate the mathematical foundation for implementing DRL. Leveraging high-fidelity simulation software (Warsim v1.0), we design a prototypical close-range aerial combat scenario. Utilizing this environment, we train mainstream DRL algorithms and analyse the training outcomes. The effectiveness of these algorithms in enabling agents to manoeuvre in aerial TZSG autonomously is summarised, providing a foundational basis for further research.

Keywords: WVR; TZSG; deep reinforcement learning; Markov decision processes; decision making

Share and Cite

MDPI and ACS Style

Lu, B.; Ru, L.; Hu, S.; Wang, W.; Xi, H.; Zhao, X. Research on Autonomous Manoeuvre Decision Making in Within-Visual-Range Aerial Two-Player Zero-Sum Games Based on Deep Reinforcement Learning. Mathematics 2024, 12, 2160. https://doi.org/10.3390/math12142160

AMA Style

Lu B, Ru L, Hu S, Wang W, Xi H, Zhao X. Research on Autonomous Manoeuvre Decision Making in Within-Visual-Range Aerial Two-Player Zero-Sum Games Based on Deep Reinforcement Learning. Mathematics. 2024; 12(14):2160. https://doi.org/10.3390/math12142160

Chicago/Turabian Style

Lu, Bo, Le Ru, Shiguang Hu, Wenfei Wang, Hailong Xi, and Xiaolin Zhao. 2024. "Research on Autonomous Manoeuvre Decision Making in Within-Visual-Range Aerial Two-Player Zero-Sum Games Based on Deep Reinforcement Learning" Mathematics 12, no. 14: 2160. https://doi.org/10.3390/math12142160

APA Style

Lu, B., Ru, L., Hu, S., Wang, W., Xi, H., & Zhao, X. (2024). Research on Autonomous Manoeuvre Decision Making in Within-Visual-Range Aerial Two-Player Zero-Sum Games Based on Deep Reinforcement Learning. Mathematics, 12(14), 2160. https://doi.org/10.3390/math12142160

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

Research on Autonomous Manoeuvre Decision Making in Within-Visual-Range Aerial Two-Player Zero-Sum Games Based on Deep Reinforcement Learning

Abstract

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI