A Continuous Space Path Planning Method for Unmanned Aerial Vehicle Based on Particle Swarm Optimization-Enhanced Deep Q-Network

In the field of unmanned aerial vehicle (UAV) path planning, the conventional deep Q-network (DQN) algorithm encounters the issue of action space discretization, which results in the generation of unsmooth and inefficient planned paths. To address this issue, we introduce the particle swarm optimization (PSO) algorithm into DQN to convert the discrete action space into a continuous one. This method divides the agent’s surrounding space into discrete and continuous action spaces. The PSO algorithm performs a global search in the continuous space to obtain a continuous candidate solution, while DQN learns a policy in the discrete space to obtain a discrete candidate solution. Then, the two candidate solutions are combined using a weighted vector method to determine a direction that balances global search and policy learning. Additionally, we introduce a novel feature matrix as the state space for DQN, providing more accurate environmental and positional representations. Furthermore, we incorporate a mechanism into the base prioritized experience replay (PER) and N-step updates, which combines the current temporal difference error (TD-error) with historical priorities and includes a policy entropy penalty term, thereby enhancing DQN’s ability to learn long-term dependencies. The performance of the PSO-DQN model is further improved through an enhanced $\epsilon$-greedy policy and learning rate decay strategy. Simulation results and experiments using the Flightmare simulator demonstrate that the proposed method generates smoother and more efficient paths for drones, exhibiting strong robustness in complex environments.