Multi-Tier 3D Trajectory Planning for Cellular-Connected UAVs in Complex Urban Environments
Abstract
:1. Introduction
- This paper introduces the MTTP method tailored for cellular-connected UAVs, which effectively addresses the optimization challenge of determining the optimal flight path in complex dynamic environments. This method achieves integration between air–ground communication service assurance and collision avoidance while considering the mixed constraints imposed by communication reliability and environmental complexity;
- This paper presents a flight risk model that takes into account the communication outage probability of the GBS-UAV link and the complexity of the flight environment. Leveraging this model, the complex 3D trajectory optimization problem is formulated as a risk distance minimization problem;
- This paper proposes a hierarchical progressive solution approach that combines a heuristic search algorithm (HSA) with deep reinforcement learning (DRL). The algorithm factors in communication conditions, statically known obstacles, and unexpected obstacles detected by the UAV’s sensors to devise an optimal flight strategy. Additionally, the proposed DRL algorithm enhances accuracy and stability by integrating the double deep Q-Network (DDQN) with the dueling network structure.
2. Related Work
2.1. Trajectory Planning Research
2.2. Trajectory Planning Algorithm
3. System Model and Problem Formulation
3.1. Scenario and UAV Mobility Model
3.2. Flight Risk Model
3.2.1. Communication Outage Probability
3.2.2. Environmental Complexity
3.3. Problem Formulation
- To minimize the cumulative flight risk from to ;
- To minimize the flight distance between and .
4. Description of the Proposed Method
Algorithm 1: Workflow of the MTTP method. |
4.1. HSA-Based Connectivity-Aware Global Planning
Algorithm 2: Improved heuristic search algorithm (IHSA). |
4.2. DRL-Based Collision-Free Local Planning
Algorithm 3: D3QN with the collaborative offline and online strategy. |
- State space: In the DRL-based local planning model, state space consists of coordinates of the UAV’s current position , the destination , the set of relative static obstacle distances , and the set of relative unexpected obstacle distances . Thus, the state space is written as .
- Action space: The action space comprises 26 directions, allowing the UAV to select any action to navigate to an adjacent grid point.
- Reward function: The reward is acquired upon executing action in state . The reward function of a long-term decision network is defined as follows:In addition, the short-term decision network reward function is set as follows:
5. Numerical Experiments
5.1. Experiment Settings
5.2. Results/Discussion
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Li, D.; Qiang, Y.; Mott, J.H. Hazard Analysis of Large Cargo Delivery UAVs Under the Chinese Air Traffic Control System. In Proceedings of the 2021 Systems and Information Engineering Design Symposium (SIEDS), Virtual, 29–30 April 2021; pp. 1–6. [Google Scholar] [CrossRef]
- Elloumi, M.; Dhaou, R.; Escrig, B.; Idoudi, H.; Saidane, L.A. Monitoring road traffic with a UAV-based system. In Proceedings of the 2018 IEEE Wireless Communications and Networking Conference (WCNC), Barcelona, Spain, 15–18 April 2018; pp. 1–6. [Google Scholar] [CrossRef]
- He, B.; Huang, B.; Lin, Y.; Wu, L. Intelligent unmanned aerial vehicle (UAV) system for aircraft surface inspection. In Proceedings of the 2020 7th International Forum on Electrical Engineering and Automation (IFEEA), Hefei, China, 25–27 September 2020; pp. 316–321. [Google Scholar] [CrossRef]
- Liu, W.; Wang, C.; Zang, Y.; Lai, S.H.; Weng, D.; Bian, X.; Lin, X.; Shen, X.; Li, J. Ground Camera Images and UAV 3D Model Registration for Outdoor Augmented Reality. In Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan, 23–27 March 2019; pp. 1050–1051. [Google Scholar] [CrossRef]
- Van der Bergh, B.; Chiumento, A.; Pollin, S. LTE in the sky: Trading off propagation benefits with interference costs for aerial nodes. IEEE Commun. Mag. 2016, 54, 44–50. [Google Scholar] [CrossRef]
- Al-Hourani, A.; Gomez, K. Modeling cellular-to-UAV path-loss for suburban environments. IEEE Wirel. Commun. Lett. 2017, 7, 82–85. [Google Scholar] [CrossRef]
- Zeng, Y.; Lyu, J.; Zhang, R. Cellular-connected UAV: Potential, challenges, and promising technologies. IEEE Wirel. Commun. 2018, 26, 120–127. [Google Scholar] [CrossRef]
- Lin, X.; Yajnanarayana, V.; Muruganathan, S.D.; Gao, S.; Asplund, H.; Maattanen, H.L.; Bergstrom, M.; Euler, S.; Wang, Y.P.E. The sky is not the limit: LTE for unmanned aerial vehicles. IEEE Commun. Mag. 2018, 56, 204–210. [Google Scholar] [CrossRef]
- Zeng, Y.; Zhang, R.; Lim, T.J. Wireless communications with unmanned aerial vehicles: Opportunities and challenges. IEEE Commun. Mag. 2016, 54, 36–42. [Google Scholar] [CrossRef]
- Zeng, Y.; Wu, Q.; Zhang, R. Accessing from the sky: A tutorial on UAV communications for 5G and beyond. Proc. IEEE 2019, 107, 2327–2375. [Google Scholar] [CrossRef]
- Zhang, S.; Zeng, Y.; Zhang, R. Cellular-enabled UAV communication: A connectivity-constrained trajectory optimization perspective. IEEE Trans. Commun. 2018, 67, 2580–2604. [Google Scholar] [CrossRef]
- Zhang, S.; Zhang, R. Radio map-based 3D path planning for cellular-connected UAV. IEEE Trans. Wirel. Commun. 2020, 20, 1975–1989. [Google Scholar] [CrossRef]
- Bulut, E.; Guevenc, I. Trajectory optimization for cellular-connected UAVs with disconnectivity constraint. In Proceedings of the 2018 IEEE International Conference on Communications Workshops (ICC Workshops), Kansas City, MO, USA, 20–24 May 2018; IEEE: New York, NY, USA, 2018; pp. 1–6. [Google Scholar]
- Yang, H.; Zhang, J.; Song, S.; Lataief, K.B. Connectivity-aware UAV path planning with aerial coverage maps. In Proceedings of the 2019 IEEE Wireless Communications and Networking Conference (WCNC), Marrakesh, Morocco, 15–18 April 2019; IEEE: New York, NY, USA, 2019; pp. 1–6. [Google Scholar]
- Khamidehi, B.; Sousa, E.S. Federated learning for cellular-connected UAVs: Radio mapping and path planning. In Proceedings of the GLOBECOM 2020-2020 IEEE Global Communications Conference, Taipei, Taiwan, 7–11 December 2020; IEEE: New York, NY, USA, 2020; pp. 1–6. [Google Scholar]
- Yin, C.; Xiao, Z.; Cao, X.; Xi, X.; Yang, P.; Wu, D. Offline and online search: UAV multiobjective path planning under dynamic urban environment. IEEE Internet Things J. 2017, 5, 546–558. [Google Scholar] [CrossRef]
- Wu, Y.; Low, K.H. An adaptive path replanning method for coordinated operations of drone in dynamic urban environments. IEEE Syst. J. 2020, 15, 4600–4611. [Google Scholar] [CrossRef]
- Zeng, Y.; Xu, X.; Jin, S.; Zhang, R. Simultaneous navigation and radio mapping for cellular-connected UAV with deep reinforcement learning. IEEE Trans. Wirel. Commun. 2021, 20, 4205–4220. [Google Scholar] [CrossRef]
- Khatib, O. Real-time obstacle avoidance for manipulators and mobile robots. Int. J. Robot. Res. 1986, 5, 90–98. [Google Scholar] [CrossRef]
- Ulrich, I.; Borenstein, J. VFH/sup*: Local obstacle avoidance with look-ahead verification. In Proceedings of the 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), San Francisco, CA, USA, 24–28 April 2000; IEEE: New York, NY, USA, 2000; Volume 3, pp. 2505–2511. [Google Scholar]
- Rao, N.S.; Stoltzfus, N.; Iyengar, S.S. A’retraction’method for learned navigation in unknown terrains for a circular robot. IEEE Trans. Robot. Autom. 1991, 7, 699–707. [Google Scholar] [CrossRef]
- Minguez, J.; Montano, L. Extending collision avoidance methods to consider the vehicle shape, kinematics, and dynamics of a mobile robot. IEEE Trans. Robot. 2009, 25, 367–381. [Google Scholar] [CrossRef]
- Fox, D.; Burgard, W.; Thrun, S. The dynamic window approach to collision avoidance. IEEE Robot. Autom. Mag. 1997, 4, 23–33. [Google Scholar] [CrossRef]
- Wu, P.P.Y.; Campbell, D.; Merz, T. Multi-objective four-dimensional vehicle motion planning in large dynamic environments. IEEE Trans. Syst. Man Cybern. Part (Cybern.) 2010, 41, 621–634. [Google Scholar] [CrossRef]
- Hernandez-Hernandez, L.; Tsourdos, A.; Shin, H.S.; Waldock, A. Multi-objective UAV routing. In Proceedings of the 2014 International Conference on Unmanned Aircraft Systems (ICUAS), Orlando, FL, USA, 27–30 May 2014; IEEE: New York, NY, USA, 2014; pp. 534–542. [Google Scholar]
- Guglieri, G.; Lombardi, A.; Ristorto, G. Operation oriented path planning strategies for rpas. Am. J. Sci. Technol. 2015, 2, 1–8. [Google Scholar]
- Liu, X.; Liu, Y.; Chen, Y. Machine learning empowered trajectory and passive beamforming design in UAV-RIS wireless networks. IEEE J. Sel. Areas Commun. 2020, 39, 2042–2055. [Google Scholar] [CrossRef]
- Wan, K.; Gao, X.; Hu, Z.; Wu, G. Robust motion control for UAV in dynamic uncertain environments using deep reinforcement learning. Remote. Sens. 2020, 12, 640. [Google Scholar] [CrossRef]
- Hu, Z.; Wan, K.; Gao, X.; Zhai, Y.; Wang, Q. Deep reinforcement learning approach with multiple experience pools for UAV’s autonomous motion planning in complex unknown environments. Sensors 2020, 20, 1890. [Google Scholar] [CrossRef]
- Liu, X.; Liu, Y.; Chen, Y. Reinforcement learning in multiple-UAV networks: Deployment and movement design. IEEE Trans. Veh. Technol. 2019, 68, 8036–8049. [Google Scholar] [CrossRef]
- Bayerlein, H.; Theile, M.; Caccamo, M.; Gesbert, D. UAV path planning for wireless data harvesting: A deep reinforcement learning approach. In Proceedings of the GLOBECOM 2020-2020 IEEE Global Communications Conference, Taipei, Taiwan, 7–11 December 2020; IEEE: New York, NY, USA, 2020; pp. 1–6. [Google Scholar]
- Theile, M.; Bayerlein, H.; Nai, R.; Gesbert, D.; Caccamo, M. UAV path planning using global and local map information with deep reinforcement learning. In Proceedings of the 2021 20th International Conference on Advanced Robotics (ICAR), Ljubljana, Slovenia, 6–10 December 2021; IEEE: New York, NY, USA, 2021; pp. 539–546. [Google Scholar]
- Xu, Z.; Wang, Q.; Kong, F.; Yu, H.; Gao, S.; Pan, D. Ga-DQN: A Gravity-aware DQN Based UAV Path Planning Algorithm. In Proceedings of the 2022 IEEE International Conference on Unmanned Systems (ICUS), Guangzhou, China, 28–30 October 2022; IEEE: New York, NY, USA, 2022; pp. 1215–1220. [Google Scholar]
- Khamidehi, B.; Sousa, E.S. A double Q-learning approach for navigation of aerial vehicles with connectivity constraint. In Proceedings of the ICC 2020-2020 IEEE International Conference on Communications (ICC), Dublin, Ireland, 7–11 June 2020; IEEE: New York, NY, USA, 2020; pp. 1–6. [Google Scholar]
- Series, P. Propagation Data and Prediction Methods Required for the Design of Terrestrial Broadband Radio Access Systems Operating in a Frequency Range from 3 to 60 GHz. Recommendation ITU-R. 2013, pp. 1410–1415. Available online: https://scholar.google.com/scholar?as_q=Propagation+data+and+prediction+methods+required+for+the+design+of+terrestrial+broadband+radio+access+systems+operating+in+a+frequency+range+from+3+to+60+GHz&as_occt=title&hl=en&as_sdt=0%2C31 (accessed on 20 August 2023).
- Xiang, H.; Tian, L. Development of a low-cost agricultural remote sensing system based on an autonomous unmanned aerial vehicle (UAV). Biosyst. Eng. 2011, 108, 174–190. [Google Scholar] [CrossRef]
- 3GPP. TR 36.777, Technical Specification Group Radio Access Network: Study on Enhanced LTE Support for Aerial Vehicles. Technical Report V15.0.0. 2017. Available online: https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3231 (accessed on 20 August 2023).
- Xie, H.; Yang, D.; Xiao, L.; Lyu, J. Connectivity-aware 3D UAV path design with deep reinforcement learning. IEEE Trans. Veh. Technol. 2021, 70, 13022–13034. [Google Scholar] [CrossRef]
- Delamer, J.A.; Watanabe, Y.; Chanel, C.P. Safe path planning for UAV urban operation under GNSS signal occlusion risk. Robot. Auton. Syst. 2021, 142, 103800. [Google Scholar] [CrossRef]
- Yao, W.; Chen, Y.; Fu, J.; Qu, D.; Wu, C.; Liu, J.; Sun, G.; Xin, L. Evolutionary utility prediction matrix-based mission planning for unmanned aerial vehicles in complex urban environments. IEEE Trans. Intell. Veh. 2022, 8, 1068–1080. [Google Scholar] [CrossRef]
- Wu, Y.; Low, K.H.; Pang, B.; Tan, Q. Swarm-based 4D path planning for drone operations in urban environments. IEEE Trans. Veh. Technol. 2021, 70, 7464–7479. [Google Scholar] [CrossRef]
- Costa Fonte, C.; Fritz, S.; Olteanu-Raimond, A.M.; Antoniou, V.; Foody, G.; Mooney, P.; See, L. A review of OpenStreetMap data. In Mapping and the Citizen Sensor; Ubiquity Press Ltd.: London, UK, 2017; pp. 37–59. [Google Scholar]
- Bertram, J.; Zambreno, J.; Wei, P. Efficient Unmanned Aerial Systems Navigation With Collision Avoidance in Dense Urban Environments. IEEE Trans. Intell. Transp. Syst. 2023, 24, 8163–8173. [Google Scholar] [CrossRef]
- Ladosz, P.; Oh, H.; Zheng, G.; Chen, W.H. A hybrid approach of learning and model-based channel prediction for communication relay UAVs in dynamic urban environments. IEEE Robot. Autom. Lett. 2019, 4, 2370–2377. [Google Scholar] [CrossRef]
Average Flight Risk | MTTP Method | Communication-Based Approach | Improvement |
---|---|---|---|
30% | 859.74 | 1050.18 | 18.13% |
60% | 1018.44 | 1161.96 | 12.35% |
90% | 1101.24 | 1203.36 | 8.49% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Luo, X.; Zhang, T.; Xu, W.; Fang, C.; Lu, T.; Zhou, J. Multi-Tier 3D Trajectory Planning for Cellular-Connected UAVs in Complex Urban Environments. Symmetry 2023, 15, 1628. https://doi.org/10.3390/sym15091628
Luo X, Zhang T, Xu W, Fang C, Lu T, Zhou J. Multi-Tier 3D Trajectory Planning for Cellular-Connected UAVs in Complex Urban Environments. Symmetry. 2023; 15(9):1628. https://doi.org/10.3390/sym15091628
Chicago/Turabian StyleLuo, Xiling, Tianyi Zhang, Wenxiang Xu, Chao Fang, Tongwei Lu, and Jialiu Zhou. 2023. "Multi-Tier 3D Trajectory Planning for Cellular-Connected UAVs in Complex Urban Environments" Symmetry 15, no. 9: 1628. https://doi.org/10.3390/sym15091628
APA StyleLuo, X., Zhang, T., Xu, W., Fang, C., Lu, T., & Zhou, J. (2023). Multi-Tier 3D Trajectory Planning for Cellular-Connected UAVs in Complex Urban Environments. Symmetry, 15(9), 1628. https://doi.org/10.3390/sym15091628