Resource Allocation and Offloading Strategy for UAVAssisted LEO Satellite Edge Computing
Abstract
:1. Introduction
 A UAVassisted airspace integrated task offload architecture is proposed in emergency scenarios, which jointly considers resource allocation and offloading schemes under the lack of ground resources of computing;
 A multisatellite joint task offload scheme is proposed, which takes full advantage of satellite computing resources to complete the task with low delay and energy consumed;
 A Deep Reinforcement Learning (DRL) algorithm is proposed, and simulation experiments prove the functionality of the algorithm, reducing the weighted sum of the energy consumed and delay by an average of 64.5%.
2. Related Work
3. System Model and Problem Description
3.1. Network Scenario
3.2. Architecture
3.3. Channel Model
3.3.1. IoT Device–UAV Channel
3.3.2. UAVLEO Channel
3.3.3. Task Offloading and Computing
3.4. Problem Definition
4. Algorithm Design
4.1. Satellite Selection
Monte CarloBased Satellite Selection Algorithm
Algorithm 1 Satellite selection algorithm 

4.2. Task and Computing Resource Allocation Strategy
DDPGBased Task Offloading and Resource Allocation Algorithm
 State
 Action.
 Reward.
Algorithm 2 DDPGLSTMbased task and resource allocation algorithm 

5. Performance Evaluation
5.1. Simulation Environment and Parameters
5.2. Performance Comparison
6. Conclusions
Author Contributions
Funding
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
 Zhang, H.; Yang, Y.; Shang, B.; Zhang, P. Joint Resource Allocation and MultiPart Collaborative Task Offloading in MEC Systems. IEEE Trans. Veh. Technol. 2022, 71, 8877–8890. [Google Scholar] [CrossRef]
 Luo, Q.; Luan, T.H.; Shi, W.; Fan, P. Deep Reinforcement Learning Based Computation Offloading and Trajectory Planning for MultiUAV Cooperative Target Search. IEEE J. Sel. Areas Commun. 2023, 41, 504–520. [Google Scholar] [CrossRef]
 Shang, B.; Yi, Y.; Liu, L. Computing over SpaceAirGround Integrated Networks: Challenges and Opportunities. IEEE Netw. 2021, 35, 302–309. [Google Scholar] [CrossRef]
 Jaiswal, K.; Dahiya, A.; Saxena, S.; Singh, V.; Singh, A.; Kushwaha, A. A Novel Computation Offloading Under 6G LEO SatelliteUAVbased IoT. In Proceedings of the 3rd International Conference on Computation, Automation and Knowledge Management (ICCAKM), Dubai, United Arab Emirates, 15–17 November 2022; pp. 1–6. [Google Scholar] [CrossRef]
 Yu, K.; Cui, Q.; Zhang, Z.; Huang, X.; Zhang, X.; Tao, X. Efficient UAV/Satelliteassisted IoT Task Offloading: A Multiagent Reinforcement Learning Solution. In Proceedings of the 27th Asia Pacific Conference on Communications (APCC), Jeju Island, Republic of Korea, 19–21 October 2022; pp. 83–88. [Google Scholar]
 Liu, Y.; Jiang, L.; Qi, Q.; Xie, S. EnergyEfficient Space–Air–Ground Integrated Edge Computing for Internet of Remote Things: A Federated DRL Approach. IEEE Internet Things J. 2023, 10, 4845–4856. [Google Scholar] [CrossRef]
 Nguyen, M.D.; Le, L.B.; Girard, A. Computation Offloading, UAV Placement, and Resource Allocation in SAGIN. In Proceedings of the 2022 IEEE Globecom Workshops (GC Wkshps), Rio de Janeiro, Brazil, 4–8 December 2022; pp. 1413–1418. [Google Scholar] [CrossRef]
 Shang, B.; Shafin, R.; Liu, L. UAV SwarmEnabled Aerial Reconfigurable Intelligent Surface (SARIS). IEEE Wirel. Commun. 2021, 28, 156–163. [Google Scholar] [CrossRef]
 Shang, B.; Liu, L. MobileEdge Computing in the Sky: Energy Optimization for Air–Ground Integrated Networks. IEEE Internet Things J. 2020, 7, 7443–7456. [Google Scholar] [CrossRef]
 Akter, S.; Kim, D.Y.; Yoon, S. Task Offloading in MultiAccess Edge Computing Enabled UAVAided Emergency Response Operations. IEEE Access 2023, 11, 23167–23188. [Google Scholar] [CrossRef]
 Wang, B.; Feng, T.; Huang, D. A Joint Computation Offloading and Resource Allocation Strategy for LEO Satellite Edge Computing System. In Proceedings of the IEEE 20th International Conference on Communication Technology (ICCT), Nanning, China, 28–31 October 2020; pp. 649–655. [Google Scholar] [CrossRef]
 Wang, Z.; Yu, H.; Zhu, S.; Yang, B. Curriculum Reinforcement LearningBased Computation Offloading Approach in SpaceAirGround Integrated Network. In Proceedings of the 13th International Conference on Wireless Communications and Signal Processing (WCSP), Virtual, 20–21 October 2021; pp. 1–6. [Google Scholar] [CrossRef]
 Tan, H.; He, M.; Xia, T.; Zheng, X.; Lai, J. A Novel Multilevel Computation Offloading Scheme at LEO Constellation Broadband Network Edge. In Proceedings of the IEEE World Congress on Services (SERVICES), Beijing, China, 18–24 October 2020; pp. 281–286. [Google Scholar] [CrossRef]
 Wang, B.; Xie, J.; Huang, D.; Xie, X. A Computation Offloading Strategy for LEO Satellite Mobile Edge Computing System. In Proceedings of the 14th International Conference on Communication Software and Networks (ICCSN), Chongqing, China, 10–12 June 2022; pp. 75–80. [Google Scholar] [CrossRef]
 Song, Z.; Hao, Y.; Liu, Y.; Sun, X. EnergyEfficient Multiaccess Edge Computing for TerrestrialSatellite Internet of Things. IEEE Internet Things J. 2021, 8, 14202–14218. [Google Scholar] [CrossRef]
 Wei, K.; Tang, Q.; Guo, J.; Zeng, M.; Fei, Z.; Cui, Q. Resource Scheduling and Offloading Strategy Based on LEO Satellite Edge Computing. In Proceedings of the IEEE 94th Vehicular Technology Conference (VTC2021Fall), Virtual, 27 Spetember–28 October 2021; pp. 1–6. [Google Scholar] [CrossRef]
 Tang, Q.; Fei, Z.; Li, B.; Han, Z. Computation Offloading in LEO Satellite Networks with Hybrid Cloud and Edge Computing. IEEE Internet Things J. 2021, 8, 9164–9176. [Google Scholar] [CrossRef]
 Yang, X.; Hong, B. CostEfficient Task Offloading for Satellite Edge Computing Systems. In Proceedings of the International Symposium on Networks, Computers and Communications (ISNCC), Shenzhen, China, 19–22 July 2022; pp. 1–5. [Google Scholar] [CrossRef]
 Tang, Z.; Zhou, H.; Ma, T.; Yu, K.; Shen, X.S. Leveraging LEO Assisted CloudEdge Collaboration for Energy Efficient Computation Offloading. In Proceedings of the IEEE Global Communications Conference (GLOBECOM), Madrid, Spain, 7–11 September 2021; pp. 1–6. [Google Scholar] [CrossRef]
 Ding, Y.; Feng, Y.; Lu, W.; Zheng, S.; Zhao, N.; Meng, L.; Nallanathan, A.; Yang, X. Online Edge Learning Offloading and Resource Management for UAVAssisted MEC Secure Communications. IEEE J. Sel. Top. Signal Process. 2023, 17, 54–65. [Google Scholar] [CrossRef]
 Lu, W.; Ding, Y.; Feng, Y.; Huang, G.; Zhao, N.; Nallanathan, A.; Yang, X. DinkelbachGuided Deep Reinforcement Learning for Secure Communication in UAVAided MEC Networks. In Proceedings of the GLOBECOM 2022—2022 IEEE Global Communications Conference, Rio de Janeiro, Brazil, 4–8 December 2022; pp. 1740–1745. [Google Scholar] [CrossRef]
 Chen, P.; Luo, X.; Guo, D.; Sun, Y.; Xie, J.; Zhao, Y.; Zhou, R. Secure Task Offloading for MECaidedUAV system. IEEE Trans. Intell. Veh. 2022, 8, 3444–3457. [Google Scholar] [CrossRef]
 Qi, X.; Yuan, M.; Zhang, Q.; Yang, Z. Joint powertrajectoryscheduling optimization in a mobile UAVenabled network via alternating iteration. China Commun. 2022, 19, 136–152. [Google Scholar] [CrossRef]
 Mao, W.; Xiong, K.; Lu, Y.; Fan, P.; Ding, Z. Energy Consumption Minimization in Secure Multiantenna UAVassisted MEC Networks with Channel Uncertainty. IEEE TRansactions Wirel. Commun. 2023, 1. [Google Scholar] [CrossRef]
 Liao, Y.; Chen, X.; Xia, S.; Ai, Q.; Liu, Q. Energy Minimization for UAV SwarmEnabled Wireless Inland Ship MEC Network with Time Windows. IEEE Trans. Green Commun. Netw. 2022, 7, 594–608. [Google Scholar] [CrossRef]
 Wang, J.; Yang, D.; Wu, F.; Xu, Y.; Xiao, L.; Zhang, T. Energy Minimization for CellularConnected UAVMEC Patrol Inspection Systems. In Proceedings of the 2022 IEEE/CIC International Conference on Communications in China (ICCC), Foshan, China, 11–13 August 2022; pp. 389–394. [Google Scholar] [CrossRef]
 Zhang, H.; Yang, Y.; Huang, X.; Fang, C.; Zhang, P. UltraLow Latency MultiTask Offloading in Mobile Edge Computing. IEEE Access 2021, 9, 32569–32581. [Google Scholar] [CrossRef]
 Chai, F.; Zhang, Q.; Yao, H.; Xin, X.; Gao, R.; Guizani, M. Joint Multitask Offloading and Resource Allocation for Mobile Edge Computing Systems in Satellite IoT. IEEE Trans. Veh. Technol. 2023, 1–15. [Google Scholar] [CrossRef]
 Ebrahim, M.A.; Ebrahim, G.A.; Mohamed, H.K.; Abdellatif, S.O. A Deep Learning Approach for Task Offloading in MultiUAV Aided Mobile Edge Computing. IEEE Access 2022, 10, 101716–101731. [Google Scholar] [CrossRef]
 Mao, S.; He, S.; Wu, J. Joint UAV Position Optimization and Resource Scheduling in SpaceAirGround Integrated Networks With Mixed CloudEdge Computing. IEEE Syst. J. 2021, 15, 3992–4002. [Google Scholar] [CrossRef]
 El Hammouti, H.; Benjillali, M.; Shihada, B.; Alouini, M.S. LearnAsYouFly: A Distributed Algorithm for Joint 3D Placement and User Association in MultiUAVs Networks. IEEE Trans. Wirel. Commun. 2019, 18, 5831–5844. [Google Scholar] [CrossRef] [Green Version]
 Du, S.; Chen, X.; Jiao, L.; Lu, Y. Energy Efficient Task Offloading for UAVassisted Mobile Edge Computing. In Proceedings of the 2021 China Automation Congress (CAC), Beijing, China, 22–24 October 2021; pp. 6567–6571. [Google Scholar] [CrossRef]
 Chen, B.; Li, N.; Li, Y.; Tao, X.; Sun, G. Energy Efficient Hybrid Offloading in SpaceAirGround Integrated Networks. In Proceedings of the 2022 IEEE Wireless Communications and Networking Conference (WCNC), Austin, TX, USA, 10–13 April 2022; pp. 1319–1324. [Google Scholar] [CrossRef]
 Chen, T.; Liu, J.; Tang, Q.; Huang, T.; Liu, Y. Deep Reinforcement Learning Based Data Offloading in MultiLayer Ka/Q Band LEO SatelliteTerrestrial Networks. In Proceedings of the IEEE 21st International Conference on Communication Technology (ICCT), Tianjin, China, 13–16 October 2021; pp. 1417–1422. [Google Scholar] [CrossRef]
 Zhao, J.; Hu, X.; Du, X. Algorithm of task offloading and resource allocation based on reinforcement learning in edge computing. In Proceedings of the 2021 IEEE 5th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Xi’an, China, 15–17 October 2021; Volume 5, pp. 1266–1269. [Google Scholar] [CrossRef]
 Chen, X.; Ge, H.; Liu, L.; Li, S.; Han, J.; Gong, H. Computing Offloading Decision Based on DDPG Algorithm in Mobile Edge Computing. In Proceedings of the IEEE 6th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA), Chengdu, China, 24–26 April 2021; pp. 391–399. [Google Scholar] [CrossRef]
 Zhang, P.; Wang, C.; Kumar, N.; Liu, L. Spaceairground integrated multidomain network resource orchestration based on virtual network architecture: A DRL method. IEEE Trans. Intell. Transp. Syst. 2021, 23, 2798–2808. [Google Scholar] [CrossRef]
 Liu, J.; Zhao, X.; Qin, P.; Geng, S.; Meng, S. Joint Dynamic Task Offloading and Resource Scheduling for WPT Enabled SpaceAirGround Power Internet of Things. IEEE Trans. Netw. Sci. Eng. 2022, 9, 660–677. [Google Scholar] [CrossRef]
 Wu, H.; Yang, X.; Bu, Z. Deep Reinforcement Learning for Computation Offloading and Resource Allocation in SatelliteTerrestrial Integrated Networks. In Proceedings of the 2022 IEEE 95th Vehicular Technology Conference: (VTC2022Spring), Helsinki, Finland, 19–22 June 2022; pp. 1–5. [Google Scholar] [CrossRef]
 Nguyen, M.D.; Le, L.B.; Girard, A. Joint Computation Offloading, UAV Trajectory, User Scheduling, and Resource Allocation in SAGIN. In Proceedings of the GLOBECOM 2022—2022 IEEE Global Communications Conference, Rio de Janeiro, Brazil, 4–8 December 2022; pp. 5099–5104. [Google Scholar] [CrossRef]
 Gao, Z.; Liu, A.; Han, C.; Liang, X. Max Completion Time Optimization for Internet of Things in LEO SatelliteTerrestrial Integrated Networks. IEEE Internet Things J. 2021, 8, 9981–9994. [Google Scholar] [CrossRef]
 Cao, H.; Yu, G.; Chen, Z. Cooperative Task offloading and Dispatching Optimization for Largescale Users via UAVs and HAP. In Proceedings of the 2023 IEEE Wireless Communications and Networking Conference (WCNC), Glasgow, UK, 26–29 March 2023; pp. 1–6. [Google Scholar] [CrossRef]
 Xie, X.; Zhang, H.; Wang, J.; Chang, Q.; Wang, J.; Pal, N.R. Learning optimized structure of neural networks by hidden node pruning with L_{1} regularization. IEEE Trans. Cybern. 2019, 50, 1333–1346. [Google Scholar] [CrossRef] [PubMed]
 Zhang, H.; Wang, J.; Sun, Z.; Zurada, J.M.; Pal, N.R. Feature selection for neural networks using group lasso regularization. IEEE Trans. Knowl. Data Eng. 2019, 32, 659–673. [Google Scholar] [CrossRef]
 Wang, B.; Li, X.; Huang, D.; Xie, J. A Profit Maximization Strategy of MEC Resource Provider in the SatelliteTerrestrial Double Edge Computing System. In Proceedings of the 2021 IEEE 21st International Conference on Communication Technology (ICCT), Tianjin, China, 13–16 October 2021; pp. 906–912. [Google Scholar] [CrossRef]
 Fujimoto, S.; van Hoof, H.; Meger, D. Addressing Function Approximation Error in ActorCritic Methods. In Proceedings of the 35th International Conference on Machine Learning; Dy, J., Krause, A., Eds.; PMLR: Cambridge, MA, USA, 2018; Volume 80, pp. 1587–1596. [Google Scholar]
 Gong, H.; Ge, H.; Ma, S.; Sun, A.; Chen, X.; Liu, L. Task Offloading Strategy Based on TD3 Algorithm in CloudEdge Collaborative MEC. In Proceedings of the 2022 4th International Conference on Natural Language Processing (ICNLP), Abu Dhabi, UAE, 7–11 December 2022; pp. 452–459. [Google Scholar] [CrossRef]
 Hazarika, B.; Singh, K.; Biswas, S.; Li, C.P. DRLBased Resource Allocation for Computation Offloading in IoV Networks. IEEE Trans. Ind. Informatics 2022, 18, 8027–8038. [Google Scholar] [CrossRef]
 Chen, N.; Zhang, S.; Qian, Z.; Wu, J.; Lu, S. When Learning Joins Edge: RealTime Proportional Computation Offloading via Deep Reinforcement Learning. In Proceedings of the IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS), Tianjin, China, 4–6 December 2019; pp. 414–421. [Google Scholar] [CrossRef]
Parameter  Value 

Computing resources of LEO satellite, ${f}_{j}$  $[3,8]$ GHz 
CPU cycles for processing one bit, $\beta $  100 cycles/bit 
Effective switched capacitance, k  ${10}^{27}$ 
IoT maximum transmit power, ${p}_{i}$  23 dBm 
UAV maximum transmit power, ${p}_{m}$  40 dBm 
Coverage radius of UAV, ${d}_{max}$  1 km 
Height of LEO satellite, ${H}_{j}$  $[700,1000]$ km 
Number of bandwidth chunks, K  6 
Noise spectral density, ${N}_{0}$  $174$ dBm/Hz 
Bandwidth of UAVs, ${\omega}_{m}$  1 MHz 
Bandwidth of LEO, B  10 MHz 
UAV altitude, ${H}_{m}$  100 m 
Light speed, c  $3\times {10}^{8}$ m/s 
Task size, ${S}_{i}$  $[2.5,3.5]$ MB 
Parameter  Value 

Buffer capacity, M  1,000,000 
Batch size, B  256 
Learning rate, ${\delta}_{a},{\delta}_{c}$  0.0001, 0.001 
Soft update rate, $\tau $  0.005 
Discount factor, $\gamma $  0.99 
Exploration rate, $\u03f5$  0.01 
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. 
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhang, H.; Xi, S.; Jiang, H.; Shen, Q.; Shang, B.; Wang, J. Resource Allocation and Offloading Strategy for UAVAssisted LEO Satellite Edge Computing. Drones 2023, 7, 383. https://doi.org/10.3390/drones7060383
Zhang H, Xi S, Jiang H, Shen Q, Shang B, Wang J. Resource Allocation and Offloading Strategy for UAVAssisted LEO Satellite Edge Computing. Drones. 2023; 7(6):383. https://doi.org/10.3390/drones7060383
Chicago/Turabian StyleZhang, Hongxia, Shiyu Xi, Hongzhao Jiang, Qi Shen, Bodong Shang, and Jian Wang. 2023. "Resource Allocation and Offloading Strategy for UAVAssisted LEO Satellite Edge Computing" Drones 7, no. 6: 383. https://doi.org/10.3390/drones7060383