# On Analyzing Routing Selection for Aerial Autonomous Vehicles Connected to Mobile Network

^{1}

^{2}

^{3}

^{4}

^{5}

^{6}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

## 3. Motivation

#### 3.1. Description of the Scenario and Tests of Communication Based on Mobile Network

#### 3.2. Description of the Solution

## 4. Formal Description of the Solution

_{1}(UAV) and a

_{2}(control station), however, each one of them taking independent decision, so the cooperation is only at the end of the decision making process. The Utility of the final action (decision), U(d), is a function of the Estimated Utilities of both agents, as shown in (1):

_{1}in any of the r = r

_{1}, … r

_{R}potential routings, i.e.,:

_{1}and a

_{2}as a function of the parameters and constrains:

_{a}

_{1}= f1(parameter_f1_1, parameter_f1_2, … subject to constrain_f1_1, constrain_f1_2, …)

_{a}

_{2}= f2(parameter_f2_1, parameter_f2_2, … subject to constrain_f2_1, constrain_f2_2, …)

_{1}, g

_{2}, …, g

_{K}}, then we can empirically select the best function, understood as the function that provides best results of the final decision. This means that the best function g

_{i}∈ G will provide those results, which are closest to max U(d) as defined in (4). Let G be the hypothesis space, with g

_{1}, …, g

_{K}being each of the hypothesis of the hypothesis space. Then the final U(d) can be calculated as follows:

## 5. Test Results

_{1}and ML

_{2}), each algorithm representing one link of the network (i.e., UAV-antenna and control station-antenna). ML

_{1}and ML

_{2}use the same models for resource usage prediction, however, since both links are measured using different parameters, then ML

_{1}and ML

_{2}will tune the layers in a different way. In addition, another algorithm, ML

_{3}, is trained using measurements of both links, this is end-to-end measurements from UAV to control station. Moreover, we assume that the selection of the routing in link 1 and link 2 are subject to the following constrain: The connection (routing path) in link 1 and in link 2 must be served by the same operator in order to avoid inter-operator roaming (IPX network, see Figure 2). ML

_{3}has the end-to-end information from link 1 and 2, so the final routing selected by ML

_{3}is the optimal one. ML

_{3}corresponds to the best trained machine (named f in the previous section).

#### 5.1. Measurements of the Network Links

_{2}.

#### 5.2. ML Algorithms and Hypothesis Space of U(d)

_{1}and ML

_{2}, implement in the first layer two well-known resource usage prediction models: An epidemic model defined in [28,29] and a model based on Autoregressive Integrated Moving Average (ARIMA) that assumes that the throughput follows a Gaussian distribution. The ARIMA model that we implemented is presented in Section 3.3 of the paper [30].

Algorithm 1: Loss Function and Optimizer |

1: procedure LossFunc_Optimizer2: Δ ← |Estimated_throughput—Output_throughput| 3: case δ > Δexit Null4: Δ1 ← Epidemic_Model_throughput—Output_throughput 5: Δ2 ← ARIMA_Model_throughput—Output_throughput 6: weight ← Δ1/Δ2 7: exit weight |

_{1}may adjust different weight for each operator, since each operator may have a different policy or policies that make resource usage prediction behaves in a different way. So, ML

_{1}(as well as ML

_{2}) are trained with only the input data of one operator.

_{3}uses the same prediction models, however the input and output data in this case are the measurements including both the links, i.e., from UAV to control station. For this, we send the messages from control station to the UAV instead to the server in the network operator. Since, in the case of ML

_{3}training, the measurements are taken in UAV, they are the same as in ML

_{1}, i.e., data throughput and the round-trip delay of signaling messages, subject to operator in link 1 is the same than operator in link 2 (in order to avoid inter-operator roaming).

_{3}has been created only for selecting the optimal function U(d), so ML

_{3}will be run only during the training phase. Afterwards, during the normal flight of the drone, only ML

_{1}, ML

_{2}and U(d) will obtain the best routing for each new connection.

_{1}(ML

_{1},),f

_{2}(ML

_{2})] ≈ f(ML

_{3}). U(d) is a function that selects the routing in link 1 and link 2, so, in our case it is the selection of the operator that will serve the connection. The inputs of U(d) are the estimated values of throughput in link 1 (calculated by ML

_{1}) and in link 2 (calculated by ML

_{2}) for each one of the operators: Est_Thr

_{link, operator}. The output of U(d) is a value from 1 to 3 (one of the three operators). The hypothesis space G of potential function U(d) includes the following hypothesis:

_{1}and G

_{2}are the boundaries of the weighted function (8):

#### 5.3. Test Methodology

- (1)
- Two Machine Learning algorithms for both agent 1 (e.g., UAV) and agent 2 (e.g., control station), called ML
_{1}and ML_{2}, respectively. ML_{1}searches the best routing (network operator that will serve the connection) in link 1 between UAV and antenna, whereas ML_{2}finds the best routing (i.e., which network operator will serve the connection) in link 2 between control station and antenna; - (2)
- 3 potential routings (3 operators) in agent 1 (e.g., UAV to antenna) and in agent 2 (e.g., control station to antenna);
- (3)
- routing in agent 1 is selected based on round-trip delay (RTD) and data throughput measurements in link 1 (between UAV and antenna). Routing in agent 2 is selected based on RTD measurements, data throughput and packet losses performed in link 2 (between control station and antenna);
- (4)
- a third Machine Learning algorithm (ML
_{3}) calculates the best routing in link 1 and 2, so the best end-to-end routing. The inputs of ML_{3}are round-trip delay and data throughput measured in end-to-end connections (between UAV and control station). This corresponds to the function max U(d) defined in Section 4; and - (5)
- a Utility function is considered such that quasi-optimal end-to-end routing can be selected based on partial link 1 and link 2 routings. Then, with the results of ML
_{1}and ML_{2}and assuming an appropriate Utility function, the system is able to select quasi-optimal end-to-end routing. The optimal Utility function will be the one that better approximates quasi-optimal solution (from partial link 1 and link 2 routings together with Utility function) and end-to-end routing.

_{1}, (2) validation of ML

_{2}, (3) validation of ML

_{3}, and (4) selection and validation of U(d).

#### 5.4. Test Results

_{1}, ML

_{2}, and ML

_{3}, we may observe that the epidemic model overestimates the throughput whereas the Autoregressive Integrated Moving Average model underestimates the throughput. It is a suitable situation for introducing a weighted average mechanism, which corrects over- and underestimation.

_{1}, ML

_{2}and ML

_{3}. These results are based on 3-fold cross-validation methodology and the confidence intervals presented in the results are based only on three measurements.

_{1}and ML

_{2}are different, so one could expect that the results differ between the two algorithms, however the table shows that the results are very similar. We may conclude that the gain of having a higher number of parameters in the prediction models is negligible in comparison to the gain of having two prediction models properly “weighted”. On the other hand, ML

_{3}achieves also similar accuracy than ML

_{1}and ML

_{2}. ML

_{3}estimates the throughput in end-to-end path containing radio links (UAV-antenna and antenna-control station) and fixed links (antenna-core-antenna, i.e., fronthaul and backhaul). The fact that the results are similar in end-to-end path means that the variability of the network (which decreases the accuracy) is present, most of all, in the radio links.

_{1}, ML

_{2}with all the 150 measurement samples obtained in link 1 and link 2. In this test we assume that ML

_{1}, ML

_{2}, and ML

_{3}do not need to be validated, so all the data can be used for training the machines. Afterwards, ML

_{3}is trained with the 150 samples of end-to-end connection.

_{1}), in link 2 (by using the trained ML

_{2}) and in end-to-end connection (by using the trained ML

_{3}). Table 2 presents the estimated values of throughput for one example sample. Let us remark that the throughput is measured in uplink direction.

_{1}is equal to 11.3 Mbps, see formula (4), and the Operator selected by G

_{1}is Operator 2; whereas G

_{2}is equal to 14.1 Mbps, see formula (5), and the Operator selected by G

_{2}is also Operator 2. In this example the values of throughput for Operator 2 are really much better than for other Operators, so it is logic that all the functions aiming to maximize the throughput will select Operator 2.

_{1}is closer to the throughput estimated by ML

_{3}in the end-to-end connection in almost all the cases and the two Utility functions, G

_{1}and G

_{2}, select the same Operator chosen by ML

_{3}. The reason of this almost unanimity of results is that in all cases link 1 is much worse than link 2, and, because of this, the values of end-to-end connection throughput are similar to link 1 throughputs (bottleneck). In this situation, G

_{1}is securely the best hypothesis of the hypothesis space G.

_{1}and ML

_{2}will estimate (after training) the value of throughput for each Operator and G

_{1}will determine the estimated end-to-end throughput. Based on this estimations, G

_{1}will select the best end-to-end Operator.

## 6. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Nawaz, H.; Ali, H.M.; Laghari, A.A. UAV Communication Networks Issues: A Review. Arch. Comput. Methods Eng.
**2020**, 1–21. [Google Scholar] [CrossRef] - Wang, J.; Juarez, N.; Kohm, E.; Liu, Y.; Yuan, J.; Song, H. Integration of SDR and UAS for Malicious Wi-Fi Hotspots Detection. In Proceedings of the 2019 Integrated Communications, Navigation and Surveillance Conference, Herndon, VA, USA, 9–11 April 2019; pp. 1–8. [Google Scholar]
- Zhang, Q.; Jiang, M.; Feng, Z.; Li, W.; Zhang, W.; Pan, M. IoT enabled UAV: Network architecture and routing algorithm. IEEE Internet Things J.
**2019**, 6, 3727–3742. [Google Scholar] [CrossRef] - Leonov, A.V.; Litvinov, G.A. About Applying AODV and OLSR Routing Protocols to Relaying Network Scenario in Fanet with Mini UAVS. In Proceedings of the 2018 XIV International Scientific-Technical Conference on Actual Problems of Electronics Instrument Engineering, Novosibirsk, Russia, 2–6 October 2018; pp. 220–228. [Google Scholar]
- Leonov, A.V.; Litvinov, G.A. Applying AODV and OLSR Routing Protocols to Air-To-Air Scenario in Flying Ad Hoc Networks Formed by Mini-UAVS. In Proceedings of the 2018 Systems of Signals Generating and Processing in the Field of on Board Communications, Moscow, Russia, 14–15 March 2018; pp. 1–10. [Google Scholar]
- Tahir, A.; Böling, J.; Haghbayan, M.-H.; Toivonen, H.T.; Plosila, J. Swarms of Unmanned Aerial Vehicles—A Survey. J. Ind. Inf. Integr.
**2019**, 16, 100106. [Google Scholar] [CrossRef] - Liu, X.; Yin, D.; Zhou, Y.; Liu, Z.; Wang, Y. Dispatching and management methods for communication of UAV swarm. In Proceedings of the 2nd International Conference on High Performance Compilation, Computing and Communications, New York, NY, USA, 15–17 March 2018; pp. 61–67. [Google Scholar]
- Sánchez-García, J.; García-Campos, J.M.; Arzamendia, M.; Reina, D.G.; Toral, S.L.; Gregor, D. A survey on unmanned aerial and aquatic vehicle multi-hop networks: Wireless communications, evaluation tools and applications. Comput. Commun.
**2018**, 119, 43–65. [Google Scholar] [CrossRef] - Sánchez-García, J.; Reina, D.G.; Toral, S.L. A distributed PSO-based exploration algorithm for a UAV network assisting a disaster scenario. Future Gener. Comput. Syst.
**2019**, 90, 129–148. [Google Scholar] [CrossRef] - Merwaday, A.; Guvenc, I. UAV Assisted Heterogeneous Networks for Public Safety Communications. In Proceedings of the IEEE Wireless Communications and Networking Conference Workshops, New Orleans, LA, USA, 9–12 March 2015; pp. 329–334. [Google Scholar]
- Batalla, J.M. On analyzing video transmission over wireless WiFi and 5G C-band in harsh IIoT environments. IEEE Access
**2020**, 8, 118534–118541. [Google Scholar] [CrossRef] - Sun, Y.; Mi, Z.; Wang, H.; Jiang, Y.; Zhao, N. Research on UAV Cluster Routing Strategy Based on Distributed SDN. In Proceedings of the 2019 IEEE 19th International Conference on Communication Technology (ICCT), Xi’an, China, 16–19 October 2019; pp. 1269–1274. [Google Scholar]
- Reina, D.G.; Tawfik, H.; Toral, S.L. Multi-subpopulation evolutionary algorithms for coverage deployment of UAV-networks. Ad Hoc Netw.
**2018**, 68, 16–32. [Google Scholar] [CrossRef] - Marler, R.T.; Arora, J.S. Survey of multi-objective optimization methods for engineering. Struct. Multidiscip. Optim.
**2004**, 26, 369–395. [Google Scholar] [CrossRef] - Hoffert, J.; Gokhale, A.; Schmidt, D.C. Timely Autonomic Adaptation of Publish/Subscribe Middleware in Dynamic Environments. Int. J. Adapt. Resilient Auton. Syst.
**2011**, 2, 1–24. [Google Scholar] [CrossRef] [Green Version] - Wierzbicki, A.P.; Makowski, M.; Wessels, J. Model-Based Decision Support Methodology with Environmental Applications; Kluwer Academic Publishers: Dordrecht, The Netherlands, 2000. [Google Scholar]
- Wierzbicki, A. The Use of Reference Objectives in Multiobjective Optimization. In Lecture Notes in Economics and Mathematical Systems; Springer: Berlin/Heidelberg, Germany, 1980; Volume 177, pp. 468–486. [Google Scholar]
- Kreglewski, T.; Granat, J.; Wierzbicki, A. A Dynamic Interactive Decision Analysis and Support System for Multicriteria Analysis of Nonlinear Models; IIASA: Laxenburg, Austria, 1991. [Google Scholar]
- Batalla, J.M.; Beben, A.; Chen, Y. Optimized decision algorithm for Information Centric Networks. Telecommun. Syst.
**2016**, 61, 247–255. [Google Scholar] - Flores, J.A.; Batalla, J.M.; Togay, C. Redundant localization system for automatic vehicles. Mech. Syst. Signal Process.
**2020**, 136, 106433. [Google Scholar] [CrossRef] - Batalla, J.M.; Cuadra Sanchez, A.; Ruiz Alonso, J.; Kopertowski, Z.; Guardalben, L.; Garcia, N.; Ophir, S. On assuring survivability of network operator’s services in evolving network environment. IEEE Access J.
**2018**, 6, 35646–35656. [Google Scholar] [CrossRef] - Karras, K.; Pallis, E.; Mastorakis, G.; Nikoloudakis, Y.; Batalla, J.M.; Mavromoustakis, C.X.; Markakis, E.K. A Hardware Acceleration Platform for AI-Based Inference at the Edge. Circuits Syst. Signal Process.
**2019**, 39, 1059–1070. [Google Scholar] [CrossRef] - Batalla, J.M.; Mavromoustakis, C.; Mastorakis, G.; Naixue Xiong, N.; Wozniak, J. Adaptive positioning system based on multiple wireless interfaces for Industrial IoT in harsh manufacturing environments. IEEE J. Sel. Areas Commun.
**2020**, 38, 899–914. [Google Scholar] [CrossRef] - Beben, A.; Batalla, J.M.; Chai, W.; Sliwinski, J. Multi-criteria decision algorithms for efficient content delivery in content networks. Ann. Telecommun.
**2013**, 68, 153–165. [Google Scholar] [CrossRef] [Green Version] - Mastorakis, G.; Mavromoustakis, C.X.; Batalla, J.M.; Pallis, E. (Eds.) Convergence of Artificial Intelligence and the Internet of Things; Springer: Berlin/Heidelberg, Germany, 2020. [Google Scholar]
- Damlapinar, M. Analytics of Life: Making Sense of Artificial Intelligence, Machine Learning and Data Analytics; NLITX: New York, NY, USA, 2019. [Google Scholar]
- Artasanchez, A.; Joshi, P. Artificial Intelligence with Python: Your Complete Guide to Building Intelligent Apps Using Python 3.x and TensorFlow 2; Packt Publishing: Birmingham, UK, 2020. [Google Scholar]
- Kryftis, Y.; Mastorakis, G.; Mavromoustakis, C.X.; Batalla, J.M.; Rodrigues, J.; Dobre, C. Resource Usage Prediction Models for Optimal Multimedia Content Provision. IEEE Syst. J.
**2016**, 11, 2852–2863. [Google Scholar] [CrossRef] - Kryftis, Y.; Mavromoustakis, C.X.; Mastorakis, G.; Pallis, E.; Batalla, J.M.; Rodrigues, J.P.C.; Dobre, C.; Kormentzas, G. Resource Usage Prediction Algorithms for Optimal Selection of Multimedia Content Delivery Methods. In Proceedings of the IEEE International Conference on Communications, London, UK, 8–12 June 2015. [Google Scholar]
- Ullah, Q.Z.; Hassan, S.; Khan, G.L. Adaptive Resource Utilization Prediction System for Infrastructure as a Service Cloud. Comput. Intell. Neurosci. J.
**2017**, 2017. [Google Scholar] [CrossRef] [Green Version]

**Figure 2.**Connectivity scenario of two links: Unmanned aerial vehicles (UAV) to antenna and antenna to Control station. N network operators.

**Figure 3.**Machine Learning (ML)-based methodology for achieving final Utility functions used in two-steps routing selection.

**Figure 4.**Input and Output data of ML

_{1}. Input data are throughput and round-trip delay (RTD) (1 min). Output is throughput (20 min). The values of <input1,input2;output> for this concrete sample are: <11.2 Mbps,242 ms; 9.3 Mbps>.

Machine Learning Algorithm | Accuracy |
---|---|

ML_{1} (UAV to antenna) | 84% ± 2% |

ML_{2} (control station to antenna) | 82% ± 3% |

ML_{3} (UAV to control station) | 81% ± 3% |

Link | Estimated Throughput [Mbps] | ||
---|---|---|---|

Operator 1 | Operator 2 | Operator 3 | |

Link 1—ML_{1} | 8.3 | 11.3 | 10.6 |

Link 2—ML_{2} | 14.7 | 16.8 | 15.4 |

End-to-end—ML_{3} | 9.1 | 11.1 | 10.8 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Mongay Batalla, J.; Mavromoustakis, C.X.; Mastorakis, G.; Markakis, E.K.; Pallis, E.; Wichary, T.; Krawiec, P.; Lekston, P.
On Analyzing Routing Selection for Aerial Autonomous Vehicles Connected to Mobile Network. *Sensors* **2021**, *21*, 399.
https://doi.org/10.3390/s21020399

**AMA Style**

Mongay Batalla J, Mavromoustakis CX, Mastorakis G, Markakis EK, Pallis E, Wichary T, Krawiec P, Lekston P.
On Analyzing Routing Selection for Aerial Autonomous Vehicles Connected to Mobile Network. *Sensors*. 2021; 21(2):399.
https://doi.org/10.3390/s21020399

**Chicago/Turabian Style**

Mongay Batalla, Jordi, Constandinos X. Mavromoustakis, George Mastorakis, Evangelos K. Markakis, Evangelos Pallis, Tomasz Wichary, Piotr Krawiec, and Przemysław Lekston.
2021. "On Analyzing Routing Selection for Aerial Autonomous Vehicles Connected to Mobile Network" *Sensors* 21, no. 2: 399.
https://doi.org/10.3390/s21020399