Evaluating the Quality of Experience Performance Metric for UAV-Based Networks

In this work, we consider a UAV-assisted cell in a single user scenario. We consider the Quality of Experience (QoE) performance metric calculating it as a function of the packet loss ratio. In order to acquire this metric, a radio-channel emulation system was developed and tested under different conditions. The system consists of two independent blocks, separately emulating connections between the User Equipment (UE) and unmanned aerial vehicle (UAV) and between the UAV and Base station (BS). In order to estimate scenario usage constraints, an analytical model was developed. The results show that, in the described scenario, cell coverage can be enhanced with minimal impact on QoE.


Introduction
Currently, 5th generation (5G) networks are standardized and deeply researched, which allows network operators to fully implement them in their architectures. However, many "challenges" that were posed to the 5G networks were not fully resolved; therefore, the discussion of the requirements and applications of the 6th generation (6G) networks is gaining momentum in the scientific community [1][2][3]. A number of new communication scenarios are predicted, which include holographic calls, e-medicine, flying networks, and Internet of skills [4]. This in turn imposes ambitious requirements for new networks. These are ultra low latency (less than 0.1 ms), and a higher level of security and privacy, and an increase in energy efficiency. It is also assumed that a level of reliability comparable to wired networks will be provided.
To achieve such a performance, especially in emergency situations or in cases when the coverage of the terrestrial BSs is limited due to terrain conditions, the development of non-terrestrial networks (NTNs) is being considered. Such networks allow operators to provide coverage for the areas where it is impossible or economically impractical to deploy terrestrial BSs. Additionally, in cases of an emergency situation, natural disaster or simple network overload (during a concert, festival or other event), such networks would deal with the situation in the shortest possible time.
Starting with release 15, the ability to use NTNs [5] has already been added to the 3GPP standard for 5G networks, but this generation of networks still remains largely 2D oriented. In 6G networks, a fully fledged three-dimensional heterogeneous network architecture is assumed to be used. In the frame of this architecture, in addition to terrestrial BSs, Unmanned Aerial Vehicles (UAVs), High Altitude Platforms (HAPs), and satellites are to be implemented [6].
The implementation of HAPs and satellites as network elements allows for a larger coverage area, especially in difficult terrain conditions. However, they present a number of significant disadvantages, such as high transmission delay, low bandwidth and high deployment complexity. At the same time, implementing UAVs makes it possible to provide communication services at a level comparable to terrestrial BSs. This fact emphasizes that UAVs have high implementation efficiency with significantly lower complexity and deployment costs. The most relevant and realistic scenario for the NTN implementation by 2030 is the implementation of UAVs as a tool to extend the coverage of the terrestrial BSs to the areas, which are hard to cover using traditional methods.
In this paper, we first present related works, given in Section 2, where we reference several papers regarding UAV implementation in telecommunications as well as some practical trials. In Section 3, a system model is presented and several trial scenarios are considered. In Section 4, a mathematical model derives some of the constraints for the scenario usages. Section 5 describes the proposed architecture of the testbed and its technical limitations. Section 6 presents the initial data for the trial, its results and their discussion. In the conclusion, we summarize the main results and define topics for further research.
The main contributions of our paper can be summarized as follows: • Proposed approach for a cell enhancement via UAV utilization as a repeater; • Proposed mathematical model to estimate the system constraints; • Developed and implement an emulator of a test network based on software defined radio (SDR) devices, which reproduces all interacting elements' channel characteristics for the scenarios described in the system model.
The experimental results show that our proposed model allows us to enhance the cell coverage while still maintaining the required QoE levels.

Related Works
The implementation of UAVs in wireless networks for purposes of building emergency communication networks with uninterrupted network coverage and reducing the load on network sections has long attracted much attention both in scientific circles and in industry [7,8]. Due to their high mobility and deployment flexibility, UAVs are widely used in 5G networks.
The authors of [9,10] show that UAVs can be cost-effectively used as aerial platforms to provide or improve communication services of ground devices. Aerial base stations (BSs) and repeaters can be considered as examples of these platforms in cases where the terrestrial cellular network does not provide sufficient coverage. The authors of [11] examined possible use cases for drones to improve network performance, highlighting two main usage types: mobile-enabled drones (MEDs) and wireless infrastructure drones (WIDs).
In [12], to achieve maximum coverage of a UAV BS, a mathematical model calculating the optimal flight height depending on the conditions of the propagation environment is proposed.
The authors of [13], using UAVs in combination with mobile edge computing (MEC) technology with multiple access, present an algorithm for unloading traffic to UAVs, with consideration given to power consumption and movement trajectory. Expanding this approach, the authors of [14] consider offloading vehicle computations on UAVs and through UAVs to ground MEC servers, using drones as radio relays.
Application of UAVs in sensor networks and as radio bridges is also of great interest. In [15], a platform for organizing an ad hoc network using UAVs to achieve the required QoS level and GPSR and AODV is considered. Additionally, the authors of [16,17] proposed using an ad hoc network for UAVs.
In [18], a spatial network configuration scheme was constructed for additional coverage by a UAV-deployed base station to increase the QoS indicator. By dividing the coverage area into three sectors in accordance with the signal level, the authors motivate users to move towards an area with a higher signal level by offering them certain rewards. This method is partially considered in our work; however, the approach is applied in reverse, so that the UAV follows the user.
For these user-centered systems, QoE plays an essential role. The work of [19] proposes a method for optimizing this indicator through prediction, using a neural network and caching the content requested by the user on the UAV in advance. This approach, however, is effective for a network section with similar user content preferences.This paper considers an approach to the analysis of QoE in a UAV-assisted cell, studying the impact of using UAVs with directional antennas to enhance cell coverage.
To assess the effectiveness of the stochastic model of the millimeter-wave channel between a UAV and ground-based antenna, the authors of [20] carried out an analysis on an experimental setup using proprietary equipment and SDR to provide a wireless channel. A similar approach was used in [21] to build a platform for an elastic cognitive network. This approach allows one to flexibly change the network configuration by using an SDR; however, it has a high implementation cost due to the usage of proprietary equipment and the need for a special location for testing. In [22], the authors presented the possibility of emulating a wireless network using SDR or commercial radio devices in conjunction with a PC. We described a similar approach in [23] to possibly emulate signal propagation channels, potentially making changes to the signal propagation environment. This method allows an experiment to be carried out autonomously and cost-effectively, as well as its validation, before carrying out a trial using real equipment. Table 1 presents a comparison analysis of the aforementioned works and our current paper. It can be noted that most works use either analytical or simulation approaches to derive the desired metrics; however, using an emulation approach is much closer to field trials than both of these approaches, since it emulates the channel behaviour unlike a simulation. The papers presenting experimental field trials are few in number and mostly focus on an emulation platform overview to showcase the platform's capabilities. The current work is a continuation of [23], where the basis for the used emulation platform was to build and upgrade said emulator to analyze the QoE impact in terms of throughput in a UAV-assisted cell enhancement scenario.

System Model
We consider a scenario where the BS is insufficient to serve the user in terms of the required QoS/QoE and it implements a UAV to improve the latency and bit rate. The connection between the base station and the UAV is realized by the new radio (NR) technology using a directional antenna. The UAV is also equipped with a directional wide-angle antenna to connect to the mobile UE via NR. There are no blockers in between the BS and UAV and between the UAV and UE; thus, the free space path loss (FSPL) signal propagation model is considered. The BS always has all the information about the UAV location and the UAV always has all the information about both the BS and UE locations.
When the user is served via a UAV and moves from its preferred position, located directly under the UAV, the signal it receives from the UAV decreases. When the signal falls below a certain given threshold, the UAV starts moving towards the location where the user was at the moment the signal reached the threshold. The BS is equipped with a rotating antenna that follows UAV movements. The BS antenna coverage has a threshold area near the edges. When a UAV enters this threshold, the antenna rotates so that it is directed towards the current UAV position. The threshold depends on the antenna angular rotation speed and the distance between the BS and UAV.
The coverage area can be divided into several areas, as depicted in Figure 1: the inner area R 0 , BS coverage R 1 , middle area R 2 and outer area R 3 .
In the inner area, it is not advisable to use a UAV to connect to the UE, since in this area, it is possible for the BS antenna to lag behind the UAV position, resulting in connection loss. Since the area is relatively close to the BS, the UE can be connected directly to the BS with relatively good link quality. The BS coverage area is the coverage area of the multisector BS antenna, which is used to serve customers. In this area, the user can be connected via a UAV as well as directly via a multisector BS antenna. The decision regarding the connection type can be considered as an optimization problem of several parameters, such as channel quality, number of connected UEs, resource allocation scheme and others, and is not considered in the current paper. The middle area corresponds to the coverage area of the directional BS antenna, which is used for the connection between the BS and UAV. In this area, the UAV is used as a repeater to connect to the UE. Thus, UAV implementation widens the BS coverage area. The outer area is the area outside the BS coverage that can be covered by the UAV, located in the middle area. The BS is unable to provide the required signal quality directly to the user or UAV located in this area. Thus, it is advisable to utilize the UAV located in the middle area as a mediator. The outer area corresponds to the UAV coverage area under the condition that the UAV maintains its connection to the BS. In this way, the UAV essentially widens the BS coverage area even in the case of a directional antenna.
For the described scenario we consider following cases depicted on Figure 2 to test and configure emulation test-bed.
• Case 1. This is a static case, where the BS antenna is turned towards UAV and does not change its angle. The UAV is located directly above the user and neither of them are moving. This case is proposed to test the emulator and to assure it is working as intended. This case is illustrated on Figure 2a. • Case 2. This is a semi-static case. The BS antenna is turned towards UAV and does not change its angle. The UAV is located in its initial position and does not move. UE moves in the area where the UAV can cover it without leaving the UAV antenna coverage. This case is to ensure that the simulator/emulator correctly reacts to the user mobility. This case is illustrated on Figure 2b. • Case 3. This is a semi-dynamic case. The BS antenna is turned towards UAV and does not change its angle. The UAV moves inside the BS antenna coverage in order to maintain user connection. The user moves in the area where the UAV can cover it without leaving the BS antenna coverage. This case tests the connection changes between BS and UAV depending on UAV position. This case is illustrated on Figure 2c.
• Case 4. This is a dynamic case. User moves in circles/spirals/given paths around the BS. The UAV moves inside the potential BS coverage in order to maintain user connection. The BS antenna is turning towards UAV when it changes its position. This scenario lets us look into the performance of the fully dynamic system. This case is illustrated on Figure 2d. The cases were tested sequentially to ensure that the designed emulator correctly reflects the changes in signal quality resulting from the mobility of corresponding nodes. The results presented in this paper correspond to case 4, the most general case. We studied the QoE characteristic of the signal according to [24]. We considered that the transmitted traffic uses an iLBC voice codec, and, in order to properly collect the data from the emulator, the BS sent a constant stream of data to the UE. We also propose an analytical model to calculate the borders of the coverage areas. Figure 2. UAV utilization scenarios: (a) Static case, when both UAV and UE are not moving. This case is considered to prepare and test the emulator. (b) Semi-static case, when UE changes its position inside static UAV coverage. Used to test the UAV-UE link. (c) Semi-dynamic case, when UAV changes its position inside the BS directional antenna coverage and UE changes its position inside the provided UAV coverage. Used to test the BS-UAV link. (d) Dynamic case, when UE freely changes its position inside coverage R 2 and R 3 and UAV changes its position inside R 2 in order to keep the link to the UE. BS directional antenna follows UAV so that the BS-UAV-UE link remains stable. This is the target test scenario.

Analytical Model
This section contains an analytical model used to calculate different constraints for the emulation. It describes ways to calculate the coverage of inner, middle and outer areas under the assumption of implementing case 4 as the most general case. The equations presented here also hold true for the other three cases described in the previous section, although in these cases they may take a more simple form. Since the UAV's location is relatively high, we implemented an FSPL model to calculate the distance. Additionally, in this section, we do not account for the Doppler effect, since its impact on the distance calculations is neglectfully small. The proposed analytical model is illustrated by Figure 3, and all the notations are described in the current section. BS sends signal with power P BS tx to UAV using carrier frequency f 1 with bandwidth ∆ν. The UAV forwards this signal towards UE with power P U AV tx using carrier frequency f 2 with the same bandwidth. At moment t, the UE is located at coordinates (x UE , y UE , 0)(t), the UAV is located at coordinates (x U AV , y U AV , h)(t) and does not change its altitude, and the BS is always located at coordinates (0, 0, h BS ), where h BS is the height of BS. Let us introduce notations SNR U AV rx (t) and SNR UE rx (t) for the SNR at the UAV and UE at moment t correspondingly. The UAV changes its position as soon as SNR UE rx (t) ≤ SNR * UE , where SNR * UE is a threshold value of the SNR received by UE from the UAV. This means that as soon as it is unable to maintain a high enough quality, the UAV would start moving to the point closest to the UE position at time t. The UAV moves in such a way that its antenna is always directed towards the BS. At the same time, the BS antenna starts following the UAV position in case SNR U AV rx (t) ≤ SNR * U AV , where SNR * U AV is a threshold value of the SNR received by the UAV from the BS.
Since we consider the FSPL propagation model, UAV coverage can be represented as where N is the noise floor.
The BS antenna has a directive diagram f (α) and thus the SNR value on the UAV can be calculated as where α is the angle between the antenna direction and the vector between the BS and UAS, and d BS,U AV (t) = (x 2 U AV + y 2 U AV + (h BS − h) 2 is the distance between the BS and UAV.
By introducing the threshold value SNR * UE , we can calculate the coverage of the middle and outer sections as solutions for the following equations Additionally, using threshold SNR * U AV , we can calculate the area where the BS antenna should start its rotation towards the drone position as a function of the UAV position G(x U AV , y U AV ) :P BS tx f arctg where φ is the current antenna angle. Coverage of the inner area can be calculated as where v U AV is the velocity of the UAV and ω is the angular speed of the BS antenna. By using data from this model and results of we can calculate the QoE as where ploss is packet loss-a percentage of packets lost during data transmission session between BS and UE on downlink channel in data plane. It is acquired through emulation, described in the next section.

Proposed Testbed Architecture
The main factors influencing the signal propagation channel characteristics for dynamic objects are (i) the Doppler effect, which occurs due to the non-stationarity of objects relative to one another, (ii) possible signal blockers, and (iii) changes in the state of the propagation environment. The use of the UAV as a relay base station introduces significant Doppler shift variations, which leads to an increase in the symbol delivery time and a change in the channel frequency. Scenarios with UAVs show significant Doppler variations with delay. Furthermore, the channel expansion varies in time and frequency. This is due to several factors, but the main Doppler shifts are caused by interacting transmitters and receivers in the multipath channel. Other factors determine local delays and Doppler spectral characteristics of channels, such as antenna radiation patterns and angular statistics of individual multipath propagation components (MPCs).
Most channel characteristics, such as the gain and phase shifts of individual multipath components, can be modeled by complex-static processes. However, the estimation of more MPC parameters is required for a more accurate channel model calculation.
The impulse response of the channel of moving objects, h(t, τ), is a continuous variable that depends on the signal transmission time, τ, and the signal processing time, t, and can be expressed as the sum of MPC: where P represents the specific MPC beam determined by the received signal strength, whereas the component h p (t) is defined [25] as follows: where η p is the attenuation constant of the ray p and v p is the Doppler shift's value on this ray. For the time-discrete output signal r(m) calculation, it is necessary to convolve the impulse response at each time with a given sampling period T, i.e., where r(t) values are calculated for t = mT values, m = 1, 2, . . . .
In this paper, we analyze the impact of the proposed approach, which utilizes UAVs to expand the terrestrial network coverage while maintaining the QoE service level. To this end, we implemented an emulator of a test network based on SDR devices, which reproduces all interacting elements' channel characteristics for the scenarios described in the system model.
The USRP 2954R radio network prototyping tool from National Instruments [26] was used for SDR devices to build the emulator. This tool has a built-in FPGA and two radio frequency channels that allow simulating networks in the range up to 6 GHz with a bandwidth of up to 160 MHz.
Since the emulator operates in real time, an FPGA module is used to perform convolution for the output signal from Equation (10), which simulates any linear time-varying channel response using simple operations, e.g., multiplication and addition. The signal pre-conversion structure specified for the FPGA is shown in Figure 4. Specifically, it is a chain of two main signal processing units between the transmitter and the receiver. These units allow modification of the characteristics of each beam at the same time following the discrete form. The transmitter sends the generated signal s[m] represented as the in-phase and quadrature (I&Q) components of each beam (MPC). Then, the signal is divided into separate beams and enters block 1 at the separate gates for each beam and is scaled in amplitudeη according to the left factor in Equation (11). In block 2 in Figure 4, the right factor e j(2πv p (t)+φ(t)) is already added, which is responsible for phase shifts in the environment. Finally, in the last block, all beams with new characteristics are combined into an output signal r[m], which the receiver obtains in the form of I&Q.
In our implementation, the system operates at a bandwidth of 40 MHz, which corresponds to 200 mbps (QAM256) with a sampling time of 25 ns.
The test setup consists of three USRP instruments in conjunction with a PC: two instruments emulate transceiver devices, and one mimics the conditions of the radio signal propagation environment. PCs obtained with a PCIe to USRP interface card are responsible for loading the compiled FPGA configuration bit files and changing SDR devices' parameters dynamically. A scheme of the organization of element interaction is illustrated in Figures 5 and 6.
The devices were connected to each other via a coaxial cable. USPR #1 was connected via the RF1 interface to RF1 USPR #2, whereas USPR #2 and #3 were connected via RF2 to RF1, respectively. When emulating data transmission, an I&Q MPC radio signal is generated at node 1 (USRP + PC) and transmitted to node 2, where the signal changes via the USRP FPGA following the specified environment parameters. The converted signal is delivered to the RF node 3 and analyzed on the PC. The reverse transmission occurs similarly. Due to the design features of the USRP, while connecting the RX and TX of two USRPs with a coaxial cable, we used signal attenuators at −30 dBi for each channel.
Each simulation object (UAV, BS, UE, signal propagation channel) is represented as an individual SDR. In view of this fact, emulation was carried out in two stages: 1.
In order to transmit a signal from the BS to UE, the signal is transmitted to the UAV via a signal propagation channel as I&Q and the data matrix are stored at the st3 node. Then, the matrix is used in the second emulation stage, while retaining all of its properties. This method lets us minimize the number of SDRs in the testbed.

Performance Evaluation
In this section, we present the results of emulation. We consider a single user scenario, where the user moves through the network. The architecture of the network is presented in Section 3. The entire network is divided into two large areas-BS area and UAV area. The BS area is limited by a circle of radius R 1 centered at the BS, i.e., cell coverage radius. Within this area, the user is serviced by the terrestrial BS since the channel quality is sufficient to meet the QoS/QoE requirements. When the user leaves the BS area, he enters the UAV area and is transferred to the UAV. Directional antennas with rotary mechanisms are used to establish the connection between the UAV and the terrestrial BS. This makes it possible to ensure the exact direction of the signal between the transceivers regardless of the UAV location. The radius of the area in which we observe the user's movement is limited to 2 km and corresponds to the area R 2 . In view of the fact that we consider a single user scenario, the optimal location of the UAV relative to the user is strictly above him. In this case, when the user moves along the network border, it should lead to losses and, accordingly, a decrease in bandwidth and QoS/QoE, which is evaluated after the emulation is carried out. In cases when the signal is too strong, it is limited to the value of −55 dB. Table 2 presents the initial data used in the trial. The BS coverage area radius is given since usually coverage areas are known due to the cells at the deployment stage.
Since the emulator uses RSSI as the threshold, the corresponding SNR values for the analytical model are presented in the table, while RSSI values themselves are calculated. Table 3 presents main calculated characteristics of the system, such as radius of different areas and SNR threshold values used for emulation.
During the trial, a single user was moving through the areas along a given trajectory. When the user moves through the inner and BS coverage areas, they are directly connected to the BS; however, when they leave the BS coverage area and enter the middle area, the UE switches to the UAV located at the point where the UE trajectory leaves the BS coverage area. Figure 7 depicts the UE movement trajectory in the trial. The figure shows the BS coverage area, user trajectory and measurement points. The BS coverage area is illustrated by a green circle; the yellow area depicts the middle area. The spiral trajectory was chosen to emulate the worst case for the BS directional antenna, when it should constantly move towards the UAV that follows the user.   Figures 8 and 9 show the emulation results of packet loss and throughput for different UE velocities. It can be noted that the packet loss plots slightly rise up for points 1 to 9, similarly, and throughput plots fall for these values. This is because the first nine points are located inside the inner and BS coverage areas of the cell, where UE is directly connected to the BS. Thus, a slight increase in packet loss and fall in throughput can be explained by the fact that UE distances itself from the center of the cell, moving towards its edge. A more sharp packet loss incline and throughput decline for case of UE velocity of 100 kmph is due to the fact that the BS connects to the UE via a multisector antenna and switching between sectors happens quite often for the fast moving UE with a given trajectory. Thus, more time is spent switching between sectors.It can be seen that, after leaving the green area, packet loss first falls to a lower value and then rises with distance. The initial fall is due to the fact that after the UE enters the middle area, it connects to the BS via the UAV, and since the UAV uses the directional antenna to connect to the BS, the signal actually becomes more stable and then falls as the distance grows. The same explanation holds true for throughput.   Figure 7. The blue line corresponds to the pedestrian UE walking along the trajectory, the red line corresponds to the UE riding a scooter, and the gray line corresponds to the UE riding a car with a velocity comparable to that of the UAV. Figure 10 presents QoE results according to the methodology of [24]. Plot behaviour is similar to that of packet loss and throughput and can be explained in a similar manner. One can see that for the considered scenario the best possible QoE value is 4.1 and it is reached inside the BS coverage area near the center of the cell. One can also note that using a UAV as a repeater to enhance the coverage keeps the QoE above 3. This means that in the considered scenario, while enhancing cell coverage from 650 m to 2 km, the overall loss in the QoE resulted in only one point at the edge of the new coverage. Even if we add an additional condition that the quality should not fall below four points, the cell coverage can still be doubled, since even for the UE moving with the same speed as the UAV, the flight speed plot falls below four points only after measurement point 30. If we consider three points as an adequate QoE, the coverage can be increased even further.

Conclusions
In order to study the performance of the UAV-assisted 5G network architecture considered in this work, a flexible functional complex testbed emulating traffic flows in such networks was developed. The deployed core network allowed us to artificially introduce delay distortions depending on the expected geographic location of the network nodes. It uses NFV in conjunction with SDR and allows virtualization of all network core functions, making it possible to quickly and flexibly change its configuration. The analytical model of the considered scenario was proposed to acquire the constraints used in the conducted emulation.
The acquired results show that using UAVs as repeaters can potentially double cell coverage area while retaining QoE level. Moreover, the coverage can be even further increased while maintaining a satisfactory QoE level with a drop of a single level.
This work scenario can be further developed by adding stationary and/or mobile blockers to a model urban environment. Another improvement would be considering multiple user scenarios while grouping users in small clusters, achieved via a single UAV. In order to prevent users from leaving their clusters, a rewards offering method can be implemented. Funding: This paper has been supported by the RUDN University Strategic Academic Leadership Program (recipient Abdukodir Khakimov). The reported study was funded by RFBR, project number 20-37-90131 (recipient Dmitry Poluektov).