Joint Trajectory and Communication Design for Buffer-Aided Multi-UAV Relaying Networks

Abstract

With the rapid development and evolvement of unmanned aerial vehicle (UAV) technology, UAV aided wireless communication technology has been widely studied recently. In this paper, a buffer aided multi-UAV relaying network is investigated to assist blocked ground communication. According to the mobility and implementation flexibility of UAV relays, it is assumed that the communication link between air-to-ground is the Rician fading channel. On the basis of information causality, we derive the state change of the information in the buffer of UAV relays and maximize the end-to-end average throughput by join the relay selection, UAV transmit power, and UAV trajectory optimization. However, the considered problem is a mixed integer non-convex optimization problem, and therefore, it is difficult to solve directly with general optimization methods. In order to make the problem tractable, an efficient iterative algorithm based on the block coordinate descent and the successive convex optimization techniques is proposed. The convergence of the proposed algorithm will be verified analytically at the end of this paper. The simulation results show that by alternately optimizing the relay selection, UAV transmit power, and UAV trajectory, the proposed algorithm is able to achieve convergence quickly and significantly improve the average throughput, as compared to other benchmark schemes.

1. Introduction

With the rapid development of unmanned aerial vehicle (UAV) technology and wireless communication technology, UAVs are beginning to be widely used in military and civil applications. Meanwhile, due to its promising performance, much research has been done on UAV assisted wireless communications [1,2,3,4,5,6]. In terms of UAV assisted wireless communications, UAVs can be used as mobile base stations (BSs) and/or relays. For example, employing UAVs as BSs can quickly restore wireless communication services when the ground infrastructure is damaged [1]. Another example is adopting UAVs as mobile relays to help provide reliable communication when the end-to-end distance is very long or direct communication is obstructed [2]. In this paper, we consider the UAVs as the mobile relays to assist terrestrial communications. Compared with traditional static relays, employing the dynamic UAVs as relays has several key advantages. First, the high flexibility of the UAV relay allows it to have the opportunity to improve the performance of the considered network by adaptively adjusting its location [2]. Second, on-demand deployment makes UAVs be widely used in emergency and extreme environments, such as border monitoring, forest fire prevention, emergency rescue, etc. [3]. Furthermore, the UAV relay has wide coverage because of its mobility.

Studies have shown that the buffer aided relaying technique can effectively solve the adverse effects brought by time varying channels. It enables relays to have the ability of data storage, breaks through the traditional transmission mode, and flexibly uses the best channel [7,8]. Therefore, in this paper, it is considered that all UAV relays are equipped with a buffer and employed in delay tolerant applications [4]. Benefiting from both relay node mobility and buffer aided relaying, UAV relays equipped with a buffer can store and forward the information under more favorable conditions when certain quality-of-service (QoS) requirements are satisfied.

Note that many studies have been done on UAVs, but little work considers buffer aided UAV relaying networks. In [2,6], the authors maximized the throughput of single-UAV and multi-UAV relaying systems, respectively, via jointly optimizing UAV trajectory and transmit power design. In [5], the authors deduced a closed form propulsion power consumption model for rotary wing UAVs and minimized the total energy consumption of the network by joint UAV trajectory and communication time allocation optimization. In [9], a basic principle combining the path loss exponent and small scale fading was proposed, which analyzed the optimization problem of large scale wireless networks by combining air-to-ground systems and widely used ground-to-ground channel models. In [10], the authors investigated the use of a UAV as a hybrid access point to provide wireless power transfer and assist obstructed ground communications in wireless powered communication networks. In [11,12,13], a new cooperative jamming method was studied to guarantee the communication security between UAVs and ground nodes, which leveraged the jamming from other nearby UAVs to prevent the eavesdropping and combined UAV trajectory and communication design to maximize the average secrecy rate. The works in [14,15,16] focused on buffer aided UAV relaying networks. The authors in [14,15] considered the secrecy rate maximization problem with a two phase buffer aided single UAV relaying network. Based on the buffer constraint, the work in [16] considered a hybrid communication link consisting of free space optical and radio frequency links and maximized the average security rate with the consideration of information causality. The channel models between UAVs and ground nodes can be divided into three types: the light-of-sight (LoS) channel [2], the probabilistic LoS channel [17], and the Rician fading channel [18,19]. Due to the fact that the LoS channel model ignores the shadow fading and small scale fading, the channel is perhaps inappropriate in urban/suburban areas. Therefore, in this paper, our work is done under the Rician fading channel.

In this paper, a two phase buffer aided multi-UAV relaying network is studied, in which we choose two different UAVs to serve the source and destination at the same time, respectively. According to the buffer state and channel state information (CSI), the network will decide which UAV receives the data from the source and which UAV serves the destination. In particular, the UAV will never be chosen if its buffer is empty. In this paper, our goal is to maximize the end-to-end average throughput by jointly optimizing the relay selection, UAV transmit power control, and UAV trajectory based on the buffer constraints. However, the relay selection, UAV transmit power control, and UAV trajectory design are tightly coupled with each other in this formulated problem, which makes it difficult to solve directly via general optimization methods. To tackle the above problem, we relax the binary variables for relay selection into continuous variables and utilize the block coordinate descent and the successive convex optimization techniques to solve the considered problem with an efficient iterative algorithm [20]. The convergence of this iterative algorithm is proven analytically at the end of this paper.

The remainder of this paper is organized as follows. In Section 2, the buffer aided multi-UAV relaying system model and the problem formulation are introduced. In Section 3, the solution approach based on the block coordinate descent and the successive convex optimization techniques is proposed. In Section 4, the simulation results are presented to verify the convergence and performance of the proposed network. Finally, we summarize the conclusions of this paper in Section 5.

2. System Model and Problem Formulation

2.1. System Model

As illustrated in Figure 1, a buffer aided multi-UAV relaying network is considered. The model consists of $K\ge 2$ UAVs, a source node S and a destination node D. Suppose S and D are stationary ground nodes and all UAVs are equipped with a buffer $\mathbb{Q}$. For the convenience of analysis, the data rate instead of the packet rate is considered, and the data in the buffer follow the “first-in-first-out” rule. It is supposed that all UAVs operate in the time-division half-duplex mode and that the communication link between S and D is unavailable due to the shadow fading and small-scale fading. In addition, it is assumed that the considered network always selects two different UAVs to communicate with S and D according to the known CSI, respectively. There is always a UAV storing the data from S, and D will be served by a UAV if and only if the UAV’s buffer is not empty. For interference avoidance, UAV j stores the data from S, and UAV k forwards the data to D over orthogonal time slots, so we do not consider the interference between UAVs.

Without loss of generality, a three-dimensional (3D) Cartesian coordinate system is considered. Hence, the locations of S and D can be denoted as ${\left[w_{{}_m}^T,0\right]}^T,m\in \left\{S,D\right\}$, where $w_m={\left[x_m,y_m\right]}^T$, $m\in \left\{S,D\right\}$. In the given limited flight period of the UAVs, these UAVs are assigned the same communication bandwidth B, and it is assumed that all UAVs fly at a fixed altitude H, which corresponds to the minimum safe altitude to avoid collision with ground obstacles without the need to adjust the flight height of UAVs frequently. The given finite flight period T is assumed to be dispersed into N equal quasi-static time slots, and $d_t$ represents the length of a time slot, i.e., $T=N{d}_t$. The duration $d_t$ is designed to be sufficiently small to approximate the positions of UAVs accurately in each time slot. In the time slot n, the location of UAV k is denoted by $\left\{q_k{\left[n\right]}^T,H\right\}$ where $q_k\left[n\right]={\left[x_k\left[n\right],y_k\left[n\right]\right]}^T$ is the horizontal trajectory of UAV k. The maximum flight speed of the UAVs is defined as $V_{max}$. Thus, during one time slot, the maximum flying distance of the UAV can be expressed as $D=V_{max}{d}_t$. In order to make the UAV trajectory feasible, the distance between the initial and final location of the UAV during the flight period is assumed to satisfy the following constraint:

$$\begin{array}{c}{∥q_k\left[1\right]-q_k\left[N\right]∥}^2\le D^2,\forall k\hfill \end{array}$$

(1)

Besides, in any time slot, all UAVs should be subjected to the maximum speed and collision avoidance constraints:

$$\begin{array}{c}{∥q_k\left[n\right]-q_k\left[n+1\right]∥}^2\le D^2,\forall k,n=1,\dots ,N-1\hfill \end{array}$$

(2)

$$\begin{array}{c}{∥q_k\left[n\right]-q_j\left[n\right]∥}^2\ge V_{max}^2,\forall n,k,j,k\ne j\hfill \end{array}$$

(3)

The distance from UAV k to the ground nodes in time slot n is given by:

$$\begin{array}{c}{d}_{k,m}\left[n\right]=\sqrt{H^2+{∥q_k\left[n\right]-w_m∥}^2},m\in \left\{S,D\right\},\forall k,n\hfill \end{array}$$

(4)

In this paper, we consider the low altitude UAVs, so too high ground obstacles and UAV flight elevation angles are the key factors affecting whether air-to-ground (A2G) links are LoS links. Under practical conditions, the A2G channels of low altitude UAVs will be affected by large scale fading and multi-path fading [21]. However, A2G links are still dominated by LoS links. Therefore, it is not completely assumed that A2G channels are LoS channels or Rayleigh channels. In the Rician fading channel, there are both LoS signals (main signals) and non-line-of-sight (NLoS) signals. Therefore, in this paper, we assume that A2G communication channels obey Rician fading [18,19]. Specifically, the channels are dominated by the LoS path with random phases, and the amplitudes of communication channels within one time slot are considered to be constant, because the horizontal location variation is virtually negligible as compared to the altitude of the UAVs during a sufficiently small time slot [18]. Then, the Rician fading channel model is given by:

$$\begin{array}{c}{h}_{k,m}\left[n\right]=h_{k,m}{\left[n\right]}^{LoS}+h_{k,m}{\left[n\right]}^{NLoS}\hfill \\ =\sqrt{\frac{\beta }{\left(G+1\right)d_{k,m}{\left[n\right]}^2}}\left(\sqrt{G}{e}^{j\frac{2\pi }{\lambda }{d}_{k,m}\left[n\right]}+g_{k,m}\left[n\right]\right)\hfill \\ =\sqrt{\frac{G}{G+1}}{\stackrel{-}{h}}_{k,m}\left[n\right]e^{j{\theta }_{k,m}\left[n\right]}+\sqrt{\frac{1}{G+1}}{\stackrel{-}{h}}_{k,m}\left[n\right]g_{k,m}\left[n\right]\hfill \\ \approx \sqrt{\frac{G}{G+1}}\sqrt{\frac{\beta }{d_{k,m}{\left[n\right]}^2}}{e}^{j{\theta }_{k,m}\left[n\right]}+\sqrt{\frac{1}{G+1}}\sqrt{\frac{\beta }{d_{k,m}{\left[n\right]}^2}}{g}_{k,m}\left[n\right]\hfill \\ m\in \left\{S,D\right\},\forall k,n\hfill \end{array}$$

(5)

where $\beta$ is defined as the channel power gain at the reference distance $d_0=1$ m and $G\ge 0$ denotes the Rician factor. ${\stackrel{-}{h}}_{k,m}\left[n\right]=\sqrt{\frac{\beta }{d_{m,k}{\left[n\right]}^2}}$ represents the free space path loss of the channel. ${\theta }_{m,k}\left[n\right]=\frac{2\pi }{\lambda }{d}_{m,k}\left[n\right]$ denotes the phase of the LoS channel, where $\lambda =\frac{c}{f_c}$ is the wavelength. In $\lambda =\frac{c}{f_c}$, c is defined as the speed of light and $f_c$ is the carrier frequency. For simplicity, we let ${\theta }_{m,k}\left[n\right]$ be modeled as a uniformly distributed random variable in $\left[0,2\pi \right)$ [18]. Moreover, $g_{m,k}\left[n\right]$ is a coefficient of the Rayleigh fading channel, subject to a complex Gaussian distribution with zero mean and unit variance.

A binary variable $\left\{a_{k,m}\left[n\right],m\in \left\{S,D\right\},\forall n,k\right\}$ is defined to indicate whether UAV k is to communicate with ground nodes S or D in time slot n, i.e., $a_{k,m}\left[n\right]=1$ means communicating, and $a_{k,m}\left[n\right]=0$ means no communicating. It is also assumed that S and D communicate with different UAVs and at most one UAV communicates with S and D in the same time slot; thus the relay selection constraints can be obtained as:

$$\begin{array}{c}\sum _{k=1}^K{a}_{k,m}\left[n\right]\le 1,\forall n,m\in \left\{S,D\right\}\hfill \end{array}$$

(6)

$$\begin{array}{c}{a}_{k,S}\left[n\right]+a_{k,D}\left[n\right]\le 1,\forall k,n\hfill \end{array}$$

(7)

$$\begin{array}{c}{a}_{k,m}\left[n\right]\in \left\{0,1\right\},m\in \left\{S,D\right\},\forall k,n\hfill \end{array}$$

(8)

Besides, $p_S$ is a constant of the transmit power of S, and $p_k\left[n\right]$ denotes the transmit power of UAV k in time slot n, which satisfies both the average power constraint and peak power constraint, expressed by $P_{mean}$ and $P_{peak}$, respectively. Thus, the transmit power satisfies the following constraints:

$$\begin{array}{c}\frac{1}{N}\sum _{n=1}^N{p}_k\left[n\right]\le P_{mean},\forall k,n\hfill \end{array}$$

(9)

$$\begin{array}{c}0\le p_k\left[n\right]\le P_{peak},\forall k,n\hfill \end{array}$$

(10)

According to the above discussion and analysis, the maximum achievable rate from S to UAV j in time slot n can be written as:

$$\begin{array}{c}{R}_{j,S}\left[n\right]=a_{j,S}\left[n\right]{log}_2\left(1+\frac{p_S{\left|h_{j,S}\left[n\right]\right|}^2}{{\sigma }^2}\right)\hfill \end{array}$$

(11)

where ${\sigma }^2$ is defined as the additive white Gaussian noise $\left(\mathrm{AWGN}\right)$ at the receiver. Accordingly, these data will be temporarily stored in the buffer of UAV j. At the end of time slot n, the remaining data in the buffer of UAV j can be expressed as:

$$\begin{array}{c}\hfill {\mathbb{Q}}_j\left[n\right]={\mathbb{Q}}_j\left[n-1\right]+R_{j,S}\left[n\right],\forall n,j\end{array}$$

(12)

Then, the maximum achievable rate from UAV k to D in time slot n can be written as:

$$\begin{array}{c}{R}_{k,D}\left[n\right]=a_{k,D}\left[n\right]{log}_2\left(1+\frac{p_k\left[n\right]{\left|h_{k,D}\left[n\right]\right|}^2}{{\sigma }^2}\right)\hfill \end{array}$$

(13)

Therefore, at the end of time slot n, the remaining data in the buffer of UAV k can be expressed as:

$$\begin{array}{c}{\mathbb{Q}}_k\left[n\right]={\mathbb{Q}}_k\left[n-1\right]+R_{k,D}\left[n\right],\forall n,k,k\ne j\end{array}$$

(14)

Considering the buffer constraint, $R_{k,D}\left[n\right]$ can be reformatted as:

$$\begin{array}{c}{R}_{k,D}^{{}^{\prime }}\left[n\right]=min\left(R_{k,D}\left[n\right],{\mathbb{Q}}_k\left[n-1\right]\right)\end{array}$$

(15)

Since the buffer equipped on the UAV is allowed to store the data collected from S in time slot n and forward it to D in any of the remaining slots, we propose the following information causality constraints:

$$\begin{array}{c}{\mathbb{Q}}_k\left[1\right]=R_{k,S}\left[1\right]\hfill \\ {\mathbb{Q}}_k\left[n\right]={\mathbb{Q}}_k\left[n-1\right]+R_{k,S}\left[n\right]-R_{k,D}^{{}^{\prime }}\left[n\right]\ge 0\hfill \\ \forall k,n=2,\dots ,N\hfill \end{array}$$

(16)

2.2. Problem Formulation

Let $A=\left\{a_{k,m}\left[n\right],m\in \left\{S,D\right\},\forall k,n\right\}$, $P=\left\{p_k\left[n\right],\forall k,n\right\}$, and $Q=\left\{q_k\left[n\right],\forall k,n\right\}$, which corresponds to the relay selection, UAV transmit power, and UAV trajectory design, respectively. It is assumed that the locations of the stationary ground nodes are known, and our goal is to maximize the average throughput of the network by jointly optimizing A, P, and Q. We formulate the average throughput maximization problem as:

$$\begin{array}{c}\underset{A,P,Q}{max}\frac{1}{N}\sum _{n=2}^N\sum _{k=1}^K{a}_{k,D}\left[n\right]R_{k,D}^{{}^{\prime }}\left[n\right]\hfill \end{array}$$

(17a)

$$\begin{array}{c}s.t.\sum _{i=2}^n{R}_{k,D}^{{}^{\prime }}\left[n\right]\le \sum _{i=1}^n{R}_{k,S}\left[n\right],\forall k,n=2,\dots ,N\hfill \end{array}$$

(17b)

$$\begin{array}{c}\left(1\right)–\left(3\right),\left(6\right)–\left(10\right)\hfill \end{array}$$

Inequation (17b) is the information causality, which implies that the forwarded data by UAV must be already stored in its buffer. Inequations (1)–(3) are constraints on the UAV’s mobility, including the UAV’s initial and final location, the maximum speed of UAV and collision avoidance. Inequations (6)–(8) are relay selection constraints, and Inequations (9) and (10) are the average power and peak power constraints of transmit power. Since all UAVs can only transmit the data that have been previously stored from S, according to the Shannon formula, Problem (17) can be rewritten as:

$$\begin{array}{c}\underset{A,P,Q}{max}\frac{1}{N}\sum _{n=2}^N\sum _{k=1}^K{a}_{k,D}\left[n\right]{log}_2\left(1+\frac{p_k\left[n\right]{\left|h_{k,D}\left[n\right]\right|}^2}{{\sigma }^2}\right)\hfill \end{array}$$

(18a)

$$\begin{array}{c}s.t.\sum _{i=2}^n{a}_{k,D}\left[n\right]{log}_2\left(1+\frac{p_k\left[n\right]{\left|h_{k,D}\left[n\right]\right|}^2}{{\sigma }^2}\right)\hfill \end{array}$$

$$\begin{array}{c}\le \sum _{i=1}^n{a}_{k,S}\left[n\right]{log}_2\left(1+\frac{p_S{\left|h_{k,S}\left[n\right]\right|}^2}{{\sigma }^2}\right),\forall k,n=2,\dots ,N\hfill \end{array}$$

(18a)

$$\begin{array}{c}\left(1\right)–\left(3\right),\left(6\right)–\left(10\right)\hfill \end{array}$$

Since the objective function is non-concave and the constraints are non-convex, Problem (18) is non-convex and difficult to solve directly. First, since the optimization variables A for relay selection are binary, the constraints in (6)–(8) involve integer constraints. Second, Constraint (3) is non-convex with regard to UAV trajectory variables Q. Finally, the information causality constraint in (18b) is non-convex because of the coupling of three optimization variable. In the next section, an efficient algorithm is proposed to make the problem mathematically tractable.

3. The Proposed Algorithm

In this section, to make Problem (18) more tractable, a method based on optimization variables iteration is proposed. We decompose the original Problem (18) into the following two sub-problems iteratively. First, Sub-problem 1 optimizes the relay selection A and transmit power P with given fixed UAV trajectory Q. Second, Sub-problem 2 optimizes the UAV trajectory Q with given relay selection A and power allocation P. Finally, we present the overall algorithm and prove the convergence of this iteration algorithm. Without loss of generality, through the iteration of the two sub-problems, we can get a suboptimal solution to the original problem.

3.1. Sub-Problem 1: Optimizing Relay Selection and UAV Power Control of a Given UAV Trajectory

For a given UAV trajectory, the relay selection and transmit power allocation of Sub-problem 1 can be expressed as the following problem:

$$\begin{array}{c}\underset{A,P}{max}\frac{1}{N}\sum _{n=2}^N\sum _{k=1}^K{a}_{k,D}\left[n\right]{log}_2\left(1+\frac{p_k\left[n\right]{\left|h_{k,D}\left[n\right]\right|}^2}{{\sigma }^2}\right)\hfill \end{array}$$

(19)

$$\begin{array}{c}s.t.\left(6\right)–\left(10\right),\left(18b\right)\hfill \end{array}$$

We note that Problem (19) is still non-convex because the constraints in (6)–(8) involve integer constraints and the information causality constraint in (18b) is non-convex. To solve the integer constraints, we relax the binary variables in (8) into continuous variables, i.e., $0\le a_{k,m}\left[n\right]\le 1,m\in \left\{S,D\right\},\forall k,n$. Furthermore, the slack variables $\left\{u_{k,m}\left[n\right]\right\}$ are introduced. First, we consider the following problem:

$$\begin{array}{c}\underset{A,P,\left\{u_{k,m}\left[n\right]\right\}}{max}\frac{1}{N}\sum _{n=2}^N\sum _{k=1}^K{a}_{k,D}\left[n\right]u_{k,D}\left[n\right]\hfill \end{array}$$

(20a)

$$\begin{array}{c}s.t.0\le a_{k,m}\left[n\right]\le 1,m\in \left\{S,D\right\},\forall k,n\hfill \end{array}$$

(20b)

$$\begin{array}{c}{log}_2\left(1+\frac{p_k\left[n\right]{\left|h_{k,D}\left[n\right]\right|}^2}{{\sigma }^2}\right)\ge u_{k,D}\left[n\right]\hfill \end{array}$$

(20c)

$$\begin{array}{c}{log}_2\left(1+\frac{p_S{\left|h_{k,S}\left[n\right]\right|}^2}{{\sigma }^2}\right)\ge u_{k,S}\left[n\right]\hfill \end{array}$$

(20d)

$$\begin{array}{c}\sum _{i=2}^n{a}_{k,D}\left[n\right]u_{k,D}\left[n\right]\le \sum _{i=1}^n{a}_{k,S}\left[n\right]u_{k,S}\left[n\right],\forall k,n=2,\dots ,N\hfill \end{array}$$

(20e)

$$\begin{array}{c}\left(6\right),\left(7\right),\left(9\right),\left(10\right)\hfill \end{array}$$

It can be proven that when the constraints (20c) and (20d) take the equal sign, there is an optimal solution for Problem (20). Therefore, Problems (20) and (19) have the same optimal value and optimal solutions of A and P. Next, we concentrate on solving Problem (20).

However, Problem (20) is still non-convex since the variables A and slack variables $\left\{u_{k,m}\left[n\right]\right\}$ are coupled and the objective function is non-concave. Generally, a maximization problem with a non-concave objective function is non-convex and thus difficult to solve optimally. Aiming at this problem, a local convex approximation method is adopted, which ensures the convergence to the local optimal solution. Thus, $a_{k,m}\left[n\right]u_{k,m}\left[n\right],m\in \left\{S,D\right\}$ can be reformulated as [22]:

$$\begin{array}{c}\hfill a_{k,m}\left[n\right]u_{k,m}\left[n\right]=\frac{{\left(a_{k,m}\left[n\right]+u_{k,m}\left[n\right]\right)}^2}{2}-\frac{a_{{}_{k,m}}^2\left[n\right]+u_{{}_{k,m}}^2\left[n\right]}{2}\end{array}$$

(21)

Then, as we all know that the first-order Taylor expansion of a convex function at any point is its global lower bound [23], the lower bound function of ${\left(a_{k,m}\left[n\right]+u_{k,m}\left[n\right]\right)}^2$ at the points $a_{k,m}^f\left[n\right]$ and $u_{k,m}^f\left[n\right]$ can be expressed as:

$$\begin{array}{cc}\hfill {\left(a_{k,m}\left[n\right]+u_{k,m}\left[n\right]\right)}^2& \ge {\left(a_{{}_{k,m}}^f\left[n\right]+u_{{}_{k,m}}^f\left[n\right]\right)}^2+2\left(a_{{}_{k,m}}^f\left[n\right]+u_{{}_{k,m}}^f\left[n\right]\right)\left(a_{k,m}\left[n\right]-a_{{}_{k,m}}^f\left[n\right]\right)\hfill \\ & +2\left(a_{{}_{k,m}}^f\left[n\right]+u_{{}_{k,m}}^f\left[n\right]\right)\left(u_{k,m}\left[n\right]-u_{{}_{k,m}}^f\left[n\right]\right)={\phi }_{{}_{k,m}}^{lb}\left[n\right]\hfill \end{array}$$

(22)

Due to $-a_{k,m}^2\left[n\right]$ and $-u_{k,m}^2\left[n\right]$ being still non-convex, the constraint (20e) is non-convex. However, we find that the value of $a_{k,m}^2\left[n\right]$ is approximate to zero or one and $0\le a_{k,m}\left[n\right]\le 1$, so we let $a_{k,m}\left[n\right]$ replace $a_{k,m}^2\left[n\right]$. Furthermore, the value of $u_{k,m}^2\left[n\right]$ is between zero and one; hence, $u_{k,m}^2\left[n\right]\le u_{k,m}\left[n\right]$. According to the analysis above, Equation (21) and Constraint (20e) can be transformed as:

$$\begin{array}{cc}\hfill a_{k,m}\left[n\right]u_{k,m}\left[n\right]& =\frac{{\left(a_{k,m}\left[n\right]+u_{k,m}\left[n\right]\right)}^2}{2}-\frac{a_{{}_{k,m}}^2\left[n\right]+u_{{}_{k,m}}^2\left[n\right]}{2}\hfill \\ & \ge \frac{{\phi }_{{}_{k,m}}^{lb}\left[n\right]-\left(a_{k,m}\left[n\right]+u_{k,m}\left[n\right]\right)}{2}\hfill \end{array}$$

(23)

$$\begin{array}{c}\sum _{i=2}^n\frac{{\phi }_{k,D}^{lb}\left[n\right]-\left(a_{k,D}\left[n\right]+u_{k,D}\left[n\right]\right)}{2}\le \sum _{i=1}^n\frac{{\phi }_{k,S}^{lb}\left[n\right]-\left(a_{k,S}\left[n\right]+u_{k,S}\left[n\right]\right)}{2},\forall k,n=2,\dots ,N\hfill \end{array}$$

(24)

After such approximation, the objective function is transformed into a concave function, so Problem (19) can be optimized by solving the following problem:

$$\begin{array}{c}\underset{A,P,\left\{u_{k,m}\left[n\right]\right\}}{max}\frac{1}{N}\sum _{n=2}^N\sum _{k=1}^K\frac{{\phi }_{{}_{k,D}}^{lb}\left[n\right]-\left(a_{k,D}\left[n\right]+u_{k,D}\left[n\right]\right)}{2}\hfill \end{array}$$

(25a)

$$\begin{array}{c}s.t.\sum _{k=1}^K{a}_{k,m}\left[n\right]\le 1,\forall n,m\in \left\{S,D\right\}\hfill \end{array}$$

(25b)

$$\begin{array}{c}{a}_{k,S}\left[n\right]+a_{k,D}\left[n\right]\le 1,\forall k,n\hfill \end{array}$$

(25c)

$$\begin{array}{c}0\le a_{k,m}\left[n\right]\le 1,m\in \left\{S,D\right\},\forall k,n\hfill \end{array}$$

(25d)

$$\begin{array}{c}\frac{1}{N}\sum _{n=1}^N{p}_k\left[n\right]\le P_{mean},\forall k\hfill \end{array}$$

(25e)

$$\begin{array}{c}0\le p_k\left[n\right]\le P_{peak},\forall k,n\hfill \end{array}$$

(25f)

$$\begin{array}{c}{log}_2\left(1+\frac{p_k\left[n\right]{\left|h_{k,D}\left[n\right]\right|}^2}{{\sigma }^2}\right)\ge u_{k,D}\left[n\right]\hfill \end{array}$$

(25g)

$$\begin{array}{c}{log}_2\left(1+\frac{p_S{\left|h_{k,S}\left[n\right]\right|}^2}{{\sigma }^2}\right)\ge u_{k,S}\left[n\right]\hfill \end{array}$$

(25h)

$$\begin{array}{c}\sum _{i=2}^n\frac{{\phi }_{k,D}^{lb}\left[n\right]-\left(a_{k,D}\left[n\right]+u_{k,D}\left[n\right]\right)}{2}\hfill \end{array}$$

$$\begin{array}{c}\le \sum _{i=1}^n\frac{{\phi }_{k,S}^{lb}\left[n\right]-\left(a_{k,S}\left[n\right]+u_{k,S}\left[n\right]\right)}{2},\forall k,n=2,\dots ,N\hfill \end{array}$$

(25i)

Thus, when the constraints (25g) and (25h) take the equal sign, there is an optimal solution for Problem (25). Problem (25) is convex, which can be solved effectively by the interior-point method or standard convex tools such as CVX [24]. It can be found that the optimal solution $\left\{A,P\right\}$ can be obtained with given Q, when the constraints (25g) and (25h) take the equal sign. In addition, both A and P will serve as the inputs for Sub-problem 2 in the next subsection.

3.2. Sub-Problem 2: Optimizing the UAV Trajectory a Given Relay Selection and Power Control

For a given relay selection and transmit power, by defining $P_n=\frac{p_k\left[n\right]{\left(\sqrt{\frac{G\beta }{G+1}}{e}^{j{\theta }_{k,D}\left[n\right]}+\sqrt{\frac{\beta }{G+1}}{g}_{k,D}\left[n\right]\right)}^2}{{\sigma }^2}$ and $P_S^f=\frac{p_S{\left(\sqrt{\frac{G\beta }{G+1}}{e}^{j{\theta }_{k,S}\left[n\right]}+\sqrt{\frac{\beta }{G+1}}{g}_{k,S}\left[n\right]\right)}^2}{{\sigma }^2}$, we can obtain Sub-problem 2 as [25]:

$$\begin{array}{c}\underset{Q}{max}\frac{1}{N}\sum _{n=2}^N\sum _{k=1}^K{a}_{k,D}\left[n\right]{log}_2\left(1+\frac{P_n}{{∥q_k\left[n\right]-w_D∥}^2+H^2}\right)\hfill \end{array}$$

(26a)

$$\begin{array}{c}s.t.\sum _{i=2}^n{log}_2\left(1+\frac{P_n}{{∥q_k\left[n\right]-w_D∥}^2+H^2}\right)\le \sum _{i=1}^n{log}_2\left(1+\frac{P_S^f}{{∥q_k\left[n\right]-w_S∥}^2+H^2}\right)\hfill \end{array}$$

$$\begin{array}{c}\forall k,n=2,\dots ,N\hfill \end{array}$$

(26b)

$$\begin{array}{c}\left(1\right)–\left(3\right)\hfill \end{array}$$

Note that Sub-problem 2 is concave with respect to $q_k\left[n\right]$. To make Problem (26) more tractable, slack variables $\left\{t_{k,m}\left[n\right]\right\},m\in \left\{S,D\right\}$ are introduced. Thus, Problem (26) can be transformed as:

$$\begin{array}{c}\underset{Q,\left\{t_{k,m}\left[n\right]\right\}}{max}\frac{1}{N}\sum _{n=2}^N\sum _{k=1}^K{a}_{k,D}\left[n\right]{log}_2\left(1+\frac{P_n}{t_{k,D}\left[n\right]}\right)\hfill \end{array}$$

(27a)

$$\begin{array}{c}s.t.\sum _{i=2}^n{log}_2\left(1+\frac{P_n}{t_{k,D}\left[n\right]}\right)\le \sum _{i=1}^n{log}_2\left(1+\frac{P_S^f}{t_{k,S}\left[n\right]}\right)\hfill \end{array}$$

$$\begin{array}{c}\forall k,n=2,\dots ,N\hfill \end{array}$$

(27b)

$$\begin{array}{c}{∥q_k\left[n\right]-w_m∥}^2+H^2-t_{k,m}\left[n\right]\le 0,m\in \left\{S,D\right\}\hfill \end{array}$$

(27c)

$$\begin{array}{c}\left(1\right)–\left(3\right)\hfill \end{array}$$

It can be shown that when the constraint (27c) takes the equal sign, there is an optimal solution for Problem (27), because we can decrease $t_{k,m}\left[n\right],m\in \left\{S,D\right\}$ to improve the objective value. Consequently, Problems (26) and (27) have the same optimal value and optimal solution of Q. Next, we concentrate on handling Problem (27).

It is easy to see that $t_{k,m}\left[n\right]$ and ${log}_2\left(1+\frac{P_n}{t_{k,m}\left[n\right]}\right)$ are convex. However, the objective function is non-concave, and the constraints in (3) and (27b) are non-convex, so Problem (27) is still non-convex. In order to make the problem (27) solvable, we obtain the lower bound of the objective function (27) by taking the first-order Taylor expansion of ${log}_2\left(1+\frac{P_n}{t_{k,m}\left[n\right]}\right)$ at the points $t_{k,m}^f\left[n\right]$ and then maximizing the concave bound to obtain the optimal solution of Sub-problem 2. Actually, the first-order Taylor expansion of function ${log}_2\left(A+\frac{B}{x}\right)$ is:

$$\begin{array}{cc}\hfill {log}_2\left(A+\frac{B}{x}\right)& \ge {log}_2\left(A+\frac{B}{x}\right)+\frac{1}{ln2\left(A+\frac{B}{x_0}\right)}\left(-\frac{B}{x_0^2}\right)\left(x-x_0\right)\hfill \\ & ={log}_2\left(A+\frac{B}{x}\right)-\frac{B\left(x-x_0\right)}{ln2\left(A{x}_0^2+B{x}_0\right)}\hfill \end{array}$$

(28)

Therefore, The first-order Taylor expansion of ${log}_2\left(1+\frac{P_n}{t_{k,m}\left[n\right]}\right)$ is given below:

$$\begin{array}{cc}\hfill {log}_2\left(1+\frac{P_n}{t_{k,m}\left[n\right]}\right)& \ge {log}_2\left(1+\frac{P_n}{t_{{}_{k,m}}^f\left[n\right]}\right)-\frac{P_n\left(t_{k,m}\left[n\right]-t_{{}_{k,m}}^f\left[n\right]\right)}{ln2\left({\left(t_{{}_{k,m}}^f\left[n\right]\right)}^2-P_n{t}_{{}_{k,m}}^f\left[n\right]\right)}\hfill \\ & ={\varphi }_{k,m}\left[n\right],m\in \left\{S,D\right\}\hfill \end{array}$$

(29)

Then, Constraint (27b) can be rewritten as:

$$\begin{array}{c}\hfill \sum _{i=2}^n{\varphi }_{k,D}\left[n\right]\le \sum _{i=1}^n{\varphi }_{k,S}\left[n\right],\forall k,n=2,\dots ,N\end{array}$$

(30)

Next, we resolve the non-concavity of ${∥q_k\left[n\right]-q_j\left[n\right]∥}^2$ in (3) by applying the first-order Taylor expansion at the given points $q_k^f\left[n\right]$ and $q_j^f\left[n\right]$, which can be written as:

$$\begin{array}{cc}\hfill {∥q_k\left[n\right]-q_j\left[n\right]∥}^2& \ge -{∥q_{{}_k}^f\left[n\right]-q_{{}_j}^f\left[n\right]∥}^2+2{\left(q_{{}_k}^f\left[n\right]-q_{{}_j}^f\left[n\right]\right)}^T\left(q_k\left[n\right]-q_j\left[n\right]\right),\forall k\ne j,n\hfill \end{array}$$

(31)

Then, Problem (26) can be approximated as the following problem:

$$\begin{array}{c}\underset{Q,\left\{t_{k,m}\left[n\right]\right\}}{max}\frac{1}{N}\sum _{n=2}^N\sum _{k=1}^K{a}_{k,D}\left[n\right]{\varphi }_{k,D}\left[n\right]\hfill \end{array}$$

(32a)

$$\begin{array}{c}s.t.\sum _{i=2}^n{\varphi }_{k,D}\left[n\right]\le \sum _{i=1}^n{\varphi }_{k,S}\left[n\right],\forall k,n=2,\dots ,N\hfill \end{array}$$

(32b)

$$\begin{array}{c}{∥q_k\left[n\right]-w_m∥}^2+H^2-t_{k,m}\left[n\right]\le 0,m\in \left\{S,D\right\}\hfill \end{array}$$

(32c)

$$\begin{array}{c}{V}_{max}^2+{∥q_{{}_k}^f\left[n\right]-q_{{}_j}^f\left[n\right]∥}^2-2{\left(q_{{}_k}^f\left[n\right]-q_{{}_j}^f\left[n\right]\right)}^T\left(q_k\left[n\right]-q_j\left[n\right]\right)\le 0,\forall k\ne j,n\hfill \end{array}$$

(32d)

$$\begin{array}{c}{∥q_k\left[1\right]-q_k\left[N\right]∥}^2\le D^2,\forall k\hfill \end{array}$$

(32e)

$$\begin{array}{c}{∥q_k\left[n\right]-q_k\left[n+1\right]∥}^2\le D^2,\forall k,n=1,\dots ,N-1\hfill \end{array}$$

(32f)

After such approximation, we note that the objective function of Problem (32) is concave and all constraints are convex. Therefore, Problem (32) is a convex optimization problem that can be efficiently solved by optimization solvers. In addition, the optimal objective value obtained from the approximate problem (32) is usually the lower bound of Problem (26).

3.3. Overall Algorithm and Convergence

In summary, the proposed algorithm can apply the block coordinate descent method to find a suboptimal solution of Problem (18) with alternately solving the two sub-problems (25) and (32) in an iterative manner. Specifically, we divide the entire optimization variables in the original problem (18) into three blocks, i.e., the relay selection A, UAV transmit power P, and UAV trajectory Q. Then, we alternately optimize the two sub-problems (25) and (32) to obtain the optimal solutions of block variables A, P, and Q. Algorithm 1 details the steps of the proposed algorithm, in which $R^l=f\left(A^l,P^l,Q^l\right)$ is the objective value of Problem (18) with variables $A^l$, $P^l$, and $Q^l$ in iteration l, and $\epsilon$ is defined as the accuracy of convergence, $\epsilon >0$.

Algorithm 1 Proposed joint relay selection, power allocation, and trajectory optimization algorithm.

1: initialization: Set the initial feasible solution $\left\{A^0,P^0,Q^0\right\}$. Set the accuracy tolerance $\epsilon$, the iteration number l = 0, and $R^l=f\left(A^l,P^l,Q^l\right)$. 2: repeat 3:  Solve Problem (25) with given $A^l,P^l,Q^l$ and feasible solution $a_{k,D}^f\left[n\right]=a_{k,D}\left[n-1\right]$, $u_{k,D}^f\left[n\right]=u_{k,D}\left[n-1\right]$, and find the optimal solution $\left\{A^{l+1},P^{l+1}\right\}$. 4:  Solve Problem (32) with given $\left\{A^{l+1},P^{l+1},Q^l\right\}$ and feasible solution $t_{k,m}^f\left[n\right]=t_{k,m}\left[n-1\right]$, and find the optimal solution $\left\{A^{l+1},P^{l+1},Q^{l+1}\right\}$. 5:  Update l = l + 1. 6: until$\frac{R^l-R^{l-1}}{R^l}\le \epsilon$.

Next, we show the convergence of the proposed algorithm. Let us define $R_{A,P}^{lb,l}\left(A,P,Q\right)=R_{{}_{A,P}}^l$ and $R_Q^{lb,l}\left(A,P,Q\right)=R_Q^l$ as the objective value of Problem (25) and (32) based on A, P, and Q, respectively. In general, we consider a bounded function that is non-negative monotonic and undiminished to be convergent. First, it is obvious that the objective values we solve are all non-negative. For given $A^l$, $P^l$, and $Q^l$, the details of obtaining the optimal solution of (25) in Step 3 can be written as [26]:

$$\begin{array}{c}\hfill R\left(A^l,P^l,Q^l\right)\stackrel{\left(a\right)}{=}{R}_{A,P}^{lb,l}\left(A^l,P^l,Q^l\right)\stackrel{\left(b\right)}{\le }{R}_{A,P}^{lb,l}\left(A^{l+1},P^{l+1},Q^l\right)\stackrel{\left(c\right)}{\le }R\left(A^{l+1},P^{l+1},Q^l\right)\end{array}$$

(33)

where $R\left(A^l,P^l,Q^l\right)$ is the pre-defined initial solution of Problem (18). Since Inequality (23) is obtained by the first-order Taylor expansion approximation of the objective function, it has the same optimal solution and optimal objective value as the original problem at $\left\{A^l,P^l\right\}$. Therefore, Equation $\left(a\right)$ is satisfied. For given $\left\{A^l,P^l,Q^l\right\}$, $\left\{A^{l+1},P^{l+1}\right\}$ is the optimal solution for Problem (25), so Inequation $\left(b\right)$ holds. According to the previous discussions, Problem (25) provides a lower bound solution for Problem (19) at $\left\{A^{l+1},P^{l+1}\right\}$; hence, Inequation $\left(c\right)$ is established. According to the inequality in (33), although Problem (25) is an approximate optimal problem for obtaining the relay selection and UAV transmit power, the objective value of the original Problem (18) is still non-decreasing. Similarly, the details of obtaining the optimal solution of (32) in Step 4 can show that:

$$\begin{array}{c}R\left(A^{l+1},P^{l+1},Q^l\right)\stackrel{\left(a\right)}{=}{R}_Q^{lb,l}\left(A^{l+1},P^{l+1},Q^l\right)\stackrel{\left(b\right)}{\le }{R}_Q^{lb,l}\left(A^{l+1},P^{l+1},Q^{l+1}\right)\stackrel{\left(c\right)}{\le }R\left(A^{l+1},P^{l+1},Q^{l+1}\right)\hfill \end{array}$$

(34)

Combining (33) and (34), it can be concluded that:

$$\begin{array}{c}R\left(A^l,P^l,Q^l\right)\le R\left(A^{l+1},P^{l+1},Q^{l+1}\right)\end{array}$$

(35)

Based on the analysis above, we can conclude that the objective value of Problem (18) is non-decreasing over the iteration l, and the objective value of Problem (18) is upper bounded by a finite value. Therefore, our optimization problem is non-negative, non-decreasing, and bounded, i.e., the proposed algorithm converges.

Next, we will discuss the complexity of the proposed algorithm. As shown in Algorithm 1, the key idea of the proposed algorithm is to optimize alternately the relay selection, UAV transmit power control, and UAV trajectory. To make the proposed problem more tractable, we decompose the original problem into the following two sub-problems iteratively. In each iteration, the main complexity of the proposed algorithm lies in Steps 3 and 4, which require solving a series of convex optimization problems. The computational costs of Steps 3 and 4 are about $O\left(L{\left(KN+2N\right)}^3\right)$ and $O\left(L{\left(2N\right)}^3\right)$, respectively, where K is the number of UAVs, N is the number of time slots, and L is the iteration numbers of the proposed algorithms [27,28].

4. Numerical Results

In this section, we provide simulation results to verify the effectiveness of the proposed algorithm for buffer aided multi-UAV relaying networks with joint relay selection, UAV transmit power, and UAV trajectory optimization. We assumed that the network had $K=2$ UAVs and that the locations of S and D were $\left(-500,0,0\right)$ and $\left(500,0,0\right)$, respectively. The initial locations of UAVs were set as $\left(-300,0,100\right)$ and $\left(300,0,100\right)$, respectively. It was assumed that all UAVs flew at the fixed altitude $H=100$ m and that the maximum speed of the UAV was $V_{max}=60$ m/s. The communication bandwidth was assumed to be $B=10$ MHz, and the noise power was ${\sigma }^2=-110$ dbm. The channel power gain at the reference distance $d_0=1$ m was set as $\beta =-70$ dbm. The transmit power of S and the maximum transmit power of UAVs were assumed to be $p_S=0.1$ W and $P_{max}=0.4$ W, respectively. In addition, the average power was $P_{mean}=P_{max}/4$. The initial transmit power of the UAVs was assumed to be equal to $p_S$, i.e., $p_k\left[n\right]=p_S$. Furthermore, we assumed the buffer size equipped with UAVs was infinite.

Figure 2 shows the UAVs optimization trajectories by the proposed algorithm for flight period T = 60 s. The initial locations of UAVs and ground nodes are marked with purple “$\square$” and black “$\square$”, respectively. In this figure, the asterisk indicates that the location of the UAV is sampled at every 0.5 s. The initial and final location of all UAVs satisfied a feasible constraint in (1), and all UAVs satisfied the minimum safety distance to avoid collision. From this figure, we can see the two UAVs tried to get close to their served ground nodes during every time slot under the flying constraints. Because the UAVs served the source or destination in different time slots, sometimes the trajectories of UAV1 and UAV2 may be a straight line from the source to the destination node.

Then, in Figure 3, the flight security of the proposed algorithm is demonstrated for period time T = 60 s. It can be obviously observed that the distance between the two UAVs varied with time and always exceeded the minimum security distance. Therefore, the trajectories of the two UAVs satisfied the collision avoidance constraints in every time slot.

Figure 4 illustrates the average system throughput versus the different flight periods T of the proposed joint optimization algorithm. For comparison, we also simulated the following benchmark schemes:

• No optimization: According to the CSI and buffer state, the system decided which UAV stored the data from the source and which UAV served the destination. The transmit power of UAVs $p_k\left[n\right]=p_S=0.1$ W. A circular flight trajectory with radius of $r=300$ m and the center of the flight trajectory above the middle of S and D were considered.
• Only the relay selection and transmit power optimized: For a given trajectory, we only optimized the relay selection and transmit power via the proposed algorithm.
• Only the trajectory optimized: For given relay selection and transmit power, we only optimized the trajectories via the proposed algorithm.

First, as expected, the average throughput could be increased by optimizing any variable (relay selection and transmit power and/or flight trajectory). Second, it is observed that the joint optimization algorithm we proposed could greatly increase the average throughput in comparison with optimizing a single variable. The results above further demonstrated that the trajectory optimization was more effective than optimizing relay selection and transmit power. Finally, it was found that the average throughput could be improved as the flight period increased with the joint optimization algorithm. In a word, the proposed algorithm could significantly improve the system performance in terms of throughput.

Next, we demonstrate the convergence behavior of Algorithm 1 in Figure 5. The average throughput of this network during different periods is illustrated. From this figure, it can be observed that with the number of iterations of the proposed algorithm increasing, the average throughput of this network gradually converged, and the convergence was achieved at about six iterations. Therefore, the curves satisfied our expectation and verified the effectiveness of the proposed algorithm in maximizing average throughput. Furthermore, it can be found that the system average throughput could be increased by prolonging the period time.

5. Conclusions

A buffer aided multi-UAV relaying network with joint trajectory and communication design was studied in this paper. Specifically, based on the information causality, aiming at maximizing the average throughput of the network, the relay selection, UAV transmit power, and UAV trajectory were jointly optimized. Since the maximization problem was a mixed integer non-convex problem, the problem was re-described and divided into iterations of the two sub-problems through the block coordinate descent and the successive convex optimization techniques. Finally, the convergence of the proposed algorithm was testified analytically. Numerical results demonstrated that Algorithm 1 could significantly improve the average throughput of the considered network. Furthermore, in this paper, we considered the transition of buffer states, and the proposed algorithm can avoid the collision of UAVs in multi-UAV relaying networks.