Joint Resource Management and Trajectory Optimization for UAV-Enabled Maritime Network

Due to the lack of places to employ communication infrastructures, there are many coverage blind zones in maritime communication networks. Benefiting from the high flexibility and maneuverability, unmanned aerial vehicles (UAVs) have been proposed as a promising method to provide broadband maritime coverage for these blind zones. In this paper, a multi-UAV-enabled maritime communication model is proposed, where UAVs are deployed to provide the transmission service for maritime users. To improve the performance of the maritime communication systems, an optimization problem is formulated to maximize the minimum average throughput among all users by jointly optimizing the user association, power allocation, and UAV trajectory. To derive the solutions with a low computational complexity, we decompose this problem into three subproblems, namely user association optimization, power allocation optimization, and UAV trajectory optimization. Then, a joint iterative algorithm is developed to achieve the solutions based on the successive convex approximation and interior-point methods. Extensive simulation results validate the effectiveness of the proposed algorithm and demonstrate that UAVs can be used to enhance the maritime coverage.


Background
Maritime communication plays an important role in many maritime applications including oil exploitation, disaster warning, and military surveillance. All these scenarios require higher data rate and more reliable wireless communication. However, these requirements cannot be met by the existing maritime communication networks, which only reach a few Mbps [1,2]. At present, without terrestrial cellular network, very-high-frequency (VHF) or satellite technologies are adopted for maritime communication [3]. While the satellite can provide wide coverage, it suffers from some problems, such as large communication delay and expensive charge. On the other hand, the on-shore base stations (BSs) can only provide a limited offshore coverage, where the coverage blind zones are inevitable. These mean that the existing infrastructures cannot meet the increasing maritime communication demands.
To overcome the problems of traditional maritime wireless communication, unmanned air vehicles (UAVs) have been proposed as a promising method to provide high-rate and stable communications for maritime users with a weak channel state [4][5][6]. Benefiting from their agile mobility, UAVs have a larger coverage range than terrestrial base stations (TBSs) and they are more appropriate for mobile users when deployed as aerial mobile base stations [7,8]. In addition, UAVs are cost-effective for large-scale applications in a maritime scenario. However, current studies mainly focus on UAV communication in the terrestrial scenario [9]. Due to the unique environment on the ocean, there are many challenges to apply UAVs in maritime wireless communication.
limited on-board energy and spectrum scarcity problem, they have some unique challenges. First, the hash maritime environment may affect the deployment of UAVs. Different from the terrestrial case, the maritime environment is greatly affected by the weather, such as typhoon and waves. Secondly, since UAVs are parts of the maritime communication networks (MCNs), they rely on the existing MCNs for backhaul links. Regarding the hybrid network architecture, the UAVs need to jointly allocate their communication resources and coordinate interference. Finally, different from previous works that utilize the free space loss model to simplify analysis in terrestrial scenarios [24,25], it is crucial to consider the unique channel model for maritime communication.
To tackle these challenges, in this paper, we consider a multi-UAV enabled maritime network for large-scale maritime communication service, especially, in blind zones. With the practical maritime channel model, the average data rate is maximized by jointly adjusting the user association, power allocation, and UAV trajectory. The main contributions of this paper are summarized as follows. • A multi-UAV enabled maritime network model is proposed to enhance the coverage of the maritime communication. To improve the maritime performance, the user association, power allocation, and the UAV trajectory are jointly optimized. Specially, the maritime two-ray model, which includes the LoS ray and sea-reflected ray, is adopted to analyze the throughput of the maritime user. Then, an optimization problem is formulated to maximize the average achievable data rate, which is an NP-hard mixed integer nonlinear programming (MINLP) problem. • To achieve the solutions with a low computational complexity, the optimization problem is decomposed into three subproblems, namely user association optimization, power allocation optimization, and UAV trajectory optimization. Consequently, the successive convex approximation (SCA) technique and block coordinate descent (BCD) method are adopted to solve the corresponding subproblems. Based on this, a joint iterative algorithm is developed. • We provide some analytical results on the computational complexity and convergence of the proposed algorithm. In addition, extensive simulations are carried out to demonstrate the effectiveness of the proposed algorithm, and that the proposed algorithm converges well and improve the performance of the maritime network. The results also demonstrates that the proposed algorithm exhibits a low computational complexity.

Organization
The rest of this paper is organized as follows. In Section II, we introduce the system model of maritime UAV communication and the problem formulation. Then, an efficient optimization algorithm is proposed to obtain the maximum average rate among all maritime users in Section III. In Section IV, we analyze the convergence and complexity of the proposed algorithm. Experimental results are presented to verify the proposed algorithm in Section V. Finally, Section VI concludes the whole paper.

System Model and Problem Formulation
In this section, we first describe the system model and then formulate an optimization problem. Table 1 presents the list of symbols used in this paper.

System Model
As shown in Figure 1, we consider a maritime wireless communication system, where M UAVs are deployed to serve K users in the open sea. These UAVs and maritime users are denoted by sets M and K, respectively. Regarding the limited spectrum resources, all UAVs reuse the same frequency to provide downlink services for the maritime users in a flight duration of T. These users share the downlink frequency with the time division multiple access (TDMA) technique. Considering the hash maritime environment, these maritime UAVs can adopt the design in [26], which can have a better performance than traditional UAVs. Considering that the moving range of maritime users is relatively small in a flight duration, it is assumed that the user's position in a flight duration T is fixed. The horizontal coordinate of the user k can be denoted as w k = (x k , y k , z k ). To simplify the theoretical analysis, it is assumed that all UAVs connect with users at a fixed height of H, which can make UAVs fly smoothly without rising or falling frequently. Hence, the trajectories of UAVs at the time t can be denoted by q m (t) = (x m (t), y m (t), H), where t ∈ T. It is also assumed that UAVs have all the information of each user, such as CSI, the maritime users locations, and so on. For the convenience of analysis, the flight duration is divided into N time slots with the length of δ t . Therefore, the trajectory of UAV m can be denoted by q m (n) = (x m (n), y m (n), H) for n = 1, · · · , N. In general, the UAV trajectory satisfies the following constraint where d max = V max δ t represents the maximum distance that the UAV can fly in a time slot, V max indicates the maximum speed of the UAV, and || · || is the Euclidean distance. To avoid the collision between UAVs, the following constraints should also be met where d min denotes the minimum collision-proof distance. Since the places where UAVs can park on the sea are very limited, we consider that UAVs return to the starting point after a fight duration. Then, they can replenish power to better complete the next flight duration. Hence, the corresponding constraint can be written as Assume that the number of users is larger than the number of UAVs. We adopt a binary variable a k,m [n] to indicate the connection between user k and UAV m at time slot n. Specifically, when a k,m [n] = 1, it means user k is connecting with UAV m. Otherwise, a k,m [n] = 0. Without a loss of system performance, it is assumed that in one time slot, each UAV can only serve one user and each user can only access one UAV at most. Hence, a k,m [n] satisfies a k,m [n] ∈ {0, 1}, ∀k, m, n, For battery-powered UAVs, the limited energy consists of two parts, i.e., the communication energy and the propulsion energy. Regarding the limited energy of UAVs, we restrict the flight duration and transmission power of UAVs. Besides, we also consider the transmission interference between UAVs, so the restriction on the transmission power can be given by 0 ≤ p k,m [n] ≤ P max , ∀k, m, n, where P max is the maximum transmission power of the UAV.

Throughput Analysis
The maritime channel models have been comprehensively investigated [10]. According to the existing results [11], the two-ray signal transmission model, including the LoS ray and the sea-reflected ray, is more appropriate for maritime communication. Based on this model, the channel power gain from UAV m to user k can be expressed as where λ denotes the carrier frequency wavelength, h k and h m represent the antenna height of user k and UAV m, respectively, and d k,m [n] = q m [n] − w k 2 is the distance between user k and UAV m at time slot n.
Since the maritime radio propagations are sparse, the Doppler shift needs to be considered in maritime wireless communication [27]. In this paper, we assume that the Doppler shift can be perfectly compensated by the phase rotation to simplify the analysis [28]. As mentioned earlier, all UAVs use the same frequency band to serve the maritime users. Hence, the co-channel interference between UAVs should be considered. Then, the achievable data rate of user k served by UAV m can be written as where σ 2 denotes the noise power. Therefore, the average achievable data rate of user k during the total flight duration can be expressed as Considering the fairness of the maritime users' throughput, we aim to maximize the minimum average rate among all maritime users via jointly optimizing the user association, transmission power, and all UAVs' trajectories. Hence, the problem is formulated as

Problem Formulation
Obviously, the problem in (11) is an MINLP problem. It is intractable to directly solve this problem by using conventional convex optimization methods. We will solve this problem in the next section by problem decomposition.

Joint User Association, Power Allocation, and Trajectory Optimization
In this section, we first decompose the problem in (11) into three subproblems, namely user association optimization, power allocation optimization, and UAV trajectory optimization. Then, the optimal solutions of these three subproblems are developed, respectively.

Problem Decomposition
To obtain the solution, we introduce a slack variable η. Then, an equivalent problem can be given as max and (12)- (18). Obviously, the above problem is still an MINLP problem and NP-hard [29], since the variables A, P, and Q are jointly coupled in the problem. On the one hand, this problem includes integer constraints, where the variable A is a binary parameter vector. On the other hand, when we fix A and another parameter vector P or Q, this problem is still non-convex and challenging to solve. To obtain the solutions, these variables should be decoupled. Then, the problem in (11) can be decomposed into three subproblems: user association optimization, power allocation optimization, and UAV trajectory optimization.

User Association Optimization
Given the power allocation P and UAV trajectory Q, the user association optimization problem can be first expressed as max It is obvious that this problem is a mixed integer linear programming (MILP) problem and the optimal solution cannot be directly obtained in general. To achieve the feasible solution with a low complexity, the problem in (21) can be transformed by relaxing the binary variable a k,m [n], as max The above problem in (26) is now convex since the objective problem and all constraints are linear with respect to a k,m [n] and η. Hence, we can efficiently solve this optimization problem by the interior-point methods [30].

Power Allocation Optimization
Similarly, given user association A and UAV trajectory Q, the problem in (11) can be expressed as max which is also non-convex since R k,m [n] in (29) is non-convex with respect to p k,m [n]. To solve this problem, we first rewrite R k,m [n] as both are concave for variable P. Since the result of subtracting two concave functions is not necessarily concave, R k,m [n] should be further simplified. By using the SCA technique [31][32][33], a convex expression can be derived to approximateR k,m [n], which is described in Lemma 1.

Lemma 1. For any given local point
, ∀k, m, n , the following inequality holdŝ where B k,j [n] is the coefficient of the Taylor expansion, and it is given in Appendix A.
Proof. Please refer to Appendix A.
With the above formula transformation, the problem in (28) can be re-formulated as The constraint (34) is convex with respect to p k,m [n], and the constraint (35) is a linear constraint with respect to p k,m [n]. As a result, the problem in (33) is a convex optimization problem. Thus, a suboptimal solution of transmission power to the problem in (28) can be efficiently solved by CVX.

UAV Trajectory Optimization
Given user association A and power allocation P, the problem in (11) can be rewritten as max which is still a non-convex problem due to the throughput-related constraint (37) and the collision avoidance constraint (40). Thus, it is difficult to solve this problem efficiently. To address this challenge, we adopt an approximate method by using the SCA technique for the UAV trajectory optimization problem in (36).
For the constraint (37), the following formula can be first derived by expanding R k,m , as In the maritime environment, the distance between the transmitter and the receiver (in the order of kilometers or tens of kilometers) is much higher than the antenna height (in the order of meters or tens of meters) [34]. Hence, the equivalent infinitesimal substitution can be used in (41). When x tends to zero, x and sin x are equivalent. By this theorem, and (38) where X k,j [n] satisfies the following constraint It can be found that q j [n] − w k 2 is a convex function with respect to q j [n], which causes the non-convexity of the problem in (42). Hence, we apply the SCA technique to transform the non-convex constraint. By applying the first-order Taylor expansion at the local point Q l = q l m [n], ∀m, n obtained in the l-th iteration, we have which is convex since Q lb [n] is linear with respect to q j [n]. However, (44) does not conform to the CVX optimization rule since X k,j [n] is in the denominator of (44). Therefore, CVX cannot be used to solve the problem in (42) directly. Then by Lemmas 2 and 3, the problem in (42) is re-formulated to satisfy the CVX optimization rule.
with an additional constraint as where z k,j [n] is the auxiliary variable. The new constraint (47) is convex.
Proof. Please refer to Appendix B.
After the above transformation,R k,m [n] is convex with respect to variable X k,j [n] and conforms to the CVX optimization rule. Next, we will transformŘ k,m [n] into a linear form, as described in Lemma 3.

Lemma 3. For any given local point
, ∀m, n , the following inequality holdš where −I l k,j [n] is the coefficient of Taylor expansion and it is given in Appendix C.
Proof. Please refer to Appendix C.
According to the previous transformation, (43) can be re-expressed as which is a convex constraint. Similarly, we apply the SCA technique to transform the collision avoidance constraint (40) into a convex one, as Therefore, after introducing the auxiliary variables X k,j [n], D k,j [n], z k,j [n] and transforming the formulas, the problem in (36) can be rewritten as where the constraints are all linear or convex. The problem in (52) is now convex and can be solved by CVX directly.

Proposed Algorithm
In this section, an overall algorithm is proposed to solve the three subproblems in (21), (28) and (36). Then, the convergence and computational complexity of this algorithm are analyzed.

Algorithm Development
According to the previous analysis, in this subsection, we propose an efficient algorithm to jointly optimize user association, power allocation, and UAV trajectory. The optimal solutions {A, P, Q} can be achieved from the above three subproblems in (21), (28) and (36), respectively. By alternatively optimizing these three subproblems, the problem in (11) can be solved with a low complexity, which is concluded in Algorithm 1.

Algorithm 1 Joint user association, power allocation, and UAV trajectory optimization algorithm
Input: The iteration index l = 0, the maximum number of iteration N max , and the convergence threshold ε. Output: The optimal user association A, power allocation P, UAV trajectory Q, and the expected throughput R. 1: Initialize the transmission power P l and the UAV trajectory Q l with any positive values. 2: Initialize the user association A l = {0}. 3: while R l − R l−1 ≥ ε or l ≤ N max do 4: Obtain the optimal A l+1 for given P l and Q l by solving problem (21).

5:
Obtain the optimal P l+1 for given A l+1 and Q l by solving problem (28). 6: Obtain the optimal Q l+1 for given A l+1 and P l+1 by solving problem (36). 7: Set l = l + 1. 8: end while 9: Obtain the optimal user association A, power allocation P, UAV trajectory Q, and the expected throughput R.

Convergence Analysis
For convenience, we define A l = a l k,m [n], ∀k, m, n , P l = p l k,m [n], ∀k, m, n , and Q l = q l m [n], ∀m, n in the l-th iteration of Algorithm 1. Let η lb p (A l , P l , Q l ) and η lb q (A l , P l , Q l ) be the objective values of the problems in (33) and (52) in the l-th iteration, respectively. According to Step 4 of Algorithm 1, we have For given local A l+1 , P l and Q l in Step 5 of Algorithm 1, we have where (a) holds since the first-order Taylor expansion in (32) is tight at the given local point P l , which means that the problem in (33) at P l has the same objective value as that of the problem in (28). (b) holds since the problem in (33) is solved optimally with solution P l+1 in step 5 with given A l+1 and Q l . (c) holds since the objective value of the problem in (33) is a lower bound of the problem in (28). The inequality (61) indicates that the objective value of the problem in (33) does not decrease after each iteration. Similarly, given local A l+1 , P l+1 , and Q l of Algorithm 1, we also have According to (60)-(62), we have which indicates that the objective value of the problem in (19) is non-decreasing after each iteration of Algorithm 1. Since the maximum value of the problem in (19) is a finite value in the feasible domain, Algorithm 1 can converge. This completes the convergence proof of our proposed algorithm.

Computational Complexity Analysis
In Step 4 of Algorithm 1, since the problem in (26)

Numerical Simulation
In this section, we verify the performance of the proposed algorithm by simulations. In the simulations, there are six maritime users located within a 2 × 2 km 2 area. Two UAVs are placed to provide communications to maritime users. Moreover, the height of the UAV is fixed at 50 m. Other main parameters are listed in Table 2.

Single UAV
We first consider a simple case where only one UAV is taken into consideration, i.e., M = 1. Figure 2 shows the real-time horizontal trajectory of a single UAV. Note that the UAV does not need to control the transmission power in this case since there is no interference. Thus, we set p = P max . From this figure, when the flight duration is short, the flight range of the UAV expands as large as possible. This is because the UAV should move closer to the maritime users to obtain a larger achievable rate. With the increase in the flight duration T, the coverage range of the UAV becomes larger. When the flight duration T is sufficiently large, the flight range of the UAV can cover all maritime users. In this situation, the UAV can approach all users in turn and hover over them for a certain time. Figure 3 presents the speed change of the UAV during the whole flight duration with T = 410 s. It can be observed that when the UAV flies close to the maritime user, it will hover over this user. At this time, its speed is close to zero. Then, it will fly to the other users with the maximum speed so that it can provide a better service within a limited flight duration.  As shown in Figure 4, we compare the proposed joint optimization scheme in Algorithm 1 with another two schemes, i.e., static scheme and circle scheme. The static scheme is that the UAV hovers over the middle point of all users, and the circle scheme represents that the UAV flies with a circular trajectory. From this figure, the proposed scheme in Algorithm 1 outperforms the other two schemes. For the static scheme, the average achievable data rate remains unchanged as the flight duration increases, since the distance between the UAV and users is fixed. For the circular scheme, the average achievable data rate first increases and then remains unchanged with the increase in the flight duration T. This is because when the flight duration T is short, the UAV can only cover a small area and the distance between the UAV and users is long. When T is sufficiently large, the UAV can cover all maritime users and provide service for each user. In terms of the proposed algorithm, the average achievable data rate R increases as the flight duration T increases. When T is sufficiently large, R increases slowly and finally remains unchanged. This is because when the flight duration T is sufficiently large, the UAV has sufficient time to fly close to the maritime users and hovers over them for a certain time. In this situation, the time for flying from one user to another can be negligible and each user can get equal-time service. In this case, the best performance can be achieved compared against the other two schemes.

Multiple UAVs
Next, we consider a more general case where two UAVs are deployed to serve multiple maritime users. Figure 5 illustrates the real-time trajectories of the two UAVs within the flight duration T = 100 s. As shown in Table 3, instead of hovering over all the users, these two UAVs collaboratively hover over the maritime users close to them and provide service for them. To avoid collision and mutual communication interference, these two UAVs must fly away from each other. However, this strategy will lead to a loss of strong direct communication links when they have to serve two users who are close to each other. By the proposed scheme, since we properly adjust the transmission power of the UAV, both strong direct link and low interference can be achieved, resulting in a larger average achievable data rate.   Figure 6 illustrates the transmission power of the two UAVs within the flight duration T = 100 s. From this figure, there is always a UAV keeping a large transmission power. When two UAVs fly away from each other, they will increase their transmission power to improve the spectrum efficiency, e.g., from t = 12 s to t = 38 s. In contrast, when they serve users who are very close to each other, they will reduce their transmission power to decrease the mutual interference, e.g., from t = 44 s to t = 87 s. Such a joint power allocation and trajectory optimization strategy decreases the mutual interference and obtains a high average achievable data rate.  Figure 7 depicts the changes of the speeds of the two UAVs within the flight duration T = 100 s. From this figure, the instantaneous speed first keeps V max = 50 m/s for a while and then drops rapidly to about 0 m/s, and holds the speed. This kind of change takes place three times in total during the flight duration. This is because each UAV sequentially serves these maritime users. It will hover over the user for a while with v = 0 m/s. Then, it will fly to the next user with the maximum speed to maximize the average achievable data rate.  Figure 8 shows the average achievable data rate of the multi-UAV scenario under different schemes. The other three baseline schemes in [3,35] are adopted to demonstrate the effectiveness of the proposed algorithm, which are introduced as follows.
• Scheme I: The user association and UAV trajectory are jointly optimized but without transmission power allocation optimization (maximum transmission power). • Scheme II: The user association and power allocation are jointly optimized but without trajectory optimization (the UAVs follow circular paths). • Scheme III: The user association and power allocation are jointly optimized but with static UAVs, which are placed at the geometric center of the users. By the performance comparison, the following observations can be made. Firstly, except for Scheme III, the max-min average achievable rates of the other three schemes increase as the flight duration T increases. Secondly, the performance gap between Scheme I and Scheme II increases as the flight duration T increases. It indicates that even without power control, the proposed trajectory optimization can achieve the throughput gain. This is because when the flight duration T is large, the trajectory design strategy can significantly decrease the mutual interference and achieve a large data rate. Thirdly, the performance gap between the proposed algorithm and Scheme I demonstrates that the transmission power allocation can bring additional throughput gain. The reason is that the communication interference between UAVs can be effectively reduced and the trajectory design becomes more flexible with the power control. Fourthly, by comparing the proposed algorithm and Scheme II, the proposed trajectory design also brings a significant throughput gain. Fifthly, when the UAVs are static, the max-min average achievable rate does not increase with the flight duration T. This is because when the UAVs are static, the communication interference between UAVs will become stable and the CSI between the UAVs and users will remain unchanged. Thus the users' data rates will remain unchanged as well. Finally, compared with the single-UAV case in Figure 4, it can be observed that the proposed algorithm under the two-UAV case takes less time to reach the same achievable data rate. For example, to achieve the same data rate 4.5 bps/Hz, the single-UAV case needs about T = 270 s, whereas the value reduces to less than T = 100 s for the two-UAV case. This is because when the number of UAVs increases, the flight time of each UAV can be saved. In this situation, the UAVs have more time to hover over the users for better communication service. Therefore, it is claimed that the multi-UAV design improves the performance of the maritime communication network. Figure 9 shows the convergence of the proposed algorithm under different flight duration T in the multi-UAV scenario. From this figure, the proposed algorithm can converge within several steps. Comparing the max-min average achievable rates of the different flight duration, the figure shows that the throughput increases with the flight duration. This is expected since a longer flight duration will result in the longer time to fly closer to the maritime users served by the UAVs.

Conclusions
In this paper, we investigate a multiple-UAV-enabled maritime network model. Specially, we construct the communication model by considering the effect of the maritime environment and adopt an approximation method to deal with the complexity of the tworay channel model. The minimum average rate among all users is maximized by jointly optimizing the user association, power allocation, and UAV trajectory. To solve this nonconvex problem, we divide it into three subproblems, i.e., user association optimization, power allocation optimization, and UAV trajectory optimization. By applying SCA and BCD methods, a local optimal solution with low computational complexity is achieved. Finally, extensive simulation results demonstrate the excellent performance of the proposed algorithm, as compared against several baseline schemes. Based on the results of this work, there are still many other research directions that could be further studied, such as considering the energy efficiency, mobile users, the time-varying maritime environment, and the integration of UAVs and satellites.
Author Contributions: Conceptualization, methodology, software, validation, formal analysis, resources, writing-original draft preparation, G.Y. and X.D.; investigation, X.D. and S.L.; writingreview and editing, X.D., S.L. and G.Y.; supervision, G.Y.; project administration, S.L. All authors have read and agreed to the published version of the manuscript.

Conflicts of Interest:
The authors declare no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript: To prove Lemma 1, we first construct the following function, as where the constant a ≥ 0, b ≥ 0. By taking the second derivative of this function, we have g (x) = −a 2 (ax + b) 2 ln 2 . (A2) It can be observed that g (x) ≤ 0. Therefore, g(x) is a concave function. The form of R k,m [n] is similar to g(x), soR k,m [n] is a concave function with respect to variable p k,j [n]. According to [36], any concave function is globally upper-bounded by its first-order Taylor expansion at any given point.