Context-aware Caching Distribution and UAV Deployment: A Game-theoretic Approach

: This paper investigates the problem of the optimal arrangement for both UAVs’ caching contents and service locations in UAV-assisted networks based on the context awareness, which considers the inﬂuence between users and environment. In the existing work, users within the coverage of UAVs are considered to be served perfectly, which ignores the communication probability caused by line-of-sight (LOS) and non-line-of-sight (NLOS) links. However, the links are related to UAV deployment. Moreover, the transmission overhead should be taken into account. To balance the tradeoff between these two factors, we design the ratio of users’ probability and transmission overhead as the performance measure mechanism to evaluate the performance of UAV-assisted networks. Then, we formulate the objective for maximizing the performance of UAV-assisted networks as a UAV-assisted caching game. It is proved that the game is an exact potential game with the performance of UAV-assisted networks serving as the potential function. Next, we propose a log-linear caching algorithm (LCA) to achieve the Nash equilibrium (NE). Finally, related simulation results reﬂect the great performance of the proposed algorithm.


Introduction
During the past decades, UAVs have been popular in military applications due to its characteristics of high mobility and low cost [1][2][3][4][5], e.g., reasonable use of UAVs can reduce casualties of front-line soldiers.Inspired by the outstanding performance in military domain, recently UAVs have been widely utilized in civilian applications, including weather monitoring [6,7], forest fire monitoring [8,9] and first aid search and rescue [10,11], which have brought convenience to the public.Specifically, when UAVs are combined with wireless communication networks, users will experience better communication services due to the fact that traditional cellular networks have several following shortcomings [12]: i) the potential damage of communication facilities caused by natural disasters; ii) an increasingly unbearable communication burden resulted from growing smart devices and communication demands.Fortunately, it happens that the unique characteristics of UAVs can make up for these deficiencies.The characteristics of high mobility and low cost ensure that UAVs can quickly fly to the disaster area to provide services.Moreover, cache-enabled UAVs may increase the degree of the file reuse when hot files are requested repeatedly among users.Thus, especially in such an era of data explosion, the combination of UAVs and the cache is considered as a promising technology to deal with the challenges.
Recently, progress on related research has shown the validity and reliability of the aforementioned technology.In our existing work [13], the problem of selecting caching nodes was studied and a selection algorithm was proposed .However, it ignored the potential cooperation among caching nodes.Authors studied the problem of hybrid caching in D2D Networks in [14], which considered the cooperation but ignored the dynamics among nodes.In [15], a secure transmission scheme based on interference alignment was proposed, which made caching UAV provide service like SBSs.Idle SBSs replaced by UAVs would disrupt eavesdropping.However, the caching content sharing of UAVs is not considered.Moreover, authors studied the problem of the resource allocation in a cloud network with cache-enabled UAVs in [16], where users' content request distribution can be predicted based on the proposed algorithm.However, the research mentioned above, to a great extent, only focuses on the resource allocation and optimization of caching contents but ignores the optimization of UAVs' service locations.This problem is rarely considered in existing work to our best knowledge.Although, authors took the change of UAVs' location into account in [17], the potential cooperation among UAVs has been not considered.In our prior work, we investigated the problem of both caching contents and service locations in UAV-assisted emergency networks by multi-hop in [18].However, there still exists several shortcomings as follows.We considered the cooperation among UAVs by multi-hop based on the assumption that users within the coverage of UAVs were served perfectly.Nevertheless, in practical scenario, the influence of propagation groups, line-of-sight (LOS) and non-line-of-sight (NLOS), which may result in the fact that users within the coverage of UAVs communicate with UAVs successfully by probability, should be taken into account.Multi-hop service will result in more delay for ground users.Therefore, it is necessary and significant to improve the shortcomings mentioned above.
In this paper, we consider the joint optimization of UAVs' caching contents and locations with the goal of balancing the tradeoff between user's service probability and transmission overhead.In proposed scenario, users need to be served in the disaster area when the communication facilities are destroyed.UAVs with caching contents fly to the destination to provide communication services.Considering the cooperation among UAVs, the caching contents of neighbor UAVs can be shared by one hop.Thus, the optimal arrangement of UAVs' caching contents and service locations are the key points to solve.However, the problem is challenging.Firstly, the transmission overhead may be affected by both UAVs' caching contents and service locations, i.e., the source of the request file and the neighbor UAV sets.Secondly, when considering the locations of UAVs, the probability that the users can communicate with UAVs successfully will be influenced due to the propagation groups, i.e., LOS and NLOS, with corresponding probabilities of occurrence.
Motivated by these observations, we consider both the probability that the users can communicate with UAVs successfully and transmission overhead that the users spend getting request files.Therefore, we apply the ratio of users' service probability and transmission overhead as the performance indicators of UAVs' actions to balance the tradeoff.Meanwhile, we formulate the challenging problem as an UAV caching game applying game theory and prove the existence of Nash equilibrium (NE).In addition, a log-linear caching algorithm (LCA) can achieve the great performance.
The main contributions are as follows: • We first investigate the problem of joint optimization of UAVs' caching contents and service locations with the goal of balancing the tradeoff between user's service probability and transmission overhead under the consideration of the share of neighbor UAVs' caching contents.• The problem is modeled as an UAV caching game, which intends to find the optimal solution that maximizes UAVs' performance indicators, i.e., the ratio of user's service probability and transmission overhead.In addition, the proposed game is proved to be a potential game with at least one pure-strategy NE.Meanwhile, the optimal solution can be achieved by its optimal NE. • The log-linear caching algorithm is proposed to achieve the desirable solution in joint caching contents and service locations optimization problem.The simulation results show that the proposed algorithm can converge to a great NE solution and guarantee the great performance of UAV-assisted networks, which demonstrates the algorithm's validity and effectiveness.
The remainder of this paper is organized as follows.In section II, we introduce the system model and formulate the problem.In Section III, we propose the UAV-assisted caching game and apply log-linear caching algorithm to achieve the desirable solution.We present the simulation results and give an expansion and discussion in section IV.Finally, we draw a conclusion in Section V.

System model
We consider a UAV-assisted networks with N cache-enabled UAVs, i.e.,N = {1, 2, ..., N}, and M users, i.e., M = {1, 2, ..., M}.Denote the set of system files as f = f 1 , f 2 , ..., f | f | , where | f | denotes the number of system files.Suppose that all users request files from f according to a special file popularity distribution, which can be modeled by a Zipf distribution [19,20]: where f i is the i-th file requested by the user, | f | is the number of system files, γ is the parameter of Zipf distribution, P f i denotes the probability that the user requests f i .
Specially, we denote user m's requested files as Q m = q 1 , q 2 , ..., q |Q m | , where |Q m | is the number of files requested by user m and q i denotes the indicator function: if user m requested i-th file, 0, if user m do not request i-th file. ( An illustration of UAV-assisted communication networks is shown in Fig. 1.There are three small UAVs, a big central UAV and some users waiting to be served.As depicted in Fig. 1, the service process of the UAV consists of four parts: firstly, the small UAVs use some prior information provided by the central UAV and make their own caching contents decisions individually; Secondly, small UAVs fly to the service location with carried caching contents.Then, UAVs start to serve for users under the assumption that the caching contents can be shared among reachable neighboring UAVs by one hop.Next, if some users' demands are still not met, the service UAV will fly to some UAVs with caching contents that the dedicated user wants, to cooperate.Meanwhile, the following file requests of corresponding users will be rejected until the service UAV comes back.Finally, the service UAV will return to the central UAV to get files if other UAVs cannot provide such services.
Motivated by [21], we denote reachable neighboring UAVs of UAV n by one hop as J n , i.e., where D(n, i) denotes the distance between UAV n and i. D n represents the communication distance thresholds for UAVs.

Problem Formulation
Note that the performance of UAVs is affected by its caching contents as well as the service locations, i.e., where to obtain the files requested by users and the successful communication probability from UAVs to dedicated users due to the influence of propagation groups.Therefore, we need to balance the tradeoff between the transmission overhead and successful communication probability.Consider a n = (l n , c n ) as the action of UAV n.Note that l n reflects the service location of UAV n, which is actually a set of space coordinates, i.e., (x n , y n , z n ).Meanwhile, number of caching contents carried by UAV n, which is the cache space of UAVs.We define the transmission overhead by considering both caching contents and service location, as follows:  where |Q m | is the number of files requested by user m, p mk denotes the probability that user m requests file k, a = (a 1 , a 2 , ..., a N ) denotes the actions of all UAVs.C mk (a) is the transmission overhead under the consideration of different sources of files which is given by: where w 1 is the overhead per file per distance, u m and m represent the UAV and its dedicated user, where w 2 , w 3 and w 4 are different overheads per file per distance according to different source of files, u k is the UAV that caches file k, c u k means the set of files that UAV u k caches.The explanation of I(k, c u k , l u m , l u k , l u o ) can be classified into four parts.The first expression indicates that UAV u m caches the file that users require; the second expression indicates that UAV u m tends to obtain files from its neighbors; the third expression denotes the fact that UAV u m tends to obtain files from other small UAVs, which are defined as potential neighbors; the truth that UAV u m will fly back to the big central UAV to cache missing files is reflected in the last expression.Moreover, it should be noted that the value of each part in indicator function is related to the distance between UAV u m and the destination for file requested by users, i.e., d(l u m , l u k ) and d(l u m , l u o ).Note that the third and last expression indicate that UAV u m will leave the service location.Therefore, w 3 and w 4 involve the flight overhead per distance.
Although the UAV can obtain the files requested by users, the successful communication probability is still an important factor which may influence the service performance.Suppose that each communication link uses pre-given channel to transmit data.It can be completed by applying unicast transmission schemes [22] where each UAV serves corresponding users by different channels.Therefore, it means each communication link will not interfere with each other.Then, for an interference-free scenario, we define the successful communication probability as the probability that the received signal-to-noise ratio (SNR) by the received user is over a predefined communication threshold , as: where r th is the communication threshold, SNR is calculated by: where P n is the transmitting power of UAV n, g u m m is the channel power gain from UAV u m to user m, N 0 is the background noise.Different from terrestrial communication network, the transmission link can be classified into two propagation groups, i.e., LOS and NLOS.Motivated by [23], g u m m for LOS and NLOS links are respectively calculated by: where d u m m reflects the distance from UAV u m to user m, α denotes the path loss exponent, η denotes an additional attenuation factor caused by the NLOS link, h, i.e., h ∼ Γ (γ, ξ), represents the Nakagami fading.According to [23], both the service location and the elevation angle θ between the UAV and the user influence the probability of LOS link.Thus, the probability of causing LOS links can be given as follows: where a and b is the parameter related to the environment, θ denotes the elevation angle specified as follows: where z u m is the height of UAV u m .The probability of causing NLOS link is P(NLOS) = 1 − P (LOS).Thus, we express the channel power gain as follows: The good arrangements of UAVs' caching contents and service locations will achieve an appropriate trade-off between high successful communication probability and low transmission overhead.Therefore, we design the ratio of users' successful communication probability and transmission overhead as the performance indicators of UAVs' actions.We denote the performance function of UAV n as: where S n denotes the set of users that UAV n serves, r m is the transmission overhead of user m to obtain requested files specified in [23], P success,m is the successful communication probability from UAV n to user m.Thus, the performance of all UAVs can be measured by: where a = (a 1 , a 2 , ..., a N ) denotes the joint strategy profile of all UAVs.Therefore, the optimization object is to find the optimal strategy profile a * = (a * 1 , a * 2 , ..., a * N ) which can maximize the performance of all UAVs, i.e., a * = arg a max U (a) = arg a max ∑ n∈N u n (a 1 , a 2 , ..., a N ), where the condition c1 reflects the fact that UAVs' cache space does not exceed the number of system files, P threshold is the communication probability threshold, the condition c2 indicates that the solution must ensure users have appropriate communication probability to communicate.

Game Model
It is well known that game theory is efficient for solving the challenge of resource competition like the anti-jamming transmission problem [24,25], the opportunistic spectrum access problem [26], the distributed channel-slot selection optimizing problem [27], even the challenging UAV relay selection problem [28] and ultra-dense small cell network problem [29].Similarly, we propose a UAV-assisted caching game in this paper from a game-theoretic perspective.The proposed game considers not only the UAVs' locations, but also the caching contents.Moreover, UAVs will cooperate with their neighbors and share the caching contents, which reflects the cooperation.Therefore, when one UAV selects its action, it will consider the influence that its action brings to its neighbors.
Let G = {N , A n , J n , U n } denotes the UAV-assisted caching game, where N denotes the set of players (UAVs), A n is the strategy space of UAV n, J n is the neighbors of UAV n, U n indicates the utility function of player n of (UAV n).
Considering the influence of marginal contribution of UAVs to the system service, we design the utility function of UAVs as follows: where a n indicates the current action of player n (UAV n), a J n is the joint action profile of UAV n's neighbors, a −n = (a 1 , ..., a n−1 , a n+1 , ..., a N ) denotes the joint action profile of UAVs except UAV n, a n = 0 reflects the fact that the current UAV n takes no action, R(a n , a −n ) = ∑ n∈N u n (a 1 , a 2 , ..., a N ) is the performance evaluation function of all UAVs, which is the same as (14).The utility function shows the individual contribution of each UAV to the UAV-assisted network performance, which is influenced by caching contents and service locations of UAVs.Therefore, when UAVs take proper arrangement of caching contents and service locations, each individual contribution will increase.Therefore, the optimization object of the UAV-assisted caching game is given by: G : max s.t. to c1 and c2.

Analysis of NE
The Nash equilibrium (NE) is the stable solution in the game theory.The existence of NE will be analyzed in the part.According to [14], we first give the definitions as follows: Definition 1 (Nash Equilibrium): An action profile a * = (a Therefore, a game is an exact potential game when the change in individual utility function is the same as the change in the potential function.
According to the definitions [30], the following theorem is given to present the properties of the proposed game.
Theorem 1: G is an exact potential game that has at least one pure strategy NE.The optimal pure strategy NE of G is the optimal solution of proposed optimization problem.
Proof.Inspired by [31], we firstly denote the potential function as follows: where R(a n , a −n ) = ∑ n∈N u n (a 1 , a 2 , ..., a N ) is the performance evaluation function of all UAVs, which is the same as (14).
The potential function indicates the performance of all UAVs.When UAV n changes its decision action from a n to ān , then the change in utility function of UAV n caused by this unilateral change is given by: Note that a n = 0 and ān = 0 all indicate the truth that UAV takes no action in service, which means that this UAV is broken and can not provide service.Therefore, the performance of UAV-assisted networks is actually the same in these two conditions, i.e., According to (21) and (22), we can get the following equation: From equation (23), we can draw the conclusion that the change in the utility function is exactly the same as the change in the potential function.Thus, according to the definition, the fact that our proposed game is an exact potential game is proved.Moreover, the potential game has the most important two properties.
• Potential game has one pure strategy NE at least.• Local or global maxima of potential function constitutes a pure strategy NE.
Therefore, the UAV-assisted caching game has one NE at least.Moreover, the NE is a stable solution to the potential function, which equals to the performance of all UAVs.It means that the optimal NE can make the performance of UAV-assisted networks largest.Thus, Theorem 1 is proved.

Log-linear Caching Algorithm
After proposing the UAV-assisted caching game, it is desirable to find the NE point mentioned above.Therefore, the log-linear caching algorithm is proposed to optimize the performance of UAV-assisted networks motivated by [14].
Different from greedy algorithms, the proposed algorithm guarantees that each UAV will not always take its best action.The mechanism avoids the dilemma of UAV-assisted networks falling into the local optimum.It will eventually result in a better NE solution by discarding a temporary good action.The detailed running mechanism of the proposed algorithm is as follows: Firstly, related parameters and actions of UAVs are initialized; Secondly, one UAV is selected randomly to enter an exploratory stage, i.e., the selected UAV randomly chooses a new action with equal probability; Finally, the selected UAV decides whether to explore the new action or to keep the original action according to its updating rules.More details are described in Algorithm 1.
Theorem 2: With a sufficiently large learning parameter, the log-linear caching algorithm converges to a NE point of the UAV-assisted caching game and maximizes the performance of UAV-assisted networks locally or globally.
Proof.According to Theorem 4 [32], the unique stationary distribution of any action profile can be determined.Based on the methodology provided in [21], it can be proved that when the learning parameter is large enough, the log-linear caching algorithm can asymptotically obtain the maximum of potential function.As is specified in (20), the potential function represents the performance of all UAVs.Therefore, the proposed algorithm can maximize the performance of UAV-assisted network.The similar proof process has been completed and more details can be read in [32].Thus, Theorem 2 is proved.(2) Moreover, set k=0 as the round count and k max as the maximum round.

Repeat iterations::
(1) Calculate U n by taking the action a n .
(2) Generate a n randomly from its strategy space (3) Calculate U n by taking the action a n and keep other UAVs' actions.(4) Update strategy: where Y = exp (βU n ) + exp (βU n ) , and β is the learning parameter.Meanwhile, all other UAVs keep their actions, and a −n (k + 1) = a −n (k).dbm respectively.The path loss exponent is α = 3.The Nakagami fading is h ∼ Γ (2, 0.5).Inspired by [23], the parameter related to the environment a and b is a=11.95 and b=0.136.

Convergence Behavior
As shown in Fig. 2, the performance of UAV-assisted networks finally converges to a stable state.Here we consider that the number of system files is | f | = 6, and there are N = 8 UAVs to serve the users.Moreover, the cache space of UAVs is |c n | = 3.At the end of iteration, the performance of UAV-assisted network will finally converge.Also, the final performance is near to the best NE, which is the best one of 1000 independent trials based on the proposed algorithm.The simulation result validates its convergence and effectiveness of the proposed algorithm.

Performance of UAV-assisted Networks versus the Number of System Files
Considering the change of the number of system files, we compare the performance of UAV-assisted networks applying four algorithms, i.e., the random selection, best NE, worst NE and the proposed algorithm.We ran the random selection algorithm 1000 times and choose the average value as the final result of the random selection algorithm.Similarly, we carried out 1000  independent trials using the proposed log-linear caching algorithm.Actually, each convergent value is a NE solution.Then the average of values is defined as the final value.In addition, we choose the best one and worst one of the values as the best NE and the worst NE respectively.Fig. 3 depicts the performance of UAV-assisted networks when varying the number of system files from 4 to 8. Several important observations can be seen in this figure:1) the performance of UAV-assisted networks decreases as the number of system files increases.2) compared with the random algorithm, the log-linear caching algorithm brings the better performance of UAV-assisted networks.3) due to the limited UAVs' processing capacity, when the number of system files increases, users' requests become more complex, which leads to a decline in performance.

Performance of UAV-assisted Networks versus the Number of Cache Space
We assess the performance of UAV-assisted networks when changing the number of cache space from 3 to 6 files.Fig. 4 reflects the final results applyinig four approaches.The simulation process is similar to Fig. 3. From Fig. 4, the simulation curves reflect the fact that: 1) the performance of UAV-assisted networks increases as the cache space of UAVs increases.The reason is that greater cache space brings high processing capacity of UAVs, which leads to an increase in performance.2) the log-linear caching algorithm is close to the best NE and outperforms the random selection algorithm which validate the effectiveness of the proposed algorithm.Considering the number of UAVs varying from 5 to 8, we compare the performance of UAV-assisted networks.Fig. 4 indicates the influence of the number of UAVs on the performance of UAV-assisted networks.The fact is reflected from the simulation curves: in general, as the number of UAVs increases, the performance of UAV-assisted network increases.The reasonable explanation is that more UAV yields larger successful communication probability.Also, the final performance of proposed algorithm is close to the best NE with respect to different numbers of UAVs which again validates the proposed algorithm.

Conclusion
We investigated the problem of the optimal arrangement for both UAV's caching contents and service locations in UAV-assisted networks.Due to the fact that UAV's actions might influence the communication probability caused by line-of-sight (LOS) and non-line-of-sight (NLOS) link as well as the transmission overhead, we designed the ratio of users' communication probability and transmission overhead as the performance measure mechanism to assess the performance of UAV-assisted networks.The objective for maximizing the performance of UAV-assisted networks was modeled as an UAV-assisted caching game.Moreover, we proved that the game is an exact potential game and analyzed the existence of Nash equilibrium (NE).Meanwhile, we also proposed the log-linear caching algorithm (LCA) to achieve the NE point.The simulation results validated the effectiveness of the proposed game and reflected the great performance of the proposed algorithm.

Figure 1 .
Figure 1.An illustration of the emergency communications network.((a).The small UAVs obtain the caching contents by the given decision from the central UAV and fly to the destination with cached contents.(b).UAVs cooperate with reachable neighboring UAVs by one hop.(c).UAV will fly to some UAVs with caching contents that the dedicated user wants (d).UAV will return to the central UAV to get files if other UAVs cannot provide such services.)

Algorithm 1 :
Log-linear caching algorithm (LCA) 1. Initialization(1) Initialize randomly users' locations, file popularity and file requests Q m , m ∈ M.Initialize randomly l n and c n ,n ∈ N satisfying to c1 and c2.

( 5 ) 3 . End rounds 4 .
until k ≥ k max or Pr [a n (k + 1)] > 0.95, n ∈ N End iterations Output : a * = (a * 1 , a * 2 , ..., a * N ) Simulation and Numerical Results and Discussion 4.1.Simulation Scenario Considering a square region of 200 × 200 m 2 , there are M = 20 users located randomly waiting to be served.Meanwhile, the communication distance threshold of UAVs is D n = 100 m and the learning parameter of the proposed algorithm is β = 40 + k/25, where k is the number of the current iterations.The different overheads per file per distance are w 1 = 0.1, w 2 = 0.3, w 3 = 2 and w 4 = 5 respectively.Moreover, the communication threshold is r th = 55 dB.The communication probability threshold is P threshold = 0.5.The transmitting power of UAVs and background noise are P n = 0.1 w and N 0 = −100

Figure 3 .
Figure 3.The performance of UAV-assisted networks versus the number of system files.(N= 8 and |c n | = 3.)

Figure 4 .Figure 5 .
Figure 4.The performance of UAV-assisted networks versus cache space of UAVs.(| f | = 6 and N = 8.) the distance from UAV u m to user m, u k and u o denote the UAV that caches file k and the central UAV respectively, a = (a 1 , a 2 , ..., a n ) is the strategy profile of all UAVs, I(k, c u k , l u m , l u k , l u o ) is the indicator function as follows:

Table 1 .
key variables used in this paper.