A Game Theoretic Approach for D2D Assisted Uncoded Caching in IoT Networks

Abstract

Content caching and exchange through device-to-device (D2D) communications can offload data from the centralized base station and improve the quality of users’ experience. However, existing studies often overlook the selfish nature of user equipment (UE) and the heterogeneity of content preferences, which limits their practical applicability. In this paper, we propose a novel incentive-driven uncoded caching framework modeled as a Stackelberg game between a base station (BS) and cache-enabled UEs. The BS acts as the leader by determining the unit incentive reward, while UEs jointly optimize their caching strategies as followers. The particular challenge in our formulation is that the uncoded caching decisions make the UEs’ total utility maximization problem into a non-convex integer programming problem. To address this, we map the UEs’ total utility maximization problem into a potential sub-game and design a potential game-based distributed caching (PGDC) algorithm that guarantees convergence to the optimal joint caching strategy. Building on this, we further develop a dynamic iterative algorithm to derive the Stackelberg equilibrium by jointly optimizing the BS’s cost and the total utility of UEs. The simulation results confirm the existence of the Stackelberg Equilibrium and demonstrate that the proposed PGDC algorithm significantly outperforms benchmark caching schemes.

1. Introduction

With the tremendous advances of Internet-of-Things (IoT) communication technologies and the ever-increasing use of smart IoT devices, wireless data traffic has exhibited explosive growth in the past few years, which puts a heavy burden on the existing IoT networks. To cope with this issue, academia and industry are beginning to explore effective approaches, such as, massive MIMO [1], coordinated multipoint transmission [2], and reconfigurable intelligent surfaces [3], to improve the IoT network transmission capacity. In addition, a lot of popular content in the IoT environment is frequently requested by various users, incurring repetitive transmissions and large-scale congestion during peak traffic time [4]. Device-to-device (D2D) communications with caching techniques can be considered as a promising solution [5] to relieve the IoT network traffic burden and improve users’ experience (QoE).

With D2D-based caching, user equipment (UE) can cache contents with individual preferences and share content with their neighbor UEs to reduce the content retrieval delay. D2D-based caching can also help mitigate the traffic load on the base station (BS) by enabling direct content sharing between UEs. UE’s willingness to share content is a critical issue in D2D networks. This is because helping the BS with caching and sharing content will inevitably incur additional costs, such as content maintenance costs and transmission delay costs. Thus, it is necessary to develop an incentive mechanism to motivate UEs to participate in content sharing via D2D communications. Recent studies have leveraged game theory to investigate the incentive mechanism and caching optimization problem in the IoT networks [6,7,8,9]. In [6,7], the authors investigate the uncoded caching optimization problem, where the content is either fully cached or not cached at all. Thus, the caching decision is the integer variable. In order to derive the closed-form Stackelberg Equilibrium (SE), the utility functions in [6,7] are modeled as the quadratic function or the logarithm function. However, the modeled utility functions are highly dependent on preset parameters and may not accurately reflect users’ true benefits. Hence, the question of how to solve the non-convex integer caching optimization in game-theoretic framework is intractable and imperative. Additionally, we find that the content caching and request probabilities in the game models of the existing works [8,9] are usually assumed to be homogeneous, i.e., all UEs have the same request probability for one particular content, which is not practical in the real world, where each UE normally has different interest and preference for network content.

In this paper, we consider a D2D-assisted IoT caching network including a base station and a set of cache-enabled UEs with the content request probability following a heterogeneous distribution. We formulate the economic interactions between BS and UEs as a one-leader–one-follower Stackelberg game in which the BS acts as the leader and all UEs jointly act as the follower. In the proposed game, the BS initially determines the unit incentive reward submitting to the cache-enabled UEs. Then, based on the given reward, UEs utilize the local cooperative caching mechanism to proactively cache appropriate content, with the aim of maximizing their total utility. Meanwhile, BS aims to keep the incentive reward as small as possible to minimize its own cost. In order to find the SE of the proposed Stackelberg game, we first formulate the total utility optimization problem (i.e., the follower’s problem), which is constrained by the binary caching decision variables of all UEs. Thus, the follower’s problem is a non-convex integer optimization problem and it is difficult to find the closed-form solution. Fortunately, we can map the total utility optimization into a sub-game and prove that this sub-game with UE’s local cooperative utility function is an exact potential game, in which at least one Nash Equilibrium (NE) exists. Building on this, we propose a potential game-based distributed caching (PGDC) algorithm to acquire the optimal joint caching placement strategy of all UEs. Although the proposed PGDC algorithm adopts a Gibbs-like update rule for caching strategy selection, it is specifically tailored for the uncoded caching problem in D2D-assisted IoT networks. The novelty lies in the integration of context-specific utility function design, potential game mapping, and distributed learning to address a challenging non-convex integer optimization problem. Then, the achieved optimal joint caching strategy can be regarded as the follower’s best response to the proposed Stackelberg game, we design a dynamic iterative algorithm based on the best response to solve the BS’s cost optimization problem (i.e., the leader’s problem) and identify the optimal incentive reward. Thus, the Stackelberg Equilibrium of the proposed game is derived.

The contributions of this paper are summarized as follows:

• Considering the content preferences and share willingness of UEs, we propose an uncoded caching incentive mechanism that motivates UEs to share cached contents, assisting the BS in alleviating content delivery burdens. The profit interactions between BS and UEs are modeled as a Stackelberg game to jointly optimize the cost of BS and UEs’ total utility.
• In order to derive the SE of the proposed game, we insightfully formulate the non-convex total utility optimization problem for all UEs as a potential sub-game, and prove this sub-game involves at least one NE. We further demonstrate that the optimal joint caching strategy must be a NE of this sub-game. Subsequently, we design a PGDC algorithm to find the optimal solution to the sub-game for a given incentive reward, and theoretically analyze the convergence of the proposed PGDC algorithm.
• Leveraging the optimal joint caching placement strategy as the follower’s best response to the Stackelberg game, we design a dynamic iterative algorithm to efficiently solve the BS’s cost optimization problem. This algorithm further enables us to determine the SE of the proposed game, which allows BS to achieve its optimal cost and UEs to obtain their maximum total utility, respectively.
• Through extensive simulation experiments, we evaluate the convergence of the proposed PGDC algorithm and validate the existence of Stackelberg Equilibrium. We compare the performance of several benchmark caching strategies with our proposed caching solution. The simulation results demonstrate the significant superiority of the PGDC algorithm.

2. Related Work

Caching in IoT networks can significantly improve the network performance, which has attracted many researchers’ attention to investigating the optimal caching policies. In [10], the authors studied the impact of bursty traffic and random availability of caching helpers on the weighted throughput and the average delay in caching helper networks by adopting the most popular caching (MPC) policy. However, MPC may not be the optimal policy in the heterogeneous content request probability model. Zhang et al. [11] proposed a cooperative caching architecture of the unmanned aerial vehicle and user terminals, and solved the caching placement by the many-to-many swap-matching algorithm. In [12], Cao et al. took account of the user’s mobility when creating their caching strategy. They developed a mobility-aware joint routing and caching strategy and designed a greedy caching algorithm to minimize the cost of the content delivery. In general, a greedy caching policy cannot acquire the global optimal solution. In [13], the authors investigated the mobility-aware content caching and updating optimization problem based on a user-interest prediction model in the IoT networks.

However, the aforementioned research ignored the selfish nature and unwillingness to share exhibited by UEs in real life. Thus, it is essential to design an incentive mechanism that stimulates UEs to be willing to share content and that benefits the two-side profits of BS and UEs. In [5], the authors proposed a green incentive mechanism to encourage mobile D2D users to cache and share content with the aim of energy saving. In [14], the authors utilized the Nash bargaining solution approach to design a novel incentive mechanism in order to encourage users to cache and share content. In [15], the authors proposed a monetary incentive-based mechanism to boost user participation, thereby facilitating successful D2D communication and achieving the high cache hit probability. Furthermore, combined with incentive mechanism, game theory [16] plays a vital role in the mutually influential economic strategies between BS and UEs in IoT caching networks, which is also an useful mathematical model to design the incentive mechanism and analyze the bilateral conflicts of economic issues.

There are a large number of works focusing on the caching issues from the game theoretic perspective. Su et al. [17] investigated the interaction between UEs and social groups in mobile social caching networks using the Stackelberg game and obtained the optimal caching strategy by the backward induction method. A joint video rental pricing and caching optimization problem is studied in [18] by considering the non-cooperative and cooperative caching of BSs under the framework of Stackelberg game. Jiang et al. [19] proposed a joint mobile edge caching and peer content-sharing framework, and derived the the game equilibrium under a generic usage-based pricing scheme. Moreover, we find that studies [17,18,19] all adopt the homogeneous content request probability, which is impractical in real life. In [20], Zheng et al. solved the large-scale uncoded caching problem in mobile edge networks using the game-based alternating direction method of multipliers. Jiang et al. [21] leveraged the crowdsourcing technique to study the edge caching system, where content providers shared a proportion of profit with UEs as an incentive reward and formulated a two-stage Stackelberg game to optimize the revenue of content providers and UEs. In [22], a blockchain incentive scheme was proposed and the Stackelberg game models using different pricing methods were formulated so as to divide uniform pricing and discriminatory pricing. In [23], in order to develop the multi-bitrate video caching strategy, the authors transformed the caching problem into the knapsack problem and applied the dominant strategy from game theory.

Unlike existing works, we propose an uncoded caching incentive mechanism for UEs with consideration of UEs’ heterogeneous content preference and share willingness. We formulate the the economic interactions between BS and UEs as a Stackelberg game to jointly optimize the cost of BS and the total utility of UEs. Since the UEs’ total utility optimization problem is a non-convex integer programming problem, we map this thorny problem into a potential sub-game, which is not investigated in most of the existing uncoded caching games.

3. System Models

3.1. Network Model

As shown in Figure 1, we consider a typical D2D-assisted IoT caching network comprising a base station (BS) and a set of cache-enabled user equipments (UEs). For clarity, let BS be indexed 0 and let $\mathcal{N}=\{1,2,\dots ,N\}$ denote the set of UEs. Each UE $n\in \mathcal{N}$ has limited caching capacity c. We assume that each UE is able to request and share content directly with its neighbor UEs via D2D links. Let $J_n\subseteq \mathcal{N}$ denote the set of neighbors of UE n, i.e.,

$$J_n=\{j\in \mathcal{N}|l_{n,j}\le r_n\},$$

(1)

where $l_{n,j}$ is the distance between UE j and UE n, $r_n$ denotes the transmission radius of UE n.

Figure 1. A typical D2D-assisted IoT caching network model.

Figure 1. A typical D2D-assisted IoT caching network model.

3.2. Content Request and Cache Placement Model

There are F contents in the IoT network, which are indexed by the set $\mathcal{F}=\{1,2,\dots ,F\}$. We assume each type of content has the same normalized size and is denoted by s. A number of prior studies adopted the homogeneous content request probability [17,18,19], which means all users in the networks have the same content request probability. In fact, UEs have different preferences about all kinds of content. Hence, we adopt a heterogeneous content request probability. That is, the request probability for the same content differs among the various UEs. Let ${\left[f\right]}_n$, $f\in \mathcal{F}$ denote the rank of content f according to UE n’s preference. For example, if ${\left[f\right]}_n=1$, that means content f is UE n’s favourite content. If ${\left[f\right]}_n=F$, that means content f is UE n’s least favourite content. We assume that each UE requests the content independently and let $P_{nf}$ denote the request probability of content f by UE n. We consider the assumption from [24], the content request probability is modeled by the Zipf distribution (i.e., the content popularity distribution) (Since we consider individual preference, here, the content popularity distribution is only for each UE, which is the same for the individual content request distribution, and not for all UEs throughout the whole network.). The probability of requesting content f by UE n is expressed as

$$P_{nf}={\displaystyle \frac{{\left[f\right]}_n^{-\alpha }}{{\sum }_{f=1}^{F}1/{\left[f\right]}_n^{\alpha }}},\forall n\in \mathcal{N},\forall f\in \mathcal{F},$$

(2)

where $\alpha >0$ determines the skewness of the content popularity distribution for each UE. When Zipf exponent $\alpha$ is large, it indicates that the request of each UE focuses on the first few popular types of content, which means the content with a small ranked index has a large request probability. In addition, we assume that the content request popularity of each UE remains stable and unchanged during the long period (e.g., a week).

During the off-peak period, each UE decides whether to cache the entire content (i.e., uncoded caching) according to their preference. We define $x_{nf},\forall n\in \mathcal{N},\forall f\in \mathcal{F}$ as the binary decision variable to determine whether the content f is cached by UE n or not. If $x_{nf}=1$, the content f is cached by UE n; otherwise, $x_{nf}=0$. Due to the limited caching storage capacity, the total number of contents cached by UE n should be satisfied by the following constraint:

$$\sum _{f=1}^F{x}_{nf}\le c,\forall n\in \mathcal{N}.$$

(3)

3.3. Communication Model

In order to avoid transmission interference among UEs, we assume UEs can request and transmit content using orthogonal frequency division multiple access (OFDMA) [25]. The transmission rate of UE n from its neighbor j can be computed as

$$T_{n,j}=B_{n,j}log(1+{\displaystyle \frac{P_j\left|h_{n,j}\right|l_{n,j}^{-\gamma }}{{\sigma }^2}}),$$

(4)

where $B_{n,j}$ denotes the communication bandwidth between UE n and its neighbor j, $P_j$ denotes the transmission power of UE j, $|h_{n,j}|$ denotes the channel gain between UE n and UE j, $\gamma$ denotes the path loss exponent of the transmission links, and ${\sigma }^2$ denotes the power of the additive white Gaussian noise. Similarly, the transmission rate of UE n from BS can be computed as

$$T_{n,0}=B_{n,0}log(1+{\displaystyle \frac{P_0\left|h_{n,0}\right|l_{n,0}^{-\gamma }}{{\sigma }^2}}),$$

(5)

where $B_{n,0}$ denotes the communication bandwidth between UE n and BS, $P_0$ denotes the transmission power of BS, and $|h_{n,0}|$ denotes the channel gain between UE n and BS.

The content transmission delay between UE n and j and between UE n and BS can be calculated by $D_{n,j}=s/T_{n,j}$ and $D_{n,0}=s/T_{n,0}$, respectively. The main notations and descriptions are listed in

Table 1. Notations and descriptions.

Notation	Description
$\mathcal{N}$	the set of UEs
$J_n$	the set of neighbors of UE n
$\mathcal{F}$	the set of contents
${\left[f\right]}_n$	the rank of content f according to UE n’s preference
$P_{nf}$	the request probability of content f of UE n
$x_{nf}$	caching decision of UE n
c	caching capacity
${\sigma }^2$	noise power
$D_{n,j}$	transmission delay between UE n and j
$D_{n,0}$	transmission delay between UE n and BS
r	unit incentive reward
${\left[n\right]}_j$	the delay rank of UE n to its neighbor UE j
${\left(k\right)}_n$	index of the neighbor UE with the kth lowest delay to UE n
${\eta }_0$	unit transmission delay cost between UEs
${\eta }_1$	unit transmission delay cost between UE and BS
${\eta }_2$	unit transmission energy cost between UE and BS
$\beta$	learning parameter

.

4. Stackelberg Game Formulation

In this section, we first design the utility function of UEs and the cost function of BS, respectively, based on the incentive mechanism. Due to the high cost of a single UE directly requesting content from BS, the proposed local cooperative caching mechanism among UEs enables the sharing of cached content between UEs and their neighbors. This not only provides the incentive reward for content sharers, but also reduces the cost of content requesters requesting content from the base station. Hence, the utility of UEs under the local cooperative caching mechanism is significantly higher than that under the non-cooperative caching mechanism. Then, we formulate the benefit interactions between UEs and BS as a single-leader–one-follower Stackelberg game to jointly optimize the incentive reward and caching placement strategy. In this game model, the leader (BS) has the priority to take action first (i.e., determining a unit reward), then the follower (all UEs) makes joint caching decisions to maximize their total utility according to the unit reward given by BS. Finally, the Stackelberg game can be decomposed into two sub-problems: the total utility maximization problem for all UEs and the cost minimization problem of BS.

4.1. Utility of the UE

The utility of UE n mainly consists of two parts: the incentive rewards paid by the base station and the delay costs incurred by requesting content. Firstly, we establish a revenue model based on the obtained incentives. Since $J_n$ denotes the neighbour set of UE n, we sort $j\in J_n$ in an increasing order of the transmission delay to UE n and let ${\left(k\right)}_n$ denote the index of the neighbor UE with the kth lowest transmission delay to UE n. We have ${\left(0\right)}_n=n$, which means the 0th lowest transmission delay to UE n is itself. Moreover, we define ${\left[n\right]}_j$ as the transmission delay rank of UE n to its neighbor UE j. Note that ${\left[j\right]}_j=0$; that is, UE j requesting the content from its own cache has the minimum delay. When UE n’s neighbor j submits a content request, UE j will first check its own local cache. If the requested content can not be found in UE j’s local cache, then UE j will submit a request to its neighbor UEs according to the sorted delay ${\left(k\right)}_j$. Let r denote the unit reward paid to UE n by the BS when the UE n caches the requested content of its neighbor j and shares the cached content with neighbor requester j. Note that the incentive reward could be of any form, such as direct monetary incentives or virtual currency, and can be implemented by smart contracts. Thus, the total incentive revenue $R\left(n\right)$ of the UE n from the incentive reward can be formulated as

$$R\left(n\right)=\sum _f\sum _{j\in J_n}{P}_{jf}\prod _{k=0}^{{\left[n\right]}_j-1}(1-x_{{\left(k\right)}_{j}f})x_{nf}r.$$

(6)

In Equation (6), if requester j’s neighbor UEs who have a lower transmission delay than UE n do not cache j’s requested content (i.e., $x_{{\left(k\right)}_{j}f}=0,\forall k<{\left[n\right]}_j$) and UE n caches the requested one (i.e., $x_{nf}=1$), then we have ${\prod }_{k=0}^{{\left[n\right]}_j-1}(1-x_{{\left(k\right)}_{j}f})x_{nf}=1$. Only in this case can UE n obtain the incentive reward from BS. From Equation (6), we find that the revenue of UE n not only depends on its own caching decision $x_{nf}$ but that it also depends on requester j’s neighbors’ caching decisions.

Next, we consider the cost of UE n. We model the operation cost of UE n as the content transmission delay cost, which is similar to the assumption in [26]. UE n can acquire the requested content in three ways according to the local cooperative caching mechanism. Hence, the transmission delay has three different conditions and we define the transmission delay between UE n and its kth neighbor according to the sorted delay as $D_{n,{\left(k\right)}_n}$. Firstly, if the requested content in its local cache, then we assume that UE n satisfies its request with no delay cost, i.e., $D_{n,{\left(0\right)}_n}=0$. Secondly, if its local cache does not have the the requested content but its neighbors have a copy, then UE n will request the content from its neighbors based on the sorted delay. Lastly, neither UE n’s local cache nor all its neighbors cache the copy; UE n has to request the content from BS. The delay between UE n and BS is denoted by $D_{n,0}$. In particular, to emphasize that the incentive mechanism facilitates cooperation among UEs, we assume that the transmission distance between UEs is much shorter than that to the BS. Consequently, the delay incurred when UEs request contents from the BS via congested backhaul links is typically larger than the transmission delay between UEs. Thus, we state that $D_{n,{\left(k\right)}_n}<D_{n,0},0\le k\le \left|J_n\right|$. Nevertheless, in advanced 5G/6G networks with extremely optimized base stations and high-capacity backhaul, the UE–BS transmission delay may be smaller than that of certain D2D links. Incorporating such cases into the model is an interesting development and will be discussed in our future work. The total cost $C\left(n\right)$ of the UE n can be formulated as

$$C\left(n\right)=\sum _f{P}_{nf}[{\eta }_0\sum _{j=1}^{|J_n|}\prod _{k=0}^{j-1}(1-x_{{\left(k\right)}_{n}f})x_{{\left(j\right)}_{n}f}{D}_{n,{\left(j\right)}_n}+{\eta }_1\prod _{k=0}^{|J_n|}(1-x_{{\left(k\right)}_{n}f})D_{n,0}],$$

(7)

where ${\eta }_0$ is the unit transmission delay cost between UEs and ${\eta }_1$ is the unit transmission delay cost between UE and BS.

In Equation (7), if UE n finds the requested content f in its local cache, i.e., $x_{{\left(0\right)}_{n}f}=1$, then two items in the square brackets are both equal to zero, which means UE n can obtain the requested content from its local cache without any delay cost. Otherwise, the transmission delay cost of UE n becomes nonzero. If the requested content can be found in the $j^{\prime }$th neighbor of UE n according to the sorted delay, i.e., $x_{{\left(j^{\prime }\right)}_{n}f}=1$ and $x_{{\left(k\right)}_{n}f}=0,\forall k<j^{\prime }$, then we have ${\displaystyle \prod _{k=0}^{j^{\prime }-1}}(1-x_{{\left(k\right)}_{n}f})x_{{\left(j^{\prime }\right)}_{n}f}=1$, which means the first term is equal to 1 while the second term in the square brackets is equal to zero. If no requested content can be found in either UE n’s local cache or UE n’ neighbours, i.e., $x_{{\left(k\right)}_{n}f}=0,\forall 0\le k\le \left|J_n\right|$, then ${\displaystyle \prod _{k=0}^{|J_n|}}(1-x_{{\left(k\right)}_{n}f})=1$, which means the first term is equal to zero while the second term in the square brackets is equal to 1.

According to the above analysis, we consider that the caching action strategy set of UE n is denoted by $x_n=x_{n1}\otimes \dots \otimes x_{nf}\dots \otimes x_{nF}$, $x_{nf}\in \{0,1\}$ for $f\in \mathcal{F}$, where symbol ⊗ is a Cartesian product. Let $x_{J_n},\forall n\in \mathcal{N}$ represent the caching action strategy set of n’s neighbor UEs. Then, the utility function $U_n(x_n,x_{J_n})$ for UE n can be defined as the difference between the incentive revenue received from BS and the UE’s content transmission delay cost. Hence, the utility $U_n(x_n,x_{J_n})$ of the UE n can be expressed as

$$\begin{array}{cc}\hfill U_n(x_n,x_{J_n})=& R\left(n\right)-C\left(n\right)\hfill \\ \hfill =& \sum _f\sum _{j\in J_n}{P}_{jf}r\prod _{k=0}^{{\left[n\right]}_j-1}(1-x_{{\left(k\right)}_{j}f})x_{nf}-\sum _f{P}_{nf}[{\eta }_0\sum _{j=1}^{|J_n|}\prod _{k=0}^{j-1}(1-x_{{\left(k\right)}_{n}f})x_{{\left(j\right)}_{n}f}{D}_{n,{\left(j\right)}_n}+\hfill \\ \hfill & {\eta }_1\prod _{k=0}^{|J_n|}(1-x_{{\left(k\right)}_{n}f})D_{n,0}].\hfill \end{array}$$

(8)

4.2. Cost of the BS

We consider that the total cost of BS is composed of two parts: the incentive rewards paid to caching UEs and the content transmission energy cost. In order to reduce the transmission burden of BS and lower the delay of content requests at UEs, BS offers incentive rewards to caching UEs to promote content transmission and sharing. Let r denote the unit incentive reward paid to UEs who have shared cached contents to their neighbors. Due to the limited caching capacity of UEs, if the requested content cannot be found in the requester UE n’s own cache or its neighbors’ caches, the requester UE n has to request the content from the BS, which incurs BS transmission energy cost. We assume that the transmission energy between BS and the requester UE n is $E_{n,0}=P_0·D_{n,0}$. Therefore, the total cost $C_{BS}$ of BS can be formulated as

$$C_{BS}=\sum _n\sum _f\left(P_{nf}{x}_{{\left(0\right)}_{n}f}r\sum _{j\in J_n}{P}_{jf}\prod _{k=0}^{{\left[n\right]}_j-1}(1-x_{{\left(k\right)}_{j}f})\right)+{\eta }_2\sum _n\sum _f{P}_{nf}\prod _{k=0}^{|J_n|}(1-x_{{\left(k\right)}_{n}f})E_{n,0},$$

(9)

where ${\eta }_2$ is the unit transmission energy cost between UE and BS. In Equation (9), if the requested content f is cached by UE n, i.e., $x_{{\left(0\right)}_{n}f}=1$, then we can easily find that the second term is equal to zero. In this case, when UE n’s neighbor j requests a desired content f according to the sorted delay ${\left(k\right)}_j$, we have two sub-cases. One is ${\prod }_{k=0}^{{\left[n\right]}_j-1}(1-x_{{\left(k\right)}_{j}f})=1$, which means UE j’s first ${\left[n\right]}_j-1$ neighbors all fail to cache the content f and UE n has the copy. Then, UE n will deliver the content to UE j and UE n will receive the incentive reward paid by BS, which incurs the incentive cost of BS. The other sub-case involves the existence of $k^{\prime }\le {\left[n\right]}_j-1$, such that $x_{{\left(k^{\prime }\right)}_{j}f}=1$, which means one of UE j’s first ${\left[n\right]}_j-1$ neighbors has cached the content f and UE j can obtain the requested content from UE $k^{\prime }$. The BS neither pays the incentive reward to the UE n nor incurs the transmission energy cost. If UE n does not cache the content f, i.e., $x_{{\left(0\right)}_{n}f}=0$, UE n wants to request the content f. In this case, we also have two sub-cases. The first sub-case is that one of UE n’s neighbors has cached the copy $x_{{\left(k\right)}_{n}f}=1,\exists 1\le k\le \left|J_n\right|$, and the BS neither pays the incentive reward to the UE n nor incurs the transmission energy cost. The second sub-case is that no requested content can be found from UE n’s neighbors, i.e., $x_{{\left(k\right)}_{n}f}=0,\forall 1\le k\le \left|J_n\right|$; thus, UE n can only acquire the requested content from BS. Thus, the first term in Equation (9) equals zero, and BS will generate the content transmission energy cost.

4.3. Stackelberg Game

In the proposed Stackelberg game, BS first determines the unit reward as the incentive for UEs. If the announced reward by the BS is higher, UEs are quite willing to cache the requested content of their neighbors and share the content via D2D communications. However, BS will accordingly incur a higher incentive reward cost. If the announced reward is lower, UEs may prefer caching the content according to their own preferences and may not be willing to share the content, which will incur a higher transmission energy cost for BS. Hence, BS needs to carefully determine the optimal unit reward to minimize its own cost. The cost optimization problem of BS can be formulated as the following Problem 1.

Problem 1.

$$\begin{array}{cccc}\hfill \underset{r}{min}& C_{BS}\hfill & \hfill & \end{array}$$

(10)

$$\begin{array}{cccc}\hfill s.t.& r\ge 0,\hfill & \hfill & \end{array}$$

(11)

$$\begin{array}{cccc}& x_{nf}\in \{0,1\},\hfill & \hfill & \forall n\in \mathcal{N},\forall f\in \mathcal{F},\hfill \end{array}$$

(12)

where $C_{BS}$ is the cost function of BS given in Equation (9), r is the unit incentive reward determined by BS, and $x_{nf}\in \{0,1\}$ is the caching decision of UE n, $\forall n\in \mathcal{N}$.

Given the reward paid by BS, all UEs try to find the optimal joint caching strategy to maximize the total utility according to the local cooperative caching mechanism. Therefore, the total utility optimization problem can be formulated as the following Problem 2.

Problem 2.

$$\begin{array}{cccc}\hfill \underset{x_n}{max}& U_{UE}=\sum _{n=1}^N{U}_n(x_n,x_{J_n})\hfill & \hfill & \hfill \end{array}$$

(13)

$$\begin{array}{cccc}\hfill s.t.& \sum _{f=1}^F{x}_{nf}\le c,\hfill & \hfill & \forall n\in \mathcal{N},\hfill \end{array}$$

(14)

$$\begin{array}{cccc}& x_{nf}\in \{0,1\},\hfill & \hfill & \forall n\in \mathcal{N},\forall f\in \mathcal{F},\hfill \end{array}$$

(15)

where $U_{UE}$ is the total utility of all UEs and $U_n(x_n,x_{J_n})$ is the utility function of the UE n given in Equation (8). The objective of Equation (13) is subject to the maximum caching storage capacity constraint and the caching decision of UE n, $\forall n\in \mathcal{N}$.

Problems 1 and 2 formulate the Stackelberg game between BS and all UEs. Each player in the game is a selfish and rational individual and tries to achieve the maximum utility/the minimum cost. Therefore, the purpose of the Stackelberg game is to find the Stackelberg Equilibrium (SE), from which neither the BS nor the UEs have the incentive to deviate. Let $\mathbf{X}=x_1\otimes x_2\otimes \dots \otimes x_N$ denote the joint caching strategy of all UEs. We have the definition of the SE for the proposed Stackelberg game in the following.

Definition 1.

Let r be the feasible solution to the cost optimization problem of BS and let $\mathit{X}$ denote a feasible joint caching strategy in response to the total utility optimization problem of UEs. Then, $(r_{SE},{\mathit{X}}_{SE})$ is a Stackelberg Equilibrium (SE) for the proposed Stackelberg game if any $(r,\mathit{X})$ in the feasible region satisfies the following conditions:

$$C_{BS}(r_{SE},{\mathit{X}}_{SE})\le C_{BS}(r,{\mathit{X}}_{SE}),$$

(16)

$$U_{UE}(r_{SE},{\mathit{X}}_{SE})\ge U_{UE}(r_{SE},\mathit{X}).$$

(17)

5. Stackelberg Game Solution

In this section, we derive the SE of the proposed Stackelberg game by using the backward induction method [27]. That is, to find an SE of the Stackelberg game, it is first necessary to determine the optimal joint caching strategy of UEs in Problem 2, which can be regarded as the best response for a given incentive reward r. Then, the optimal incentive reward can be acquired by solving the cost optimization problem of BS according to the obtained optimal joint caching strategy. However, in the UEs’ total utility maximization problem, the caching decision $x_{nf},\forall n\in \mathcal{N},\forall f\in \mathcal{F}$ is a binary variable, which incurs the UEs’ total utility sub-problem; this is a non-convex integer programming problem. Hence, it is difficult to obtain the optimal joint caching strategy in the closed form. Using a method different from that used to solve the analytical solution of the best response in the existing game models, we formulate the total utility maximization problem of UEs as a sub-game and prove that the sub-game is an exact potential game. Then, we develop a potential game-based distributed caching algorithm to obtain the optimal joint caching strategy. In the BS’s cost-optimal sub-problem, we design a dynamic iterative algorithm to find the optimal incentive reward based on the optimal joint caching strategy.

5.1. UEs’ Potential Sub-Game

In this subsection, we first prove that the sub-game of all UEs is an exact potential game utilizing local caching information exchange among UEs and that it possesses at least Nash Equilibrium (NE). However, in many cases, NE is not necessarily the optimal solution (i.e., the optimal joint caching strategy). Similarly, the optimal solution may not be NE. Here, we will prove that the total utility function of all UEs satisfies the monotone and submodular. Then, we will theoretically derive the bound of the total utility under the condition of the NE according to the submodular property. Moreover, we can prove that the optimal solution of the total utility optimization problem must be a NE.

According to the cooperative caching mechanism, if UE n changes its current caching strategy, then it will not only affect its own utility but also its neighbors’ utility. Thus, we define a local cooperative utility function $U_n^{lc}(x_n,x_{J_n})$ of UE n as follows:

$$U_n^{lc}(x_n,x_{J_n})=U_n(x_n,x_{J_n})+\sum _{j\in J_n}{U}_j(x_j,x_{J_j}).$$

(18)

In Equation (18), the local cooperative utility is composed of two parts. The first term is UE n’s own utility and the second part is the sum of neighbors’ utilities of UE n.

From Equation (18), it can be seen that the local cooperative utility improvement problem depends on the caching decision of UE n and its neighbor UEs. Thus, we consider the caching decision as the action strategy in a game and formulate the local cooperative utility improvement problem as a caching action game ${\mathcal{G}}_c=[\mathcal{N},{\left\{x_n\right\}}_{n\in \mathcal{N}},{\left\{U_n^{lc}\right\}}_{n\in \mathcal{N}}]$, where $\mathcal{N}=\{1,2,\dots ,N\}$ is the set of players (UEs), $\left\{x_n\right\}$ is the caching action strategy of player n, and $\left\{U_n^{lc}\right\}$ is the local cooperative utility function of player n. Below, we provide proof that ${\mathcal{G}}_c$ is an exact potential game.

Definition 2

([28]). A caching action game ${\mathcal{G}}_c$ is called an exact potential game if there exists a potential function $\Phi$ such that for every $n\in \mathcal{N}$, the following holds:

$$\Phi (x_n^{\prime },{\mathit{X}}_{-n})-\Phi (x_n,{\mathit{X}}_{-n})=U_n^{lc}(x_n^{\prime },x_{J_n})-U_n^{lc}(x_n,x_{J_n}),$$

(19)

where ${\mathit{X}}_{-n}=x_1...\otimes x_{n-1}\otimes x_{n+1}...\otimes x_N$ denotes the caching action strategy of all the players except player n.

Theorem 1.

${\mathcal{G}}_c$ is an exact potential game and has at least a Nash Equilibrium (NE).

Proof. 

First, we consider the total utility $U_{UE}$ of all UEs as the potential function; that is,

$$\Phi (x_n,{\mathbf{X}}_{-n})=U_{UE}=\sum _{n=1}^N{U}_n(x_n,x_{J_n}).$$

(20)

Suppose that a randomly selected UE n changes its caching strategy $x_n$ into $x_n^{\prime }$, while other UEs’ caching action strategies remain unchanged. Then, the change in the local cooperative utility of UE n is given by

$$U_n^{lc}(x_n^{\prime },x_{J_n})-U_n^{lc}(x_n,x_{J_n})=U_n(x_n^{\prime },x_{J_n})-U_n(x_n,x_{J_n})+\sum _{j\in J_n}\left(U_j(x_j,x_{J_j}^{\prime })-U_j(x_j,x_{J_j})\right),$$

(21)

where $U_j(x_j,x_{J_j}^{\prime })$ denotes UE j’s new utility function after its neighbor UE n changes the caching strategy. On the other hand, the change in the potential function due to the caching strategy change of UE n is

$$\begin{array}{cc}\hfill \Phi (x_n^{\prime },{\mathbf{X}}_{-n})-\Phi (x_n,{\mathbf{X}}_{-n})=& U_n(x_n^{\prime },x_{J_n})-U_n(x_n,x_{J_n})+\sum _{j\in J_n}\left(U_j(x_j,x_{J_j}^{\prime })-U_j(x_j,x_{J_j})\right)\hfill \\ \hfill & +\sum _{i\in \mathcal{N}\setminus J_n,i\ne n}\left(U_i(x_i,x_{J_i}^{\prime })-U_i(x_i,x_{J_i})\right),\hfill \end{array}$$

(22)

where $\mathcal{N}\setminus J_n$ means UE n’s neighbor set $J_n$ is not included in set $\mathcal{N}$. Since UE n’s caching decision only affects its neighbor UEs, we have

$$U_i(x_i,x_{J_i}^{\prime })-U_i(x_i,x_{J_i})=0,\forall i\in \mathcal{N}\setminus J_n,i\ne n.$$

(23)

Thus, we have

$$\Phi (x_n^{\prime },{\mathbf{X}}_{-n})-\Phi (x_n,{\mathbf{X}}_{-n})=U_n^{lc}(x_n^{\prime },x_{J_n})-U_n^{lc}(x_n,x_{J_n}).$$

(24)

From Equation (24), we find that the change in UE n’s local cooperative caching utility caused by UE n’s unilateral caching action strategy change is equal to the change of the potential function; that is, the change in the total utility $U_{UE}$ of all UEs. Hence, the caching action game ${\mathcal{G}}_c$ is an exact potential game. According to [28], it has been proved that each potential game has at least a Nash Equilibrium.    □

From Theorem 1, we know that ${\mathcal{G}}_c$ has at least one Nash Equilibrium. Next, we take advantage of submodular property to theoretically prove that the total utility under the NE condition can achieve at least ${\displaystyle \frac{1}{1+\delta }}$ optimal, where $0\le \delta \le 1$ is a constant. Meanwhile, we can prove that the optimal caching strategy must be a NE. In the subsequent analysis, we first provide the definition of the submodular function and then prove that the total utility function of all UEs is submodular. Then, we prove the bound for the total utility $U_{UE}$ of all UEs under the Nash Equilibrium condition.

Definition 3

(Submodular funtions [29]). Let Q be a finite ground set, a set of function U: $2^Q\to \mathbb{R}$ is submodular if sets $X,Y$ satisfy $X\subseteq Y\subseteq Q$, $f\in Q-Y$ then

$$U(X\cup \{f\left\}\right)-U\left(X\right)\ge U(Y\cup \{f\left\}\right)-U\left(Y\right).$$

(25)

The submodular funciton has the similar property with the diminishing marginal returns, that is, the marginal gain of adding an element to the larger set will decrease.

Define a finite ground set $Q=\left\{Q_{nf}\right|n\in \mathcal{N},f\in \mathcal{F}\}$, where $Q_{nf}$ denotes that the content f may be placed in the cache of UE n. The ground set can be divided into N disjoint subsets $Q_1,Q_2,\dots ,Q_N$, where $Q_n=\{Q_{n1},...,Q_{nF}\}$ denotes the all content placement configurations in the UE n. The joint caching strategy $\mathbf{X}$ is a subset of Q such that $Q_{nf}\in \mathbf{X}$ if and only if $Q_{nf}=1$. Thus, for a feasible joint caching strategy $\mathbf{X}$, the caching capacity constraint of UE n can be defined as $|\mathbf{X}\cap Q_n| \le c, \forall n\in \mathcal{N}$.

Lemma 1.

The total utility $U_{UE}$ given in Equation (13) is monotone and submodular.

Proof. 

We consider two caching placement sets X and Y which satisfy that $X\subset Y\subset Q$. For any randomly selected UE n, let $x_n$ and $x_{J_n}$ denote UE n’s caching placement strategy and the caching placement strategies of UE n’s neighbors, respectively. We assume that $(x_n\cup x_{J_n})\subset X\subset Y\subset Q$. Considering adding one element $Q_{a{f}^{\prime }}\in Q-Y,a\in \mathcal{N}$ into caching placement sets X and Y, we have the following two cases:

Case 1: UE n chooses to add $Q_{n{f}^{\prime }}$ to its own caching. For the convenience of analysis, we assume UE n acquires $Q_{n{f}^{\prime }}$ from its ith neighbor ${\left(i\right)}_n$. (If we consider that UE n acquires $Q_{n{f}^{\prime }}$ from BS, we have similar results.) That is, $Q_{n{f}^{\prime }}=x_{n{f}^{\prime }}=1$. Then, according to the caching placement X, the gain of UE n’s utility is

$$U_n(X\cup Q_{n{f}^{\prime }})-U_n\left(X\right)=\sum _{j\in J_n}{P}_{j{f}^{\prime }}\prod _{k=0}^{{\left[n\right]}_j-1}(1-x_{{\left(k\right)}_j{f}^{\prime }})r+P_{n{f}^{\prime }}{D}_{n,{\left(i\right)}_n}{\eta }_0>0.$$

(26)

According to the caching placement Y, the gain of UE n’s utility is

$$U_n(Y\cup Q_{n{f}^{\prime }})-U_n\left(Y\right)=\sum _{j\in J_n}{P}_{j{f}^{\prime }}\prod _{k=0}^{{\left[n\right]}_j-1}(1-x_{{\left(k\right)}_j{f}^{\prime }})r+P_{n{f}^{\prime }}{D}_{n,{\left(i\right)}_n}{\eta }_0.$$

(27)

From above analysis, we find that $U_n(X\cup \left\{Q_{n{f}^{\prime }}\right\})-U_n\left(X\right)=U_n(Y\cup \left\{Q_{n{f}^{\prime }}\right\})-U_n\left(Y\right)$, which satisfies the property of submodular function. And it can be seen that the utility gain of adding an element in this case is always monotone increase.

Case 2: UE n’s $j_{th}$ neighbor chooses to add $f^{\prime }$ to its own caching $(Q_{{\left(j\right)}_n{f}^{\prime }}=1)$. In this case, we have following two sub-cases:

2-1: Based on the caching placement Y, UE n can acquire the content $f^{\prime }$ from its neighbor ${\left(i\right)}_n$ with $i<j$, i.e., $Q_{{\left(i\right)}_n{f}^{\prime },}\in Y$. Then, the gain of UE n’s utility is $U_n(Y\cup Q_{{\left(j\right)}_n{f}^{\prime }})-U_n\left(Y\right)=0$. Based on the caching placement X, UE n can obtain the content $f^{\prime }$ from its neighbor ${\left(i^{\prime }\right)}_n$ with $i^{\prime }\ge i$. If $i^{\prime }<j$, then the gain of UE n’s utility is zero. Otherwise $i^{\prime }>j$, the gain of UE n’s utility is $U_n(X\cup Q_{{\left(j\right)}_n{f}^{\prime }})-U_n\left(X\right)={\eta }_0{P}_{n{f}^{\prime }}(D_{n,{\left(i^{\prime }\right)}_n}-D_{n,{\left(j\right)}_n})>0$. In this sub-case, we have $U_n(X\cup Q_{{\left(j\right)}_n{f}^{\prime }})-U_n\left(X\right)\ge U_n(Y\cup Q_{{\left(j\right)}_n{f}^{\prime }})-U_n\left(Y\right)$.

2-2: Based on the caching placement Y, UE n can obtain the content $f^{\prime }$ from its neighbor ${\left(i\right)}_n$ with $i>j$, and the gain of UE n’s utility is $U_n(Y\cup Q_{{\left(j\right)}_n{f}^{\prime }})-U_n\left(Y\right)={\eta }_0{P}_{n{f}^{\prime }}(D_{n,{\left(i\right)}_n}-D_{n,{\left(j\right)}_n})$. Based on the caching placement X, UE n can obtain the content $f^{\prime }$ from its neighbor ${\left(i^{\prime }\right)}_n$ with $i^{\prime }\ge i$. The gain of UE n’s utility is $U_n(X\cup Q_{{\left(j\right)}_n{f}^{\prime }})-U_n\left(X\right)={\eta }_0{P}_{n{f}^{\prime }}(D_{n,{\left(i^{\prime }\right)}_n}-D_{n,{\left(j\right)}_n})$. Then, we have

$$U_n(X\cup Q_{{\left(j\right)}_n{f}^{\prime }})-U_n\left(X\right)-\left(U_n(Y\cup Q_{{\left(j\right)}_n{f}^{\prime }})-U_n\left(Y\right)\right)={\eta }_0{P}_{n{f}^{\prime }}(D_{n,{\left(i^{\prime }\right)}_n}-D_{n,{\left(i\right)}_n})\ge 0.$$

(28)

In this case, we have $U_n(X\cup \left\{Q_{{\left(j\right)}_n{f}^{\prime }}\right\})-U_n\left(X\right)\ge U_n(Y\cup \left\{Q_{{\left(j\right)}_n{f}^{\prime }}\right\})-U_n\left(Y\right)$, which satisfies the property of submodular function.

Hence, the utility function $U_n(x_n,x_{J_n})$ of UE n is monotone and submodular. The submodular function is closed, i.e., the submodular function remains a submodular function after a nonnegative linear combination [30]. Hence, the total utility $U_{UE}$ of all UEs is also a submodular function.    □

From Lemma 1, we conclude that the total utility $U_{UE}$ is a submodular function. We have the following theorem to theoretically prove that the total utility $U_{UE}$ under the NE condition is at least $1/1+\delta , 0\le \delta \le 1$ of the optimal total utility (lower bound) and when the optimal caching strategy is the Nash equilibrium, we can acquire the optimal total utility (upper bound).

Theorem 2.

Let ${\mathit{X}}_{opt}$ and ${\mathit{X}}_{NE}$ denote the optimal joint caching strategy and the Nash equilibrium joint caching strategy, respectively. Then, there exists $0\le \delta \le 1$ such that the total utility $U_{UE}$ of all UEs under the Nash equilibrium satisfies

$${\displaystyle \frac{1}{1+\delta }}{U}_{UE}\left({\mathit{X}}_{opt}\right)\le U_{UE}\left({\mathit{X}}_{NE}\right)\le U_{UE}\left({\mathit{X}}_{opt}\right),$$

(29)

where $U_{UE}\left({\mathit{X}}_{opt}\right)$ is optimal total utility for all UEs.

Proof. 

From Lemma 1, we know that the total utility $U_{UE}$ of all UEs is a monotone increasing submodular function. Then, according to [31], we have the lower bound of total utility under the condition of NE. For any $0\le \delta \le 1$, we have

$$U_{NE}\left({\mathbf{X}}_{NE}\right)\ge {\displaystyle \frac{1}{1+\delta }}{U}_{UE}\left({\mathbf{X}}_{opt}\right).$$

(30)

Next, we prove the upper bound of total utility under the condition of NE. We assume that $U_{UE}\left({\mathbf{X}}_{opt}\right)$ is the global optimal total utility, i.e., $U_{UE}\left({\mathbf{X}}_{opt}\right)\ge U_{UE}\left(\mathbf{X}\right)$, $\mathbf{X}$ is any feasible joint caching strategy, and assume the optimal caching strategy ${\mathbf{X}}_{opt}=\{x_1,x_2,...,x_N\}$ is not the NE. Then, there must exist one UE n that can increase its local cooperative utility by changing its caching strategy $x_n$ to $x_n^{\prime }$, i.e., $U_n^{lc}(x_n^{\prime },x_{J_n})>U_n^{lc}(x_n,x_{J_n})$. According to the definition of the potential game, we have

$$\Phi (x_n^{\prime },{\mathbf{X}}_{-n})>\Phi (x_n,{\mathbf{X}}_{-n}).$$

(31)

Since $\Phi (x_n,{\mathbf{X}}_{-n})=U_{UE}\left({\mathbf{X}}_{opt}\right)$, that is contradicted by the assumption that $U_{UE}\left({\mathbf{X}}_{opt}\right)$ is the global optimal total utility. Hence, the optimal joint caching strategy must be the NE and $U_{UE}\left({\mathbf{X}}_{opt}\right)\ge U_{UE}\left({\mathbf{X}}_{NE}\right)$.    □

5.2. PGDC Algorithm

In this subsection, we develop a potential game-based distributed caching (PGDC) algorithm to directly maximize the local cooperative utility and further optimize the total utility of all UEs. The proposed algorithm is a distributed caching scheme that only requires local communications between UEs without BS control. According to Theorem 1, when the local cooperative utility reaches the maximum, the total utility of all UEs is also maximized. Meanwhile, we can acquire the optimal joint caching strategy. From Theorem 2, we know that the obtained optimal joint caching strategy must be a NE of the sub-game. Therefore, in the proposed Stackelberg game, we can first solve the follower’s total utility optimization problem based on Algorithm 1 and obtain the optimal joint caching strategy, which can be regarded as the best response for the given incentive reward r. Then, we can solve the BS’s cost optimization problem based on the best response in the next subsection.

In order to solve the UEs sub-game, we develop the potential game-based distributed caching algorithm as shown in Algorithm 1. In this algorithm, we assume that each UE can communicate and exchange caching information with its neighbors. The main idea of the proposed algorithm can be described as follows: First, we initialize the caching placement strategy randomly for each UE. In each iteration, a player (UE) is randomly selected to explore its new caching strategy (note that the content in the explored caching strategy has not really been cached yet). Then, the selected UE exchanges the explored caching information with its neighbors and calculates its new local cooperative utility. Furthermore, the probability that the selected UE updates its new caching strategy follows the Gibbs distribution [32], which is coupled with the calculated local cooperative utility of the selected UE as shown in Equation (18). When the local cooperative utility of the new explored caching strategy is larger than that of the current caching strategy, the Gibbs distribution will guarantee that the selected UE chooses the new explored caching strategy with high probability and, accordingly, the local cooperative utility of the selected UE will be improved. The proposed algorithm will end when the maximum iteration is reached or we achieve the maximum local cooperative utility.

Algorithm 1 Potential game-based distributed caching algorithm (PGDC)

Initialization: At time $t=0$, each UE n, $\forall n\in \mathcal{N}$ caches c contents randomly. Set maximum iteration $t_{max}$.

Loop for$t=1,2,3,...,t_{max}$

1: Selecting UE: UE n is selected randomly with equal probability. Then, each UE n calculates the local cooperative utility $U_n^{lc}(x_n,x_{J_n})$ by exchanging its current caching information with its neighbors.   2: Exploring new caching strategy: Each selected UE n chooses a trial caching strategy $x_n^{\prime }$ independently, i.e, UE n explores c new content randomly and calculates its new local cooperative utility by exchanging its new caching information with its neighbors, denoted as $U_n^{lc}(x_n^{\prime },x_{J_n})$.   3: Updating caching probability: The selected UE n chooses the trial content caching strategy in the next iteration or retains the current content caching strategy according to the following probability: $$\begin{gathered} Pr[x_n(t+1)=x_n^{\prime }]={\displaystyle \frac{exp\left(\beta U_n^{lc}(x_n^{\prime },x_{J_n})\right)}{\phi }}, \\ Pr[x_n(t+1)=x_n]={\displaystyle \frac{exp\left(\beta U_n^{lc}(x_n,x_{J_n})\right)}{\phi }}, \end{gathered}$$ (32) $\phi =exp\left(\beta U_n^{lc}(x_n,x_{J_n})\right)+exp\left(\beta U_n^{lc}(x_n^{\prime },x_{J_n})\right)$ and $\beta$ is a learning parameter. All other UEs keep their caching strategies unchanged, i.e., $x_{-n}(t+1)=x_{-n}\left(t\right)$.   4: $t=t+1$, if $t<t_{max}$, then return to step 1, else return the optimal joint caching strategy.

End Loop

In the following, we will prove the proposed algorithm is convergent and can reach stationary distribution by leveraging the balanced equation.

Theorem 3.

In the potential game-based distributed caching algorithm, the unique stationary distribution $\pi \left(X\right)$ of any joint caching strategy $X\in \mathit{X}$ is given as follows:

$$\pi \left(X\right)={\displaystyle \frac{exp(\beta \Phi (X\left)\right)}{{\sum }_{\chi \in \mathit{X}}exp(\beta \Phi \left(\chi \right))}},$$

(33)

where $\mathit{X}$ represents all the possible feasible joint caching strategies and $\Phi (·)$ is the potential function given by (20).

Proof. 

According to [33], let $X\left(t\right)=\{x_1\left(t\right),...,x_N\left(t\right)\}$ denote the joint caching strategy of all UEs at the t-th iteration. According to the Algorithm 1, the caching state $X\left(t\right)$ is determined by state $X(t-1)$, $X\left(t\right)$ is a discrete-time Markov process. Moreover, the caching states of all UEs is limited and any two of the caching states can be arbitrarily transformed. It is easily concluded that $X\left(t\right)$ is irreducible and aperiodic, which means $X\left(t\right)$ has a unique stationary distribution. Thus, $X\left(t\right)$ should satisfy the following balanced equation:

$$\pi \left(X\right)P_{XY}=\pi \left(Y\right)P_{YX},$$

(34)

where $X,Y\in \mathbf{X}$ are two different joint caching strategies and $P_{XY}$ denotes the transition probability from caching state X to state Y.

From Algorithm 1, the probability that UE n selected to explore the caching strategy is $1/N$ and the probability that the caching strategy $x_n$ of UE n changing to $x_n^{\prime }$ is ${\displaystyle \frac{exp\left(\beta U_n^{lc}(x_n^{\prime },x_{J_n})\right)}{\phi }}$. Hence, at the t-th iteration, we have

$$\begin{array}{cc}\hfill \pi \left(X\right)P_{XY}=& {\displaystyle \frac{exp(\beta \Phi (X\left)\right)}{{\sum }_{\chi \in \mathbf{X}}exp(\beta \Phi \left(\chi \right))}}\times \left({\displaystyle \frac{1}{N}}\right)\times \left({\displaystyle \frac{exp\left(\beta U_n^{lc}(x_n^{\prime },x_{J_n})\right)}{\phi }}\right).\hfill \end{array}$$

(35)

Let $\tau$ be denoted as

$$\tau ={\displaystyle \frac{1}{N\phi [{\sum }_{\chi \in \mathbf{X}}exp(\beta \Phi \left(\chi \right))]}}.$$

(36)

Then, we have

$$\pi \left(X\right)P_{XY}=\tau exp\left(\beta \Phi \left(X\right)+\beta U_n^{lc}(x_n^{\prime },x_{J_n})\right).$$

(37)

Based on the symmetry property, we can derive the following equation,

$$\pi \left(Y\right)P_{YX}=\tau exp\left(\beta \Phi \left(Y\right)+\beta U_n^{lc}(x_n,x_{J_n})\right).$$

(38)

According to the balanced equation and the definition of potential game that $\Phi \left(X\right)-\Phi \left(Y\right)=U_n^{lc}(x_n,x_{J_n})-U_n^{lc}(x_n^{\prime },x_{J_n})$, we have

$$\sum _{X\in \mathbf{X}}\pi \left(X\right)P_{XY}=\sum _{X\in \mathbf{X}}\pi \left(Y\right)P_{YX}=\pi \left(Y\right)\sum _{X\in \mathbf{X}}{P}_{YX}=\pi \left(Y\right).$$

(39)

Therefore, we prove the potential game-based distributed caching algorithm has a unique stationary distribution.    □

The proposed PGDC algorithm can be readily applied in D2D-assisted IoT networks for practical deployment. For instance, in a smart community environment, UEs (e.g., smartphones, mobile terminals) can leverage the PGDC algorithm to dynamically update their cached content based on both their own demand and their neighbors’ preferences. This not only reduces latency and offloads the central base station but also aligns with the incentive mechanisms provided by the base station. Since the algorithm only requires local information exchange between neighbors and does not rely on global network knowledge, it is suitable for large-scale privacy-sensitive applications. Therefore, PGDC provides a practical solution that bridges theoretical game-theoretic modeling with implementable caching strategies in real IoT networks.

5.3. BS’s Cost Optimization Sub-Problem

In this subsection, we design a dynamic iterative algorithm according to the optimal joint caching strategy to solve the BS’s cost optimization problem and seek the optimal incentive reward $r_{opt}$. After obtaining the $r_{opt}$ and the corresponding ${\mathbf{X}}_{opt}$, we then acquire the SE of the proposed Stackelberg game. The designed dynamic iterative algorithm includes the interactions between BS and UEs, as shown in Algorithm 2. In Algorithm 2, BS as the leader first sets the reward $r=0$ and starts the Stackelberg game. The follower (all UEs) executes the PGDC algorithm to obtain the corresponding optimal joint caching strategy for a given incentive reward r, and computes the corresponding optimal total utility $U_{UE}$ according to Equation (13). Then, BS calculates its own cost based on the obtained optimal joint caching strategy and increases the incentive reward r by a very small increment ${\Delta }_r>0$. The above interactions are repeated iteratively.

Algorithm 2 Dynamic iterative algorithm

1: function BS-game $(\mathbf{X},r)$ 2:         BS: set $r=0$ 3:         UEs-game $(\mathbf{X},r)$ 4:         calculate BS cost according to Equation (9) 5:         $r\leftarrow r+{▵}_r$ 6: until $C_{BS}$ reaches the minimum costs, the corresponding incentive reward denoted by $r_{opt}$ and the corresponding joint caching strategy denoted by ${\mathbf{X}}_{opt}$ 7: return the optimal reward $r_{opt}$ and the optimal joint caching strategy ${\mathbf{X}}_{opt}$ 8: End function 9: function UEs-game $(\mathbf{X},r)$ 10:     repeat PGDC algorithm until it reaches the maximum iteration or achieves the maximum local cooperative utility 11: return the optimal joint caching strategy ${\mathbf{X}}_r$ and the maximum total utility $U_{UE}$ according to Equation (13) for a given r 12: End function

Convergence analysis:At the beginning of iterations, the cost of BS will decrease as the reward r increases. This is because UEs are willing to cache and share content based on their neighbors’ preferences when they receive the incentive reward. However, the decreasing trend of the cost will not last very long. When the cost of the BS minimizes at a certain incentive reward, it will slowly increase after this certain point due to the limited caching capacity of UEs. That is, UEs’ requests that cannot be satisfied by their neighbors will be submitted to the BS, which increases the BS transmission energy cost. The certain incentive reward makes the cost of the BS reach the minimum value denoted by $r_{opt}$ and the corresponding optimal joint caching strategy is denoted by ${\mathbf{X}}_{opt}$, which contains the SE point of the Stackelberg game. That is, $(r_{opt},{\mathbf{X}}_{opt})$ is the SE of the proposed Stackelberg game.

5.4. Complexity Analysis

Next, we analyze the complexity of the PGDC algorithm and the dynamic iterative algorithm. In each iteration of the PGDC algorithm, the computation complexity of utility for the selected UE is determined by the number of neighbors and the caching capacity. The upper bounds of the number of neighbors and the caching capacity are N and F, respectively. Thus, the computation complexity of each iteration is $O\left(NF\right)$, and the overall computational complexity of the PGDC algorithm is $O\left(t_{max}NF\right)$. The dynamic iterative algorithm mainly comprises two steps: calculating the cost of BS and executing the PGDC algorithm. The complexity of calculating the cost is $O\left(N^{2}F\right)$, and the complexity of the PGDC algorithm is $O\left(t_{max}NF\right)$. Therefore, given that the dynamic iterative algorithm runs for ${\displaystyle \frac{r_{max}}{{\Delta }_r}}$ iterations, its overall computational complexity is $O({\displaystyle \frac{r_{max}}{{\Delta }_r}}NF(N+t_{max}))$.

6. Performance Evaluation

In this section, we show the numerical results to validate the existence of SE in the proposed Stackelberg game. Moreover, we compare the performance of the PGDC algorithm with other benchmark caching schemes under different simulation parameters.

6.1. Experimental Setups

We consider a D2D assisted IoT caching network with a base station and 50 UEs, where the transmission radius of BS is 500 m and the maximum communication distance of UEs is 50 m. There are 40 pieces of content in the network, each with a size of 100 MB. The location distribution of BS and UEs in the simulation environment is shown in Figure 2. Each UE has no more than four neighbors and every UE has heterogeneous content request probability based on its own preference. In the simulations, we set the ranks of contents for each UE randomly to model different preferences of UEs. We further compare the PGDC algorithm in Algorithm 1 with three other benchmark caching schemes: (1) the most popular caching (MPC) scheme [10], in which each UE caches the c most favourite pieces of content according to their preference for a given reward r; (2) the greedy caching (GC) scheme [12], in which each UE caches c pieces of content based on its own maximum utility $U_n(x_n,x_{J_n})$ for a given reward r, without considering its local cooperative utility $U_n^{lc}(x_n,x_{J_n})$; and (3) the random caching (RC) scheme, in which each UE caches c pieces of content randomly for a given reward r. In addition, the primary simulation parameters are listed in

Table 2. Simulation parameters.

Parameter	Value
Number of UEs $\left(N\right)$	50
Number of contents $\left(F\right)$	40
Size of content $\left(S\right)$	100 MB
Transmission power of BS $\left(P_0\right)$	30 w
Transmission power of UE $\left(P_n\right)$	0.5 w
Bandwidth for D2D links $\left(B_{n,j}\right)$	30 MHz
Bandwidth for BS-UE links $\left(B_{n,0}\right)$	10 MHz
Unit transmission delay cost $\left({\eta }_0\right)$	0.05
Unit transmission delay cost $\left({\eta }_1\right)$	0.1
Unit transmission energy cost $\left({\eta }_2\right)$	0.1
Noise power $\left({\sigma }^2\right)$	−174 dBm/Hz
Pathloss exponent $\left(\gamma \right)$	3
Channel gain $\left(\right|h_{n,j}\left|\right)$	${10}^{-2}$
Unit transmission delay cost $\left({\eta }_0\right)$	0.05
Learning parameter $\left(\beta \right)$	50

.

6.2. Numerical Results

We first evaluate the convergence of the PGDC algorithm in Algorithm 1 with different caching capacities when we set the incentive reward $r=1$. In Figure 3, we find that the number of iterations needed for the total utility of all UEs to reach the stable state under different caching capacity is no more than 1500, which indicates that the designed PGDC algorithm can converge quickly to a stable cache state. Moreover, we can further conclude that the larger caching capacity is, the greater the total utility will be. This is because a larger caching capacity helps UEs share more content under the given incentive reward, while reducing the delay cost of requesting content from the base station.

Figure 4 shows the impact of incentive reward r on the total utility $U_{UE}$ for all UEs and the cost $C_{BS}$ of BS and illustrates the existence of the Stackelberg Equilibrium of the proposed game. It can be observed that the total utility increases monotonically with the given reward r. That is because a larger r encourages UEs to be more willing to cache content based on their neighbors’ preferences, which leads to larger incentive revenue. Furthermore, we find that the cost of BS decreases with the increase in reward r at the beginning. However, when the reward r gradually increases to a point (i.e., $r_{opt}$), the cost of BS will begin to increase. This is because UEs will take into account neighbors’ preferences with regard to the incentive reward, which enables the requested content to be directly obtained from neighbor UEs and reduces the request from the base station, thereby lowering the transmission delay cost of the base station. In addition, the initial incentive reward offered by the base station is significantly less than the unit transmission energy cost. Therefore, the total cost of the BS decreases in the early stage. As the incentive rewards gradually increase, the incentive expenses for BS start to rise. Additionally, due to the limited caching capacity of UEs, when the preferences of their neighbors cannot be satisfied, some requested contents will inevitably not be retrieved in the local cache and the neighbor caches. These requests will be sent to the BS, thereby increasing the transmission energy expenses of the BS. In Figure 4, when we gradually increase the incentive reward, we can find that the optimal incentive reward $r_{opt}$ is 0.12, which corresponds to the minimum cost of BS. This point, together with its corresponding optimal joint caching strategy, constitutes the equilibrium point we aim to identify in the Stackelberg game.

Figure 5 illustrates the impact of Zipf exponent $\alpha$ on the total utility $U_{UE}$ for the proposed PGDC algorithm and the other three caching schemes when we set the incentive reward $r=1$ and each UE’s capacity is five. It can be seen that the total utility in PGDC algorithm, MPC scheme and GC scheme increases respectively when $\alpha$ increases from 0.8 to 1.6. The total utility of the RC scheme shows some fluctuations when $\alpha$ increases. When $\alpha$ is larger, there are fewer popular pieces of content in the network, which means that the top-ranked content for each UE has a larger request probability. Hence, once the popular content has been cached, resulting in larger request demands, the total utility in the PGDC algorithm, MPC scheme, and GC scheme will increase accordingly. In RC scheme, the popular content may not be cached when $\alpha$ increases, which leads to fluctuations in the total utility. Moreover, we find that our proposed PGDC algorithm achieves the highest total utility compared with the other three caching schemes, indicating that it can yield additional utility gains. Since each UE has heterogeneous caching preference, in the MPC scheme, UEs only cache their own favourite c content, and do not consider their neighbors’ preferences. Hence, the total utility of the MPC scheme is smaller than that of the proposed PGDC algorithm.

Figure 6 shows the impact of caching capacity on the total utility for the proposed PGDC algorithm and the other three caching schemes when we set the incentive reward $r=1$ and Zipf exponent $\alpha =1.2$. We find that the total utility of the proposed PGDC algorithm is the highest compared with other benchmark caching schemes. When the caching capacity is five, quantitative analysis shows the PGDC algorithm achieves a total utility that is 25.1% higher than the GC scheme, and 86.1% higher than the MPC scheme. Moreover, it can be observed that the total utility increases with the caching capacity. This can be explained based on the fact that when the caching capacity becomes larger, each UE is able to cache more content in its local cache, which can satisfy its own needs or the requests of neighbors. Accordingly, a larger caching capacity enables UEs to either gain more incentive benefits or reduce delay costs, thereby contributing to an increase in total utility.

Figure 7 compares the total utility under different amounts of content when the incentive reward r is 1, Zipf exponent $\alpha$ is 1.2 and the caching capacity is 5. When the amount of content in the network increases, the total utility of UEs shows a decreasing trend in all four caching algorithms. The main reason for this change is that the caching capacity is limited and the requests of UEs also become more diverse as the amount of content increases. When the limited local caching capacity and the content in the neighbor caches cannot meet the diverse demands, UEs will request content from the BS, which increases the corresponding transmission delay cost and thereby reduces the total utility.

Figure 8 illustrates the impact of different numbers of neighbors on the total utility in all four caching algorithms. (In order to highlight the impact of the number of neighbors on the total utility, the location distribution of BS and UEs shown in Figure 2 is no longer adopted. In each comparison, it is assumed that the number of neighbors for each UE is the same. For example, when we set the number of neighbors to 2, all UEs have two neighbors in all caching algorithms.) It can be observed that the total utility improves significantly when the number of neighbors increases from 1 to 4. Furthermore, the proposed PGDC algorithm exhibits the highest growth rate in this scenario. This is because increasing the number of neighbors can enhance the chances of content sharing or content requests, which increases the incentive benefits or reduces the transmission delay cost accordingly. Hence, the total utility presents a notable growth when the number of neighbors increases.

Finally, we conduct experiments to validate the performance gap between the Nash Equilibrium (NE) solutions derived via the PGDC algorithm and the global optimal solutions—a gap for which a theoretical bound is provided in Theorem 2. Since finding the true global optimum $U_{UE}\left({\mathbf{X}}_{opt}\right)$ for larger networks is computationally prohibitive (NP-hard), we instead use a small-scale network setup with a limited number of UEs $N=3$ and content $F=5$.

Table 3. Different capacity vs. iterations.

Capacity	${\mathit{U}}_{\mathit{UE}}\left({\mathbf{X}}_{\mathit{opt}}\right)$	10 Iterations		25 Iterations		50 Iterations
		${\mathit{U}}_{\mathit{UE}}\left({\mathbf{X}}_{\mathit{NE}}\right)$	Ratio	${\mathit{U}}_{\mathit{UE}}\left({\mathbf{X}}_{\mathit{NE}}\right)$	Ratio	${\mathit{U}}_{\mathit{UE}}\left({\mathbf{X}}_{\mathit{NE}}\right)$	Ratio
2	2.43	1.78	0.73	2.37	0.98	2.43	1.0
3	2.05	1.59	0.77	2.05	1.0	2.05	1.0
4	1.43	1.43	1.0	1.43	1.0	1.43	1.0

compares the ratio ${\displaystyle \frac{U_{UE}\left({\mathbf{X}}_{NE}\right)}{U_{UE}\left({\mathbf{X}}_{opt}\right)}}$ under different caching capacities when the incentive reward r is 1, Zipf exponent $\alpha$ is 1.2. We find that all ratios ${\displaystyle \frac{U_{UE}\left({\mathbf{X}}_{NE}\right)}{U_{UE}\left({\mathbf{X}}_{opt}\right)}}$ for different NE strategies fall within the range $[{\displaystyle \frac{1}{1+\delta }},1]$, and increasing iterations of the PGDC algorithm can improve the ratio. When the number of iterations is large enough, we find the ratio is equal to 1, which fully demonstrates that the PGDC algorithm is capable of finding the optimal joint caching strategy. Furthermore, when capacity is equal to 4, the optimal total utility decreases compared to the other two scenarios. This is because the content cached by each UE can basically meet its own needs, and the mutual requests between UEs are also reduced accordingly. This leads to a decrease in the incentive earnings, thereby causing a reduction in the total utility.

Table 4. Different Zipf exponents vs. iterations.

Zipf Exponents	${\mathit{U}}_{\mathit{UE}}\left({\mathbf{X}}_{\mathit{opt}}\right)$	10 Iterations		25 Iterations		50 Iterations
		${\mathit{U}}_{\mathit{UE}}\left({\mathbf{X}}_{\mathit{NE}}\right)$	Ratio	${\mathit{U}}_{\mathit{UE}}\left({\mathbf{X}}_{\mathit{NE}}\right)$	Ratio	${\mathit{U}}_{\mathit{UE}}\left({\mathbf{X}}_{\mathit{NE}}\right)$	Ratio
0.8	2.23	1.58	0.71	2.18	0.97	2.23	1.0
1	2.34	1.66	0.71	2.28	0.98	2.34	1.0
1.2	2.43	1.78	0.73	2.37	0.98	2.43	1.0

compares the ratio ${\displaystyle \frac{U_{UE}\left({\mathbf{X}}_{NE}\right)}{U_{UE}\left({\mathbf{X}}_{opt}\right)}}$ under different Zipf exponents when the incentive reward r is 1, and the caching capacity is 2. We also find that, for different NE strategies, all ratios ${\displaystyle \frac{U_{UE}\left({\mathbf{X}}_{NE}\right)}{U_{UE}\left({\mathbf{X}}_{opt}\right)}}$ lie within the range $[{\displaystyle \frac{1}{1+\delta }},1]$. Similarly, increasing iterations of the PGDC algorithm can improve the ratio. When the number of iterations is sufficiently large, the PGDC algorithm can find the optimal joint caching strategy.

7. Conclusions

In this paper, we investigated a joint caching incentive and caching placement optimization problem in a D2D-assisted IoT caching network which consists of one BS and several cache-enabled UEs. We first introduced an uncoded caching incentive mechanism with the consideration of users’ different content preferences. Then, we formulated the the economic interactions between BS and UEs as a one-leader–one-follower Stackelberg game. In the proposed game, the BS initially determines the unit incentive reward to all UEs to encourage content sharing between UEs. Based on the given reward, UEs make the joint caching decision to maximize their total utility. In order to find an SE of the proposed Stackelberg game, we proposed a PGDC algorithm to achieve the maximum total utility of UEs and to obtain the optimal joint caching strategy as the follower’s best response. Furthermore, we proved the convergence of the PGDC algorithm. Based on the optimal joint caching strategy, we developed a dynamic iterative algorithm to solve the BS’s cost optimization problem. Finally, we presented the numerical results to validate the existence of SE in the proposed Stackelberg game. We also compared the performance of the proposed PGDC algorithm with other benchmark caching schemes. The simulation results demonstrate a significant performance advantage for our approach. For instance, under typical network settings with caching capacity of $c=5$, the PGDC algorithm yields a total utility gain of 25.1% over GC algorithm and 86.3% over MPC algorithm. This highlights the effectiveness of our PGDC algorithm in achieving substantial extra utility gain for the D2D-assisted IoT caching network.