On the Retrial-Queuing Model for Strategic Access and Equilibrium-Joining Strategies of Cognitive Users in Cognitive-Radio Networks with Energy Harvesting

: This article studies the strategic access of single-server retrial queue with two types of customers, where priority is given according to their category. On the basis of this concept, a cognitive-radio network was developed as retrial queue with energy harvesting. Cognitive radio allows for a secondary user to opportunistically access the idle spectrum of a primary user (PU). Upon arrival of a primary user, the service given to the secondary user by the cognitive radio is interrupted, and the PU band is available for the primary user. After completion of service for the primary user, the PU band is again available to secondary users. Performance metrics are derived to study the equilibrium strategies of secondary users. A Stackelberg game was formulated and Nash equilibrium was derived for the noncooperative strategy of the secondary user. Game-theory concepts are incorporated with queuing theory ideas to obtain the net beneﬁt for the noncooperative strategy and social beneﬁt for cooperative strategy. Lastly, analytical results are veriﬁed with numerical examples, and the effects of energy-harvesting rate are discussed.


Introduction
In computer and telecommunication technologies, there exists a phenomenon called retry that often occurs to obtain a service, i.e., users try for their service while the server is serving another user. In these circumstances, they enter the waiting pool called orbit and try for their service after a random time period. These types of queuing systems are called retrial-queuing systems. For the detailed study about retrial queues one can refer Falin and Templeton [1]. These features also apply for wireless communication systems.
Currently, cognitive radio (CR) is a promising technology that allows for a user to solve the problem of spectrum scarcity. CR is an intelligent radio that senses the surrounding environment, understands a physical situation, decides and adjusts the transmission parameters according to the environment for the benefit of users. If there is no affirmative situation for transmission, it embraces it according to the surroundings.
In 1999, Mitola III [2] outlined a new technology that allows for a user to exploit the unused spectrum. A survey by the Federal Communication Commission in the United States stated that 70% of the spectrum that was allocated to the user upon their demand was not used properly. Only 2% of the spectrum was utilised by demanding user. In the past few decades, software radio has existed in practice, and this defined software radio gave infrastructural development for CR. A CR network can be fashioned via using CR by additionally encompassing radio link properties to network-layer functions. The main objective of CR is to solve the spectrum demand of users and ensure optimal spectrum usage for users. analysed a noncooperative game for an M/M/1 queuing system with breakdowns, i.e., the service interruption of a secondary user was considered as a breakdown, and showed that the game had an inefficient Nash equilibrium; by imposing admission fees, the social welfare at the equilibrium point coincided with the social welfare of the socially optimal strategy. Wang and Zhang [9] analysed strategic joining in M/M/1 retrial and investigated customers balking strategic behaviour for observable and unobservable cases. Chang et al. [10] discussed individual and social strategies, and pricing strategy by considering only one base station for serving primary and secondary users. Secondary users are dismissed in the system when a primary user arrives.Through game theory, the decision-making process of whether secondary users would join the queue is studied in this paper.Wang et al. [11] examined opportunistic spectrum sharing with imperfect sensing from a game-theory perspective. In this, they modelled the system with constant retrial and server breakdowns, where secondary users followed the carrier-sense multiple-access (CSMA) protocol with imperfect spectrum sensing. Zhao and Yue [12] discussed multiple secondary users through the construction of three-dimensional Markov chain steady-state distribution of the model. They obtained and compared the Nash equilibrium, and the socially optimal strategy for secondary users for different schemes. Wang and Li [13] examined noncooperative and cooperative joining strategies of secondary users in cognitive-radio networks and treated them as M/M/1 retrial queuing systems, but did not implement the energy-harvesting state; in this, admission fees were imposed on secondary users to bridge the gap between individual and social strategy. Wang et al. [14] discussed equilibrium strategies and service pricing in the Internet of Things (IoT) as an M/G/1 queue with energy harvesting; since IoT systems exist with homogeneous users, they did not consider service interruption in their model. IoT is considered to be a leader of the Stackelberg game and a social planner that adjusts the service price to maximise its and its users' revenue; the social behaviour of customers in IoT systems was also discussed. In all the above papers, equilibrium strategies for individually optimal strategies and optimal social welfare strategies of secondary users (SUs) were imposed with admission fees by the CR network base station; an individual decision of SUs coincides with a socially optimal strategy that optimises the total welfare of SUs.
Currently, radio-frequencies (RFs) in energy harvesting are a favourable solution for power-energy-constrained wireless networks. RFs in CR networks are capable of converting received RF signals into electricity. In RF-powered CR networks, the energyharvesting technique allows for secondary users to harvest and store energy from primary users if they are neighbours of primary users. Even if they are far away from primary users, they can transmit data. Due to this, secondary users search for both spectrum holes for opportunistic access and for the occupied spectrum of a primary user to harvest energy. The authors in [6][7][8][9][10][11]13,14] reported equilibrium strategies in CR networks with different environments. However, no study considered energy harvesting in CR networks. Wang et al. [14] analysed energy harvesting in the IoT for reliable servers with general service time. However, in practice, there may be server interruption due to breakdowns or service priority is given for different categories of arrivals. The present article was prepared with the implementation of Wang and Li [13] energy harvesting as an additional factor for consideration in CR networks.
This paper focuses on energy harvesting as a major factor in CR networks, particularly the performance metrics and equilibrium-joining strategies [15] of secondary users with the help of queuing theory in addition to a game-theory aspect. A noncooperative game was formulated, and the Nash equilibrium was identified. The rest of the paper is organised as follows. Section 2 explains the mathematical formulation and steady-state equations of the problem, and their solutions. In Section 3, we presents the results with important performance metrics and equilibrium-joining strategies. Section 4 discusses the numerical results to validate our analytical findings. In Section 5, we presents the conclusions.

Mathematical Model
Let us consider a CR system with a single band, which acts as a server and is termed a PU band. In the system, there are two types of customers: licensed and unlicensed users. Licensed users are called primary users (PUs), and unlicensed users are called secondary users (SUs). The PU band serves either a primary or a secondary user, but not both simultaneously. Due to high priority of primary users, whenever they enter into the cognitive-radio system, the PU band is allocated to the primary user. Even though the PU band is in transmission for a secondary user, transmission is interrupted by the arrival of a primary user, and the PU band allocates the spectrum to the primary user. The interrupted secondary user waits in front of the PU band instead of entering the orbit. If the PU band entirely completes its transmission for the primary user, it again allocates the PU band to the secondary user. Once the PU band completes its service for either a primary or a secondary user, it spends some time for its energy harvesting. Figure 1 shows this type of system model, i.e., a primary band is being accessed from the base station. This primary band acts as a server equipped with cognitive radio for primary and secondary users. The system is capable of energy harvesting through radio-frequency, solar, or wind power. If primary users are not present in the system, secondary or cognitive users access the spectrum, and the remaining secondary users enter the orbit and try for their service through retrials. When primary users enter the system, service for secondary users is interrupted. After completing the service for the primary user, the interrupted secondary user's service continues. During energy-harvesting mode, the PU band does not serve any more customers, either primary or secondary. Incoming users join the orbit and repeatedly try for the PU band. After energy consumption takes place, the PU band goes idle, and again serves new or blocked customers. In this study, heterogeneous users such as primary and secondary users access the cognitive-radio system. The arrival pattern of users follows Poisson distribution. The arrival rates of primary and secondary users are denoted as α and λ, respectively. Service time follows exponential distribution. Service rates are denoted as β and µ for primary and secondary users, respectively. If an incoming user finds that the PU band is not available, i.e., either in serving a primary or a secondary user or in the state of energy harvesting, they join the retrial orbit by joining probability q. A user in orbit repeatedly tries for the PU band until they obtain the service that is exponentially distributed; the retrial rate is denoted as θ. The rate of retrial increases as the users in orbit increase, that is, each user in the retrial orbit can independently attempt for the PU band. Other users decide to leave the system or balk with probability 1-q (q). The energy-harvesting rate of the cognitive-radio system is denoted as η. The sensing of the primary-user band by a user results in a positive cost per unit time. If a secondary user decides to enter the system, the waiting time costs some positive amount. Let C be cost per unit time for the delayed secondary user. Upon successful service, the user obtains the reward, which is denoted as R. Let us denote ρ 1 = λ µ , and ρ 2 = α β are the traffic intensities of secondary and primary users, respectively. For our convenience, we denoted ρ = ρ 1 (1 + ρ 2 ) and ρ θ = µ θ . Table 1 gives summarization of the above notations. Here, we consider a linear retrial rate from the orbit to the CR network system. We assumed that the interarrival time of secondary users, the service time of secondary users, the retrial time, the interarrival time of primary users, and the service time of primary users are mutually independent. Let us denote T(q) as the average delay for secondary users who join the orbit; for this entering user, we identified joining probability q ∈ [0, 1] with the constraint that we have to maximise benefit argmax q∈[0,1] S(q; q) =q.S(q) =q[R − CT(q)], where S(q) is the expected benefit for secondary users if they decide to join the system. Since we are interested in analysing the behaviour of secondary users, we considered the existence of a symmetric equilibrium among them. If q e represents the equilibrium, argmaxq ∈[0.1] S(q, q) = q e . Due to the demand for the PU band, there is competitiveness among secondary users, so that each user behaves to obtain its own benefit. Meanwhile, the CR system provider can fix the reward and cost for waiting time to obtain their own benefit. Under this situation, we analysed the noncooperative and cooperative strategies of secondary users. Joining probability ρ 1 Traffic intensity of secondary user ρ 2 Traffic intensity of primary user ρ θ θ µ ρ ρ 1 (1 + ρ 2 )

Steady-State Equations
Let S(t) be the state of the PU band at any instant t, and N(t) be the number of secondary users in the retrial orbit at any time t. Then, S(t) = {0, 1, 2, 3, 4} corresponds to the states in which the PU band is idle, occupied with a secondary user, a primary user with one interrupted secondary user, a primary user with no interrupted secondary user, or in energy harvesting. A CR network system can be considered as a retrial-queuing system, then the stochastic process {(S(t), N(t)), t ≥ 0} is a two-dimensional continuous-time Markov chain with state space {0, 1, 2, 3, 4} × Z + . The state transition diagram is given by Figure 2. The balance equations are expressed as

Steady State Solutions
Theorem 1. In a CR network system, the probabilities that the primary band is idle, occupied with a secondary user, occupied with a primary user with one interrupted secondary user, occupied with a primary user with no interrupted secondary user, or in energy harvesting are given by Proof. Equations (1)-(5) can be solved by using the generating function techniques. We define the partial generating function as follows.

Results
Theorem 2. The mean numbers of secondary users in the orbit when the primary band is idle, occupied with a secondary user, occupied with a primary user with one interrupted secondary user, occupied with a primary user with no interrupted secondary user, or in energy harvesting are given by Proof. Differentiating (12)-(16) with respect to z and setting z = 1, we obtain Differentiating (21) and inserting z = 1, we obtain Using (21) and inserting (33) into (28)-(32), we obtain Using (34) and N 0 = P 0 (1), we can obtain Equation (23). Inserting (33) and (34) in (29)-(32), and using N 1 = P 1 (1), N 2 = P 2 (1), N 3 = P 3 (1) and N 4 = P 4 (1), we can obtain Equations (24)-(27), which completes the proof. Theorem 3. The mean number of secondary users in the orbit is given by Proof. The expected number of repeating users is given by Adding (23)-(27), we can obtain the mean number of secondary users in the orbit.
Theorem 4. The expected waiting time of an arriving secondary user that finds that the primary band is occupied by either a primary or a secondary user and decides to join the orbit is given by Proof. Effective arrival rate is given by Substituting (6) in (37), we can obtain By Little's law, we can obtain the mean waiting time T q of a secondary user in the orbit by Inserting (35) and (38) in (39), we obtain the expected waiting time of an arriving secondary user in the orbit.

System Stability
If ρ < 1, the system is stable; if ρ > 1, the system is unstable. This is possible because the denominator of T q contains term (qρ − 1); if this term gives a positive value, then only the system is stable. So, qρ < 1 . This is the condition for system stability.

Noncooperative Strategy
In noncooperative strategy, all secondary users behave for their own benefit without considering other secondary users and the cognitive-radio system. Depending upon cost C, which is charged for the waiting time in the orbit, and reward R, received after completion of the service, either they can enter or balk. Secondary users decide to join the system if the reward for service exceeds the expected cost in the orbit. Since all secondary users act for their own benefit, this affects the strategy of other secondary users and of the cognitive-radio system. On the basis of the behaviour of a secondary user, this system can be modelled as a symmetrical noncooperative game. Under this competitive situation, each secondary user is considered to be a player of the game, and our aim was to find the Nash equilibrium using a game-theory concept.
The expected net benefit of a secondary user who decides to join the system and enters the system when the primary band is unavailable is given by Inserting (36) in (40), we obtain where R is the reward received after each service, and C is the waiting cost per unit time.
Theorem 5. Considering qρ < 1 in the cognitive-radio network system, a Nash equilibrium mixed strategy exists with entrance probability when secondary users observing the primary band are busy with the other user, where q e is given by where a, b, and c are given in Appendix A.1.
Proof. For qρ < 1, S(q) is strictly decreasing and has a unique maximum and a unique minimum S(1) = R − C * T(1) When R C < T(0), then S(q) is negative for all q; therefore, q e = 0. When T(0) ≤ R C ≤ T(1), then there exists a solution for S(q) = 0 that lies in interval [0, 1], which gives q e for this case.
When R C > T(1), incoming secondary users obtain the net benefit for all q. Therefore, q e = 1 in this case.

Theorem 6.
Considering ρ ≥ 1 in the cognitive-radio system, a Nash equilibrium mixed strategy with entrance probability q e when the secondary users observing the primary band is busy with the other user exists, where q e , is given by where T(0) and T(1) were already given in (43) and (44).

Cooperative Strategy
The social benefit per unit time when all secondary users follow a mixed policy entered with probability q while observing that the primary band is either occupied with a primary user or in energy harvesting is given by where λ * = λP 0 (1) + λq(P 1 (1) + P 2 (1) + P 3 (1) + P 4 (1)) (50) Inserting (51) and (35) into (49), social benefit per unit time S soc is obtained. The maximisation of S soc (q) over q, that is, finding social optimal joining probability is very tedious since (35) is very complicated, as it contains q terms in both numerator and denominator. So, we continued this numerically by using MATLAB. The following algorithm outlines the maximisation problem for finding social optimal probability q * .
Step 2: Differentiate S soc (q) with respect to q.

Discussion
In this section, analytical results are validated with numerical examples due to the effects of various parameters in the joining behaviour of a noncooperative strat-egy. Figures 3-10 show how the values of parameters α, β, λ, µ, θ, η, R and C influence equilibrium-joining probabilities. Figures 11-14 show the effect of energy-harvesting rate η on the total mean number of users in the orbit, total waiting time in the orbit and the net benefit of users when they follow a noncooperative strategy, and the social benefit of secondary users when they follow a cooperative strategy. Figure 3 shows that joining probability monotonically decreases with respect to the arrival rate of primary user α. When α increases, more primary users join the system, and they can immediately obtain the primary band by interrupting a secondary user no matter whether they are occupied with the primary band or trying for the primary band in orbit. The interrupted secondary user waits in front of the primary band; whenever the service for a primary user is completed by the primary band, the waiting secondary user obtains the primary band. The waiting time duration in front of the server makes other secondary users in the orbit unable to occupy the primary band. This congests the orbit more, and secondary users have less interest in joining the orbit, since waiting time in the orbit adds a waiting cost for a secondary user. As a result, fewer secondary users try for the primary band. Figure 4 shows that the joining probability monotone increases with respect to service rate β of primary users. That means that, if β increases, more primary users are served; if the service rate is increased, then more primary users leave the system, which makes secondary users easily occupy the primary band, which reduces the waiting time in orbit and the waiting cost of secondary users. This leads to more secondary users joining the system.             Figure 5 shows that the joining probability monotone decreases with respect to the arrival rate of secondary users' λ. When λ increases, arriving secondary users expect that the orbit is occupied by either a primary or a secondary user. So, they have less interest in joining the orbit because of the waiting cost in orbit. This makes fewer secondary users join the system. Figure 6 shows that joining probability monotone increases with respect to the service rate of µ of secondary users. This can be interpreted as follows: if more secondary users are served, the system has far fewer primary users and has more available service for secondary users, reducing the orbit crowd; this reduces waiting time in orbit and waiting cost. This makes secondary users have much interest in joining the system. Figure 7 shows that joining probability monotone increases with respect to energy-harvesting rate. If η is higher, then the cognitive-radio system works for longer. If it is with energy, the system can serve for a long period of time. Then, if the period for service is increased, the number of served users is also increased. Since service rate increases, the system can serve more users. This make the orbit have fewer users. So, more secondary users decide to join the system. Figure 8 shows that joining probability monotone increases with respect to retrial rate θ of secondary users. If θ increases, that is, if secondary users increasingly make attempts for the primary band, then there may be a chance to obtain it. This makes the orbit have far fewer secondary users. Due to the far fewer secondary users in orbit, incoming secondary users join the system without worrying about the waiting cost. That leads to more secondary users entering the system. Figure 9 shows that joining probability monotone increases with respect to reward R for each service. If the R reward gained after the completion of service increases, then secondary users are not bothered about the waiting time in orbit and can endure the waiting cost there. For a higher value of R, more secondary users are inclined to join the system. This makes R an increasing function. Figure 10 shows that joining probability monotone decreases with respect to the waiting cost of secondary users in the orbit. If C increases, incoming secondary users are less inclined to join the system. There may be a chance to balk into the system because of the higher waiting cost. If cost C is increased, the net benefit obtained by a secondary user is decreased, even though they expect their net benefit to be at the maximum. Therefore, they have much less interest in joining the system. Figure 11 explains the effect of energy-harvesting rate η on the total mean number of secondary users in the orbit for different values of q. If energy-harvesting rate η increases, then users in orbit decrease. This happens since, if η is increased, more secondary users obtain power consumption, which makes users in the orbit quickly obtain power consumption. Therefore, more users are served; this makes the system less congested, and it can transmit data over a long interval. Figure 12 shows the effect of energy-harvesting rate η with respect to the total waiting time of secondary users in the orbit for different values of q. When η increases, the total waiting time of secondary users in the orbit decreases. When power consumption is quicker, secondary users take less time in the system and can quickly leave the orbit. This makes the system more beneficial to secondary users and reduces waiting cost C for them. Figure 13 indicates the influence of energy-harvesting rate η in the net benefit of secondary users when they follow a noncooperative strategy. When η increases, S(q) also increases. Since the power-consumption rate is increased, this make the secondary users easily obtain power, and makes a user spend less time in system and orbit, so waiting-time cost C is decreased, but reward R remains as same for each service. This makes it more beneficial to secondary users. Figure 14 shows the effect of energy-harvesting rate η with respect to the social benefit of other secondary users and the cognitive-radio network (CRN) system. When η increases, S soc (q) also increases. As the power-consumption rate is increased, more users quickly obtain power and service. This makes the CRN systems more efficient, they can serve more users, and secondary users obtain power for consumption very quickly.
In a cooperative strategy, optimising social strategy makes the expression very complicated. So, results can be compared using both categories in Figures 13 and 14. When we compare S(q) and S soc (q) for q = 0.1, 0.2, 0.3 and 0.4, S(q) is higher than S soc (q). By imposing proper admission fees on secondary users that enter the system makes S(q) and S soc (q) as equal or S soc (q) is greater than S(q). From a system-provider point of view, the system-parameter service rate and energy-harvesting rate can be adjusted according to parameters that depends on a user to obtain the profit. The remaining parameters all depend only on users. Here, we present a comparison for both probabilities, i.e., equilibrium joining probability q e and social optimal probability q * ; corresponding values are given in Appendix A. 2 Tables A1 and A2. Usually, individual joining probability q e is greater than social optimal probability q * . A graph is presented to compare q e and q * against µ and η in Figures 15 and 16.

Conclusions
In this article, a CR network system was presented equipped with energy harvesting, treated as a retrial queue with two types of customers where priority is given according to their category. Through generating a function approach, important system performance metrics were identified. Queuing theory concepts were incorporated with game theory, and the Nash equilibrium and socially joining strategies of secondary users were identified when they acted according to noncooperative and cooperative strategies. Results were validated with numerical examples for different parameters, and the effect of the energyharvesting rate on various performance metrics was presented. Numerical observations showed that the joining probability of a social optimisation problem was smaller than the individual optimisation problem. By imposing appropriate values for R and C, a system provider could obtain an optimal benefit for secondary users. This concept can be extended with general service time and for more than one server. Data Availability Statement: The study did not report any data.

Acknowledgments:
The authors would like to thank two anonymous referees for their helpful comments on this article that significantly improved the quality of the paper.

Conflicts of Interest:
The authors declare no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript: