Robustness of the cμ-Rule for an Unreliable Single-Server Two-Class Queueing System with Constant Retrial Rates

Abstract

We study the robustness of the $c\mu$-rule for the optimal allocation of a resource consisting of one unreliable server to parallel queues with two different classes of customers. The customers in queues can be served with respect to a FIFO retrial discipline, when the customers at the heads of queues repeatedly try to occupy the server at a random time. It is proved that for scheduling problems in the system without arrivals, the $c\mu$-rule minimizes the total average cost. For the system with arrivals, it is difficult directly to prove the optimality of the same policy with explicit relations. We derived for an infinite-buffer model a static control policy that also prescribes the service for certain values of system parameters exclusively for the class-i customers if both of the queues are not empty, with the aim to minimize the average cost per unit of time. It is also shown that in a finite buffer case, the $c\mu$-rule fails.

1. Introduction

The modeling and analysis of telecommunications and computer systems is now inconceivable without the various tasks associated with resource allocation and formulated in the framework of queueing theory. One such classic problem is to allocate a server to multiple parallel queues with the most-studied objective of minimizing the average cost per unit of time. There is a great diversity in the application of such controlled systems, for example, the task of scheduling a processor in multilevel programming, the transmission of different types of packages in a telecommunications network, or a machine tool that processes different types of work pieces. It was shown that for many systems with such kinds of resource allocation problems, the allocation policy in the form of the $c\mu$-rule is optimal. In the literature, this rule is also known as Smith’s rule or weighted shortest processing time. According to this rule, the waiting class-i customer from a non-empty queue is allocated to the server if it has a maximal weight of the form $c_i{\mu }_i$, where $c_i$ is a holding cost per unit of time the customer is in the queue or at the server and ${\mu }_i$ is an overall service rate of the class-i customer. The $c\mu$-rule is a very attractive policy since it is a static one and requires only the information regarding whether a certain queue is empty or not. Obviously, to apply such a policy, the values $c_i$ and ${\mu }_i$ must be known, which unfortunately is not always the case, especially for the overall service intensity.

The optimality of the $c\mu$-rule for ordinary multi-class single-server queues in different settings was proved; see e.g., [1,2]. The result was extended in [3] for the discrete-time dynamic scheduling problems of three types of general $G/G/1$ queue with K classes of customers. A detailed review of works on the $c\mu$-rule can also be found in M.Sc. thesis [4]. In [5], the authors analyzed the $c\mu$-rule for queueing models with non-linear costs. The optimality of this rule for discrete-time queueing models with a general distributed inter-arrival time and geometrically distributed service time was established in [6]. The concept of a generalized $c\mu$-rule was proposed in [5], where it was shown that this rule is asymptotically optimal for non-decreasing convex delay costs. A classic scheduling problem with a single resource shared by two competing queues was considered in [7] in the context of the stochastic flow model, where it was shown that the $c\mu$-rule is optimal. The non-preemptive assignment of a single server to two infinite-capacity retrial queues was analyzed in [8], where the $c\mu$-rule was optimal for a scheduling problem in a system without arrivals. The authors in [9] considered the learning-based variants of the $c\mu$-rule where the service rates $\mu$ are unknown.

Most often, systems with retrials assume that customers make repeated attempts independently of each other to occupy the server after a random time. Such systems are referred to as the systems with level-dependent retrial rates or with classical retrial policies. A detailed review of such systems can be found, e.g., in [10,11]. Nevertheless, in computer networks, the conditions according to the classical retrial policy are not always fulfilled. Sometimes, there is a situation when the customer at the head of the queue in the ordinary queueing system has no information about the state of the server and retries for service after a random time. There may also be situations where the server needs to check whether a transmission facility is available, or it may need time to find a specified customer. In these tasks, the retrial intensity is independent of the number of customers in the orbit; that is, a constant retry policy is quite appropriate to describe the behavior of customers when retries occur. The constant retrial policy was introduced in [12], where it was assumed that only the customer at the head of the queue can request service after an exponentially distributed retrial time. The single-class queueing systems with constant retrial rates and different options were analyzed intensively, e.g., in [13,14,15] and other papers. The uncontrolled analog of a two-class queueing system has also been investigated by a number of authors. For example, in [16], the authors studied a two-class system with a single exponential service requirement and constant retrial policy. The regenerative approach was used there to derive a set of necessary stability conditions of such a system. However, in this system, no dynamic control was assumed to be possible. The generating function of the stationary distribution of the number of orbiting customers at service completion epochs was obtained in [17]. Among recent works on parallel queues with constant retrial policies, we can mention a study proposed in [18], where stability conditions for both an uncontrolled system and a $c\mu$-controlled system were derived.

This paper deals with a controllable unreliable single-server two-class queueing system with constant retrial rates. The optimal allocation problem for this queueing model with retrial and reliability attributes is a new one, and our research aims are different from those previously studied. The emphasis of the paper is on answering the question of how robust the $c\mu$-rule is as an optimal allocation policy in the queueing system under study. The queueing system is studied without and with arrivals. In the first case, explicit relations of the $c\mu$-rule can be derived. In the second case, the relations for the optimality of a variant of the $c\mu$-rule were obtained for the model with a certain constraint on the arrival process.

In Section 2, the queueing model is described. The main results, including the analysis of the optimal allocation policy and some numerical experiments, are presented in Section 3.

2. Model Description

We analyze the Markovian single-server queueing system servicing two classes of customers, as illustrated in Figure 1. The customers of each class $i=1,2$ arrive at the system according to a Poisson stream with a rate ${\lambda }_i$. Independently of the state of the server, the class-i customers join the corresponding waiting line or queue with infinite capacity $N_i=\infty$ upon arrival. The service of customers from the queue occurs according to a FIFO retrial discipline; i.e., the customer waiting at the head of the queue retries to occupy the server in exponentially distributed time with a rate ${\theta }_i$. The service rate of the ith class customer is ${\mu }_i$. The server during a service process of the ith-class customer can fail in exponentially distributed time with a rate ${\alpha }_i$. In this case, the customer leaves the service area and joins the head of its queue again. In the failed state, the server can be repaired in exponentially distributed time with a rate $\beta$. All types of time intervals are assumed to be mutually independent. At moments of retrial arrival, the idle server may accept the customer of a certain class who attempts to occupy the server or can deny the service. The system performance is described by the steady state average cost, which is of the form

$$\begin{array}{c}\hfill g^f=c_1{L}_1^f+c_2{L}_2^f,\end{array}$$

(1)

where $L_i^f$ is the average number of the ith-class customers present in the system, given the allocation policy is f and $c_i$ is the holding cost per unit of time the ith-class customer spends in the system. The objective of the present analysis is to provide how robust a static policy f is, defined as a $c\mu$-rule for the system under study by minimizing the average cost per unit of time (1).

Denote by $Q_i\left(t\right)$ the number of ith-class customers in the system at time t and by $D\left(t\right)$ the state of the server at time t, which is defined as follows:

$$D\left(t\right)=\left\{\begin{array}{cc}0\hfill & \mathrm{the}\mathrm{server}\mathrm{is}\mathrm{idle},\hfill \\ 1\hfill & \mathrm{the}\mathrm{server}\mathrm{services}\mathrm{the}1\mathrm{st}\mathrm{class}\mathrm{customer},\hfill \\ 2\hfill & \mathrm{the}\mathrm{server}\mathrm{services}\mathrm{the}2\mathrm{nd}\mathrm{class}\mathrm{customer},\hfill \\ 3\hfill & \mathrm{the}\mathrm{server}\mathrm{is}\mathrm{failed}.\hfill \end{array}\right.$$

Consider the three-dimensional continuous-time Markov chain

$$\begin{array}{c}\hfill {\left\{X\left(t\right)\right\}}_{t\ge 0}={\{Q_1\left(t\right),Q_2\left(t\right),D\left(t\right)\}}_{t\ge 0},\end{array}$$

(2)

with a state space

$$E=\{x=(q_1,q_2,d):q_1,q_2\in {\mathbb{N}}_0,d\in \{0,1,2,3\}\}$$

and policy-dependent infinitesimal matrix ${\Lambda }^f=\left[{\lambda }_{xy}^f\right]$ with components for $x\ne y$,

$$\begin{array}{c}\hfill {\lambda }_{xy}^f=\left\{\begin{array}{cc}{\lambda }_i\hfill & y=x+{\mathbf{e}}_i,\hfill \\ {\mu }_i\hfill & y=x-i{\mathbf{e}}_3,d\left(x\right)=i,\hfill \\ {\theta }_i\hfill & y=x-{\mathbf{e}}_i+i{\mathbf{e}}_3,d\left(x\right)=0,f\left(x\right)=i,\hfill \\ {\alpha }_i\hfill & y=x+{\mathbf{e}}_i-i{\mathbf{e}}_3,d\left(x\right)=i,\hfill \\ \beta \hfill & y=x-3{\mathbf{e}}_3,\hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

(3)

where ${\mathbf{e}}_i$ stands for a vector of dimension 3 with 1 in the ith position, $i=1,2,3$, and 0 elsewhere. Here, f is a control policy which prescribes the allocation of the ith-class customer to the server at the moments of retrial arrival. The set of admissible control actions is defined by $A=\{1,2\}$, where $f\left(x\right)=i\in A$ means that upon retrial arrival in state x, the ith-class customer must be accepted for service. The sets of admissible control actions in state x are denoted by $A\left(x\right)\subseteq A$, where $A\left(x\right)=A$ for $x=(q_1,q_2,0)$, $q_1,q_2\ge 1$, $A\left(x\right)=1$ for $x=(q_1,0,0)$ and $A\left(x\right)=2$ for $x=(0,q_2,0)$. It is assumed that the stability condition is fulfilled. According to a general result for the $M/G/1$ system with parallel queues, see, e.g., [2]; the system is stable if the total load $\rho ={\rho }_1+{\rho }_2<1$, where ${\rho }_i$ is the load of the class-i queue. The value ${\rho }_i$ can be obtained if we treat the class-i queue as an independent single-server queueing system with parameters ${\lambda }_i$, ${\mu }_i$, ${\theta }_i$, ${\alpha }_i$, and $\beta$. The corresponding continuous-time Markov-chain is then a quasi-birth-and-death (QBD) process with a three-diagonal block infinitesimal matrix:

$$\begin{array}{cc}\hfill & Q=\left(\begin{array}{ccccc}{B}_0& B_1& 0& 0& \\ A_2& A_1& A_0& 0& \\ 0& A_2& A_1& A_0& \cdots \\ 0& 0& A_2& A_1& \cdots \\ & & ⋮& ⋮\end{array}\right),\mathrm{where}\hfill \\ \hfill & B_0=\left(\begin{array}{ccc}-{\lambda }_i& 0& 0\\ {\mu }_i& -({\lambda }_i+{\mu }_i+{\alpha }_i)& 0\\ 0& 0& 1\end{array}\right),B_1=\left(\begin{array}{ccc}{\lambda }_i& 0& 0\\ 0& {\lambda }_i& {\alpha }_i\\ 0& 0& 0\end{array}\right),\hfill \\ \hfill & A_1=\left(\begin{array}{ccc}-({\lambda }_i+{\theta }_i)& 0& 0\\ {\mu }_i& -({\lambda }_i+{\mu }_i+{\alpha }_i)& 0\\ \beta & 0& -({\lambda }_i+\beta ),\end{array}\right),\hfill \\ \hfill & A_0=\left(\begin{array}{ccc}{\lambda }_i& 0& 0\\ 0& {\lambda }_i& {\alpha }_i\\ 0& 0& {\lambda }_i\end{array}\right),A_2=\left(\begin{array}{ccc}0& {\theta }_i& 0\\ 0& 0& 0\\ 0& 0& 0\end{array}\right).\hfill \end{array}$$

According to a matrix-analytic approach for QBD-processes [19], the queue i is stable if the following inequality holds:

$$\begin{array}{c}\hfill \mathbf{p}{A}_2\mathbf{e}>\mathbf{p}{A}_0\mathbf{e},\mathbf{e}={(1,1,1)}^{\prime }.\end{array}$$

(4)

where $\mathbf{p}=(p_1,p_2,p_3)$ is a stationary distribution for the infinitesimal transition matrix

$$A=A_0+A_1+A_2=\left(\begin{array}{ccc}-{\theta }_i& {\theta }_i& 0\\ {\mu }_i& -({\alpha }_i+{\mu }_i)& {\alpha }_i\\ \beta & 0& -\beta \end{array}\right)$$

which satisfies the system $\mathbf{p}A=\mathbf{0}$ and $\mathbf{p}\mathbf{e}=1$. The solution of the system is given by

$$\begin{array}{c}\hfill p_1=\frac{({\alpha }_i+{\mu }_i)\beta }{C},p_2=\frac{\beta {\theta }_i}{C},p_3=\frac{\alpha {\theta }_i}{C},\end{array}$$

(5)

where $C={\alpha }_i(\beta +{\theta }_i)+\beta ({\mu }_i+{\theta }_i)$. By substituting the solution (5) into (4) we obtain

$$\begin{array}{c}\hfill {\rho }_i=\frac{{\lambda }_{i}C}{{\beta }_i{\mu }_i{\theta }_i}={\lambda }_i\frac{{\alpha }_i+{\mu }_i}{{\mu }_i{\theta }_i}\left(\frac{{\theta }_i}{\beta }\frac{{\alpha }_i+\beta }{{\alpha }_i+{\mu }_i}+1\right)<1.\end{array}$$

and the stability condition is then defined as

$$\begin{array}{c}\hfill \sum _{i=1}^2{\lambda }_i\frac{{\alpha }_i+{\mu }_i}{{\mu }_i{\theta }_i}\left(\frac{{\theta }_i}{\beta }\frac{{\alpha }_i+\beta }{{\alpha }_i+{\mu }_i}+1\right)<1.\end{array}$$

(6)

Below, we present the main results of this paper obtained for a system with two classes of customers. However, these results can also be generalized to the case of an arbitrary number of classes.

3. Optimal Allocation Problem

Consider first a classic scheduling problem for the allocation of customers in the system without arrivals, i.e., when ${\lambda }_1={\lambda }_2=0$, in which the customers are queued. The waiting customers must be served in such a way that the overall average cost is minimized. It is assumed that the allocation policy $f(q_1,0,0)=1$ and $f(0,q_2,0)=2$ for $q_1,q_2\ge 1$. Here, we have a classical scheduling problem.

Proposition 1.

In state $x=(q_1,q_2,0)$, the optimal allocation policy can be defined in the form of a $c\mu$-rule:

$$\begin{array}{c}\hfill f\left(x\right)=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{c}_2{\tilde{\mu }}_1\le c_1{\tilde{\mu }}_2,\hfill \\ 2,\hfill & \mathrm{if}{c}_2{\tilde{\mu }}_1\ge c_1{\tilde{\mu }}_2\hfill \end{array}\right.,\end{array}$$

(7)

where ${\tilde{\mu }}_i=m_i\left(\frac{{\alpha }_i+\beta }{m_i{\mu }_i\beta }+1\right)$, $m_i=\frac{{\alpha }_i+{\mu }_i}{{\mu }_i{\theta }_i}$, $i=1,2$. In the case of equality $c_2{\tilde{\mu }}_1=c_1{\tilde{\mu }}_2$, the control actions 1 and 2 are equivalent.

Proof. 

Denote by $V\left(x\right)$ the total minimal average cost given the initial state is $x\in E$. This function is given by

$$\begin{array}{cc}\hfill & V(q_1,q_2,0)=min\left\{\frac{1}{{\theta }_1}(q_1{c}_1+q_2{c}_2)+V(q_1-1,q_2,1),\frac{1}{{\theta }_2}(q_1{c}_1+q_2{c}_2)+V(q_1,q_2-1,2)\right\},\hfill \\ \hfill & q_1,q_2\ge 1,\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill & V(q_1,q_2,1)=\frac{1}{{\mu }_1+{\alpha }_1}((q_1+1)c_1+q_2{c}_2)+\frac{{\mu }_1}{{\mu }_1+{\alpha }_1}V(q_1,q_2,0)+\frac{{\alpha }_1}{{\mu }_1+{\alpha }_1}V(q_1+1,q_2,3),\hfill \\ \hfill & q_1,q_2\ge 0,\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill & V(q_1,q_2,2)=\frac{1}{{\mu }_2+{\alpha }_2}(q_1{c}_1+(q_2+1)c_2)+\frac{{\mu }_2}{{\mu }_2+{\alpha }_2}V(q_1,q_2,0)+\frac{{\alpha }_2}{{\mu }_2+{\alpha }_2}V(q_1,q_2+1,3),\hfill \\ \hfill & q_1,q_2\ge 0,\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill & V(q_1,q_2,3)=\frac{1}{\beta }(q_1{c}_1+q_2{c}_2)+V(q_1,q_2,0),q_1,q_2\ge 0.\hfill \end{array}$$

(11)

We need to prove that $f(q_1,q_2,0)=1$ if $c_2{\tilde{\mu }}_1<c_1{\tilde{\mu }}_2$, as defined in a proposition. First, we show that $f(q_1,q_2-1,0)=1$ implies $f(q_1,q_2,0)=1$. In the case $f(q_1,q_2,0)=2$ and $f(q_1,q_2-1,0)=1$, we obtain from (8), (10), and (11), after some simple algebra, the relation for $V(q_1,q_2,0)$,

$$\begin{array}{cc}\hfill V(q_1,q_2,0)& =\left(\frac{1}{{\mu }_2}+\frac{{\alpha }_2}{\beta {\mu }_2}+\frac{{\alpha }_2+{\mu }_2}{{\mu }_2{\theta }_2}\right)(q_1{c}_1+q_2{c}_2)+\left(\frac{1}{{\mu }_1}+\frac{{\alpha }_1}{\beta {\mu }_1}+\frac{{\alpha }_1+{\mu }_1}{{\mu }_1{\theta }_1}\right)(q_1{c}_1+(q_2-1)c_2)\hfill \\ \hfill & +V(q_1-1,q_2-1,0).\hfill \end{array}$$

(12)

In the case $f(q_1,q_2,0)=1$, if in state $(q_1,q_2-1,0)$, action 2 is chosen instead of action 1, and we obtain the following inequality:

$$\begin{array}{cc}\hfill V(q_1,q_2,0)& \ge \left(\frac{1}{{\mu }_1}+\frac{{\alpha }_1}{\beta {\mu }_1}+\frac{{\alpha }_1+{\mu }_1}{{\mu }_1{\theta }_1}\right)(q_1{c}_1+q_2{c}_2)+\left(\frac{1}{{\mu }_2}+\frac{{\alpha }_2}{{\mu }_2\beta }+\frac{{\alpha }_2+{\mu }_2}{{\mu }_2{\theta }_2}\right)((q_1-1)c_1+q_2{c}_2)\hfill \\ \hfill & +V(q_1-1,q_2-1,0).\hfill \end{array}$$

(13)

Now, if $f(q_1,q_2,0)=1$ is optimal, then the difference of expressions (12) and (13) must be positive; i.e.,

$$\begin{array}{cc}\hfill & c_1\left(\frac{1}{{\mu }_2}+\frac{{\alpha }_2}{\beta {\mu }_2}+\frac{{\alpha }_2+{\mu }_2}{{\mu }_2{\theta }_2}\right)-c_2\left(\frac{1}{{\mu }_1}+\frac{{\alpha }_1}{\beta {\mu }_1}+\frac{{\alpha }_1+{\mu }_1}{{\mu }_1{\theta }_1}\right)\hfill \\ \hfill & =c_1\frac{{\alpha }_2+{\mu }_2}{{\mu }_2{\theta }_2}\left(\frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}+1\right)-c_2\frac{{\alpha }_1+{\mu }_1}{{\mu }_1{\theta }_1}\left(\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}+1\right)\hfill \\ \hfill & =c_1{m}_2\left(\frac{{\alpha }_2+\beta }{m_2{\mu }_2\beta }+1\right)-c_2{m}_1\left(\frac{{\alpha }_1+\beta }{m_1{\mu }_1\beta }+1\right)\ge 0.\hfill \end{array}$$

(14)

which coincides with the statement.

In the case of equality $c_2{\tilde{\mu }}_1=c_1{\tilde{\mu }}_2$, the control actions 1 and 2 are equivalent; i.e., $f\left(x\right)=1\equiv 2$ for $x=(q_1,q_2,0)$. We show that the optimal policy $f(q_1-1,q_2,0)=1\equiv 2$ and $f(q_1,q_2-1,0)=1\equiv 2$ implies that $f(q_1,q_2,0)=1\equiv 2$. Note that $f(0,q_2,0)=2$ and $f(q_1,0,0)=1$ for $q_1,q_2\ge 1$. For the minimum in (8), in the case $f(q_1,q_2-1,0)=1\equiv 2$, we obtain then for the control action $f(q_1,q_2,0)=2$ the relation

$$\frac{1}{{\theta }_2}(q_1{c}_1+q_2{c}_2)+V(q_1,q_2-1,2)$$

in the form (12). For the control action $f(q_1,q_2,0)=1$, the relation

$$\frac{1}{{\theta }_1}(q_1{c}_1+q_2{c}_2)+V(q_1-1,q_2,1)$$

is equal to the right-hand side of the inequality (13). The difference in the total average cost given by the relation in (14) for equivalent actions must be equal to zero. Therefore, if $c_2{\tilde{\mu }}_1=c_1{\tilde{\mu }}_2$, the control actions 1 and 2 are equivalent. □

It is assumed now that new customers can arrive at the system, i.e., ${\lambda }_1>0$, ${\lambda }_2>0$. We expect that the same $c\mu$-rule defined in (7) will be optimal for the system with arrivals, but for technical reasons, it is quite difficult to derive expressions for the mean overall service times of the ith-class customers. Therefore, to analyze the properties of an optimal control policy, we have to introduce a queueing model with a constraint on possible arrivals. This model differs from the original one since we assume that a new arrival can occur only in state $x=(q_1,q_2,0)$, $q_1,q_2\ge 0$, where the server is empty. Without the proposed constraint, it is impossible to obtain recurrence relations for the dynamic programming value function v, which could be used in the proof by induction. But, we assume that the structure of the optimal control policy will not change due to this constraint since decision making is only performed in the state where the server is idle. Incoming new customers in states with a busy server will only lead to a proportional growth of the corresponding queues, which should not affect the structure of the optimal control. The dynamic programming approach, see, e.g., [20,21,22], is used to establish the properties of the optimal control policy in the following Proposition. Note that the state space of the corresponding Markov decision process (MDP) is infinite and the costs are unbounded. The existence of an average cost optimal stationary policy and convergence of the value iteration algorithm can be verified in the same way as it was in [23], where the authors generalized the existence of the optimal policy for the discounted expected total cost minimization to the average cost criterion.

Proposition 2.

In state $x=(q_1,q_2,0)$, the optimal allocation policy for the system with a constraint on arrivals can be defined in the form

$$\begin{array}{c}\hfill f\left(x\right)=\left\{\begin{array}{cc}1,\hfill & if{c}_2{m}_1\le c_1{m}_2,\frac{{\alpha }_1+\beta }{m_1{\mu }_1}\le \frac{{\alpha }_2+\beta }{m_2{\mu }_2},\hfill \\ 2\hfill & if{c}_2{m}_1\ge c_1{m}_2,\frac{{\alpha }_1+\beta }{m_1{\mu }_1}\ge \frac{{\alpha }_2+\beta }{m_2{\mu }_2}\hfill \end{array}\right..\end{array}$$

(15)

In the case of equalities $c_2{m}_1=c_1{m}_2$ and $\frac{{\alpha }_1+\beta }{m_1{\mu }_1}=\frac{{\alpha }_2+\beta }{m_2{\mu }_2}$, the control actions 1 and 2 are equivalent.

Proof. 

For the introduced cost structure, the average cost stationary optimal policy exists. This policy can be found as a solution of the system of optimality equations for the dynamic programming relative value function $v:E\to \mathbb{R}$ and gain g,

$$\begin{array}{cc}\hfill v(q_1,q_2,0)& =q_1{c}_1+q_2{c}_2-g+{\lambda }_{1}v(q_1+1,q_2,0)+{\lambda }_{2}v(q_1,q_2+1,0)\hfill \\ \hfill & +min\{{\theta }_{1}v(q_1-1,q_2,1)+(1-{\theta }_1-{\lambda }_1-{\lambda }_2)v(q_1,q_2,0),\hfill \\ \hfill & {\theta }_{2}v(q_1,q_2-1,2)+(1-{\theta }_2-{\lambda }_1-{\lambda }_2)v(q_1,q_2,0)\},\hfill \end{array}$$

(16)

where $q_1,q_2\ge 1$ and $c(q_1,q_2,0)=q_1{c}_1+q_2{c}_2$ are an immediate cost of the corresponding MDP. In states with the only one nonempty queue, the optimal policy consists of the service of the corresponding customer, and the relative value functions are of the form

$$\begin{array}{cc}\hfill v(q_1,0,0)& =q_1{c}_1-g+{\lambda }_{1}v(q_1+1,0,0)+{\lambda }_{2}v(q_1,1,0)\hfill \\ \hfill & +{\theta }_{1}v(q_1-1,0,1)+(1-{\theta }_1-{\lambda }_1-{\lambda }_2)v(q_1,0,0),\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill v(0,q_2,0)& =q_2{c}_2-g+{\lambda }_{1}v(1,q_2,0)+{\lambda }_{2}v(0,q_2+1,0)\hfill \\ \hfill & +{\theta }_{2}v(0,q_2-1,2)+(1-{\theta }_2-{\lambda }_1-{\lambda }_2)v(0,q_2,0),\hfill \end{array}$$

(18)

where $q_1,q_2\ge 1$. The value $v(0,0,0)=0$, and then $g={\lambda }_{1}v(1,0,0)+{\lambda }_{2}v(0,1,0)$. The equations for states where the server is busy or failed are

$$\begin{array}{cc}\hfill v(q_1,q_2,1)& =(q_1+1)c_1+q_2{c}_2-g+{\mu }_{1}v(q_1,q_2,0)+{\alpha }_{1}v(q_1+1,q_2,3)\hfill \\ \hfill & +(1-{\mu }_1-{\alpha }_1)v(q_1,q_2,1),\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill v(q_1,q_2,2)& =q_1{c}_1+(q_2+1)c_2-g+{\mu }_{2}v(q_1,q_2,0)+{\alpha }_{2}v(q_1,q_2+1,3)\hfill \\ \hfill & +(1-{\mu }_2-{\alpha }_2)v(q_1,q_2,2),\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill v(q_1,q_2,3)& =q_1{c}_1+q_2{c}_2+\beta v(q_1,q_2,0)+(1-\beta )v(q_1,q_2,3).\hfill \end{array}$$

(21)

Equation (16) can be expressed using (19)–(21) in the following way:

$$\begin{array}{cc}\hfill v(q_1,q_2,0)& =q_1{c}_1+q_2{c}_2-g+{\lambda }_{1}v(q_1+1,q_2,0)+{\lambda }_{2}v(q_1,q_2+1,0)\hfill \\ \hfill & +min\{\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}(q_1{c}_1+q_2{c}_2-g)+\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}v(q_1-1,q_2,0)\hfill \\ \hfill & +\left(\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\right)v(q_1,q_2,0),\hfill \\ \hfill & \frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}(q_1{c}_1+q_2{c}_2-g)+\frac{{\mu }_2{\theta }_2}{{\alpha }_2+{\mu }_2}v(q_1,q_2-1,0)\hfill \\ \hfill & +\left(\frac{{\alpha }_2{\theta }_2}{{\alpha }_2+{\mu }_2}+(1-{\lambda }_1-{\lambda }_2-{\theta }_2)\right)v(q_1,q_2,0)\}.\hfill \end{array}$$

(22)

We rewrite Equations (17) and (18) in the same way:

$$\begin{array}{cc}\hfill v(q_1,0,0)& =q_1{c}_1-g+{\lambda }_{1}v(q_1+1,0,0)+{\lambda }_{2}v(q_1,1,0)+\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}(c_1{q}_1-g)\hfill \\ \hfill & +\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}v(q_1-1,0,0)+\left(\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\right)v(q_1,0,0),\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill v(0,q_2,0)& =q_2{c}_2-g+{\lambda }_{1}v(1,q_2,0)+{\lambda }_{2}v(0,q_2+1,0)+\frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}(c_2{q}_2-g)\hfill \\ \hfill & +\frac{{\mu }_2{\theta }_2}{{\alpha }_2+{\mu }_2}v(0,q_2-1,0)+\left(\frac{{\alpha }_2{\theta }_2}{{\alpha }_2+{\mu }_2}+(1-{\lambda }_1-{\lambda }_2-{\theta }_2)\right)v(0,q_2,0).\hfill \end{array}$$

(24)

The solution for the system of optimality equations can be calculated recursively using an equivalent discrete-time model on a finite horizon obtained by a uniformization procedure. The corresponding recursive relations have almost the same structure. Namely, the Equations (22)–(24) can be rewritten in such a way that on the left-hand side, the function $v\left(x\right)$ is replaced by the $n+1$-stage cost function $v_{n+1}\left(x\right)$, and on the right-hand side, we put $g=0$, and function $v\left(x\right)$ is replaced by the n-stage cost function $v_n\left(x\right)$. It is assumed that the initial condition is $v_0\left(x\right)=0$, $x\in E$. Let the inequalities in the first row of (15) hold. Consider the term for action selection in the obtained recursive relations. In the case

$$\begin{array}{cc}\hfill & \frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}(q_1{c}_1+q_2{c}_2)+\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}{v}_n(q_1-1,q_2,0)\hfill \\ \hfill & +\left(\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\right)v_n(q_1,q_2,0)\hfill \\ \hfill & \le \frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}(q_1{c}_1+q_2{c}_2)+\frac{{\mu }_2{\theta }_2}{{\alpha }_2+{\mu }_2}{v}_n(q_1,q_2-1,0)\hfill \\ \hfill & +\left(\frac{{\alpha }_2{\theta }_2}{{\alpha }_2+{\mu }_2}+(1-{\lambda }_1-{\lambda }_2-{\theta }_2)\right)v_n(q_1,q_2,0)\hfill \end{array}$$

(25)

for any $n\in {\mathbb{N}}_0$, the optimal control action is $f(q_1,q_2,0)=1$ for an arbitrary n. The statement is proved by induction on n. If $n=0$, the inequality (25) obviously holds. Assume the validity of this inequality for some n. Then, it must be proved that (25) holds for $n\to n+1$. Expressions for $v_{n+1}(q_1-1,q_2,0)$, $v_{n+1}(q_1,q_2-1,0)$, and $v_{n+1}(q_1,q_2,0)$ can be obtained from (22)–(24). The first terms by $q_1{c}_1+q_2{c}_2$ in inequality (25) are multiplied by the factor $\nu =1$ defined as

$$\begin{array}{c}\hfill \nu ={\lambda }_1+{\lambda }_2+\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+\left(1-{\lambda }_1-{\lambda }_2-{\theta }_1\right).\end{array}$$

In the case $q_1\ge 2$, $q_2\ge 1$, we obtain from (25) the following inequality:

$$\begin{array}{cc}\hfill & \nu \frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}(q_1{c}_1+q_2{c}_2)+\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}[\left(\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}+1\right)((q_2-1)c_1+q_2{c}_2)\hfill \\ \hfill & +{\lambda }_1{v}_n(q_1,q_2,0)+{\lambda }_2{v}_n(q_1-1,q_2+1,0)+\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}{v}_n(q_1-2,q_2,0)\hfill \\ \hfill & +\left(\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\right)v_n(q_1-1,q_2,0)]\hfill \\ \hfill & +\left(\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\right)[\left(\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}+1\right)\times \hfill \\ \hfill & \times (q_1{c}_1+q_2{c}_2)+{\lambda }_1{v}_n(q_1+1,q_2,0)+{\lambda }_2{v}_n(q_1,q_2+1,0)\hfill \\ \hfill & +\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}{v}_n(q_1-1,q_2,0)+\left(\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\right)v_n(q_1,q_2,0)]\hfill \\ \hfill & \le \nu \frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}(q_1{c}_1+q_2{c}_2)+\frac{{\mu }_2{\theta }_2}{{\alpha }_2+{\mu }_2}[\left(\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}+1\right)(q_1{c}_1+(q_2-1)c_2)\hfill \\ \hfill & +{\lambda }_1{v}_n(q_1+1,q_2-1,0)+{\lambda }_2{v}_n(q_1,q_2,0)+\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}{v}_n(q_1-1,q_2-1,0)\hfill \\ \hfill & +\left(\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\right)v_n(q_1,q_2-1,0)]\hfill \\ \hfill & +\left(\frac{{\alpha }_2{\theta }_2}{{\alpha }_2+{\mu }_2}+(1-{\lambda }_1-{\lambda }_2-{\theta }_2)\right)[\left(\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}+1\right)\times \hfill \\ \hfill & \times (q_1{c}_1+q_2{c}_2)+{\lambda }_1{v}_n(q_1+1,q_2,0)+{\lambda }_2{v}_n(q_1,q_2+1,0)\hfill \\ \hfill & +\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}{v}_n(q_1-1,q_2,0)+\left(\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\right)v_n(q_1,q_2,0)],\hfill \end{array}$$

(26)

where we multiply the first term in the left- and right-hand side by the factor $\nu =1$. The inequality (26) is valid due to the following reasons. For the terms with the factor ${\lambda }_1$, by adding to the left-hand side the item $c_1\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}$ and to the right-hand side the item $c_1\frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}$, we obtain

$$\begin{array}{cc}\hfill & \frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}((q_1+1)c_1+q_2{c}_2)+\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}{v}_n(q_1,q_2,0)\hfill \\ \hfill & +\left(\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\right)v_n(q_1+1,q_2,0)\le \hfill \\ \hfill & \frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}((q_1+1)c_1+q_2{c}_2)+\frac{{\mu }_2{\theta }_2}{{\alpha }_2+{\mu }_2}{v}_n(q_1+1,q_2-1,0)\hfill \\ \hfill & +\left(\frac{{\alpha }_2{\theta }_2}{{\alpha }_2+{\mu }_2}+(1-{\lambda }_1-{\lambda }_2-{\theta }_2)\right)v_n(q_1+1,q_2,0).\hfill \end{array}$$

(27)

The inequality (27) holds due to the induction assumption (25) in state $x=(q_1+1,q_2,0)$. In the same way, prove the inequality for the terms with the factor ${\lambda }_2$. In this case, we add to the left-hand side and right-hand side of the corresponding inequality, respectively, the elements $c_2\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}$ and to the right-hand side $c_2\frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}$. The induction assumption (25) in state $x=(q_1,q_2+1,0)$ can be applied. The inequality obtained for the terms with a factor $\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}$ by subtracting $c_1\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}$ and $c_1\frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}$, respectively, from the left-hand side and right-hand side holds as well due to the induction assumption in state $x=(q_1-1,q_2,0)$. The inequalities for the terms with factors $\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}$ and $(1-{\lambda }_1-{\lambda }_2-\theta )$ directly satisfy the induction assumption (25). The rest of the terms build the inequality of the form

$$\begin{array}{cc}\hfill & \frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}\left(-{\lambda }_1{c}_1-{\lambda }_2{c}_2+c_1\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}\right)+\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}\left(\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}+1\right)((q_1-1)c_1+q_2{c}_2)\hfill \\ \hfill & +\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}\left(\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}+1\right)(q_1{c}_1+q_2{c}_2)+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\left(\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}+1\right)(q_1{c}_1+q_2{c}_2)\hfill \\ \hfill & \le \frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}\left(-{\lambda }_1{c}_1-{\lambda }_2{c}_2+c_1\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}\right)+\frac{{\mu }_2{\theta }_2}{{\alpha }_2+{\mu }_2}\left(\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}+1\right)(q_1{c}_1+(q_2-1)c_2)\hfill \\ \hfill & +\frac{{\alpha }_2{\theta }_2}{{\alpha }_2+{\mu }_2}\left(\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}+1\right)(q_1{c}_1+q_2{c}_2)+(1-{\lambda }_1-{\lambda }_2-{\theta }_2)\left(\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}+1\right)(q_1{c}_1+q_2{c}_2).\hfill \end{array}$$

(28)

After some simple algebra, we obtain from (28) the inequality

$$\begin{array}{cc}\hfill & \frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}\left(-{\lambda }_1{c}_1-{\lambda }_2{c}_2+c_2\frac{{\mu }_2{\theta }_2}{{\alpha }_2+{\mu }_2}\right)-c_1\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}\hfill \\ \hfill & \le \frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}\left(-{\lambda }_1{c}_1-{\lambda }_2{c}_2+c_1\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}\right)-c_2\frac{{\mu }_2{\theta }_2}{{\alpha }_2+{\mu }_2}\hfill \end{array}$$

or by means of the variable $m_i$, it is of the form

$$\begin{array}{cc}\hfill & \frac{{\alpha }_1+\beta }{m_1{\mu }_1\beta }\left(-{\lambda }_1{c}_1-{\lambda }_2{c}_2+\frac{c_2}{m_2}\right)-\frac{c_1}{m_1}\hfill \\ \hfill & \le \frac{{\alpha }_2+\beta }{m_2{\mu }_2\beta }\left(-{\lambda }_1{c}_1-{\lambda }_2{c}_2+\frac{c_1}{m_1}\right)-\frac{c_2}{m_2}.\hfill \end{array}$$

(29)

The inequality (29) holds due to the assumptions for the control action 1; i.e., $\frac{c_2}{m_2}\le \frac{c_1}{m_1}$ and $\frac{{\alpha }_1+\beta }{m_1{\mu }_1}\le \frac{{\alpha }_2+\beta }{m_2{\mu }_2}$ and the inequality

$${\lambda }_1{m}_1+{\lambda }_2{m}_2={\lambda }_1\frac{{\alpha }_1+{\mu }_1}{{\mu }_1{\theta }_1}+{\lambda }_2\frac{{\alpha }_2+{\mu }_2}{{\mu }_2{\theta }_2}<1$$

obtained directly from the stability condition (6). In fact, the expressions in brackets of the inequality (29) can be rewritten, respectively, as

$$\begin{array}{c}\hfill \frac{c_2}{m_2}(1-{\lambda }_2{m}_2)-\frac{c_1}{m_1}{\lambda }_1{m}_1\mathrm{and}\frac{c_1}{m_1}(1-{\lambda }_1{m}_1)-\frac{c_2}{m_2}{\lambda }_2{m}_2\ge 0.\end{array}$$

(30)

The second expression in (30) is obviously non-negative due to conditions $\frac{c_1}{m_1}\ge \frac{c_2}{m_2}$ and $1-{\lambda }_1{m}_1>{\lambda }_2{m}_2$. If

$$\frac{c_2}{m_2}(1-{\lambda }_2{m}_2)-\frac{c_1}{m_1}{\lambda }_1{m}_1\le 0,$$

then (29) is true. If

$$\frac{c_2}{m_2}(1-{\lambda }_2{m}_2)-\frac{c_1}{m_1}{\lambda }_1{m}_1\ge 0,$$

then

$$\frac{c_2}{m_2}(1-{\lambda }_2{m}_2)-\frac{c_1}{m_1}{\lambda }_1{m}_1-\frac{c_1}{m_1}(1-{\lambda }_1{m}_1)+\frac{c_2}{m_2}{\lambda }_2{m}_2=\frac{c_2}{m_2}-\frac{c_1}{m_1}\le 0,$$

which confirms the validity of the inequality (29).

The inequality (25) for $q_1=1$ can be proved using the same technique as before. Indeed, the inequality (26) is converted to

$$\begin{array}{cc}\hfill & \nu \frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}(c_1+q_2{c}_2)+\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}[\left(\frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}+1\right)q_2{c}_2+{\lambda }_1{v}_n(1,q_2,0)+{\lambda }_2{v}_n(0,q_2+1,0)\hfill \\ \hfill & +\frac{{\mu }_2{\theta }_2}{{\alpha }_2+{\mu }_2}{v}_n(0,q_2-1,0)+\left(\frac{{\alpha }_2{\theta }_2}{{\alpha }_2+{\mu }_2}+(1-{\lambda }_1-{\lambda }_2-{\theta }_2)\right)v_n(0,q_2,0)]\hfill \\ \hfill & +\left(\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\right)[\left(\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}+1\right)\times \hfill \\ \hfill & \times (c_1+q_2{c}_2)+{\lambda }_1{v}_n(2,q_2,0)+{\lambda }_2{v}_n(1,q_2+1,0)\hfill \\ \hfill & +\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}{v}_n(0,q_2,0)+\left(\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\right)v_n(1,q_2,0)]\le \hfill \\ \hfill & \nu \frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}(c_1+q_2{c}_2)+\frac{{\mu }_2{\theta }_2}{{\alpha }_2+{\mu }_2}[\left(\frac{{\theta }_1}{\beta }\frac{{\alpha }_1+\beta }{{\alpha }_1+{\mu }_1}+1\right)\times \hfill \\ \hfill & \times (c_1+(q_2-1)c_2)+{\lambda }_1{v}_n(2,q_2-1,0)+{\lambda }_2{v}_n(1,q_2,0)+\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}{v}_n(0,q_2-1,0)\hfill \\ \hfill & +\left(\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\right)v_n(1,q_2-1,0)]\hfill \\ \hfill & +\left(\frac{{\alpha }_2{\theta }_2}{{\alpha }_2+{\mu }_2}+(1-{\lambda }_1-{\lambda }_2-{\theta }_2)\right)[\left(\frac{{\theta }_2}{\beta }\frac{{\alpha }_2+\beta }{{\alpha }_2+{\mu }_2}+1\right)\times \hfill \\ \hfill & \times (c_1+q_2{c}_2)+{\lambda }_1{v}_n(2,q_2,0)+{\lambda }_2{v}_n(1,q_2+1,0)\hfill \\ \hfill & +\frac{{\mu }_1{\theta }_1}{{\alpha }_1+{\mu }_1}{v}_n(0,q_2,0)+\left(\frac{{\alpha }_1{\theta }_1}{{\alpha }_1+{\mu }_1}+(1-{\lambda }_1-{\lambda }_2-{\theta }_1)\right)v_n(1,q_2,0)].\hfill \end{array}$$

By further comparing the terms of the corresponding inequality for the parameters of the system, we obtain inequality (29) using the proof by induction as for the case $q_2\ge 2$. And finally, we note that the inequalities in (15) automatically lead to the $c\mu$-rule defined in (7), but this does not always hold true in the reverse. □

Conjecture 1.

We expect that the policy defined by (15) is also optimal for the system where a constraint on arrival is omitted. This can be explained by the fact that the proportion of the class-i customers who arrived at the system during the time when the server was idle will also be maintained when customers arrive in states where the server is busy or in a failed state. Therefore, the incentive to service the customer of a certain class remains the same, and hence the policy (15) seems to be valid for the original queueing system with arrivals.

Example 1.

Consider the system with fixed parameters $({\lambda }_1,{\alpha }_1,\beta ,c_1)=(0.13,0.20,5,1.00)$ and the six cases of varied parameters given in

Table 1. Simulation results.

${\mathit{\lambda }}_2$	${\mathit{\mu }}_1$	${\mathit{\mu }}_2$	${\mathit{\theta }}_1$	${\mathit{\theta }}_2$	${\mathit{\alpha }}_2$	${\mathit{c}}_2$	${\mathit{g}}^{\mathit{f}=1}$	${\mathit{g}}^{\mathit{f}=2}$
0.30	5	3	0.55	0.5	0.10	1.00	25.657	29.403
0.26	5	3	0.70	0.40	0.10	2.00	36.909	34.238
0.26	5	3	0.70	0.43	0.12	1.64	18.223	18.291
0.30	3	5	0.51	0.5	0.10	1.00	25.430	22.955
0.30	3	5	0.51	0.5	0.10	0.90	23.421	24.709
0.30	5	3	0.55	0.5	0.10	1.20	33.348	31.610

. The last two columns of the table represent the values of the average cost g evaluated using a simulation technique for the policy with $f(q_1,q_2,0)=1$ and $f(q_1,q_2,0)=2$, $q_1,q_2\ge 1$.

The inequalities from (15) are

$$\begin{array}{cc}\hfill & Case1\text{:}{c}_2{\tilde{\mu }}_1=2.098<2.407=c_1{\tilde{\mu }}_2\Rightarrow f=1,\hfill \\ \hfill & c_2{m}_1=1.890<2.067=c_1{m}_2,\hfill \\ \hfill & \frac{{\alpha }_1+\beta }{m_1{\mu }_1}=0.550<0.823=\frac{{\alpha }_2+\beta }{m_2{\mu }_2};\hfill \\ \hfill & Case2\text{:}{c}_2{\tilde{\mu }}_1=3.387>2.923=c_1{\tilde{\mu }}_2\Rightarrow f=2,\hfill \\ \hfill & c_2{m}_1=2.971>2.583=c_1{m}_2,\hfill \\ \hfill & \frac{{\alpha }_1+\beta }{m_1{\mu }_1}=0.700>0.658=\frac{{\alpha }_2+\beta }{m_2{\mu }_2};\hfill \\ \hfill & Case3\text{:}{c}_2{\tilde{\mu }}_1=2.779=2.779=c_1{\tilde{\mu }}_2\Rightarrow f=1\equiv 2,\hfill \\ \hfill & c_2{m}_1=2.438=2.438=c_1{m}_2,\hfill \\ \hfill & \frac{{\alpha }_1+\beta }{m_1{\mu }_1}=0.700=0.700=\frac{{\alpha }_2+\beta }{m_2{\mu }_2};\hfill \\ \hfill & Case4\text{:}{c}_2{\tilde{\mu }}_1=2.438>2.244=c_1{\tilde{\mu }}_2\Rightarrow f=2,\hfill \\ \hfill & c_2{m}_1=2.091>2.040=c_1{m}_2,\hfill \\ \hfill & \frac{{\alpha }_1+\beta }{m_1{\mu }_1}=0.828>0.500=\frac{{\alpha }_2+\beta }{m_2{\mu }_2};\hfill \\ \hfill & Case5\text{:}{c}_2{\tilde{\mu }}_1=2.194<2.244=c_1{\tilde{\mu }}_2\Rightarrow f=1,\hfill \\ \hfill & c_2{m}_1=1.882<2.040=c_1{m}_2,\hfill \\ \hfill & \frac{{\alpha }_1+\beta }{m_1{\mu }_1}=0.829>0.500=\frac{{\alpha }_2+\beta }{m_2{\mu }_2};\hfill \\ \hfill & Case6\text{:}{c}_2{\tilde{\mu }}_1=2.519>2.407=c_1{\tilde{\mu }}_2\Rightarrow f=2,\hfill \\ \hfill & c_2{m}_1=2.269>2.067=c_1{m}_2,\hfill \\ \hfill & \frac{{\alpha }_1+\beta }{m_1{\mu }_1}=0.550<0.823=\frac{{\alpha }_2+\beta }{m_2{\mu }_2}.\hfill \end{array}$$

In Case 3, we obtain equivalent policies, and the simulation results for the average cost are very similar. In Cases 5 and 6, the relations are converse, but the optimal policy, as expected, still follows the rule $\left(\right)$.

For the system with finite buffer capacities, if the optimal allocation policy can have, in general, another structure, then the $c\mu$-rule is due to the influence of the boundary states. In the next example, we illustrate such a result.

Example 2.

Consider the system with the parameters of Case 4 from Example 1 and finite buffer capacity for both of queues $N_1=N_2=20$. The state-dependent optimal control actions $f(q_1,q_2,0)$ evaluated by a dynamic programming approach are summarized as a matrix represented in

Table 2. Optimal control actions $f(q_1,q_2,0)$.

${\mathit{q}}_2$\${\mathit{q}}_1$	0	1	2	3	4	5	6	7	8	9	10	⋯
0	0	2	2	2	2	2	2	2	2	2	2	...
1	1	2	2	2	2	2	2	2	2	2	2	...
2	1	2	2	2	2	2	2	2	2	2	2	...
3	1	2	2	2	2	2	2	2	2	2	1	...
4	1	2	2	2	2	2	2	2	2	1	1	...
5	1	2	2	2	2	2	2	1	1	1	1	...
6	1	2	2	2	2	1	1	1	1	1	1	...
7	1	2	2	2	1	1	1	1	1	1	1	...
8	1	2	1	1	1	1	1	1	1	1	1	...
9	1	1	1	1	1	1	1	1	1	1	1	...
⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋱

. The columns describe the number of customers in queue 1 and the rows in queue 2. It can be seen that the optimal policy is not a static anymore. The optimal average cost here is $g=9.980$. The average cost for the policy $f(q_1,q_2,0)=2$, $q_1,q_2\ge 1$, is equal to $g=11.591$, and for the policy $f(q_1,q_2,0)=1$, $q_1,q_2\ge 1$, the average cost takes the lower value $g=10.077$. This results in the optimal policy differing from those obtained for higher buffer capacities. As $N_1$ and $N_2$ increase, the boundary between areas 1 and 2 in a control matrix shifts right. In the infinite buffer case, the optimal policy is defined exclusively by actions $f(q_1,q_2,0)=2$, $q_1,q_2\ge 1$, with the average cost $g=22.955$, while the alternative policy $f(q_1,q_2,0)=1$, $q_1,q_2\ge 1$ leads now to the higher average cost of $g=25.430$.

The last example shows that for queueing systems with small buffer capacities, the optimal policy differs significantly from the static $c\mu$ policy. However, according to numerical experiments, as the capacity grows, the dynamic control policy will converge to the $c\mu$-rule. At what value of capacity both policies will become indistinguishable is a separate question requiring additional analysis.

4. Conclusions

In this paper, we have analyzed the optimal routing problem for the unreliable single-server two-class queueing system with constant retrial rates. We derived conditions for the optimality of a static policy to serve the customers from a certain queue. The scheduling problem for the system with parallel retrial queues and with no new arrivals can be treated as the same problem for an ordinary multi-class system with a generally distributed service time. In this case, we found explicitly the corresponding $c\mu$-rule. For the system with new arrivals under specified constraint for the arrival process, we proved theoretically the optimality of a policy consisting of two inequalities which imply the original $c\mu$-rule, but they are not equivalent to it. However, numerical experiments show that the $c\mu$-rule is still valid for the main system. As we were unable to find any counter-examples, we can assume that the specified $c\mu$-rule is actually optimal in the infinite buffer case. If the buffers are finite, this rule fails. We have provided a dynamic programming approach to find explicit conditions when static control policies are guaranteed to be optimal. In the future, it would be interesting to consider the possibility of generalizing the results to a main system with new arrivals for an arbitrary number of servers. The process of convergence of the dynamic control policy to the static one with increasing buffer capacity for the system with finite queues should also be investigated in more detail.