Optimal Control Problems for Erlang Loss Systems

Abstract

An Erlang loss system, which is an $M/G/s/s$ queue, is a model used in various applications. In this paper, a controlled version of the process is defined. The objective is to maximize the expected time until the system is full when the service time is exponentially distributed. The control variable is the service rate. The dynamic programming equation satisfied by the value function F, from which the optimal control follows at once, is derived, and F is found explicitly when $s=2$ and $s=3$. The problem of minimising the probability of the system being saturated is also considered.

1. Introduction

In an $M/G/s/s$ queue, called an Erlang loss system, customers arrive according to a Poisson process of rate $\lambda$. The service time S can follow any distribution. There are s servers, and no room for potential customers to wait for service because the capacity of the system is also equal to s. As a result, if a customer arrives while the system is in state s, this customer is lost.

This model is used in various applications. In Smith et al. [1], it was applied to traffic distribution in cellular networks. Lee and Dudin [2] analysed the sojourn time and loss probability in a model of traffic control in communication networks.

Restrepo et al. [3] wanted to find out how to allocate a fleet of ambulances in order to keep response times to calls as small as possible. van Dijk and Kortbeek [4] computed bounds for the probability of rejection in operating theatre and intensive care unit systems.

In de Bruin et al. [5], the problem of interest was to determine the size of hospital wards. Hyytiä et al. [6] proposed a model related to an Erlang loss system to describe a single vehicle in a dial-a-ride transport system. They presented closed-form expressions for the blocking probability, the acceptance rate, and the mean sojourn time of the vehicle in the system.

Rastpour et al. [7] worked on the duration of red alerts for ambulance systems, which corresponds to times at which all ambulances are busy. Morozov et al. [8] were interested in the stability of a model for cognitive wireless networks and used a modified Erlang loss system. Finally, in Zimmerman et al. [9], the aim was to effectively manage the distribution of ventilators during the COVID-19 pandemic.

Recently, Lefebvre and Yaghoubi studied optimal control problems for queueing systems. In [10], they assumed that in the $M/G/1$ model, the server can choose between two service time distributions. The aim was to minimize the expected time needed to clear the queue of customers. In [11], there were k servers, and the objective was to determine how many servers should be used at any time to reduce the number of customers from $k+l$ to $k+r$, where $0\le r<l$, as rapidly as possible. In both cases, the optimizer had to take quadratic control costs into account.

In this paper, we first suppose that the service time S has an exponential distribution with the parameter $\mu$, so that the Erlang loss system becomes an $M/M/s/s$ queue. However, we assume that the parameter $\mu$ is actually a function, $\mu \left[X\right(t\left)\right]$, of the number of customers in the system at time t and that the server can choose any (positive) service rate. The aim will be to determine the value of $\mu \left[X\right(t\left)\right]$ that maximizes the expected value of the time $\tau \left(k\right)$ until the system is full, where k is the initial number of customers in the system. Then, we will consider the problem of minimising the probability that the system, when in equilibrium, is saturated in the interval $[0,\tau (k\left)\right)$. Finally, we will suppose that the parameter $\mu$ is a random variable R and compute the probability that the system (when in equilibrium) is saturated for various important distributions of R.

2. Optimal Service Rate

Let $X\left(t\right)$ denote the number of customers at time t in an $M/M/s/s$ queue. We assume that $X\left(0\right)=k\in \{0,1,2,\dots ,s-1\}$. We define the random variable as follows:

$$\tau \left(k\right)=inf\{t>0:X(t)=s\mid X(0)=k\}.$$

(1)

In probability theory, $\tau \left(k\right)$ is called the first-passage time. It is the first time that the system is full, given that there were k customers at the initial time 0. All the servers are assumed to serve at the same rate $\mu \left[X\right(t\left)\right]$. The larger $\mu \left[X\right(t\left)\right]$ is, the longer it will take until $X\left(t\right)=s$. However, we take it for granted that serving faster is more costly for the system. More precisely, we consider the cost functional:

$$J\left(k\right):={\int }_0^{\tau \left(k\right)}\left\{{\displaystyle \frac{1}{2}}{q}_0{\mu }^2\left[X\left(t\right)\right]+\theta \right\}\mathrm{d}t+K\left[\tau \left(k\right)\right],$$

(2)

where $q_0>0$ and $\theta <0$ are constants and $K(·)$ is a terminal cost function. Since $\theta$ is negative, a reward is received as long as the system is not full. There must therefore be a trade-off between working faster (so that $\mu \left[X\right(t\left)\right]$ is larger) and the quadratic costs thus incurred. The optimizer must also take into account the final cost $K\left[\tau \right(k\left)\right]$. In this paper, we assume that $K\left[\tau \right(k\left)\right]$ is a constant $K_0$.

When the service time S has an exponential distribution, we can use dynamic programming to determine the value of $\mu \left[X\right(t\left)\right]$ in the interval $[0,\tau (k\left)\right)$ that minimizes the expected value of the cost criterion $J\left(k\right)$.

Many papers have been published on the optimal control of queueing systems; see, for instance, Chen and Xia [12], Su and Li [13], and Chen et al. [14]. The main difference between these papers and the present one is the fact that in our case, the final time is a random variable. Such problems are known as homing problems (see Whittle [15]). In a homing problem, the optimizer controls a stochastic process until a certain event occurs. In many applications, the final time is indeed neither fixed nor infinite, but rather random. For example, suppose that $X\left(t\right)$ denotes the number of people suffering from a certain disease during an epidemic, and that we want to put an end to this epidemic as quickly as possible. In reality, the duration of the epidemic is obviously a random variable. Similarly, in the problem that we are studying, we cannot know how long it will take until the system is saturated.

Homing problems were first considered by Whittle [15] as n-dimensional diffusion processes. In practice, it is very difficult to find explicit and exact solutions to these problems in two or more dimensions, as this usually requires solving non-linear partial differential equations. Whittle [16] also generalised homing problems by considering a cost functional that takes into account the optimizer’s sensitivity to risk.

In Rishel [17], a homing problem for diffusion processes that can model the wear of devices was created. In addition to their papers on the optimal control of queueing systems, the author has solved homing problems for autoregressive processes ([18]) and in discrete times ([19]). In [19], the problem considered was in fact a stochastic dynamic game.

Now, we define the value function:

$$F\left(k\right)=\underset{\begin{array}{c}\mu \left[X\right(t\left)\right]\\ t\in [0,\tau (k\left)\right)\end{array}}{inf}E\left[J\left(k\right)\right],$$

(3)

where “E” denotes the expected value. This function gives us the expected cost (or reward) obtained if the optimizer chooses the optimal service rate in the interval $[0,\tau (k\left)\right)$. Moreover, it satisfies the boundary condition

$$F\left(s\right)=K_0.$$

(4)

Remark 1.

Contrary to the problem in [10], here the problem that we study is time-invariant. Therefore, the value function depends only on the current state k. In addition, we now assume that the optimizer can choose any positive value of $\mu \left[X\right(t\left)\right]$. In [10], there were only two possible values for the service rate.

In an $M/M/s/s$ queue, the random time A needed for a new customer to arrive is exponentially distributed with parameter $\lambda$. It follows that the probability that a customer will arrive in the interval $(0,\mathrm{\Delta }t]$ is given by

$$P[A\le \mathrm{\Delta }t]=1-e^{-\lambda \mathrm{\Delta }t}=\lambda \mathrm{\Delta }t+o\left(\mathrm{\Delta }t\right).$$

(5)

When there are $k\in \{1,2,\dots ,s-1\}$ customers in the system, the time $S\left(k\right)$ until a customer departs has an exponential distribution with the parameter $k\mu \left(k\right)$. Hence,

$$P\left[S\right(k)\le \mathrm{\Delta }t]=k\mu \left(k\right)\mathrm{\Delta }t+o\left(\mathrm{\Delta }t\right).$$

(6)

The probability of at least two events (arrivals or departures) in any interval of length $\mathrm{\Delta }t$ is equal to $o\left(\mathrm{\Delta }t\right)$. Moreover, the random variables A and $S\left(k\right)$ are assumed to be independent, which implies that $min\{A,S(k\left)\right\}\sim$ Exp$(\lambda +k\mu (k\left)\right)$.

When $X\left(0\right)=k\in \{1,2,\dots ,s-1\}$, the random number of customers in the system at time $\mathrm{\Delta }t$ is

$$X\left(\mathrm{\Delta }t\right)=\left\{\begin{array}{cc}k+1& \mathrm{with}\mathrm{probability}\lambda \mathrm{\Delta }t+o\left(\mathrm{\Delta }t\right),\hfill \\ k-1& \mathrm{with}\mathrm{probability}k\mu \left(k\right)\mathrm{\Delta }t+o\left(\mathrm{\Delta }t\right),\hfill \\ k& \mathrm{with}\mathrm{probability}1-[\lambda +k\mu (k\left)\right]\mathrm{\Delta }t+o\left(\mathrm{\Delta }t\right).\hfill \end{array}\right.$$

(7)

If $X\left(0\right)=0$, we find that

$$X\left(\mathrm{\Delta }t\right)=\left\{\begin{array}{cc}k+1& \mathrm{with}\mathrm{probability}\lambda \mathrm{\Delta }t+o\left(\mathrm{\Delta }t\right),\hfill \\ k& \mathrm{with}\mathrm{probability}1-\lambda \mathrm{\Delta }t+o\left(\mathrm{\Delta }t\right).\hfill \end{array}\right.$$

(8)

Next, since $\tau \left(0\right)$ has the same distribution as $A+\tau \left(1\right)$, we can write that

$$E\left[\tau \left(0\right)\right]={\displaystyle \frac{1}{\lambda }}+E\left[\tau \left(1\right)\right]⟹F\left(0\right)={\displaystyle \frac{\theta }{\lambda }}+F\left(1\right).$$

(9)

For $k\in \{1,2,\dots ,s-1\}$, we find that

$$\begin{array}{ccc}\hfill F\left(k\right)& :=& \underset{\begin{array}{c}\mu \left[X\right(t\left)\right]\\ t\in [0,\tau (k\left)\right)\end{array}}{inf}E\left[{\int }_0^{\tau \left(k\right)}\left\{{\displaystyle \frac{1}{2}}{q}_0{\mu }^2\left[X\left(t\right)\right]+\theta \right\}\mathrm{d}t\right]\hfill \\ & =& \underset{\begin{array}{c}\mu \left[X\right(t\left)\right]\\ t\in [0,\tau (k\left)\right)\end{array}}{inf}E\left[{\int }_0^{\mathrm{\Delta }t}\left\{{\displaystyle \frac{1}{2}}{q}_0{\mu }^2\left[X\left(t\right)\right]+\theta \right\}\mathrm{d}t+{\int }_{\mathrm{\Delta }t}^{\tau \left(k\right)}\left\{{\displaystyle \frac{1}{2}}{q}_0{\mu }^2\left[X\left(t\right)\right]+\theta \right\}\mathrm{d}t\right]\hfill \\ & =& \underset{\begin{array}{c}\mu \left[X\right(t\left)\right]\\ t\in [0,\tau (k\left)\right)\end{array}}{inf}E[\left({\displaystyle \frac{1}{2}}{q}_0{\mu }^2\left(k\right)+\theta \right)\mathrm{\Delta }t+{\int }_{\mathrm{\Delta }t}^{\tau \left(k\right)}\left\{{\displaystyle \frac{1}{2}}{q}_0{\mu }^2\left[X\left(t\right)\right]+\theta \right\}\mathrm{d}t\hfill \\ & & +o\left(\mathrm{\Delta }t\right)].\hfill \end{array}$$

(10)

Making use of Bellman’s principle of optimality, we can express the expected value of the integral from $\mathrm{\Delta }t$ to $\tau \left(k\right)$ in terms of the value function at time $\mathrm{\Delta }t$:

$$\underset{\begin{array}{c}\mu \left[X\right(t\left)\right]\\ t\in [\mathrm{\Delta }t,\tau (k\left)\right)\end{array}}{inf}E\left[{\int }_{\mathrm{\Delta }t}^{\tau \left(k\right)}\left\{{\displaystyle \frac{1}{2}}{q}_0{\mu }^2\left[X\left(t\right)\right]+\theta \right\}\mathrm{d}t\right]=E\left[F\left[X\right(\mathrm{\Delta }t\left)\right]\right]+o\left(\mathrm{\Delta }t\right).$$

(11)

We deduce from Equation (7) that

$$E\left[F\left[X\right(\mathrm{\Delta }t\left)\right]\right]=F(k+1)\lambda \mathrm{\Delta }t+F(k-1)k\mu \left(k\right)\mathrm{\Delta }t+F\left(k\right)\left\{1-[\lambda +k\mu (k\left)\right]\mathrm{\Delta }t\right\}+o\left(\mathrm{\Delta }t\right).$$

(12)

It then follows from Equation (10) that

$$\begin{array}{ccc}\hfill 0& =& \underset{\mu \left(k\right)}{inf}\{\left({\displaystyle \frac{1}{2}}{q}_0{\mu }^2\left(k\right)+\theta \right)\mathrm{\Delta }t+F(k+1)\lambda \mathrm{\Delta }t+F(k-1)k\mu \left(k\right)\mathrm{\Delta }t-F\left(k\right)[\lambda +k\mu \left(k\right)]\mathrm{\Delta }t\hfill \\ & & +o\left(\mathrm{\Delta }t\right)\}.\hfill \end{array}$$

(13)

That is,

$$0=\underset{\mu \left(k\right)}{inf}\left\{{\displaystyle \frac{1}{2}}{q}_0{\mu }^2\left(k\right)+\theta +F(k+1)\lambda +F(k-1)k\mu \left(k\right)-F\left(k\right)[\lambda +k\mu \left(k\right)]+{\displaystyle \frac{o\left(\mathrm{\Delta }t\right)}{\mathrm{\Delta }t}}\right\}.$$

(14)

Letting $\mathrm{\Delta }t$ decrease to zero, we can state the following proposition.

Proposition 1.

The value function $F\left(k\right)$ satisfies the dynamic programming equation (DPE)

$$0=\underset{\mu \left(k\right)}{inf}\left\{{\displaystyle \frac{1}{2}}{q}_0{\mu }^2\left(k\right)+\theta +F(k+1)\lambda +F(k-1)k\mu \left(k\right)-F\left(k\right)[\lambda +k\mu \left(k\right)]\right\}$$

(15)

for $k\in \{1,2,\dots ,s-1\}$. Moreover, we obtain Equation (9) and the boundary condition in Equation (4).

Now, by differentiating the expression within the curly brackets in Equation (15) with respect to $\mu \left(k\right)$ and setting it to zero, we find that

$$q_0\mu \left(k\right)+kF(k-1)-kF\left(k\right)=0.$$

(16)

Corollary 1.

The optimal control can be expressed in terms of the value function as follows:

$${\mu }^{*}\left(k\right)={\displaystyle \frac{k}{q_0}}\left[F\left(k\right)-F(k-1)\right]$$

(17)

for $k\in \{1,2,\dots ,s-1\}$.

Substituting the expression for $u^{*}\left(k\right)$ in Equation (17) into the DPE, we find (after simplification) that to obtain the value function $F\left(k\right)$ we must solve the non-linear second-order difference equation

$$0=\theta -{\displaystyle \frac{k^2}{2q_0}}{\left[F\left(k\right)-F(k-1)\right]}^2+\lambda \left[F(k+1)-F\left(k\right)\right],$$

(18)

where $k\in \{1,2,\dots ,s-1\}$, subject to the boundary condition $F\left(s\right)=K_0$. In the remainder of this paper, we will assume that $K_0=0$, so that $F\left(s\right)=0$.

Remark 2.

Suppose that we set $\mu \left(k\right)=0$ for any k. That is, we consider the uncontrolled process $\{X_0\left(t\right),t\ge 0\}$. Then, we find that

$$J_0\left(k\right):={\int }_0^{{\tau }_0\left(k\right)}\theta \mathrm{d}t⟹E\left[J_0\left(k\right)\right]=\theta E\left[{\tau }_0\left(k\right)\right].$$

(19)

Moreover, we can write that

$${\tau }_0\left(k\right)=A_k+A_{k+1}+\cdots +A_{s-1},$$

(20)

where $A_i\sim$ Exp$\left(\lambda \right)$ for $i=1,2,\dots ,s-1$. Hence,

$$E\left[{\tau }_0\left(k\right)\right]={\displaystyle \frac{s-k}{\lambda }}⟹F_0\left(k\right)={\displaystyle \frac{(s-k)\theta }{\lambda }}.$$

(21)

It follows that the value function must satisfy the inequation

$$F\left(k\right)\le {\displaystyle \frac{(s-k)\theta }{\lambda }}<0.$$

(22)

2.1. The Case When $s=2$

When $s=2$, we find that $F\left(2\right)=0$. Equation (18) for $k=1$ becomes

$$0=\theta -{\displaystyle \frac{1}{2q_0}}{\left[F\left(1\right)-F\left(0\right)\right]}^2-\lambda F\left(1\right).$$

(23)

Moreover, Equation (9) implies that

$$-\lambda q_{0}F\left(0\right)+2\theta q_0-{\displaystyle \frac{{\theta }^2}{2{\lambda }^2}}=0,$$

(24)

so that

$$F\left(0\right)={\displaystyle \frac{2\theta }{\lambda }}-{\displaystyle \frac{{\theta }^2}{2{\lambda }^3{q}_0}},$$

(25)

and $F\left(1\right)=F\left(0\right)-\theta /\lambda$. From Equation (17), we see that the optimal control is given by

$${\mu }^{*}\left(1\right)={\displaystyle \frac{1}{q_0}}\left[F\left(1\right)-F\left(0\right)\right]=-{\displaystyle \frac{\theta }{q_0\lambda }}.$$

(26)

2.2. The Case When $s=3$

We now use $F\left(3\right)=0$ and the following non-linear difference equations:

$$\begin{array}{ccc}\hfill 0& =& \theta -{\displaystyle \frac{1}{2q_0}}{\left[F\left(1\right)-F\left(0\right)\right]}^2+\lambda \left[F\left(2\right)-F\left(1\right)\right],\hfill \end{array}$$

(27)

$$\begin{array}{ccc}\hfill 0& =& \theta -{\displaystyle \frac{2}{q_0}}{\left[F\left(2\right)-F\left(1\right)\right]}^2-\lambda F\left(2\right).\hfill \end{array}$$

(28)

Making use of the mathematical software program Mathematica (https://www.wolfram.com/mathematica/online/), we find that the solution to systems (27) and (28), together with Equation (9), is

$$\begin{array}{ccc}\hfill F\left(0\right)& =& {\displaystyle \frac{\theta (6{\lambda }^6{q}_0^3-5{\lambda }^4{q}_0^2\theta +4{\lambda }^2{q}_0{\theta }^2-{\theta }^3)}{2{\lambda }^7{q}_0^3}},\hfill \end{array}$$

(29)

$$\begin{array}{ccc}\hfill F\left(1\right)& =& {\displaystyle \frac{\theta (4{\lambda }^6{q}_0^3-5{\lambda }^4{q}_0^2\theta +4{\lambda }^2{q}_0{\theta }^2-{\theta }^3)}{2{\lambda }^7{q}_0^3}},\hfill \end{array}$$

(30)

$$\begin{array}{ccc}\hfill F\left(2\right)& =& {\displaystyle \frac{\theta (2{\lambda }^6{q}_0^3-4{\lambda }^4{q}_0^2\theta +4{\lambda }^2{q}_0{\theta }^2-{\theta }^3)}{2{\lambda }^7{q}_0^3}}.\hfill \end{array}$$

(31)

In the particular case where $\theta =-1$ and $\lambda =1$, we obtain

$$F\left(0\right)=-{\displaystyle \frac{6q_0^3+5q_0^2+4q_0+1}{2q_0^3}}.$$

(32)

This function of $q_0$ is shown in Figure 1. Notice that

$$\underset{q_0\to \infty }{lim}F\left(0\right)=-3,$$

(33)

which is the bound deduced from Equation (22). This result follows from the fact that when $q_0$ tends to infinity, the optimizer should choose $\mu \left(k\right)=0$ for any $k\le s-1$. Similarly, with $\theta =-1$ and $\lambda =1$, we obtain ${lim}_{q_0\to \infty }F\left(1\right)=-2$ and ${lim}_{q_0\to \infty }F\left(2\right)=-1$.

The optimal controls are

$$\begin{array}{ccc}\hfill {\mu }^{*}\left(1\right)& =& -{\displaystyle \frac{\theta }{q_0\lambda }},\hfill \end{array}$$

(34)

$$\begin{array}{ccc}\hfill {\mu }^{*}\left(2\right)& =& -{\displaystyle \frac{2\theta }{q_0\lambda }}+{\displaystyle \frac{{\theta }^2}{q_0^2{\lambda }^3}}.\hfill \end{array}$$

(35)

3. Minimising the Probability of Saturation

When the Erlang loss system is in statistical equilibrium (or a steady state), the probability that it is in state k is given by the following (see [20]):

$${\pi }_k:=\underset{t\to \infty }{lim}P[X\left(t\right)=k]={\displaystyle \frac{{\left(\lambda E\left[S\right]\right)}^k/k!}{{\sum }_{j=0}^s{\left(\lambda E\left[S\right]\right)}^j/j!}}$$

(36)

for $k\in \{0,1,\dots ,s\}$. Thus, the probability that the system in equilibrium is saturated (or full), such that all the servers are busy, is

$${\pi }_s={\displaystyle \frac{{\left(\lambda E\left[S\right]\right)}^s/s!}{{\sum }_{j=0}^s{\left(\lambda E\left[S\right]\right)}^j/j!}}.$$

(37)

Suppose now that we start controlling the process when it is in equilibrium and that $X\left(0\right)=k\in \{0,1,\dots ,s-1\}$. Moreover, we assume, as in the preceding section, that the service time is an exponential random variable with parameter $\mu$ that depends on the state of the process, such that ${\pi }_s={\pi }_s\left[\mu \left(X\left(t\right)\right)\right]$. We want to minimize the probability of saturation in the interval $[0,\tau (k\left)\right)$. We consider the cost functional

$$J_2\left(k\right):={\int }_0^{\tau \left(k\right)}\left\{{\displaystyle \frac{1}{2}}{q}_0{\mu }^2\left[X\left(t\right)\right]+{\pi }_s\left[\mu \left(X\left(t\right)\right)\right]\right\}\mathrm{d}t.$$

(38)

Proceeding as in Section 2, we obtain the following corollary to Proposition 1.

Corollary 2.

The value function $F\left(k\right)$ satisfies the DPE

$$0=\underset{\mu \left(k\right)}{inf}\left\{{\displaystyle \frac{1}{2}}{q}_0{\mu }^2\left(k\right)+{\displaystyle \frac{{[\lambda /\mu \left(k\right)]}^s/s!}{{\sum }_{j=0}^s{[\lambda /\mu \left(k\right)]}^j/j!}}+F(k+1)\lambda +F(k-1)k\mu \left(k\right)-F\left(k\right)[\lambda +k\mu \left(k\right)]\right\}$$

(39)

for $k\in \{1,2,\dots ,s-1\}$. The boundary condition is $F\left(s\right)=0$.

Even in the case where $s=2$, trying to express the optimal control ${\mu }^{*}\left(k\right)$ in terms of the value function does not seem possible. Therefore, we will simplify the problem. Assume that the optimizer can only choose between $\mu ={\mu }_1$ or ${\mu }_2$ and cannot switch from ${\mu }_1$ to ${\mu }_2$ (or vice versa) in the interval $[0,\tau (k\left)\right)$. Then,

$$E\left[J_2\left(k\right)\right]=E\left[{\int }_0^{\tau \left(k\right)}\left\{{\displaystyle \frac{1}{2}}{q}_0{\mu }_i^2+{\pi }_s\left({\mu }_i\right)\right\}\mathrm{d}t\right]=\left\{{\displaystyle \frac{1}{2}}{q}_0{\mu }_i^2+{\pi }_s\left({\mu }_i\right)\right\}E\left[\tau \left(k\right)\right],$$

(40)

where ${\mu }_i={\mu }_1$ or ${\mu }_2$.

3.1. The Case When $s=2$

Let $F_{i,j}$ be the event: the first transition of the process from state i is to state j. Using the fact that

$$P[A<S]={\displaystyle \frac{\lambda }{{\mu }_i+\lambda }},$$

(41)

we can write that

$$\begin{array}{ccc}\hfill E\left[\tau \right(1\left)\right]& =& E[\tau \left(1\right)\mid F_{1,0}]P\left[F_{1,0}\right]+E[\tau \left(1\right)\mid F_{1,2}]P\left[F_{1,2}\right]\hfill \\ & =& \left\{{\displaystyle \frac{1}{{\mu }_i+\lambda }}+{\displaystyle \frac{1}{\lambda }}+E\left[\tau \left(1\right)\right]\right\}{\displaystyle \frac{{\mu }_i}{{\mu }_i+\lambda }}+{\displaystyle \frac{1}{{\mu }_i+\lambda }}{\displaystyle \frac{\lambda }{{\mu }_i+\lambda }}\hfill \end{array}$$

(42)

for $i=1,2$. It follows that

$$E\left[\tau \left(1\right)\right]={\displaystyle \frac{{\mu }_i+\lambda }{{\lambda }^2}}.$$

(43)

Therefore, in order to determine whether $E\left[J_2\left(1\right)\right]$ is smaller when $\mu ={\mu }_1$ or ${\mu }_2$, we must calculate

$$E\left[J_2\left(1\right)\right]=\left[{\displaystyle \frac{1}{2}}{q}_0{\mu }_i^2+{\displaystyle \frac{{(\lambda /{\mu }_i)}^2/2}{1+(\lambda /{\mu }_i)+\frac{1}{2}{(\lambda /{\mu }_i)}^2}}\right]{\displaystyle \frac{{\mu }_i+\lambda }{{\lambda }^2}}$$

(44)

for $i=1,2$ and compare the two values of $E\left[J_2\left(1\right)\right]$.

Suppose that $\lambda =1$, ${\mu }_1=1$ and ${\mu }_2=2$. We must compare

$$E\left[J_2\left(1\right)\right]\stackrel{\mu =1}{=}{q}_0+{\displaystyle \frac{2}{5}}\mathrm{with}E\left[J_2\left(1\right)\right]\stackrel{\mu =2}{=}6q_0+{\displaystyle \frac{3}{13}}.$$

(45)

The two straight lines cross at $q_0=q^{*}:=11/325$. As can be seen in Figure 2, the optimal solution is to choose $\mu =2$ for $q_0\in [0,q^{*}]$ and $\mu =1$ for $q_0\ge q^{*}$.

3.2. The Case When $s=3$

We know that

$$\begin{array}{ccc}\hfill E\left[\tau \right(1\left)\right]& =& E[\tau \left(1\right)\mid F_{1,0}]P\left[F_{1,0}\right]+E[\tau \left(1\right)\mid F_{1,2}]P\left[F_{1,2}\right]\hfill \\ & =& \left\{{\displaystyle \frac{1}{{\mu }_i+\lambda }}+{\displaystyle \frac{1}{\lambda }}+E\left[\tau \left(1\right)\right]\right\}{\displaystyle \frac{{\mu }_i}{{\mu }_i+\lambda }}+\left\{{\displaystyle \frac{1}{{\mu }_i+\lambda }}+E\left[\tau \left(2\right)\right]\right\}{\displaystyle \frac{\lambda }{{\mu }_i+\lambda }}\hfill \end{array}$$

(46)

and

$$\begin{array}{ccc}\hfill E\left[\tau \right(2\left)\right]& =& E[\tau \left(2\right)\mid F_{2,1}]P\left[F_{2,1}\right]+E[\tau \left(2\right)\mid F_{2,3}]P\left[F_{2,3}\right]\hfill \\ & =& \left\{{\displaystyle \frac{1}{{\mu }_i+\lambda }}+E\left[\tau \left(1\right)\right]\right\}{\displaystyle \frac{{\mu }_i}{{\mu }_i+\lambda }}+{\displaystyle \frac{1}{{\mu }_i+\lambda }}{\displaystyle \frac{\lambda }{{\mu }_i+\lambda }}.\hfill \end{array}$$

(47)

We find that the solution to systems (46) and (47) is

$$E\left[\tau \left(1\right)\right]={\displaystyle \frac{1}{{\lambda }^3}}(2{\lambda }^2+2\lambda \mu +{\mu }^2)\mathrm{and}E\left[\tau \left(2\right)\right]={\displaystyle \frac{1}{{\lambda }^3}}({\lambda }^2+\lambda \mu +{\mu }^2).$$

(48)

With these values, we can proceed as in the case when $s=2$ to determine whether $\mu ={\mu }_1$ or $\mu ={\mu }_2$ is the optimal choice for $k=1,2$. First, when $k=1$, we compute

$$E\left[J_2\left(1\right)\right]=\left[{\displaystyle \frac{1}{2}}{q}_0{\mu }_i^2+{\displaystyle \frac{{(\lambda /{\mu }_i)}^3/6}{1+(\lambda /{\mu }_i)+\frac{1}{2}{(\lambda /{\mu }_i)}^2+\frac{1}{6}{(\lambda /{\mu }_i)}^3}}\right]{\displaystyle \frac{1}{{\lambda }^3}}(2{\lambda }^2+2\lambda {\mu }_i+{\mu }_i^2)$$

(49)

for $i=1,2$. Let us again take $\lambda =1$, ${\mu }_1=1$, and ${\mu }_2=2$. We obtain

$$E\left[J_2\left(1\right)\right]\stackrel{\mu =1}{=}{\displaystyle \frac{5}{2}}{q}_0+{\displaystyle \frac{5}{16}}\mathrm{and}E\left[J_2\left(1\right)\right]\stackrel{\mu =2}{=}20q_0+{\displaystyle \frac{10}{79}}.$$

(50)

The optimal service rate is $\mu =2$ for $q_0\in [0,47/4424]$ and $\mu =1$ for $q_0\ge 47/4424$; see Figure 3.

Similarly, when $k=2$, we find that

$$E\left[J_2\left(2\right)\right]\stackrel{\mu =1}{=}{\displaystyle \frac{3}{2}}{q}_0+{\displaystyle \frac{3}{16}}\mathrm{and}E\left[J_2\left(2\right)\right]\stackrel{\mu =2}{=}14q_0+{\displaystyle \frac{7}{79}}.$$

(51)

This time, the optimal service rate is $\mu =2$ for $q_0\in [0,5/632]$ and $\mu =1$ for $q_0\ge 5/632$; see Figure 4.

4. The Case in Which the Service Rate Is a Random Variable

To conclude this paper, we will consider the following problem: suppose that the service time S has an exponential distribution with parameter R, where R is itself a random variable. We would like to determine for which distribution of R, as it has a given expected value, the probability of saturation is minimized when the system is in equilibrium. To do so, assuming that R possesses a probability density function $f_R\left(r\right)$, we would have to minimize

$${\pi }_s={\int }_0^{\infty }{\displaystyle \frac{{(\lambda /r)}^s/s!}{{\sum }_{j=0}^s{(\lambda /r)}^j/j!}}{f}_R\left(r\right)\mathrm{d}r,$$

(52)

under the constraints that

$${\int }_0^{\infty }{f}_R\left(r\right)\mathrm{d}r=1\mathrm{and}{\int }_0^{\infty }r{f}_R\left(r\right)\mathrm{d}r=r_0,$$

(53)

where $r_0$ is a given constant. The random variable R could be discrete, or even of mixed type, so that the density function $f_R\left(r\right)$ may contain Dirac delta functions.

Instead of trying to find the function $f_R\left(r\right)$ that minimizes ${\pi }_s$, we will consider the most important probability distributions for R. We will treat this as a case in which $s=2$ and, for the sake of simplicity, set $\lambda$ equal to 1. Moreover, we assume that $E\left[R\right]=1$. Hence, we must compute

$${\pi }_2={\int }_0^{\infty }{\displaystyle \frac{{(1/r)}^2/2}{1+(1/r)+\frac{1}{2}{(1/r)}^2}}{f}_R\left(r\right)\mathrm{d}r$$

(54)

for random variables such that

$${\int }_0^{\infty }r{f}_R\left(r\right)\mathrm{d}r=1.$$

(55)

Suppose first that R is actually the constant 1. Then, we find that

$$f_R\left(r\right)=\delta (r-1)⟹{\pi }_2={\displaystyle \frac{1/2}{1+1+\frac{1}{2}}}={\displaystyle \frac{1}{5}}.$$

(56)

Next, if R has a uniform distribution at the interval $(0,2)$, we find that

$${\pi }_2={\displaystyle \frac{1}{2}}{\int }_0^2{\displaystyle \frac{{(1/r)}^2/2}{1+(1/r)+\frac{1}{2}{(1/r)}^2}}\mathrm{d}r={\displaystyle \frac{arctan\left(5\right)}{2}}-{\displaystyle \frac{\pi }{8}}\approx 0.2940.$$

(57)

Assume now that S has a gamma distribution with both parameters equal to $\alpha$, so that

$$f_R(r;\alpha )={\displaystyle \frac{1}{\mathsf{\Gamma }\left(\alpha \right)}}{\left(\alpha r\right)}^{\alpha -1}\alpha e^{-\alpha r}\mathrm{for}r>0.$$

(58)

We obtain $E\left[R\right]=1$, as required. If $\alpha =1$, $R\sim$ Exp$\left(1\right)$. We compute

$${\int }_0^{\infty }{\displaystyle \frac{{(1/r)}^2/2}{1+(1/r)+\frac{1}{2}{(1/r)}^2}}{f}_R(r;\alpha =0.1)\mathrm{d}r\approx 0.7474.$$

(59)

The value of ${\pi }_2$ when $\alpha =1,2,\dots ,20$ is shown in Figure 5. We see that ${\pi }_2$ decreases with an increasing $\alpha$. For large $\alpha$, the limiting probability ${\pi }_2$ is approximately equal to 0.21.

Let

$$f_R(r;\sigma ):={\displaystyle \frac{r}{{\sigma }^2}}{e}^{-r^2/\left(2{\sigma }^2\right)}\mathrm{for}r>0.$$

(60)

That is, the random variable R has a Rayleigh distribution with parameter $\sigma$. When $\sigma =\sqrt{2/\pi }$, we find that $E\left[R\right]=1$. We calculate

$${\int }_0^{\infty }{\displaystyle \frac{{(1/r)}^2/2}{1+(1/r)+\frac{1}{2}{(1/r)}^2}}{f}_R(r;\sigma =\sqrt{2/\pi })\mathrm{d}r\approx 0.2619.$$

(61)

Notice that for all the distributions considered above, the value of ${\pi }_2$ is larger than when $R=1$.

If R is a discrete random variable taking the values 0.5 and 1.5 with probability of 1/2, we find that ${\pi }_2\approx 0.2588$. When $R=r_1$ or $2-r_1$ with a probability of 1/2, where $r_1\in (0,1)$, ${\pi }_2\in (1/5,7/13)$ and decreases as $r_1$ increases. More generally, if $R=r_1<1$ or $r_2>1$, such that $E\left[R\right]=1$, the value of ${\pi }_2$ is greater than 1/5.

5. Conclusions

In this paper, a homing problem for an $M/G/s/s$ queueing model was considered. As mentioned in Section 1, this model is used in many applications. We were interested in maximising the expected time until the system becomes saturated. We were able to determine the optimal value of the service rate when the service time S has an exponential distribution, such that we are dealing with an $M/M/s/s$ queue. In this case, we can appeal to dynamic programming to obtain the equation satisfied by the value function. This equation is a non-linear second-order difference equation. The optimal control can be deduced from the value function.

In Section 3, we examined a related optimal control problem. However, we had to simplify the problem because it is not possible to explicitly express the optimal control in terms of the value function.

In Section 4, we wanted to minimize the probability of saturation when the service rate $\mu$ is a random variable R. Various types of distribution of R were considered.

In general, when the service time is not exponentially distributed, it is not possible to make use of dynamic programming. Indeed, if the probability in Equation (6) is not proportional to $\mathrm{\Delta }t$, theIS technique will fail. The same holds true if the random variable A does not have an exponential distribution. In those cases, another technique must be used for the $M/G/s/s$ queue.

Finally, we could modify the assumptions leading us to the $M/M/s/s$ model. For instance, the servers could serve at different rates and/or perhaps more than one customer at a time, or there could be batch arrivals, and so on.