On the Optimal Input Rate in Queues with Batch Service

Abstract

In recent years, queuing systems with batch service are emerging as powerful and flexible mathematical models in different frameworks. In this paper, we consider a single server queuing system with Poissonian arrivals, infinite buffers, and a constant batch size b. This paper addresses a little-studied optimization problem, namely the existence of an optimal arrival rate that minimizes the average sojourn time. Unlike the classical $M/M/1$ queue, for any batch size b, the problem admits a non-trivial solution that can be found by solving a polynomial equation of degree $b+1$. Since, in general, only numerical solutions are available, a simple first-order approximation is also derived and the corresponding deviations (in terms of input rate and sojourn time) are calculated. In more detail, it is shown that the approximation improves as the batch size increases and, in any case, the relative error for the average sojourn time is less than 0.34%. Finally, the paper provides new theoretical results about the asymptotic service rate in the equivalent birth–death process, highlighting how it depends on all queue parameters.

1. Introduction

Queues and queuing networks are often used as powerful mathematical models of many real systems, in which a given service is shared by multiple customers. Starting from the pioneering works by Erlang [1], queuing theory has emerged as a powerful analysis tool in a wide range of cargo and passenger transport systems (including river ports, seaports, airports, and multimodal transport hubs) [2], in healthcare [3], in IoT–Fog–Cloud cooperation systems [4], and, in general, in telecommunication systems with Quality of Service guarantees [5], just to mention a few recent works.

Traditionally, it has been assumed that the service discipline is work conserving, i.e., a server (or, more in general, servers) cannot be free if the waiting line is not empty. Although such an assumption seems to be quite natural, there is currently a growing interest in developing new analysis methods for queuing systems that violate this, namely queues with batch service (also known in the literature as bulk service queues), since they represent suitable models in different real-life scenarios, ranging from production systems [6,7] to cloud computing and computer systems in general [8,9]. The widespread practical use of batch services makes the study of this kind of queue relevant in the 21st century [10,11].

The first analysis of queuing systems with batch service was published in [12], where the equilibrium distribution of the queue length was obtained using the embedded Markov chain method for a queuing system with one server that serves a batch of customers of a fixed maximum size at intervals with a given distribution. Then, in [13], the Laplace transform of the waiting time distribution was derived from the probability generating function of the queue length distribution. Explicit formulas for the mean and variance of the waiting time were shown and higher moments, if necessary, could be easily found. Later, the study of systems with a batch service was extended by Neuts, Bagchi, and Serfozo [14,15,16], considering different strategies of service, as highlighted below. Moreover, the fundamental results for batch service systems can be found in [17,18,19].

In more detail, several types of batch service have been considered in the literature. The general case, introduced in [14], assumes that the batch size has lower and upper thresholds, denoted in the following as a and b, respectively. This means that three situations are possible:

• If the number of customers in the system is less than a, then the server is idle;
• If the number of customers in the waiting line is between the two thresholds a and b, then they are all served as soon as the previous batch leaves the queue;
• Otherwise, a group of b customers is selected for service.

Such systems are known in the literature as general bulk service (GBS) queues [20,21], and for $a=1$, they include the original model proposed in [12,13].

As another special case of GBS, the customers might be serviced in batches of a given fixed size [10,11,22], i.e., $a=b$, with the underlying assumption that the server remains idle when the number of customers in the queue is less then the batch size.

It is worth mentioning that batching based only on the number of customers in the queue has several drawbacks, for instance, when clients are perishable, i.e., the customer may rot before the service is completed. Recently, to overcome this issue, [23] introducd the flexible general bulk service (FGBS) rule, which modifies the GBS rule by taking into account the average waiting time. In FGBS, the server starts bulk service when:

• There are at least a customers in the queue;
• There are less than a customers in the queue but their measured average waiting time is greater than or equal to the waiting time tolerance T.

In other words, the server becomes idle only if at the service completion epochs, the number of customers is less than a and the average waiting time of a customer in the queue is less than T.

Another interesting variant of the GBS discipline has been proposed in [24], where the service process comprises of k stages (with distinct input queues) and customers can join the service batch (provided its size is less than b) at any stage depending on their service requirements. Moreover, the service rate at each stage depends on the stage as well as on the current batch size m (with $a\le m\le b$). Finally, customers cannot leave the system after completing service in an intermediate stage.

In general, systems with batch service are very difficult to analyze, but, as in other queuing models, the assumptions that arrivals are Poissonian and service times for batches of customers have an exponential distribution permit to obtain some closed-form results. For example, in [25], it was proposed to use the supplementary variable technique and the theory of difference equations in the analysis of a single server queue with batch arrivals and services. In more detail, the authors obtained explicit expressions of the steady-state system–content distribution at pre-arrival and arbitrary epochs in terms of the roots of the associated characteristic equation. The authors of [10] obtained the closed-form steady-state distribution for a class of queues that serve batches of customers with different configurations of service rates that may depend on the load of the server, with special attention to the conditions that must be satisfied to make the system quasi-reversible.

In [11], using the probability generating function method, the authors derived the closed-form expression for a steady-state queue length distribution, the expected queue length, and the expected waiting time of a customer. In [26], a multiplicative form of the stationary distribution for the $M/M^b/1$ queue (i.e., with a fixed batch size b) was obtained on the basis of an equivalent birth–death process with (adequately chosen) state-dependent service rates, which can be derived by solving a polynomial equation of degree $b+1$.

Among all the works on queuing systems with batch services, only a few of them deal with various optimization problems. For example, in [27], the optimal number of servers that minimizes the total expected cost per unit time was found for both infinite and finite capacity systems. In [16], the authors analyzed the problem of finding policies which minimize the expected discounted cost, or the expected cost per unit time, over an infinite time horizon. The strategic customer behavior is considered in [28]; the customers decide whether or not to join the queue taking into account a linear reward–cost structure.

In this paper, we extend our previous work [29] on the $M/M^b/1$ queue, focusing on the derivation of the optimal input rate that minimizes the average sojourn time for fixed values of the other system parameters (namely, batch service rate $\mu$ and batch size b).

To the best of our knowledge, the existence of an optimal input rate has never been proven in the literature; indeed, as highlighted above, optimization techniques have been applied to other characteristics. Moreover, in our previous work [29], the batch size was considered as an optimization parameter, while this paper completes the investigation of the batch service mechanism, focusing the dependence of the average waiting time on the arrival and service rates. In more detail, the optimal input rate is determined as a unique solution (under the system stability condition) of a polynomial equation of degree $b+1$, which, in general, can be solved only numerically. Moreover, the first order approximation is provided and the relative errors for the optimal input rate and the corresponding sojourn time are investigated for different values of b.

Finally, as a side contribution, the equivalent birth–death process is revised, focusing on the mathematical properties of the characteristic equation and deriving auxiliary expressions, used in the following.

The rest of the paper is organized as follows. Section 2 deals with the analysis method based on the equivalent birth–death process we proposed in [26], which permitted us to find an explicit formula for the sojourn time. Then, the main contributions are reported in Section 3, which includes the general theoretical results on the existence of an optimal arrival rate ${\lambda }_{\mathrm{opt}}$ as well as the first order approximation ${\lambda }_{\mathrm{appr}}$ and the corresponding values of the minimum sojourn time as a function of the batch size. Finally, Section 4 discusses the goodness of the proposed approximation and highlights future research directions.

2. Materials and Methods

The aim of this section is twofold. On one hand, it briefly recalls the main results for the $M/M^b/1$ queue (see Section 2.1), focusing on the analysis method proposed in [26] and on the general expression of the sojourn time (more details can be found in [29] and the references therein). On the other hand, in Section 2.2, the characteristic equation is deeply investigated, highlighting the dependence of the asymptotic equivalent death rate M on the queue parameters; apart from the theoretical relevance of these results, some of the expressions will be used in Section 3 to derive the main contribution of the paper. Finally, Section 2.3 comes back to the equivalent birth–death process, highlighting that only the chosen value of M corresponds to a well-defined Markov chain.

2.1. The $M/M^b/1$ Queue

Let us consider a single server queue with infinite buffers and assume that the input process is Poissonian with rate $\lambda$. Customers are served in batches of fixed size b ($b\ge 2$) and the service time is exponentially distributed with parameter $\mu$. Such a system is not work conserving since, after a batch departure, the server stays idle until b customers accumulate in the waiting line.

The existence of a non degenerate steady-state behavior requires that the utilization factor satisfies the stability condition, i.e.,

$$\rho =\frac{\lambda }{b\mu }<1.$$

(1)

Product form state probabilities of the underlying Markov chain can be derived, as originally proposed by the authors (see [26] for details), by introducing an equivalent birth–death process, with constant birth rate $\lambda$ and state-dependent death rates $\tilde{\mu }\left(n\right)$, determined by the following system of equations:

$$\left\{\begin{array}{c}{\displaystyle \tilde{\mu }\left(n\right)=\lambda -\mu \frac{{\lambda }^b}{\tilde{\mu }(n+1)\times \dots \times \tilde{\mu }(b+n)},1⩽n⩽b-1,}\hfill \\ {\displaystyle \tilde{\mu }\left(n\right)=\lambda +\mu -\mu \frac{{\lambda }^b}{\tilde{\mu }(n+1)\times \dots \times \tilde{\mu }(b+n)},n⩾b.}\hfill \end{array}\right.$$

(2)

Let $M=\underset{n\to \infty }{lim}\tilde{\mu }\left(n\right)$; if the limit exists, for sufficiently high values of n we can assume that $\tilde{\mu }\left(n\right)=M$ and the second expression in (2), valid for $n\ge b$, reduces to

$$M^{b+1}-(\lambda +\mu )M^b+{\lambda }^b\mu =0,$$

(3)

and the existence of the equivalent birth–death process depends upon the existence of a solution to the previous equation, henceforth referred to as the characteristic equation of the system, satisfying stability condition (1).

Although (3) provides an implicit relation between the asymptotic service rate M and all the queuing characteristics ($\lambda$, $\mu$ and b), i.e.,

$$f(M,\lambda ,\mu ,b)=M^{b+1}-(\lambda +\mu )M^b+{\lambda }^b\mu =0,$$

in the following, for the sake of clarity, we will highlight just the dependence on the relevant variables and omit the others, taking constant values in the considered settings (see, for instance, the calculation of the total derivatives of M with respect to $\lambda$, $\mu$, and b in Section 2.2 as well as the derivation of the optimal arrival rate in Section 3).

As it is shown in [26], for any consistent set of queue parameters, the characteristic equation has only one root satisfying the stability condition, which will be denoted by M in the following (in analogy with the notation already used in [29]). Some interesting mathematical properties of M will be further discussed in Section 2.2, where new material is presented, while this subsection just summarizes previous results and can be skipped by readers familiar with [29].

Taking into account that M is a solution of (3), from the second set of equations in (2), it follows that

$$\tilde{\mu }\left(b\right)=\tilde{\mu }(b+1)=\tilde{\mu }(b+2)=\dots =M,$$

and the equivalent service rates for the remaining states can be easily derived recursively (as also shown in Section 2.3). Moreover, the state probabilities can be written as (see [29] for details)

$$\left\{\begin{array}{c}\pi \left(n\right)=\pi \left(0\right){\displaystyle \sum _{i=0}^n}{x}^i,1\le n\le b-1,\hfill \\ \pi \left(n\right)=\pi \left(0\right)x^{n-b}\frac{\lambda }{\mu },n\ge b,\hfill \end{array}\right.$$

where $x=\frac{\lambda }{M}$ and the probability of an empty system $\pi \left(0\right)$ is

$$\pi \left(0\right)=\frac{1-x}{b}.$$

Then, taking into account Little’s law [19] and the definition of the average number of customers in the system, the average sojourn time is given by

$$\overline{u}=\frac{1}{\lambda }\sum _{n=0}^{\infty }n\pi \left(n\right),$$

which, after non trivial manipulations (see [29] for details), leads to the following compact expression:

$$\overline{u}=\frac{b-1}{2\lambda }+\frac{1}{M-\lambda }.$$

(4)

2.2. A Deeper Insight into the Roots of the Characteristic Equation

Let us consider the characteristic Equation (3). As shown in [26], $f\left(M\right)=0$ has two positive roots, the largest of which belongs to the open interval

$$(M^{\prime },M^{\prime \prime })=\left(\frac{b(\lambda +\mu )}{b+1},\frac{{(\lambda +\mu )}^{b+1}-{\lambda }^b\mu }{{(\lambda +\mu )}^b}\right)$$

(5)

and satisfies the stability condition. Note that

$$M^{\prime }=\frac{b(\lambda +\mu )}{b+1}$$

is the only positive root of $f^{\prime }\left(M\right)=0$ and $\rho <1$ (i.e., $\lambda <b\mu$) implies that $M>M^{\prime }>\lambda$.

Let us consider how this root depends on $\lambda$. Indeed, for fixed values of $\mu$ and b, Equation (3) defines an implicit function of M and $\lambda$

$$f(M,\lambda )=0$$

and, taking into account the differentiation rules for implicit functions (see, for instance, [30]), we get:

$$\frac{dM}{d\lambda }=-\frac{f_{\lambda }^{\prime }(M,\lambda )}{f_M^{\prime }(M,\lambda )},$$

(6)

where

$$f_{\lambda }^{\prime }(M,\lambda )=\frac{\partial f(M,\lambda )}{\partial \lambda }=b{\lambda }^{b-1}\mu -M^b$$

(7)

and

$$f_M^{\prime }(M,\lambda )=\frac{\partial f(M,\lambda )}{\partial M}=(b+1)M^b-b(\lambda +\mu )M^{b-1}.$$

(8)

As a direct implication of the inequality $M>M^{\prime }$ in (5), it is easy to see that $f_M^{\prime }(M,\lambda )>0$ when M is the root of the characteristic equation satisfying the stability condition. Moreover, taking into account (3), $f_M^{\prime }(M,\lambda )$ can be rewritten in a more compact form, i.e.,

$$f_M^{\prime }(M,\lambda )=M^b+b\left(M^b-(\lambda +\mu )M^{b-1}\right)=M^b-\frac{b{\lambda }^b\mu }{M},$$

(9)

which will be used in the next section.

Taking advantage of the same equality (3) that can be rewritten as

$$M^b\left(M-(\lambda +\mu )\right)=-{\lambda }^b\mu ,$$

the expression of $f_{\lambda }^{\prime }(M,\lambda )$ becomes

$$f_{\lambda }^{\prime }(M,\lambda )=b{\lambda }^{b-1}\mu -M^b=\frac{{\lambda }^{b-1}\mu }{M-(\lambda +\mu )}\left(\lambda +b(M-(\lambda +\mu \left)\right)\right),$$

and it is easy to see that for the value of M of interest

$$f_{\lambda }^{\prime }(M,\lambda )<0⟺M>M_0=\lambda +\mu -\lambda /b.$$

To prove the last inequality, let us consider the function

$$f\left(x\right)=x^{b+1}-(\lambda +\mu )x^b+{\lambda }^b\mu ,$$

which is an increasing function for $x>M^{\prime }$, since $M^{\prime }$ is the only positive root of $f^{\prime }\left(x\right)=0$, as stated above. Moreover, by definition $f\left(M\right)=0$; hence, it is enough to show that $f\left(M_0\right)<0$, where the latter inequality can be verified by direct calculation, taking into account that $\lambda <b\mu$ and truncating the binomial expansion

$${\left(\lambda +(\mu -\lambda /b)\right)}^b>{\lambda }^b+b{\lambda }^{b-1}(\mu -\lambda /b)=b\mu {\lambda }^{b-1}.$$

Indeed, for $x=M_0=\lambda +\mu -\lambda /b$, we have:

$$\begin{array}{cc}\hfill f\left(M_0\right)& ={(\lambda +\mu -\lambda /b)}^b\left[\lambda +\mu -\lambda /b-(\lambda +\mu )\right]+{\lambda }^b\mu \hfill \\ \hfill & =-\frac{\lambda }{b}{\left(\lambda +(\mu -\lambda /b)\right)}^b+{\lambda }^b\mu <-\frac{\lambda }{b}b\mu {\lambda }^{b-1}+{\lambda }^b\mu =0.\hfill \end{array}$$

As a side result, we have found another lower bound $M_0$ for the largest root of (3), with $M_0=M^{\prime }+\Delta$, where

$$\Delta =M_0-M^{\prime }=\frac{1}{b+1}(\lambda +\mu )-\frac{\lambda }{b}=\frac{1}{b(b+1)}(b\mu -\lambda )>0.$$

In other words, we improved the relation (5), with a tighter lower bound

$$(M_0,M^{\prime \prime })=\left(\mu +\lambda \frac{b-1}{b},\frac{{(\lambda +\mu )}^{b+1}-{\lambda }^b\mu }{{(\lambda +\mu )}^b}\right)\subset (M^{\prime },M^{\prime \prime }).$$

(10)

Finally, if M is the largest positive root of the characteristic equation (and hence $M>M_0>M^{\prime }$), it is easy to see from Equation (6) that

$$\frac{dM}{d\lambda }=-\frac{f_{\lambda }^{\prime }(M,\lambda )}{f_M^{\prime }(M,\lambda )}>0$$

and so M is an increasing function of $\lambda$.

In a similar way, it is easy to prove that M is also an increasing function of $\mu$ and b, considering the batch size as a continuous parameter in order to take the derivative (note that the latter property has already been proven in [29] considering discrete values of b). Indeed, it is straightforward to verify that the partial derivatives with respect to M are still given by (8),

$$f_{\mu }^{\prime }(M,\mu )=\frac{\partial f(M,\mu )}{\partial \mu }={\lambda }^b-M^b<0$$

and

$$f_b^{\prime }(M,b)=\frac{\partial f(M,b)}{\partial b}=\left(M^{b+1}-(\lambda +\mu )M^b\right)logM+\mu {\lambda }^{b}log\lambda ,$$

which can be rewritten as (again taking advantage of the characteristic Equation (3))

$$f_b^{\prime }(M,b)={\lambda }^b\mu (log\lambda -logM)<0.$$

Hence, the corresponding total derivatives

$$\frac{dM}{d\mu }=-\frac{f_{\mu }^{\prime }(M,\mu )}{f_M^{\prime }(M,\mu )}\mathrm{and}\frac{dM}{db}=-\frac{f_b^{\prime }(M,b)}{f_M^{\prime }(M,b)}$$

are positive and the root M (i.e., the asymptotic equivalent service rate) is a monotone increasing function of all the queue parameters ($\lambda$, $\mu$, and b).

Finally, it is worth mentioning another property of the roots of the characteristic equation: for a fixed value of the ratio $\eta =\lambda /\mu$, the roots are linear in $\mu$. Indeed, dividing all terms by ${\mu }^{b+1}$, the characteristic equation becomes

$${\xi }^{b+1}-(1+\eta ){\xi }^b+{\eta }^b=0,$$

where $\xi =M/\mu$. The change in variable highlights that the roots of the previous equation depend only on the ratio $\eta$ between $\lambda$ and $\mu$, so for a fixed value of $\eta$ (and a constant utilization $\rho$ if the batch size is also fixed), $M=\xi \mu$ is linear in $\mu$ (and in $\lambda$, since $\eta$ is assumed to be a constant).

2.3. Analysis of the Equivalent Birth–Death Process

Let us consider the death rates of the equivalent birth–death process and verify that all the rates are positive. Since

$$\tilde{\mu }\left(b\right)=\tilde{\mu }(b+1)=\tilde{\mu }(b+2)=\dots =M,$$

we need to verify that $\tilde{\mu }\left(n\right)>0$ for $1\le n\le b-1$. To this aim, let us focus at first on the expression of $\tilde{\mu }(b-1)$:

$$\tilde{\mu }(b-1)=\lambda -\mu \frac{{\lambda }^b}{\tilde{\mu }\left(b\right)\times \dots \times \tilde{\mu }(2b-1)}=\lambda -\mu \frac{{\lambda }^b}{M^b}.$$

Taking into account (3) and the lower bound $M_0$ in (10), it is easy to verify that

$$\tilde{\mu }(b-1)=M-\mu >M_0-\mu >0.$$

Next, let us consider the expression of

$$\tilde{\mu }(b-2)=\lambda -\mu \frac{{\lambda }^b}{(M-\mu )M^{b-1}},$$

from which by direct comparison it follows that $\tilde{\mu }(b-1)>\tilde{\mu }(b-2)$. In a similar way, it is easy to see that

$$\tilde{\mu }(b-1)>\tilde{\mu }(b-2)>\cdots >\tilde{\mu }\left(1\right).$$

Therefore, to complete the proof, it is enough to show that $\tilde{\mu }\left(1\right)>0$, where

$$\tilde{\mu }\left(1\right)=\lambda -\mu \frac{{\lambda }^b}{\tilde{\mu }\left(2\right)\times \dots \times \tilde{\mu }(b+1)}.$$

Taking advantage of the following equality, derived in [29],

$$\frac{\lambda }{\mu }=\frac{{\lambda }^b}{\tilde{\mu }\left(1\right)\tilde{\mu }\left(2\right)\times \dots \times \tilde{\mu }\left(b\right)},$$

we get

$$\tilde{\mu }\left(1\right)=\lambda -\mu \frac{\tilde{\mu }\left(1\right){\lambda }^b}{\tilde{\mu }\left(1\right)\tilde{\mu }\left(2\right)\times \dots \times \tilde{\mu }(b+1)}=\lambda -\mu \frac{\lambda }{\mu }\frac{\tilde{\mu }\left(1\right)}{M}.$$

Therefore,

$$\tilde{\mu }\left(1\right)=\frac{\lambda M}{M+\lambda }>0$$

and

$$\tilde{\mu }(b-1)>\tilde{\mu }(b-2)>\cdots >\tilde{\mu }\left(1\right)>0,$$

completing the proof.

3. Optimal Input Rate for Batch Services

Let us consider the general expression of the average sojourn time in $M/M^b/1$ queuing systems, given by (4), with the constraint (3) that provides the implicit relation between M and the queue parameters, i.e.,

$$\left\{\begin{array}{c}{\overline{u}}_b(\lambda ,M)=\frac{b-1}{2\lambda }+\frac{1}{M-\lambda },\hfill \\ f(M,\lambda )=M^{b+1}-(\lambda +\mu )M^b+{\lambda }^b\mu =0,\hfill \end{array}\right.$$

(11)

where we highlighted the dependence of the sojourn time on $\lambda$ (directly and through M), while $\mu$ and b are fixed parameters, determined by the specific queuing system under study, since we are interested in the optimization with respect to $\lambda$.

It is well known that in the classical $M/M/1$ queue, the average sojourn time

$${\overline{u}}_1=\frac{1}{\mu -\lambda },$$

which can be easily derived from (4) noticing that $b=1$ and the largest root of the characteristic equation is $M=\mu$, is a monotone increasing function of $\lambda$ and ${\overline{u}}_1\to \infty$ as the utilization $\rho \to 1$. Instead, in the case of batch services (i.e., $b\ge 2$), there is an additional delay due to the completion of a batch, represented by the first term in (4). Indeed, ${\overline{u}}_b\to \infty$ also as $\lambda \to 0$; hence, it is reasonable to expect the existence of some intermediate values of $\lambda$ between 0 and $b\mu$ (stability condition), for which the sojourn time is minimal.

This consideration is confirmed by Figure 1, in which the behavior of ${\overline{u}}_b$ is plotted as a function of $\lambda$ for $b=1,2,3$. Without loss of generality, as further discussed at the beginning of Section 4.2, $\mu =1$ is assumed in the plots.

In other words, we are looking for the optimal arrival rate $\lambda$ (for fixed values of $b\ge 2$ and $\mu$), i.e.,

$${\overline{u}}_b\left({\lambda }_{\mathrm{opt}}\right)=\underset{\lambda }{min}{\overline{u}}_b\left(\lambda \right),$$

that corresponds to the search for a conditional minimum [30] of a function of several variables, which can be obtained through the total derivative

$$\frac{d{\overline{u}}_b}{d\lambda }=0.$$

(12)

In the next Section 3.1, we derive the mathematical condition for the existence of a unique minimum of the sojourn time, while in Section 3.2 and Section 3.3, we determine the mathematical expression of the optimal arrival rate and provide its approximation, respectively. Finally, the goodness of the approximation will be discussed in Section 4.

3.1. Condition for the Existence of an Optimal Arrival Rate

In the calculation of the total derivative (12), it is important to take into account that the variables $\lambda$ and M in (11) are not independent. The general solution is provided by the method of Lagrange multipliers [30], but in our case, the same condition can be achieved through direct calculation. Indeed, omitting the arguments of the functions, we can write:

$$\begin{array}{cc}\hfill \frac{d\overline{u}}{d\lambda }& =\frac{\partial \overline{u}}{\partial \lambda }+\frac{\partial \overline{u}}{\partial M}\frac{dM}{d\lambda }\hfill \\ \hfill & =-\frac{b-1}{2{\lambda }^2}+\frac{1}{{(M-\lambda )}^2}-\frac{1}{{(M-\lambda )}^2}\frac{dM}{d\lambda }\hfill \\ \hfill & =-\frac{b-1}{2{\lambda }^2}+\frac{1}{{(M-\lambda )}^2}\left(1-\frac{dM}{d\lambda }\right)\hfill \end{array}$$

and, taking into account the expressions in Section 2.2,

$$\begin{array}{cc}\hfill \frac{d\overline{u}}{d\lambda }& =-\frac{b-1}{2{\lambda }^2}+\frac{1}{{(M-\lambda )}^2}\left(1+\frac{f_{\lambda }^{\prime }(M,\lambda )}{f_M^{\prime }(M,\lambda )}\right)\hfill \\ \hfill & =-\frac{b-1}{2{\lambda }^2}+\frac{1}{{(M-\lambda )}^2}\frac{b{\lambda }^{b-1}\mu (M-\lambda )}{M^{b+1}-b{\lambda }^b\mu }\hfill \\ \hfill & =\frac{-(b-1)\left((M-\lambda )(M^{b+1}-b{\lambda }^b\mu )-\frac{2b}{b-1}{\lambda }^{b-1}\mu \right)}{2{\lambda }^2(M-\lambda )(M^{b+1}-b{\lambda }^b\mu )}\hfill \\ \hfill & =\frac{-\mu (b-1)\phi (M,\lambda )}{2{\lambda }^2(M-\lambda )(M^{b+1}-b{\lambda }^b\mu )},\hfill \end{array}$$

where

$$\mu \phi (M,\lambda )=(M-\lambda )(M^{b+1}-b{\lambda }^b\mu )-\frac{2b}{b-1}{\lambda }^{b-1}\mu .$$

Since $M>\lambda$ and, according to Equation (9),

$$f_M^{\prime }(M,\lambda )=M^b-\frac{b{\lambda }^b\mu }{M}>0,$$

it is easy to see that

$$\frac{d\overline{u}}{d\lambda }\ge 0⟺\phi (M,\lambda )\le 0.$$

(13)

Taking into account the characteristic Equation (3), $\mu \phi (M,\lambda )$ can be rewritten as follows:

$$\begin{array}{cc}\hfill \mu \phi (M,\lambda )& =(M-\lambda )(M^{b+1}-b{\lambda }^b\mu )-\frac{2b}{b-1}{\lambda }^{b+1}\mu \hfill \\ \hfill & =(M-\lambda )\left((\lambda +\mu )M^b-(b+1){\lambda }^b\mu \right)-\frac{2b}{b-1}{\lambda }^{b+1}\mu \hfill \\ \hfill & =\mu M^{b+1}+\lambda \left(M^{b+1}-(\lambda +\mu )M^b\right)-(b+1)M{\lambda }^b\mu +(b-1){\lambda }^{b+1}\mu -\frac{2b}{b-1}{\lambda }^{b+1}\mu \hfill \\ \hfill & =\mu \left(M^{b+1}-(b+1){\lambda }^{b}M+b{\lambda }^{b+1}\right)-\frac{2b}{b-1}{\lambda }^{b+1}\mu \hfill \\ \hfill & =\mu \left(M^{b+1}-(b+1){\lambda }^{b}M+b\frac{b-3}{b-1}{\lambda }^{b+1}\right),\hfill \end{array}$$

i.e.,

$$\phi (M,\lambda )=M^{b+1}-(b+1){\lambda }^{b}M+b\frac{b-3}{b-1}{\lambda }^{b+1}.$$

Noticing that $\phi (M,\lambda )$ is a homogeneous polynomial of degree $b+1$, we can rewrite $\phi (M,\lambda )$ in terms of a new function $\psi$ of the ratio $M/\lambda$:

$$\phi (M,\lambda )={\lambda }^{b+1}\left({\left(\frac{M}{\lambda }\right)}^{b+1}-(b+1)\frac{M}{\lambda }+b\frac{b-3}{b-1}\right)={\lambda }^{b+1}\psi \left(\frac{M}{\lambda }\right)$$

with

$$\psi \left(x\right)=x^{b+1}-(b+1)x+b\frac{b-3}{b-1}.$$

Since $\lambda$ represents the arrival rate and cannot be negative, (13) is equivalent to

$$\frac{d\overline{u}}{d\lambda }\ge 0⟺\left\{\begin{array}{cc}\psi \left(x\right)\hfill & =x^{b+1}-(b+1)x+b\frac{b-3}{b+1}\le 0,\hfill \\ \hfill & x>1,\hfill \end{array}\right.$$

(14)

where the condition $x>1$ is derived from the tighter lower bound in (10) and the stability condition $\lambda <b\mu$. Indeed,

$$M>M_0=\mu +\lambda -\frac{\lambda }{b}>\frac{\lambda }{b}+\lambda -\frac{\lambda }{b}=\lambda .$$

In this way, the existence of a conditional minimum for the average sojourn time is determined by the behavior of $\psi \left(x\right)$ for $x>1$.

It is easy to see that $\psi \left(x\right)$ is monotone increasing for $x>1$, since

$${\psi }^{\prime }\left(x\right)=(b+1)\left(x^b-1\right).$$

Moreover, $\psi \left(x\right)\to +\infty$ as $x\to +\infty$ and

$$\psi \left(1\right)=-\frac{2b}{b-1}<0,$$

so there exists a unique root $x_0>1$ of the equation $\psi \left(x\right)=0$, which corresponds to the unique extremum of ${\overline{u}}_b(\lambda ,M)$, i.e.,

$$x_0=\frac{M}{{\lambda }_{\mathrm{opt}}}.$$

(15)

Figure 2 provides a visual illustration of the behavior of $\psi \left(x\right)$ for different values of b, highlighting the existence and uniqueness of $x_0$.

Finally, taking into account the general shape of ${\overline{u}}_b$ that diverges to $+\infty$ for $\lambda \to 0$ and $\lambda \to b\mu$, the extremum corresponds to the global minimum of the sojourn time.

3.2. Calculation of the Optimal Rate

The optimal input rate ${\lambda }_{\mathrm{opt}}$ can be easily derived from the value of $x_0$ taking into account the characteristic equation. To this aim, let us divide (3) by ${\lambda }^{b+1}$, taking into account (15) for $\lambda ={\lambda }_{\mathrm{opt}}$, we get

$$x_0^{b+1}-\left(1+\frac{\mu }{{\lambda }_{\mathrm{opt}}}\right)x_0^b+\frac{\mu }{{\lambda }_{\mathrm{opt}}}=0$$

and, rearranging the terms:

$${\lambda }_{\mathrm{opt}}=\mu \frac{x_0^b-1}{x_0^{b+1}-x_0^b}=\mu \frac{x_0^b-1}{x_0^b(x_0-1)},$$

(16)

i.e., the optimal input rate is proportional to $\mu$. The corresponding minimum sojourn time can be simply derived by direct substitution in (4):

$${\overline{u}}_{\min }=\frac{b-1}{2{\lambda }_{\mathrm{opt}}}+\frac{1}{x_0{\lambda }_{\mathrm{opt}}-{\lambda }_{\mathrm{opt}}}=\frac{1}{{\lambda }_{\mathrm{opt}}}\left(\frac{b-1}{2}+\frac{1}{x_0-1}\right).$$

(17)

Unfortunately, there is no general closed-form expression for $x_0=x_0\left(b\right)$, so the previous equation can be used to analytically derive ${\lambda }_{\mathrm{opt}}$ just for $b=2$ and $b=3$. In more detail, for $b=2$:

$${\psi }_2\left(x\right)=x^3-3x-2=(x-2){(x+1)}^2$$

and the unique solution of ${\psi }_2\left(x\right)=0$ satisfying the condition $x>1$ is $x_0=2$. Hence, from (16):

$${\lambda }_{2,\mathrm{opt}}=\mu \frac{x_0^2-1}{x_0^2(x_0-1)}=\frac{3}{4}\mu .$$

In a similar way, for $b=3$:

$${\psi }_3\left(x\right)=x^4-4x=x(x^3-4)\Rightarrow x_0=\sqrt[3]{4}$$

and

$${\lambda }_{3,\mathrm{opt}}=\mu \frac{x_0^3-1}{x_0^3(x_0-1)}=\frac{3}{4}\frac{\mu }{\sqrt[3]{4}-1}.$$

Although for higher values of b$x_0$ can be easily determined using numerical methods, in the next subsection, we provide an approximate formula, based on the assumption that $x_0\to 1$ as $b\to \infty$, as suggested by Figure 2.

3.3. Approximate Expression for the Optimal Input Rate

Assume that

$$x_0=1+\frac{\alpha }{b}+o\left(\frac{1}{b}\right),$$

where the coefficient $\alpha >0$ is determined from the condition $\psi \left(x_0\right)=0$. For $b\to \infty$,

$$\frac{b-3}{b-1}\stackrel{b\to \infty }{\to }1-\frac{2}{b}+o\left(\frac{1}{b}\right)$$

and

$${\left(1+\frac{\alpha }{b}+o\left(\frac{1}{b}\right)\right)}^{b+1}\stackrel{b\to \infty }{\to }{e}^{\alpha }+o\left(1\right),$$

so we can rewrite the condition $\psi \left(x_0\right)=0$ as follows:

$$e^{\alpha }-(b+1)\left(1+\frac{\alpha }{b}\right)+b-2+o\left(1\right)=0$$

and, neglecting the infinitesimal terms:

$$e^{\alpha }-(\alpha +3)=0.$$

(18)

As shown in Figure 3, the latter equation has two roots, ${\alpha }_1\approx -2.947531$ and ${\alpha }_2\approx 1.505241$. Since $x>1$, and hence $\alpha >0$, the following approximation

$$x_0\approx 1+\frac{1.505241}{b}$$

(19)

can be used to estimate the optimal input rate

$${\lambda }_{\mathrm{appr}}=\mu \frac{1-{\left(1+\alpha /b\right)}^{-b}}{\alpha /b}$$

(20)

as well as the corresponding minimum value of the average sojourn time from (17):

$${\overline{u}}_{\mathrm{appr}}=\frac{1}{{\lambda }_{\mathrm{appr}}}\left(\frac{b-1}{2}+\frac{b}{\alpha }\right)=\frac{b(2+\alpha )-\alpha }{2\alpha {\lambda }_{\mathrm{appr}}}.$$

(21)

In the next section, at first we will verify the correctness of the approximation for $x_0$, and then calculate the relative error for both the optimal input rate and the corresponding average sojourn time.

4. Discussion

The goodness of the first order approximation and the behavior of the minimum sojourn time (including its asymptotic value as $b\to \infty$) are illustrated in the next two subsections, while some remarks about optimization with respect to the other queue parameters are presented in Section 4.3. Finally, extensions to this work are briefly investigated in Section 4.4.

4.1. Numerical Validation of the First Order Approximation

As already stated above, the function $\psi \left(x\right)$ has a unique solution satisfying the stability condition ($x_0>1$), which can be found by numerical methods for $b\ge 4$ (exact values are available only for $b=2$ and $b=3$). However, for a better understanding of the service mechanism as a function of the batch size, it would be useful to obtain an analytical expression (or, at least, some kind of approximation) of $x_0$ as a function of b. Hence, for the sake of simplicity, in Section 3.3, we proposed a first order approximation, given by (19).

In order to assess the goodness of (19), we compare our approximation with the numerical solutions for $b\ge 4$, which can be easily found, for instance, with the bisection method applied to the interval $[1,2]$, since $\psi \left(2\right)>0$ for $b>3$ as can be straightforwardly checked by direct substitution. As highlighted in

Table 1. Comparison between exact and approximate roots of $\psi \left(x\right)$.

b	${\mathit{x}}_0$	${\mathit{x}}_{\mathbf{appr}}$	RE (%)
4	1.419552	1.376310	3.04613
5	1.327085	1.301048	1.96194
6	1.268258	1.250874	1.37074
7	1.227461	1.215034	1.01239
8	1.197479	1.188155	0.77859
9	1.174502	1.167249	0.617527
10	1.156327	1.150524	0.501826
20	1.076636	1.075262	0.127612
30	1.050775	1.050175	0.057092
40	1.037966	1.037631	0.0322264
60	1.025235	1.025087	0.0143748
80	1.018898	1.018816	0.00810094
100	1.015105	1.015052	0.00519053
200	1.007539	1.007526	0.00130072
300	1.005023	1.005017	0.0005786
400	1.003766	1.003763	0.000325619
500	1.003013	1.003010	0.000208465
600	1.002510	1.002509	0.000144804
700	1.002151	1.002150	0.000106408
800	1.001882	1.001882	$8.148\times {10}^{-5}$
900	1.001673	1.001672	$6.439\times {10}^{-5}$
1000	1.001506	1.001505	$5.217\times {10}^{-5}$

, the root $x_0$ converges to 1 for high values of b; moreover, the relative error decreases monotonically and is already below 1% for $b=8$. Hence, our assumption seems to be adequate and it is meaningful to check the validity of the corresponding approximations (20) and (21) for the optimal arrival rate and the minimum sojourn time.

4.2. Evaluation of the Optimal Rate and Corresponding Sojourn Time

At first, let us consider the general expression of the sojourn time, focusing on its dependence on the batch service rate $\mu$ for a fixed value of $\eta =\lambda /\mu$. Although (4) does not depend explicitly on $\mu$, taking advantage of the notation introduced in Section 2.2, i.e., $M=\mu \xi$ and $\lambda =\eta \mu$, the average sojourn time can be rewritten as

$$\overline{u}=\frac{1}{\mu }\left(\frac{b-1}{2\eta }+\frac{1}{\xi -\eta }\right).$$

Since $\xi$ is independent of $\mu$, for a fixed value of $\eta$, the quantity in brackets does not depend on $\mu$; therefore, the sojourn time $\overline{u}$ is inversely proportional to $\mu$, and its minimum is achieved for the same value of $\eta =\eta \left(b\right)$, in agreement with Equation (16). Hence, without loss of generality, in the numerical analysis we will consider the case $\mu =1$.

Table 2. Numerical results—${\lambda }_{\mathrm{opt}}$ and sojourn time ($\mu =1$).

b	${\mathit{\lambda }}_{\mathbf{opt}}$	${\mathit{\lambda }}_{\mathbf{appr}}$	RE (%)	${\overline{\mathit{u}}}_{\mathbf{min}}$	${\overline{\mathit{u}}}_{\mathbf{appr}}$	RE (%)
2	0.7500	0.8961	19.4839	2.0000	2.0407	2.0328
3	1.2768	1.4046	10.0058	2.1165	2.1309	0.6803
4	1.7965	1.9168	6.6928	2.1617	2.1689	0.3372
5	2.3146	2.4307	5.0175	2.1850	2.1894	0.2008
6	2.8320	2.9455	4.009	2.1991	2.2020	0.1332
7	3.3491	3.4609	3.3371	2.2084	2.2105	0.0947
8	3.8662	3.9766	2.857	2.2151	2.2166	0.0708
9	4.3831	4.4926	2.4976	2.2200	2.2212	0.0549
10	4.9001	5.0088	2.2182	2.2238	2.2248	0.0439
20	10.0689	10.1742	1.0460	2.2394	2.2397	0.0103
30	15.2376	15.3419	0.6841	2.2441	2.2442	0.0045
40	20.4064	20.5101	0.5083	2.2463	2.2464	0.0025
60	30.7440	30.8472	0.3357	2.2485	2.2485	0.0011
80	41.0817	41.1846	0.2506	2.24955	2.24957	$6.15\times {10}^{-4}$
100	51.4194	51.5223	0.1999	2.25018	2.25019	$3.92\times {10}^{-4}$
200	103.108	103.210	0.0994	2.25141	2.25141	$9.76\times {10}^{-5}$
300	154.796	154.899	0.0661	2.25182	2.25182	$4.33\times {10}^{-5}$
400	206.485	206.587	0.0496	2.25202	2.25202	$2.43\times {10}^{-5}$
500	258.173	258.275	0.0396	2.25214	2.25214	$1.56\times {10}^{-5}$
600	309.862	309.964	0.0330	2.25222	2.25222	$1.08\times {10}^{-5}$
700	361.550	361.652	0.0283	2.25228	2.25228	$7.94\times {10}^{-6}$
800	413.234	413.341	0.0248	2.25232	2.25232	$6.08\times {10}^{-6}$
900	464.927	465.029	0.0220	2.25235	2.25235	$4.80\times {10}^{-6}$
1000	516.616	516.718	0.0198	2.25238	2.25238	$3.89\times {10}^{-6}$

highlights the effect of the previous approximation of $x_0$ on the values of the optimal input rate and the corresponding minimum sojourn time. For the sake of completeness we also report the cases $b=2$ and $b=3$ in italics, for which exact analytical values are available and hence it is meaningless to use the approximate value of $x_0$. As expected, both relative errors are decreasing functions of b (in agreement with Table 1 since all the values are based on $x_0$ and $x_{\mathrm{appr}}$).

The numerical values in Table 2 highlight two relevant properties of the sojourn time:

• The shape of the curve ${\overline{u}}_{\mathrm{opt}}\left(\lambda \right)$ is quite flat around its minimum; indeed, even significant deviations in $\lambda$ lead to small variations in the sojourn time, as shown in the first lines of the table. In practice, this implies a high stability of the system even in case of variations in the input rate;
• In spite of the fact that $M\to \lambda$ as $b\to +\infty$ (i.e., $x_0\to 1$), the increments in ${\overline{u}}_{\min }$ become increasingly smaller as b goes to $+\infty$.

To investigate the existence of an upper bound for ${\overline{u}}_{\min }$, let us estimate the optimal arrival rate ${\lambda }_{\infty }$ as $b\to \infty$, according to Equation (20):

$${\lambda }_{\mathrm{appr}}=\mu \frac{1-{\left(1+\alpha /b\right)}^{-b}}{\alpha /b}\stackrel{b\to \infty }{\to }\frac{\mu b}{\alpha }\left(1-e^{-\alpha }\right)=\frac{\mu b}{\alpha }\frac{\alpha +2}{\alpha +3},$$

where we made use of equality (18), which defines $\alpha$. The corresponding minimum value of the average sojourn time from (17) becomes

$${\overline{u}}_{\mathrm{appr}}=\frac{b(2+\alpha )-\alpha }{2\alpha {\lambda }_{\mathrm{appr}}}\stackrel{b\to \infty }{\to }\frac{1}{\mu }\frac{\alpha (\alpha +3)}{b(\alpha +2)}\frac{b(2+\alpha )-\alpha }{2\alpha }=\frac{\alpha +3}{2\mu },$$

i.e.,

$${\overline{u}}_{\infty }\approx \frac{1}{\mu }\frac{\alpha +3}{2}\approx \frac{2.25262}{\mu }$$

in perfect agreement with the last lines in Table 2. Moreover, as a side result, we obtained the following approximation of ${\lambda }_{\mathrm{opt}}$ for high values of b:

$${\lambda }_b\approx \mu \frac{\alpha +2}{\alpha (\alpha +3)}b\approx 0.51688\mu b,$$

i.e., asymptotically ${\lambda }_{\mathrm{opt}}$ grows linearly with the batch size. This final result also justifies the finite values of the sojourn time in spite of the fact that both terms inside the brackets in (4) diverge when $b\to \infty$.

4.3. Optimal Sojourn Time with Respect to Other Queue Parameters

As already stated, the average sojourn time depends (directly and/or through M) on all the queue parameters, namely b, $\lambda$, and $\mu$. To complete the analysis of $\overline{u}$, we briefly consider the optimization with respect to b and $\mu$, in both cases assuming that the other two parameters are fixed.

The optimization with respect to b (for fixed values of the input and service rates) has been deeply investigated in [29]; in a nutshell, ${\overline{u}}_b$ has a unique minimum that can be attained for the minimal batch size (i.e., $b=2$) or some intermediate value of b, depending on the specific values of $\lambda$ and $\mu$.

Finally, the optimization with respect to $\mu$ is trivial; the faster the server, the lower the sojourn time. This conclusion can be easily verified taking the total derivative of $\overline{u}$ with respect to $\mu$, as we did in Section 3.1, but with a simpler calculation. Indeed, we can write:

$$\frac{d\overline{u}}{d\mu }=\frac{\partial \overline{u}}{\partial \mu }+\frac{\partial \overline{u}}{\partial M}\frac{dM}{d\mu }=-\frac{1}{{(M-\lambda )}^2}\frac{dM}{d\mu }<0,$$

since

$$\frac{dM}{d\mu }>0$$

as shown in Section 2.2; hence, $\overline{u}$ is a decreasing function of $\mu$ and attains its minimum as $\mu \to \infty$, with

$${\overline{u}}_{\min }\left(\mu \right)=\frac{b-1}{2\lambda }.$$

4.4. Final Remarks and Future Work

This paper presented a first order approximation for the root $x_0$ of the auxiliary equation $\psi \left(x\right)=0$ which, together with the characteristic equation, permits to determine the optimal arrival rate ${\lambda }_{\mathrm{opt}}$, which minimizes the average sojourn time in the queue. It is shown that the goodness of both approximations increases with the batch size b and, even for $b=4$ (for $b=2$ and $b=3$ the exact values are available analytically), the relative errors for ${\lambda }_{\mathrm{opt}}$ and ${\overline{u}}_{\min }$ are less than 6.70% and 0.34%, respectively. This result can be exploited for the optimization of a wide variety of real-life systems, including collective transportation, batch request processes in cloud systems, and databases employing multi-query optimization.

As a future extension of this work, higher order approximations of $x_0$ deserve consideration, mainly from a theoretical point of view (as highlighted above, the first order expansion is good enough for practical purposes), to get a deeper insight into the asymptotic convergence of the sojourn time to its maximum value.

The existence of an optimal arrival rate can be employed for the minimization of the average response time of open queuing networks with batch services, where the response time is defined as the weighted sum (according to the node arrival rate) of the node sojourn times. In this case, the routing matrix is the optimization parameter, with the additional constraint that the number of possible destinations from each node should be significantly larger than the batch size; otherwise, the formulas for a queue in isolation based on the equivalent birth–death process cannot be extended to an open queuing network, as highlighted in [29], where we addressed the optimization problem with respect to the batch size.