Optimization of Open Queuing Networks with Batch Services

Abstract

In this paper, open queuing networks with Poisson arrivals and single-server infinite buffer queues are considered. Unlike traditional queuing models, customers are served (with exponential service time) in batches, so that the nodes are non-work-conserving. The main contribution of this work is the design of an efficient algorithm to find the batch sizes which minimize the average response time of the network. As preliminary steps at the basis of the proposed algorithm, an analytical expression of the average sojourn time in each node is derived, and it is shown that this function, depending on the batch size, has a single minimum. The goodness of the proposed algorithm and analytical formula were verified through a discrete-event simulation for an open network with a non-tree structure.

1. Introduction

The recent growing interest in queuing networks with batch services is motivated by their use as mathematical models of packet switching networks [1], wireless sensor networks [2], manufacturing systems [3,4,5,6], Web servers [7], data-processing systems [8] and transportation systems [9].

The analysis of queues with batch services was originally introduced in [10,11] and extended to queuing networks in [12]. A review of the main results on queues with batch services can be found in [13,14].

Such a feature as batch service significantly complicates the model analysis, since it violates the one step assumption, which is at the basis of product-form queuing networks. Indeed, in general it is not possible to obtain a product-form of the stationary distribution [6,15] for a queuing network with batch services, and this consideration strongly limits their practical applicability. To overcome this problem, approximate analysis methods, based on mean value analysis (MVA) [1] or the decomposition method [6,16], have been proposed in the literature. In [15], an infinitesimal operator was constructed to obtain a stationary distribution. In [17,18,19,20,21,22], a product form of the stationary distribution for queuing networks was obtained under certain assumptions. In more detail, Chao et al. [18] supposed that at service completion, the entire batch coalesces into a single unit, and it either leaves the system or goes to another node according to given routing probabilities; the product-form solution of such a model can provide relevant bounds for the behavior of assembly processes, especially when the system is operating under heavy traffic. In [19], conditions were obtained under which Markovian queues (i.e., with Poisson arrivals of batches of tasks and exponential task-service times), with both batch arrivals and departures, have a geometric queue length probability distribution at equilibrium. Furthermore, in [20], response time density was obtained for a tandem pair of such Markovian queues. Miyazawa and Taylor [21] considered queuing networks with batch arrivals and departures, in which additional batches arrive to the nodes when they are empty. The introduction of these extra arrivals made it possible to obtain a geometric stationary distribution, which is a (stochastic) upper bound for the original network. Such systems were further analyzed by Chao [22], who proved that the stationary distribution satisfies a set of non-standard partial balance equations and that the extra arrivals are the necessary and sufficient conditions for having a product form solution, as well as for the partial balance equation to hold.

In addition to the above-discussed probabilistic analysis, both theoretical [23,24] and practical [3,4,5,8,25] parameter optimization of queuing networks with batch services represent open issues of significant relevance. In more detail, one of the classes of optimization problems involves determining the optimal size of the batch, since it has a significant impact on network performance. Rabta and Reiner [3] proposed a general-purpose genetic algorithm and an approximate decomposition procedure for determining optimal batch sizes in a multi-product manufacturing system with the goal of minimizing the total cycle time. The problem of determining the optimal buffer size in a material handling system with constraints on throughput and cycle time was addressed in [4] by using queue decomposition and an iterative method. In more detail, in a material handling system a vehicle moves back and forth between consecutive workstations that have input/front and output/rear buffers. The system was analyzed as an open queuing network, in which input buffers corresponded to single-server queues with batch arrivals, workstations were modeled as traditional single-server queues and the output buffers corresponded to queues with batch services.

In [5], a queuing network with batch services was used to model semiconductor fabrication facilities. A procedure for distributing plant resources was proposed, which ensures the minimum cost of equipment for a given set of technical characteristics (volume and cycle time targets). Kar and Harrison [8] investigated data processing systems: the optimization problem of finding the batch size by maximizing the throughput was solved using mean-field techniques. Finally, [25] dealt with $M/G^{[a,b]}/1/N$ queues with bulking threshold a and maximum service capacity b; by means of renewal theory, busy period analysis and decomposition techniques. It was shown how increasing the bulking threshold affects performance indexes, such as the mean waiting time and the time-averaged number of loss customers. Then, a necessary and sufficient condition for the optimal bulking threshold that minimizes the expected waiting time was established, and an algorithm which guarantees to find the optimal threshold in polynomial time was proposed.

This paper extends the results published in [15,26] for queuing networks consisting of $M/M^b/1$ systems, where b denotes the batch size. In more detail, in [26], based on the fact that the stationary distributions of the considered $M/M^b/1$ queue and an $M/M/1$ queue with (adequately chosen) state-dependent service rates are identical, a product-form for the stationary distribution of a general queuing network with batch services has been obtained. Taking advantage of our previous result, in this work the analytical expression of the average response time of the queuing network is obtained, and it is shown that it has one minimum. This is the basis for the main original contribution of the paper, represented by an algorithm for finding the optimal batch sizes that minimize the mean response time.

It should be noted that similar results were recently derived in [27] for infinite servers queues: Using the factorial moment generating function, the authors obtained a product-form for the stationary distribution of the $M/M^b/\infty$ system; showed that the stationary distributions of the $M/M^1/\infty$ (i.e., $b=1$) and $M/M/\infty$ queues are identical; and derived some performance indexes. In general, the assumption of infinite servers simplifies the tractability of queuing systems, and to the best of our knowledge a similar analysis has not been extended to the single server case.

The rest of the paper is organized as follows. Section 2 describes the main features of open queuing networks with batch services, and Section 3 introduces the equivalent Jackson network, originally proposed by the authors in [26]. Then, in Section 4, the expressions of the average sojourn time in each node and of the response time of the network are derived. Section 5 presents the main original contribution of this paper, namely, an optimization algorithm for finding the vector of the batch sizes for which the average response time is minimal. Finally, Section 6 numerically illustrates the original contributions of the paper.

2. Statement of the Problem

The paper deals with large-scale networks with batch services and individual routing of the customers. In more detail, a continuous-time open queuing network N consisting of L nodes, $S_i$, $i=1,\dots ,L$, and an external traffic source $S_0$ is considered. The arrival process is Poissonian with rate ${\lambda }_0$, and customer transitions between nodes $S_i$, $i=1,\dots ,L$, are described by the routing matrix $\Theta =\left({\theta }_{ij}\right)$, $i,j=0,\dots ,L$, where ${\theta }_{ij}$ defines the transition probability from node $S_i$ to node $S_j$.

As in classic open queuing networks, each node $S_i$, $i=1,\dots ,L$, consists of an infinite capacity single-server queue, and arriving customers are put in the waiting queue if the server is busy. It is assumed that in node $S_i$ the customers are served in batches of size $b_i$. In more detail, the server stays idle until $b_i$ customers arrive at the node; if more customers are present in the waiting queue when the server is idle, then $b_i$ customers are selected in any order, while the others remain in the queue. The service times are exponentially distributed with parameter ${\mu }_i$, $i=1,\dots ,L$. At the end of the service, the destination node $S_j$ of each customer, independently of the others, is determined accordingly to the routing probability ${\theta }_{ij}$, $i=1,\dots ,L$, $j=0,1,\dots ,L$.

The following analysis is based on the assumption that the number of possible destinations from each node $S_i$ is significantly larger than the batch size. Therefore, the simultaneous arrival of two or more customers in a node has an infinitesimal probability, and the input flow in each node of N can be approximated as a Poisson stream of customers with intensity depending on ${\lambda }_0$ and $\Theta =\left({\theta }_{ij}\right)$.

The state of the network is described by a vector $s=(s_1,\dots ,s_L)$, where $s_i$ indicates the number of customers at node $S_i$. Hence, the state space of the queuing network N can be denoted as $X=\{s:s_i\ge 0\}$.

The first step is to determine the stationary distribution $\pi \left(s\right)=({\pi }_1\left(s_1\right),\dots ,{\pi }_L\left(s_L\right))$, $s\in X$, for the queuing network N, where the stationary distributions ${\pi }_i\left(s_i\right)$ in the nodes $S_i$, $s_i=0,1,\dots$, $i=1,\dots ,L$, can be calculated considering each node in isolation, as originally proved in [26] and briefly summarized in the next section.

3. Single Node Analysis and Equivalent Jackson Network

Let us consider the generic node $S_i$, $i=1,\dots ,L$, in isolation, with input rate ${\lambda }_i$ given by

$${\lambda }_i=\frac{{\omega }_i}{{\omega }_0}{\lambda }_0,i=1,\dots ,L,$$

(1)

where the vector of visitation rates $\omega =({\omega }_0,{\omega }_1,\dots ,{\omega }_L)$ is the solution of the equation $\omega \Theta =\omega$ with the normalization condition ${\sum }_{i=0}^L{\omega }_i=1$.

It is easy to derive the equilibrium equations for node $S_i$:

$$\left\{\begin{array}{c}{\lambda }_i{\pi }_i\left(n\right)={\mu }_i{\pi }_i\left(b_i\right),n=0,\hfill \\ {\lambda }_i{\pi }_i\left(n\right)={\lambda }_i{\pi }_i(n-1)+{\mu }_i{\pi }_i(b_i+n),1\le n\le b_i-1,\hfill \\ ({\lambda }_i+{\mu }_i){\pi }_i\left(n\right)={\lambda }_i{\pi }_i(n-1)+{\mu }_i{\pi }_i(b_i+n),n\ge b_i.\hfill \end{array}\right.$$

(2)

Let us introduce a birth–death process ${\xi }_i$, equivalent in steady-state probabilities to the Markov process associated with the node $S_i$. The process ${\xi }_i$ is defined on the set of states $\{0,1,\dots \}$, with birth rates ${\lambda }_i={\lambda }_i\left(n\right)$, which do not depend on the state n, $n\in \{0,1,\dots \}$, and death rates ${\tilde{\mu }}_i\left(n\right)$, $n\in \{1,2,\dots \}$. The state space and the parameters ${\lambda }_i$ of the process ${\xi }_i$ correspond to the state space $\{0,1,\dots \}$ and the input rate of node $S_i$ defined by Equation (1). To determine the rates ${\tilde{\mu }}_i\left(n\right)$, $n=1,2,\dots$, note that the steady-state probabilities of the birth–death process ${\xi }_i$ are given by

$${\pi }_i\left(k\right)={\pi }_i\left(0\right)\prod _{n=1}^k\frac{{\lambda }_i}{{\tilde{\mu }}_i\left(n\right)},k=1,2,\dots ,$$

(3)

where

$${\pi }_i\left(0\right)={\left(1+\sum _{k=1}^{\infty }\prod _{n=1}^k\frac{{\lambda }_i}{{\tilde{\mu }}_i\left(n\right)}\right)}^{-1},i=1,\dots ,L.$$

By replacing (3) in (2), the following expressions are obtained for ${\tilde{\mu }}_i\left(n\right)$, $n=1,2,\dots$:

$$\left\{\begin{array}{c}{\displaystyle {\tilde{\mu }}_i\left(n\right)={\lambda }_i-{\mu }_i\frac{{\lambda }_i^{b_i}}{{\tilde{\mu }}_i(n+1)·\dots ·{\tilde{\mu }}_i(b_i+n)},1\le n\le b_i-1,}\hfill \\ {\displaystyle {\tilde{\mu }}_i\left(n\right)={\lambda }_i+{\mu }_i-{\mu }_i\frac{{\lambda }_i^{b_i}}{{\tilde{\mu }}_i(n+1)·\dots ·{\tilde{\mu }}_i(b_i+n)},n\ge b_i.}\hfill \end{array}\right.$$

(4)

Let $M_i=\underset{n\to \infty }{lim}{\tilde{\mu }}_i\left(n\right)$; if the limit exists, then:

$${\mu }_i{\lambda }_i^{b_i}=({\lambda }_i+{\mu }_i-M_i)M_i^{b_i}$$

or

$$M_i^{b_i+1}-({\lambda }_i+{\mu }_i)M_i^{b_i}+{\lambda }_i^{b_i}{\mu }_i=0.$$

(5)

The existence of the equivalent birth–death process ${\xi }_i$ depends upon the existence of a positive solution of the previous equation, which satisfies the stability condition for each node $S_i$.

As proved in [26], taking into account the stability condition, Equation (5) has a unique root $M_i$, belonging to the open interval

$$\left(\frac{b_i({\lambda }_i+{\mu }_i)}{b_i+1},\frac{{({\lambda }_i+{\mu }_i)}^{b_i+1}-{\lambda }_i^{b_i}{\mu }_i}{{({\lambda }_i+{\mu }_i)}^{b_i}}\right),$$

(6)

which in the general case can be determined numerically. From (4) it follows that

$${\tilde{\mu }}_i\left(b_i\right)={\tilde{\mu }}_i(b_i+1)={\tilde{\mu }}_i(b_i+2)=\dots =M_i,$$

and the service rates ${\tilde{\mu }}_i(b_i-1)$, ${\tilde{\mu }}_i(b_i-2)$,…, ${\tilde{\mu }}_i\left(1\right)$ can be easily derived recursively.

Since the equivalent process ${\xi }_i$ can be built for any node $S_i$, it is possible to define a Jackson network $\tilde{N}$ with nodes ${\tilde{S}}_i$, equivalent in stationary distribution to the original queuing network N with batch services. According to the previous results for the single node, $\tilde{N}$ has state-dependent service rates ${\tilde{\mu }}_i\left(n\right)$, where n is the number of customers in the node ${\tilde{S}}_i$, $n=1,2,\dots$, $i=1,\dots ,L$, and its arrival rates and routing probabilities are the same as in the original queuing network N.

Moreover, N and its equivalent network $\tilde{N}$ are stable if and only if the utilization coefficient in the node $S_i$, $i=1,\dots ,L$,

$${\rho }_i=\frac{{\lambda }_i}{b_i{\mu }_i}<1.$$

Under such conditions, the stationary distribution for $\tilde{N}$ is given by

$$\pi \left(s\right)=\prod _{i=1}^L{\pi }_i\left(s_i\right),s\in X,$$

(7)

where

$${\pi }_i\left(s_i\right)={\pi }_i\left(0\right)\prod _{n=1}^{s_i}\frac{{\lambda }_i}{{\tilde{\mu }}_i\left(n\right)}.$$

4. Stationary Response Time

The stationary distribution $\pi \left(s\right)$ for $\tilde{N}$ allows us to calculate the average values of various performance parameters.

To characterized the queuing network as a whole, the most relevant index is the average time spent by a customer in the queuing network, known as response time:

$$\overline{\tau }=\frac{1}{{\lambda }_0}\sum _{i=1}^L{\lambda }_i{\overline{u}}_i,$$

which is given by the weighted sum of the average sojourn times ${\overline{u}}_i$ in all the nodes $S_i$.

An elegant closed-form expression for the average sojourn time is provided by the following Theorem, which is valid for a generic node $S_i$, $i=1,\dots ,L$. For sake of simplicity, in the proof of the theorem, the subscript i identifying the node is omitted.

Theorem 1.

The average sojourn time in a queuing system with batch services of fixed size b is given by

$$\overline{u}=\frac{b-1}{2\lambda }+\frac{1}{M-\lambda },$$

(8)

where λ is the arrival rate and M is the unique solution of (5), satisfying the stability condition.

Proof of Theorem 1.

By substituting (3) into the first equation of (2), we get

$$\frac{\lambda }{\mu }=\frac{{\lambda }^b}{\tilde{\mu }\left(1\right)\tilde{\mu }\left(2\right)\dots \tilde{\mu }\left(b\right)},$$

and by taking into account the latter equality:

$$\pi (n+b)=\pi \left(0\right)\frac{\lambda }{\mu }{\left(\frac{\lambda }{M}\right)}^n,n\ge 0,b\ge 1.$$

(9)

Having defined $x=\frac{\lambda }{M}$, the system (2) can be rewritten as follows:

$$\left\{\begin{array}{c}\pi \left(b\right)=\pi \left(0\right)\frac{\lambda }{\mu },\hfill \\ \pi \left(n\right)=\pi \left(0\right){\displaystyle \sum _{i=0}^n}{x}^i,1\le n\le b-1,\hfill \\ \pi \left(n\right)=\pi \left(0\right)x^{n-b}\frac{\lambda }{\mu },n\ge b.\hfill \end{array}\right.$$

(10)

When taking into account Little’s law and the definition of the average number of customers in the system, the average sojourn time is given by

$$\overline{u}=\frac{1}{\lambda }\sum _{n=0}^{\infty }n\pi \left(n\right),$$

and by substituting the expressions of the state probabilities given by (10), the previous expression becomes

$$\begin{gathered} \overline{u}=\frac{1}{\lambda }(\pi \left(0\right)(1+x)+2\pi \left(0\right)(1+x+x^2)+\dots +(b-1)\pi \left(0\right)(1+x+\dots +x^{b-1})+ \\ +\sum _{n=b}^{\infty }n\pi \left(0\right)x^{n-b}\frac{\lambda }{\mu }) \end{gathered}$$

or rearrange the terms according to the powers of x:

$$\overline{u}=\frac{\pi \left(0\right)}{\lambda }\left((1+x)\sum _{i=1}^{b-1}i+x^2\sum _{i=2}^{b-1}i+\dots +x^{b-1}(b-1)+\sum _{n=b}^{\infty }n{x}^{n-b}\frac{\lambda }{\mu }\right).$$

The next step of the proof consists in finding the value of $\pi \left(0\right)$. To this end, the system (2) can be rewritten in the form

$$\left\{\begin{array}{c}\lambda \pi \left(0\right)=\mu \pi \left(b\right),\hfill \\ \lambda \pi \left(1\right)=\mu \pi \left(b\right)+\mu \pi (b+1),\hfill \\ \dots \hfill \\ \lambda \pi (b-1)=\mu \left(\pi \right(b)+\pi (b+1)+\dots +\pi (2b-1\left)\right),\hfill \\ \lambda \pi \left(b\right)=\mu \left(\pi \right(b+1)+\pi (b+2)+\dots +\pi (2b+1\left)\right),\hfill \\ \dots \hfill \end{array}\right.$$

By summing up all the equations of this system, we obtain

$$\lambda \sum _{n=0}^{\infty }\pi \left(n\right)=b\mu \sum _{n=b}^{\infty }\pi \left(n\right)$$

or

$$\sum _{n=b}^{\infty }\pi \left(n\right)=\frac{\lambda }{b\mu }.$$

(11)

By substituting (9) into (11) and applying the formula for the sum of an infinite geometric progression, the probability of an empty system becomes

$$\pi \left(0\right)=\frac{1-x}{b}.$$

The last part of the proof consists in manipulating the expression of $\overline{u}$, which can be significantly simplified thanks to the specific structure of the state probabilities of the system and the properties of M. In more detail:

$$\overline{u}=\frac{1-x}{b\lambda }\left((1+x)\sum _{i=1}^{b-1}i+x^2\sum _{i=2}^{b-1}i+\dots +x^{b-1}(b-1)+\sum _{n=b}^{\infty }n{x}^{n-b}\frac{\lambda }{\mu }\right)$$

or, by rearranging the terms:

$$\begin{gathered} \overline{u}=\frac{1}{b\lambda }\left((1-x^2)\sum _{i=1}^{b-1}i+(1-x)x^2\sum _{i=2}^{b-1}i+\dots +(1-x)x^{b-1}(b-1)\right)+ \\ +\frac{1-x}{b\lambda }\sum _{n=b}^{\infty }n{x}^{n-b}\frac{\lambda }{\mu }=\frac{1}{b\lambda }(\sum _{i=1}^{b-1}i-[\left(\sum _{i=1}^{b-1}i-\sum _{i=2}^{b-1}i\right)x^2+\left(\sum _{i=2}^{b-1}i-\sum _{i=3}^{b-1}i\right)x^3+\dots \\ +((b-2)+(b-1)-(b-1))x^{b-1}+(b-1)x^b])+\frac{1-x}{b\lambda }\sum _{n=b}^{\infty }n{x}^{n-b}\frac{\lambda }{\mu } \end{gathered}$$

and after simple algebraic manipulations:

$$\overline{u}=\frac{1}{b\lambda }\left(\sum _{i=1}^{b-1}i-\sum _{i=1}^{b-1}(i-1)x^i\right)+\frac{1-x}{b\lambda }\sum _{n=b}^{\infty }n{x}^{n-b}\frac{\lambda }{\mu }.$$

Applying the formulas for the sums of arithmetic and arithmetic-geometric progressions, we obtain

$$\overline{u}=\frac{b-1}{2\lambda }-\frac{x^2(1-x^{b-1})}{\lambda b{(1-x)}^2}+\frac{(b-1)x^{b+1}}{\lambda b(1-x)}+\frac{x}{b\mu (1-x)}+\frac{1}{\mu }$$

and, after the inverse substitution of $x=\frac{\lambda }{M}$:

$$\begin{gathered} \overline{u}=\frac{b-1}{2\lambda }-\frac{\frac{{\lambda }^2}{M^2}(1-\frac{{\lambda }^{b-1}}{M^{b-1}})}{\lambda b{(1-\frac{\lambda }{M})}^2}+\frac{(b-1)\frac{{\lambda }^{b+1}}{M^{b+1}}}{\lambda b(1-\frac{\lambda }{M})}+\frac{\frac{\lambda }{M}}{b\mu (1-\frac{\lambda }{M})}+\frac{1}{\mu }= \\ =\frac{b-1}{2\lambda }-\frac{\lambda (M^{b-1}-{\lambda }^{b-1})}{b{M}^{b-1}{(M-\lambda )}^2}+\frac{(b-1){\lambda }^b}{b{M}^b(M-\lambda )}+\frac{\lambda }{b\mu (M-\lambda )}+\frac{1}{\mu }. \end{gathered}$$

The sum of the second, third and fourth terms in the latter equation becomes

$$\begin{gathered} \frac{-\lambda M(M^{b-1}-{\lambda }^{b-1})+(M-\lambda )(b-1){\lambda }^b}{b{M}^b{(M-\lambda )}^2}+\frac{\lambda }{b\mu (M-\lambda )}= \\ =\frac{-\lambda M^b+{\lambda }^{b}M+Mb{\lambda }^b-M{\lambda }^b-{\lambda }^{b+1}b+{\lambda }^{b+1}}{b{M}^b{(M-\lambda )}^2}+\frac{\lambda }{b\mu (M-\lambda )}= \\ =\frac{-\lambda M^b+Mb{\lambda }^b-{\lambda }^{b+1}b+{\lambda }^{b+1}}{b{M}^b{(M-\lambda )}^2}+\frac{\lambda }{b\mu (M-\lambda )}=\frac{b{\lambda }^b(M-\lambda )-\lambda (M^b-{\lambda }^b)}{b{M}^b{(M-\lambda )}^2}+ \\ +\frac{\lambda }{b\mu (M-\lambda )}=\frac{b{\lambda }^b\mu (M-\lambda )-\lambda \mu (M^b-{\lambda }^b)+M^b(M-\lambda )\lambda }{b{M}^b{(M-\lambda )}^2\mu } \end{gathered}$$

and, taking into account the well-known formula:

$$a^n-b^n=(a-b)(a^{n-1}+a^{n-2}b+a^{n-3}{b}^2+\dots +a{b}^{n-2}+b^{n-1}),$$

and the average sojourn time can be rewritten as:

$$\overline{u}=\frac{b-1}{2\lambda }+\frac{b{\lambda }^b\mu -\lambda \mu (M^{b-1}+M^{b-2}\lambda +\dots +M{\lambda }^{b-2}+{\lambda }^{b-1})+M^b\lambda }{b{M}^b\mu (M-\lambda )}+\frac{1}{\mu }.$$

Finally, from Equation (5) it is easy to find that

$$M^{b+1}-(\lambda +\mu )M^b+{\lambda }^b\mu =(M-\lambda )(M^b-\mu M^{b-1}-\mu M^{b-2}\lambda -\dots -\mu M{\lambda }^{b-2}-\mu {\lambda }^{b-1})=0.$$

Therefore, since $M>\lambda$,

$$M^b-\mu (M^{b-1}+M^{b-2}\lambda +\dots +M{\lambda }^{b-2}+{\lambda }^{b-1})=0$$

and then:

$$\begin{gathered} \overline{u}=\frac{b-1}{2\lambda }+\frac{{\lambda }^b}{M^b(M-\lambda )}+\frac{1}{\mu }= \\ =\frac{b-1}{2\lambda }+\frac{{\lambda }^b\mu +M^{b+1}-M^b\lambda }{M^b(M-\lambda )\mu }=\frac{b-1}{2\lambda }+\frac{M^b\mu }{M^b(M-\lambda )\mu }=\frac{b-1}{2\lambda }+\frac{1}{M-\lambda }. \end{gathered}$$

Thus, we got expression (8). □

Note that for $b=1$, the root of the Equation (5) is $M=\mu$, and hence $\overline{u}=\frac{1}{\mu -\lambda }$ in accordance with the well-known result for the $M/M/1$ queue.

5. Average Response Time Optimization

The average response time is a relevant performance index that depends, as highlighted in the previous section, on the routing matrix and the node sojourn times. The latter depend on the network parameters not only explicitly, but also through the values of the roots $M_i$ of (5).

Assuming that the network structure and the incoming traffic rate are fixed parameters, in this section we propose an efficient algorithm to find the optimal batch size vector $b=(b_1,\dots ,b_L)$, for which the average response time of the queuing network N is minimized.

Our optimization algorithm is based on the following two considerations. Firstly, as shown above, the evolution of each node is independent of the rest of the network in a probabilistic sense: the arrival rate to any node of N does not change when the service discipline (namely, the batch size) changes in other nodes, and the input rates ${\lambda }_i$ depends only on the routing matrix. Therefore, the problem of finding the minimum average response time $\overline{\tau }$ of N is decomposed into solving L minimization problems for the functions ${\overline{u}}_i\left(b_i\right)$ in the nodes $S_i$, $i=1,\dots ,L$. Secondly, the function ${\overline{u}}_i\left(b_i\right)$ has only one minimum for values of $b_i$ satisfying the stability condition ${\lambda }_i<b_i{\mu }_i$.

As in the previous section, for sake of clarity, in the following we omit the subscript i, since the same considerations are valid for all the nodes. To prove that $\overline{u}\left(b\right)$ has one minimum, we treat b as a continuous parameter and calculate the second derivative of $\overline{u}\left(b\right)$ with respect to b:

$${\overline{u}}^{″}=\frac{(M-\lambda )M^{″}-2{\left(M^{\prime }\right)}^2}{{(\lambda -M)}^3}.$$

From (6) it is easy to see that M converges to $\lambda +\mu$ as $b\to \infty$. Moreover, the sequence of the roots $M_b$ is monotone increasing for any $b>\frac{\lambda }{\mu }$. To prove the latter statement, consider the function

$$f_b\left(M\right)=M^{b+1}-(\lambda +\mu )M^b+{\lambda }^b\mu ;$$

i.e., $M_b$ is the largest root of $f_b\left(M\right)=0$. Hence:

$$M_b^{b+1}=(\lambda +\mu )M_b^b-{\lambda }^b\mu$$

and

$$f_{b+1}\left(M_b\right)=M_b^{b+2}-(\lambda +\mu )M_b^{b+1}+{\lambda }^{b+1}\mu ={\lambda }^b\mu (\lambda -M_b)<0,$$

since $M_b>\lambda$. This means that for the batch size $b+1$, the largest solution of $f_{b+1}\left(M\right)=0$ must satisfy the inequality $M_{b+1}>M_b$, since $f_{b+1}\left(M\right)$ is continuous and $f_{b+1}(\lambda +\mu )>0$. Hence, the function $y=M\left(b\right)$, again considering b as a continuous variable, has an horizontal asymptote as $b\to \infty$ and is monotone increasing; as a consequence of the existence of a finite limit, its increase rate decreases for sufficiently high values of b, and its second derivative $M^{″}$ must be negative. Moreover, as shown in the Figure 1, Figure 2 and Figure 3 below for various ratios of $\lambda$ and $\mu$, the function $y=M\left(b\right)$ is concave (convex upwards) for all values of b satisfying the stability condition. Furthermore, since $M>\lambda$, ${{\overline{u}}_i}^{″}>0$ and the average sojourn time is a convex function (and has one minimum).

Hence, we propose the following optimization algorithm to determine the optimal batch size vector:

• Find the vector $\omega$ solution of equation $\omega \Theta =\omega$ under the normalization condition ${\sum }_{i=0}^L{\omega }_i=1$.
• Compute the arrival rates vector $\lambda =({\lambda }_1,\dots ,{\lambda }_L)$, as

  $${\lambda }_i=\frac{{\omega }_i}{{\omega }_0}{\lambda }_0,i=1,\dots ,L.$$
• Initialize $i=0$.
• $i=i+1$. If $i>L$, then go to step 14.
• Initialize $k=0$ that represents the iteration number.
• Find the initial value $b_i$ satisfying the condition ${\lambda }_i<b_i{\mu }_i$.
• Find the root $M_i$ of the Equation (5).
• Calculate ${\tilde{\mu }}_i^k=({\tilde{\mu }}_i^k\left(1\right),\dots ,{\tilde{\mu }}_i^k\left(b_i\right))$, where ${\tilde{\mu }}_i^k\left(n\right)=M_i$, if $n\ge b_i$ and the rates ${\tilde{\mu }}_i^k\left(n\right)$ are calculated according to (4), if $n<b_i$.
• Calculate the stationary distribution according to (7).
• Calculate the average sojourn time at the node $S_i$ according to (8).
• $k=k+1$ and increase $b_i$ by 1.
• Repeat steps 7–10.
• If ${\overline{u}}_i^k<{\overline{u}}_i^{k-1}$, then go to step 11. If ${\overline{u}}_i^k\ge {\overline{u}}_i^{k-1}$, then decrease $b_i$ by 1 and go to step 4.
• Return the vector b. The algorithm is complete.

6. Numerical Examples

To highlight the correctness of the results in the previous sections, we provide some numerical examples for a single node and a large queuing network, satisfying the assumptions under which the product-form solution was derived.

At first we focus on the sojourn time in a generic node $S_i$, $i\in \{1,\dots ,L\}$, with input rate ${\lambda }_i$ and service rate ${\mu }_i$ for batches of $b_i$ customers. Figure 4 and Figure 5 show the behavior of the sojourn time ${\overline{u}}_i\left(b_i\right)$ for ${\mu }_i=2.0$ and ${\mu }_i=3.0$, respectively. In more detail, ${\overline{u}}_i\left(b_i\right)$ is plotted for different values of ${\lambda }_i$ (namely, ${\lambda }_i=1.0,\dots ,5.0$) as a function of the "allowed" values of the batch size, i.e., the values of $b_i$ satisfying the condition ${\rho }_i<1$ for the existence of a stationary regime.

Both figures show that the function ${\overline{u}}_i\left(b_i\right)$ has one minimum for a specific value of ${\rho }_i$ depending on the system’s parameters. For instance, if ${\lambda }_i=1$, then the optimal batch size is one for both values of ${\mu }_i$, whereas for ${\lambda }_i=2$ the optimal $b_i$ is already equal to two; and for higher input rates it depends on ${\mu }_i$ (compare the graphs for ${\lambda }_i=5$ in Figure 4 and Figure 5).

Next, consider the queuing network N, consisting of $L=11$ nodes, with input rate ${\lambda }_0=2$, service rate vector $\mu =(1.1,0.8,1.1,0.3,0.4,0.2,0.4,0.3,0.2,0.4,0.1)$ and routing matrix

$$\Theta =\left(\begin{array}{c}0.00.30.30.40.00.00.00.00.00.00.00.0\\ 0.50.00.00.00.10.10.10.10.10.00.00.0\\ 0.50.00.00.00.10.10.10.10.10.00.00.0\\ 0.50.00.00.00.10.10.10.10.10.00.00.0\\ 0.70.00.00.00.00.00.00.00.00.10.10.1\\ 0.70.00.00.00.00.00.00.00.00.10.10.1\\ 0.70.00.00.00.00.00.00.00.00.10.10.1\\ 0.70.00.00.00.00.00.00.00.00.10.10.1\\ 0.70.00.00.00.00.00.00.00.00.10.10.1\\ 0.70.00.10.10.00.00.00.10.00.00.00.0\\ 0.80.10.10.00.00.00.00.00.00.00.00.0\\ 0.70.10.00.10.10.00.00.00.00.00.00.0\end{array}\right).$$

From the source $S_0$, the nodes $S_1$, $S_2$ and $S_3$ may be reached. After being served in these systems, the customers may return to the source $S_0$ (i.e., leave the network) with a probability of 0.5, and with small probabilities equal to 0.1, may move to nodes $S_4$–$S_8$. After service is completed in $S_4$–$S_8$, the customers may go to $S_0$ with a probability of 0.7 or to one of the nodes $S_9$–$S_{11}$ with probabilities equal to 0.1. Finally, customers from $S_9$–$S_{11}$ leave the network with sufficiently high probabilities or move to other nodes with the same probability of 0.1.

Links and route probabilities were chosen in such a way that the developed analysis method can be applied. Indeed, small and comparable (in our case, equal) transition probabilities permit one to approximate the input flow, coming to any queue from a large number of nodes, close to a Poisson flow. Moreover, backwards paths (from $S_9$–$S_{11}$) have been introduced to verify the applicability of the proposed methodology also to queuing networks with a non-tree structure.

Using the optimization algorithm proposed in this paper, the optimal batch size vector is $b=(2,2,2,2,1,2,2,2,1,1,3)$. This choice leads to an average response time equal to 7.61, and a mean number of customers in the network equal to 15.22 (in good agreement with the simulation estimates of 7.64 and 15.28, respectively). Here and in the following, all the parameters, derived by discrete-event simulation, were calculated in stationary conditions with a confidence interval of 0.01 and a confidence level higher than 0.95.

For the sake of completeness,

Table 1. Average numbers of customers in the nodes.

Node	${\mathit{S}}_1$	${\mathit{S}}_2$	${\mathit{S}}_3$	${\mathit{S}}_4$	${\mathit{S}}_5$	${\mathit{S}}_6$	${\mathit{S}}_7$	${\mathit{S}}_8$	${\mathit{S}}_9$	${\mathit{S}}_{10}$	${\mathit{S}}_{11}$
Approximation	1.174	1.554	1.493	1.447	1.065	2.118	1.139	1.381	1.111	0.357	2.289
Simulation	1.175	1.553	1.494	1.451	1.076	2.221	1.145	1.388	1.131	0.365	2.283

and

Table 2. Average sojourn times in the nodes.

Node	${\mathit{S}}_1$	${\mathit{S}}_2$	${\mathit{S}}_3$	${\mathit{S}}_4$	${\mathit{S}}_5$	${\mathit{S}}_6$	${\mathit{S}}_7$	${\mathit{S}}_8$	${\mathit{S}}_9$	${\mathit{S}}_{10}$	${\mathit{S}}_{11}$
Approximation	1.890	2.501	1.818	6.672	5.163	10.750	5.254	6.694	10.556	3.393	21.647
Simulation	1.890	2.501	1.820	6.691	5.219	10.769	5.280	6.720	10.735	3.468	21.684

, respectively, show the average number of customers and the average sojourn time in each node of the network. Note that all the analytical values and the corresponding estimates by discrete-event simulation are very close, confirming the applicability of the analytical approach proposed in this paper not only for the whole queue, but also at the node level. In more detail, the largest difference for the average number of customers, observed at node $S_6$, is less than $5\%$, whereas for the average sojourn time the maximum was attained at nodes $S_9$ and $S_{10}$, and it was equal to $2\%$.

Moreover, Table 2 highlights that the average sojourn times in nodes $S_6$, $S_9$ and $S_{11}$ significantly exceed the values in the other nodes of the network. Since in our example the transition probabilities are comparable, this is due to the service rates, which in these nodes are lower than in the rest of the network.

7. Conclusions

In this paper we proposed an efficient method for the analysis of large-scale open queuing networks with batch services and an original algorithm to determine the optimal vector of the service batch sizes, which minimizes the average response time of the network. The proposed algorithm can be used to determine the optimal capacity of vehicles for various purposes, such as the transportation of passengers or goods. The algorithm is also applicable for optimizing systems for the accumulation and subsequent processing of customers requests, which may include information in electronic or paper forms, financial resources, materials, parts and production waste. In more detail, due to the conditions under which the product-form was obtained, the proposed algorithm is applicable to large-scale networks with individual routing of the customers, assuming that the number of possible destinations is significantly larger than the batch size.

Finally, it is worth mentioning that the memory requirement of the algorithm is $\mathcal{O}\left(1\right)$, and its computational complexity is $\mathcal{O}\left(n\right)$, where n is the number of the values of the $\overline{u}$ function that must be calculated (in general, n depends on the queuing network parameters).