# A New Method for Markovian Adaptation of the Non-Markovian Queueing System Using the Hidden Markov Model

^{1}

^{2}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Analysis of the Current Solution Queueing Systems M/E_{r}/1/∞ and E_{k}/E_{r}/1/∞

_{k}/r queueing system from 1968/69 [18,19] was based on the assumption that customers arrive at Poisson’s arrival process and are serviced by the Erlang distribution Er(k, µ). A well-known convolution was used to reduce the Markov queueing system (1). The analytical solution assumes the following flow diagram (Figure 1).

_{n}= x

^{n}, requires the solution of the polynomial (2).

_{k}/E

_{r}/1 had a similar problem which was also successfully mathematically solved in a similar way [21,22,23,24].

_{k}/r/∞ systems leads us to a kind of paradox. Their relationship is as follows (3):

_{k}/r is “mechanically” configured to serve a group of “k” clients. If each client is served by the exponential distribution, a group of “k” clients will be served by the Erlang distribution. In practice, we have found this system in stochastic inventory management. At the same time intervals 1/λ, stocks are supplemented with the same quantity of “k” integer “customers” (Figure 2). The current solution of the M/E

_{k}/r is not entirely adequate for the following reasons:

- Customers do not arrive at Poisson’s arrival process—a group of “k” customers arrive at Poisson’s arrival process. For this reason, the ratio of the number of clients in the queueing systems is (3);
- Customers are not served by Erlang’s distribution—customers are served by the exponential distribution. A group of “k” customers are served by Erlang’s distribution.

_{k}/r, the state of the queue is redefined as the number of service stages remaining in the system [23]. Solutions of the first half of the century are still dominant [25], although these do not allow for successive transitions through the states of the system, as in the system M/M/1/∞. Simply, the queueing system M/E

_{k}/r from Figure 1 does not have mutual transitions from neighboring states.

_{k}/1/∞ system must have a smaller mean number of customers in the queue. The reason for this relationship is found in the projected flow diagram (Figure 1). Exponential phases of customer services through which the Erlang distribution is convoluted are partially realized, depending on the quantity λ and μ. For the interval 0 < ρ < 1, for small values, ρ → 0 solutions are satisfactory, but when ρ → 1, there are large deviations. When dynamically observed, the higher arrival rate λ, the earlier the convolution of the exponential customer services is interrupted, and we obtain a service of the Erlang order, which is an average of less than k. Therefore, a valid solution is only approximate for favorable relationships λ and μ.

## 3. Analytical Solution of the Queueing System M(λ)/Ek(μ)/1/∞

#### 3.1. Elementary Case of Poison’s Birth and Erlang’s Death

_{0}and X

_{1}. Let the time of birth be exponentially distributed E(λ) in this system, and the dying time be distributed according to Erlang’s distribution Er(k, μ

_{k}).

_{k−1}, h

_{k−2}, …, h

_{2}, h

_{1}(k ∈ N) are abstract states and have a catalitic role. In order to achieve the convolution of “k” exponential distributions, the system needs to introduce “k − 1” abstract hidden Markov states in the process of customer services. The indexes of abstract states h

_{i}are descendent. The symbolic equivalence of the birth–death process can be reduced to convincing by Markov with abstract, hidden states (Figure 3).

_{0}) = p

_{0}and P(X

_{1}) = p

_{1}, as well as the probability of the state of abstract HMS h

_{k−1}, h

_{k−2}, …, h

_{2}, h

_{1}, respectively P(h

_{i}) = p

_{h}

_{(i)}. We can calculate them with ease based on the existing analytical rules of the stationary regime of the birth–death process.

_{h}

_{(i)}classified? The existing system (Figure 2) has a “k + 1” state, and we only need two basic states: P

_{0}and P

_{1}.

_{1}. Analogously, the intensity of dying μ

_{k}from the beginning of the convolution translates the system into X

_{0}state, and the probability of abstract HMS is thus associated with the preceding state according to the following rule (4):

_{k}) and dying along the exponential E(μ), abstract HMS are formed for the purpose of convolution, but their probability will be added to the state P

_{1}.

#### 3.2. The Probability of State of the Queueing System M(λ)/E_{k}(μ)/∞

_{0}, X

_{1}, …, X

_{n}, with the exponentially distributed inter-arrival-time E(λ) and the service time distributed according to Erlang’s distribution Er(k, μ

_{k}), k ∈ N, is shown in Figure 5.

_{0}, X

_{1}, …, X

_{n}, in the stationary mode of operation, are denoted by the large letter P

_{i}(t) = P

_{i}= const, i ∈ [1, n]. The probabilities of the abstract states h

_{i,j}are denoted by the small letter p

_{i}

_{,j}(t) = p

_{i}

_{,j}= const, i ∈ [1, k−1], j ∈ [1, n]. For the X

_{0}state, the balance (differential) equation of the stady-state (stationary operating mode) is (5)

_{k}

_{−1,1}, h

_{k}

_{−2,1}, …, h

_{2,1}, h

_{1,1}which are between the states X

_{0}and X

_{1}, the system of balance equations of the stady-state is (6)

_{1}, the balance equation of the stady-state is (7)

_{k}

_{−1,2}, h

_{k}

_{−2,2}, …, h

_{2,2}, h

_{1,2}that are located between the states X

_{1}and X

_{2}, the system of balance equations of the stady-state is (8)

_{2}, the balance equation of the stady-state is (9)

_{0}can be obtained from the normative Equation (11):

_{k}gives the equation for the P

_{0}(14) calculation.

_{0}is equal (15) to

_{k}

_{−1,j}, h

_{k}

_{−2,j}, …, h

_{2,j}, h

_{1,j}(j ∈ N) are abstract probabilities. The analytical process takes them to the catalytic at each transition from the state X

_{j}to the state X

_{j−}

_{1}, for j ∈ [1, ∞). Therefore, the probabilities of p

_{i}

_{,j}states h

_{i}

_{,j}are abstractly transmitted only for the realization of the service by Erlang’s distribution. Now, the new code of the sum of probabilities of real and abstract states S

_{i}(Figure 6) can be marked:

_{k}). We temporarily observe the label “k, μ” with the note that the service rate is distributed by Erlang’s “k” order with the intensity “μ

_{k}” (Figure 7):

_{i}) = s

_{i}can be given by (16)

#### 3.3. Comparison of the System M(λ)/M(μ_{e})/1/∞ and M(λ)/E_{k}(μ_{k})/1/∞

_{k}(μ

_{k})/1 have an identical arrival rate and equal mean service time. Then, this relationship can be expanded with “λ” (17), as follows:

_{k}, it follows that (18)

_{k}(μ

_{k})/1/∞ (19) can be obtained:

_{k}(μ

_{k})/1/∞.

_{e})/1/∞ with smoothness, we proceed to the formula of the system M(λ)/E

_{k}(μ

_{k})/1/∞. Now, the mean number of customers in the system M(λ)/E

_{k}(μ

_{k})/1/∞ and in the queue equals (21)

_{k}(μ

_{k})/1/∞ is always lower than the mean number of customers in the system M(λ)/M(μ

_{e})/1/∞. The same applies to the mean number of customers in the queue (22):

_{k}(μ

_{k})/1/∞ (3), the solution with the abstract HMS (21) completely meets the logic of the Pollaczek–Khinchine formula.

_{k}(μ

_{k})/1/∞, we can verify the validity of the patterns (21) and the relation (22). Mean service time and variance are distributed according to Erlang’s distribution with known parameters of mathematical expectation and standard deviation k/μ

_{k}and k/μ

_{k}

^{2}, respectively. The ratio ρ is “k” times greater than ρ

_{k}(18). If in the Pollaczek–Khinchine formula we reduce the intensity and the variance, we obtain (23), which is identical to the form (22) obtained:

_{e})/1/∞ system. The amount of time $\phi $ is the same (24), as follows:

## 4. Queueing System M(λ)/N(ω, σ)/1/∞

#### 4.1. An Approximate Solution

_{k}(k, μ

_{k}) and a normal distribution N(ω, σ) with a probability density function (25),

_{k}, ω, and σ, from the expression (26), we can obtain a quadratic equation. The solution of this square equation by “k ≥ 1” is equal to (27) the following:

_{2}. In the case of distributing the service time by the normal distribution N(ω, σ), which in the application of the queueing system does not have negative values f(t) ≥ 0, t ∈ (0, ∞), due to the known rule (ω − 3σ) > 0 (with probability p = 0.9996), or the minimum value of the parameter “k”, we can expect 9. Thanks to the relation (26), the non-Markov queueing system M(λ)/N(ω, σ)/1/∞ can be reduced to the Markov queueing system with Erlang’s distribution of service times M(λ)/E(ω

^{2}/σ

^{2},ω/σ

^{2})/1/∞.

^{2}/σ

^{2}) ∈ R

^{+}, i.e., a parameter of the Erlang distribution “k” will often not be the positive integer, k $\notin $ N. For practical use, the substitution (28) is approximative:

#### 4.2. Calibration of the Solution for a Normal Service Time

^{+}of the Erlang distribution curve obtained on the basis of (28), we can determine the value of Δ using the integer finite. We recall that in the case of the normal distribution of the service time N(ω, σ), the value of the parameter k ≥ 9 (29):

^{+}can be represented by a fraction formed by two selected natural numbers (a, b) ∈ N, which are greater than zero. Let k′ = int(k) and k″ = int(k) + 1. It follows that (30)

- The first Erlang system of order k′, i.e., queueing system M(λ)/E(k′, ω
^{2}/σ^{2})/1/∞. Let us denote its probability as s_{i}′; - The second Erlang system of order k″, i.e., queueing system M(λ)/E(k″, ω
^{2}/σ^{2})/1/∞. Let us identify its probabilities with s_{i}″.

_{i}denotes the probability of the state of the system M(λ)/N(ω, σ)/1/∞, after calculating the numbers (a, b) ∈ N, the probabilities of the state n

_{i}(31) can be calculated as follows:

^{+}for the purpose of the compulsory adjustment of μ

_{n}= ω/σ according to formulas (26) and (27).

## 5. Conclusions

_{k}(μ

_{k})/1/∞ and M(λ)/N(ω, σ)/1/∞ were intended to demonstrate the effectiveness of the new proposed HMS-based method. The correctness of the new method is confirmed by the Pollaczek–Khinchine formula.

_{k}(μ

_{k})/n/m and M(λ)/N(ω, σ)/n/m. The extension of the HMS application principle provides solutions with queueing systems E

_{i}/E

_{j}/m/n (i ≠ j) i, j ∈ N, and by applying calibration, all queueing systems with a normally distributed time are analytically available.

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Pollaczek, F. Uebereine Aufgabe der Wahrscheinlichkeits theory. Math. Z.
**1930**, 32, 64–100. [Google Scholar] [CrossRef] - Kingman, J.F.C. The single server queue in heavy traffic. Math. Proc. Camb. Philos. Soc.
**1961**, 57, 902–904. [Google Scholar] [CrossRef] - Kolmogorov, A. Sur le problèmed’attente. MatematicheskiiSbornik
**1931**, 38, 101–106. (In French) [Google Scholar] - Koenigsberg, E. Is queueing theory dead? Omega
**1991**, 19, 69–78. [Google Scholar] [CrossRef] - Schwarz, J.A.; Selinka, G.; Stolletz, R. Performance analysis of time-dependent queueing systems: Survey and classification. Omega
**2016**, 63, 170–189. [Google Scholar] [CrossRef] [Green Version] - Baum, L.; Petrie, T. Statistical Inference for Probabilistic Functions of Finite State Markov Chains. Ann. Math. Stat.
**1966**, 37, 1554–1563. [Google Scholar] [CrossRef] - Li, Y. Hidden Markov models with states depending on observations. Pattern Recognit. Lett.
**2005**, 26, 977–984. [Google Scholar] [CrossRef] - Chen, M.Y.; Kundu, A.; Zhou, J. Off-line handwritten word recognition using a hidden Markov model type stochastic network. IEEE Trans. Pattern Recognit. Mach. Intell.
**1994**, 16, 481–496. [Google Scholar] [CrossRef] - Uğuz, H.; Kodaz, H. Classification of internal carotid artery Doppler signals using fuzzy discrete hidden Markov model. Expert Syst. Appl.
**2011**, 38, 7407–7414. [Google Scholar] [CrossRef] - Gu, F.; Zhang, H.; Zhu, D. Blind separation of non-stationary sources using continuous density hidden Markov models. Digit. Signal Process.
**2013**, 23, 1549–1564. [Google Scholar] [CrossRef] - Kobayashi, K.; Kaito, K.; Lethanh, N. A statistical deterioration forecasting method using hidden Markov model for infrastructure management. Transp. Res. Part B Methodol.
**2012**, 46, 544–561. [Google Scholar] [CrossRef] [Green Version] - Rongrong, F.; Hong, W.; Wenbo, Z. Dynamic driver fatigue detection using hidden Markov model in real driving condition. Expert Syst. Appl.
**2016**, 63, 397–411. [Google Scholar] - Holzmann, H.; Schwaiger, F. Testing for the number of states in hidden Markov models. Comput. Stat. Data Anal.
**2016**, 100, 318–330. [Google Scholar] [CrossRef] - Fadiloglu, M.M.; Yeralanm, S. Models of production lines as quasi-birth-death processes. Math. Comput. Model.
**2002**, 35, 913–930. [Google Scholar] [CrossRef] [Green Version] - Ching, W.K.; Huang, X.; Ng, M.K.; Siu, T.K. Hidden Markov Chains. In Markov Chains. International Series in Operations Research & Management Science; Mamon, R.S., Elliott, R.J., Eds.; Springer: Boston, MA, USA, 2013; Volume 189, pp. 201–230. [Google Scholar]
- Tanackov, I.; Stojić, G.; Tepić, J.; Kostelac, M.; Sinani, F.; Sremac, S. Golden Ratio (Sectiona Aurea) in Markovian Ants AI Hybrid. Adaptive and Intelligent Systems, ICAIS 2011, Lecture Notes in Computer Science; Springer: Berlin, Germany, 2011; Volume 6943, pp. 356–367. [Google Scholar]
- Whitt, W. A broad view of queueing theory through one issue. Queueing Syst.
**2018**, 89, 3–14. [Google Scholar] [CrossRef] - Heffer, J.C. Steady-state solution of the M/Ek/c (0, FIFO) queueing system. INFOR J. Can. Oper. Res. Soc.
**1969**, 17, 16–30. [Google Scholar] - Mayhugh, J.O.; Mc Cormick, R.E. Steady state solution of the queue M/Ek/r. Manag. Sci.
**1968**, 14, 692–712. [Google Scholar] [CrossRef] - Poyntz, C.D.; Jackson, R.R.P. The steady-state solution for the queueing process Ek/Em/r. Oper. Res. Q.
**1973**, 24, 615–625. [Google Scholar] [CrossRef] - Adan, I.; Zhao, Y. Analyzing GI/Er/1 queues. Oper. Res. Lett.
**1996**, 19, 183–190. [Google Scholar] [CrossRef] - Adan, I.J.B.F.; Van de Waarsenburg, W.A.; Wessels, J. Analyzing Ek/E/c queues. Eur. J. Oper. Res.
**1996**, 92, 112–124. [Google Scholar] [CrossRef] - Martínez, J.M.V.; Vallejos, R.A.C.; Barría, M.M. On the limiting probabilities of the M/Er/1 queueing system. Stat. Probab. Lett.
**2014**, 88, 56–61. [Google Scholar] [CrossRef] - Wang, K.H.; Kuo, M.Y. Profit analysis of the M/Er/1 machine repair problem with a non-reliable service station. Comput. Ind. Eng.
**1997**, 32, 587–594. [Google Scholar] [CrossRef] - Adan, I.; Resing, J. Queueing Systems; Einhoven University of Technology: Eindhoven, The Netherlands, 2015. [Google Scholar]

**Figure 3.**Process of Markovization of the elementary birth–death process based on the abstract hidden Markov states (HMS) and the convolution of the exponential distribution.

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Tanackov, I.; Prentkovskis, O.; Jevtić, Ž.; Stojić, G.; Ercegovac, P.
A New Method for Markovian Adaptation of the Non-Markovian Queueing System Using the Hidden Markov Model. *Algorithms* **2019**, *12*, 133.
https://doi.org/10.3390/a12070133

**AMA Style**

Tanackov I, Prentkovskis O, Jevtić Ž, Stojić G, Ercegovac P.
A New Method for Markovian Adaptation of the Non-Markovian Queueing System Using the Hidden Markov Model. *Algorithms*. 2019; 12(7):133.
https://doi.org/10.3390/a12070133

**Chicago/Turabian Style**

Tanackov, Ilija, Olegas Prentkovskis, Žarko Jevtić, Gordan Stojić, and Pamela Ercegovac.
2019. "A New Method for Markovian Adaptation of the Non-Markovian Queueing System Using the Hidden Markov Model" *Algorithms* 12, no. 7: 133.
https://doi.org/10.3390/a12070133