An Approach to Obtain Upper Ergodicity Bounds for Some QBDs with Countable State Space

Abstract

Usually, when the computation of limiting distributions of (in)homogeneous (in)finite continuous-time Markov chains (CTMC) has to be performed numerically, the algorithm has to be told when to stop the computation. Such an instruction can be constructed based on available ergodicity bounds. One of the analytical methods to obtain ergodicity bounds for CTMCs is the logarithmic norm method. It can be applied to any CTMC; however, since the method requires a guessing step (search for proper Lyapunov functions), which may not be successful, the obtained bounds are not always meaningful. Moreover, the guessing step in the method cannot be eliminated or automated and has to be performed in each new use-case, i.e., for each new structure of the infinitesimal matrix. However, the simplicity of the method makes attempts to expand its scope tempting. In this paper, such an attempt is made. We present a new technique that allows one to apply, in one unified way, the logarithmic norm method to QBDs with countable state spaces. The technique involves the preprocessing of the infinitesimal matrix of the QBD, finding bounding for its blocks, and then merging them into the single explicit upper bound. The applicability of the technique is demonstrated through a series of examples.

1. Introduction

When one thinks of analytical modeling tools in such areas as operations research [1,2], chemistry [3], biology, and epidemiology [4,5], among the first that come to mind are the classic birth-and-death processes (BDP), or the more versatile quasi-birth-and-death processes (QBD) [6,7,8]. A detailed bibliography (up to 1982) and a more recent one (up to 2004), which contain a large body of papers on particular applications of BDPs, can be found in [9,10]. Despite numerous available results for BDPs, they remain a topic of active research, with rich and practically useful results (see, for example, [11]).

Among all of the types of BDPs/QBDs, in this paper, attention is devoted only to two-dimensional (i.e., certain types of QBDs) possibly inhomogeneous continuous-time Markov chains (CTMC) with (countable) discrete state space. We recall that the inhomogeneity property implies that (some or all) transition intensities are non-random functions of time and (may or may not) depend on the state of the chain. These processes have been studied from various point of views: existence, numerical algorithms (see also the recent review in [12]), asymptotics, approximations, etc. One of the attempts to provide a systematic classification of the available approaches (up to 2016 within the framework of queueing theory) can be found in [13].

In what follows, one specific question is addressed concerning the considered class of CTMCs, that is, the computation of the long-run, (limiting) time-dependent performance characteristics. This question can be considered from various point of views (computation time, accuracy, complexity, storage use, etc.). Therefore, various solution techniques are already available in the literature. One of the ways to improve the efficiency of a solution technique is to supply it with a method for the limiting regime detection or, in other words, a method providing ergodicity bounds. Indeed, once the limiting regime is reached, there is no need to continue the computation indefinitely. Therefore, the main contribution of this paper is finding new ergodicity bounds for certain well-known types of QBDs.

The results presented further continue the work whose first results were presented in [14], which considered the well-known problem of finding upper bounds on the rate of convergence (in the $l_1$ norm) to the limiting regime of possibly inhomogeneous Markov chains with discrete state space and continuous time. One of the convenient methods for obtaining such bounds is the logarithmic norm method. However, when the logarithmic norm is positive (and for complicated models, this is often the case), the method yields meaningless bounds. One of the techniques that eliminates this critical drawback of the method, i.e., expands the scope of its application, is the use of weighted norms. However, by switching to the weighted norms, another problem comes to replace the original one: the problem of selecting the correct weights. Its solution in the general case is not known. Therefore, for each specific Markov chain, the proper weights have to be guessed, often by trial and error, and therefore this venture is not always successful. When one analyzes the available literature on this issue, it can be seen that for one type of infinitesimal matrices (generators), it is easier to guess the weights (if it is possible at all) than for others. Suggesting a classification is perhaps a matter for future research. However, as shown in [14], the suggested observation is sufficient to develop a new technique that expands the scope of the logarithmic norm method. This is achieved by introducing the new step—preprocessing of the infinitesimal matrix. Later on, we will see that this expansion of the scope does not come for free: the resulting bounds may turn out to be very rough or (again) meaningless.

The preprocessing proposed in [14] implied that, at first, one searches in the intensity matrix for blocks that are easier to analyze individually than the original matrix. Then, having obtained the bounds for the blocks, it remains to combine them into the single one estimate for the original Markov chain. In [1], a corresponding method for combining estimates is proposed when the number of chain states is finite.In this paper, it will be shown that the idea of [14] can be extended to countable Markov chains. Along the way, one has to overcome a number of analytical difficulties and introduce a number of conditions on the initial characteristics of the Markov chain. In this paper, we did not care about making these conditions as general and easy to check as possible. This is the subject of further research. Here, the emphasis is on demonstrating the efficiency of the method proposed in [14] for countable Markov chains.

It is important to note that the results obtained in this article, together with the results of [14], form one general approach for applying the simple logarithmic norm method to study the convergence of QBDs, which was previously possible to do only in very special cases.

2. Preliminaries

Let $\left\{X\right(t),t\ge 0\}$ be a (possibly inhomogeneous) CTMC with the state space $\mathcal{X}=\{-K_1,\dots ,-1,0,1,\dots ,K_2\}$, which is allowed to be finite or countable. In the latter case, one assumes that either $K_1=0$ and $K_2=\infty$ or $K_1=-\infty$ and $K_2=\infty$. The transition probabilities will be denoted by $p_{ij}(s,t)$, i.e., $p_{ij}(s,t)=Pr\left\{X\left(t\right)=j\left|X\left(s\right)=i\right.\right\}$, $i,j\ge 0,0\le s\le t$. Let us also denote the state probability by $p_i\left(t\right)=Pr\left\{X\left(t\right)=i\right\}$, and the vector of state probabilities by $\mathbf{p}\left(t\right)={\left(\dots ,p_{-1}\left(t\right),p_0\left(t\right),p_1\left(t\right),\dots \right)}^T$. In what follows, it is assumed that

$$Pr\left\{X\left(t+h\right)=j|X\left(t\right)=i\right\}=\left\{\begin{array}{cc}{q}_{ij}\left(t\right)h+{\alpha }_{ij}\left(t,h\right),\hfill & ifj\ne i\hfill \\ 1-{\displaystyle \sum _{k\ne i}{q}_{ik}\left(t\right)h+{\alpha }_i\left(t,h\right)},\hfill & ifj=i,\hfill \end{array}\right.$$

(1)

where all ${\alpha }_i(t,h)$ are $o\left(h\right)$ uniformly in i, that is, ${sup}_i\left|{\alpha }_i(t,h)\right|=o\left(h\right)$. Moreover, if $X\left(t\right)$ is inhomogeneous, then all its infinitesimal characteristics (intensity functions) $q_{ij}\left(t\right)$ are integrable in t on any interval $[a,b]$, $0\le a\le b$.

Denote $a_{ij}\left(t\right)=q_{ji}\left(t\right)$ for $j\ne i$ and $a_{ii}\left(t\right)=-{\sum }_{j\ne i}{a}_{ji}\left(t\right)=-{\sum }_{j\ne i}{q}_{ij}\left(t\right)$. Further, in order to be able to obtain tighter estimates, it is assumed that

$$|a_{ii}\left(t\right)|\le L_1<\infty$$

for almost all $t\ge 0$. The state probabilities then satisfy the forward Kolmogorov system

$${\displaystyle \frac{d}{dt}}\mathbf{p}\left(t\right)=A\left(t\right)\mathbf{p}\left(t\right),t\ge 0,$$

(2)

where the initial distribution $\mathbf{p}\left(0\right)$ is given, $A\left(t\right)=Q^T\left(t\right)$, and $Q\left(t\right)$ is the infinitesimal matrix of the CTMC.

Let $\parallel ·\parallel$ be the usual $l_1$-norm, i.e., for a vector $\mathbf{x}=\left(x_i\right)$ and a matrix $B=\left(b_{ij}\right)$, it holds that $\parallel \mathbf{x}\parallel ={\sum }_i\left|x_i\right|$ and $\parallel B\parallel ={sup}_j{\sum }_i\left|b_{ij}\right|$. Denote $\Omega =\left\{\mathbf{x}:\mathbf{x}\in l_1^{+}\&\parallel \mathbf{x}\parallel =1\right\}$.

Therefore, $\parallel A\left(t\right)\parallel =2{sup}_k\left|a_{kk}\left(t\right)\right|\le 2L_1$ for almost all $t\ge 0$, and we can apply the results of [15] to Equation (2) in the space $l_1$. Namely, in [15], it was shown that the Cauchy problem for (2) has a unique solution for an arbitrary initial condition, i.e., if the initial condition is $\mathbf{p}\left(0\right)$, then $\mathbf{p}\left(t\right)=U(t,0)\mathbf{p}\left(0\right)$, $t>0$, where $U(t,0)=\left(p_{ij}(0,t)\right)$ is the Cauchy operator for (2). Moreover, if $\mathbf{p}\left(s\right)\in \Omega$, then $\mathbf{p}\left(t\right)\in \Omega$, for any $0\le s\le t$ and any initial condition $\mathbf{p}\left(s\right)$.

In what follows, we will make use of the fact that subtraction of a given function from any row of the matrix $A\left(t\right)$ does not change the CTMC’s convergence rate (see, for example, [16,17,18]). Indeed, let $\overrightarrow{1}$ denote the vector of ones, and ⊗ the Kronecker product. Then, for the vector $\overrightarrow{r}\left(t\right)={(\cdots ,r_{-1}\left(t\right),r_0\left(t\right),r_1\left(t\right),\cdots )}^T$ of arbitrary functions $r_i\left(t\right)$, the system (2) can be rewritten in the form

$${\displaystyle \frac{d}{dt}}\mathbf{p}\left(t\right)=\left(A\left(t\right)-{\overrightarrow{1}}^T\otimes \overrightarrow{r}\left(t\right)\right)\mathbf{p}\left(t\right)+\left({\overrightarrow{1}}^T\otimes \overrightarrow{r}\left(t\right)\right)\mathbf{p}\left(t\right).$$

Since ${\overrightarrow{1}}^T\mathbf{p}\left(t\right)=1$ for any $t\ge 0$, one has that $\left({\overrightarrow{1}}^T\otimes \overrightarrow{r}\left(t\right)\right)\mathbf{p}\left(t\right)=\overrightarrow{r}\left(t\right)$. Therefore, for any two solutions ${\mathbf{p}}^{\ast }\left(t\right)$ and ${\mathbf{p}}^{\ast \ast }\left(t\right)$ of the system (2), one has that

$${\displaystyle \frac{d}{dt}}{\mathbf{p}}^{\ast }\left(t\right)=(A\left(t\right)-{\overrightarrow{1}}^T\otimes \overrightarrow{r}\left(t\right)){\mathbf{p}}^{\ast }\left(t\right)+\overrightarrow{r}\left(t\right),$$

$${\displaystyle \frac{d}{dt}}{\mathbf{p}}^{\ast \ast }\left(t\right)=(A\left(t\right)-{\overrightarrow{1}}^T\otimes \overrightarrow{r}\left(t\right)){\mathbf{p}}^{\ast \ast }\left(t\right)+\overrightarrow{r}\left(t\right).$$

By subtracting, for example, the second equation from the first, for any constant diagonal matrix, say D, one obtains the new system of ODE

$${\displaystyle \frac{d}{dt}}\mathbf{x}\left(t\right)=\tilde{A}\left(t\right)\mathbf{x}\left(t\right),t\ge 0,$$

(3)

where $\mathbf{x}\left(t\right)=D({\mathbf{p}}^{\ast }\left(t\right)-{\mathbf{p}}^{\ast \ast }\left(t\right)$) and $\tilde{A}\left(t\right)=D(A\left(t\right)-\overrightarrow{1}\otimes \overrightarrow{r}\left(t\right))D^{-1}$. In order for the Cauchy problem (3) with the initial condition $\mathbf{x}\left(0\right)=D({\mathbf{p}}^{\ast }\left(0\right)-{\mathbf{p}}^{\ast \ast }\left(0\right))$ to ensure a unique solution, one assumes that D is chosen such that $\parallel \tilde{A}\left(t\right)\parallel$ is uniformly bounded in t, i.e., there exists $L_2>0$ such that $\parallel \tilde{A}\left(t\right)\parallel \le L_2<\infty$ for $t\ge 0$. Clearly, if D is the identity matrix, $L_2=L_1$.

3. General Upper Bounds

Let us assume that one can represent the matrix $\tilde{A}\left(t\right)$ in block three-diagonal form:

$$\tilde{A}\left(t\right)=\left(\begin{array}{ccccccc}\hfill \ddots & ⋮& ⋮\hfill & ⋮& ⋮& \dots & \\ \hfill \dots & {\mathfrak{B}}_{-1}& {\mathfrak{A}}_0\hfill & 0& 0& \dots \\ \hfill \dots & {\mathfrak{R}}_{-1}& {\mathfrak{B}}_0\hfill & {\mathfrak{A}}_1& 0& \dots \\ \hfill \dots & 0& {\mathfrak{R}}_0\hfill & {\mathfrak{B}}_1& {\mathfrak{A}}_2& \dots \\ \hfill \dots & 0& 0\hfill & {\mathfrak{R}}_1& {\mathfrak{B}}_2& \dots \\ \hfill \dots & ⋮& ⋮\hfill & ⋮& ⋮& \ddots & \end{array}\right)$$

(4)

with the diagonal blocks being square matrices of sizes $m_i\ge 1$, $-K_1\le i\le K_2$. If one introduces the column-vectors

$$\begin{array}{ccc}\hfill \mathbf{x}\left(t\right)& =& {(\cdots ,{\mathbf{x}}_{-1}\left(t\right),{\mathbf{x}}_0\left(t\right),{\mathbf{x}}_1\left(t\right),\cdots )}^T,\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill {\mathbf{x}}_i\left(t\right)& =& {(x_1^i\left(t\right),\dots ,x_{m_i}^i\left(t\right))}^T,\hfill \end{array}$$

(6)

then the system (3) can be split into the coupled systems of equations

$$\begin{array}{cc}\hfill {\displaystyle \frac{d}{dt}}{\mathbf{x}}_i\left(t\right)& ={\mathfrak{R}}_{i-1}\left(t\right){\mathbf{x}}_{i-1}\left(t\right)+{\mathfrak{B}}_i\left(t\right){\mathbf{x}}_i\left(t\right)+{\mathfrak{A}}_{i+1}\left(t\right){\mathbf{x}}_{i+1}\left(t\right).\hfill \end{array}$$

(7)

where, for the sake of brevity, the convention ${\mathbf{x}}_{-K_1-1}\left(t\right)={\mathbf{x}}_{K_2+1}\left(t\right)=\overrightarrow{0}$ is used. The solutions of these systems can be written in the form

$$\begin{array}{cc}{\mathbf{x}}_i\left(t\right)=U_i(t,0){\mathbf{x}}_i\left(0\right)+{\int }_0^t{U}_i(t,s){\mathfrak{R}}_{i-1}\left(s\right){\mathbf{x}}_{i-1}\left(s\right)ds+& \\ & +{\int }_0^t{U}_i(t,s){\mathfrak{A}}_{i+1}\left(s\right){\mathbf{x}}_{i+1}\left(s\right)ds,\end{array}$$

(8)

where $U_i(t,0)$ is the Cauchy operator for the ODE system ${\displaystyle \frac{d}{dt}}{\mathbf{x}}_i\left(t\right)={\mathfrak{B}}_i\left(t\right){\mathbf{x}}_i\left(t\right)$ with the initial condition ${\mathbf{x}}_i\left(0\right)$. For each $-K_1\le i\le K_2$, denote by $\gamma \left({\mathfrak{B}}_i\left(t\right)\right)={\gamma }_i\left(t\right)$ the logarithmic norm of the operator ${\mathfrak{B}}_i\left(t\right)$ and assume that one can find such a function, say $\gamma \left(t\right)$, that bounds from below all of the norms for all t, i.e., $\gamma \left(t\right)\le {inf}_i{\gamma }_i\left(t\right)$. Then, since $\parallel U_i(t,0)\parallel \le exp\left(-{\int }_0^t{\gamma }_i\left(s\right)ds\right)$, we have that $\parallel U_i(t,0)\parallel \le exp\left(-{\int }_0^t\gamma \left(s\right)ds\right)$, i.e., all Cauchy operators become bounded by the same function.

Assume that the blocks ${\mathfrak{R}}_i\left(t\right)$ and ${\mathfrak{A}}_i\left(t\right)$ of the matrix $\tilde{A}\left(t\right)$ are such that the constants $L={sup}_i{sup}_t\parallel {\mathfrak{R}}_i\left(t\right)\parallel$ and $K={sup}_i{sup}_t\parallel {\mathfrak{A}}_i\left(t\right)\parallel$ are finite and positive. Consider (8) for $i=0$ and choose an initial condition such that $\parallel {\mathbf{x}}_j\left(0\right)\parallel =0$ for all $j\ne n$ and $\parallel {\mathbf{x}}_n\left(0\right)\parallel \ne 0$ for any fixed $n\in \{K_1,\dots ,K_2\}$.

By sequentially substituting the expressions for ${\mathbf{x}}_i\left(t\right)$, given by (8), into the integrals on the right-hand side of (8) when $i=0$, and then by taking norm of both sides, remembering that

$$\begin{array}{c}\parallel {\mathbf{x}}_i\left(t\right)\parallel \le \parallel U_i(t,s)\parallel \parallel {\mathbf{x}}_i\left(s\right)\parallel +{\int }_0^t\parallel U_i(t,s)\parallel \parallel {\mathfrak{R}}_{i-1}\left(s\right)\parallel \parallel {\mathbf{x}}_{i-1}\left(s\right)\parallel ds+\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+{\int }_0^t\parallel U_i(t,s)\parallel \parallel {\mathfrak{A}}_{i+1}\left(s\right)\parallel \parallel {\mathbf{x}}_{i+1}\left(s\right)\parallel ds,\end{array}$$

and using the standard inequality for the binomial coefficients $\left({\displaystyle \genfrac{}{}{0pt}{}{i+2k}{i+k}}\right)\le 4^{k+{\displaystyle \frac{i}{2}}}$, one obtains the following sequence of upper bounds for $\parallel {\mathbf{x}}_0\left(t\right)\parallel$:

$$\begin{array}{c}\parallel {\mathbf{x}}_0\left(t\right)\parallel \le K^n{t}^{n}exp\left(-{\int }_0^t\gamma \left(s\right)ds\right){\displaystyle \sum _{k=0}^{\infty }}\left({\displaystyle \genfrac{}{}{0pt}{}{n+2k}{n+k}}\right){\displaystyle \frac{K^k{L}^k{t}^{2k}}{(n+2k)!}}\parallel {\mathbf{x}}_n\left(0\right)\parallel \le \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\le {\left({\displaystyle \frac{K}{L}}\right)}^{{\displaystyle \frac{n}{2}}}exp\left(-{\int }_0^t\gamma \left(s\right)ds\right){\displaystyle \sum _{k=0}^{\infty }}{\displaystyle \frac{{\left(4KL\right)}^{k+\frac{n}{2}}{t}^{2k+n}}{(n+2k)!}}\parallel {\mathbf{x}}_n\left(0\right)\parallel \le \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\le exp\left(-{\int }_0^t\gamma \left(s\right)ds\right){\left({\displaystyle \frac{K}{L}}\right)}^{{\displaystyle \frac{n}{2}}}exp\left(2t\sqrt{KL}\right)\parallel {\mathbf{x}}_n\left(0\right)\parallel .\end{array}$$

(9)

These bounds are valid only in the case when $\parallel {\mathbf{x}}_j\left(0\right)\parallel =0$ for all $j\ne n$ and $\parallel {\mathbf{x}}_n\left(0\right)\parallel \ne 0$ for any fixed $n\in \{K_1,\dots ,K_2\}$. If one allows for an arbitrary initial condition $\{\parallel {\mathbf{x}}_i\left(0\right)\parallel ,-K_1\le i\le K_2\}$, then along the same lines, it can be shown that $\parallel {\mathbf{x}}_0\left(t\right)\parallel$ is upper bounded by

$$\begin{array}{c}\hfill \parallel {\mathbf{x}}_0\left(t\right)\parallel \le exp\left(-{\int }_0^t\gamma \left(s\right)ds\right)\sum _{k=-K_1}^{K_2}{\left({\displaystyle \frac{K}{L}}\right)}^{{\displaystyle \frac{k}{2}}}exp\left(2t\sqrt{KL}\right)\parallel {\mathbf{x}}_k\left(0\right)\parallel .\end{array}$$

(10)

One of the conclusions that can be immediately drawn from (10) is that for the considered type of CTMCs with the finite state space, one does not need to care about the starting state of the chain: the obtained upper bound will always be meaningful (finite). If the state space is the whole set of integers (as, for example, happens in the analysis of double-ended queues; see [19]), one can make the infinite sum in the right-hand side of (10) converge by choosing the proper initial conditions. The same argument applies in the case when the state space has either maxima or minima (i.e., either $K_1=\infty$ or $K_2=\infty$). However, as the analysis shows, these cases need to be treated with different methods, and furthermore, we dwell only on the latter case, which is also, within the classic Markovian queueing framework, the most common one.

Assume for definiteness that $K_1=0$ and $K_2=\infty$, it can be shown that the upper bounds for the components $\parallel {\mathbf{x}}_i\left(t\right)\parallel$ of the solution $\mathbf{x}\left(t\right)$ have a form similar to (10), and since $\parallel \mathbf{x}\left(t\right)\parallel ={\sum }_{k=0}^{\infty }\parallel {\mathbf{x}}_k\left(t\right)\parallel$, it holds that

$$\begin{array}{c}\parallel \mathbf{x}\left(t\right)\parallel \le exp\left(-{\int }_0^t\gamma \left(s\right)ds\right)\left(1+{\left({\displaystyle \frac{K}{L}}\right)}^{{\displaystyle \frac{1}{2}}}+\left({\displaystyle \frac{K}{L}}\right)+{\left({\displaystyle \frac{K}{L}}\right)}^{{\displaystyle \frac{3}{2}}}+\cdots \right)exp\left(2t\sqrt{KL}\right)\parallel {\mathbf{x}}_0\left(0\right)\parallel +\hfill \\ exp\left(-{\int }_0^t\gamma \left(s\right)ds\right)\left({\left({\displaystyle \frac{L}{K}}\right)}^{{\displaystyle \frac{1}{2}}}+1+{\left({\displaystyle \frac{K}{L}}\right)}^{{\displaystyle \frac{1}{2}}}+\left({\displaystyle \frac{K}{L}}\right)+{\left({\displaystyle \frac{K}{L}}\right)}^{{\displaystyle \frac{3}{2}}}+\cdots \right)exp\left(2t\sqrt{KL}\right)\parallel {\mathbf{x}}_1\left(0\right)\parallel +\\ exp\left(-{\int }_0^t\gamma \left(s\right)ds\right)\left(\left({\displaystyle \frac{L}{K}}\right)+{\left({\displaystyle \frac{L}{K}}\right)}^{{\displaystyle \frac{1}{2}}}+1+{\left({\displaystyle \frac{K}{L}}\right)}^{{\displaystyle \frac{1}{2}}}+\left({\displaystyle \frac{K}{L}}\right)+\cdots \right)exp\left(2t\sqrt{KL}\right)\parallel {\mathbf{x}}_2\left(0\right)\parallel +\\ \hfill exp\left(-{\int }_0^t\gamma \left(s\right)ds\right)\left({\left({\displaystyle \frac{L}{K}}\right)}^{{\displaystyle \frac{3}{2}}}+\left({\displaystyle \frac{L}{K}}\right)+{\left({\displaystyle \frac{L}{K}}\right)}^{{\displaystyle \frac{1}{2}}}+1+{\left({\displaystyle \frac{K}{L}}\right)}^{{\displaystyle \frac{1}{2}}}+\cdots \right)exp\left(2t\sqrt{KL}\right)\parallel {\mathbf{x}}_3\left(0\right)\parallel +\cdots ,\end{array}$$

where in the brackets one has a geometric series, which converges whenever $K/L<1$. Therefore, by collapsing the infinite sums, the final bound can be represented as

$$\begin{array}{c}\hfill \parallel \mathbf{x}\left(t\right)\parallel \le exp\left(2t\sqrt{KL}-{\int }_0^t\gamma \left(s\right)ds\right){\displaystyle \frac{\sqrt{L}}{\sqrt{L}-\sqrt{K}}}\sum _{k=0}^{\infty }{\left({\displaystyle \frac{L}{K}}\right)}^{{\displaystyle \frac{k}{2}}}\parallel {\mathbf{x}}_k\left(0\right)\parallel .\end{array}$$

(11)

Depending on the behavior of the exponent and the infinite sum, the value of the right part of (11) may go down to 0 as $t\to \infty$, or may grow as t increases. The assumptions made so far seem to be insufficient to come up with the exact conditions that would allow one to distinguish these cases. Therefore, we resort to the analysis of each case separately, which will clarify what is being missed.

4. Null Ergodicity

Consider the system (2). Put

$$D=\mathrm{diag}(1,\dots ,1,d,\dots ,d,d^2,\dots ,d^2,d^3,\dots ,d^3,\dots ),$$

where the number of identical elements is equal to the corresponding block size. Assume that there exists such $d\in (0,1)$ that $\parallel \tilde{A}\left(t\right)\parallel$ is uniformly bounded in t. This guarantees the uniqueness of the solution of (3).

Theorem 1. 

Let there exist such d that $d^{2}L<K<L$, where $L={sup}_i{sup}_t\parallel {\mathfrak{R}}_i\left(t\right)\parallel$ and $K={sup}_i{sup}_t\parallel {\mathfrak{A}}_i\left(t\right)\parallel$, and such a function $\gamma \left(t\right)$ that $\parallel U_i(t,0)\parallel \le e^{-{\int }_0^t\gamma \left(s\right)ds}$, where $U_i(t,0)$ is the Cauchy operator for the ODE system ${\displaystyle \frac{d}{dt}}{\mathbf{x}}_i\left(t\right)={\mathfrak{B}}_i\left(t\right){\mathbf{x}}_i\left(t\right)$. Then, if ${lim}_{t\to \infty }\left(2t\sqrt{KL}-{\int }_0^t\gamma \left(s\right)ds\right)=-\infty$, the Markov chain $\left\{X\right(t),t\ge 0\}$ is null-ergodic and the rate of convergence of $p_i\left(t\right)\to 0$ is upper bounded by

$$p_i\left(t\right)\le {\displaystyle \frac{2}{d^i}}exp\left(2t\sqrt{KL}-{\int }_0^t\gamma \left(s\right)ds\right){\displaystyle \frac{\sqrt{KL}}{(\sqrt{L}-\sqrt{K})(\sqrt{K}-d\sqrt{L})}},t>0.$$

Proof. 

Choose a non-negative vector $\mathbf{p}\left(t\right)$, satisfying $\parallel \mathbf{p}\left(0\right)\parallel =1$ and otherwise arbitrary. Since $1=\parallel \mathbf{p}\left(0\right)\parallel ={\sum }_{k=0}^{\infty }{p}_k\left(0\right)\ge {\sum }_{k=0}^{\infty }{d}^k{p}_k\left(0\right)=\parallel D\mathbf{p}\left(0\right)\parallel$, the norm $\parallel D\mathbf{p}\left(0\right)\parallel$ is bounded. Note that $\overrightarrow{0}$ is also the solution of (3). Let $\mathbf{x}\left(t\right)=D(\mathbf{p}\left(t\right)-\overrightarrow{0})$. Since $\parallel {\mathbf{x}}_k\left(0\right)\parallel =d^k\parallel \mathbf{p}\left(t\right)\parallel \le d^k$, one can simplify (11) to obtain

$$\begin{array}{c}\hfill \parallel D\mathbf{p}\left(t\right)\parallel \le exp\left(2t\sqrt{KL}-{\int }_0^t\gamma \left(s\right)ds\right){\displaystyle \frac{\sqrt{KL}}{(\sqrt{L}-\sqrt{K})(\sqrt{K}-d\sqrt{L})}}.\end{array}$$

(12)

However, since $p_i\left(t\right)d^i\le \parallel D\mathbf{p}\left(t\right)\parallel$, the statements of the theorem follow. □

Several examples, which follow, aim at demonstrating how Theorem 1 can be applied to obtain the exact expressions for the upper convergence bounds.

Example 1. 

As the first example, let us consider the standard (possibly inhomogeneous) birth–death process on $\{0,1,\dots \}$ with the birth rates ${\lambda }_i\left(t\right)$ and the death rates ${\mu }_i\left(t\right)$. The infinitesimal matrix $A\left(t\right)$ has a simple block structure

$$\begin{array}{c}\hfill A\left(t\right)=\left(\begin{array}{cccc}{\lambda }_0\left(t\right)& {\mu }_1\left(t\right)& 0& \dots \\ {\lambda }_0\left(t\right)& -{\lambda }_1\left(t\right)-{\mu }_1\left(t\right)& {\mu }_2\left(t\right)& \dots \\ 0& {\lambda }_1\left(t\right)& -{\lambda }_2\left(t\right)-{\mu }_2\left(t\right)& \dots \\ ⋮& ⋮& ⋮& \dots \end{array}\right)\end{array}$$

(13)

The size of each block in $A\left(t\right)$ is $1\times 1$, and therefore the appropriate matrix D is $D=\mathrm{diag}(1,d,d^2,\dots )$. Since

$$\begin{array}{c}\hfill \tilde{A}\left(t\right)=DA\left(t\right)D^{-1}=\left(\begin{array}{cccc}{\lambda }_0\left(t\right)& {\displaystyle \frac{{\mu }_1\left(t\right)}{d}}& 0& \dots \\ d{\lambda }_0\left(t\right)& -{\lambda }_1\left(t\right)-{\mu }_1\left(t\right)& {\displaystyle \frac{{\mu }_2\left(t\right)}{d}}& \dots \\ 0& d{\lambda }_1\left(t\right)& -{\lambda }_2\left(t\right)-{\mu }_2\left(t\right)& \dots \\ ⋮& ⋮& ⋮& \dots \end{array}\right),\end{array}$$

(14)

it is straightforward to see that $L={sup}_i{sup}_t\left|{\displaystyle \frac{{\mu }_i\left(t\right)}{d}}\right|$, $K={sup}_i{sup}_t\left|d{\lambda }_i\left(t\right)\right|$, and $\gamma \left(t\right)=inf({\lambda }_0\left(t\right),{\lambda }_1\left(t\right)+{\mu }_1\left(t\right),{\lambda }_2\left(t\right)+{\mu }_2\left(t\right),\cdots )$. In order for the inequality $d^{2}L<K<L$ to hold, one must choose d such that it satisfies the double inequality $d<\sqrt{{\displaystyle \frac{{sup}_i{sup}_t\left|{\mu }_i\left(t\right)\right|}{{sup}_i{sup}_t\left|{\lambda }_i\left(t\right)\right|}}}$ and ${sup}_i{sup}_t\left|{\mu }_i\left(t\right)\right|<{sup}_i{sup}_t\left|{\lambda }_i\left(t\right)\right|$ (in which the right part is always less than 1). For the considered birth–death process to be null-ergodic, the latter inequality must hold simultaneously with the inequality $\gamma \left(t\right)>2\sqrt{KL}$.

In the state-independent homogeneous case, i.e., when ${\lambda }_i\left(t\right)=\lambda$ and ${\mu }_i\left(t\right)=\mu$, this gives $d<\sqrt{{\displaystyle \frac{\mu }{\lambda }}}$ and the overly optimistic null-ergodicity condition $\lambda >4\mu$ (compared to the standard one, $\lambda >\mu$) with the rate of convergence $\lambda -2\sqrt{\lambda \mu }$.

Example 2. 

As the second example, consider the model of the server with breakdown and repair from [20] (see Model 2). If the server, being a classical $M_t/M_t/1$ queue with the parameters $\lambda \left(t\right)$ and $\mu \left(t\right)$, is operating, it breaks down with a time to breakdown distribution that is exponential with parameter $a\left(t\right)$. Once the breakdown occurs, the repair starts. The time to the end of repair is also exponentially distributed with parameter $b\left(t\right)$. During the breakdown, the part that was being processed at the time breakdown occurred is not processed. Processing starts on the same part after the repair. The infinitesimal matrix $A\left(t\right)$ for the QBD process $(X_1\left(t\right),X_2\left(t\right))$ ($X_1\left(t\right)$—the queue-size when the server is working; $X_2\left(t\right)$—the queue-size when the server is broken) has a block structure

$$\begin{array}{c}\hfill A\left(t\right)=\left(\begin{array}{ccccc}-a\left(t\right)-\lambda \left(t\right)& b\left(t\right)& \mu \left(t\right)& 0& \dots \\ a\left(t\right)& -b\left(t\right)-\lambda \left(t\right)& 0& 0& \dots \\ \lambda \left(t\right)& 0& -a\left(t\right)-\lambda \left(t\right)-\mu \left(t\right)& b& \mu \left(t\right)& 0& \dots \\ 0& \lambda \left(t\right)& a& -b\left(t\right)-\lambda \left(t\right)& 0& 0& \dots \\ & & \lambda \left(t\right)& 0& -a\left(t\right)-\lambda \left(t\right)-\mu \left(t\right)& b\left(t\right)& \mu \left(t\right)& \dots \\ & & 0& \lambda \left(t\right)& a\left(t\right)& -b\left(t\right)-\lambda \left(t\right)& 0& \dots \\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& \dots \end{array}\right)\end{array}$$

(15)

in which each block is of size $2\times 2$, and therefore the appropriate matrix D is $D=\mathrm{diag}(1,1,d,d,d^2,d^2\cdots )$. Since

$$\begin{array}{c}\hfill \tilde{A}\left(t\right)=\left(\begin{array}{ccccc}-a\left(t\right)-\lambda \left(t\right)& b& {\displaystyle \frac{\mu \left(t\right)}{d}}& 0& \dots \\ a& -b\left(t\right)-\lambda \left(t\right)& 0& 0& \dots \\ d\lambda \left(t\right)& 0& -a\left(t\right)-\lambda \left(t\right)-\mu \left(t\right)& b\left(t\right)& {\displaystyle \frac{\mu \left(t\right)}{d}}& 0& \dots \\ 0& d\lambda \left(t\right)& a\left(t\right)& -b\left(t\right)-\lambda \left(t\right)& 0& 0& \dots \\ & & d\lambda \left(t\right)& 0& -a\left(t\right)-\lambda \left(t\right)-\mu \left(t\right)& b\left(t\right)& {\displaystyle \frac{\mu \left(t\right)}{d}}& \dots \\ & & 0& d\lambda \left(t\right)& a& -b\left(t\right)-\lambda \left(t\right)& 0& \dots \\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& \dots \end{array}\right)\end{array}$$

(16)

it is straightforward to see that $L={sup}_t\left|{\displaystyle \frac{\mu \left(t\right)}{d}}\right|$ and $K={sup}_t\left|d\lambda \left(t\right)\right|$ and $\gamma \left(t\right)=\lambda \left(t\right)$. In order for the inequality $d^{2}L<K<L$ to hold, one must choose d such that it satisfies the double inequality $d<\sqrt{{\displaystyle \frac{{sup}_t\left|\mu \left(t\right)\right|}{{sup}_t\left|\lambda \left(t\right)\right|}}}$ and ${sup}_t\left|\mu \left(t\right)\right|<{sup}_t\left|\lambda \left(t\right)\right|$. For the considered QBD to be null-ergodic, the latter inequality must hold simultaneously with the inequality $\gamma \left(t\right)>2\sqrt{KL}$.

In the homogeneous case, one of the suitable choices is $d<\sqrt{\mu /\lambda }$, which gives the simple and overly pessimistic null-ergodicity condition $\lambda >4\mu$.

Example 3. 

As the third example, we consider the classic possibly inhomogeneous $M/E_2/1$ queue (see, for example, Model 3 in [20]). The arrival rate is denoted by $\lambda \left(t\right)$, and stages of the Erlang service time are exponentially distributed with the parameter $\mu \left(t\right)$. The infinitesimal matrix $A\left(t\right)$ for the QBD process (excluding the state when the queue is empty, which does not impact the analysis of ergodicity bounds) $(X_1\left(t\right),X_2\left(t\right))$ ($X_1\left(t\right)$—the queue-size; $X_2\left(t\right)$—the Erlang stage) has a block structure

$$\begin{array}{c}\hfill A\left(t\right)=\left(\begin{array}{ccccc}-\lambda \left(t\right)-\mu \left(t\right)& \mu \left(t\right)& 0& \mu \left(t\right)& \dots \\ \mu \left(t\right)& -\lambda \left(t\right)-\mu \left(t\right)& 0& 0& \dots \\ \lambda \left(t\right)& 0& -\lambda \left(t\right)-\mu \left(t\right)& 0& 0& \mu \left(t\right)& \dots \\ 0& \lambda \left(t\right)& \mu \left(t\right)& -\lambda \left(t\right)-\mu \left(t\right)& 0& 0& \dots \\ & & \lambda \left(t\right)& 0& -\lambda \left(t\right)-\mu \left(t\right)& 0& 0& \mu \left(t\right)& \dots \\ & & 0& \lambda \left(t\right)& \mu \left(t\right)& -\lambda \left(t\right)-\mu \left(t\right)& 0& 0& \dots \\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& \dots \end{array}\right),\end{array}$$

(17)

in which each block is of size $2\times 2$, and therefore the appropriate matrix D is $D=\mathrm{diag}(1,1,d,d,d^2,d^2\cdots )$. Since

$$\begin{array}{c}\hfill \tilde{A}\left(t\right)=\left(\begin{array}{ccccc}-\lambda \left(t\right)-\mu \left(t\right)& \mu \left(t\right)& 0& {\displaystyle \frac{\mu \left(t\right)}{d}}& \dots \\ \mu \left(t\right)& -\lambda \left(t\right)-\mu \left(t\right)& 0& 0& \dots \\ d\lambda \left(t\right)& 0& -\lambda \left(t\right)-\mu \left(t\right)& 0& 0& {\displaystyle \frac{\mu \left(t\right)}{d}}& \dots \\ 0& d\lambda \left(t\right)& \mu \left(t\right)& -\lambda \left(t\right)-\mu \left(t\right)& 0& 0& \dots \\ & & d\lambda \left(t\right)& 0& -\lambda \left(t\right)-\mu \left(t\right)& 0& 0& {\displaystyle \frac{\mu \left(t\right)}{d}}& \dots \\ & & 0& d\lambda \left(t\right)& \mu \left(t\right)& -\lambda \left(t\right)-\mu \left(t\right)& 0& 0& \dots \\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& \dots \end{array}\right)\end{array}$$

(18)

it is straightforward to see that $L={sup}_t\left|{\displaystyle \frac{\mu \left(t\right)}{d}}\right|$, $K={sup}_t\left|d\lambda \left(t\right)\right|$, and $\gamma \left(t\right)=\lambda \left(t\right)$. In order for the inequality $d^{2}L<K<L$ to hold, one much choose d such that it satisfies the double inequality $d<\sqrt{{\displaystyle \frac{{sup}_t\left|\mu \left(t\right)\right|}{{sup}_t\left|\lambda \left(t\right)\right|}}}$ and ${sup}_t\left|\mu \left(t\right)\right|<{sup}_t\left|\lambda \left(t\right)\right|$. For the considered QBD to be null-ergodic, the latter inequality must hold simultaneously with the inequality $\gamma \left(t\right)>2\sqrt{KL}$.

As in the previous example, in the homogeneous case one of the suitable choices is $d<\sqrt{\mu /\lambda }$, which gives the simple and overly pessimistic null-ergodicity condition $\lambda >4\mu$.

Example 4.

As the final example in this section, let us consider Model 5 from [20], which is a tandem queue $M_t/M_t/1/0\to ·/M_t/1$. The arrival rate is, as usual, denoted by $\lambda \left(t\right)$, and the service rates are ${\mu }_1\left(t\right)$ and ${\mu }_2\left(t\right)$, respectively. The infinitesimal matrix $A\left(t\right)$ for the QBD process $(X_1\left(t\right),X_2\left(t\right))$ ($X_1\left(t\right)$—the state of the first stage; $X_2\left(t\right)$—the queue-size in the second stage) has a block structure

$$\begin{array}{c}\hfill A\left(t\right)=\left(\begin{array}{ccccc}-\lambda \left(t\right)& 0& {\mu }_2\left(t\right)& 0& \dots \\ \lambda \left(t\right)& -{\mu }_1\left(t\right)& 0& {\mu }_2\left(t\right)& \dots \\ 0& {\mu }_1\left(t\right)& -\lambda \left(t\right)-{\mu }_2\left(t\right)& 0& {\mu }_2\left(t\right)& 0& \dots \\ 0& 0& \lambda \left(t\right)& -{\mu }_1\left(t\right)-{\mu }_2\left(t\right)& 0& {\mu }_2\left(t\right)& \dots \\ & & 0& {\mu }_1\left(t\right)& -\lambda \left(t\right)-{\mu }_2\left(t\right)& 0& {\mu }_2\left(t\right)& 0& \dots \\ & & 0& 0& \lambda \left(t\right)& -{\mu }_1\left(t\right)-{\mu }_2\left(t\right)& 0& {\mu }_2\left(t\right)& \dots \\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& \dots \end{array}\right)\end{array}$$

(19)

in which each block is of size $2\times 2$, and therefore the appropriate matrix D is $D=\mathrm{diag}(1,1,d,d,d^2,d^2,d^3,d^3\dots )$, since

$$\begin{array}{c}\hfill \tilde{A}\left(t\right)=\left(\begin{array}{ccccc}-\lambda \left(t\right)& 0& {\displaystyle \frac{{\mu }_2\left(t\right)}{d}}& 0& \dots \\ \lambda \left(t\right)& -{\mu }_1\left(t\right)& 0& {\displaystyle \frac{{\mu }_2\left(t\right)}{d}}& \dots \\ 0& d{\mu }_1\left(t\right)& -\lambda \left(t\right)-{\mu }_2\left(t\right)& 0& {\displaystyle \frac{{\mu }_2\left(t\right)}{d}}& 0& \dots \\ 0& 0& \lambda \left(t\right)& -{\mu }_1\left(t\right)-{\mu }_2\left(t\right)& 0& {\displaystyle \frac{{\mu }_2\left(t\right)}{d}}& \dots \\ & & 0& d{\mu }_1\left(t\right)& -\lambda \left(t\right)-{\mu }_2\left(t\right)& 0& {\displaystyle \frac{{\mu }_2\left(t\right)}{d}}& 0& \dots \\ & & 0& 0& \lambda \left(t\right)& -{\mu }_1\left(t\right)-{\mu }_2\left(t\right)& 0& {\displaystyle \frac{{\mu }_2\left(t\right)}{d}}& \dots \\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& \dots \end{array}\right).\end{array}$$

(20)

Let us bound $\parallel U_0(t,0)\parallel$, where $U_0(t,0)$ is the Cauchy operator for the ODE system ${\displaystyle \frac{d}{dt}}{\mathbf{x}}_0\left(t\right)={\mathfrak{B}}_0\left(t\right){\mathbf{x}}_0\left(t\right)$ and

$$\begin{array}{c}\hfill {\mathfrak{B}}_0\left(t\right)=\left(\begin{array}{cc}-\lambda \left(t\right)& 0\\ \lambda \left(t\right)& -{\mu }_1\left(t\right)\end{array}\right).\end{array}$$

(21)

Put $D_0=\mathrm{diag}(1,k)$, $0<k<1$. Then

$$\begin{array}{c}\hfill D_0{\mathfrak{B}}_0\left(t\right)D_0^{-1}=\left(\begin{array}{cc}-\lambda \left(t\right)& 0\\ k\lambda \left(t\right)& -{\mu }_1\left(t\right)\end{array}\right).\end{array}$$

(22)

For arbitrary positive t and $t_0$, such that $t>t_0$, it holds that

$${\displaystyle \frac{1}{k}}\le exp\left(ln{\displaystyle \frac{1}{k}}\right)\le exp\left({\displaystyle \frac{t}{t_0}}ln{\displaystyle \frac{1}{k}}\right)\le exp\left({\int }_0^t{\displaystyle \frac{1}{t_0}}ln{\displaystyle \frac{1}{k}}ds\right).$$

Therefore,

$$\begin{array}{c}\parallel U_0(t,0)\parallel \le \parallel D_0\parallel \parallel D_0^{-1}\parallel \parallel D_0{U}_0(t,0)D_0^{-1}\parallel \le \hfill \\ \hspace{1em}\hspace{1em}\le {\displaystyle \frac{1}{k}}exp\left(-\underset{0}{\overset{t}{\int }}min\left(\lambda \left(s\right)\left(1-k\right),{\mu }_1\left(s\right)\right)ds\right)\le \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\le exp\left(\underset{0}{\overset{t}{\int }}{\displaystyle \frac{1}{t_0}}ln{\displaystyle \frac{1}{k}}-min\left(\lambda \left(s\right)\left(1-k\right),{\mu }_1\left(s\right)\right)ds\right).\end{array}$$

By applying the same reasoning for the next main diagonal block of $\tilde{A}\left(t\right)$, which is the matrix

$$\begin{array}{c}\hfill {\mathfrak{B}}_1\left(t\right)=\left(\begin{array}{cc}-\lambda \left(t\right)-{\mu }_2\left(t\right)& 0\\ \lambda \left(t\right)& -{\mu }_1\left(t\right)-{\mu }_2\left(t\right)\end{array}\right)\end{array}$$

(23)

instead of ${\mathfrak{B}}_0\left(t\right)$, one obtains:

$$\begin{array}{c}\parallel U_1(t,0)\parallel \le \parallel D_1\parallel \parallel D_1^{-1}\parallel \parallel D_1{U}_1(t,0)D_1^{-1}\parallel \le \hfill \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\le {\displaystyle \frac{1}{k}}exp\left(-\underset{0}{\overset{t}{\int }}min\left(\lambda \left(s\right)\left(1-k\right)+{\mu }_2\left(s\right),{\mu }_1\left(s\right)+{\mu }_2\left(s\right)\right)ds\right).\end{array}$$

All other main diagonal blocks of $\tilde{A}\left(t\right)$ are identical with ${\mathfrak{B}}_1\left(t\right)$. Therefore,

$$\parallel U(t,0)\parallel \le \underset{i=1,2}{inf}\parallel U_i(t,0)\parallel \le exp\left(\underset{0}{\overset{t}{\int }}{\displaystyle \frac{1}{t_0}}ln{\displaystyle \frac{1}{k}}-min\left(\lambda \left(s\right)\left(1-k\right),{\mu }_1\left(s\right)\right)ds\right).$$

Since $L={sup}_t\left|{\displaystyle \frac{{\mu }_2\left(t\right)}{d}}\right|$, $K={sup}_t\left|d{\mu }_1\left(t\right)\right|$, and

$$\gamma \left(t\right)=-\left({\displaystyle \frac{1}{t_0}}ln{\displaystyle \frac{1}{k}}-min\left(\lambda \left(t\right)\left(1-k\right),{\mu }_1\left(t\right)\right)\right)$$

for the double inequality $d^{2}L<K<L$ to hold, one much choose d such that it satisfies the condition $d<\sqrt{{\displaystyle \frac{{sup}_t\left|{\mu }_2\left(t\right)\right|}{{sup}_t\left|{\mu }_1\left(t\right)\right|}}}$ and ${sup}_t\left|{\mu }_2\left(t\right)\right|<{sup}_t\left|{\mu }_1\left(t\right)\right|$. For the considered QBD to be null-ergodic, the latter inequality must hold simultaneously with the inequality $\gamma \left(t\right)>2\sqrt{KL}$ and $t>t_0>0$. In the homogeneous case, the two inequalities do hold if

$$\left\{\begin{array}{ccc}\hfill {\mu }_1& >& {\mu }_2,\hfill \\ \hfill min\left(\lambda ,{\mu }_1\right)& >& 2\sqrt{{\mu }_1{\mu }_2},\hfill \end{array}\right.$$

which is again too optimistic, compared to the null-ergodicity ${\mu }_1^2>{\mu }_2(\lambda +{\mu }_1)$, derived from the classic QBD results (positive drift condition).

5. Weak Erdogicity

Recall that a CTMC is called weakly ergodic if $\parallel {\mathbf{p}}^{\ast }\left(t\right)-{\mathbf{p}}^{\ast \ast }\left(t\right)\parallel \to 0$ as $t\to \infty$ for any initial conditions ${\mathbf{p}}^{\ast }\left(0\right),{\mathbf{p}}^{\ast \ast }\left(0\right)$, where ${\mathbf{p}}^{\ast }\left(t\right)$ and ${\mathbf{p}}^{\ast \ast }\left(t\right)$ are the corresponding solutions of (2).

The route to obtain the upper bounds is as follows. At first, one guesses the functions $\overrightarrow{r}\left(t\right)$ and subtracts them from all rows of the matrix (the transposed infinitesimal matrix of the original CTMC or the censored CTMC with the state space $\{1,2,\dots \}$) $A\left(t\right)$. Then, one performs the similarity transformation $T\left(A\left(t\right)-({\overrightarrow{1}}^T\otimes \overrightarrow{r}\left(t\right))\right)T^{-1}$, where T is the upper triangular matrix of the form

$$\begin{array}{c}\hfill T=\left(\begin{array}{cccc}1& 1& 1& \dots \\ 0& 1& 1& \dots \\ 0& 0& 1& \dots \\ ⋮& ⋮& ⋮& \ddots \end{array}\right)\end{array}$$

Finally, the diagonal matrix

$$D=\mathrm{diag}(1,\dots ,1,d,\dots ,d,d^2,\dots ,d^2,d^3,\dots ,d^3,\dots ),$$

having the same structure as in Section 4 but with $d>1$, may be applied in the form of a similarity transformation in order to make the exponent in (11) as large as possible. Therefore, the matrix $\tilde{A}\left(t\right)$ in (3) is either

$$\tilde{A}\left(t\right)=T\left(A\left(t\right)-({\overrightarrow{1}}^T\otimes \overrightarrow{r}\left(t\right))\right)T^{-1} \mathrm{or} \tilde{A}\left(t\right)=DT\left(A\left(t\right)-({\overrightarrow{1}}^T\otimes \overrightarrow{r}\left(t\right))\right)T^{-1}{D}^{-1}.$$

Since both K and L in (11) depend on the form of the matrix $\tilde{A}\left(t\right)$, one has to determine them in each particular case. Note that, unlike the null-ergodicity case, we have that $K<L\le dL$, and thus in infinite sum ${\sum }_{k=0}^{\infty }{\left({\displaystyle \frac{L}{K}}\right)}^{{\displaystyle \frac{k}{2}}}\parallel {\mathbf{x}}_k\left(0\right)\parallel$ is not finite for an arbitrary $\parallel {\mathbf{x}}_k\left(0\right)\parallel$. Therefore, we assume that $\parallel {\mathbf{x}}_k\left(0\right)\parallel$ are chosen such that $\parallel {\mathbf{x}}_i\left(0\right)\parallel =0$ for i larger than some fixed j; this guarantees the convergence of the series.

Theorem 2. 

Let there exist such d that $K<L\le dL$, where $L={sup}_i{sup}_t\parallel {\mathfrak{R}}_i\left(t\right)\parallel$ and $K={sup}_i{sup}_t\parallel {\mathfrak{A}}_i\left(t\right)\parallel$, and such a function $\gamma \left(t\right)$ that $\parallel U_i(t,0)\parallel \le e^{-{\int }_0^t\gamma \left(s\right)ds}$, where $U_i(t,0)$ is the Cauchy operator for the ODE system ${\displaystyle \frac{d}{dt}}{\mathbf{x}}_i\left(t\right)={\mathfrak{B}}_i\left(t\right){\mathbf{x}}_i\left(t\right)$. Then, if ${lim}_{t\to \infty }\left(2t\sqrt{KL}-{\int }_0^t\gamma \left(s\right)ds\right)=-\infty$, the Markov chain $\left\{X\right(t),t\ge 0\}$ is weakly ergodic, and the upper ergodicity bound is

$$\begin{array}{c}\hfill \parallel {\mathbf{p}}^{\ast \ast }\left(t\right)-{\mathbf{p}}^{\ast }\left(t\right)\parallel \le exp\left(2t\sqrt{KL}-{\int }_0^t\gamma \left(s\right)ds\right){\displaystyle \frac{\sqrt{L}}{\sqrt{L}-\sqrt{K}}}{\left({\displaystyle \frac{L}{K}}\right)}^{{\displaystyle \frac{j}{2}}}\parallel DT({\mathbf{p}}^{\ast \ast }\left(0\right)-{\mathbf{p}}^{\ast }\left(0\right))\parallel .\end{array}$$

Since the considered CTMC is a QDB, one may be interested either in the expected value of its level or phase, or some other probabilistic characteristics. Denote by $L^{\ast }\left(t\right)={\sum }_{i=0}^{\infty }i\parallel {\mathbf{p}}_i^{\ast }\left(t\right)\parallel$ and $L^{\ast \ast }\left(t\right)={\sum }_{i=0}^{\infty }i\parallel {\mathbf{p}}_i^{\ast \ast }\left(t\right)\parallel$ the expected levels of the considered CTMC with the initial conditions ${\mathbf{p}}^{\ast \ast }\left(0\right)$ and ${\mathbf{p}}^{\ast }\left(0\right)$, respectively. If Theorem 2 holds, then

$$\parallel L^{\ast \ast }\left(t\right)-L^{\ast }\left(t\right)\parallel \le {\displaystyle \frac{exp\left(2t\sqrt{KL}-{\int }_0^t\gamma \left(s\right)ds\right)}{elnd}}{\displaystyle \frac{\sqrt{L}}{\sqrt{L}-\sqrt{K}}}{\left({\displaystyle \frac{L}{K}}\right)}^{{\displaystyle \frac{j}{2}}}\parallel DT({\mathbf{p}}^{\ast \ast }\left(0\right)-{\mathbf{p}}^{\ast }\left(0\right))\parallel .$$

Now, as in Section 4, we proceed with the same examples, which demonstrate how Theorem 2 can be applied to obtain the exact expressions for the upper convergence bounds.

Example 5. 

We begin with the standard (possibly inhomogeneous) birth–death process on $\{0,1,\dots \}$. Let $r_0\left(t\right)={\lambda }_0\left(t\right)+{\mu }_1\left(t\right)$ and $r_i\left(t\right)=0$ for $i\ge 1$. Then, the matrix

$$\tilde{A}\left(t\right)=T\left(A\left(t\right)-({\overrightarrow{1}}^T\otimes \overrightarrow{r}\left(t\right))\right)T^{-1}$$

will have the block structure:

$$\begin{array}{c}\hfill \tilde{A}\left(t\right)=\left(\begin{array}{cccc}-{\lambda }_0-{\mu }_1& 0& 0& \dots \\ {\lambda }_0& -{\lambda }_0-{\mu }_1& {\mu }_1& \dots \\ 0& {\lambda }_1& -{\lambda }_1-{\mu }_2& \dots \\ ⋮& ⋮& ⋮& \dots \end{array}\right).\end{array}$$

(24)

It is straightforward to see that $K={sup}_i{sup}_t\left|{\lambda }_i\left(t\right)\right|$, $L={sup}_i{sup}_t\left|{\mu }_i\left(t\right)\right|$, and $\gamma \left(t\right)=inf({\lambda }_0\left(t\right)+{\mu }_1\left(t\right),{\lambda }_1\left(t\right)+{\mu }_2\left(t\right),\cdots )$. Therefore, the weak ergodicity condition is $\gamma \left(t\right)>2\sqrt{KL}$. Moreover, since it must hold that $K<L\le dL$, the feasible region for d is $1\le d\le \sqrt{{\displaystyle \frac{{sup}_i{sup}_t|{\mu }_i\left(t\right)|}{{sup}_i{sup}_t\left|{\lambda }_i\left(t\right)\right|}}}$. In the state-independent homogeneous case, one can put $d={\displaystyle \frac{\sqrt{\mu }}{\sqrt{\lambda }}}$. Then, $K={sup}_i{sup}_t\left|d{\lambda }_i\left(t\right)\right|=\sqrt{\mu \lambda }$, $L={sup}_i{sup}_t{\displaystyle \frac{|{\mu }_i\left(t\right)|}{d}}=\sqrt{\mu \lambda }$, and $\gamma \left(t\right)=inf({\lambda }_0\left(t\right)+{\mu }_1\left(t\right),{\lambda }_1\left(t\right)+{\mu }_2\left(t\right),\dots )=-(\mu +\lambda )$. The exact rate of convergence is $\gamma \left(t\right)-2\sqrt{KL}={(\sqrt{\mu }-\sqrt{\lambda })}^2$.

Example 6. 

Let us proceed to the second example in Section 4, in which the model of the server with breakdown and repair was considered. The infinitesimal matrix $A\left(t\right)$ remains unchanged. Let $r_0\left(t\right)=\lambda \left(t\right)+{\mu }_1\left(t\right)$ and $r_i\left(t\right)=0$ for $i\ge 1$. Then, the matrix $\tilde{A}\left(t\right)=D\left(A\left(t\right)-({\overrightarrow{1}}^T\otimes \overrightarrow{r}\left(t\right))\right)D^{-1}$ will have the block structure:

$$\left(\begin{array}{ccccc}-\lambda -\mu & 0& 0& 0& \dots \\ a& -b-a-\lambda & {\displaystyle \frac{b-\mu }{d}}& {\displaystyle \frac{\mu }{d}}& \dots \\ d\lambda & 0& -\lambda -\mu & \mu & 0& 0& \dots \\ 0& d\lambda & a& -b-a-\lambda & {\displaystyle \frac{b-\mu }{d}}& {\displaystyle \frac{\mu }{d}}& \dots \\ & & d\lambda & 0& -\lambda -\mu & \mu & 0& \dots \\ & & 0& d\lambda & a& -b-a-\lambda & {\displaystyle \frac{b-\mu }{d}}& \dots \\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& \dots \end{array}\right),$$

wherefrom it follows that $L={\displaystyle \frac{1}{d}}{sup}_{t}max(\mu \left(t\right),|b-\mu \left(t\right)\left|\right)$ and $K=d{sup}_t\left|\lambda \left(t\right)\right|$. - Since it must hold that $K<L\le dL$, the feasible range for d is $1\le d<\sqrt{{\displaystyle \frac{max(\mu ,|b-\mu \left|\right)}{\lambda }}}$.

Let us bound $\parallel U_0(t,0)\parallel$, where $U_0(t,0)$ is the Cauchy operator for the ODE system ${\displaystyle \frac{d}{dt}}{\mathbf{x}}_0\left(t\right)={\mathfrak{B}}_0\left(t\right){\mathbf{x}}_0\left(t\right)$ and

$$\begin{array}{c}\hfill {\mathfrak{B}}_0=\left(\begin{array}{cc}-\lambda -\mu & 0\\ a& -b-a-\lambda \end{array}\right).\end{array}$$

(25)

Put $D_0=\mathrm{diag}(1,k)$, $k>0$. Then, since

$$\begin{array}{c}\hfill D_0{\mathfrak{B}}_0{D}_0^{-1}=\left(\begin{array}{cc}-\lambda -\mu & 0\\ ka& -b-a-\lambda \end{array}\right),\end{array}$$

(26)

we have that

$$\begin{array}{c}\parallel U_0(t,0)\parallel \le \parallel D_0\parallel \parallel D_0^{-1}\parallel \parallel D_0{U}_0(t,0)D_0^{-1}\parallel \le \hfill \\ \le {\displaystyle \frac{1}{k}}exp\left(-\underset{0}{\overset{t}{\int }}min\left(\lambda \left(s\right)+\mu \left(s\right)-ka,b+a+\lambda \left(s\right)\right)ds\right)\le \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}exp\left(\underset{0}{\overset{t}{\int }}{\displaystyle \frac{1}{t_0}}\left|ln{\displaystyle \frac{1}{k}}\right|-min\left(\lambda \left(s\right)+\mu \left(s\right)-ka,b+a+\lambda \left(s\right)\right)ds\right).\end{array}$$

Using the same reasoning for the ith main diagonal block ${\mathfrak{B}}_i\left(t\right)$ of $\tilde{A}\left(t\right)$ with $D_i=\mathrm{diag}(1,k)$, $k>0$, one obtains that

$$\begin{array}{c}\hfill D_i{\mathfrak{B}}_i{D}_i^{-1}=\left(\begin{array}{cc}-\lambda -\mu & {\displaystyle \frac{\mu }{k}}\\ ka& -b-a-\lambda \end{array}\right)\end{array}$$

(27)

and, for arbitrary $t>t_0>0$,

$$\begin{array}{c}\parallel U_i(t,0)\parallel \le \parallel D_i\parallel \parallel D_i^{-1}\parallel \parallel D_i{U}_i(t,0)D_i^{-1}\parallel \le \hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\le {\displaystyle \frac{1}{k}}exp\left(-\underset{0}{\overset{t}{\int }}min\left(\lambda \left(s\right)+\mu \left(s\right)-ka,b+a+\lambda \left(s\right)-{\displaystyle \frac{\mu \left(s\right)}{k}}\right)ds\right)\le \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\le exp\left(\underset{0}{\overset{t}{\int }}{\displaystyle \frac{1}{t_0}}\left|ln{\displaystyle \frac{1}{k}}\right|-min\left(\lambda \left(s\right)+\mu \left(s\right)-ka,b+a+\lambda \left(s\right)-{\displaystyle \frac{\mu \left(s\right)}{k}}\right)ds\right).\end{array}$$

Therefore, the logarithmic norm $\gamma \left(s\right)$ is

$$\begin{array}{c}\hfill \gamma \left(s\right)=-{\displaystyle \frac{1}{t_0}}\left|ln{\displaystyle \frac{1}{k}}\right|+min\left(\lambda \left(s\right)+\mu \left(s\right)-ka,b+a+\lambda \left(s\right)-{\displaystyle \frac{\mu \left(s\right)}{k}}\right).\end{array}$$

(28)

and thus the rate of convergence is equal to

$$\gamma \left(t\right)-2\sqrt{KL}=-{\displaystyle \frac{1}{t_0}}\left|ln{\displaystyle \frac{1}{k}}\right|+min\left(\lambda \left(t\right)+\mu \left(t\right)-ka,b+a+\lambda \left(t\right)-{\displaystyle \frac{\mu \left(t\right)}{k}}\right)-2\sqrt{KL}.$$

In the homogeneous case, we have that $k=\sqrt{a}/\sqrt{\mu }$, and one has the following bound for the rate:

$$\gamma \left(t\right)-2\sqrt{KL}=\left\{\begin{array}{cc}-{\displaystyle \frac{1}{t_0}}\left|ln{\displaystyle \frac{\sqrt{\mu }}{\sqrt{a}}}\right|+\lambda +b-\sqrt{a\mu }-2\sqrt{\mu \lambda },\hfill & \mu >b,\mu >\lambda ,\hfill \\ -{\displaystyle \frac{1}{t_0}}\left|ln{\displaystyle \frac{\sqrt{\mu }}{\sqrt{a}}}\right|+\lambda +\mu -\sqrt{a\mu }-2\sqrt{\mu \lambda },\hfill & 2\mu >b>\mu ,2\mu >\lambda ,\hfill \\ -{\displaystyle \frac{1}{t_0}}\left|ln{\displaystyle \frac{\sqrt{\mu }}{\sqrt{a}}}\right|+\lambda +\mu -\sqrt{a\mu }-2\sqrt{(b-\mu )\lambda },\hfill & b>2\mu ,b-\mu >\lambda .\hfill \end{array}\right.$$

Example 7. 

Along the same lines, one can obtain the upper weak ergodicity bounds in the last example of Section 4, i.e., for the tandem queue $M_t/M_t/1/0\to ·/M_t/1$. The infinitesimal matrix $A\left(t\right)$, of course, remains unchanged. Let $r_0\left(t\right)=\lambda \left(t\right)$ and $r_i\left(t\right)=0$ for $i\ge 1$. Then, the matrix $\tilde{A}\left(t\right)=D\left(A\left(t\right)-({\overrightarrow{1}}^T\otimes \overrightarrow{r}\left(t\right))\right)D^{-1}$ will have the block structure:

$$\begin{array}{c}\hfill \left(\begin{array}{ccccc}-\lambda & 0& {\mu }_2& 0& \dots \\ \lambda & -\lambda & -{\mu }_2& {\mu }_2& \dots \\ 0& {\mu }_1& -{\mu }_1-{\mu }_2& 0& {\mu }_2& 0& \dots \\ 0& 0& \lambda & -\lambda -{\mu }_2& -{\mu }_2& {\mu }_2& \dots \\ & & 0& {\mu }_1& -{\mu }_1-{\mu }_2& 0& {\mu }_2& 0& \dots \\ & & 0& 0& \lambda & -\lambda -{\mu }_2& -{\mu }_2& {\mu }_2& \dots \\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& \dots \end{array}\right).\end{array}$$

(29)

Let us bound $\parallel U_0(t,0)\parallel$, where $U_0(t,0)$ is the Cauchy operator for the ODE system ${\displaystyle \frac{d}{dt}}{\mathbf{x}}_0\left(t\right)={\mathfrak{B}}_0\left(t\right){\mathbf{x}}_0\left(t\right)$ and

$$\begin{array}{c}\hfill {\mathfrak{B}}_0=\left(\begin{array}{cc}-\lambda & 0\\ \lambda & -\lambda \end{array}\right)\end{array}$$

(30)

Put $D_0=\mathrm{diag}(1,k)$, $k>0$. Then,

$$\begin{array}{c}\hfill D_0{\mathfrak{B}}_0{D}_0^{-1}=\left(\begin{array}{cc}-\lambda & 0\\ k\lambda & -\lambda \end{array}\right)\end{array}$$

(31)

and $\parallel U_0(t,0)\parallel$ is upper bounded by

$$\parallel U_0(t,0)\parallel \le \parallel D_0\parallel \parallel D_0^{-1}\parallel \parallel D_0{U}_0(t,0)D_0^{-1}\parallel \le {\displaystyle \frac{1}{k}}exp\left(-\underset{0}{\overset{t}{\int }}\lambda \left(s\right)\left(1-k\right)ds\right).$$

For the next main diagonal block ${\mathfrak{B}}_1\left(t\right)$ of $\tilde{A}\left(t\right)$

$$\begin{array}{c}\hfill {\mathfrak{B}}_1=\left(\begin{array}{cc}-{\mu }_1-{\mu }_2& 0\\ \lambda & -\lambda -{\mu }_2\end{array}\right)\end{array}$$

(32)

with $D_1=\mathrm{diag}(1,k)$, $k>0$, one obtains

$$\begin{array}{c}\hfill D_1{\mathfrak{B}}_1{D}_0^{-1}=\left(\begin{array}{cc}-{\mu }_1-{\mu }_2& 0\\ k\lambda & -\lambda -{\mu }_2\end{array}\right),\end{array}$$

(33)

and the upper bound

$$\begin{array}{c}\parallel U_1(t,0)\parallel \le \parallel D_1\parallel \parallel D_1^{-1}\parallel \parallel D_1{U}_1(t,0)D_1^{-1}\parallel \le \hfill \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\le {\displaystyle \frac{1}{k}}exp\left(-\underset{0}{\overset{t}{\int }}min\left({\mu }_1\left(s\right)+{\mu }_2\left(s\right)-k\lambda \left(s\right),\lambda \left(s\right)+{\mu }_2\left(s\right)\right)ds\right).\end{array}$$

By considering the other main diagonal blocks of $\tilde{A}\left(t\right)$ in the same manner, one eventually arrives at the single upper bound for the Cauchy operator:

$$\parallel U(t,0)\parallel \le \underset{i}{inf}\parallel U_i(t,0)\parallel \le {\displaystyle \frac{1}{k}}exp\left(-\underset{0}{\overset{t}{\int }}min\left({\mu }_1\left(s\right)+{\mu }_2\left(s\right)-k\lambda \left(s\right),\lambda \left(s\right)\left(1-k\right)\right)ds\right),$$

wherefrom it follows that $L=2{sup}_t\left|{\mu }_2\left(t\right)\right|$ and $K={sup}_t\left|{\mu }_1\left(t\right)\right|$, and $\gamma \left(t\right)=-{\displaystyle \frac{1}{t_0}}\left|ln{\displaystyle \frac{1}{k}}\right|+min\left({\mu }_1\left(t\right)+{\mu }_2\left(t\right)-k\lambda \left(t\right),\lambda \left(t\right)\left(1-k\right)\right)$. Since it must hold that $K<L\le dL$, one must make sure that the inequality ${sup}_t|{\mu }_1\left(t\right)|<2{sup}_t\left|{\mu }_2\left(t\right)\right|$ is not violated for any t. In the homogeneous case, the rate of convergence is simply $\gamma \left(t\right)-2\sqrt{KL}=-{\displaystyle \frac{1}{t_0}}\left|ln{\displaystyle \frac{1}{k}}\right|+min(\lambda ,{\mu }_1+{\mu }_2)-k\lambda -2\sqrt{2{\mu }_1{\mu }_2}$.

The obtained bound does not work for any intensities’ values and is particularly useful for $\lambda \left(t\right)\gg 1$. For example, in the homogeneous case, the bound holds if ${\mu }_1<(3-2\sqrt{2}){\mu }_2$. However, the exact ergodicity condition, as was mentioned in the previous section, is ${\mu }_1^2<{\mu }_2(\lambda +{\mu }_1)$. This implies that for $\lambda \gg 1$, it is sufficient for stability to have ${\mu }_1<{\mu }_2$.

As an illustration, let us consider the following numerical example. Put

$${\mu }_1=1,{\mu }_2=7,\lambda \left(t\right)=10+cos\left(2\pi t\right).$$

Then, $L=2{sup}_t\left|{\mu }_2\left(t\right)\right|=14$, $K={sup}_t\left|{\mu }_1\left(t\right)\right|=1$, and

$$\begin{array}{c}\gamma \left(t\right)-2\sqrt{KL}=-{\displaystyle \frac{1}{t_0}}\left|ln{\displaystyle \frac{1}{k}}\right|+min\left({\mu }_1\left(t\right)+{\mu }_2\left(t\right)-k\lambda \left(t\right),\lambda \left(t\right)\left(1-k\right)\right)=\hfill \\ \hfill =-{\displaystyle \frac{1}{t_0}}\left|ln{\displaystyle \frac{1}{k}}\right|+8-k(10+cos\left(2\pi t\right))-2\sqrt{14}.\end{array}$$

If one puts $k=1/1000$ and $t_0=70$, then $\gamma \left(t\right)-2\sqrt{KL}\ge 0.4$. Therefore, according to Theorem 2, we have the following ergodicity bound:

$$\begin{array}{c}\hfill \parallel {\mathbf{p}}^{\ast \ast }\left(t\right)-{\mathbf{p}}^{\ast }\left(t\right)\parallel \le exp(-0.4t){\displaystyle \frac{\sqrt{14}}{\sqrt{14}-1}}{\left(14\right)}^{{\displaystyle \frac{j}{2}}}\parallel DT({\mathbf{p}}^{\ast \ast }\left(0\right)-{\mathbf{p}}^{\ast }\left(0\right))\parallel .\end{array}$$

This bound also holds if the total number of customers in the model is finite. For the sake of convenience, assume that the maximum queue size is 48. Consider two different initial conditions for the model, ${\mathbf{p}}^{\ast }\left(0\right))=0$ and ${\mathbf{p}}^{\ast \ast }\left(0\right)=49$, i.e., the model either starts empty or full. Therefore, $\parallel DT({\mathbf{p}}^{\ast \ast }\left(0\right)-{\mathbf{p}}^{\ast }\left(0\right))\parallel =50$, and the ergodicity bound is

$$\begin{array}{c}\hfill \parallel {\mathbf{p}}^{\ast \ast }\left(t\right)-{\mathbf{p}}^{\ast }\left(t\right)\parallel \le 4exp(-0.4t){10}^{15}.\end{array}$$

For $t>120$, we have that $\parallel {\mathbf{p}}^{\ast \ast }\left(t\right)-{\mathbf{p}}^{\ast }\left(t\right)\parallel \le {10}^{-5}$. The plots of the probabilities $p_0\left(t\right)$ and $p_1\left(t\right)$ as well as the average queue-size ${\sum }_{i=0}^{49}i{p}_i\left(t\right)$ are given in Figure 1, Figure 2, Figure 3, Figure 4, Figure 5, Figure 6, Figure 7, Figure 8 and Figure 9.

As can be seen, the plots are drawn for three different time periods, $[0,20]$, $[0,120]$, and $[120,121]$, wherefrom it can be seen how the the influence of the initial conditions vanishes.

6. Conclusions

As in the finite case, presented in [14], the obtained convergence bounds for the countable case are not tight but, in general, are sufficient to build truncation bounds. They can cover quite a number of countable QBDs (and allow for various variants of partitions of infinitesimal matrices, and not only those which are chosen in this paper), and therefore the numerical solutions of systems of ODEs become feasible. One of the promising directions for further research is the relaxation of the assumptions made to obtain the upper bound (11) and establishing the convergence conditions of (10) in the case when the state space is the whole set of integers.