On the Rate of Convergence for a Characteristic of Multidimensional Birth-Death Process

Abstract

We consider a multidimensional inhomogeneous birth-death process. In this paper, a general situation is studied in which the intensity of birth and death for each coordinate (“each type of particle”) depends on the state vector of the whole process. A one-dimensional projection of this process on one of the coordinate axes is considered. In this case, a non-Markov process is obtained, in which the transitions to neighboring states are possible in small periods of time. For this one-dimensional process, by modifying the method previously developed by the authors of the note, estimates of the rate of convergence in weakly ergodic and null-ergodic cases are obtained. The simplest example of a two-dimensional process of this type is considered.

1. Introduction and Preliminaries

Multidimensional birth-death processes (BDP) were objects of a number of studies in queueing theory and other applied fields. The authors of these papers studied different special classes of homogeneous multidimensional BDPs under some restrictions and considered fluid approximations [1], simulations [2,3,4,5,6], large deviations [7], stability [8,9], and other features. The problem of the product from solutions for such models was considered, for instance, in [10,11] (also, see the references therein). If the process is inhomogeneous and the transition intensities have a more general form, then the problem of computation of any probabilistic characteristics of the queueing model is much more difficult.

In the general case, it is impossible to obtain explicit solutions and their characteristics, as well as to construct any significant characteristics of the processes, as can be seen from the above list of works. This paper fills this gap and proposes a method of research and evaluation allowing one to estimate the rate of convergence for a one-dimensional projection of the multidimensional birth-death process. The approach also makes it possible to evaluate the main characteristics of the projection, as is demonstrated by the simplest example of an inhomogeneous two-dimensional process.

The background of our approach is the method of investigation of inhomogeneous BDP, see the detailed discussion and some preliminary results in [12,13,14,15]. Estimates for the state probabilities of one-dimensional projections of a multidimensional BDP were studied in [16,17]. However, within that methodology, it was impossible to obtain estimates of the rate of convergence, since the logarithmic norm of the operator cannot be applied to the corresponding nonlinear systems.

Here, we substantially modify that approach so that it can be used for estimation and construction of some explicit bounds on the rate of convergence for one-dimensional projection of a multidimensional BDP. Namely, in Section 2, we develop a simple but efficient method for bounding the rate of convergence for an arbitrary (which may be nonlinear, depending on the number of parameters and so on) differential equation in the space of sequences $l_1$, and in Section 3, we apply this method to bounding the rate of convergence for one-dimensional projections of BDP.

Let $\mathbf{X}\left(t\right)=(X_1\left(t\right),\dots ,X_d\left(t\right))$ be a d-dimensional BDP such that in the interval $(t,t+h)$, the following transitions are possible with order h: birth of a particle of type j, death of a particle of type j.

Let ${\lambda }_{j,\mathbf{m}}\left(t\right)$ be the corresponding birth rate (from the state $\mathbf{m}=(m_1,\dots ,m_d)={\sum }_{i=1}^d{m}_i{\mathbf{e}}_i$ to the state $\mathbf{m}+{\mathbf{e}}_j$) and let ${\mu }_{j,\mathbf{m}}\left(t\right)$ be the corresponding death intensity (from the state $\mathbf{m}=(m_1,\dots ,m_d)={\sum }_{i=1}^d{m}_i{\mathbf{e}}_i$ to the state ${\mathbf{m}-\mathbf{e}}_j$). Denote $p_{\mathbf{m}}\left(t\right)=Pr\left(\mathbf{X}\left(t\right)=\mathbf{m}\right)$.

To consider the existence and uniqueness, we renumber the states (only in this section), transforming the process into a one-dimensional one. Now, let the (finite or countable) state space of the vector process under consideration be arranged in a special order, say $0,1,\cdots$. Denote by $p_i\left(t\right)$, the corresponding state probabilities, and by $\mathbf{p}\left(t\right)$, the corresponding column vector of state probabilities. Applying our standard approach (see details in [12,14,15]), we suppose in addition that all intensities are nonnegative functions locally integrable on $[0,\infty )$, and, moreover, in new enumeration,

$$Pr\left(X(t+h)=j/X\left(t\right)=i\right)=\left\{\begin{array}{cc}{q}_{ij}\left(t\right)h+{\alpha }_{ij}(t,h),\hfill & j\ne i,\hfill \\ 1-{\displaystyle \sum _{k\ne i}}{q}_{ik}\left(t\right)h+{\alpha }_i(t,h),\hfill & j=i,\hfill \end{array}\right.$$

(1)

where $q_{ij}\left(t\right)$ are the corresponding transition intensities and all ${\alpha }_i(t,h)$ are $o\left(h\right)$ uniformly in i, that is, ${lim}_{h\to 0}\frac{1}{h}{sup}_i|{\alpha }_i(t,h)|=0$, for any $t\ge 0$.

We suppose that ${\lambda }_{j,\mathbf{m}}(t)\le L<\infty ,{\mu }_{j,\mathbf{m}}(t)\le M<\infty ,$ for any j, $\mathbf{m}$ and almost all $t\ge 0$.

The probabilistic dynamics of the process is represented by the forward Kolmogorov system:

$$\frac{d\mathbf{p}}{dt}=A(t)\mathbf{p}(t),$$

(2)

where $A(t)$ is the corresponding infinitesimal (intensity) matrix.

Throughout the paper, we denote the $l_1$-norm by $\parallel ·\parallel$, i.e., $\parallel \mathbf{x}\parallel =\sum |x_i|$, and $\parallel B\parallel ={sup}_j{\sum }_i|b_{ij}|$ for $B={(b_{ij})}_{i,j=0}^{\infty }$.

Let $\Omega$ be the set all stochastic vectors, i.e., $l_1$-vectors with nonnegative coordinates and unit norm. We have the inequality $\parallel A(t)\parallel \le 2d\left(L+M\right)<\infty ,$ for almost all $t\ge 0$. Hence, the operator function $A(t)$ from $l_1$ into itself is bounded for almost all $t\ge 0$ and is locally integrable on $[0;\infty )$. Therefore, we can consider (2) as a differential equation in the space $l_1$ with bounded operator.

It is well known, see [18], that the Cauchy problem for differential Equation (2) has unique solution for an arbitrary initial condition, and $\mathbf{p}(s)\in \Omega$ implies $\mathbf{p}(t)\in \Omega$ for $t\ge s\ge 0$.

We recall that a Markov chain $X(t)$ is called null-ergodic, if all $p_i(t)\to 0$ as $t\to \infty$ for any initial condition, and it is called weakly ergodic, if $\parallel {\mathbf{p}}^{*}(t)-{\mathbf{p}}^{**}(t)\parallel \to 0$ as $t\to \infty$ for any initial condition ${\mathbf{p}}^{*}(0),{\mathbf{p}}^{**}(0)$, see for instance [12,14].

2. Bounds on the Rate of Convergence for a Differential Equation

Consider a general (linear or nonlinear) differential equation

$$\frac{d\mathbf{y}}{dt}=H\mathbf{y}(t),$$

(3)

in the space of sequences $l_1$ under the assumption of existence and uniqueness of a solution for any initial condition $\mathbf{y}(0)$.

Let $H=(h_{ij})$, where all $h_{ij}$ depend on some parameters (for instance, on y, $t,\dots$).

We have

$$\frac{d{y}_i}{dt}=h_{ii}{y}_i+\sum _{j\ne i}{h}_{ij}{y}_j.$$

Now, if $y_i>0$, then

$$\frac{d|y_i|}{dt}=\frac{d{y}_i}{dt}=h_{ii}|y_i|+\sum _{j\ne i}{h}_{ij}{y}_j\le h_{ii}|y_i|+\sum _{j\ne i}|h_{ij}||y_j|,$$

and if $y_i<0$, then we also have

$$\frac{d|y_i|}{dt}=-\frac{d{y}_i}{dt}=-h_{ii}{y}_i-\sum _{j\ne i}{h}_{ij}{y}_j\le h_{ii}|y_i|+\sum _{j\ne i}|h_{ij}||y_j|.$$

Finally, using the continuity of all coordinates of the solution and the absolute convergence of all series, we obtain the estimate

$$\begin{array}{c}\hfill \frac{d\parallel y\parallel }{dt}=\sum _i\frac{d|y_i|}{dt}\le \sum _i\left(h_{ii}|y_i|+\sum _{j\ne i}|h_{ij}||y_j|\right)\le {\beta }^{*}\parallel y\parallel ,\end{array}$$

(4)

where

$${\beta }^{*}=\underset{i}{sup}\left(h_{ii}+\sum _{j\ne i}|h_{ji}|\right).$$

(5)

Remark 1.

One can see that inequality (4) implies the bound

$$\parallel y(t)\parallel \le e^{{\int }_0^t{\beta }^{*}du}\parallel y(0)\parallel .$$

(6)

Moreover, if H is bounded for any t linear operator function from $l_1$ to itself, then ${\beta }^{*}(t)=\gamma (H(t))$ is the corresponding logarithmic norm of $H(t)$, see [12,13,14,15].

On the other hand, in a nonlinear situation, ${\beta }^{*}(t)$ yields a generalization of this notion.

3. Bounds on the Rate of Convergence for a Projection of Multidimensional BDP

Again, consider the forward Kolmogorov system (2) in the original vector form. Then, we have

$$\begin{array}{c}\hfill \frac{d{p}_{\mathbf{m}}}{dt}=\sum _l{\lambda }_{l,\mathbf{m}-{\mathbf{e}}_{\mathbf{l}}}(t)p_{\mathbf{m}-{\mathbf{e}}_{\mathbf{l}}}+\\ \hfill \sum _l{\mu }_{l,\mathbf{m}+{\mathbf{e}}_{\mathbf{l}}}(t)p_{\mathbf{m}+{\mathbf{e}}_{\mathbf{l}}}-\sum _l\left({\lambda }_{l,\mathbf{m}}+{\mu }_{l,\mathbf{m}}\right)(t)p_{\mathbf{m}},\end{array}$$

(7)

for any $\mathbf{m}$.

In this section, we consider the one-dimensional process $X_j(t)$ for a fixed j. Denote $x_k(t)=Pr\left(X_j(t)=k\right)$. Then, $x_k(t)={\sum }_{\mathbf{m},m_j=k}{p}_{\mathbf{m}}(t)$. The process $X_j(t)$ has nonzero jump rates only for unit jumps ($\pm 1$), namely, if $X_j(t)=k$, then for small positive h only the jumps $X_j(t+h)=k\pm 1$ are possible with positive intensities, say ${\tilde{\lambda }}_k$ and ${\tilde{\mu }}_k$, respectively. Moreover, (7) implies the equalities

$${\tilde{\lambda }}_k{x}_k(t)=\sum _{\mathbf{m},m_j=k}{\lambda }_{j,\mathbf{m}}(t)p_{\mathbf{m}}(t),$$

(8)

$${\tilde{\mu }}_k{x}_k(t)=\sum _{\mathbf{m},m_j=k}{\mu }_{j,\mathbf{m}}(t)p_{\mathbf{m}}(t),$$

(9)

and hence

$${\tilde{\lambda }}_k=\frac{{\sum }_{\mathbf{m},m_j=k}{\lambda }_{j,\mathbf{m}}(t)p_{\mathbf{m}}(t)}{{\sum }_{\mathbf{m},m_j=k}{p}_{\mathbf{m}}(t)},$$

(10)

and

$${\tilde{\mu }}_k=\frac{{\sum }_{\mathbf{m},m_j=k}{\mu }_{j,\mathbf{m}}(t)p_{\mathbf{m}}(t)}{{\sum }_{\mathbf{m},m_j=k}{p}_{\mathbf{m}(t)}}.$$

(11)

Then, $X_j(t)$ is a (in general, non-Markovian) birth and death process with birth and death intensities ${\tilde{\lambda }}_k$ and ${\tilde{\mu }}_k$, respectively, (that is, it is a process with possible infinitesimal jumps $\pm 1$, the intensities of which depend on t and on the initial condition for the original multidimensional process $X(t)$.)

For any fixed initial distribution $\mathbf{p}(0)$ and any $t>0$, the probability distribution $\mathbf{p}(t)$ is unique. Hence, ${\tilde{\lambda }}_k={\lambda }_k\left(\mathbf{p}(0),t\right)$ and ${\tilde{\mu }}_k={\mu }_k\left(\mathbf{p}(0),t\right)$ uniquely define the system

$$\frac{d\mathbf{x}}{dt}=\tilde{A}\mathbf{x}(t),$$

(12)

for the vector $\mathbf{x}(t)$ of state probabilities of the projection $X_j(t)$ under the given initial condition. Obviously, different initial conditions specify different systems.

Here, $\tilde{A}$ is the corresponding three-diagonal “birth-death” transposed intensity matrix such that all off-diagonal elements are nonnegative and all column-wise sums are equal to zero.

Let for all $\mathbf{m}$ and any $t\ge 0$

$$l_j\le {\lambda }_{j,\mathbf{m}}(t)\le L_j,m_j\le {\mu }_{j,\mathbf{m}}(t)\le M_j.$$

(13)

Then, from (10) and (11), we obtain the two-sided bounds

$$l_j\le {\tilde{\lambda }}_k\le L_j,m_j\le {\tilde{\mu }}_k\le M_j,$$

(14)

for any k, any t, and any initial conditions.

1. Let the state space of $X_j(t)$ be countable and

$$M_j<l_j.$$

(15)

Put $\sigma =\sqrt{M_j/l_j}<1$, ${\delta }_n={\sigma }^n,n\ge 0,$ ${\tilde{x}}_n={\delta }_n{x}_n$, and $\tilde{\mathbf{x}}=\left({\tilde{x}}_0,{\tilde{x}}_1,\cdots \right)$. Let $\Lambda$ be a diagonal matrix, $\Lambda =diag\left({\delta }_0,{\delta }_1,\cdots \right)$.

Note that in this situation, $\parallel \tilde{\mathbf{x}}(t)\parallel ={\sum }_{i=0}^{\infty }{\delta }_k{x}_k(t)$, and $\parallel \tilde{\mathbf{x}}(t)\parallel \to 0$ as $t\to \infty$ implying null ergodicity of $X_j(t)$, that is $p_k(t)=Pr\left(X_j(t)=k\right)\to 0$ as $t\to \infty$ for any k.

Then,

$$\frac{d\tilde{\mathbf{x}}}{dt}=\Lambda \tilde{A}{\Lambda }^{-1}\tilde{\mathbf{x}}(t).$$

(16)

Then, we have

$$\begin{array}{c}\hfill {\tilde{\lambda }}_k+{\tilde{\mu }}_k-\frac{{\delta }_{k+1}}{{\delta }_k}{\tilde{\lambda }}_k-\frac{{\delta }_{k-1}}{{\delta }_k}{\tilde{\mu }}_k\ge {\tilde{\lambda }}_k\left(1-\sigma \right)-{\tilde{\mu }}_k\left(1/\sigma -1\right)\ge \\ \hfill l_j\left(1-\sigma \right)-M_j\left(1/\sigma -1\right)={\left(\sqrt{l_j}-\sqrt{M_j}\right)}^2={\alpha }^{*},\end{array}$$

(17)

implying the estimate

$$\begin{array}{c}\hfill \frac{d\parallel \tilde{\mathbf{x}}\parallel }{dt}\le \underset{k}{sup}\left(\frac{{\delta }_{k+1}}{{\delta }_k}{\tilde{\lambda }}_k+\frac{{\delta }_{k-1}}{{\delta }_k}{\tilde{\mu }}_k-{\tilde{\lambda }}_k-{\tilde{\mu }}_k\right)\parallel \tilde{\mathbf{x}}\parallel =\\ \hfill -\underset{k}{inf}\left({\tilde{\lambda }}_k+{\tilde{\mu }}_k-\frac{{\delta }_{k+1}}{{\delta }_k}{\tilde{\lambda }}_k-\frac{{\delta }_{k-1}}{{\delta }_k}{\tilde{\mu }}_k\right)\parallel \tilde{\mathbf{x}}\parallel \le -{\alpha }^{*}\parallel \tilde{\mathbf{x}}\parallel ,\end{array}$$

(18)

and the following statement.

Theorem 1.

Let (15) hold for some j. Then, $X_j(t)$ is null-ergodic and the following bounds hold:

$$\parallel \tilde{\mathbf{x}}(t)\parallel \le e^{-{\alpha }^{*}t}\parallel \tilde{\mathbf{x}}(0)\parallel ,$$

(19)

and

$$Pr\left(X_j(t)\le n/X_j(0)=k\right)\le {\sigma }^{k-n}·e^{-{\alpha }^{*}t}.$$

(20)

Hence,

$$Pr\left(X_j(t)>n/X_j(0)=k\right)>1-{\sigma }^{k-n}·e^{-{\alpha }^{*}t},$$

(21)

and $Pr\left(X_j(t)>n/X_j(0)=k\right)\to 1$ as $t\to \infty ,$ for any $n,k$.

Remark 2.

It should be noted that the above requirements are imposed only on this one coordinate.

2. Let

$$L_j<m_j,{\alpha }_{*}=l_j+m_j-2\sqrt{L_j{M}_j}>0.$$

(22)

We have $\mathbf{x}(t)\in \Omega$ for any $t\ge 0$. Set $x_0(t)=1-{\sum }_{i\ge 1}{x}_i(t)$. Then, from (12), we obtain the system

$$\frac{d\mathbf{z}}{dt}=\tilde{B}\mathbf{z}+\tilde{\mathbf{f}},$$

(23)

where $\mathbf{z}={(x_1,x_2,\cdots )}^{\top }$, $\tilde{\mathbf{f}}={\left({\tilde{\lambda }}_0,0,0,\cdots \right)}^{\top }$, and the corresponding matrix $\tilde{B}={\left({\tilde{b}}_{ij}\right)}_{i,j=1}^{\infty }$, where ${\tilde{b}}_{ij}={\tilde{a}}_{ij}-{\tilde{a}}_{i0}$ for the corresponding elements of the matrix $\tilde{A}$.

For the solutions of system (23), the rate of convergence is determined by the system

$$\frac{d\mathbf{w}}{dt}=\tilde{B}\mathbf{w},$$

(24)

where all elements of $\tilde{B}$ depend on t and the initial condition of the original process.

Now, let $\beta =\sqrt{\frac{M_j}{L_j}}>1$ in accordance with (22). Let $d_{k+1}={\beta }^k$, $k\ge 0$. Denote by D, the upper triangular matrix

$$D=\left(\begin{array}{cccc}{d}_1& d_1& d_1& \cdots \\ 0& d_2& d_2& \cdots \\ 0& 0& d_3& \cdots \\ & \ddots & \ddots & \ddots \end{array}\right).$$

(25)

Let $\tilde{\mathbf{w}}=D\mathbf{w}$. Then, the following bound holds:

$$\begin{array}{c}\hfill \frac{d\parallel \tilde{\mathbf{w}}\parallel }{dt}\le \underset{i\ge 0}{sup}\left(\frac{d_{i+1}}{d_i}{\tilde{\lambda }}_{i+1}+\frac{d_{i-1}}{d_i}{\tilde{\mu }}_i-\left({\tilde{\lambda }}_i+{\tilde{\mu }}_{i+1}\right)\right)\parallel \tilde{\mathbf{w}}\parallel =\\ \hfill -\underset{i\ge 0}{inf}\left(\left({\tilde{\lambda }}_i+{\tilde{\mu }}_{i+1}-\beta {\tilde{\lambda }}_{i+1}-{\tilde{\mu }}_i/\beta \right)\right)\parallel \tilde{\mathbf{w}}\parallel \le -{\alpha }_{*}\parallel \tilde{\mathbf{w}}\parallel .\end{array}$$

(26)

Note that $\parallel \tilde{\mathbf{w}}\parallel =\parallel D\mathbf{w}\parallel \ge \frac{1}{2}\parallel \mathbf{w}\parallel$, see detailed discussion in [15], therefore, if $\parallel \tilde{\mathbf{w}}(t)\parallel \to 0$ as $t\to \infty$, then $X_j(t)$ is weakly ergodic.

Thus, we obtain the following statement.

Theorem 2.

Let (22) hold for some j. Then, $X_j(t)$ is weakly ergodic and the following bound holds:

$$\parallel D\mathbf{w}(t)\parallel \le e^{-{\alpha }_{*}t}\parallel D\mathbf{w}(0)\parallel ,$$

(27)

for any $t\ge 0$ and any corresponding initial conditions.

4. Example

Consider a simple two-dimensional BDP with finite state space $\{i,j\}$, $0\le i\le 10$, $0\le j\le 10$ and the following transition intensities:

(i)

${\lambda }_{1,i,0}(t)=\frac{i+1}{11}{\lambda }_1(t)$ from $(i,0)$ to $(i+1,0)$;

(ii)

${\lambda }_{1,i,j}(t)={\lambda }_1(t)$ from $(i,j)$ to $(i+1,j)$ if $j\ne 0$;

(iii)

${\lambda }_{2,i,j}(t)={\lambda }_2(t)$ from $(i,j)$ to $(i+1,j)$;

(iv)

${\mu }_{1,i,j}(t)={\mu }_1(t)$ from $(i,j)$ to $(i-1,j)$;

(v)

${\mu }_{2,i,j}(t)={\mu }_2(t)$ from $(i,j)$ to $(i,j-1)$;

where ${\lambda }_1(t)=1+cos(2\pi t)$, ${\lambda }_2(t)=5+sin(2\pi t)$, ${\mu }_1(t)=11+sin(2\pi t)$, ${\mu }_2=3$.

Then, $\beta =\sqrt{\frac{M_1}{L_1}}=\sqrt{6}$, and Theorem 2 gives bound (27) with ${\alpha }_{*}=10-4\sqrt{6}$.

We computed some important characteristics for the original process and its projection $X_1(t)$, namely:

Figure 1, Figure 2 and Figure 3 show the behaviour of the state probabilities for $X_1(t)$, namely $Pr(X_1(t)=0)$, $Pr(X_1(t)=1)$, and $Pr(X_1(t)=2)$ under two initial conditions for the original BDP:

Figure 1. $Pr(X_1(t)=0)$ under initial conditions (i) and (ii).

Figure 2. $Pr(X_1(t)=1)$ under initial conditions (i) and (ii).

Figure 3. $Pr(X_1(t)=2)$ under initial conditions (i) and (ii).

(i)

$p_{i,j}(0)=\frac{1}{121}$, for any $i,j$ (blue); and

(ii)

$p_{0,0}(0)=\frac{109}{121}$, $p_{i,j}(0)=\frac{1}{1210}$, for any $i,j$ such that $i+j>0$ (green).

Note that the corresponding initial conditions for the projection are $\mathbf{x}(0)={\left(\frac{1}{11},\cdots ,\frac{1}{11}\right)}^T$, and $\mathbf{x}(0)={\left(10/11,1/110,\cdots ,1/110\right)}^T$.

These Figures illustrate the rate of convergence in a weak ergodic situation.

Figure 4 and Figure 5 show the ’birth intensities’ ${\tilde{\lambda }}_0$ and ${\tilde{\lambda }}_1$ for $X_1(t)$ under the same initial conditions.

Figure 4. ${\tilde{\lambda }}_0$ under initial conditions (i) and (ii).

Figure 5. ${\tilde{\lambda }}_1$ under initial conditions (i) and (ii).

Note that all the quantities are found by numerically solving the Cauchy problem for the forward Kolmogorov system (2) and the corresponding system (12) for its projection on the corresponding interval.

As can be seen from Theorem 2 and the figures below, to construct all the characteristics of interest with good accuracy, it suffices to carry out the numerical solution on the interval $[0,5]$.

As was already noted, the projection of the original process is not a Markov process, and all probabilistic characteristics depend on the initial conditions of the original process.

It can be seen that these characteristics present comprehensive information concerning the behavior of the projection of the original process.

5. Conclusions

In the paper, some estimates of the rate of convergence were discussed for one-dimensional projections of multidimensional inhomogeneous birth and death processes. Some specific queueing models were considered. The applied approach allows one to use an analogue of the logarithmic norm of an operator function for a nonlinear system of differential equations, as was shown in Section 2. In addition, similar estimates can be obtained for other one-dimensional processes related to the original one. For example, the total number of “particles” of all types can be studied. Moreover, it is possible to study multidimensional processes with possible transformations of particles from one type to another. Such processes play a very important role in stochastic models of epidemics, see, for example, References [19,20] and the references therein.