Numerical Computation of Distributions in Finite-State Inhomogeneous Continuous Time Markov Chains, Based on Ergodicity Bounds and Piecewise Constant Approximation

Abstract

In this paper it is shown, that if a possibly inhomogeneous Markov chain with continuous time and finite state space is weakly ergodic and all the entries of its intensity matrix are locally integrable, then, using available results from the perturbation theory, its time-dependent probability characteristics can be approximately obtained from another Markov chain, having piecewise constant intensities and the same state space. The approximation error (the taxicab distance between the state probability distributions) is provided. It is shown how the Cauchy operator and the state probability distribution for an arbitrary initial condition can be calculated. The findings are illustrated with the numerical examples.

1. Introduction

Operations research possesses a great deal of applications for which continuous-time Markov chains (CTMC) with discrete state space serve as adequate mathematical models (see, for example, [1] (Section 5) and [2]). The scope of use of CTMC is broadened if one allows for inhomogeneity, i.e., if one allows some (or all) transition intensities to be non-random functions of time and depend on the state of the chain. A large body of works exists on the analysis of inhomogeneous CTMCs, which covers all the vital questions: existence, numerical solution, asymptotics, and approximations (for a systematic view see, for example, [1] and [3] (Sections 1 and 1.2)).

In this paper, we revisit the problem of the computation of the time-dependent probability distributions of inhomogeneous CTMCs. It is well-known that for such chains analytical solutions are usually not possible and one must resort to other approaches. There exist various techniques to calculate performance measures for CTMCs (See, for example, [4] (Introduction) and [5,6], for a short review related to queuing theory): point-wise stationary approximation [7], uniformization [8,9], heavy-traffic approximation [10,11], robust optimization [3] (Section 2), Monte Carlo simulation, stationary independent period by period approach, state-space enrichment, etc. Probably the most widely applicable technique (or at least popular currently due to the increasing computer power) is comprised of numerical methods for systems of ordinary differential equations (ODEs) [12,13,14,15,16]. In this paper the novel variation of the method for the computation of the Cauchy matrix is proposed for the case when the intensity matrix of a Markov chain is time–dependent. The main requirement for the method to work is that the Markov chain under consideration is weakly ergodic and the entries of its intensity matrix are locally integrable. If these conditions are satisfied then the (approximate) computation of the main characteristics of the chain can be brought down to the computation of those in another chain with the intensity matrix, having piecewise-constant entries. This method is, of course, not new and can be found already in [17]. The main contribution of this paper comes from the observation that the new chain can be considered as the perturbed version of the original chain. Therefore, the estimates for perturbed chains available from the literature can be used (see [18]). Thus, if the chains start from the same state, then the “difference” between them (in $l_1$-norm) for any time $t>0$ does not exceed the constant, obtained here for the first time in the literature (see (7)). Since exact solutions are rarely available, here we show what difference in the solutions can be expected (see (6)): the main contribution into the approximation error comes from approximate computation of the Cauchy operators (see (17)). These effects are illustrated in the numerical section. Here, a comparison is made with the well-known Runge–Kutta method: the numerical results show that the performance (with respect to the accuracy) is almost identical. Finally, the new approximation method can be executed in parallel and therefore can be efficiently used in a chain with a large number of states.

The paper is structured as follows. In the next section, the preliminary notation and the required definitions are given. Section 3 contains the main result of the paper, i.e., the bounds for the difference between the two chains, original with time-varying intensities and the new one with the piecewise constant intensities. Section 4 discusses the solution algorithm, which is illustrated in the Section 5 with three numerical examples. In the concluding section, a summary and further direction of research is given.

2. Preliminaries

Let $X\left(t\right)$, $t\ge 0$, be, in general, an inhomogeneous continuous-time Markov chain with a finite or countable state space $E_S=0,1,\dots ,S$, $S<\infty$. The transition probabilities for $X\left(t\right)$ will be denoted $p_{ij}(s,t)=Pr\left\{X\left(t\right)=j\left|X\left(s\right)=i\right.\right\}$, $i,j\ge 0,0\le s\le t$. Let $p_i\left(t\right)=Pr\left\{X\left(t\right)=i\right\}$ be the state probabilities of the chain and $\mathbf{p}\left(t\right)={\left(p_0\left(t\right),p_1\left(t\right),\dots \right)}^T$ be the corresponding vector of state probabilities. In what follows, it is assumed that

$$Pr\left\{X\left(t+h\right)=j|X\left(t\right)=i\right\}=\left\{\begin{array}{cc}{q}_{ij}\left(t\right)h+{\alpha }_{ij}\left(t,h\right),\hfill & \mathrm{i}\mathrm{f} j\ne i\hfill \\ 1-{\displaystyle \sum _{k\ne i}{q}_{ik}\left(t\right)h+{\alpha }_i\left(t,h\right)},\hfill & \mathrm{i}\mathrm{f} j=i,\hfill \end{array}\right.$$

(1)

where all ${\alpha }_i(t,h)$ are $o\left(h\right)$ uniformly in i, that is, ${sup}_i\left|{\alpha }_i(t,h)\right|=o\left(h\right)$. Moreover, if $X\left(t\right)$ is inhomogeneous, then all its infinitesimal characteristics (intensity functions) $q_{ij}\left(t\right)$ are integrable in t on any interval $[a,b]$, $0\le a\le b$.

Denote $a_{ij}\left(t\right)=q_{ji}\left(t\right)$ for $j\ne i$ and $a_{ii}\left(t\right)=-{\sum }_{j\ne i}{a}_{ji}\left(t\right)=-{\sum }_{j\ne i}{q}_{ij}\left(t\right)$. Further, in order to be able to obtain tighter estimates, its is assumed that

$$|a_{ii}\left(t\right)|\le L<\infty$$

(2)

for almost all $t\ge 0$. The state probabilities then satisfy the forward Kolmogorov system

$$\frac{d}{dt}\mathbf{p}\left(t\right)=A\left(t\right)\mathbf{p}\left(t\right),$$

(3)

where $A\left(t\right)=Q^T\left(t\right)$, and $Q\left(t\right)$ is the infinitesimal matrix of the process.

Let $\parallel ·\parallel$ be the usual $l_1$-norm, i.e., $\parallel \mathbf{x}\parallel =\sum |x_i|$, and $\parallel B\parallel ={sup}_j{\sum }_i\left|b_{ij}\right|$ for $B={\left(b_{ij}\right)}_{i,j=0}^{\infty }$. Denote $\mathrm{\Omega }=\left\{\mathbf{x}:\mathbf{x}\in l_1^{+}\&\parallel \mathbf{x}\parallel =1\right\}$. Therefore, $\parallel A\left(t\right)\parallel =2{sup}_k\left|a_{kk}\left(t\right)\right|\le 2L$ for almost all $t\ge 0$, and we can apply the results of [19] to Equation (3) in the space $l_1$. Namely, in [19] it was shown that the Cauchy problem for Equation (3) has a unique solution for an arbitrary initial condition. Moreover, if $\mathbf{p}\left(s\right)\in \mathrm{\Omega }$, then $\mathbf{p}\left(t\right)\in \mathrm{\Omega }$, for any $0\le s\le t$ and any initial condition $\mathbf{p}\left(s\right)$.

By $\overline{X}=\overline{X}\left(t\right)$, we will denote the “perturbed” Markov chain with the same state space, state probabilities ${\overline{p}}_i\left(t\right)$, transposed infinitesimal matrix $\overline{A}\left(t\right)={\left({\overline{a}}_{ij}\left(t\right)\right)}_{i,j=0}^{\infty }$ and the “perturbations” themselves, that is, the differences between the corresponding “perturbed” and original characteristics will be denoted by ${\widehat{a}}_{ij}\left(t\right)$, $\widehat{A}\left(t\right)$.

Let $E(t,k)=\mathbf{E}\left\{X\left(t\right)\left|X\left(0\right)=k\right.\right\}$ be the conditional mean value of $X\left(t\right)$ at an arbitrary instant t. Recall that a Markov chain $X\left(t\right)$ is weakly ergodic if $\parallel {\mathbf{p}}^{*}\left(t\right)-{\mathbf{p}}^{**}\left(t\right)\parallel \to 0$ as $t\to \infty$ for any initial condition, and it has the limiting mean $\varphi \left(t\right)$ if $\left|E\right(t,k)-\varphi (t\left)\right|\to 0$ as $t\to \infty$ for any k.

3. Estimation through Piecewise Constant Approximation

Consider the system of linear differential equations in the vector-matrix form (3), where all the elements $a_{ij}\left(t\right)$ of the matrix $A\left(t\right)$ are locally integrable non-random functions on $[0,\infty )$, i.e., $|{\int }_a^b{a}_{ij}\left(t\right)dt|<\infty$ for any $(a,b)\subset [0,\infty )$.

Let T be a real positive number and N be a positive integer. Put $h=T/N$ and denote $\overline{A}\left(t\right)=A\left(⌊\frac{N}{T}t⌋\frac{T}{N}\right)$, where $\lfloor x\rfloor$ returns the largest integer less than or equal to x. Therefore, $\overline{A}\left(t\right)$ is the transposed intensity matrix, consisting of piecewise constant locally integrable non-random functions on $[0,\infty )$, which satisfies the forward Kolmogorov system of equations

$$\frac{d}{dt}\overline{\mathbf{p}}\left(t\right)=\overline{A}\left(t\right)\overline{\mathbf{p}}\left(t\right).$$

(4)

The difference between the systems (3) and (4) can be described by [18] (Theorem 1), which repeat below without the proof. The basic idea behind this result is that if one knows the rate of convergence of the original chain $X\left(t\right)$, the estimate for the difference between the systems can be calculated as well.

 Theorem 1 

([18]). Let the Markov chain $X\left(t\right)$ be exponentially weakly ergodic; that is, for any initial conditions ${\mathbf{p}}^{*}\left(s\right)\in \mathrm{\Omega }$,${\mathbf{p}}^{**}\left(s\right)\in \mathrm{\Omega }$ and any $s\ge 0,t\ge s$, there holds the inequality

$$\parallel {\mathbf{p}}^{*}\left(t\right)-{\mathbf{p}}^{**}\left(t\right)\parallel \le 2c{e}^{-b(t-s)}.$$

(5)

Therefore, for the perturbations small enough ($\parallel A\left(t\right)-\overline{A}\left(t\right)\parallel \le \epsilon$ for almost all $t\ge 0$), the perturbed chain $\overline{X}\left(t\right)$ is also exponentially weakly ergodic, and the following perturbation bound takes place:

$$\parallel \mathbf{p}\left(t\right)-\overline{\mathbf{p}}\left(t\right)\parallel \le$$

$$\left\{\begin{array}{cc}\parallel \mathbf{p}\left(s\right)-\overline{\mathbf{p}}\left(s\right)\parallel +(t-s)\epsilon ,\hfill & if0<t-s<\frac{1}{b}ln\frac{c}{2},\hfill \\ \frac{c}{2}{e}^{-b(t-s)}\parallel \mathbf{p}\left(s\right)-\overline{\mathbf{p}}\left(s\right)\parallel +\frac{1}{b}(ln\frac{c}{2}+1-c{e}^{-b(t-s)})\epsilon ,\hfill & ift-s\ge \frac{1}{b}ln\frac{c}{2}\hfill \end{array}\right.$$

(6)

and

$$\underset{t\to \infty }{lim\; sup}\parallel \mathbf{p}\left(t\right)-\overline{\mathbf{p}}\left(t\right)\parallel \le \frac{(1+log(c/2\left)\right)\epsilon }{b}.$$

(7)

Thus, if the uniform grid is used, one can apply the Theorem 1 to estimate the difference between the state vectors of the original system and of the system with the piecewise constant intensities. One can also notice the following. For large perturbations in $A\left(t\right)$ (large $\epsilon$) and low rate of convergence b, the number of steps N must be large to compensate $\epsilon$ and decrease the right-hand side of (6). If one does not pay attention to such differences as (6), which always appear if the piecewise constant matrix replaces the original one, the error in the final result can be severely underestimated.

In what follows, we consider two ways to obtain the estimation of the state vector. One is through the discretization of the intensity matrix and exact solution of the system with constant coefficients. The other is through the discretization of the intensity matrix and the approximate calculation of the Cauchy operator.

4. Estimation of the State Probabilities

 Theorem 2. 

If conditions of the [18] (Theorem 1) are satisfied and $\mathbf{p}\left(0\right)=\overline{\mathbf{p}}\left(0\right)$, then the following bound holds:

$$\parallel \mathbf{p}\left(T\right)-\overline{\mathbf{p}}\left(T\right)\parallel \le \left\{\begin{array}{cc}T\epsilon ,\hfill & if0<T<\frac{1}{b}ln\frac{c}{2},\hfill \\ \frac{1}{b}(ln\frac{c}{2}+1-c{e}^{-bT})\epsilon ,\hfill & ifT\ge \frac{1}{b}ln\frac{c}{2}.\hfill \end{array}\right.$$

(8)

 Theorem 3. 

If conditions of the [18] (Theorem 1) are satisfied and $A\left(t\right)$ and $\overline{A}\left(t\right)$ are T-periodic, then the following bound for periodic limiting solutions of the corresponding systems (3) and (4) holds:

$$\underset{0\le t\le T}{sup}\parallel \mathbf{p}\left(t\right)-\overline{\mathbf{p}}\left(t\right)\parallel \le \frac{(1+log(c/2\left)\right)\epsilon }{b}.$$

(9)

Exact solutions of the system with the constant intensities can (in principle) be found using computer algebra systems. However, as it is demonstrated in Example 1, even in the simplest cases the solution may turn out to be too cumbersome. Therefore, in what follows we dwell on the direct numerical method for the solution of the system and provide the bound of its approximation error. Firstly, from (3) and (4) we have the relations

$$\mathbf{p}\left(t\right)=U(t,s)\mathbf{p}\left(s\right)$$

(10)

$$\overline{\mathbf{p}}\left(t\right)=\overline{U}(t,s)\overline{\mathbf{p}}\left(s\right)$$

(11)

which imply that

$$\overline{U}(h(k+1),hk)={\overline{U}}_n(h(k+1),hk)+G_k,$$

(12)

where $G_k$ is a matrix and ${\overline{U}}_n(t+h,t)=I+{\sum }_{k=1}^n\frac{{\left(\overline{A}\left(t\right)h\right)}^k}{k!}$. Note that the column-sum of a column in $U_n\left(t\right)$ is always equal to 1 unless n is small; in the latter case, negative values may appear. Therefore, during calculations a non-negativity check is required, otherwise $\parallel U_n\left(t\right)\parallel$ can become larger than one. Henceforth, we can assume that ${\overline{U}}_n(h(i+1),hi)\ge 0$ for $\forall i$ and thus $\parallel {\overline{U}}_n(h(i+1),hi)\parallel =1$.

Let us estimate the approximation error on $[0,T]$. By using the Cauchy operator’s property we have

$$\overline{U}(T,0)=\prod _{i=0}^{N-1}(\overline{U}(h(i+1),hi)=\prod _{i=0}^{N-1}({\overline{U}}_n(h(i+1),hi)+G_i),$$

(13)

or, after the expansion and collection of the common terms,

$$\overline{U}(T,0)=\prod _{i=0}^{N-1}{\overline{U}}_n(h(i+1),hi)+\sum _{i=0}^{N-1}{G}_i\prod _{k=0;k\ne i}^{N-1}{\overline{U}}_n(h(k+1),hk)+\cdots$$

(14)

If one denotes $G={max}_{0\le k\le N-1}\parallel G_k\parallel$, then it follows that

$$\parallel \overline{U}(T,0)-{\overline{U}}_n{(T,0)\parallel \le (1+G)}^N-1.$$

(15)

Therefore, the following theorem holds.

 Theorem 4. 

If conditions of the [18] (Theorem 1), are satisfied, then for sufficiently large n, such that ${\overline{U}}_n(h(i+1),hi)\ge 0,i=1,\cdots ,n$, and if $\mathbf{p}\left(0\right)=\overline{\mathbf{p}}\left(0\right)$, the following bound holds:

$$\parallel \mathbf{p}\left(T\right)-\overline{\mathbf{p}}{\left(T\right)\parallel \le (1+G)}^N-1+\left\{\begin{array}{cc}T\epsilon ,\hfill & if0<T<\frac{1}{b}ln\frac{c}{2},\hfill \\ \frac{1}{b}(ln\frac{c}{2}+1-c{e}^{-bT})\epsilon ,\hfill & ifT\ge \frac{1}{b}ln\frac{c}{2}.\hfill \end{array}\right.$$

(16)

Therefore, to construct the Cauchy operator it is sufficient to compute ${\overline{U}}_n(t+h,t)=I+{\sum }_{k=1}^n\frac{{\left(\overline{A}\left(t\right)h\right)}^k}{k!}$ and ${\overline{U}}_n(kh,0)={\displaystyle \prod _{i=0}^{k-1}}{\overline{U}}_n(h(i+1),hi)$. A specific solution is found by multiplying the Cauchy operator by the initial condition, i.e., $\mathbf{p}\left(kh\right)=\left({\displaystyle \prod _{i=0}^{k-1}}{\overline{U}}_n(h(i+1),hi)\right)\mathbf{p}\left(0\right)$. Using the associative property of matrices, and by multiplying them in a proper order, the computation of $\mathbf{p}\left(kh\right)$ can be performed in parallel and thus can be made quite fast. In particular, if $A\left(t\right)$ contains many non-zero intensities, this method may outperform standard ones (see the Introduction).

What is left is the estimation of G. From (12), it follows that $G_k={\sum }_{i=n+1}^{\infty }\frac{{\left(A\left(hk\right)h\right)}^i}{i!}$. Therefore,

$$G={max}_{0\le k\le N-1}\parallel G_k\parallel \le \sum _{i=n+1}^{\infty }\frac{{(\parallel A\left(hk\right)\parallel h)}^i}{i!}\le \sum _{i=n+1}^{\infty }\frac{{\left(2Lh\right)}^i}{i!}=e^{2hL}\left(1-\frac{\mathrm{\Gamma }(n+1,2hL)}{n!}\right),$$

(17)

where $\mathrm{\Gamma }(s,x)$ denotes the incomplete Gamma function.

5. Numerical Examples

Below, three examples will be considered. Firstly, we consider the example in which the original system is solved with the Runge–Kutta method and the exact solution of the system with a piecewise constant is found. In the second example, we are concerned with the computation of the value of the probability vector at a given time $t=T$. In the third example, the same computation is carried out, except for the fact that here the intensity matrix $A\left(t\right)$ does not depend on t. This substantially reduces the approximation error: in the right part of (16), $\epsilon$ becomes equal to 0 (since discretization is no longer needed) and the only error left is due to (15).

5.1. Example 1

In this example, we will build the exact solution and obtain the exact form of the limiting distribution of the process with piecewise constant intensity matrix and compare it (see (20)) with the limiting distribution of the original process with the matrix $A\left(t\right)$. Assume a birth-death process has two states and the arrival and death intensities are equal to $\lambda \left(t\right)=1-cos\left(t\right)$, $\mu \left(t\right)=5-sin\left(t\right)$, respectively. Break the $[0,2\pi ]$ interval into four intervals of equal length, i.e., the breakpoints are $0,\frac{\pi }{2},\pi ,\frac{3\pi }{2},2\pi$. Then $N=4$, $h=\frac{\pi }{2}$, and $\epsilon =2$ The solutions of the system (4) in each of the intervals are

$$p_0\left(t\right)=1+e^{-5t}a,p_0\left(t\right)=\frac{4}{5}+e^{-5t}b,p_0\left(t\right)=\frac{5}{7}+e^{-7t}c,p_0\left(t\right)=\frac{6}{7}+e^{-7t}d.$$

Note that this solution can be continued on the whole interval $[0,\infty )$, and it will be a piecewise function comprised of elementary ones:

$$p_0\left(t\right)=\left\{\begin{array}{cc}1+e^{-5t}{a}_i& 2i\pi \le a_i\le 2i\pi +\frac{\pi }{2}\\ \frac{4}{5}+e^{-5t}{b}_i& 2i\pi +\frac{\pi }{2}\le b_i\le 2i\pi +\pi \\ \frac{5}{7}+e^{-7t}{c}_i& 2i\pi +\pi \le c_i\le 2i\pi +\frac{3\pi }{2}\\ \frac{6}{7}+e^{-7t}{d}_i& 2i\pi +\frac{3\pi }{2}\le d_i\le 2i\pi +2\pi \end{array}\right.$$

(18)

The constants must be chosen in such a way so that the solution is continuous. Ergodicity guarantees the existence of the unique (periodic) solution. Irrespectively of the initial conditions for large t, the solution will be indistinguishable from the periodic. Therefore, the idea is to construct the periodic solution, which will represent the system behavior when t is large. Note that since the (periodic) solution is unique, then the following system of algebraic equations has a unique solution:

$$\left\{\begin{array}{c}1+e^{-\frac{5\pi }{2}}a=\frac{4}{5}+e^{-\frac{5\pi }{2}}b\\ \frac{4}{5}+e^{-5\pi }b=\frac{5}{7}+e^{-7\pi }c\\ \frac{5}{7}+e^{-\frac{21\pi }{2}}c=\frac{6}{7}+e^{-\frac{21\pi }{2}}d\\ \frac{6}{7}+e^{-14\pi }d=1+a\end{array}\right.$$

(19)

Since the solution is cumbersome, it is not stated here. The system (3) can be solved numerically. The behavior of the $p_0\left(t\right)$ probability can be seen in the Figure 1.

Since the convergence rate $\alpha \left(t\right)$ for the original process is $\alpha \left(t\right)=6-sin\left(t\right)-cos\left(t\right)$, then for $M=1,b=4,\epsilon =2$ from (9) we obtain

$$\underset{0\le t\le 2\pi }{sup}\parallel \mathbf{p}\left(t\right)-\overline{\mathbf{p}}\left(t\right)\parallel \le \frac{(1+log(1\left)\right)2}{4}\le 0.5.$$

(20)

5.2. Example 2

Consider the birth–death process with 31 states, with arrival and death intensities equal to $\lambda \left(t\right)=1-sin\left(t\right)$, $\mu \left(t\right)=5-cos\left(t\right)$, respectively. Break the interval $[0,2\pi ]$ interval into N intervals of equal length. Let $d=3$. Then, the convergence rate $\alpha \left(t\right)$ is equal to $\alpha \left(t\right)=4-\frac{2}{3}sin\left(t\right)-\frac{2}{3}cos\left(t\right)$. Put $\alpha =\frac{8}{3},M=1,N=50,L=8,\epsilon \le 0.36$. Then, for any $T>0$ we have the bound

$$\parallel \mathbf{p}\left(T\right)-\overline{\mathbf{p}}{\left(T\right)\parallel \le (1+G)}^N-1+\left\{\begin{array}{cc}T\epsilon ,\hfill & if0<T<\frac{1}{b}ln\frac{c}{2},\hfill \\ \frac{1}{b}(ln\frac{c}{2}+1-c{e}^{-bT})\epsilon \le 0.136,\hfill & ifT\ge \frac{1}{b}ln\frac{c}{2}.\hfill \end{array}\right.$$

(21)

In the figures below (see Figure 2 and Figure 3), one can see the solution (blue lines) for the initial conditions $X\left(0\right)=\overline{X}\left(0\right)=0$ using the Runge–Kutta method and the intensity matrix $A\left(t\right)$, solution (yellow lines) using the Runge–Kutta method and the piecewise constant matrix, the solution (green dots) using matrix multiplication.

From the

Table 1. Example 2: approximation error for various values of n.

n	$\parallel \mathsf{p}\left(2\mathbf{\pi }\right)-\overline{\mathsf{p}}\left(2\mathbf{\pi }\right)\parallel$	G	${(1+\mathit{G})}^{\mathit{N}}-1$	${(1+\mathit{G})}^{\mathit{N}}-1+0.136$
4	0.02144931153977276	≤0.41	≤28,903,845	≤28,903,846
5	0.02137755886467509	≤0.13	≤450	≤451
6	0.02138771528374614	≤0.035	≤5	≤6
7	0.021386393661019132	≤0.009	≤0.57	≤0.706
8	0.021386550440098923	≤0.002	≤0.11	≤0.245
9	0.021386533366325292	≤0.0004	≤0.021	≤0.157
10	0.02138653508102207	≤0.00007	≤0.004	≤0.14

, it can be seen that there is no sense in increasing the value of n indefinitely, since the most significant part of the approximation error is due to the substitution of the piecewise constant matrix instead of the original one. In general, to reduce the approximation error, one should increase both n and N.

5.3. Example 3

The major contribution in the approximation error is made when the initial matrix $A\left(t\right)$ is changed to the piecewise constant matrix under condition that $A\left(t\right)$ changes rapidly due to the large value of $\epsilon$. In this example will be made the same computation as in second example, except for the fact that here the intensity matrix $A\left(t\right)$ does not depend on t. This substantially reduces the approximation error: in the right part of (16), $\epsilon$ becomes equal to 0 (since discretization is no longer needed) and the only error left is due to (15).

Consider the birth–death process with 31 states, with arrival and death intensities equal to $\lambda \left(t\right)=1$, $\mu \left(t\right)=5$, respectively. Break the interval $[0,2\pi ]$ interval into N intervals of equal length. The approximation error is bounded by

$$\parallel \mathbf{p}\left(T\right)-\overline{\mathbf{p}}{\left(T\right)\parallel \le (1+G)}^N-1.$$

(22)

The

Table 2. Example 3: approximation error for various values of n.

n	$\parallel \mathsf{p}\left(2\mathbf{\pi }\right)-\overline{\mathsf{p}}\left(2\mathbf{\pi }\right)\parallel$	G	${(1+\mathit{G})}^{\mathit{N}}-1$
4	≤4 $\times {10}^{-10}$	≤0.283	21,333
5	≤2 $\times {10}^{-11}$	≤0.09	31
6	≤7 $\times {10}^{-13}$	≤0.022	$1.34$
7	≤3 $\times {10}^{-14}$	≤0.005	$0.23$
8	≤9.99 $\times e^{-14}$	≤0.002	$0.09$
9	≤9.99 $\times e^{-14}$	≤0.0002	$0.009$
10	≤9.99 $\times e^{-14}$	≤0.00004	$0.002$

demonstrates its quality for $N=40$.

As can be seen from the second column, the solutions by the Runge–Kutta method and the matrix method are identical; the approximation error, shown in the fourth column, significantly overestimates the difference.

Note that in the considered context, i.e., when $A\left(t\right)$ does not depend on t, the increase of N makes the situation worse and increases the approximation error. For example, for $N=1$ and $n=10$, we have that $\parallel \mathbf{p}\left(T\right)-\overline{\mathbf{p}}\left(T\right)\parallel \le 0.00004$.

In the figures below (see Figure 4 and Figure 5), one can see the solution (yellow lines) for the initial conditions $X\left(0\right)=\overline{X}\left(0\right)=0$ using the Runge–Kutta method and the intensity matrix $A\left(t\right)$ (one can see that the solution for the system with the piecewise constant matrix will be the same), the solution (green dots) using matrix multiplication.

6. Conclusions

In this paper, the novel variation of the well-known method for the computation of the transient distribution of the finite state time-varying Markov chains has been proposed. Its main advantage is that, if the two chains start in the same state, then within a finite horizon one can bound the difference between their probability distributions. The generalization to the countable state space is the direction for further research and it seems to be feasible since the truncation bounds are available for inhomogeneous CTMC with countable state space.