Multi-Dimensional Markov Chains of M/G/1 Type

Abstract

We consider an irreducible discrete-time Markov process with states represented as (k, i) where k is an M-dimensional vector with non-negative integer entries, and i indicates the state (phase) of the external environment. The number n of phases may be either finite or infinite. One-step transitions of the process from a state (k, i) are limited to states (n, j) such that n ≥ k−1, where 1 represents the vector of all 1s. We assume that for a vector k ≥ 1, the one-step transition probability from a state (k, i) to a state (n, j) may depend on i, j, and n − k, but not on the specific values of k and n. This process can be classified as a Markov chain of M/G/1 type, where the minimum entry of the vector n defines the level of a state (n, j). It is shown that the first passage distribution matrix of such a process, also known as the matrix G, can be expressed through a family of nonnegative square matrices of order n, which is a solution to a system of nonlinear matrix equations.

1. Introduction

The Markov chains of M/G/1 type represent a basic model that explains the temporal evolution of population size. It is a two-component process, the first component of which is an integer-valued process describing the dynamics of population size, and the other represents the state (phase) of the external environment. The state space of Markov chains of M/G/1 type can be partitioned into non-empty disjoint subsets known as levels, such that one-step state transitions are limited to states at the same level, the adjacent lower level, or any higher level. The transition matrix of a Markov chain of M/G/1 type has a block upper Hessenberg form and all blocks along the main diagonal, except for the top ones, are identical.

The matrix geometric method proposed by M.F. Neuts in [1] and his study of Markov chains of M/G/1 type in [2] led to the rapid development of matrix-analytic methods in operations research. These methods provide a powerful framework for the unified analysis of large classes of Markov processes and, more importantly, for their numerical solution. Matrix-analytic methods and their applications as they stand today are outlined in [3,4,5,6,7,8,9,10,11].

In this paper, we study discrete-time Markov chains $\xi (t)=(\alpha (t),\beta (t))$ on the state space $X={\mathbb{Z}}_{+}^M\times J$, where ${\mathbb{Z}}_{+}$ is the set of nonnegative integers and $J=\{1,2,\dots ,n\}$. The number of elements in $J$ may be either finite or infinite. One-step transitions of the process from a state $(k,i)$ are limited to states $(n,j)$ such that $n\ge k-1,$ where $1$ represents the vector of all 1s. We assume that the process $\xi (t)$ is spatially homogeneous, meaning that for $k\ge 1$ the one-step transition probability from a state $(k,i)$ to a state $(n,j)$ may depend on $i,j$, and $n-k$ but not on the specific values of $k$ and $n.$ We will refer to these processes as M-dimensional Markov chains of M/G/1 type $(M\mathrm{d}-\mathrm{M}/\mathrm{G}/1)$. The $M\mathrm{d}-\mathrm{M}/\mathrm{G}/1$ processes are Markov chains of M/G/1 type, with the level $l$ consisting of states $(k_1,k_2,\dots ,k_M,j)$ for which $\min k_i=l$.

Multidimensional quasi-birth-and-death processes ($M\mathrm{d}-\mathrm{QBD}$) are a specific type of multidimensional Markov chains of M/G/1 type. They are characterized by having one-step transitions from a state $(k,i)$ restricted to states $(n,j)$ such that $n-k\in {\{-1,0,1\}}^M$. The explicit analytical representation for the stationary distribution of $M\mathrm{d}-\mathrm{QBD}$ processes are unknown, and most work has been devoted to deriving asymptotic formulas of the stationary distribution (see [12,13,14]). The conditions ensuring a positive recurrent or transient 2d-QBD process were analyzed in [15]. Specific cases of $M\mathrm{d}-\mathrm{QBD}$ processes, where only one component of the vector $\alpha (t)$ may change at a time, were studied in [16].

In the theory of Markov chains of M/G/1 type, a crucial role is played by the so-called matrix $G$ [2]. If it is known, many relevant quantities may be efficiently computed. Different algorithms for the numerical computation of the matrix $G$ have been proposed [2,17,18,19,20,21]. However, the structure of the levels of $Md-\mathrm{M}/\mathrm{G}/1$ processes can make it challenging to use these methods effectively.

This study aims to describe the matrix $G$ of $M\mathrm{d}-\mathrm{M}/\mathrm{G}/1$ processes in terms of matrices of order $n$. Section 2 explores classical Markov chains of M/G/1 type. Section 3 defines the multidimensional Markov chains of M/G/1 type and introduces the concept of the state sectors and the sector exit probabilities. We analyze the first passage probabilities and demonstrate that the family of matrices representing the sector exit probabilities satisfies a system of nonlinear matrix equations. Section 4 illustrates how a solution to this system can be obtained through successive substitution. An example of the Md-QBD process is discussed in Section 5. Finally, we offer some concluding remarks in Section 6.

We use bold capital letters to denote matrices and bold lowercase letters to denote vectors. Unless otherwise stated, all vectors in this paper have integer components and the length $M$. For any vector $x$ we use the notation $x_i$ for the $i\mathrm{th}$ component of $x$. For vectors $x=(x_1,x_2,\dots ,x_M)$ and $y=(y_1,y_2,\dots ,y_M)$, $x\ge y$ means that $x_j\ge y_j$ for all $j$, and $x\cancel{\ge }y$ means that $x_j<y_j$ for at least one value of $j$. Notations $x\le y$ and $x\cancel{\le }y$ are defined similarly. Vector $1$ represents the vector of all 1s, and the vector $e_m$ indicates the vector with zero entries except the mth entry, which equals one. Given a vector $w\ge 1$ and an integer $k\ge -1$, we define two sets $Z_k(w)=\{n\in {\mathbb{Z}}^M|\min (n_i-w_i)=k\}$ and $X_k(w)=Z_k(w)\times J$. Sets $Z(w)$, $X(w)$, and $X^c(w)$ are defined as $Z(w)=\{n\in Z^M|n\ge w\}$, $X(w)=Z(w)\times J$, and $X^c(w)=X\backslash X(w).$ We refer to the set of states $X(w)$ as the sector.

2. Markov Chains of M/G/1 Type

A Markov chain of M/G/1 type with a state space ${\mathbb{Z}}_{+}\times J$ is characterized by a block Hessenberg transition matrix of the form

$$P=\left[\begin{array}{lllll}{B}_0\hfill & B_1\hfill & B_2\hfill & B_3\hfill & \cdots \hfill \\ A_{-1}\hfill & A_0\hfill & A_1\hfill & A_2\hfill & \ddots \hfill \\ O\hfill & A_{-1}\hfill & A_0\hfill & A_1\hfill & \ddots \hfill \\ O\hfill & O\hfill & A_{-1}\hfill & A_0\hfill & \ddots \hfill \\ ⋮\hfill & \ddots \hfill & \ddots \hfill & \ddots \hfill & \ddots \hfill \end{array}\right],$$

where blocks $B_i$ and $A_i$ are nonnegative square matrices, such that ${\displaystyle {\sum }_{i=0}^{\infty }{B}_i}$ and ${\displaystyle {\sum }_{i=-1}^{\infty }{A}_i}$ are stochastic matrices. Entries of the matrix $P$ blocks are indexed by the elements of the phase space $J$. The subset of all states $(l,i)$, such that $i\in J$, is called the level $l$.

A fundamental role in the theory of Markov chains of M/G/1 type is played by the matrix $G$, described by Neuts in [2]. The element $g(i,j)$ of this matrix represents the probability that, starting from the state $(l,i)$ $(l\ge 1)$, the chain will first appear at the level $l-1$ in state $(l-1,j)$. Neuts demonstrated that $G$ is the minimal nonnegative solution to the equation

$$G={\displaystyle \sum _{v=0}^{\infty }{A}_{v-1}{G}^v}.$$

(1)

There are several algorithms for calculating the invariant measures of the transition matrix $P$ using the matrix $G$ [22,23,24].

3. Multidimensional Processes of M/G/1 Type

Consider an irreducible discrete-time Markov chain $\xi (t)=(\alpha (t),\beta (t))$ on the state space $X={\mathbb{Z}}_{+}^M\times J$. Let us denote the probability of a one-step transition from $(k,i)$ to $(n,j)$ as $p_{k,n}(i,j)$. We assume that the transition probability matrix $P=[p_{k,n}(i,j)]$ $(k,n\in {\mathbb{Z}}_{+}^M$, $i,j\in J)$, partitioned into blocks $P_{k,n}$ of order $n$, for all $k\ge 1$ has the following properties:

$$P_{k,n}=\left\{\begin{array}{cc}{Q}_{n-k},& n\ge k-1,\\ O,& n\cancel{\ge }k-1,\end{array}\right.$$

(2)

where $Q_r$, $r\ge -1$, are nonnegative square matrices such that $Q={\displaystyle {\sum }_{r\ge -1}{Q}_r}$ is a stochastic matrix. We refer to this process as an M-dimensional Markov chain of M/G/1 type $(M\mathrm{d}-\mathrm{M}/\mathrm{G}/1).$

3.1. Sectors of the Process States

Let us define the sequence of passage times as follows:

$${\theta }_0=0,{\theta }_{k+1}=\min \{t>{\theta }_k|\xi (t)\in X^c(\alpha ({\theta }_k)\},k\ge 0.$$

We say that at time $t$, the process $\xi (t)=(\alpha (t),\beta (t))$ is in the sector $X(r)$ if ${\theta }_k\le t<{\theta }_{k+1}$ and if $\alpha ({\theta }_k)=r$. For a vector $w\ge 1$, we define the sector $X(w)$ exit time as the moment ${\tau }_w$ when the process first enters the set $X^c(w)$,

$${\tau }_w=\min \{t\ge 0|\xi (t)\in X^c(w)\}.$$

Additionally, we define the number of sectors visited along a path to $X^c(w)$ as

$${\kappa }_w=\max \{k\ge 0|{\theta }_k\le {\tau }_w\}.$$

If an initial state $\xi (0)$ of the process belongs to $X^c(w)$, then we have ${\tau }_w=0$ and ${\kappa }_w=0.$ If $\xi (0)=(k,i)$ with $k\ge w$, then at the first hitting time of $X^c(w)$, the process exits the sector $X(\alpha ({\theta }_{{\kappa }_w-1}))$ and enters the sector $X(\alpha ({\theta }_{{\kappa }_w}))$, which implies equality ${\tau }_w={\theta }_{{\kappa }_w}$. The set $X^c(w)$ is reached at the moment of transition from the set $X_0(\alpha ({\theta }_{{\kappa }_w-1}))\cap X_0(w)$ to the state $\xi ({\theta }_{{\kappa }_w})\in X_{-1}(\alpha ({\theta }_{{\kappa }_w-1}))\cap X_{-1}(w).$

For vectors $w\ge 1$, $k\in X(w)$, and $n\in X^c(w)$ we define matrices ${\Phi }_{k,n}^{(v)}(w)=[{\phi }_{(k,i),(r,j)}^{(v)}(w)]$ $(i,j\in J)$ and ${\Phi }_{k,n}(w)=[{\phi }_{(k,i),(n,j)}(w)]$ $(i,j\in J)$ as follows.

The element ${\phi }_{(k,i),(n,j)}^{(v)}(w)$ of the matrix ${\Phi }_{k,n}^{(v)}(w)$ is the conditional probability that the process $\xi (t)$, starting in the state $(k,i)\in X(w)$, reaches the set $X^c(w)$ by hitting the state $(n,j)$ after passing through exactly $v$ sectors,

$${\phi }_{(k,i),(n,j)}^{(v)}(w)=\mathbb{P}\{{\kappa }_w=v,\xi ({\theta }_{{\kappa }_w})=(n,j)|\xi (0)=(k,i)\}$$

$$=\mathbb{P}\{{\kappa }_w=v,\xi ({\tau }_w)=(n,j)|\xi (0)=(k,i)\} .$$

(3)

The element ${\phi }_{(k,i),(n,j)}(w)$ of the matrix ${\Phi }_{k,n}(w)$ is the conditional probability that the process $\xi (t)$ will eventually hit the set $X^c(w)$ in the state $(n,j)$, given that it starts in the state $(k,i)$,

$${\phi }_{(k,i),(n,j)}(w)=\mathbb{P}\{{\kappa }_w<\infty ,\xi ({\theta }_{{\kappa }_w})=(n,j)|\xi (0)=(k,i)\} =\mathbb{P}\{\xi ({\tau }_w)=(n,j)|\xi (0)=(k,i)\} .$$

(4)

It is clear that matrices ${\Phi }_{k,n}(w)$ and ${\Phi }_{k,n}^{(v)}(w)$, $v=1,2,\dots$ are related to each other by the equality

$${\Phi }_{k,n}(w)={\displaystyle \sum _{v=1}^{\infty }{\Phi }_{k,n}^{(v)}(w)}.$$

(5)

For $l\ge 0$, any path of $\xi (t)$ leading from a state $(k,i)\in X_l(w)$ to a state $(n,j)\in X^c(w)$ must successively visit sets $X_{l-1}(w),X_{l-2}(w),\dots ,X_0(w),X_{-1}(w)$, which will require visiting at least $l+1$ sectors. Therefore, we have

$${\Phi }_{k,n}^{(v)}(w)=O, \mathrm{for} \mathrm{all} k\in Z_l(w), n\in Z_{-1}(w)\mathrm{and} v\le l.$$

(6)

Since the process $\xi (t)$ is spatially homogeneous, for any vector $k\ge w\ge 1$ and any vector $n\in Z_{-1}(w)$, the probabilities ${\phi }_{(k,i),(n,j)}(w)$ may depend on $i,j,$ $k-w$, and $n-w$ but not on the specific values of $k$,$n$ and $w$, i.e., ${\Phi }_{k,n}(w)={\Phi }_{k-w+1,n-w+1}(1).$ This means that the matrix ${\Phi }_{k,n}(w)$ may be expressed as

$${\Phi }_{k,n}(w)={\Psi }_{k-w,n-w},$$

(7)

where the matrix ${\Psi }_{\alpha ,\beta }$ is defined as

$${\Psi }_{\alpha ,\beta }={\Phi }_{w+\alpha ,w+\beta }(w)$$

(8)

independently of the vector $w\ge 1$.

The element ${\psi }_{(\alpha ,i),(\beta ,j)}$ of the matrix ${\Psi }_{\alpha ,\beta }$ is the conditional probability that the process $\xi (t)$ will hit the state $(w+\beta ,j)$ on the first visit to the set $X^c(w)$, given that it starts in the state $(w+\alpha ,i).$ Here, the vectors $\alpha$ and $\beta$ satisfy conditions $w+\alpha \ge w$ and $w+\beta \in Z_{-1}(w)$. Therefore, the index $\alpha$ of a matrix ${\Psi }_{\alpha ,\beta }$ is a nonnegative vector and its index $\beta$ belongs to the set $\mathcal{E}$ defined as

$$\mathcal{E}=\{\beta \in {\mathbb{Z}}^M|\min {\beta }_m=-1\}.$$

We refer to the matrices ${\Psi }_{\alpha ,\beta }$ as the matrices of the first passage probabilities.

The matrices ${\Phi }_{k,n}(k)={\Psi }_{0,n-k}$ determine the transition probabilities of the embedded Markov chain $\xi ({\theta }_k)$, since for $k\ge 1$ we have

$${\phi }_{(k,i),(n,j)}(k)=\mathbb{P}\{\xi ({\tau }_k)=(n,j)|\xi (0)=(k,i)\}$$

$$=\mathbb{P}\{\xi ({\theta }_1)=(n,j)|\xi (0)=(k,i)\}=\mathbb{P}\{\xi ({\theta }_k)=(n,j)|\xi ({\theta }_{k-1})=(k,i)\}.$$

(9)

We define matrices $F_{\beta }$ as

$$F_{\beta }={\Psi }_{0,\beta }, \beta \in \mathcal{E},$$

(10)

and refer to these matrices as the matrices of the sector exit probabilities.

3.2. Matrices of the First Passage Probabilities

Theorem 1.

The matrices of the first passage probabilities ${\Psi }_{\alpha ,\beta }$ satisfy the system

$${\Psi }_{\alpha ,\beta }=Q_{\beta -\alpha }+{\displaystyle \sum _{\gamma \ge 0}{Q}_{\gamma -\alpha }}{\Psi }_{\gamma ,\beta }, \alpha \ge 0,\beta \in \mathcal{E}.$$

(11)

Proof of Theorem 1. 

We will initially demonstrate the validity of the following equality for all vectors $k\in Z(w)$ and $n\in Z_{-1}(w)$:

$${\Phi }_{k,n}(w)=P_{k,n}+{\displaystyle \sum _{r\ge w}{P}_{k,r}}{\Phi }_{r,n}(w).$$

(12)

This formula adheres to the law of total probability, taking into account all possible states of the process following the first transition. Consider two states: $(k,i)\in X(w)$ and $(n,j)\in X_{-1}(w)$. The state $(n,j)$ can be reached from the state $(k,i)$ after a single transition. This contributes to (12) the term $P_{k,n}$.The first transition may take the process to some state $(r,j)$ with $r\ge w$. To reach the set $X_{-1}(w)$ from state $(r,j)$, the process $\xi (t)$ must necessarily cross one or more sectors and then hit the state $(n,j)\in X_{-1}(w)$. This contributes to the second term on the right-hand side of (12). Equation (10) for the matrices ${\Psi }_{\alpha ,\beta }$ is derived from (12) using Formulas (2) and (7). □

For vectors $\alpha \ge 0$ and $\beta \in \mathcal{E}$, let the set ${\mathrm{E}}_{\alpha ,\beta }^{(v)}$ be defined as the set of all $v-$ tuples $({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathcal{E}}^v$, satisfying $\alpha +{\epsilon }_1\ge 0,$ $\alpha +{\epsilon }_1+{\epsilon }_2\ge 0,$…, $\alpha +{\epsilon }_1+\dots +{\epsilon }_{v-1}\ge 0$, and $\alpha +{\epsilon }_1+\dots +{\epsilon }_v=\beta$, and let the set ${\mathrm{E}}_{\alpha ,\beta }$ be defined as ${\mathrm{E}}_{\alpha ,\beta }={\displaystyle {\cup }_{v=1}^{\infty }{\mathrm{E}}_{\alpha ,\beta }^{(v)}}$.

Any nonnegative vector $\alpha$ can be represented as $\alpha =\delta +k1$, with $k=\min {\alpha }_m$, $\delta \in \mathcal{N},$ and the set $\mathcal{N}$ being defined as $\mathcal{N}=\{n\in {\mathbb{Z}}^M|\min n_m=0\}$. Since the inequality $\epsilon \ge -1$ holds for all vectors $\epsilon \in \mathcal{E}$, only elements ${\epsilon }_1,\dots ,{\epsilon }_v$ of the set $\mathcal{E}$ in quantity $v>k$ can satisfy the condition $\delta +k1+{\epsilon }_1+\dots +{\epsilon }_v\in \mathcal{E}$. Therefore, for any $\delta \in \mathcal{N}$, $\beta \in \mathcal{E}$ and $k\ge v\ge 1$, the set ${\mathrm{E}}_{\delta +k1,\beta }^{(v)}$ is empty. If this set is not empty, then the following decomposition is valid in terms of the Cartesian products of sets of the form ${\mathrm{E}}_{\gamma ,\epsilon }^{(u)}$, where $\gamma \in \mathcal{N}$, $\epsilon \in \mathcal{E}$, and $u\ge 1$,

Lemma 1. 

For any vectors $\delta \in \mathcal{N}$, $\beta \in \mathcal{E}$ and an integer $k\ge 1$ the sets ${\mathrm{E}}_{\delta +k1,\beta }^{(v)}$, $v\ge 1$, and the set ${\mathrm{E}}_{\delta +k1,\beta }$ can be represented as follows:

$${\mathrm{E}}_{\delta +k1,\beta }^{(v)}={\displaystyle \underset{\begin{array}{c}{u}_1,\dots ,u_{k+1}\ge 1\\ u_1+\dots +u_{k+1}=v\end{array}}{\cup } {\displaystyle \underset{{\gamma }_1,\dots ,{\gamma }_k\in \mathcal{N}}{\cup }{\mathrm{E}}_{\delta ,{\gamma }_1-1}^{(u_1)}\times {\mathrm{E}}_{{\gamma }_1,{\gamma }_2-1}^{(u_2)}\times \dots \times {\mathrm{E}}_{{\gamma }_{k-1},{\gamma }_k-1}^{(u_k)}\times }{\mathrm{E}}_{{\gamma }_k,\beta }^{(u_{k+1})}},$$

(13)

$${\mathrm{E}}_{\delta +k1,\beta }={\displaystyle \underset{{\gamma }_1,\dots ,{\gamma }_k\in \mathcal{N}}{\cup }{\mathrm{E}}_{\delta ,{\gamma }_1-1}\times {\mathrm{E}}_{{\gamma }_1,{\gamma }_2-1}\times \dots \times {\mathrm{E}}_{{\gamma }_{k-1},{\gamma }_k-1}\times {\mathrm{E}}_{{\gamma }_k,\beta }}.$$

(14)

Proof of Lemma 1. 

Since $\delta +l1\in Z_{l-1}(1)$ and $\beta \in Z_{-1}(1)$, for any $v-$ tuple $({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{\delta +l1,\beta }^{(v)}$ of the length $v>l\ge 1$, in the interval $1\le w<v$ there necessarily exist numbers $w$, such that $\delta +l1+{\epsilon }_1+\dots +{\epsilon }_w\in Z_{l-2}(1)$. Let us denote by $u$ the minimum of such numbers. The number $v-u$ of the elements ${\epsilon }_{u+1},{\epsilon }_{u+2},\dots ,{\epsilon }_v$, satisfying the condition $(\delta +l1+{\epsilon }_1+\dots +{\epsilon }_u)$$+{\epsilon }_{u+1}+\dots +{\epsilon }_v=\beta$, must be at least $l$. Therefore, the set ${\mathrm{E}}_{\delta +l1,\beta }^{(v)}$ can be represented as follows:

$${\mathrm{E}}_{\delta +l1,\beta }^{(v)}={\displaystyle \underset{u=1}{\overset{v-l}{\cup }}\{({\epsilon }_1,\dots ,{\epsilon }_v)\in \mathcal{E}|\delta +l1+{\epsilon }_1\ge l1,\delta +l1+{\epsilon }_1+{\epsilon }_2\ge l1,}$$

$$\dots ,\delta +l1+{\epsilon }_1+\dots +{\epsilon }_{u-1}\ge l1,\delta +l1+{\epsilon }_1+\dots +{\epsilon }_u\in Z_{-1}(l1),$$

(15)

$$\delta +l1+{\epsilon }_1+\dots +{\epsilon }_{u+1}\ge 0,\dots ,\delta +l1+{\epsilon }_1+\dots +{\epsilon }_{v-1}\ge 0,$$

$$\delta +l1+{\epsilon }_1+\dots +{\epsilon }_v=\beta \}.$$

Each element $z$ of the set $Z_{-1}(l1)$ can be written as $z=\gamma +(l-1)1,$ where $\gamma \in \mathcal{N}$. Therefore, Equation (15) implies the following equalities:

$${\mathrm{E}}_{\delta +l1,\beta }^{(v)}={\displaystyle \underset{u=1}{\overset{v-l}{\cup }}{\displaystyle \underset{\gamma \in \mathcal{N}}{\cup }\{({\epsilon }_1,\dots ,{\epsilon }_v)\in \mathcal{E}|\delta +l1+{\epsilon }_1\ge l1,\delta +l1+{\epsilon }_1+{\epsilon }_2\ge l1,}}$$

$$\dots ,\delta +l1+{\epsilon }_1+\dots +{\epsilon }_{u-1}\ge l1,\delta +l1+{\epsilon }_1+\dots +{\epsilon }_u=\gamma +(l-1)1,$$

$$\delta +l1+{\epsilon }_1+\dots +{\epsilon }_{u+1}\ge 0,\dots ,\delta +l1+{\epsilon }_1+\dots +{\epsilon }_{v-1}\ge 0,$$

$$\delta +l1+{\epsilon }_1+\dots +{\epsilon }_v=\beta \}$$

$$={\displaystyle \underset{u=1}{\overset{v-l}{\cup }}{\displaystyle \underset{\gamma \in \mathcal{N}}{\cup }\{({\epsilon }_1,\dots ,{\epsilon }_v)\in \mathcal{E}|\delta +{\epsilon }_1\ge 0,\delta +{\epsilon }_1+{\epsilon }_2\ge 0,}}\dots ,\delta +{\epsilon }_1+\dots +{\epsilon }_{u-1}\ge 0,$$

$$\delta +{\epsilon }_1+\dots +{\epsilon }_u=\gamma -1,\gamma +(l-1)1+{\epsilon }_{u+1}\ge 0,\gamma +(l-1)1+{\epsilon }_{u+1}+{\epsilon }_{u+2}\ge 0,\dots ,$$

$$\gamma +(l-1)1+{\epsilon }_{u+1}+\dots +{\epsilon }_{v-1}\ge 0,\gamma +(l-1)1+{\epsilon }_{u+1}+\dots +{\epsilon }_v=\beta \}$$

$$={\displaystyle \underset{u=1}{\overset{v-l}{\cup }}{\displaystyle \underset{\gamma \in \mathcal{N}}{\cup }{\mathrm{E}}_{\delta ,\gamma -1}^{(u)}\times {\mathrm{E}}_{\gamma +(l-1)1,\beta }^{(v-u)}}}$$

(16)

Thus, for any integers $v>l\ge 1$, the following formula is valid

$${\mathrm{E}}_{\delta +l1,\beta }^{(v)}={\displaystyle \underset{u=1}{\overset{v-l}{\cup }}{\displaystyle \underset{\gamma \in \mathcal{N}}{\cup }{\mathrm{E}}_{\delta ,\gamma -1}^{(u)}\times {\mathrm{E}}_{\gamma +(l-1)1,\beta }^{(v-u)}}}.$$

(17)

Applying this formula for $l=k,k-1,\dots ,1$, we can obtain the decomposition (13) for the set ${\mathrm{E}}_{\delta +k1,\beta }^{(v)}$.

Introducing a new variable $w=v-u$ in (17) and changing the order of summation, we obtain the following result:

$${\mathrm{E}}_{\delta +l1,\beta ,\beta }={\displaystyle \underset{u=1}{\overset{\infty }{\cup }}{\displaystyle \underset{w=l}{\overset{\infty }{\cup }}{\displaystyle \underset{\gamma \in \mathcal{N}}{\cup }{\mathrm{E}}_{\delta ,\gamma -1}^{(u)}\times {\mathrm{E}}_{\gamma +(l-1)1,\beta }^{(w)}}}}={\displaystyle \underset{\gamma \in \mathcal{N}}{\cup }{\mathrm{E}}_{\delta ,\gamma -1}\times {\mathrm{E}}_{\gamma +(l-1)1,\beta }}.$$

Applying this formula for $l=k,k-1,\dots ,1$, we can achieve the decomposition (14) for the set ${\mathrm{E}}_{\delta +k1,\beta }$, which proves Lemma 1. □

Theorem 2. 

For $\alpha \ge 0$ and $\beta \in \mathcal{E}$, the matrices ${\Psi }_{\alpha ,\beta }$ are given by

$${\Psi }_{\alpha ,\beta }={\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{\alpha ,\beta }^{(v)}}{F}_{{\epsilon }_1}{F}_{{\epsilon }_2}\cdots F_{{\epsilon }_v}}}.$$

(18)

Proof of Theorem 2. 

Entries of matrices ${\Phi }_{r,n}(r)$ are the one-step transition probabilities (9) of the embedded Markov chain $\xi ({\theta }_k)$, while entries of the matrices ${\Phi }_{k,n}^{(v)}(w)$ are the probabilities of reaching the set $X^c(w)$ after $v$ steps of this Markov chain. Therefore, for all $k\in Z(w)$ and $n\in Z_{-1}(w)$, the matrices ${\Phi }_{k,n}^{(v)}(w)$ are completely determined by the matrices ${\Phi }_{r,n}(r)$, $r\ge w$, as follows:

$${\Phi }_{k,n}^{(1)}(w)={\Phi }_{k,n}(k),$$

$${\Phi }_{k,n}^{(u+1)}(w)={\displaystyle \sum _{(r_1,\dots ,r_u)\in T_{k,n}^{(u)}(w)}{\Phi }_{k,r_1}(k){\Phi }_{r_1,r_2}(r_1)\cdots {\Phi }_{r_{u-1},r_u}(r_{u-1}){\Phi }_{r_u,n}(r_u)}, u\ge 1.$$

(19)

Here, the summation extends over the set $T_{k,n}^{(u)}(w)$ of all $u-$tuples $(r_1,\dots ,r_u)$, satisfying $r_1,\dots ,r_u\in Z(w)$, $r_1\in Z_{-1}(k)$, $r_2\in Z_{-1}(r_1)$,…, $r_u\in Z_{-1}(r_{u-1})$, $n\in Z_{-1}(r_u)$.

From (5) and (19), it follows that for all $k\in Z(w)$ and $n\in Z_{-1}(w)$, we have

$${\Phi }_{k,n}(w)={\displaystyle \sum _{v=1}^{\infty }{\Phi }_{k,n}^{(v)}(w)}$$

$$={\Phi }_{k,n}(k)+{\displaystyle \sum _{u=1}^{\infty }{\displaystyle \sum _{(r_1,\dots ,r_u)\in {\mathrm{T}}_{k,n}^{(u)}(w)}{\Phi }_{k,r_1}(k){\Phi }_{r_1,r_2}(r_1)\cdots {\Phi }_{r_{u-1},r_u}(r_{u-1}){\Phi }_{r_u,n}(r_u)}}$$

$$=F_{n-k}(k)+{\displaystyle \sum _{u=1}^{\infty }{\displaystyle \sum _{(r_1,\dots ,r_u)\in {\mathrm{T}}_{k,n}^{(u)}(w)}{F}_{r_1-k}{F}_{r_2-r_1}\cdots F_{r_u-r_{u-1}}{F}_{n-r_u}}}.$$

(20)

Using Formulas (7) and (20), we can obtain the matrix ${\Psi }_{\alpha ,\beta }$ as

$${\Psi }_{\alpha ,\beta }={\Phi }_{\alpha +1,\beta +1}(1)$$

$$=F_{\beta -\alpha }+{\displaystyle \sum _{u=1}^{\infty }{\displaystyle \sum _{(r_1,\dots ,r_u)\in {\mathrm{T}}_{\alpha +1,\beta +1}^{(u)}(1)}{F}_{r_1-\alpha -1}{F}_{r_2-r_1}\cdots F_{r_u-r_{u-1}}{F}_{\beta +1-r_u}}}.$$

(21)

Let us introduce new variables: $v=u+1$, ${\epsilon }_1=r_1-\alpha -1$, ${\epsilon }_2=r_2-r_1$, …, ${\epsilon }_{v-1}=r_{v-1}-r_{v-2},$ and ${\epsilon }_v=\beta +1-r_{v-1}$. From the definition of the set ${\mathrm{T}}_{\alpha +1,\beta +1}^{(u)}(1)$, it follows that vectors ${\epsilon }_1,\dots ,{\epsilon }_v$ belong to the set $\mathcal{E}$ and satisfy the following conditions: $\alpha +{\epsilon }_1\ge 0$, $\alpha +{\epsilon }_1+{\epsilon }_2\ge 0,$ …, and $\alpha +{\epsilon }_1+\dots +{\epsilon }_{v-1}\ge 0$. Taking into account that $\alpha +{\epsilon }_1+\dots +{\epsilon }_v=\beta$, we finally obtain (18). □

It follows from Lemma 1 and Theorem 2 that matrices ${\Psi }_{\alpha ,\beta }$ with arbitrary nonnegative index $\alpha$ can be expressed in terms of the matrices ${\Psi }_{\delta ,\gamma }$, with $\delta \in \mathcal{N}$.

Corollary 1. 

For any vectors $\delta \in \mathcal{N}$, $\beta \in \mathcal{E}$ and integers $k\ge 1$, the matrix ${\Psi }_{\delta +k1,\beta }$ can be represented as

$${\Psi }_{\delta +k1,\beta }={\displaystyle \sum _{{\gamma }_1,\dots ,{\gamma }_k\in \mathcal{N}}{\Psi }_{\delta ,{\gamma }_1-1}{\Psi }_{{\gamma }_1,{\gamma }_2-1}\dots {\Psi }_{{\gamma }_{k-1},{\gamma }_k-1}{\Psi }_{{\gamma }_k,\beta }}.$$

(22)

Proof of Corollary 1. 

It follows from (13) and (18) that the matrix

$${\Psi }_{\delta +k1,\beta }={\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{\delta +k1,\beta }^{(v)}}{F}_{{\epsilon }_1}{F}_{{\epsilon }_2}\cdots F_{{\epsilon }_v}}}$$

can be represented as follows:

$${\Psi }_{\delta +k1,\beta }={\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{\begin{array}{c}{u}_1,\dots ,u_{k+1}\ge 1\\ u_1+\dots +u_{k+1}=v\end{array}}{\displaystyle \sum _{{\gamma }_1,\dots ,{\gamma }_k\in \mathcal{N}}({\displaystyle \sum _{({\epsilon }_{1,1},\dots ,{\epsilon }_{1,u_1})\in {\mathrm{E}}_{\delta ,{\gamma }_1-1}^{(u_1)}}{F}_{{\epsilon }_{1.1}}\cdots F_{{\epsilon }_1,u_1})}}}}$$

$$({\displaystyle \sum _{({\epsilon }_{2,1},\dots ,{\epsilon }_{2,u_2})\in {\mathrm{E}}_{{\gamma }_1,{\gamma }_2-1}^{(u_2)}}{F}_{{\epsilon }_{2.1}}\cdots F_{{\epsilon }_{2,u_2}})}\dots ({\displaystyle \sum _{({\epsilon }_{k,1},\dots ,{\epsilon }_{k,u_k})\in {\mathrm{E}}_{{\gamma }_{k-1},{\gamma }_k-1}^{(u_k)}}{F}_{{\epsilon }_{k.1}}\cdots F_{{\epsilon }_{k,u_k}})}$$

$$({\displaystyle \sum _{({\epsilon }_{k+1,1},\dots ,{\epsilon }_{k+1,u_{k+1}})\in {\mathrm{E}}_{{\gamma }_k,\beta }^{(u_{k+1})}}{F}_{{\epsilon }_{k+1.1}}\cdots F_{{\epsilon }_{k+1,u_{k+1}}})}$$

$$={\displaystyle \sum _{{\gamma }_1,\dots ,{\gamma }_k\in \mathcal{N}}[{\displaystyle \sum _{u_1=1}^{\infty }({\displaystyle \sum _{({\epsilon }_{1,1},\dots ,{\epsilon }_{1,u_1})\in {\mathrm{E}}_{\delta ,{\gamma }_1-1}^{(u_1)}}{F}_{{\epsilon }_{1.1}}\cdots F_{{\epsilon }_1,u_1})}}}$$

$${\displaystyle \sum _{u_2=1}^{\infty }({\displaystyle \sum _{({\epsilon }_{2,1},\dots ,{\epsilon }_{2,u_2})\in {\mathrm{E}}_{{\gamma }_1,{\gamma }_2-1}^{(u_2)}}{F}_{{\epsilon }_{2.1}}\cdots F_{{\epsilon }_{2,u_2}})}\dots }$$

$${\displaystyle \sum _{u_k=1}^{\infty }({\displaystyle \sum _{({\epsilon }_{k,1},\dots ,{\epsilon }_{k,u_k})\in {\mathrm{E}}_{{\gamma }_{k-1},{\gamma }_k-1}^{(u_k)}}{F}_{{\epsilon }_{k.1}}\cdots F_{{\epsilon }_{k,u_k}})}}$$

$${\displaystyle \sum _{u_{k+1}=1}^{\infty }({\displaystyle \sum _{({\epsilon }_{k+1,1},\dots ,{\epsilon }_{k+1,u_{k+1}})\in {\mathrm{E}}_{{\gamma }_k,\beta }^{(u_{k+1})}}{F}_{{\epsilon }_{k+1.1}}{F}_{{\epsilon }_{k+1,2}}\cdots F_{{\epsilon }_{k+1,u_{k+1}}})}}].$$

(23)

After applying Formula (18) to each sum inside the square brackets in (23), we obtain Formula (22), which proves Corollary 1. □

3.3. Md-M/G/1 Processes as One-Dimensional Markov Chains of M/G/1 Type

Let $\xi (t)=(\alpha (t),\beta (t))$ be a Md-M/G/1 process and processes $\ell (t)$ and $\delta (t)$ be defined, respectively, as $\ell (t)=\underset{i}{\min }{\alpha }_i(t)$ and $\delta (t)=\alpha (t)-\ell (t)1$. Consider a discrete-time process $\vartheta (t)=(\ell (t),\delta (t),\beta (t))$ on the state space ${\mathbb{Z}}_{+}\times \mathcal{N}\times J$, where the set $\mathcal{N}$ is defined as $\mathcal{N}=\{(n_1,n_2,\dots ,n_M)\in {\mathbb{Z}}_{+}^M|\min n_i=0\}$. The process $\vartheta (t)$ is the Markov chain of M/G/1 type with the phase space $\mathcal{N}\times J$. Mapping $\phi (n,j)=(l,n-l1,j)$, with $l=\min n_i$, is a bijection from ${\mathbb{Z}}_{+}^M\times J$ onto ${\mathbb{Z}}_{+}\times \mathcal{N}\times J$. When the process $\vartheta (t)$ is in a state $(l,a,j)$, the process $\xi (t)=(\alpha (t),\beta (t))$ is in the state ${\phi }^{-1}(l,a,j)=(l1+a,j)$. Therefore, the probability of a one-step transition of $\vartheta (t)$ from $(l_1,a_1,i)$ to $(l_2,a_2,j)$ is given by $p_{l_{1}1+a_1,l_{2}1+a_2}(i,j).$ Element $g((a,i),(b,j))$ of the matrix $G$ of the process $\vartheta (t)$ represents the probability that, starting from the state $(l,a,i)$ $(l\ge 1)$, the process $\vartheta (t)$ will first appear at the level $l-1$ in state $(l-1,b,j)$. It may be expressed by elements ${\phi }_{(k,i),(n,j)}(1)$ of the matrix ${\Phi }_{k,n}(1)$ of the process $\xi (t)$ as

$$g((a,i),(b,j))={\phi }_{(a+1,i),(b,j)}(1), (a,i),(b,j)\in E.$$

(24)

Let us partition the matrix $G$ into blocks of order $n$: $G_{a,b}=[g((a,i),(b,j))]$ $(i,j\in J)$, $a,b\in \mathcal{N},$ As a result of the equality ${\phi }_{(a+1,i),(b,j)}(1)={\psi }_{a,i),(b-1,j)}$, Equation (24) can be expressed in matrix form as

$$G_{a,b}={\Psi }_{a,b-1}.$$

(25)

It implies that the matrices $F_{\beta }={\Psi }_{0,\beta }$ are expressed through the matrices $G_{a,b}$ as

$$F_{\beta }=G_{0,\beta +1}, \beta \in \mathcal{E}.$$

(26)

From (26) and Theorem 2, it follows that

$$G_{a,b}={\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{a,b-1}^{(v)}}{F}_{{\epsilon }_1}{F}_{{\epsilon }_2}\cdots F_{{\epsilon }_v}}}, a,b\in \mathcal{N}.$$

(27)

Hence, the matrices $F_{\beta }$ uniquely define the matrix $G$ and vice versa.

4. Matrices of the Sector Exit Probabilities

As a direct consequence of Theorem 1, we can derive the following representation for the matrices of the sector exit probabilities $F_{\beta }={\Psi }_{0,\beta }$:

$$F_{\beta }=Q_{\beta }+{\displaystyle \sum _{\alpha \ge 0}{Q}_{\alpha }}{\Psi }_{\alpha ,\beta }, \beta \in \mathcal{E}.$$

(28)

By combining Formulas (11) and (12), we obtain the system of nonlinear equations for the matrices $F_{\beta }$ of the sector exit probabilities:

$$F_{\beta }=Q_{\beta }+{\displaystyle \sum _{\alpha \ge 0}{Q}_{\alpha }}{\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{\alpha ,\beta }^{(v)}}{F}_{{\epsilon }_1}{F}_{{\epsilon }_2}\cdots F_{{\epsilon }_v}}}, \beta \in \mathcal{E}.$$

(29)

This system may be solved by successive substitutions, starting with zero matrices.

Theorem 3. 

Let $(X_{\beta }(k), \beta \in \mathcal{E})$, $k\ge 0$, be families of matrices of order $n$, recursively defined by

$$X_{\beta }(0)=O,$$

$$X_{\beta }(k+1)=Q_{\beta }+{\displaystyle \sum _{\alpha \ge 0}{Q}_{\alpha }}{\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{\alpha ,\beta }^{(v)}}{X}_{{\epsilon }_1}(k)\cdots X_{{\epsilon }_v}}(k)}, k\ge 0.$$

(30)

Then, each sequence  $(X_{\beta }(k),k\ge 0)$ is element-wise monotonically increasing and converges to a nonnegative matrix  $X_{\beta }$. The family of matrices  $X_{\beta }$,  $\beta \in \mathcal{E}$, is the minimal solution of the system

$$Y_{\beta }=Q_{\beta }+{\displaystyle \sum _{\alpha \ge 0}{Q}_{\alpha }}{\displaystyle \sum _{v=1}^{\infty } }{\displaystyle \underset{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{\alpha ,\beta }^{(v)}}{\dots }{Y}_{{\epsilon }_1}{Y}_{{\epsilon }_2}\cdots Y_{{\epsilon }_v}}, \beta \in \mathcal{E},$$

(31)

in the set of families $Y_{\beta }$, $\beta \in \mathcal{E}$, of nonnegative matrices of order $n$.

Proof of Theorem 3.

We first demonstrate that the sequences $(X_{\beta }(k),k\ge 0)$, $\beta \in \mathcal{E}$, are monotonically increasing and satisfy $X_{\beta }(k)\le F_{\beta }$ for all $\beta \in \mathcal{E}$ and for all $k\ge 0$. We proceed using induction.

Since $X_{\beta }(0)=O$ and $X_{\beta }(1)=Q_{\beta }$, we know that $X_{\beta }(0)\le X_{\beta }(1)\le F_{\beta }$. Let us assume that $X_{\beta }(k-1)\le X_{\beta }(k)\le F_{\beta }$ for some $k$ and for all $\beta \in \mathcal{E}$. Then, it follows from (30) that

$$X_{\beta }(k+1)=Q_{\beta }+{\displaystyle \sum _{\alpha \ge 0}{Q}_{\alpha }}{\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{\alpha ,\beta }^{(v)}}{X}_{{\epsilon }_1}(k)\cdots X_{{\epsilon }_v}}(k)}$$

$$\ge Q_{\beta }+{\displaystyle \sum _{\alpha \ge 0}{Q}_{\alpha }}{\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{\alpha ,\beta }^{(v)}}{X}_{{\epsilon }_1}(k-1)\cdots X_{{\epsilon }_v}}(k-1)}=X_{\beta }(k)$$

(32)

and

$$X_{\beta }(k+1)=Q_{\beta }+{\displaystyle \sum _{\alpha \ge 0}{Q}_{\alpha }}{\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{\alpha ,\beta }^{(v)}}{X}_{{\epsilon }_1}(k)\cdots X_{{\epsilon }_v}}(k)}$$

$$\le Q_{\beta }+{\displaystyle \sum _{\alpha \ge 0}{Q}_{\alpha }}{\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{\alpha ,\beta }^{(v)}}{F}_{{\epsilon }_1}{F}_{{\epsilon }_2}\cdots F_{{\epsilon }_v}}}=F_{\beta },$$

(33)

which proves the induction step. Thus, each entry in the sequence $X_{\beta }(k)$, $k\ge 0,$ is bounded and monotonically increasing for every $\beta \in \mathcal{E}$. This implies the existence of the limits $X_{\beta }=\underset{k\to \infty }{\lim }{X}_{\beta }(k)\le F_{\beta }$, $\beta \in \mathcal{E}$, satisfying the system (31).

Assume that ${\overline{X}}_{\beta }$, $\beta \in \mathcal{E}$, is another nonnegative solution of (31). We show via induction that $X_{\beta }(k)\le {\overline{X}}_{\beta }$ for all $\beta \in \mathcal{E}$ and all $k\ge 0$. Since ${\overline{X}}_{\beta }\ge O$, we know that $X_{\beta }(0)=O\le {\overline{X}}_{\beta }$ for all $\beta \in \mathcal{E}$. Now, let us assume that $X_{\beta }(k)\le {\overline{X}}_{\beta }$ for some $k$ and all $\beta \in \mathcal{E}$. Then, we obtain inequalities

$$X_{\beta }(k+1)=Q_{\beta }+{\displaystyle \sum _{\alpha \ge 0}{Q}_{\alpha }}{\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{\alpha ,\beta }^{(v)}}{X}_{{\epsilon }_1}(k)\cdots X_{{\epsilon }_v}}(k)}$$

$$\le Q_{\beta }+{\displaystyle \sum _{\alpha \ge 0}{Q}_{\alpha }}{\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{\alpha ,\beta }^{(v)}}{\overline{X}}_{{\epsilon }_1}(k)\cdots {\overline{X}}_{{\epsilon }_v}}(k)}={\overline{X}}_{\beta }(k+1).$$

(34)

Therefore, $X_{\beta }(k)\le {\overline{X}}_{\beta }$ for all $\beta \in \mathcal{E}$ and all $k\ge 0$, which proves the induction step and the minimality property of the family $X_{\beta }$, $\beta \in \mathcal{E}$. □

Note that in one-dimensional cases, when $M=1$, the sets $\mathcal{N}=\left\{0\right\}$ and $\mathcal{E}=\{-1\}$ are singletons, and we have equality $F_{-1}=G_{0,0}=G$. In these cases, a system of matrix Equations (29) consists of a single Equation (1).

5. Example of the Md-QBD Process

Consider a Md-QBD process $\xi (t)=(\alpha (t),\beta (t))$ characterized by having one-step transitions from a state $(k,i)$ restricted to states $(n,j)$ such that $n-k\in \mathcal{C}$, where $\mathcal{C}=\{-e_v,e_v,v=1,2,\dots ,M\}\cup \{e_v-e_w,v,w=1,2,\dots ,M\}$. Matrices of transition probabilities $P_{k,n}$ of such process have the following form:

$$P_{k,n}=O\mathrm{if} n-k\notin \mathcal{C},$$

$$P_{n-e_v,n}=Q_{e_v}, P_{n+e_v,n}=Q_{-e_v}\mathrm{for} v=1,2,\dots ,M,$$

$$P_{n,n+e_v-e_w}=Q_{e_v-e_w}\mathrm{for} v,w=1,2,\dots ,M,$$

where $Q_{-e_v}$, $Q_{e_v}$, $Q_{e_v-e_w}$ are nonnegative square matrices such that

$$Q={\displaystyle \sum _{v=1}^M(Q_{-e_v}+Q_{e_v})+}{\displaystyle \sum _{v=1}^M{\displaystyle \sum _{w=1}^M{Q}_{e_v-e_w}}}$$

is a stochastic matrix. In this case, the representation (28) for the matrices of the sector exit probabilities has the following form:

$$F_{\beta }=Q_{\beta }+Q_0{\Psi }_{0,\beta }+{\displaystyle \sum _{i=1}^M{Q}_{e_i}{\Psi }_{e_i,\beta }}, \beta \in \mathcal{E}.$$

(35)

We show that the matrices ${\Psi }_{\alpha ,\beta }$, $\alpha \ge 0$, $\beta \in \mathcal{E}$ in (35) are nonzero only if the vector $\beta$ has a single negative component. For any vector $w\ge 1$, the element ${\psi }_{(\alpha ,i),(\beta ,j)}$ of the matrix ${\Psi }_{\alpha ,\beta }$ is the conditional probability that the process $\xi (t)$ will visit the state $(w+\beta ,j)$ on the first visit to the set $X^c(w)$, given that it starts in the state $(w+\alpha ,i).$ The probability ${\psi }_{(\alpha ,i),(\beta ,j)}$ is nonzero if and only if there is some state $(w+\gamma ,m)\in X(w)$, the probability of transition from which to the state $(w+\beta ,j)\in X^c(w)$ is positive. For this, it is necessary that the matrix $Q_{\beta -\gamma }$ be nonzero. Since $\gamma \ge 0$ and $\min {\beta }_m=-1$, the vector $\beta -\gamma$ has negative components. Since $Q_{\beta -\gamma }\ne O$, we have either $\beta =\gamma -e_w$ and ${\gamma }_w=0$ for some $w$, or $\beta =\gamma +e_v-e_w$ and ${\gamma }_w=0$ for some $w$ and $v\ne w$. In both cases, the vector $\beta$ has a single negative component ${\beta }_w=-1$.

Let us denote by ${\mathcal{E}}_1$ the set of vectors $\beta \in \mathcal{E}$ with a single negative component and by ${\mathcal{E}}_0=\mathcal{E}\backslash {\mathcal{E}}_1$ the set of vectors with a large number of negative components. Given the above, the matrices ${\Psi }_{\alpha ,\beta }$ are zero for all $\alpha \ge 0$ and $\beta \in {\mathcal{E}}_0$. In addition, for $\beta \in {\mathcal{E}}_0$ matrices $Q_{\beta }$ are also zero. Therefore, as follows from equality (35), the matrices $F_{\beta }$, $\beta \in {\mathcal{E}}_0,$ are also zero. After removing the zero matrices, system (29) acquires the following form:

$$F_{\beta }=Q_{\beta }+Q_0{\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\overline{\mathrm{E}}}_{0,\beta }^{(v)}}{F}_{{\epsilon }_1}{F}_{{\epsilon }_2}\cdots F_{{\epsilon }_v}}}$$

$$+{\displaystyle \sum _{i=1}^M{Q}_{e_i}{\displaystyle \sum _{v=1}^{\infty }{\displaystyle \sum _{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\overline{\mathrm{E}}}_{e_i,\beta }^{(v)}}{F}_{{\epsilon }_1}{F}_{{\epsilon }_2}\cdots F_{{\epsilon }_v}}}}, \beta \in {\mathcal{E}}_1,$$

(36)

where the sets ${\overline{\mathrm{E}}}_{\alpha ,\beta }^{(v)}$ are defined as

$${\overline{\mathrm{E}}}_{\alpha ,\beta }^{(v)}=\{({\epsilon }_1,\dots ,{\epsilon }_v)\in {\mathrm{E}}_{\alpha ,\beta }^{(v)}|{\epsilon }_1,\dots ,{\epsilon }_v\in {\mathcal{E}}_1\}.$$

(37)

Specific cases of $M\mathrm{d}-\mathrm{QBD}$ processes, where only one component of the vector $\alpha (t)$ can change at a time, were studied in [16]. In these cases, the matrices $Q_{e_v-e_w}$ are zero for all indices $v\ne w$. Equation (18) in [16] applies only when $M=1$ as there is an error in the proof of Theorem 3. As a result, the findings presented in Section 3 of the mentioned work that rely on this theorem are only valid in the one-dimensional case. For multidimensional $M\mathrm{d}-\mathrm{QBD}$ processes, Formula (27) of this article provides the correct matrix-multiplicative representation of the matrix $G$ through matrices of order $n$, utilizing the solution of the system (36).

6. Conclusions and Future Work

This study presents several theoretical results that aim to simplify the analysis of multidimensional Markov chains of M/G/1 type. The novelty is that we have introduced new concepts of the state sectors and the sector exit probabilities. We have been able to demonstrate that Equation (1), which involves a matrix $G$, can be replaced by a system of Equation (29) that utilizes a family of matrices $F_{\beta }$ of the sector exit probabilities. Entries of the matrix $G$ are indexed by elements of the set $\mathcal{N}\times J$, where $\mathcal{N}=\{(k_1,k_2,\dots ,k_M)\in {\mathbb{Z}}_{+}^M|\min k_i=0\}$ and $J=\{1,2,\dots ,n\}$, while the family of the matrices $F_{\beta }$ of the order $n$ is indexed by the set $\mathcal{E}=\{(k_1,k_2,\dots ,k_M)\in {\mathbb{Z}}^M|\min k_i=-1\}.$ In one-dimensional cases, the results of the article are reduced to the existing results by Neuts in [2].

However, there are several challenges to overcome in practically implementing these results. In multidimensional cases, the family of the matrices $F_{\beta }$ is infinite. Since it is not feasible to compute infinite families of matrices, future research should focus on developing a method to select an appropriate truncation approximation for the proposed algorithm and conducting complexity and error analyses. It remains unclear whether in multi-dimensional cases the family of matrices $F_{\beta }$ is the minimal nonnegative solution to the system (29). Future research should also address this question.