Next Article in Journal
Exploring the Structure of Possibility Multi-Fuzzy Soft Ordered Semigroups Through Interior Ideals
Next Article in Special Issue
Accurate Approximation for Resource Queuing Systems with Losses and Signals
Previous Article in Journal
Persistent Topological Laplacians—A Survey
Previous Article in Special Issue
Scaling Limits of a Tandem Queue with Two Infinite Orbits
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Multi-Dimensional Markov Chains of M/G/1 Type

by
Valeriy Naumov
1,* and
Konstantin Samouylov
2
1
Service Innovation Research Institute, Annankatu 8 A, 00120 Helsinki, Finland
2
Institute of Computer Science and Telecommunications, RUDN University, 6 Miklukho-Maklaya St., Moscow 117198, Russia
*
Author to whom correspondence should be addressed.
Mathematics 2025, 13(2), 209; https://doi.org/10.3390/math13020209
Submission received: 30 November 2024 / Revised: 6 January 2025 / Accepted: 6 January 2025 / Published: 9 January 2025
(This article belongs to the Special Issue Queue and Stochastic Models for Operations Research, 3rd Edition)

Abstract

:
We consider an irreducible discrete-time Markov process with states represented as (k, i) where k is an M-dimensional vector with non-negative integer entries, and i indicates the state (phase) of the external environment. The number n of phases may be either finite or infinite. One-step transitions of the process from a state (k, i) are limited to states (n, j) such that nk1, where 1 represents the vector of all 1s. We assume that for a vector k1, the one-step transition probability from a state (k, i) to a state (n, j) may depend on i, j, and n − k, but not on the specific values of k and n. This process can be classified as a Markov chain of M/G/1 type, where the minimum entry of the vector n defines the level of a state (n, j). It is shown that the first passage distribution matrix of such a process, also known as the matrix G, can be expressed through a family of nonnegative square matrices of order n, which is a solution to a system of nonlinear matrix equations.

1. Introduction

The Markov chains of M/G/1 type represent a basic model that explains the temporal evolution of population size. It is a two-component process, the first component of which is an integer-valued process describing the dynamics of population size, and the other represents the state (phase) of the external environment. The state space of Markov chains of M/G/1 type can be partitioned into non-empty disjoint subsets known as levels, such that one-step state transitions are limited to states at the same level, the adjacent lower level, or any higher level. The transition matrix of a Markov chain of M/G/1 type has a block upper Hessenberg form and all blocks along the main diagonal, except for the top ones, are identical.
The matrix geometric method proposed by M.F. Neuts in [1] and his study of Markov chains of M/G/1 type in [2] led to the rapid development of matrix-analytic methods in operations research. These methods provide a powerful framework for the unified analysis of large classes of Markov processes and, more importantly, for their numerical solution. Matrix-analytic methods and their applications as they stand today are outlined in [3,4,5,6,7,8,9,10,11].
In this paper, we study discrete-time Markov chains  ξ ( t ) = ( α ( t ) , β ( t ) )  on the state space  X = + M × J , where  +  is the set of nonnegative integers and  J = { 1 , 2 , , n } . The number of elements in  J  may be either finite or infinite. One-step transitions of the process from a state  ( k , i )  are limited to states  ( n , j )  such that  n k 1 ,  where  1  represents the vector of all 1s. We assume that the process  ξ ( t )  is spatially homogeneous, meaning that for  k 1  the one-step transition probability from a state  ( k , i )  to a state  ( n , j )  may depend on  i , j , and  n k  but not on the specific values of  k  and  n .  We will refer to these processes as M-dimensional Markov chains of M/G/1 type  ( M d - M / G / 1 ) . The  M d - M / G / 1  processes are Markov chains of M/G/1 type, with the level  l  consisting of states  ( k 1 , k 2 , , k M , j )  for which  min k i = l .
Multidimensional quasi-birth-and-death processes ( M d - QBD ) are a specific type of multidimensional Markov chains of M/G/1 type. They are characterized by having one-step transitions from a state  ( k , i )  restricted to states  ( n , j )  such that  n k { 1 , 0 , 1 } M . The explicit analytical representation for the stationary distribution of  M d - QBD  processes are unknown, and most work has been devoted to deriving asymptotic formulas of the stationary distribution (see [12,13,14]). The conditions ensuring a positive recurrent or transient 2d-QBD process were analyzed in [15]. Specific cases of  M d - QBD  processes, where only one component of the vector  α ( t )  may change at a time, were studied in [16].
In the theory of Markov chains of M/G/1 type, a crucial role is played by the so-called matrix  G  [2]. If it is known, many relevant quantities may be efficiently computed. Different algorithms for the numerical computation of the matrix  G  have been proposed [2,17,18,19,20,21]. However, the structure of the levels of  M d - M / G / 1  processes can make it challenging to use these methods effectively.
This study aims to describe the matrix  G  of  M d - M / G / 1  processes in terms of matrices of order  n . Section 2 explores classical Markov chains of M/G/1 type. Section 3 defines the multidimensional Markov chains of M/G/1 type and introduces the concept of the state sectors and the sector exit probabilities. We analyze the first passage probabilities and demonstrate that the family of matrices representing the sector exit probabilities satisfies a system of nonlinear matrix equations. Section 4 illustrates how a solution to this system can be obtained through successive substitution. An example of the Md-QBD process is discussed in Section 5. Finally, we offer some concluding remarks in Section 6.
We use bold capital letters to denote matrices and bold lowercase letters to denote vectors. Unless otherwise stated, all vectors in this paper have integer components and the length  M . For any vector  x  we use the notation  x i  for the  i th  component of  x . For vectors  x = ( x 1 , x 2 , , x M )  and  y = ( y 1 , y 2 , , y M ) x y  means that  x j y j  for all  j , and  x y  means that  x j < y j  for at least one value of  j . Notations  x y  and  x y  are defined similarly. Vector  1  represents the vector of all 1s, and the vector  e m  indicates the vector with zero entries except the mth entry, which equals one. Given a vector  w 1  and an integer  k 1 , we define two sets  Z k ( w ) = { n M | min ( n i w i ) = k }  and  X k ( w ) = Z k ( w ) × J . Sets  Z ( w ) X ( w ) , and  X c ( w )  are defined as  Z ( w ) = { n Z M | n w } X ( w ) = Z ( w ) × J , and  X c ( w ) = X \ X ( w ) .  We refer to the set of states  X ( w )  as the sector.

2. Markov Chains of M/G/1 Type

A Markov chain of M/G/1 type with a state space  + × J  is characterized by a block Hessenberg transition matrix of the form
P = B 0 B 1 B 2 B 3 A 1 A 0 A 1 A 2 O A 1 A 0 A 1 O O A 1 A 0 ,
where blocks  B i  and  A i  are nonnegative square matrices, such that  i = 0 B i  and  i = 1 A i  are stochastic matrices. Entries of the matrix  P  blocks are indexed by the elements of the phase space  J . The subset of all states  ( l , i ) , such that  i J , is called the level  l .
A fundamental role in the theory of Markov chains of M/G/1 type is played by the matrix  G , described by Neuts in [2]. The element  g ( i , j )  of this matrix represents the probability that, starting from the state  ( l , i )   ( l 1 ) , the chain will first appear at the level  l 1  in state  ( l 1 , j ) . Neuts demonstrated that  G  is the minimal nonnegative solution to the equation
G = v = 0 A v 1 G v .
There are several algorithms for calculating the invariant measures of the transition matrix  P  using the matrix  G  [22,23,24].

3. Multidimensional Processes of M/G/1 Type

Consider an irreducible discrete-time Markov chain  ξ ( t ) = ( α ( t ) , β ( t ) )  on the state space  X = + M × J . Let us denote the probability of a one-step transition from  ( k , i )  to  ( n , j )  as  p k , n ( i , j ) . We assume that the transition probability matrix  P = [ p k , n ( i , j ) ]   ( k , n + M i , j J ) , partitioned into blocks  P k , n  of order  n , for all  k 1  has the following properties:
P k , n = Q n k , n k 1 , O , n k 1 ,
where  Q r r 1 , are nonnegative square matrices such that  Q = r 1 Q r  is a stochastic matrix. We refer to this process as an M-dimensional Markov chain of M/G/1 type  ( M d - M / G / 1 ) .

3.1. Sectors of the Process States

Let us define the sequence of passage times as follows:
θ 0 = 0 , θ k + 1 = min { t > θ k | ξ ( t ) X c ( α ( θ k ) } , k 0 .
We say that at time  t , the process  ξ ( t ) = ( α ( t ) , β ( t ) )  is in the sector  X ( r )  if  θ k t < θ k + 1  and if  α ( θ k ) = r . For a vector  w 1 , we define the sector  X ( w )  exit time as the moment  τ w  when the process first enters the set  X c ( w ) ,
τ w = min { t 0 | ξ ( t ) X c ( w ) } .
Additionally, we define the number of sectors visited along a path to  X c ( w )  as
κ w = max { k 0 | θ k τ w } .
If an initial state  ξ ( 0 )  of the process belongs to  X c ( w ) , then we have  τ w = 0  and  κ w = 0 .  If  ξ ( 0 ) = ( k , i )  with  k w , then at the first hitting time of  X c ( w ) , the process exits the sector  X ( α ( θ κ w 1 ) )  and enters the sector  X ( α ( θ κ w ) ) , which implies equality  τ w = θ κ w . The set  X c ( w )  is reached at the moment of transition from the set  X 0 ( α ( θ κ w 1 ) ) X 0 ( w )  to the state  ξ ( θ κ w ) X 1 ( α ( θ κ w 1 ) ) X 1 ( w ) .
For vectors  w 1 k X ( w ) , and  n X c ( w )  we define matrices  Φ k , n ( v ) ( w ) = [ φ ( k , i ) , ( r , j ) ( v ) ( w ) ]   ( i , j J )  and  Φ k , n ( w ) = [ φ ( k , i ) , ( n , j ) ( w ) ]   ( i , j J )  as follows.
The element  φ ( k , i ) , ( n , j ) ( v ) ( w )  of the matrix  Φ k , n ( v ) ( w )  is the conditional probability that the process  ξ ( t ) , starting in the state  ( k , i ) X ( w ) , reaches the set  X c ( w )  by hitting the state  ( n , j )  after passing through exactly  v  sectors,
φ ( k , i ) , ( n , j ) ( v ) ( w ) = { κ w = v , ξ ( θ κ w ) = ( n , j ) | ξ ( 0 ) = ( k , i ) }  
= { κ w = v , ξ ( τ w ) = ( n , j ) | ξ ( 0 ) = ( k , i ) }   .
The element  φ ( k , i ) , ( n , j ) ( w )  of the matrix  Φ k , n ( w )  is the conditional probability that the process  ξ ( t )  will eventually hit the set  X c ( w )  in the state  ( n , j ) , given that it starts in the state  ( k , i ) ,
φ ( k , i ) , ( n , j ) ( w ) = { κ w < , ξ ( θ κ w ) = ( n , j ) | ξ ( 0 ) = ( k , i ) }   = { ξ ( τ w ) = ( n , j ) | ξ ( 0 ) = ( k , i ) }   .
It is clear that matrices  Φ k , n ( w )  and  Φ k , n ( v ) ( w ) v = 1 , 2 ,  are related to each other by the equality
Φ k , n ( w ) = v = 1 Φ k , n ( v ) ( w ) .
For  l 0 , any path of  ξ ( t )  leading from a state  ( k , i ) X l ( w )  to a state  ( n , j ) X c ( w )  must successively visit sets  X l 1 ( w ) , X l 2 ( w ) , , X 0 ( w ) , X 1 ( w ) , which will require visiting at least  l + 1  sectors. Therefore, we have
Φ k , n ( v ) ( w ) = O ,   for   all   k Z l ( w ) ,   n Z 1 ( w ) and   v l .
Since the process  ξ ( t )  is spatially homogeneous, for any vector  k w 1  and any vector  n Z 1 ( w ) , the probabilities  φ ( k , i ) , ( n , j ) ( w )  may depend on  i , j ,   k w , and  n w  but not on the specific values of  k , n  and  w , i.e.,  Φ k , n ( w ) = Φ k w + 1 , n w + 1 ( 1 ) .  This means that the matrix  Φ k , n ( w )  may be expressed as
Φ k , n ( w ) = Ψ k w , n w ,
where the matrix  Ψ α , β  is defined as
Ψ α , β = Φ w + α , w + β ( w )
independently of the vector  w 1 .
The element  ψ ( α , i ) , ( β , j )  of the matrix  Ψ α , β  is the conditional probability that the process  ξ ( t )  will hit the state  ( w + β , j )  on the first visit to the set  X c ( w ) , given that it starts in the state  ( w + α , i ) .  Here, the vectors  α  and  β  satisfy conditions  w + α w  and  w + β Z 1 ( w ) . Therefore, the index  α  of a matrix  Ψ α , β  is a nonnegative vector and its index  β  belongs to the set  E  defined as
E = { β M | min β m = 1 } .
We refer to the matrices  Ψ α , β  as the matrices of the first passage probabilities.
The matrices  Φ k , n ( k ) = Ψ 0 , n k  determine the transition probabilities of the embedded Markov chain  ξ ( θ k ) , since for  k 1  we have
φ ( k , i ) , ( n , j ) ( k ) = { ξ ( τ k ) = ( n , j ) | ξ ( 0 ) = ( k , i ) }  
= { ξ ( θ 1 ) = ( n , j ) | ξ ( 0 ) = ( k , i ) } = { ξ ( θ k ) = ( n , j ) | ξ ( θ k 1 ) = ( k , i ) } .
We define matrices  F β  as
F β = Ψ 0 , β ,   β E ,
and refer to these matrices as the matrices of the sector exit probabilities.

3.2. Matrices of the First Passage Probabilities

Theorem 1.
The matrices of the first passage probabilities  Ψ α , β  satisfy the system
Ψ α , β = Q β α + γ 0 Q γ α Ψ γ , β ,   α 0 , β E .
Proof of Theorem 1. 
We will initially demonstrate the validity of the following equality for all vectors  k Z ( w )  and  n Z 1 ( w ) :
Φ k , n ( w ) = P k , n + r w P k , r Φ r , n ( w ) .
This formula adheres to the law of total probability, taking into account all possible states of the process following the first transition. Consider two states:  ( k , i ) X ( w )  and  ( n , j ) X 1 ( w ) . The state  ( n , j )  can be reached from the state  ( k , i )  after a single transition. This contributes to (12) the term  P k , n .The first transition may take the process to some state  ( r , j )  with  r w . To reach the set  X 1 ( w )  from state  ( r , j ) , the process  ξ ( t )  must necessarily cross one or more sectors and then hit the state  ( n , j ) X 1 ( w ) . This contributes to the second term on the right-hand side of (12). Equation (10) for the matrices  Ψ α , β  is derived from (12) using Formulas (2) and (7). □
For vectors  α 0  and  β E , let the set  E α , β ( v )  be defined as the set of all  v  tuples  ( ε 1 , , ε v ) E v , satisfying  α + ε 1 0 ,   α + ε 1 + ε 2 0 , …,  α + ε 1 + + ε v 1 0 , and  α + ε 1 + + ε v = β , and let the set  E α , β  be defined as  E α , β = v = 1 E α , β ( v ) .
Any nonnegative vector  α  can be represented as  α = δ + k 1 , with  k = min α m δ N ,  and the set  N  being defined as  N = { n M | min n m = 0 } . Since the inequality  ε 1  holds for all vectors  ε E , only elements  ε 1 , , ε v  of the set  E  in quantity  v > k  can satisfy the condition  δ + k 1 + ε 1 + + ε v E . Therefore, for any  δ N β E  and  k v 1 , the set  E δ + k 1 , β ( v )  is empty. If this set is not empty, then the following decomposition is valid in terms of the Cartesian products of sets of the form  E γ , ε ( u ) , where  γ N ε E , and  u 1 ,
Lemma 1. 
For any vectors  δ N ,  β E  and an integer  k 1  the sets  E δ + k 1 , β ( v ) ,  v 1 , and the set  E δ + k 1 , β  can be represented as follows:
E δ + k 1 , β ( v ) = u 1 , , u k + 1 1 u 1 + + u k + 1 = v   γ 1 , , γ k N E δ , γ 1 1 ( u 1 ) × E γ 1 , γ 2 1 ( u 2 ) × × E γ k 1 , γ k 1 ( u k ) × E γ k , β ( u k + 1 ) ,
E δ + k 1 , β = γ 1 , , γ k N E δ , γ 1 1 × E γ 1 , γ 2 1 × × E γ k 1 , γ k 1 × E γ k , β .
Proof of Lemma 1. 
Since  δ + l 1 Z l 1 ( 1 )  and  β Z 1 ( 1 ) , for any  v  tuple  ( ε 1 , , ε v ) E δ + l 1 , β ( v )  of the length  v > l 1 , in the interval  1 w < v  there necessarily exist numbers  w , such that  δ + l 1 + ε 1 + + ε w Z l 2 ( 1 ) . Let us denote by  u  the minimum of such numbers. The number  v u  of the elements  ε u + 1 , ε u + 2 , , ε v , satisfying the condition  ( δ + l 1 + ε 1 + + ε u ) + ε u + 1 + + ε v = β , must be at least  l . Therefore, the set  E δ + l 1 , β ( v )  can be represented as follows:
E δ + l 1 , β ( v ) = u = 1 v l { ( ε 1 , , ε v ) E | δ + l 1 + ε 1 l 1 , δ + l 1 + ε 1 + ε 2 l 1 ,
, δ + l 1 + ε 1 + + ε u 1 l 1 , δ + l 1 + ε 1 + + ε u Z 1 ( l 1 ) ,
δ + l 1 + ε 1 + + ε u + 1 0 , , δ + l 1 + ε 1 + + ε v 1 0 ,
δ + l 1 + ε 1 + + ε v = β } .
Each element  z  of the set  Z 1 ( l 1 )  can be written as  z = γ + ( l 1 ) 1 ,  where  γ N . Therefore, Equation (15) implies the following equalities:
E δ + l 1 , β ( v ) = u = 1 v l γ N { ( ε 1 , , ε v ) E | δ + l 1 + ε 1 l 1 , δ + l 1 + ε 1 + ε 2 l 1 ,
, δ + l 1 + ε 1 + + ε u 1 l 1 , δ + l 1 + ε 1 + + ε u = γ + ( l 1 ) 1 ,
δ + l 1 + ε 1 + + ε u + 1 0 , , δ + l 1 + ε 1 + + ε v 1 0 ,
δ + l 1 + ε 1 + + ε v = β }
= u = 1 v l γ N { ( ε 1 , , ε v ) E | δ + ε 1 0 , δ + ε 1 + ε 2 0 , , δ + ε 1 + + ε u 1 0 ,
δ + ε 1 + + ε u = γ 1 , γ + ( l 1 ) 1 + ε u + 1 0 , γ + ( l 1 ) 1 + ε u + 1 + ε u + 2 0 , ,
γ + ( l 1 ) 1 + ε u + 1 + + ε v 1 0 , γ + ( l 1 ) 1 + ε u + 1 + + ε v = β }
= u = 1 v l γ N E δ , γ 1 ( u ) × E γ + ( l 1 ) 1 , β ( v u ) ,
Thus, for any integers  v > l 1 , the following formula is valid
E δ + l 1 , β ( v ) = u = 1 v l γ N E δ , γ 1 ( u ) × E γ + ( l 1 ) 1 , β ( v u ) .
Applying this formula for  l = k , k 1 , , 1 , we can obtain the decomposition (13) for the set  E δ + k 1 , β ( v ) .
Introducing a new variable  w = v u  in (17) and changing the order of summation, we obtain the following result:
E δ + l 1 , β , β = u = 1 w = l γ N E δ , γ 1 ( u ) × E γ + ( l 1 ) 1 , β ( w ) = γ N E δ , γ 1 × E γ + ( l 1 ) 1 , β .
Applying this formula for  l = k , k 1 , , 1 , we can achieve the decomposition (14) for the set  E δ + k 1 , β , which proves Lemma 1. □
Theorem 2. 
For  α 0  and  β E , the matrices  Ψ α , β  are given by
Ψ α , β = v = 1 ( ε 1 , , ε v ) E α , β ( v ) F ε 1 F ε 2 F ε v .
Proof of Theorem 2. 
Entries of matrices  Φ r , n ( r )  are the one-step transition probabilities (9) of the embedded Markov chain  ξ ( θ k ) , while entries of the matrices  Φ k , n ( v ) ( w )  are the probabilities of reaching the set  X c ( w )  after  v  steps of this Markov chain. Therefore, for all  k Z ( w )  and  n Z 1 ( w ) , the matrices  Φ k , n ( v ) ( w )  are completely determined by the matrices  Φ r , n ( r ) r w , as follows:
Φ k , n ( 1 ) ( w ) = Φ k , n ( k ) ,
Φ k , n ( u + 1 ) ( w ) = ( r 1 , , r u ) T k , n ( u ) ( w ) Φ k , r 1 ( k ) Φ r 1 , r 2 ( r 1 ) Φ r u 1 , r u ( r u 1 ) Φ r u , n ( r u ) ,   u 1 .
Here, the summation extends over the set  T k , n ( u ) ( w )  of all  u tuples  ( r 1 , , r u ) , satisfying  r 1 , , r u Z ( w ) r 1 Z 1 ( k ) r 2 Z 1 ( r 1 ) ,…,  r u Z 1 ( r u 1 ) n Z 1 ( r u ) .
From (5) and (19), it follows that for all  k Z ( w )  and  n Z 1 ( w ) , we have
Φ k , n ( w ) = v = 1 Φ k , n ( v ) ( w )
= Φ k , n ( k ) + u = 1 ( r 1 , , r u ) T k , n ( u ) ( w ) Φ k , r 1 ( k ) Φ r 1 , r 2 ( r 1 ) Φ r u 1 , r u ( r u 1 ) Φ r u , n ( r u )
= F n k ( k ) + u = 1 ( r 1 , , r u ) T k , n ( u ) ( w ) F r 1 k F r 2 r 1 F r u r u 1 F n r u .
Using Formulas (7) and (20), we can obtain the matrix  Ψ α , β  as
Ψ α , β = Φ α + 1 , β + 1 ( 1 )
= F β α + u = 1 ( r 1 , , r u ) T α + 1 , β + 1 ( u ) ( 1 ) F r 1 α 1 F r 2 r 1 F r u r u 1 F β + 1 r u .
Let us introduce new variables:  v = u + 1 ε 1 = r 1 α 1 ε 2 = r 2 r 1 , …,  ε v 1 = r v 1 r v 2 ,  and  ε v = β + 1 r v 1 . From the definition of the set  T α + 1 , β + 1 ( u ) ( 1 ) , it follows that vectors  ε 1 , , ε v  belong to the set  E  and satisfy the following conditions:  α + ε 1 0 α + ε 1 + ε 2 0 ,  …, and  α + ε 1 + + ε v 1 0 . Taking into account that  α + ε 1 + + ε v = β , we finally obtain (18). □
It follows from Lemma 1 and Theorem 2 that matrices  Ψ α , β  with arbitrary nonnegative index  α  can be expressed in terms of the matrices  Ψ δ , γ , with  δ N .
Corollary 1. 
For any vectors  δ N ,  β E  and integers  k 1 , the matrix  Ψ δ + k 1 , β  can be represented as
Ψ δ + k 1 , β = γ 1 , , γ k N Ψ δ , γ 1 1 Ψ γ 1 , γ 2 1 Ψ γ k 1 , γ k 1 Ψ γ k , β .
Proof of Corollary 1. 
It follows from (13) and (18) that the matrix
Ψ δ + k 1 , β = v = 1 ( ε 1 , , ε v ) E δ + k 1 , β ( v ) F ε 1 F ε 2 F ε v
can be represented as follows:
Ψ δ + k 1 , β = v = 1 u 1 , , u k + 1 1 u 1 + + u k + 1 = v γ 1 , , γ k N ( ( ε 1 , 1 , , ε 1 , u 1 ) E δ , γ 1 1 ( u 1 ) F ε 1.1 F ε 1 , u 1 )
( ( ε 2 , 1 , , ε 2 , u 2 ) E γ 1 , γ 2 1 ( u 2 ) F ε 2.1 F ε 2 , u 2 ) ( ( ε k , 1 , , ε k , u k ) E γ k 1 , γ k 1 ( u k ) F ε k .1 F ε k , u k )
( ( ε k + 1 , 1 , , ε k + 1 , u k + 1 ) E γ k , β ( u k + 1 ) F ε k + 1.1 F ε k + 1 , u k + 1 )
= γ 1 , , γ k N [ u 1 = 1 ( ( ε 1 , 1 , , ε 1 , u 1 ) E δ , γ 1 1 ( u 1 ) F ε 1.1 F ε 1 , u 1 )
u 2 = 1 ( ( ε 2 , 1 , , ε 2 , u 2 ) E γ 1 , γ 2 1 ( u 2 ) F ε 2.1 F ε 2 , u 2 )
u k = 1 ( ( ε k , 1 , , ε k , u k ) E γ k 1 , γ k 1 ( u k ) F ε k .1 F ε k , u k )
u k + 1 = 1 ( ( ε k + 1 , 1 , , ε k + 1 , u k + 1 ) E γ k , β ( u k + 1 ) F ε k + 1.1 F ε k + 1 , 2 F ε k + 1 , u k + 1 ) ] .
After applying Formula (18) to each sum inside the square brackets in (23), we obtain Formula (22), which proves Corollary 1. □

3.3. Md-M/G/1 Processes as One-Dimensional Markov Chains of M/G/1 Type

Let  ξ ( t ) = ( α ( t ) , β ( t ) )  be a Md-M/G/1 process and processes  ( t )  and  δ ( t )  be defined, respectively, as  ( t ) = min i α i ( t )  and  δ ( t ) = α ( t ) ( t ) 1 . Consider a discrete-time process  ϑ ( t ) = ( ( t ) , δ ( t ) , β ( t ) )  on the state space  + × N × J , where the set  N  is defined as  N = { ( n 1 , n 2 , , n M ) + M | min n i = 0 } . The process  ϑ ( t )  is the Markov chain of M/G/1 type with the phase space  N × J . Mapping  φ ( n , j ) = ( l , n l 1 , j ) , with  l = min n i , is a bijection from  + M × J  onto  + × N × J . When the process  ϑ ( t )  is in a state  ( l , a , j ) , the process  ξ ( t ) = ( α ( t ) , β ( t ) )  is in the state  φ 1 ( l , a , j ) = ( l 1 + a , j ) . Therefore, the probability of a one-step transition of  ϑ ( t )  from  ( l 1 , a 1 , i )  to  ( l 2 , a 2 , j )  is given by  p l 1 1 + a 1 , l 2 1 + a 2 ( i , j ) .  Element  g ( ( a , i ) , ( b , j ) )  of the matrix  G  of the process  ϑ ( t )  represents the probability that, starting from the state  ( l , a , i )   ( l 1 ) , the process  ϑ ( t )  will first appear at the level  l 1  in state  ( l 1 , b , j ) . It may be expressed by elements  φ ( k , i ) , ( n , j ) ( 1 )  of the matrix  Φ k , n ( 1 )  of the process  ξ ( t )  as
g ( ( a , i ) , ( b , j ) ) = φ ( a + 1 , i ) , ( b , j ) ( 1 ) ,   ( a , i ) , ( b , j ) E .
Let us partition the matrix  G  into blocks of order  n G a , b = [ g ( ( a , i ) , ( b , j ) ) ]   ( i , j J ) a , b N ,  As a result of the equality  φ ( a + 1 , i ) , ( b , j ) ( 1 ) = ψ a , i ) , ( b 1 , j ) , Equation (24) can be expressed in matrix form as
G a , b = Ψ a , b 1 .
It implies that the matrices  F β = Ψ 0 , β  are expressed through the matrices  G a , b  as
F β = G 0 , β + 1 ,   β E .
From (26) and Theorem 2, it follows that
G a , b = v = 1 ( ε 1 , , ε v ) E a , b 1 ( v ) F ε 1 F ε 2 F ε v ,   a , b N .
Hence, the matrices  F β  uniquely define the matrix  G  and vice versa.

4. Matrices of the Sector Exit Probabilities

As a direct consequence of Theorem 1, we can derive the following representation for the matrices of the sector exit probabilities  F β = Ψ 0 , β :
F β = Q β + α 0 Q α Ψ α , β ,   β E .
By combining Formulas (11) and (12), we obtain the system of nonlinear equations for the matrices  F β  of the sector exit probabilities:
F β = Q β + α 0 Q α v = 1 ( ε 1 , , ε v ) E α , β ( v ) F ε 1 F ε 2 F ε v ,   β E .
This system may be solved by successive substitutions, starting with zero matrices.
Theorem 3. 
Let  ( X β ( k ) ,   β E ) k 0 , be families of matrices of order  n , recursively defined by
X β ( 0 ) = O ,
X β ( k + 1 ) = Q β + α 0 Q α v = 1 ( ε 1 , , ε v ) E α , β ( v ) X ε 1 ( k ) X ε v ( k ) ,   k 0 .
Then, each sequence   ( X β ( k ) , k 0 ) is element-wise monotonically increasing and converges to a nonnegative matrix   X β . The family of matrices   X β ,   β E , is the minimal solution of the system
Y β = Q β + α 0 Q α v = 1   ( ε 1 , , ε v ) E α , β ( v ) Y ε 1 Y ε 2 Y ε v ,   β E ,
in the set of families  Y β ,  β E , of nonnegative matrices of order n .
Proof of Theorem 3.
We first demonstrate that the sequences  ( X β ( k ) , k 0 ) β E , are monotonically increasing and satisfy  X β ( k ) F β  for all  β E  and for all  k 0 . We proceed using induction.
Since  X β ( 0 ) = O  and  X β ( 1 ) = Q β , we know that  X β ( 0 ) X β ( 1 ) F β . Let us assume that  X β ( k 1 ) X β ( k ) F β  for some  k  and for all  β E . Then, it follows from (30) that
X β ( k + 1 ) = Q β + α 0 Q α v = 1 ( ε 1 , , ε v ) E α , β ( v ) X ε 1 ( k ) X ε v ( k )
Q β + α 0 Q α v = 1 ( ε 1 , , ε v ) E α , β ( v ) X ε 1 ( k 1 ) X ε v ( k 1 ) = X β ( k )
and
X β ( k + 1 ) = Q β + α 0 Q α v = 1 ( ε 1 , , ε v ) E α , β ( v ) X ε 1 ( k ) X ε v ( k )
Q β + α 0 Q α v = 1 ( ε 1 , , ε v ) E α , β ( v ) F ε 1 F ε 2 F ε v = F β ,
which proves the induction step. Thus, each entry in the sequence  X β ( k ) k 0 ,  is bounded and monotonically increasing for every  β E . This implies the existence of the limits  X β = lim k X β ( k ) F β β E , satisfying the system (31).
Assume that  X ¯ β β E , is another nonnegative solution of (31). We show via induction that  X β ( k ) X ¯ β  for all  β E  and all  k 0 . Since  X ¯ β O , we know that  X β ( 0 ) = O X ¯ β  for all  β E . Now, let us assume that  X β ( k ) X ¯ β  for some  k  and all  β E . Then, we obtain inequalities
X β ( k + 1 ) = Q β + α 0 Q α v = 1 ( ε 1 , , ε v ) E α , β ( v ) X ε 1 ( k ) X ε v ( k )
Q β + α 0 Q α v = 1 ( ε 1 , , ε v ) E α , β ( v ) X ¯ ε 1 ( k ) X ¯ ε v ( k ) = X ¯ β ( k + 1 ) .
Therefore,  X β ( k ) X ¯ β  for all  β E  and all  k 0 , which proves the induction step and the minimality property of the family  X β β E . □
Note that in one-dimensional cases, when  M = 1 , the sets  N = { 0 }  and  E = { 1 }  are singletons, and we have equality  F 1 = G 0 , 0 = G . In these cases, a system of matrix Equations (29) consists of a single Equation (1).

5. Example of the Md-QBD Process

Consider a Md-QBD process  ξ ( t ) = ( α ( t ) , β ( t ) )  characterized by having one-step transitions from a state  ( k , i )  restricted to states  ( n , j )  such that  n k C , where  C = { e v , e v , v = 1 , 2 , , M } { e v e w , v , w = 1 , 2 , , M } . Matrices of transition probabilities  P k , n  of such process have the following form:
P k , n = O if   n k C ,
P n e v , n = Q e v ,   P n + e v , n = Q e v for   v = 1 , 2 , , M ,
P n , n + e v e w = Q e v e w for   v , w = 1 , 2 , , M ,
where  Q e v Q e v Q e v e w  are nonnegative square matrices such that
Q = v = 1 M ( Q e v + Q e v ) + v = 1 M w = 1 M Q e v e w
is a stochastic matrix. In this case, the representation (28) for the matrices of the sector exit probabilities has the following form:
F β = Q β + Q 0 Ψ 0 , β + i = 1 M Q e i Ψ e i , β ,   β E .
We show that the matrices  Ψ α , β α 0 β E  in (35) are nonzero only if the vector  β  has a single negative component. For any vector  w 1 , the element  ψ ( α , i ) , ( β , j )  of the matrix  Ψ α , β  is the conditional probability that the process  ξ ( t )  will visit the state  ( w + β , j )  on the first visit to the set  X c ( w ) , given that it starts in the state  ( w + α , i ) .  The probability  ψ ( α , i ) , ( β , j )  is nonzero if and only if there is some state  ( w + γ , m ) X ( w ) , the probability of transition from which to the state  ( w + β , j ) X c ( w )  is positive. For this, it is necessary that the matrix  Q β γ  be nonzero. Since  γ 0  and  min β m = 1 , the vector  β γ  has negative components. Since  Q β γ O , we have either  β = γ e w  and  γ w = 0  for some  w , or  β = γ + e v e w  and  γ w = 0  for some  w  and  v w . In both cases, the vector  β  has a single negative component  β w = 1 .
Let us denote by  E 1  the set of vectors  β E  with a single negative component and by  E 0 = E \ E 1  the set of vectors with a large number of negative components. Given the above, the matrices  Ψ α , β  are zero for all  α 0  and  β E 0 . In addition, for  β E 0  matrices  Q β  are also zero. Therefore, as follows from equality (35), the matrices  F β β E 0 ,  are also zero. After removing the zero matrices, system (29) acquires the following form:
F β = Q β + Q 0 v = 1 ( ε 1 , , ε v ) E ¯ 0 , β ( v ) F ε 1 F ε 2 F ε v
+ i = 1 M Q e i v = 1 ( ε 1 , , ε v ) E ¯ e i , β ( v ) F ε 1 F ε 2 F ε v ,   β E 1 ,
where the sets  E ¯ α , β ( v )  are defined as
E ¯ α , β ( v ) = { ( ε 1 , , ε v ) E α , β ( v ) | ε 1 , , ε v E 1 } .
Specific cases of  M d - QBD  processes, where only one component of the vector  α ( t )  can change at a time, were studied in [16]. In these cases, the matrices  Q e v e w  are zero for all indices  v w . Equation (18) in [16] applies only when  M = 1  as there is an error in the proof of Theorem 3. As a result, the findings presented in Section 3 of the mentioned work that rely on this theorem are only valid in the one-dimensional case. For multidimensional  M d - QBD  processes, Formula (27) of this article provides the correct matrix-multiplicative representation of the matrix  G  through matrices of order  n , utilizing the solution of the system (36).

6. Conclusions and Future Work

This study presents several theoretical results that aim to simplify the analysis of multidimensional Markov chains of M/G/1 type. The novelty is that we have introduced new concepts of the state sectors and the sector exit probabilities. We have been able to demonstrate that Equation (1), which involves a matrix  G , can be replaced by a system of Equation (29) that utilizes a family of matrices  F β  of the sector exit probabilities. Entries of the matrix  G  are indexed by elements of the set  N × J , where  N = { ( k 1 , k 2 , , k M ) + M | min k i = 0 }  and  J = { 1 , 2 , , n } , while the family of the matrices  F β  of the order  n  is indexed by the set  E = { ( k 1 , k 2 , , k M ) M | min k i = 1 } .  In one-dimensional cases, the results of the article are reduced to the existing results by Neuts in [2].
However, there are several challenges to overcome in practically implementing these results. In multidimensional cases, the family of the matrices  F β  is infinite. Since it is not feasible to compute infinite families of matrices, future research should focus on developing a method to select an appropriate truncation approximation for the proposed algorithm and conducting complexity and error analyses. It remains unclear whether in multi-dimensional cases the family of matrices  F β  is the minimal nonnegative solution to the system (29). Future research should also address this question.

Author Contributions

Conceptualization, V.N. and K.S.; formal analysis, V.N.; investigation, V.N. and K.S.; methodology, V.N. and K.S.; funding acquisition, K.S. All authors have read and agreed to the published version of the manuscript.

Funding

This publication has been supported by the RUDN University Scientific Projects Grant System, project No. 021937-2-000.

Data Availability Statement

Data are contained within the article.

Acknowledgments

The authors wish to thank the anonymous reviewers for their valuable and helpful comments, which have significantly improved the study.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Neuts, M.F. Matrix-Geometric Solutions in Stochastic Models: An Algorithmic Approach; The Johns Hopkins University Press: Baltimore, MD, USA, 1981. [Google Scholar] [CrossRef]
  2. Neuts, M.F. Structured Stochastic Matrices of M/G/1 Type and Their Applications; Marcel Dekker: New York, NY, USA, 1989. [Google Scholar] [CrossRef]
  3. Latouche, G.; Ramaswami, V. Introduction to Matrix Analytic Methods in Stochastic Modeling; SIAM: Philadelphia, PA, USA, 1999. [Google Scholar] [CrossRef]
  4. Lipsky, L. Queueing Theory. A Linear Algebraic Approach; Springer: New York, NY, USA, 2009. [Google Scholar]
  5. Li, Q.L. Constructive Computation in Stochastic Models with Applications. The RG-Factorizations; Springer: Berlin/Heidelberg, Germany, 2010. [Google Scholar]
  6. He, Q.M. Fundamentals of Matrix-Analytic Methods; Springer: New York, NY, USA, 2014. [Google Scholar]
  7. Phung-Duc, T. Retrial Queueing Models: A Survey on Theory and Applications. In Stochastic Operations Research in Business and Industry; Dohi, T., Ano, K., Kasahara, S., Eds.; World Scientific Publisher: Singapore, 2017; pp. 1–26. Available online: https://arxiv.org/abs/1906.09560 (accessed on 24 January 2024).
  8. Dudin, A.N.; Klimenok, V.I.; Vishnevsky, V.M. The Theory of Queueing Systems with Correlated Flows; Springer: Basel, Switzerland, 2020. [Google Scholar]
  9. Naumov, V.; Gaidamaka, Y.; Yarkina, N.; Samouylov, K. Matrix and Analytical Methods for Performance Analysis of Telecommunication Systems; Springer: Cham, Switzerland, 2021. [Google Scholar]
  10. Chakravarthy, S.R. Introduction to Matrix-Analytic Methods in Queues 1: Analytical and Simulation Approach—Basics; Wiley: London, UK, 2022. [Google Scholar]
  11. Chakravarthy, S.R. Introduction to Matrix-Analytic Methods in Queues 2: Analytical and Simulation Approach—Queues and Simulation; Wiley: London, UK, 2022. [Google Scholar]
  12. Kobayashi, M.; Miyazawa, M. Tail asymptotics of the stationary distribution of a two-dimensional reflecting random walk with unbounded upward jumps. Adv. Appl. Probab. 2014, 46, 365–399. [Google Scholar] [CrossRef]
  13. Ozawa, T.; Kobayashi, M. Exact asymptotic formulae of the stationary distribution of a discrete-time two-dimensional QBD process. Queueing Syst. 2018, 90, 351–403. [Google Scholar] [CrossRef]
  14. Ozawa, T. Tail Asymptotics in any direction of the stationary distribution in a two-dimensional discrete-time QBD process. Queueing Syst. 2022, 102, 227–267. [Google Scholar] [CrossRef]
  15. Ozawa, T. Stability condition of a two-dimensional QBD process and its application to estimation of efficiency for two-queue models. Perf. Eval. 2019, 130, 101–118. [Google Scholar] [CrossRef]
  16. Naumov, V.A. Matrix-Multiplicative Solution for Multi-Dimensional QBD Processes. Mathematics 2024, 12, 444. [Google Scholar] [CrossRef]
  17. Ramaswami, V. Nonlinear matrix equations in applied probability—Solution techniques and open problems. SIAM Rev. 1988, 30, 256–263. [Google Scholar] [CrossRef]
  18. Bini, D.A.; Latouche, G.; Meini, B. Numerical Methods for Structured Markov Chains; Oxford University Press: Oxford, UK, 2005. [Google Scholar] [CrossRef]
  19. Chiang, C.-Y.; Chu, E.K.-W.; Guo, C.-H.; Huang, T.-M.; Lin, W.-W.; Xu, S.-F. Convergence analysis of the doubling algorithm for several nonlinear matrix equations in the critical case. SIAM J. Matrix Anal. Appl. 2009, 31, 227–247. [Google Scholar] [CrossRef]
  20. Bini, D.A.; Meini, B. The cyclic reduction algorithm: From Poisson equation to stochastic processes and beyond. Numer. Algorithms 2009, 51, 23–60. [Google Scholar] [CrossRef]
  21. Bini, D.A.; Latouche, G.; Meini, B. A family of fast fixed point iterations for M/G/1-type Markov chains. IMA J. Numer. Anal. 2022, 42, 1454–1477. [Google Scholar] [CrossRef]
  22. Ramaswami, V. Stable recursion for the steady state vector for Markov chains of M/G/1 type. Stoch. Models 1988, 4, 183–188. [Google Scholar] [CrossRef]
  23. Meini, B. An improved FFT-based version of Ramaswami’s formula. Stoch. Models 1997, 13, 223–238. [Google Scholar] [CrossRef]
  24. Li, Q.; Zhao, Y.Q. A constructive method for finding β-invariant measures for transition matrices of M/G/1-type. In Matrix-Analytic Methods. Theory and Applications; Latouche, G., Taylor, P.G., Eds.; World Scientific: Hackensack, NJ, USA, 2002; pp. 237–263. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Naumov, V.; Samouylov, K. Multi-Dimensional Markov Chains of M/G/1 Type. Mathematics 2025, 13, 209. https://doi.org/10.3390/math13020209

AMA Style

Naumov V, Samouylov K. Multi-Dimensional Markov Chains of M/G/1 Type. Mathematics. 2025; 13(2):209. https://doi.org/10.3390/math13020209

Chicago/Turabian Style

Naumov, Valeriy, and Konstantin Samouylov. 2025. "Multi-Dimensional Markov Chains of M/G/1 Type" Mathematics 13, no. 2: 209. https://doi.org/10.3390/math13020209

APA Style

Naumov, V., & Samouylov, K. (2025). Multi-Dimensional Markov Chains of M/G/1 Type. Mathematics, 13(2), 209. https://doi.org/10.3390/math13020209

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop