Modified Engel Algorithm and Applications in Absorbing/Non-Absorbing Markov Chains and Monopoly Game

Abstract

The Engel algorithm was created to solve chip-firing games and can be used to find the stationary distribution for absorbing Markov chains. Kaushal et al. developed a matlab-based version of the generalized Engel algorithm based on Engel’s probabilistic abacus theory. This paper introduces a modified version of the generalized Engel algorithm, which we call the modified Engel algorithm, or the mEngel algorithm for short. This modified version is designed to address issues related to non-absorbing Markov chains. It achieves this by breaking down the transition matrix into two distinct matrices, where each entry in the transition matrix is calculated from the ratio of the numerator and denominator matrices. In a nested iteration setting, these matrices play a crucial role in converting non-absorbing Markov chains into absorbing ones and then back again, thereby providing an approximation of the solutions of non-absorbing Markov chains until the distribution of a Markov chain converges to a stationary distribution. Our results show that the numerical outcomes of the mEngel algorithm align with those obtained from the power method and the canonical decomposition of absorbing Markov chains. We provide an example, Torrence’s problem, to illustrate the application of absorbing probabilities. Furthermore, our proposed algorithm analyzes the Monopoly transition matrix as a form of non-absorbing probabilities based on the rules of the Monopoly game, a complete information dynamic game, particularly the probability of landing on the Jail square, which is determined by the order of the product of the movement, Jail, Chance, and Community Chest matrices. The Long Jail strategy, the Short Jail strategy, and the strategy for getting out of Jail by rolling consecutive doubles three times have been formulated and tested. In addition, choosing which color group to buy is also an important strategy. By comparing the probability distribution of each strategy and the profit return for each property and color group of properties, and the color group property, we find which one should be used when playing Monopoly. In conclusion, the mEngel algorithm, implemented using R codes, offers an alternative approach to solving the Monopoly game and demonstrates practical value.

1. Introduction

The chip-firing game (CFG), also known as a probabilistic abacus, is a dynamic process described in [1,2]. It involves a graph with a set of finite vertices and edges, on which multiple tokens, or “chips”, are placed. In this game, if the number of chips on a vertex is greater than or equal to the degree of that vertex, it can be "fired" by moving one chip along each incident edge and placing it on the adjacent vertex. When a vertex is fired, its chip count is reduced by its degree, while its neighbors’ chip counts are increased by one. The CFG is a directed weighted multigraph with a set of vertices and multiple edges sharing the same vertex. Firing a vertex means sending one chip along each outgoing edge from that vertex. It can be formulated as the transition digraph for an absorbing Markov chain with transition probabilities. Engel provided another method for solving chip-firing games based on the canonical decomposition of absorbing Markov chains in [3]. To start the game, a pile of pre-assigned chips is given. An indicated number of chips is set at each transient state, and zero chips are set at each absorbing state. Then, a vertex, k, is considered a critical position, and the number of chips, $g_k$, is placed on each adjacent vertex neighbor of that vertex, k, based on the transition probabilities between adjacent vertices and the vertex, k. This is referred to as “critical loading” in Kaushal et al.’s work [4,5]. If the number of chips at the critical position (vertex, k) is not enough to fire, an additional chip (or additional chips) will be taken from the pile and placed on the vertex, k, and then fired to each adjacent vertex. This process continues as long as the critical position does not reappear in the transient states in the transition diagram. Once the number of chips in all the transient states matches the initial number of chips in those states, the game stops [6]. Chips placed in absorbing states can have a value of either nonzero or zero. By using a normalization procedure, we can determine the stationary distribution of the Markov chains from the transient states to the absorbing states based on the number of chips in each absorbing state. Engel’s method involves representing each absorbing Markov chain as a directed graph in order to find a solution. This method is applicable to any Markov chains with at least one absorbing state.

The generalized Engel algorithm, as proposed by Kaushal et al. [4,5], is designed to solve the CFG by focusing on the transition states and absorbing states of a Markov chain. This algorithm is known for its higher accuracy compared to other methods but does depend on several factors. Kaushal et al. [4] implemented the Engel algorithm into a computer program to expand its application to more complex absorbing Markov chains, which they termed the generalized Engel algorithm. They achieved this by splitting the transition matrix into two separate matrices: the numerator matrix containing the numerators of entries in the transition matrix, and the denominator matrix containing the denominators of entries in the transition matrix as weights or for scaling. The numerator matrix represents the proportional distribution values of each transition state, while the denominator matrix represents the capacity of each transition state. They used the numerator matrix to simulate “firing” and the denominator matrix to reduce the divided chips. Our modified algorithm has a broader application and can provide process values. For example, in a directed graph, we can construct the set of configurations obtained from the CFG by a sequence of firings. Through process values, we can track the movement of every chip, determine how many chips pass through states, and record each configuration with the updated chips.

Our paper introduces a modified version of the generalized Engel algorithm, which we call the mEngel algorithm. This modified algorithm is designed to solve both absorbing and non-absorbing Markov chain problems. The key difference between the generalized Engel algorithm and our modified version is how it handles non-absorbing Markov chains. We accomplish this by reformulating the transition matrix decomposition into two separate matrices, similar to the approach in [4,5]. However, we further split the numerator matrix into nine sub-block matrices, representing a transformation from non-absorbing Markov chains to absorbing Markov chains and back to non-absorbing Markov chains. The mEngel algorithm works by determining the number of chips at each vertex and then using the algorithm to find all probability distribution integral values and store them in each corresponding vertex. The process continues until it stops when the tolerance is satisfied. Essentially, the mEngel algorithm provides approximate solutions for non-absorbing Markov chains. The study of Monopoly as a Markov process and its solution solver using the power method and the canonical decomposition of absorbing Markov chains has been investigated by several researchers, such as [7,8,9,10,11,12,13,14,15,16], among others. Various strategies related to Monopoly’s Jail were explored by analyzing the size of the transition matrix of non-absorbing Markov chains [17,18,19,20], as well as the Long and Short Jail scenarios [21]. There are more than two players in one game, and the last player who is not bankrupt wins. Strategies in the game include deciding whether to spend as long as possible in prison to avoid paying tolls to other players, known as Long Jail, or to leave prison immediately to buy as much land as possible, known as Short Jail. The mEngel algorithm was utilized to compare the ranking results of the Long and Short Jail strategies.

The rest of the paper is organized as follows. Section 2 introduces the transition diagram for the non-absorbing Markov chain model using a nested iterative setting. Section 3 first introduces the formulation of the transition probability matrix for absorbing/non-absorbing Markov chains [22] and then provides a proof of the mEngel algorithm for solving the non-absorbing Markov chain problem using a nested iterative approach. This method is associated with the direct approaches of the power method and the canonical decomposition of absorbing Markov chains. The flow of the mEngel algorithm is provided, along with detailed descriptions of Algorithms 1–3. Section 4 presents the numerical results of the mEngel algorithm in solving for the absorbing probabilities in Torrence’s problem [23,24]. Finding the stationary probabilities of the non-absorbing Markov chains based on the rules of the Monopoly game regarding Jail are presented, i.e., the probability of landing on each state is found. Emphasis is placed upon the Monopoly player regarding “Go to Jail” for three reasons: (1) landing on “Go to Jail”; (2) drawing a “Go to Jail” card; (3) rolling doubles three times. Players use of the Long Jail and Short Jail strategies was studied in [19,21]; these can be formulated as the $43\times 43$ model and the $41\times 41$ model, respectively, where the number of a player’s turns to profit is studied and ranked based on the properties of the same color group. Another model, known as the $123\times 123$ model [17,20], is also presented in this paper. It involves getting out of Jail by rolling consecutive doubles three times. Section 5 draws conclusions and identifies opportunities for further research.

2. From the Complete Graph to the Modified Graph

The CFG, a discrete dynamic model, can be played on undirected and directed graphs. In this paper, we focus on directed graphs and their relationship with Markov chains, which can be viewed as an absorbing Markov chain. A directed graph, $G(V,E)$, is defined with vertices, $V\left(G\right)$, and edges, $E\left(G\right)$. Each vertex, $v\in V\left(G\right)$, has a capacity, d. An arbitrary number of chips, $g_v$, are placed on each vertex. If the number of chips, $g_v$, exceeds the capacity, d, i.e., $g_v\ge d$, then some chips will be passed to each neighboring vertex along the outgoing edges, which is called firing [4,5,6].

Suppose that a CFG starts with critical loading and ends with reoccurring critical loading. In that case, the chip distribution of this CFG is equal to the stationary distribution of the absorbing Markov chain. The critical loading occurs when each vertex, $v_i$, has one less chip than it needs to fire. In other words, $g_{v_i}=d_i-1$. The critical loading of each CFG will reoccur, which has been proven by [6]. Hence, the Engel algorithm is complete, meaning it always provides a solution for the absorbing Markov chain.

Let us have the directed graph, $G(V,E)$, of a Markov Chain, represented as a transition diagram, as shown in Figure 1. Assume that every jump passes through this directed graph. We take the vertex and edge of it to create a new directed graph, as shown in Figure 2.

Figure 1. The directed graph of the original Markov chain.

Figure 2. The directed graph of the reconstructed Markov chain.

Let the new graph be $\widehat{G}$. We have the vertex set $V\left(\widehat{G}\right)$ and the edge set $E\left(\widehat{G}\right)$, where $V\left(\widehat{G}\right)=\left\{s\right\}\cup V\left(G\right)\cup V\left(G^{\prime }\right)$, and $V\left(G^{\prime }\right)$ is a set of the same size as $V\left(G\right)$, for ${\widehat{e}}_{ij}\in E\left(\widehat{G}\right)$, $e_{ij}\in E\left(G\right)$, ${\widehat{e}}_{ij}=e_{ij}$ if $v_i\in V\left(G\right),v_j\in V\left(G^{\prime }\right)$, ${\widehat{e}}_{ij}=0$ if $v_i,v_j\in V\left(G\right)$ or $v_i,v_j\in V\left(G^{\prime }\right)$.

The mEngel algorithm proposed for solving the absorbing Markov chain as well as the non-absorbing Markov chain consists of three steps:

Step 1:

We begin with the state space of a non-absorbing Markov chain, denoted by S, then create another state space of an absorbing Markov chain with the same number of states as in the original chain of the transition diagram, denoted by $S_1$. In this new absorbing chain, the state space $S_1$ is not fully connected. Then, we introduce the additional state space denoted by $S_2$, which is also not fully connected. All states in $S_2$ are recurrent, meaning they eventually return to themselves. In general, a state is considered recurrent if, whenever we leave that state, we will return to it in the future with probability one. $S_2$ cannot go back to $S_1$.

Step 2:

We introduce an artificial state s in S as a starting state. This state is not part of the original Markov chain but is added to the algorithm. The transition probabilities from S to each state in $S_1$ are set to specific probability distribution values (i.e., certain transitions). In contrast, the transition probabilities from S to each state in $S_2$ are all set to zeros (i.e., impossible transitions).

Step 3:

The stationary distribution of this modified Markov chain (including S, $S_1$, and $S_2$) is equivalent to that of the original non-absorbing Markov chain. The proof of this equivalence is provided in Section 3.

In Figure 2, we pick a one-time jump (the player moves one time (one turn of the game)) of the Markov chain. With this time limit (a chain with only one jump), the Markov chain becomes an absorbing one. Then, we can iterate this absorbing Markov chain to obtain the stationary distribution of the whole Markov chain. The mEngel algorithm allows us to obtain the probability of the evolution process through an iterative process, i.e.,

$$\left\{S\to S_1\to S_2\right\}\to \left\{S\to S_1\to S_2\right\}\to \dots .$$

For example, one can observe how often chips pass through one state before the game ends, through a sequence of configurations.

3. Structure of the mEngel Algorithm

The transition probability matrix, P, of the Markov chain for the mEngel algorithm can then be represented in the following canonical form, assuming time homogeneity (e.g., [22]), that is

$$P=\left[{\displaystyle \frac{n_{ij}}{d_{ij}}}\right],i,j=1,\dots ,n$$

where $N=\left[n_{ij}\right]$ is a state (nominator) matrix, $D=\left[d_{ij}\right]$ is a scaling (denominator) matrix, and n is the total number of states. Note that $P=N/D$ is a non-absorbing matrix and $\widehat{P}=\widehat{N}/\widehat{D}$ is an absorbing matrix. The distinction between P and $\widehat{P}$ arises from the operation of Algorithms 1 and 2. Specifically, when we input P into Algorithm 2, it generates N and D as outputs. Similarly, when we input $\widehat{P}$, the algorithm produces $\widehat{N}$ and $\widehat{D}$. It is important to note that in our computations, we only utilize $\widehat{P}$.

The mEngel algorithm’s transition diagram is composed of three state processes: (1) $S=\left\{s\right\}$ is a source state; (2) $S_1=\{1,\dots ,n\}$ is a set of original states from the given problem; (3) $S_2=\{1^{\prime },\dots ,n^{\prime }\}$ is a set of fictitious states for an mEngel algorithm setting. We use labels (e.g., s) instead of full names for vertices, $V_s$, i.e., $S=\left\{s\right\}=\left\{V_s\right\}$. The transition diagram is shown in Figure 2.

Algorithm 1: Structure of the mEngel algorithm

Algorithm 2: Matrix formulations of N and D

Input: P: the transition matrix; n: the # of states; $c=10000$. Output: N and D.   1  $S1\leftarrow round(c\ast P)$   2  $d(0:n)\leftarrow 0$   3  $d\left(1\right)\leftarrow 1$   4  $z\leftarrow 0$   5  $z1(0:n)\leftarrow 0$   6  $z2(0:2n)\leftarrow 0$   7  $z3\leftarrow matrix\left(\right(n,n),0)$   8  $z4\leftarrow diag\left(n\right)$   9  $p1\leftarrow [z;z2]$ 10  $p2\leftarrow [d;z3;z3]$ 11  $p3\leftarrow [z1;S1;z4]$ 12  $N\leftarrow [p1,p2,p3]$ 13  $q1\leftarrow matrix\left(\right(n,2\ast n+1),1)$ 14  $q2\leftarrow matrix\left(\right(n,2\ast n+1),c)$ 15  $q3\leftarrow matrix\left(\right(2,n+1),1)$ 16  $D\leftarrow [q3;q2;q1]$

Define

$$\widehat{N}=\begin{array}{cccccc}& & s& \{1,\dots ,n\}& \{1^{\prime },\dots ,n^{\prime }\}& \\ \begin{array}{c}s\\ \{1,\dots ,n\}\\ \{1^{\prime },\dots ,n^{\prime }\}\end{array}& (& \begin{array}{c}{\mathbf{0}}_{1\times 1}\\ {\mathbf{0}}_{n\times 1}\\ {\mathbf{0}}_{n\times 1}\end{array}& \begin{array}{c}{\mathit{d}}_{1\times n}^{\left(k\right)}\\ {\mathbf{0}}_{n\times n}\\ {\mathbf{0}}_{n\times n}\end{array}& \begin{array}{c}{\mathbf{0}}_{1\times n}\\ \mathrm{ceil}(c·P)\\ I_n\end{array}& )\end{array}$$

(1)

and

$$\widehat{D}=\begin{array}{ccccccccc}& & s& \{1,\dots ,n\}& \{1^{\prime },\dots ,n^{\prime }\}& & & & \\ \begin{array}{c}s\\ \{1,\dots ,n\}\\ \{1^{\prime },\dots ,n^{\prime }\}\end{array}& (& \begin{array}{c}{m}_{1\times 1}^{\left(k\right)}\\ c·{\mathbf{1}}_{n\times 1}\\ {\mathbf{1}}_{n\times 1}\end{array}& \begin{array}{c}{\mathit{m}}_{1\times n}^{\left(k\right)}\\ c·{\mathbf{1}}_{n\times n}\\ {\mathbf{1}}_{n\times n}\end{array}& \begin{array}{c}{\mathit{m}}_{1\times n}^{\left(k\right)}\\ c·{\mathbf{1}}_{n\times n}\\ {\mathbf{1}}_{n\times n}\end{array}& )& (& \begin{array}{c}{\mathit{m}}_{1\times 2n+1}^{\left(k\right)}\\ c·{\mathbf{1}}_{n\times 2n+1}\\ {\mathbf{1}}_{n\times 2n+1}\end{array}& )\end{array}$$

(2)

where ${\displaystyle m_{1\times 1}^{\left(k\right)}=\sum _{i=1}^n{d}_i^{\left(k\right)}}$, $\mathit{m}={\left[\begin{array}{cccc}{\displaystyle \sum _{i=1}^n{d}_i^{\left(k\right)}}& {\displaystyle \sum _{i=1}^n{d}_i^{\left(k\right)}}& \dots & {\displaystyle \sum _{i=1}^n{d}_i^{\left(k\right)}}\end{array}\right]}_{1\times n}$ and c is a suitably sized scaling number.

• The ceil operation is used here to ensure that every element of the matrix $\widehat{N}$ is an integer since the distributing chips are in an integer process. For example, if an element of the transition matrix, P, for a non-absorbing Markov chain/absorbing chain, is 0.949 and $c=100$, then the unrounded value will be $94.9$. Thus, the rounded-up value is 95. Alternatively, one could choose c as 1000, thus avoiding the rounding-up procedure. Similarly, each corresponding entry of the ratio of $\widehat{N}$ and $\widehat{D}$ is equal to each position of the transition matrix P. Hence, $\widehat{D}$ is a matrix scaled by c.
• From S to $\{1,\dots ,n\}$, each arrow means a directed movement, where the number of chips being distributed is reshuffled and added. At the end of the iteration process, the number of configurations (chips) between the initial and final transient states remains the same.
• From $\{1,\dots ,n\}$ to $\{1^{\prime },\dots ,n^{\prime }\}$, the original transition matrix, P, is scaled by c. As a result of this rounding-up process, the chips in a vertex are distributed to its neighboring vertices based on the probabilities obtained from rounding up the values of $c·P$ to the nearest integers.
• From $\{1^{\prime },\dots ,n^{\prime }\}$ to $\{1^{\prime },\dots ,n^{\prime }\}$, a set of transient states will be forced into a set of recurrent states, e.g., a state j is called an absorbing state if, with probability 1, the process will eventually return to j after it leaves $j.$ Hence, this is crucial for transforming the transient state into the absorbing state. Typically, a self-loop at each state means that a state of a Markov chain is called absorbing if, once entered, it cannot be left. Chips are stored at each vertex.
• At the end of the iterative process, within the pre-assigned tolerance and the number of iterations, the stationary distribution for non-absorbing Markov chains is obtained by summing all chips from all vertices, i.e., ${\displaystyle m_{1\times 1}^{\left(k\right)}=\sum _{i=1}^n{d}_i^{\left(k\right)}}$, using the normalization procedure, i.e., $d_i^{\left(k\right)}/{\sum }_{i=1}^n{d}_i^{\left(k\right)}$ for all i.

Given the initial probability distribution ${\mathit{d}}^{\left(0\right)}$, our aim here is to show that

$$\mathrm{mEngel}\left({\mathit{d}}^{\left(0\right)}\right)={\mathit{d}}^{\left(1\right)}={\mathit{d}}^{\left(0\right)}P\to \mathrm{mEngel}\left({\mathit{d}}^{\left(1\right)}\right)={\mathit{d}}^{\left(2\right)}={\mathit{d}}^{\left(1\right)}P\to$$

$$\mathrm{mEngel}\left({\mathit{d}}^{\left(2\right)}\right)={\mathit{d}}^{\left(3\right)}={\mathit{d}}^{\left(2\right)}P\to \dots \to \mathrm{mEngel}\left({\mathit{d}}^{\left(k\right)}\right)={\mathit{d}}^{(k+1)}={\mathit{d}}^{\left(k\right)}P.$$

For each iteration in the mEngel algorithm, we will get the same probability distribution for a non-absorbing/absorbing Markov chain using the power method, i.e.,

$${\mathit{d}}^{\left(1\right)}={\mathit{d}}^{\left(0\right)}P\to {\mathit{d}}^{\left(2\right)}={\mathit{d}}^{\left(1\right)}P\to {\mathit{d}}^{\left(3\right)}={\mathit{d}}^{\left(2\right)}P\to \dots \to {\mathit{d}}^{(k+1)}={\mathit{d}}^{\left(k\right)}P.$$

What this needs to show is that

$$\mathrm{mEngel}\left({\mathit{d}}^{\left(1\right)}\right)={\mathit{d}}^{\left(2\right)}={\mathit{d}}^{\left(1\right)}P$$

is true using the first step transition.

We aim to prove $\mathrm{mEngel}\left({\mathit{d}}^{\left(1\right)}\right)={\mathit{d}}^{\left(1\right)}P$. By the power method, we have ${\displaystyle {\left({\mathit{d}}^{\left(1\right)}P\right)}_i=\sum _{j=1}^n{d}_j^{\left(1\right)}{p}_{ji}}$. This means that we need to prove ${\displaystyle {\left(\mathrm{mEngel}\left({\mathit{d}}^{\left(1\right)}\right)\right)}_i=\sum _{j=1}^n{d}_j^{\left(1\right)}{p}_{ji}}$.

The canonical decomposition of P is defined as

$$P=\begin{array}{cccc}& & \mathrm{TR}& \mathrm{ABS}\\ \begin{array}{c}\mathrm{TR}\\ \mathrm{ABS}\end{array}& (& \begin{array}{c}Q\\ \mathbf{0}\end{array}& \begin{array}{c}R\\ I\end{array}& )\end{array}$$

where TR states are transient states, while ABS states are absorbing states (e.g., see Figure 3). $\mathbf{0}$ represents a zero matrix and I represents an identity matrix with appropriate dimensions.

Figure 3. Transition diagram for Torrence’s problem.

To do so, let us recall

$$\widehat{P}=\begin{array}{ccccccccc}& & s& 1& \dots & n& 1^{\prime }& \dots & n^{\prime }\\ \begin{array}{c}s\\ 1\\ ⋮\\ n\\ 1^{\prime }\\ ⋮\\ n^{\prime }\end{array}& (& \begin{array}{c}0\\ 0\\ 0\\ 0\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}{d}_1^{\left(1\right)}\\ 0\\ 0\\ 0\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}\dots \\ \dots \\ \ddots \\ \dots \\ \dots \\ \ddots \\ \dots \end{array}& \begin{array}{c}{d}_n^{\left(1\right)}\\ 0\\ 0\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ p_{11}\\ ⋮\\ p_{n1}\\ 1\\ ⋮\\ 0\end{array}& \begin{array}{c}\dots \\ \dots \\ \ddots \\ \dots \\ \dots \\ \ddots \\ \dots \end{array}& \begin{array}{c}0\\ p_{1n}\\ 0\\ ⋮\\ p_{nn}\\ 0\\ ⋮\\ 1\end{array}& )\end{array}$$

Using the absorbing Markov chain formulation, we have

$$Q=\begin{array}{cccccc}& & s& 1& \dots & n\\ \begin{array}{c}s\\ 1\\ ⋮\\ n\end{array}& (& \begin{array}{c}0\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}{d}_1^{\left(1\right)}\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}\dots \\ \dots \\ \ddots \\ \dots \end{array}& \begin{array}{c}{d}_n^{\left(1\right)}\\ 0\\ 0\\ 0\end{array}& )\end{array}$$

$$R=\begin{array}{ccccc}& & 1^{\prime }& \dots & n^{\prime }\\ \begin{array}{c}s\\ 1\\ ⋮\\ n\end{array}& (& \begin{array}{c}0\\ p_{11}\\ ⋮\\ p_{n1}\end{array}& \begin{array}{c}\dots \\ \dots \\ \ddots \\ \dots \end{array}& \begin{array}{c}0\\ p_{1n}\\ ⋮\\ p_{nn}\end{array}& )\end{array}$$

The fundamental matrix for $\widehat{P}$ is

$$J={({\mathbf{1}}_{n+1\times n+1}-Q)}^{-1}=\begin{array}{cccccc}& & s& 1& \dots & n\\ \begin{array}{c}s\\ 1\\ ⋮\\ n\end{array}& (& \begin{array}{c}1\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}{d}_1^{\left(1\right)}\\ 1\\ 0\\ 0\end{array}& \begin{array}{c}\dots \\ \dots \\ \ddots \\ \dots \end{array}& \begin{array}{c}{d}_n^{\left(1\right)}\\ 0\\ 0\\ 1\end{array}& )\end{array}$$

and the time-to-absorption matrix is

$$\begin{gathered} B=JR=\begin{array}{ccccccccccccc}& & s& 1& \dots & n& & & & 1^{\prime }& \dots & n^{\prime }& \\ \begin{array}{c}s\\ 1\\ ⋮\\ n\end{array}& (& \begin{array}{c}1\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}{d}_1^{\left(1\right)}\\ 1\\ 0\\ 0\end{array}& \begin{array}{c}\dots \\ \dots \\ \ddots \\ \dots \end{array}& \begin{array}{c}{d}_n^{\left(1\right)}\\ 0\\ 0\\ 1\end{array}& )& \begin{array}{c}s\\ 1\\ ⋮\\ n\end{array}& (& \begin{array}{c}0\\ p_{11}\\ ⋮\\ p_{n1}\end{array}& \begin{array}{c}\dots \\ \dots \\ \ddots \\ \dots \end{array}& \begin{array}{c}0\\ p_{1n}\\ ⋮\\ p_{nn}\end{array}& )\end{array} \\ =\begin{array}{ccccccccccccc}& & 1^{\prime }& \dots & n^{\prime }& & & & & 1^{\prime }& \dots & n^{\prime }& \\ \begin{array}{c}s\\ 1\\ ⋮\\ n\end{array}& (& \begin{array}{c}{\displaystyle \sum _{j=1}^n{d}_j^{\left(1\right)}{p}_{j1}}\\ p_{11}\\ ⋮\\ p_{n1}\end{array}& \begin{array}{c}\dots \\ \dots \\ \ddots \\ \dots \end{array}& \begin{array}{c}{\displaystyle \sum _{j=1}^n{d}_j^{\left(1\right)}{p}_{jn}}\\ p_{1n}\\ ⋮\\ p_{nn}\end{array}& )& =& \begin{array}{c}s\\ 1\\ ⋮\\ n\end{array}& (& \begin{array}{c}{d}_1^{\left(2\right)}\\ p_{11}\\ ⋮\\ p_{n1}\end{array}& \begin{array}{c}\dots \\ \dots \\ \ddots \\ \dots \end{array}& \begin{array}{c}{d}_n^{\left(2\right)}\\ p_{1n}\\ ⋮\\ p_{nn}\end{array}& )\end{array} \end{gathered}$$

So, the ${\displaystyle {\left(\mathrm{mEngel}\left({\mathit{d}}^{\left(1\right)}\right)\right)}_i}$ is $B_{1i}$, which is ${\displaystyle \sum _{j=1}^n{d}_j^{\left(1\right)}{p}_{ji}}$. This can also be expressed as ${\left(d^{\left(1\right)}P\right)}_i$.

From s to $\{1^{\prime },\dots ,n^{\prime }\}$ in B, the distribution of $\{1^{\prime },\dots ,n^{\prime }\}$ is ${\mathit{d}}^{\left(2\right)}$, i.e.,

$$\mathrm{mEngel}\left({\mathit{d}}^{\left(1\right)}\right)={\mathit{d}}^{\left(2\right)}={\mathit{d}}^{\left(1\right)}P.$$

The rest of the iteration results follow, i.e.,

$$\mathrm{mEngel}\left({\mathit{d}}^{\left(k\right)}\right)={\mathit{d}}^{(k+1)}={\mathit{d}}^{\left(k\right)}P,$$

where P can be a transition matrix or an absorbing transition matrix. From $\{1,\dots ,n\}$ to $\{1^{\prime },\dots ,n^{\prime }\}$ in B, we check to see that P remains unchanged at each iteration. The $k+1$-step transition will follow.

Implementation of the mEngel Algorithm

Algorithm 1 presents the structure of the pseudo-code, while Algorithm 2 provides the matrix formulations of $\widehat{N}$ and $\widehat{D}$ for (1) and (2), respectively. The details of Algorithm 2 are described in Section 3. Algorithm 3 demonstrates the nested mEngel algorithm with an accuracy tolerance as the stopping criterion. We use N and D for illustrative purposes in these three algorithms to avoid confusion.

Algorithm 3: Illustration of the nested mEngel algorithm

Let

$${\mathit{d}}^{\left(0\right)}=\left[\begin{array}{cccc}1& 0& \dots & 0\end{array}\right]$$

be the initial probability distribution vector. The details of Algorithm 1 are described as follows:

• In Lines 3–5, $MD$ represents the capacity array of the states. An element in $MD$ is equal to 0 when it is a recurrent state and not equal to zero when it is a transient state. This part is also called critical loading.
• In Lines 15–39, the while loop ends when the initial transient states’ chips are the same as the end states’ chips.
• In Lines 20–30, chips are firing if available. Otherwise, it simply counts the number of unavailable states.

  –

  In Lines 21–22, if the available chips are more than the capacity for chips, then the chips are stored in $new1$.

  –

  In Lines 24–28, $j\ne i$, where j is the order of states that the chips move to, i is the order of states that the chips start from, and $j\ne i$ means that, in this round of the loop, the current state is not the starting state.

• In Lines 31–32, if the number of states that are not available is equal to the transient states, the while loop ends.
• In vector $temp$, the total of d is needed, i.e., $sum\left(d\right)$, because the starting state needs $sum\left(d\right)$ chips to be available to be fired. We set $counteq=0$, which is the number of unavailable transient states. The loop will stop when $counteq$ is equal to the number of transient states, n.
• In Lines 37–38, the number of the current transient state’s chips are checked to determine whether it is the same as that of the initial one, i.e., critical loading.
• In Lines 40–43, the probability distribution vector is updated and normalized.

4. Numerical Results

In this section, we apply the mEngel algorithm to the absorbing problem, which is Torrence’s problem, and non-absorbing problems such as Monopoly. We use our homemade R code to generate all numerical results and graphical plots. All the transition matrices defined below are input in Algorithms 1–3.

Let $X_n$ represent the state at time n, and $P(X_n=j|X_{n-1}=i)$ denote the conditional probability that the state at time $n-1$ is i and that at time n is j. Then, we have $P(X_n=j|X_{n-1}=i_{n-1},\dots ,X_0=i_0)=P(X_n=j|X_{n-1}=i_{n-1})$, which is known as the Markov property. It is important to note that $P\left(j\right|i)=P(X_n=j|X_{n-1}=i)$. For instance, in the Markov matrix, M, we have $M_{ij}=P\left(j\right|i)$, and it has the property that ${\displaystyle \sum _{j\in S}{M}_{ij}=1}$ for all $i\in S$, where S is the state set.

4.1. Torrence’s Problem

Torrence’s problem is a random walk problem given as a weekly Riddler puzzle on the FiveThirtyEight website [23]. The transition diagram for Torrence’s problem shown in Figure 3 is the one from [24].

The transition matrix for Torrence’s problem is given by

$$P=\begin{array}{ccccccccccccc}& & 1& 2& 3& 4& 5& 6& 7& 8& 9& 10& \\ \begin{array}{c}1\\ 2\\ 3\\ 4\\ 5\\ 6\\ 7\\ 8\\ 9\\ 10\end{array}& (& \begin{array}{c}0\\ 1/3\\ 0\\ 0\\ 1/3\\ 0\\ 0\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}1/3\\ 0\\ 1/3\\ 0\\ 0\\ 0\\ 0\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 1/3\\ 0\\ 1/3\\ 0\\ 0\\ 0\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 0\\ 1/3\\ 0\\ 1/3\\ 0\\ 0\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}1/3\\ 0\\ 0\\ 1/3\\ 0\\ 0\\ 0\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}1/3\\ 0\\ 0\\ 0\\ 0\\ 1\\ 0\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 1/3\\ 0\\ 0\\ 0\\ 0\\ 1\\ 0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 0\\ 1/3\\ 0\\ 0\\ 0\\ 0\\ 1\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 0\\ 0\\ 1/3\\ 0\\ 0\\ 0\\ 0\\ 1\\ 0\end{array}& \begin{array}{c}0\\ 0\\ 0\\ 0\\ 1/3\\ 0\\ 0\\ 0\\ 0\\ 1\end{array}& )\end{array}.$$

A person begins in the dark inner circle state and aims to reach the other colored states. The probability of each state is shown in Figure 4. The probability of each recurrent state is 5/11 for the closest state, 2/11 for the second closest two states, and 1/11 for the furthest state. These distribution values are consistent with the existing ones. The mEngel’s algorithm has 22 configurations, resulting in a stationary distribution mirroring the final round’s chip distribution. All transient states eventually hold an equal number of chips to the initial configuration, as shown in Table 1. These findings are consistent with those in [24].

Figure 4. Torrence’s distribution.

Table 1. Process value distribution of chips for state 1.

	State	6	7	8	9	10	1	2	3	4	5	A Firing State
Configuration
0		0	0	0	0	0	2	2	2	2	2
1		0	0	0	0	0	3	2	2	2	2	0
2		1	0	0	0	0	0	3	2	2	3	1
3		1	1	0	0	0	1	0	3	2	3	2
4		1	1	1	0	0	1	1	0	3	3	3
5		1	1	1	1	0	1	1	1	0	4	4
6		1	1	1	1	1	2	1	1	1	1	5
7		1	1	1	1	1	3	1	1	1	1	0
8		2	1	1	1	1	0	2	1	1	2	1
9		2	1	1	1	1	1	2	1	1	2	0
10		2	1	1	1	1	2	2	1	1	2	0
11		2	1	1	1	1	3	2	1	1	2	0
12		3	1	1	1	1	0	3	1	1	3	1
13		3	2	1	1	1	1	0	2	1	3	2
14		3	2	1	1	2	2	0	2	2	0	5
15		3	2	1	1	2	3	0	2	2	0	0
16		4	2	1	1	2	0	1	2	2	1	1
17		4	2	1	1	2	1	1	2	2	1	0
18		4	2	1	1	2	2	1	2	2	1	0
19		4	2	1	1	2	3	1	2	2	1	0
20		5	2	1	1	2	0	2	2	2	2	1
21		5	2	1	1	2	1	2	2	2	2	0
22		5	2	1	1	2	2	2	2	2	2	0

4.2. Monopoly

Monopoly, a widely enjoyed board game, can be modeled as a non-absorbing Markov chain problem. The game’s intricate rules result in a complex transition matrix. We plan to construct this transition matrix by multiplying four separate matrices (e.g., [19]), as described below. We dissect the entire Markov chain into segments, each of which can be considered a sub-chain. Each sub-chain corresponds to a specific rule, and their combination reconstitutes the original chain.

In Monopoly, a player’s decision is primarily determined by the ownership of land on the board. In decision-making, the player only needs to consider the probability of landing on each piece of land and the benefits of landing on each piece of land. In what follows, we calculate the probability of landing on each piece of land and the benefits of purchasing each piece of land in the two Jail strategies.

Using the Long and Short Jail strategies [21], we aim to find the probability of landing in Jail for three scenarios: (1) landing on “Go to Jail”; (2) drawing a “Go to Jail” card; (3) rolling doubles three times. Within the non-absorbing Monopoly Markov chains consisting of 41 states, where the first 40 states are regular Monopoly squares as listed, for a Short Jail strategy, at state 41, if a player starts their turn in Jail, they have the option to play a “Get Out of Jail Free” card (Community Chest or Chance) or pay USD 50 to roll the dice and exit Jail normally. For a Long Jail strategy, assuming the player is incarcerated, the non-absorbing Monopoly Markov chains consist of 43 states, where the first 40 states are regular Monopoly squares as listed, state 41 represents the initial turn in Jail, state 42 represents the second turn in Jail, and state 43 represents the third turn in Jail. In Monopoly, the player moves around the 40 squares by rolling a pair of fair dice. If the player rolls doubles, then they have an extra turn. If they roll doubles again on this extra turn, they have yet another additional turn. However, if the player rolls doubles for a third time, they are sent to Jail instead of proceeding as usual. This is called the $120\times 120$ model [17]. In what follows, we also use Long and Short Jail strategies for obtaining the stationary distribution of the Monopoly Markov chains formulated by the $123\times 123$ model.

4.2.1. Construction of Monopoly Matrices

Firstly, players use two dice to move, with the number of steps corresponding to the sum of the two dice values, as shown in Table 2.

Table 2. Two dice sum probabilities.

Sum of Two Dice	Probability
2	$P\left(2\right)={\displaystyle \frac{1}{36}}$
3	$P\left(3\right)={\displaystyle \frac{2}{36}}$
4	$P\left(4\right)={\displaystyle \frac{3}{36}}$
5	$P\left(5\right)={\displaystyle \frac{4}{36}}$
6	$P\left(6\right)={\displaystyle \frac{5}{36}}$
7	$P\left(7\right)={\displaystyle \frac{6}{36}}$
8	$P\left(8\right)={\displaystyle \frac{5}{36}}$
9	$P\left(9\right)={\displaystyle \frac{4}{9}}$
10	$P\left(10\right)={\displaystyle \frac{3}{36}}$
11	$P\left(11\right)={\displaystyle \frac{2}{36}}$
12	$P\left(12\right)={\displaystyle \frac{1}{36}}$

To end up in Jail, a player must roll three doubles, land on the square, or obtain the card that sends them there. Once in Jail, the player can either pay to leave on their next turn or try to roll doubles. If neither of these options happens, the player must stay in Jail for three turns. These rules are important for understanding the Jail matrix and the movement matrix. For example, the movement matrix involves a Short Jail strategy with an extra state and a Long Jail strategy with three extra states. In the case of three extra states, the states are as follows: (1) if the player chooses not to use a card or pay the fee, then they still need to roll the dice; (2) if the player rolls doubles, then they move out of Jail the designated number of squares without earning an additional turn, even though doubles are rolled; (3) failing to roll doubles on the third attempt requires one to pay USD 50 to leave and to proceed the indicated number of squares. Landing on a Chance or Community Chest square prompts players to draw a card that could lead them to other squares (the player takes the top card from the draw pile and immediately places it on the discard pile). The probabilities for the cards in the Chance or Community Chest squares are shown in Table 3 and Table 4. The “Stay” card groups all non-moving square cases. In Table 3, there are 16 cards available, where 14 of them do not involve moving to another square (they involve either gaining or losing money in some way). Of the 14 “Stay” cards, 9 reward the player with cash, 4 make the player pay money to the bank or other players, and 1 is a “Get Out of Jail Free” card. One card sends the player directly to Jail (“Go to Jail”), and another card sends the player to the start position (“Advance to Go”). Table 4 shows that one card sends the player directly to Jail (“Go to Jail”), three of the six “Stay” cards make the player pay money to the bank or other players, two reward the player with cash, and one is a “Get Out of Jail Free” card. The remaining seven direct the player to a property, railroad, or utility, highlighted in bold.

Table 3. Community Chest destinations.

Card	Probability
Stay	14/16
Advance to Go	1/16
Go to Jail	1/16

Table 4. Chance destinations.

Card	Probability
Take a Trip to Reading Railroad	1/16
Advance to the nearest Railroad	2/16
Advance to St. Charles Place	1/16
Advance to the Nearest Utility	1/16
Advance to Illinois Avenue	1/16
Advance to Boardwalk	1/16
Advance to Go	1/16
Go Directly to Jail	1/16
Go back three spaces	1/16
Stay	6/16

We aim to categorize these rules into four groups, each represented by a matrix. These matrices are named according to the rules they represent: the movement matrix $\mathit{M}=\left[M_{ij}\right]$, the Jail matrix $\mathit{J}=\left[J_{ij}\right]$, the Chance matrix $\mathit{Ch}=\left[C{h}_{ij}\right]$, and the Community Chest matrix $\mathit{Cc}=\left[C{c}_{ij}\right]$. All four matrices adhere to an order that mirrors the priority of the rules in Monopoly. For instance, a player must first move and then act based on the new square they land on, making the movement matrix the first. The Jail matrix is second as we consider the possibility of Jail upon arriving at a new square. The order of the Chance and Community Chest matrices is also fixed. If a player lands on a Chance square, they may be sent to a Community Chest square. This is due to one of the 16 Chance cards instructing the player to “Go back 3 spaces”, and the Chance square is three ahead of the Community Chest square.

This approach differs from that presented in [19], where we attempted to change the order of the Chance and Community Chest matrices, resulting in a different outcome. This paper considers the scenario where a player can move from a Chance square to a Community Chest square. Upon researching the commutative law of matrix multiplication, we found that these two matrices do not satisfy the conditions of the commutative law.

Upon integrating these matrices, we derive the transition matrix, defined as follows:

$$P=\mathit{M}\mathit{J}\mathit{Ch}\mathit{Cc}.$$

Two strategies emerge: Long Jail formulated by the $43\times 43$ model and Short Jail formulated by the $41\times 41$ model. The Long Jail strategy involves players remaining in Jail for as long as possible, implying they will not pay to leave. Conversely, the Short Jail strategy permits players to leave immediately on their next turn. The numerical distinction between these two strategies lies in the last three rows of the movement matrix, while the other matrices remain unchanged.

Let us define the movement matrix in the Long Jail strategy as ${\mathit{M}}^L$ and the movement matrix in the Short Jail strategy as ${\mathit{M}}^S$. The transition matrix in the Long Jail strategy is given by $P={\mathit{M}}^L\mathit{J}\mathit{Ch}\mathit{Cc}$, and the transition matrix in the Short Jail strategy is given by $P={\mathit{M}}^S\mathit{J}\mathit{Ch}\mathit{Cc}$, as shown in Table A1 and Table A2 in Appendix A. Next, we will construct the Markov matrix for ${\mathit{M}}^L,{\mathit{M}}^S,\mathit{J},\mathit{Ch}$, and $\mathit{Cc}$, as shown in Table A3–Table A7, respectively, in Appendix A.

The movement matrix in the Long Jail strategy is given by:

$$M_{ij}^L=\left\{\begin{array}{cc}1/36\hfill & \mathrm{if}j=i+2,i-38,i+12,i-28\mathrm{and}1\le i,j\le 40\mathrm{or}j=13,23,i=43\hfill \\ \hfill & \mathrm{or}j=13,15,17,19,21,23,i=41,42\hfill \\ 2/36\hfill & \mathrm{if}j=i+3,i-37,i+11,i-29\mathrm{and}1\le i,j\le 40\mathrm{or}j=14,22,i=43\hfill \\ 3/36\hfill & \mathrm{if}j=i+4,i-36,i+10,i-30\mathrm{and}1\le i,j\le 40\mathrm{or}j=15,21,i=43\hfill \\ 4/36\hfill & \mathrm{if}j=i+5,i-35,i+9,i-31\mathrm{and}1\le i,j\le 40\mathrm{or}j=16,20,i=43\hfill \\ 5/36\hfill & \mathrm{if}j=i+6,i-34,i+8,i-32\mathrm{and}1\le i,j\le 40\mathrm{or}j=17,19,i=43\hfill \\ 6/36\hfill & \mathrm{if}j=i+7,i-33\mathrm{and}1\le i,j\le 40\mathrm{or}j=18,i=43\hfill \\ 30/36\hfill & \mathrm{if}j=42,i=41\mathrm{or}j=43,i=42\hfill \end{array}\right.$$

For the 1st–40th rows: Take the first row as an example. This means that the first row starts from “Go” (1st state) and goes to other states based on the probability of the outcome of the roll of the two dice.

The probability of moving from the ith state to the jth state is $P\left(j\right|i)=P(j-i)$. Apply it, and we can obtain the movement matrix, $M_{ij}^L=P\left(j\right|i)$. The last three rows have two steps:

Step 1:

The probability of leaving from Jail is $1/6,$ while the probability of staying in Jail is $1-1/6=5/6$.

Step 2:

They have the same formation as the 1st–40th rows if ‘leaving Jail’. However, in states 41 and 42, the entries $M_{ij}^L$ (where $i=41,42,j<41$), are $M_{ij}^L=(5/6)=P\left(j\right|13)$ ($5/6$ because ‘if leaving’, from Step 1). Furthermore, $M_{ij}^L$ (where $i=43,j<41$), is $M_{ij}^L=P\left(j\right|13)$.

In Monopoly, the last three states show how to leave Jail. In Jail, in states 41 and 42, players have a $1/36$ chance to roll doubles. If they do, they leave Jail and move forward that many spaces. Because we added extra states for being in Jail, but rolling doubles means they move from the Jail square, if they do roll doubles, they move to state 11, “Just Visiting Jail”.

Squares from Jail	11	12	13	14	15	16	17	18	19	20	21	22	23
	11	11+1	11+2	11+3	11+4	11+5	11+6	11+7	11+8	11+9	11+10	11+11	11+12
Probability of
Leaving after the	0	0	1/36	0	1/36	0	1/36	0	1/36	0	1/36	0	1/36
First Turn in Jail
Probability of
Leaving after the	0	0	1/36	0	1/36	0	1/36	0	1/36	0	1/36	0	1/36
Second Turn in Jail

A player has a 5/6 chance of staying in Jail, meaning the chance of moving from state 41 to 42 is 5/6. This is because state 41 is when they are first put in Jail, and if they do not roll doubles, they move to their first turn in Jail, state 42. The same 5/6 chance applies to moving from state 42 to 43. After the third turn in Jail, they must pay to leave. So, they leave from state 43 with chances based on the rolling of two dice, like the rest of the matrix. The chances of moving from state 43 to other states on the board are outlined below:

Squares	1 … 11	12	13	14	15	16	17	18	19	20	21	22	23	24 … 43
Probability of
Leaving after the	0 … 0	0	1/36	2/36	3/36	4/36	5/36	6/36	5/36	4/36	3/36	2/36	1/36	0 … 0
Third Turn in Jail

The movement matrix in the Short Jail strategy is given by:

$$M_{ij}^S=\left\{\begin{array}{cc}1/36\hfill & \mathrm{if}j=i+2,i-38,i+12,i-28\mathrm{and}1\le i,j\le 40\hfill \\ & \mathrm{or}j=13,23,i=41,42,43\hfill \\ 2/36\hfill & \mathrm{if}j=i+3,i-37,i+11,i-29\mathrm{and}1\le i,j\le 40\hfill \\ & \mathrm{or}j=14,22,i=41,42,43\hfill \\ 3/36\hfill & \mathrm{if}j=i+4,i-36,i+10,i-30\mathrm{and}1\le i,j\le 40\hfill \\ & \mathrm{or}j=15,21,i=41,42,43\hfill \\ 4/36\hfill & \mathrm{if}j=i+5,i-35,i+9,i-31\mathrm{and}1\le i,j\le 40\hfill \\ & \mathrm{or}j=16,20,i=41,42,43\hfill \\ 5/36\hfill & \mathrm{if}j=i+6,i-34,i+8,i-32\mathrm{and}1\le i,j\le 40\hfill \\ & \mathrm{or}j=17,19,i=41,42,43\hfill \\ 6/36\hfill & \mathrm{if}j=i+7,i-33\mathrm{and}1\le i,j\le 40\hfill \\ & \mathrm{or}j=18,i=41,42,43\hfill \end{array}\right.$$

For the first row and any of the 2nd to 40th rows of $M_{ij}^S$, the probability of moving from the ith state to the jth state is $P\left(j\right|i)=P(j-i)$, which is the same as the ones in $M_{ij}^L$. The last three rows are computed as follows:

• For Short Jail, the probability of staying in Jail for one more turn is 0 as opposed to Long Jail.
• They have the same formation as the 1st-40th rows if ‘leaving Jail’. (like $M_{ij}^L$)
• Starting with the 11st state (shared with “Just Visiting Jail”), the entries $M_{ij}^S=P\left(j\right|i)$ are the probabilities from rolling two dice, where $j=13,\dots ,23$, and $i=41,42,43$.

The Jail matrix is given by

$$J_{ij}=\left\{\begin{array}{cc}215/216\hfill & \mathrm{if}i=j\mathrm{and}1\le i\le 40\mathrm{with}i\ne 31\hfill \\ 1/216\hfill & \mathrm{if}j\mathrm{and}1\le i\le 40\mathrm{with}i\ne 31\hfill \\ 1\hfill & \mathrm{if}i=31,j=41,\mathrm{or}41\le i,j\le 43\hfill \\ 0\hfill & \mathrm{if}\mathrm{otherwise}\hfill \end{array}\right.$$

where it describes the rule for rolling doubles three times, the probability of which is ${(1/6)}^3=1/216$, so the probability of staying out of Jail is $1-1/216=215/216$, and $J_{ij}=P\left(j\right|i)=215/216$, when $i=j,j\ne 41$, and $J_{ij}=P\left(j\right|i)=0$, when $i\ne j,j\ne 41$, $J_{ij}=P\left(j\right|i)=1/216$ when $j=41$.

Special cases: 31st, 41st, 42nd, and 43rd.

• For the 31st, when players land in this state, they will go to Jail. So, $J_{ij}=P\left(j\right|i)=0$ when $j\ne 41$, $J_{ij}=1$ when $j=41$.
• For the 41st, 42nd, and 43rd, players who are already in Jail will stay in Jail. So, $J_{ij}=P\left(j\right|i)=0$ when $i\ne j$, and $J_{ij}=1$ when $i=j$.

We will now examine the Jail matrix. Adding two states and the doubles rule makes it more complex. The chance of rolling a double is ${(6/36)}^2=(1/6)(1/6)=6/36$. Each roll is independent, meaning the chance of rolling a double does not change, no matter the previous rolls. So, the chance of getting three doubles in a row is ${(6/36)}^3=(1/6)(1/6)(1/6)=(1/216)$. In the $43\times 43$ Jail matrix, all the diagonal entries are ones, except for state 31, the Policeman square. Here, the chance to stay in each state is $215/216$, and to go to Jail in state 41 is $1/216$. Players can be sent to Jail anytime by rolling three doubles in a row. The exception is state 31, where going to Jail is certain. Players in states 41, 42, or 43 stay put. So, we made a Jail matrix that accounts for the chance of rolling doubles three times.

The Chance matrix is given by

$$C{h}_{ij}=\left\{\begin{array}{cc}1/16\hfill & \mathrm{if}j=1,6,12,25,40,41,i=8,23,37,\hfill \\ & [8,6],[8,5],[8,13],[23,20],[23,29],[37,29],[37,34]\hfill \\ 2/16\hfill & \mathrm{if}[23,26],[37,36]\hfill \\ 3/16\hfill & \mathrm{if}j=6\mathrm{and}i=8\hfill \\ 6/16\hfill & \mathrm{if}j=i\mathrm{and}i=8,23,37\hfill \\ 1\hfill & \mathrm{if}1\le i,j\le 43\mathrm{with}8,23,37\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

The rules for calculating probabilities are as follows:

• For the first and last rows, as well as for all other rows except the 8th, 23rd, and 37th, there is no change. Therefore, $C{h}_{ij}=P\left(j\right|i)=0$ for $i\ne j$ when $i\ne 8,23,37$.
• Moreover, $C{h}_{ij}=P\left(j\right|i)=1$ when $i=j$ and $i\ne 8,23,37$.
• In the special case of the 8th, 23rd, and 37th rows, the probability depends on randomly drawn cards. Therefore, there are no specific formulations, and the probability is equal to the probability of the drawn cards as shown in Table 4.

The Community Chest matrix is given by

$$C{c}_{ij}=\left\{\begin{array}{cc}1/16\hfill & \mathrm{if}i=3,18,34\mathrm{and}1\le j\le 41\hfill \\ 4/16\hfill & \mathrm{if}i=3,18,34\mathrm{and}i=j\hfill \\ 1\hfill & \mathrm{if}i\ne 3,18,34\mathrm{and}i=j\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

For the 1st and last row (in fact for all rows except the 3rd, 18th, and 34th), there is no change. So, $C{c}_{ij}=P\left(j\right|i)=0$ for $i\ne j$, when $i\ne 3,18,34$. For $i=j$, when $i\ne 3,18,34$, $C{c}_{ij}=P\left(j\right|i)=1$. For the 3rd, 18th, and 34th rows, the probability depends on randomly drawn cards, as shown in Table 3.

This matrix has ones down the diagonal except in the 3rd, 18th, and 34th states, where the diagonal is $14/16$. This is because 14 cards in the Community Chest will not move the player, and there are two that do—one being “Advance to Go” and the other being “Go to Jail”. For the 3rd, 18th, and 34th rows, these cards will send a player to state 1 or state 41, with the probability of each being $1/16$.

4.2.2. Numerical Results for the $43\times 43$ Model (Long Jail) and for the $41\times 41$ Model (Short Jail)

Solutions for the stationary distribution of the Monopoly Markov chains using the mEngel algorithm, the power method, and the canonical decomposition of absorbing Markov chains are shown in Table 5. There is no sizable difference in the results for any of the distribution values when they are judged to three decimal places. A comparison of the convergence of the Monopoly Markov chains using the mEngel algorithm and the power method is shown in Table 6. These results indicate that, by reducing the tolerance, adjusting parameters such as c, and increasing the number of iterations, the differences between the two results become smaller, implying that the solutions for the mEngel algorithm are close to the results for the power method. Ranking based on the stationary distribution of the Monopoly Markov chains using the Long Jail strategy and the Short Jail strategy is shown in Table 7. The ranking order of our mEngel results is the same as [10,19,25]. The probabilities of Jail using the $43\times 43$ model and the $41\times 41$ model, as shown in Table 8, are 11.70% and 6.302%, respectively. The most-landed-on square in both strategies is Jail. Following “In Jail/Just Visiting”, the second most frequently visited square on the board is Illinois Avenue. Note that there exists an Advance to Illinois Avenue card in the Chance deck. In terms of the Short Jail strategy, the subsequent most commonly landed-on squares are St. James Place, Tennessee Avenue, and New York Avenue, situated 6, 8, and 9 squares beyond Jail, respectively. Based on the results of these two strategies, the player should aim to get out of Jail as quickly as possible by paying the fine before all properties have been purchased.

Table 5. Solutions for the stationary distribution of the Monopoly Markov chains using the mEngel algorithm, the power method, and the canonical decomposition of absorbing Markov chains.

	mEngel		Power		Canonical Form
States	Long Jail	Short Jail	Long Jail	Short Jail	Long Jail	Short Jail
Go	2.9220 × 10−2	3.1050 × 10−2	2.9225 × 10−2	3.1048 × 10−2	2.9225 × 10−2	3.1048 × 10−2
Mediterranean Avenue	2.0355 × 10−2	2.1629 × 10−2	2.0360 × 10−2	2.1622 × 10−2	2.0360 × 10−2	2.1622 × 10−2
Community Chest (South)	1.8052 × 10−2	1.9184 × 10−2	1.8056 × 10−2	1.9172 × 10−2	1.8056 × 10−2	1.9172 × 10−2
Baltic Avenue	2.0722 × 10−2	2.2016 × 10−2	2.0725 × 10−2	2.2004 × 10−2	2.0725 × 10−2	2.2004 × 10−2
Income Tax	2.2233 × 10−2	2.3617 × 10−2	2.2238 × 10−2	2.3607 × 10−2	2.2238 × 10−2	2.3607 × 10−2
Reading Railroad	2.8312 × 10−2	2.9964 × 10−2	2.8316 × 10−2	2.9953 × 10−2	2.8316 × 10−2	2.9953 × 10−2
Oriental Avenue	2.1544 × 10−2	2.2874 × 10−2	2.1544 × 10−2	2.2863 × 10−2	2.1544 × 10−2	2.2863 × 10−2
Chance (South)	8.2434 × 10−3	8.7516 × 10−3	8.2425 × 10−3	8.7468 × 10−3	8.2425 × 10−3	8.7468 × 10−3
Vermont Avenue	2.2046 × 10−2	2.3404 × 10−2	2.2042 × 10−2	2.3388 × 10−2	2.2042 × 10−2	2.3388 × 10−2
Connecticut Avenue	2.1919 × 10−2	2.3266 × 10−2	2.1911 × 10−2	2.3248 × 10−2	2.1911 × 10−2	2.3248 × 10−2
Just Visiting	2.1630 × 10−2	2.2954 × 10−2	2.1617 × 10−2	2.2934 × 10−2	2.1617 × 10−2	2.2934 × 10−2
St. Charles Place	2.5807 × 10−2	2.7281 × 10−2	2.5792 × 10−2	2.7262 × 10−2	2.5792 × 10−2	2.7262 × 10−2
Electric Company	2.5050 × 10−2	2.4882 × 10−2	2.5036 × 10−2	2.4864 × 10−2	2.5036 × 10−2	2.4864 × 10−2
States Avenue	2.2000 × 10−2	2.4003 × 10−2	2.1982 × 10−2	2.3990 × 10−2	2.1982 × 10−2	2.3990 × 10−2
Virginia Avenue	2.4455 × 10−2	2.4864 × 10−2	2.4438 × 10−2	2.4855 × 10−2	2.4438 × 10−2	2.4855 × 10−2
Pennsylvania Railroad	2.3798 × 10−2	2.6501 × 10−2	2.3781 × 10−2	2.6498 × 10−2	2.3781 × 10−2	2.6498 × 10−2
St. James Place	2.6921 × 10−2	2.8063 × 10−2	2.6909 × 10−2	2.8065 × 10−2	2.6909 × 10−2	2.8065 × 10−2
Community Chest (West)	2.2957 × 10−2	2.5970 × 10−2	2.2949 × 10−2	2.5976 × 10−2	2.2949 × 10−2	2.5976 × 10−2
Tennessee Avenue	2.8093 × 10−2	2.9245 × 10−2	2.8087 × 10−2	2.9254 × 10−2	2.8087 × 10−2	2.9254 × 10−2
New York Avenue	2.7827 × 10−2	3.0561 × 10−2	2.7822 × 10−2	3.0575 × 10−2	2.7822 × 10−2	3.0575 × 10−2
Free Parking	2.7947 × 10−2	2.8516 × 10−2	2.7949 × 10−2	2.8531 × 10−2	2.7949 × 10−2	2.8531 × 10−2
Kentucky Avenue	2.5740 × 10−2	2.7947 × 10−2	2.5742 × 10−2	2.7963 × 10−2	2.5742 × 10−2	2.7963 × 10−2
Chance (North)	1.0270 × 10−2	1.0290 × 10−2	1.0272 × 10−2	1.0296 × 10−2	1.0272 × 10−2	1.0296 × 10−2
Indiana Avenue	2.5216 × 10−2	2.6884 × 10−2	2.5219 × 10−2	2.6900 × 10−2	2.5219 × 10−2	2.6900 × 10−2
Illinois Avenue	2.9513 × 10−2	3.1416 × 10−2	2.9519 × 10−2	3.1432 × 10−2	2.9519 × 10−2	3.1432 × 10−2
B&O Railroad	2.8496 × 10−2	3.0215 × 10−2	2.8502 × 10−2	3.0231 × 10−2	2.8502 × 10−2	3.0231 × 10−2
Atlantic Avenue	2.5031 × 10−2	2.6705 × 10−2	2.5033 × 10−2	2.6717 × 10−2	2.5033 × 10−2	2.6717 × 10−2
Ventnor Avenue	2.4844 × 10−2	2.6434 × 10−2	2.4846 × 10−2	2.6444 × 10−2	2.4846 × 10−2	2.6444 × 10−2
Water Works	2.7545 × 10−2	2.9165 × 10−2	2.7547 × 10−2	2.9175 × 10−2	2.7547 × 10−2	2.9175 × 10−2
Marvin Gardens	2.4035 × 10−2	2.5507 × 10−2	2.4036 × 10−2	2.5512 × 10−2	2.4036 × 10−2	2.5512 × 10−2
Go to Jail	0	0	0	0	0	0
Pacific Avenue	2.4926 × 10−2	2.6460 × 10−2	2.4927 × 10−2	2.6461 × 10−2	2.4927 × 10−2	2.6461 × 10−2
North Carolina Avenue	2.4466 × 10−2	2.5974 × 10−2	2.4469 × 10−2	2.5974 × 10−2	2.4469 × 10−2	2.5974 × 10−2
Community Chest (East)	2.2259 × 10−2	2.3655 × 10−2	2.2262 × 10−2	2.3657 × 10−2	2.2262 × 10−2	2.3657 × 10−2
Pennsylvania Avenue	2.3381 × 10−2	2.4836 × 10−2	2.3384 × 10−2	2.4840 × 10−2	2.3384 × 10−2	2.4840 × 10−2
Short Line Railroad	2.5532 × 10−2	2.7122 × 10−2	2.5536 × 10−2	2.7130 × 10−2	2.5536 × 10−2	2.7130 × 10−2
Chance (East)	8.1276 × 10−3	8.6306 × 10−3	8.1287 × 10−3	8.6337 × 10−3	8.1287 × 10−3	8.6337 × 10−3
Park Place	2.0597 × 10−2	2.1869 × 10−2	2.0601 × 10−2	2.1877 × 10−2	2.0601 × 10−2	2.1877 × 10−2
Luxury Tax	2.0564 × 10−2	2.1834 × 10−2	2.0566 × 10−2	2.1839 × 10−2	2.0566 × 10−2	2.1839 × 10−2
Boardwalk	2.4946 × 10−2	2.6390 × 10−2	2.4951 × 10−2	2.6391 × 10−2	2.4951 × 10−2	2.6391 × 10−2
Jail (First turn)	3.7755 × 10−2	4.0069 × 10−2	3.7755 × 10−2	4.0073 × 10−2	3.7755 × 10−2	4.0073 × 10−2
Jail (Second turn)	3.1430 × 10−2	0	3.1462 × 10−2	0	3.1462 × 10−2	0
Jail (Third turn)	2.6192 × 10−2	0	2.6219 × 10−2	0	2.6219 × 10−2	0

Table 6. Comparison of the convergence of the Monopoly Markov chains using the mEngel algorithm and the power method.

	$\Vert {\mathit{d}}_{\mathit{i},\mathbf{mEngel}}-{\mathit{d}}_{\mathit{i},\mathbf{Power}}{\Vert }_2$	Number of Iterations	Long Jail	Short Jail
Tolerance
0.0001		1	1.2969 × 10−5	1.2969 × 10−5
0.00001		2	8.2213 × 10−6	8.2733 × 10−6
0.001		3	1.6723 × 10−4	1.7547 × 10−4
0.001		4	6.2985 × 10−4	7.6839 × 10−4
0.001 (0.01)		5	9.8909 × 10−4	1.1038 × 10−3
0.01		6	1.3026 × 10−3	1.5551 × 10−3
0.01		7	1.4098 × 10−3	1.8264 × 10−3
0.01		8	1.3658 × 10−3	1.8486 × 10−3
0.01		9	1.3059 × 10−3	2.1228 × 10−3
0.01		10	1.3650 × 10−3	2.3236 × 10−3
0.001 (0.01)		11	9.6764 × 10−4	2.1712 × 10−3
0.001 (0.01)		12	9.0800 × 10−4	1.8861 × 10−3
0.001 (0.01)		13	7.6921 × 10−4	1.6592 × 10−3
0.001 (0.01)		14	5.6854 × 10−4	1.4104 × 10−3
0.001 (0.01)		15	6.2546 × 10−4	1.3701 × 10−3
0.001 (0.01)		16	4.5819 × 10−4	1.4093 × 10−3
0.001 (0.01)		17	3.1808 × 10−4	1.2781 × 10−3
0.001		18	2.4727 × 10−4	8.9537 × 10−4
0.001		19	1.3592 × 10−4	6.5633 × 10−4
0.001		20	1.1921 × 10−4	6.0545 × 10−4
0.0001 (0.001)		21	9.8218 × 10−5	4.3479 × 10−4
0.001		22	1.0943 × 10−4	3.1397 × 10−4
0.0001 (0.001)		23	8.0613 × 10−5	2.4965 × 10−4
0.0001 (0.001)		24	5.5567 × 10−5	2.4997 × 10−4
0.0001 (0.001)		25	3.5889 × 10−5	1.5996 × 10−4
0.0001 (0.001)		26	5.0507 × 10−5	1.8358 × 10−4
0.0001 (0.001)		27	7.1482 × 10−5	2.5761 × 10−4
0.0001 (0.001)		28	7.8835 × 10−5	2.4739 × 10−4
0.0001 (0.001)		29	3.1048 × 10−5	1.7960 × 10−4
0.0001		30	1.7391 × 10−5	8.6980 × 10−5

Table 7. Ranking based on the stationary distribution of the Monopoly Markov chains using the Long Jail strategy for the $43\times 43$ model and the Short Jail strategy for the $41\times 41$ model.

Rank	States	Long Jail	States	Short Jail
1	Jail (First turn)	3.7755 × 10−2	Jail (First turn)	4.0069 × 10−2
2	Jail (Second turn)	3.1430 × 10−2	Illinois Avenue	3.1416 × 10−2
3	Illinois Avenue	2.9513 × 10−2	Go	3.1050 × 10−2
4	Go	2.9220 × 10−2	New York Avenue	3.0561 × 10−2
5	B&O Railroad	2.8496 × 10−2	B&O Railroad	3.0215 × 10−2
6	Reading Railroad	2.8312 × 10−2	Reading Railroad	2.9964 × 10−2
7	Tennessee Avenue	2.8094 × 10−2	Tennessee Avenue	2.9245 × 10−2
8	Free Parking	2.7947 × 10−2	Water Works	2.9165 × 10−2
9	New York Avenue	2.7827 × 10−2	Free Parking	2.8516 × 10−2
10	Water Works	2.7545 × 10−2	St. James Place	2.8063 × 10−2
11	St. James Place	2.6921 × 10−2	Kentucky Avenue	2.7948 × 10−2
12	Jail (Third turn)	2.6192 × 10−2	St. Charles Place	2.7281 × 10−2
13	St. Charles Place	2.5807 × 10−2	Short Line Railroad	2.7122 × 10−2
14	Kentucky Avenue	2.5740 × 10−2	Indiana Avenue	2.6885 × 10−2
15	Short Line Railroad	2.5533 × 10−2	Atlantic Avenue	2.6705 × 10−2
16	Indiana Avenue	2.5216 × 10−2	Pennsylvania Railroad	2.6501 × 10−2
17	Electric Company	2.5050 × 10−2	Pacific Avenue	2.6460 × 10−2
18	Atlantic Avenue	2.5031 × 10−2	Ventnor Avenue	2.6435 × 10−2
19	Boardwalk	2.4946 × 10−2	Boardwalk	2.6387 × 10−2
20	Pacific Avenue	2.4926 × 10−2	North Carolina Avenue	2.5974 × 10−2
21	Ventnor Avenue	2.4844 × 10−2	Community Chest (West)	2.5970 × 10−2
22	North Carolina Avenue	2.4466 × 10−2	Marvin Gardens	2.5507 × 10−2
23	Virginia Avenue	2.4455 × 10−2	Electric Company	2.4881 × 10−2
24	Marvin Gardens	2.4035 × 10−2	Virginia Avenue	2.4864 × 10−2
25	Pennsylvania Railroad	2.3798 × 10−2	Pennsylvania Avenue	2.4837 × 10−2
26	Pennsylvania Avenue	2.3381 × 10−2	States Avenue	2.4003 × 10−2
27	Community Chest (West)	2.2957 × 10−2	Community Chest (East)	2.3655 × 10−2
28	Community Chest (East)	2.2259 × 10−2	Income Tax	2.3618 × 10−2
29	Income Tax	2.2233 × 10−2	Vermont Avenue	2.3404 × 10−2
30	Vermont Avenue	2.2046 × 10−2	Connecticut Avenue	2.3266 × 10−2
31	States Avenue	2.1999 × 10−2	Just Visiting	2.2954 × 10−2
32	Connecticut Avenue	2.1919 × 10−2	Oriental Avenue	2.2874 × 10−2
33	Just Visiting	2.1630 × 10−2	Baltic Avenue	2.2016 × 10−2
34	Oriental Avenue	2.1544 × 10−2	Park Place	2.1869 × 10−2
35	Baltic Avenue	2.0722 × 10−2	Luxury Tax	2.1834 × 10−2
36	Park Place	2.0598 × 10−2	Mediterranean Avenue	2.1629 × 10−2
37	Luxury Tax	2.0564 × 10−2	Community Chest (South)	1.9181 × 10−2
38	Mediterranean Avenue	2.0355 × 10−2	Chance (North)	1.0290 × 10−2
39	Community Chest (South)	1.8052 × 10−2	Chance (South)	8.7516 × 10−3
40	Chance (North)	1.0270 × 10−2	Chance (East)	8.6306 × 10−3
41	Chance (South)	8.2434 × 10−3	Go to Jail	0
42	Chance (East)	8.1276 × 10−3	Jail (Second turn)	0
43	Go to Jail	0	Jail (Third turn)	0

Table 8. The probabilities of landing on Jail using the Long Jail strategy and the Short Jail strategy for the $43\times 43$ model, the $41\times 41$ model, and the $123\times 123$ model.

Rank	States	Long Jail	Rank	States	Short Jail
		$\mathbf{43}\times \mathbf{43}$Model			$\mathbf{41}\times \mathbf{41}$Model
1	Jail (First turn)	3.7755 × 10−2	1	Jail (First turn)	4.0069 × 10−2
2	Jail (Second turn)	3.1430 × 10−2	31	Just Visiting	2.2954 × 10−2
12	Jail (Third turn)	2.6192 × 10−2	43	Go to Jail	0
33	Just Visiting	2.1630 × 10−2
43	Go to Jail	0
		1.1701 × 10−1			6.3024 × 10−2
Rank	States	Long Jail	Rank	States	Short Jail
		$123\times 123$ model			$123\times 123$ model
1	Jail	8.3424 × 10−2	1	Jail	3.5222 × 10−2
28	Just Visiting	2.2933 × 10−2	30	Just Visiting	2.3671 × 10−2
41	Go to Jail	0	41	Go to Jail	0
		1.0636 × 10−1			5.8892 × 10−2

4.3. The $123\times 123$ Model

Let us define $\widehat{S}$ as the order set of 123 states and S as the order set of 43 states. Two sets $\widehat{S}$ and S have the same order as their matrices. Furthermore, we have divided each state $s_i$ (except Jail) into three states $s_i^1,s_i^2,s_i^3$. For

$$S=\{s_1,s_2,\dots ,s_{40},j_1,j_2,j_3\},$$

then

$$\widehat{S}=\{s_1^1,s_2^1,\dots ,s_{40}^1,s_1^2,s_2^2,\dots ,s_{40}^2,s_1^3,s_2^3,\dots ,s_{40}^3,j_1,j_2,j_3\}.$$

Let us construct the transition matrix $\widehat{P}$ for the $123\times 123$ model by making use of the first 40 squares of the regular transition matrix

$$P={\mathit{M}}_{43\times 43}{\widehat{\mathit{J}}}_{43\times 43}{\mathit{Ch}}_{43\times 43}{\mathit{Cc}}_{43\times 43}$$

additionally, for the last three states in the Long Jail strategy, $\mathit{M}$, $\mathit{Ch}$, and $\mathit{Cc}$ are same as for the $43\times 43$ model, and $\widehat{\mathit{J}}$ is different from $\mathit{J}$. $\widehat{\mathit{J}}$ is a diagonal matrix, but in the 31st row, ${\widehat{\mathit{J}}}_{31,31}=0$ and ${\widehat{\mathit{J}}}_{31,41}=1$, which is “Go to Jail”:

$$\widehat{P}=\begin{array}{cc}\begin{array}{c}1-40\\ 41-80\\ 81-120\\ 121-123\end{array}& (\end{array}\begin{array}{cc}\begin{array}{cccc}1-40& 41-80& 81-120& 121-123\\ \left({\displaystyle \frac{5}{6}}\right)P& \left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{5}{6}}\right)P& \left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{1}{6}}\right)P& J1\\ \left({\displaystyle \frac{5}{6}}\right)P& \left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{5}{6}}\right)P& \left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{1}{6}}\right)P& J1\\ \left({\displaystyle \frac{5}{6}}\right)P& \left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{5}{6}}\right)P& \left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{1}{6}}\right)P& J1\\ J3& 0& 0& J2\end{array}& ),\end{array}$$

where $J{1}_{ij}=\left\{\begin{array}{cc}{P}_{\left(imod40\right)41}\hfill & j=121\hfill \\ 0\hfill & j=122,123\hfill \end{array}\right.$, $J{2}_{ij}=P_{(i-80)(j-80)}$, and $J{3}_{ij}=P_{(i-80)j}$.

To clarify, $J1$ represents the transition probability from 40 normal states to 3 Jail states. Each state is equivalent to the state with an index 40 greater and 80 greater when used as the starting state for this round. Therefore, the probability $\widehat{P}$ in [1–40, 121–123], which denotes the probability of moving from the ‘1st to the 40th’ state to the ‘121st to the 123rd’ state, should be the same as $\widehat{P}$ in [41–80, 121–123] and [81–120, 121–123]. As a result, [1–40, 121–123], [41–80, 121–123], and [81–120, 121–123] are all denoted as $J1$. Moreover, the transition from [121–123] to [1–40] represents starting from 3 Jail states and transitioning to 40 normal states, which is equivalent to [41–43, 1–40] in P. Thus, this part should be labeled as $J3$. The transition from [121–123] to [41–120] represents starting from 3 Jail states and transitioning to 40 normal states with some doubles. However, since players cannot roll doubles to go to Jail when “leaving Jail”, the probability for this transition is 0. The transition from 3 Jail states to 3 Jail states is labeled as $J2$ because it is equivalent to [41–43, 41–43] in P.

If we use the transition matrix in the Long Jail strategy to calculate P, we will obtain the transition matrix for the Long Jail strategy with 120 states. Similarly, if we use the transition matrix in the Short Jail strategy to calculate P, we will obtain the transition matrix for the Short Jail strategy with 120 states.

It is important to note that both ${\widehat{P}}_{ij}$ and $P_{ij}$ represent the probability of going from state i to state j. The difference lies in the order of states, and $\widehat{P}$ has more states than P, as explained above (see S and $\widehat{S}$). Understanding the $123\times 123$ model is crucial for determining the relationship between the ith state in P and $\widehat{P}$. In ${\widehat{P}}_{ij}$, i is the starting state, and j is the ending state, using the symbols in S and $\widehat{S}$. When $s_i$ is the starting state, it is the same as $s_i^1$, $s_i^2$, and $s_i^3$, and this implies $P_{x|s_i}={\widehat{P}}_{x|s_i^1}={\widehat{P}}_{x|s_i^2}={\widehat{P}}_{x|s_i^3}$. However, when $s_i$ is the ending state, it is not the same as $s_i^1$, $s_i^2$, and $s_i^3$, but follows the relationship $P_{si|x}=\left({\displaystyle \frac{5}{6}}\right){\widehat{P}}_{s_i^1|x}=\left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{5}{6}}\right){\widehat{P}}_{s_i^1|x}=\left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{1}{6}}\right){\widehat{P}}_{s_i^1|x}$, where $x\in \widehat{S}$.

The structure consists of 43 states, including 40 regular states and 3 Jail states, and 123 states, including 120 regular states and 3 Jail states. No additional Jail states have been added.

Ranking based on the stationary distribution of the Monopoly Markov chains using the Long Jail strategy and the Short Jail strategy for the $123\times 123$ model is shown in Table 9. These two ranking results are similar. The probabilities of landing on Jail using the $123\times 123$ model as shown in Table 8 are 10.64% and 5.89%, respectively. The most-landed-on square in both strategies is Jail. Following “In Jail/Just Visiting”, the subsequent frequently visited square on the board is Illinois Avenue for Long Jail, but Illinois Avenue is in the third place for Short Jail.

Table 9. Ranking based on the stationary distribution of the Monopoly Markov chains using the Long Jail strategy and the Short Jail strategy for the $123\times 123$ model.

Rank	States	Long Jail	States	Short Jail
1	Jail	8.3424 × 10−2	Jail	3.5222 × 10−2
2	Illinois Avenue	2.9743 × 10−2	Go	3.1326 × 10−2
3	Go	2.9160 × 10−2	Illinois Avenue	3.1233 × 10−2
4	Reading Railroad	2.9144 × 10−2	Reading Railroad	3.0606 × 10−2
5	Tennessee Avenue	2.8775 × 10−2	New York Avenue	3.0448 × 10−2
6	New York Avenue	2.8685 × 10−2	B&O Railroad	2.9929 × 10−2
7	B&O Railroad	2.8552 × 10−2	Tennessee Avenue	2.9115 × 10−2
8	Free Parking	2.8474 × 10−2	Water Works	2.8870 × 10−2
9	St. James Place	2.7770 × 10−2	Free Parking	2.8416 × 10−2
10	St. Charles Place	2.7218 × 10−2	St. James Place	2.8065 × 10−2
11	Water Works	2.7150 × 10−2	St. Charles Place	2.8034 × 10−2
12	Kentucky Avenue	2.6364 × 10−2	Kentucky Avenue	2.7836 × 10−2
13	Electric Company	2.6236 × 10−2	Short Line Railroad	2.7040 × 10−2
14	Indiana Avenue	2.5571 × 10−2	Indiana Avenue	2.6761 × 10−2
15	Virginia Avenue	2.5452 × 10−2	Pennsylvania Railroad	2.6662 × 10−2
16	Pennsylvania Railroad	2.4969 × 10−2	Boardwalk	2.6606 × 10−2
17	Atlantic Avenue	2.4905 × 10−2	Atlantic Avenue	2.6411 × 10−2
18	Short Line Railroad	2.4834 × 10−2	Pacific Avenue	2.6226 × 10−2
19	Boardwalk	2.4734 × 10−2	Ventnor Avenue	2.6132 × 10−2
20	Ventnor Avenue	2.4582 × 10−2	Community Chest (West)	2.5814 × 10−2
21	Pacific Avenue	2.4270 × 10−2	North Carolina Avenue	2.5762 × 10−2
22	Community Chest (West)	2.3791 × 10−2	Electric Company	2.5514 × 10−2
23	North Carolina Avenue	2.3776 × 10−2	Marvin Gardens	2.5245 × 10−2
24	Marvin Gardens	2.3557 × 10−2	Virginia Avenue	2.5165 × 10−2
25	States Avenue	2.3298 × 10−2	Pennsylvania Avenue	2.4701 × 10−2
26	Vermont Avenue	2.3168 × 10−2	States Avenue	2.4455 × 10−2
27	Connecticut Avenue	2.3137 × 10−2	Income Tax	2.4130 × 10−2
28	Just Visiting	2.2933 × 10−2	Vermont Avenue	2.4088 × 10−2
29	Income Tax	2.2788 × 10−2	Connecticut Avenue	2.3972 × 10−2
30	Pennsylvania Avenue	2.2704 × 10−2	Just Visiting	2.3671 × 10−2
31	Oriental Avenue	2.2373 × 10−2	Community Chest (East)	2.3498 × 10−2
32	Community Chest (East)	2.1616 × 10−2	Oriental Avenue	2.3464 × 10−2
33	Baltic Avenue	2.1038 × 10−2	Baltic Avenue	2.2425 × 10−2
34	Mediterranean Avenue	2.0362 × 10−2	Luxury Tax	2.1955 × 10−2
35	Luxury Tax	2.0201 × 10−2	Park Place	2.1925 × 10−2
36	Park Place	2.0149 × 10−2	Mediterranean Avenue	2.1925 × 10−2
37	Community Chest (South)	1.8191 × 10−2	Community Chest (South)	1.9492 × 10−2
38	Chance (North)	1.0369 × 10−2	Chance (North)	1.0243 × 10−2
39	Chance (South)	8.6136 × 10−3	Chance (South)	8.9927 × 10−3
40	Chance (East)	7.9217 × 10−3	Chance (East)	8.6250 × 10−3
41	Go to Jail	0	Go to Jail	0

4.4. The Return of Monopoly

We know that some properties (states) in Monopoly have the same colors. If we collect all states with the same color, we will profit from it. Return is the ratio of expected money and cost. It shows how much money a player can take back at every turn. Players should buy all the properties of one color and choose the color based on which will provide the highest returns (e.g., [25]).

The formula for determining the return based on the number of turns is given by [26]:

$$\mathrm{Turn}={\displaystyle \frac{\mathrm{Cost}}{p·R·E\left(x\right)}}$$

where $\mathrm{Cost}$ is the development cost for a house or a hotel and is different for each property, p is the probability of landing on the property, R is the rent earnings, and $E\left(x\right)$ is the expectation of rolling x times in one turn. If a player rolls doubles three times, this player will go to Jail. So, a player can roll doubles at most three times before the next player rolls. Let x be the number of rolls in one turn, $x\in \{1,2,3\}$. Then, $p\left(x\right)$ is the probability of a player rolling x times in one turn. Then, $p\left(1\right)={\displaystyle \frac{30}{36}}={\displaystyle \frac{5}{6}}$, where the player rolls no doubles. $p\left(2\right)={\displaystyle \frac{6}{36}}\left({\displaystyle \frac{30}{36}}\right)={\displaystyle \frac{1}{6}}\left({\displaystyle \frac{5}{6}}\right)$, where the player rolls doubles once and non-doubles once. Furthermore, $p\left(3\right)={\displaystyle \frac{6}{36}}\left({\displaystyle \frac{6}{36}}\right)\left(1\right)={\displaystyle \frac{1}{6}}\left({\displaystyle \frac{1}{6}}\right)$, where the player rolls two doubles and then, regardless of what the player rolls next, their turn will end. So, the expectation of rolling x times in one turn is:

$$E\left(x\right)=1·p\left(1\right)+2·p\left(2\right)+3·p\left(3\right)=1.19\overline{4}.$$

The rent and cost values are sourced from [21,27]. Let us consider Mediterranean Avenue as an example, using the Long Jail strategy shown in Table A8 (see Appendix B). If the player owns Mediterranean Avenue, it takes approximately 1234 turns to recoup the cost of the purchase price. Similarly, if the player builds a house, generating revenue will take approximately 699 turns. Here are the rest of the cases:

• For $p=0.020355$, $\mathrm{Cost}=60$, $R=2$, the number of the turns is calculated as $\mathrm{Turn}=60·{\left(0.02\right)}^{-1}·2^{-1}·{\left(1.19\overline{4}\right)}^{-1}=1233.914$.
• For 1 house, with $R=10$ and $\mathrm{Cost}=170$, $\#\mathrm{of} \mathrm{Turns}=170·{\left(0.020355\right)}^{-1}·{10}^{-1}·{\left(1.19\overline{4}\right)}^{-1}=699.2179$.
• For 2 houses, with $R=30$ and $\mathrm{Cost}=220$, $\#\mathrm{of} \mathrm{Turns}=220·{\left(0.020355\right)}^{-1}·{30}^{-1}·{\left(1.19\overline{4}\right)}^{-1}=301.6234$.
• For 3 houses, with $R=90$ and $\mathrm{Cost}=270$, $\#\mathrm{of} \mathrm{Turns}=270·{\left(0.020355\right)}^{-1}·{90}^{-1}·{\left(1.19\overline{4}\right)}^{-1}=123.3914$.
• For 4 houses, with $R=160$ and $\mathrm{Cost}=320$, $\#\mathrm{of} \mathrm{Turns}=320·{\left(0.020355\right)}^{-1}·{160}^{-1}·{\left(1.19\overline{4}\right)}^{-1}=82.2609$.
• For a hotel, with $R=250$ and $\mathrm{Cost}=370$, $\#\mathrm{of} \mathrm{Turns}=370·{\left(0.020355\right)}^{-1}·{250}^{-1}·{\left(1.19\overline{4}\right)}^{-1}=60.8731$.

Now, let us examine the example of Mediterranean Avenue using the Long Jail strategy for the $123\times 123$ model, as shown in Table A9 (see Appendix B). An inspection of Table A9 reveals that the number of turns required to recoup the purchase price of Mediterranean Avenue (including properties, houses, and hotel), is nearly identical to that of the $43\times 43$ model.

Similar calculations were performed for all the states to calculate the number of turns needed in Monopoly to recoup the cost of the purchase price for each property. This was performed using the Long Jail strategy for both the $43\times 43$ model and the $123\times 123$ model, and using the Short Jail strategy for both the $41\times 41$ model and the $123\times 123$ model. The results are summarized in Table A8–Table A11 in Appendix B, respectively, where NA stands for not applicable. Although the stationary probabilities varied slightly for each property, the number of turns remained the same. In Table A8–Table A11 in Appendix B, we also observe how fast a player can start to make a profit from each state and each color group. For the state, Boardwalk and New York Avenue require the least number of turns before they start making money.

The rule for building on properties of the same color in Monopoly is that a player must own all the sites of the same color before building on them. We take Brown (Mediterranean Avenue and Baltic Avenue) as an example. To build one house on Mediterranean Avenue, they must first buy all the brown land. So, a player must pay for all the land before building a house. The land cost is 60 for each. Then, the house cost is 50 for each. So, the cost of land on brown is $2\left(60\right)=120$, the cost to build one house on brown is $2(60+50)=220$, and the cost to build two houses on brown is $2(60+(50\left)2\right)=320$. Therefore the cost of building one house on Mediterranean Avenue is $2\left(60\right)+50=170$. We want to calculate how much we need to pay to build h houses in some states. Let $\mathrm{baseCost}$ be the cost of buying all lands of the same color, and $\mathrm{houseCost}$ be the cost of building one house in a state. We first need to buy all the land of the same color to build a house. So, we need to pay $\mathrm{baseCost}$ first, and then we pay $h\times \mathrm{houseCost}$ to build h houses. Hence, we pay $\mathrm{baseCost}+h\times \mathrm{houseCost}$ to build h houses on the state, which is noted as $\mathrm{Cost}=\mathrm{baseCost}+h\times \mathrm{houseCost}$. Let n be the number of states in some color. If we want to calculate the cost of building h houses on each site in the same color group, which is noted as ${\mathrm{Cost}}_{\mathrm{color}}$, we need to pay $\mathrm{baseCost}$ to buy all lands and pay $h\times n\times \mathrm{houseCost}$ to build houses. So, ${\mathrm{Cost}}_{\mathrm{color}}=\mathrm{baseCost}+h\times n\times \mathrm{houseCost}$. We have $h\times \mathrm{houseCost}=\mathrm{Cost}-\mathrm{baseCost}$ then ${\mathrm{Cost}}_{\mathrm{color}}=\mathrm{baseCost}+n\times (\mathrm{Cost}-\mathrm{baseCost})$. For properties of the same color group, the formula is given as:

$$\mathrm{Turn}={\displaystyle \frac{{\mathrm{Cost}}_{\mathrm{color}}}{\sum p·\mathrm{Average}\mathrm{of}\mathrm{Rent}·E\left(x\right)}}$$

where ${\mathrm{Cost}}_{\mathrm{color}}=\mathrm{Cost}·n-\mathrm{baseCost}·(n-1).$

For instance, let us consider the Brown property using the Short Jail strategy. Below is the number of turns required for the Brown property group to generate income, given the player’s properties and two houses. Similar calculations apply for other houses and a hotel:

• $p=0.02125+0.02158=0.04282$, baseCost = 60 + 60 = 120, $R={\displaystyle \frac{2+4}{2}}=3$, the number of the turns is calculated as $\#\mathrm{of} \mathrm{Turns}=120·{\left(0.04282\right)}^{-1}·3^{-1}·{\left(1.19\overline{4}\right)}^{-1}=782.0731$.
• For two houses, with $R={\displaystyle \frac{30+60}{2}}=45$, $\mathrm{cost}=220+220-\mathrm{baseCost}·(2-1)=440$, $\#\mathrm{of} \mathrm{Turns}=320·{\left(0.04282\right)}^{-1}·{45}^{-1}·{\left(1.19\overline{4}\right)}^{-1}=307.0004$.

Table 10 summarizes similar calculations for all color groups using the Long Jail strategy for both the $43\times 43$ model and $123\times 123$ model, and using the Short Jail strategy for both the $41\times 41$ model and the $123\times 123$ model. The table shows the number of turns it takes for a player to recoup the cost of the purchase price. We compared the ranking based on the number of turns in Monopoly required to recoup the costs of owning hotels for the same color groups. Our observations indicate that the Orange and Light Blue properties are the quickest to start making money of all the colors.

Table 10. Turns in Monopoly for the same color property using the Long Jail strategy for both the $43\times 43$ model and the $123\times 123$ model, and using the Short Jail strategy for both the $41\times 41$ model and the $123\times 123$ model.

Same Color Property	Probability	Rent	Cost	Turn	Rent	Cost	Turn	Rent	Cost	Turn	Rent	Cost	Turn	Rent	Cost	Turn	Rent	Cost	Turn
Brown	4.108 × 10−2	3	120	815.1989	15	220	298.9063	45	320	144.9243	135	420	63.4044	240	520	44.1566	350	620	36.1017
Light Blue	6.551 × 10−2	6.67	320	613.1272	33.33	470	180.2142	93.33	620	84.8979	280	770	35.1446	416.67	920	28.2177	566.67	1070	24.1313
Light Purple	7.226 × 10−2	10.67	440	477.7760	53.33	740	160.7667	160	1040	75.3094	466.67	1340	33.2683	650	1640	29.2325	800	1940	28.0962
Orange	8.284 × 10−2	14.67	560	385.7908	73.33	860	118.5252	206.67	1160	56.7250	566.67	1260	22.4716	766.67	1560	20.5641	966.67	1860	19.4459
Red	8.047 × 10−2	18.67	680	378.9349	93.33	1130	125.9671	266.67	1580	61.6429	716.67	2030	29.4698	891.67	2480	28.9366	1066.67	2930	28.5784
Yellow	7.391 × 10−2	22.66	800	399.9088	113.33	1250	124.9384	340	1700	56.6371	816.67	2150	29.8210	991.67	2600	29.6987	1166.67	3050	29.6130
Green	7.277 × 10−2	20	920	529.2240	136.67	1520	127.9535	410	2120	59.4886	933.33	2720	33.5286	1133.33	3320	33.7026	1316.67	3920	34.2524
Dark Blue	4.554 × 10−2	42.5	750	324.4243	187.5	1150	112.7555	550	1550	51.8096	1250	1950	28.6791	1500	2350	28.8017	1750	2750	28.8892
Railroad	1.061 × 10−1	200	800	31.5511
Utility	5.260 × 10−2	70	300	68.2137
Same Color Property	Probability	Rent	Cost	Turn	Rent	Cost	Turn	Rent	Cost	Turn	Rent	Cost	Turn	Rent	Cost	Turn	Rent	Cost	Turn
Brown	4.140 × 10−2	3	120	808.8979	15	220	296.5959	45	320	143.8041	135	420	62.9143	240	520	43.8153	350	620	35.8226
Light Blue	6.868 × 10−2	6.67	320	584.8277	33.33	470	171.8962	93.33	620	80.9793	280	770	33.5225	416.67	920	26.9153	566.67	1070	23.0175
Light Purple	7.597 × 10−2	10.67	440	454.4438	53.33	740	152.9157	160	1040	71.6317	466.67	1340	31.6437	650	1640	27.8050	800	1940	26.7241
Orange	8.523 × 10−2	14.67	560	374.9725	73.33	860	115.2016	206.67	1160	55.1343	566.67	1260	21.8415	766.67	1560	19.9875	966.67	1860	18.9006
Red	8.168 × 10−2	18.67	680	373.3214	93.33	1130	124.1011	266.67	1580	60.7297	716.67	2030	29.0332	891.67	2480	28.5079	1066.67	2930	28.1550
Yellow	7.304 × 10−2	22.66	800	404.6722	113.33	1250	126.4266	340	1700	57.3117	816.67	2150	30.1762	991.67	2600	30.0524	1166.67	3050	29.9658
Green	7.075 × 10−2	20	920	544.3340	136.67	1520	131.6067	410	2120	61.1875	933.33	2720	34.4859	1133.33	3320	34.6649	1316.67	3920	35.2303
Dark Blue	4.488 × 10−2	42.5	750	329.1952	187.5	1150	114.4136	550	1550	52.5715	1250	1950	29.1009	1500	2350	29.2252	1750	2750	29.3141
Railroad	1.075 × 10−1	200	800	31.1520
Utility	5.339 × 10−2	70	300	67.2043
Same Color Property	Probability	Rent	Cost	Turn	Rent	Cost	Turn	Rent	Cost	Turn	Rent	Cost	Turn	Rent	Cost	Turn	Rent	Cost	Turn
Brown	4.365 × 10−2	3	120	767.2021	15	220	281.3074	45	320	136.3915	135	420	59.6713	240	520	41.5568	350	620	33.9761
Light Blue	6.954 × 10−2	6.67	320	577.5951	33.33	470	169.7704	93.33	620	79.9779	280	770	33.1079	416.67	920	26.5825	566.67	1070	22.7328
Light Purple	7.615 × 10−2	10.67	440	453.3696	53.33	740	152.5542	160	1040	71.4624	466.67	1340	31.5689	650	1640	27.7392	800	1940	26.6610
Orange	8.787 × 10−2	14.67	560	363.7067	73.33	860	111.7404	206.67	1160	53.4779	566.67	1260	21.1853	766.67	1560	19.3870	966.67	1860	18.3328
Red	8.625 × 10−2	18.67	680	353.5408	93.33	1130	117.5255	266.67	1580	57.5119	716.67	2030	27.4949	891.67	2480	26.9974	1066.67	2930	26.6632
Yellow	7.865 × 10−2	22.66	800	375.8075	113.33	1250	117.4087	340	1700	53.2237	816.67	2150	28.0238	991.67	2600	27.9088	1166.67	3050	27.8283
Green	7.727 × 10−2	20	920	498.4034	136.67	1520	120.5018	410	2120	56.0241	933.33	2720	31.5760	1133.33	3320	31.7399	1316.67	3920	32.2576
Dark Blue	4.826 × 10−2	42.5	750	306.1393	187.5	1150	106.4004	550	1550	48.8895	1250	1950	27.0627	1500	2350	27.1784	1750	2750	27.2610
Railroad	1.138 × 10−1	200	800	29.4274
Utility	5.405 × 10−2	70	300	66.3837
Brown	4.435 × 10−2	3	120	755.0929	15	220	276.8674	45	320	134.2387	135	420	58.7295	240	520	40.9009	350	620	33.4398
Light Blue	7.152 × 10−2	6.67	320	561.6046	33.33	470	165.0704	93.33	620	77.7637	280	770	32.1914	416.67	920	25.8465	566.67	1070	22.1035
Light Purple	7.765 × 10−2	10.67	440	444.6117	53.33	740	149.6073	160	1040	70.0819	466.67	1340	30.9590	650	1640	27.2034	800	1940	26.1459
Orange	8.763 × 10−2	14.67	560	364.7028	73.33	860	112.0464	206.67	1160	53.6243	566.67	1260	21.2433	766.67	1560	19.4401	966.67	1860	18.3830
Red	8.583 × 10−2	18.67	680	355.2708	93.33	1130	118.1006	266.67	1580	57.7934	716.67	2030	27.6294	891.67	2480	27.1295	1066.67	2930	26.7937
Yellow	7.779 × 10−2	22.66	800	379.9622	113.33	1250	118.7067	340	1700	53.8121	816.67	2150	28.3336	991.67	2600	28.2174	1166.67	3050	28.1360
Green	7.600 × 10−2	20	920	506.7319	136.67	1520	122.5154	410	2120	56.9603	933.33	2720	32.1036	1133.33	3320	32.2702	1316.67	3920	32.7967
Dark Blue	4.853 × 10−2	42.5	750	304.4361	187.5	1150	105.8084	550	1550	48.6175	1250	1950	26.9122	1500	2350	27.0272	1750	2750	27.1093
Railroad	1.142 × 10−1	200	800	29.3141
Utility	5.438 × 10−2	70	300	66.1024

The hotel returns ranking results for both strategies, as shown in Table 11. These results are consistent with sources [18,26].

Table 11. Ranking the number of turns in Monopoly for properties of the same color with a hotel, using the Long Jail strategy for both the $43\times 43$ model and the $123\times 123$ model, and using the Short Jail strategy for both the $41\times 41$ model and the $123\times 123$ model.

Rank of Hotel Return	States	Long Jail $43\times 43$ Model	States	Short Jail $41\times 41$ Model
1	Brown	36.10167 ≈ 36	Brown	33.97609 ≈ 34
2	Green	34.25238 ≈ 34	Green	32.25762 ≈ 32
3	Yellow	29.61302 ≈ 30	Yellow	27.82833 ≈ 28
4	Dark Blue	28.88921 ≈ 29	Dark Blue	27.26097 ≈ 27
5	Red	28.57838 ≈ 29	Red	26.66321 ≈ 27
6	Light Purple	28.09622 ≈ 28	Light Purple	26.66097 ≈ 27
7	Light Blue	24.13126 ≈ 24	Light Blue	22.7328 ≈ 23
8	Orange	19.44593 ≈ 19	Orange	18.33277 ≈ 18
Rank of Hotel Return	States	Long Jail $\mathbf{123}\times \mathbf{123}$Model	States	Short Jail $\mathbf{123}\times \mathbf{123}$Model
1	Brown	35.82262 ≈ 36	Brown	33.43983 ≈ 33
2	Green	35.23033 ≈ 35	Green	32.79666 ≈ 33
3	Yellow	29.96575 ≈ 30	Yellow	28.13598 ≈ 28
4	Dark Blue	29.31405 ≈ 29	Dark Blue	27.10931 ≈ 27
5	Red	28.15502 ≈ 28	Red	26.79369 ≈ 27
6	Light Purple	26.72414 ≈ 27	Light Purple	26.14594 ≈ 26
7	Light Blue	23.01746 ≈ 23	Light Blue	22.10345 ≈ 22
8	Orange	18.90063 ≈ 19	Orange	18.38298 ≈ 18

5. Conclusions

In the present study, we have unveiled the mEngel algorithm, which addresses the computation of absorbing and non-absorbing Markov chains through a nested iterative process. We have also furnished proof to demonstrate the efficacy of the mEngel algorithm in resolving non-absorbing Markov chain issues, correlating it with the power method’s procedures and the canonical decomposition of absorbing Markov chains. The mEngel algorithm was implemented in a sequential manner using R, Algorithms 1–3 being elucidated in detail. Our proposed algorithm has two key features that set it apart from the original Engel algorithm [1,2,3] and the approach in [4]:

• It can be applied to non-absorbing and absorbing Markov chains, whereas the original Engel algorithm is limited to absorbing cases.
• It provides process values, such as the passing frequency of intermediate states, which traditional methods cannot provide. This capability allowed us to identify which state had been firing, referred to as a firing state, and record this information for each configuration.

We explored the Engel algorithm’s capability for computing the absorbing probabilities for Torrence’s problem, with findings that align with established scholarly works. Determining the steady-state probabilities for non-absorbing states in Monopoly, particularly concerning the Jail rules, was also investigated. Using the Long Jail strategy, the Short Jail strategy, and the strategy of getting out of Jail by rolling consecutive doubles three times, the $43\times 43$ model, the $41\times 41$ model, and the $123\times 123$ model, were formulated and tested, and their results are consistent with those in the existing literature. Our findings show that using the Short Jail strategy is a common practice. Early in the game, being in Jail reduces the player’s ability to purchase property so the player should get out of Jail immediately if the player can pay USD 50 or use a “Get Out of Jail Free” card from either the Community Chest or the Chance card pile, moving the player’s token to “Just Visiting”. Late in the game, being in Jail can protect the player from paying an opponent’s rent. Furthermore, rewards to each state and color group under different Jail strategies are calculated, allowing players to determine their best strategy based on the results. Further research is necessary when considering a case-by-case card evaluation function [28] to assess the value of properties using the mEngel algorithm. The main drawback of the mEngel algorithm is that it takes more computational time to achieve high accuracy compared to other methods in the same problem setting. To decrease the computational time, it is essential to reorganize the architecture of Algorithms 1–3 by, for example, adjusting the number of iterations, nested loops, and recursive calls. These results will be published elsewhere.