The Waiting Time Distribution of Competing Patterns in Markov-Dependent Bernoulli Trials

Abstract

Competing patterns are compound patterns that compete to be the first to occur a pattern-specific number of times, known as a stopping rule. In this paper, we study a higher-order Markovian dependent Bernoulli trials model with competing patterns. The waiting time distribution refers to the distribution of the number of trials required until the stopping rule is met. Based on a finite Markov chain, a hierarchical algorithm is proposed to derive the conditional probability generating function (pgf) of the waiting time of the competing patterns model. By applying the law of total expectation, the final pgf is then obtained. Using examples, we further demonstrate that the proposed algorithm is an effective and easy-to-implement tool.

1. Introduction and Literature Review

The waiting time distribution for a sequence of trials refers to the distribution of the number of trials required until a specific stopping rule is satisfied. A series of identical trials is referred to as a “run”, while a general sequence is called a “pattern”. The waiting times of a sequence of trials have been extensively studied with different stopping rules. Most of these studies employ combinatorial analysis, assuming a sequence of independent and identically distributed (i.i.d.) trials, each of which ends in two or more outcomes.

The most basic model dates to i.i.d. Bernoulli trials (each resulting in either a success “1” or a failure “0”), where the stopping rule is the occurrence of the first success (“1”). Here, the waiting time is a geometrically distributed random variable (r.v.). When the stopping rule is extended to the k-th success, the waiting time is Negative Binomial (NB) r.v. More complex stopping rules include specific sequences, combinations of multiple stopping rules, multi-state trials, and Markovian models (with dependent trials).

The waiting time distributions of runs and patterns, such as geometric, geometric of order k, negative binomial, and sooner and later, have been successfully applied in numerous areas of statistics and applied probability. In recent decades, the theory of waiting time distributions has become an indispensable tool for studying various applications, including DNA sequence homology (Schwager [1], Karwe and Naus [2,3]), epidemiology (Kulldorff [4]), and system reliability (Aki [5], Aki and Hirano [6], Chang and Huang [7]). We will now present some specific examples.

Studying the distribution of patterns assists in DNA sequence analysis by modeling the occurrence of specific motifs, repeats, or base combinations. Each nucleotide position is treated as a Bernoulli trial, allowing researchers to detect biologically significant patterns such as transcription factor binding sites, identify mutation hotspots, and study codon usage bias. It also aids in evaluating the statistical significance of sequence alignments and testing whether observed patterns are random or functionally important, providing insights into DNA structure, function, and evolution. In this regard, Kulldorff [4] investigated the occurrences of sudden infant death syndrome or birth defects using the Bernoulli model. In psychology, Schwager [1] explored the concept of “success breeds success” or “failure breeds failure” which is applied in achievement testing, animal learning studies, athletic competition, and study performance improvement. He further demonstrated that the behavior of groups of people forming lines and other structures can be modeled as a Markov-dependent sequence of trials in which some characteristic, such as the sex of the individual, is taken as a trial outcome. Dafnis [8] demonstrated a practical application for the waiting time distribution of binary trials in the topic of meteorology and agriculture, by considering the cultivation of raisins. The harvesting of certain varieties of raisins in Greece must occur between August and September. For the harvesting, a period of at least four consecutive dry days is required. Before this period begins, a period of at least two consecutive rainy days is required to water the raisins. Dafnis [8] used “0” to denote the occurrence of a rainy day, and “1” to denote the occurrence of a dry one. The probabilities of “0” and “1” were estimated using previous years’ statistics.

In the field of agriculture, the concept of r-weak runs distribution was applied to identify the rate of development of crops (which recognizes that plant development will occur only when the temperature exceeds a specific base temperature for a certain number of days), and to investigate the impact of critical factors on honeydew honey production (Dafnis et al. [9], Dafnis et al. [10]). Drawing our attention to reliability studies, we mention the consecutive k-out-of-n systems. In these systems, each component is either working or failed, and the overall system functions only when at least k consecutive components are working within the total n components in the sequence. Dafnis et al. [11] studied the k-out-of-n system that fails if and only if a string of k non-functioning components is interrupted by a string of at most r consecutive functioning ones. They showed that there are ${(r+1)}^{k-1}$ different patterns of appearance that can cause a system failure. In the financial field, Dafnis and Marki [12] showed how the concept of r-weak runs can be adapted by financial advisors and technical analysts for the determination of a personalized and effective investing strategy with controlled risk.

For the literature review, we date back to Feller [13], who studied the probability generating function (pgf) for the waiting time distribution of a sequence of i.i.d Bernoulli trials where the stopping condition is the first occurrence of a series of k consecutive successes. Philippou et al. [14] extended Feller [13] by studying a geometric distribution of order k, defined as the waiting time distribution until k consecutive successes occur. Aki [5] generalized the model to geometric, NB, Poisson, logarithmic series, and binomial distributions of order k with dependencies. Philippou and Makri [15] studied the binomial distribution of order k in a finite number of Bernoulli trials. Philippou [16] studied multi-state trials, and Ling [17] extended the k-th ordered geometric model with parameter p to $(k,\dots ,k_m)$-order with parameter $(p_1,\dots ,p_n)$; the exact distributions and the pgfs were further derived for some special cases of $(k,\dots ,k_m)$. Shmueli and Cohen [18] used recursive formulas to compute the exact probability functions of a model with switching rules. Koutras and Eryilmaz [19] considered a compound geometric distribution of order k determined by another random process, such as Poisson or binomial.

The above literature assumes that the experiment ends after the first k consecutive successes. A more general stopping rule is to stop the experiment with the first occurrence of a particular sequence of i.i.d. Bernoulli experiments, known as a pattern. A pattern has a specific sequence and may include different symbols. In this context, Blom and Thorburn [20] analyzed the waiting time distribution until a k-digit sequence is obtained or, more generally, until one of several k-digit sequences is obtained. For the latter case, the mean waiting time was also derived. Ebneshahrashoob and Sobel [21] studied a generalized pgf, means and variances for the waiting time until obtaining a sequence of s successes or a sequence of r failures, whichever comes sooner. Huang [22] introduced a generalized stopping $(k_1,k_2)$-rule as the occurrence of $k_1$ consecutive failures followed by $k_2$ consecutive successes. Dafnis et al. [8], Makri [23] and Kumar and Upadhye [24] considered different types of $(k_1,k_2)$-rules. Zhao et al. [25], Kong [26] and Chadjiconstantinidis and Eryilmaz [27] studied the distributions of $(k_1,\dots ,k_m)$-runs of multi-state trials.

To date, we focused on the distribution of the first instance of a pattern, or the $(k_1,\dots ,k_m)$-type model. We next reviewed the literature dealing with the distribution for the r-th occurrence of a pattern. Aki [28] investigated the waiting time distribution until the r-th occurrence of a pattern in a sequence of i.i.d. Bernoulli trials. Koutras [29] derived several moments of the waiting time for the non-overlapping appearance of a pair of successes separated by at most k − 2 failures ($k>2$). Robin and Daudin [30] obtained the distribution of distances between the two successive occurrences of a specific pattern, as well as between the n-th and the $(n+m)$-th occurrences. Aki and Hirano [31] further explored a two-dimensional pattern model.

Focusing on models under the assumption of Markov dependency, Hirano and Aki [32] studied the distribution of the number of success runs of length k until the n-th trial in a two-state dependent Markov chain. Fu and Koutras [33] presented an approach for the distribution theory of runs based on a finite Markov chain embedding technique that covers identical and nonidentical Bernoulli trials. In addition, the exact distribution of the waiting time for the m-th occurrence of a specific run, and the distribution of the number of at least k successes were derived. The number of failures, successes, and the first consecutive k successes were studied in Aki and Hirano [31]. Using non-overlapping counting, Fu [34] studied the multi-state trials model. Koutras [35] developed a general technique for the waiting time distribution in a two-state dependent Markov chain. Antzoulakos [36] introduced a variation of the finite Markov chain embedding method to derive the pgf until the r-th occurrence of a pattern, considering both non-overlapping and overlapping cases. Fisher and Cui [37] introduced a mathematical framework for determining the expected time for a specific pattern to emerge in higher-order Markov chains, both with and without a predefined starting sequence. This approach was extended to the calculation of the first occurrence of any pattern from a collection, along with the probability that each individual pattern is the first to appear. Chang et al. [38] studied the dual relationship between the probability of the number of patterns and the probability of the waiting time in a sequence of multi-state trials.

We next focused on models with stopping rules that involve a few simple patterns, known as the compound patterns rule, where the stopping rule is triggered by the first appearance of one of the patterns. Applying pgf techniques, Fu and Chang [39] and Han and Hirano [40] investigated multi-state trial models with compound patterns for both i.i.d. trials and first-order Markov-dependent trials. For the r-th order Markov-dependent chain, Fu and Lou [41] examined the waiting time distributions of the first occurrence of simple and compound patterns in the sequences of Bernoulli trials. Wu [42] applied a finite Markov chain embedding technique to analyze the conditional waiting time distributions. Using a matrix form, Aston and Martin [43] presented an algorithm that computes the distribution of the waiting time until the m-th occurrence of a compound pattern. For an excellent summary of various calculation methods, we refer readers to the comprehensive book by Fu and Lou [44] and Balakrishnan and Koutras [45].

Our research addresses models with a competing pattern stopping rule. Here, the experiment ends when a simple or compound pattern occurs a specific number of times (note that the compound pattern rule is a special case, where the experiment ends after the first occurrence). For a real-life example, consider again the cultivation of raisins (Dafnis [8]). To obtain raisins of fine quality, grapes need successions of short rainy and dry periods. Thus, an agriculturalist cares about the frequent occurrence of patterns with at most 2 consecutive rainy days followed by at most 2 consecutive dry days. Agriculturalists further claim that the occurrence of at least five such patterns in the three months has a significant effect on the quality of raisins.

Another real-life example is taken from Dafnis et al. [9]. They studied the effect of the number of cold days on the life cycle of Marchalina hellenica. They showed that the number of cold periods (each of which is a run of k consecutive cold days) is a more critical factor to the completion time of the insect’s biological cycle than the total number of cold days.

For models with competing patterns, Aston and Martin [43] investigated the waiting time distribution for competing patterns in m-th order dependent multi-state Markov trials. They analyzed several compound patterns, each associated with a specific required number of occurrences. Martin and Aston [3] introduced the generalized later patterns model, in which all patterns must appear multiple times. We also mention the closely related sooner and later models. Sooner waiting time distribution captures the number of trials required for the first occurrence of one of two competing patterns, and conversely, later waiting time distribution refers to the number of trials required for both patterns to occur (see, e.g., Han and Hirano [40], Balakrishnan and Koutras [45]).

For other related models dealing with waiting time distributions under the Markovian assumption, we mention the hidden Markov processes (Aston and Martin [46]), in which the states are not directly observable, or the sparse Markov process, in which the transition probability matrix includes many zero or near-zero entries (see, e.g., Martin [47,48]). Dafnis [10] studied the model with independent but not necessarily identically distributed trials. Michael [49] and Vaggelatou [50] presented a framework for a continuous time Markov chain. Dafnis et al. [9] introduced the r-weak run of length at least k in a sequence of binary trials. Their model was extended to include minimum and maximum constraints (Dafnis and Makri [12]). In a series of papers, Makri ([51,52]) investigated a sequence of binary trials with a specific length threshold.

Our paper contributes to this study by deriving simple and closed-form expressions for the pgf of waiting time distributions associated with higher-order Markovian-dependent Bernoulli trials with competing patterns. To the best of our knowledge, the studies presented to date are based on Markov chain-embedding techniques, where the state space is large, leading to a complicated transition probabilities matrix, high computational complexity and, thus, it is difficult to implement. The suggested algorithm is based on a hierarchical approach and includes three steps. In the first step, the framework is designed by including the state space, stopping rules, and steady-state probabilities. The second step determines all the paths that terminate the experiment; each such path is divided into sub-paths (components) where the total pgf is the product of the pgfs of these sub-paths. The last step derives the pgf of each sub-path by considering whether the path is longer or shorter than the number of trials of the longest pattern. The final pgf is obtained by applying the law of total expectation.

The rest of the paper is organized as follows. Section 2 introduces the definitions and preliminaries to be used. Using a hierarchical approach, Section 3 provides a mathematical description of the model and derives the pgf of the waiting time; this derivation is demonstrated by an example. A summarizing algorithm and additional examples are provided in Section 4. Finally, Section 5 concludes the paper and suggests some future directions. Following the convention, we indicate vectors by bold letters, and matrices by blackboard bold letters. We let $1_{\left\{A\right\}}$ be the indicator of an event $A,$ $\mathit{e}={(1,\dots ,1)}^T$ be the column vector of all ones, and $\mathbb{I}$ be the identity matrix, all of the appropriate dimensions. We use $\left|\mathcal{A}\right|$ to denote the number of elements in a set $\mathcal{A}.$ Summarizing our abbreviations, we use pgf(s) for probability generating function(s), i.i.d r.v.(s) for independent and identical distributed random variable(s).

2. Definitions and Preliminaries

We use the terminology of Fo and Lou [41] and Aston and Martin [43].

Bernoulli trial. Consider a sequence of Bernoulli trials $X_1,X_2,\dots ,$ where each trial (symbol)$X_t$ has two possible outcomes, success and failure (0 and 1, respectively) with $p(X_t=1)=p$ and $p(X_t=0)=1-p=q.$ Let $\mathcal{S}$ be the state space of an individual result $X_t,$ i.e., $\mathcal{S}=\{0,1\}.$

A simple pattern. A simple pattern $\mathsf{\Lambda }$ is a specific sequence of k trials (symbols), $x_{i,1},$ $x_{i,2}\dots ,$ $x_{i,k}$ from $\mathcal{S}$, i.e., $\mathsf{\Lambda }=x_{i,1},\dots x_{i,k}.$ The waiting time random variable of a simple pattern, $W\left(\mathsf{\Lambda }\right),$ is defined as

$$W\left(\mathsf{\Lambda }\right)=inf\{n\in \mathbb{N}:n\ge l,X_{n-k+1}=x_{i,1},\dots ,X_n=x_{i,k}\},$$

i.e., … $W\left(\mathsf{\Lambda }\right)$ counts the number of trials until the first occurrence of pattern $\mathsf{\Lambda }$. In the following, we assume that we start at $t=1$.

A compound pattern $\mathsf{\Lambda }={\cup }_{i=1}^l{\mathsf{\Lambda }}_i$ is the union of l simple patterns. We use ${\mathsf{\Lambda }}_i\cup {\mathsf{\Lambda }}_j$ to denote the occurrence of either pattern ${\mathsf{\Lambda }}_i$ or pattern ${\mathsf{\Lambda }}_j$. Accordingly, the waiting time $W\left(\mathsf{\Lambda }\right)$ is defined as

$$W\left(\mathsf{\Lambda }\right)=inf\{n\in \mathbb{N}:\mathrm{the}\mathrm{occurrence}\mathrm{of}\mathrm{any}\mathrm{pattern}{\mathsf{\Lambda }}_1,\dots ,{\mathsf{\Lambda }}_l\mathrm{during}\mathrm{the}n\mathrm{trials}\}.$$

Note that $W\left(\mathsf{\Lambda }\right)=inf\{W\left({\mathsf{\Lambda }}_1\right),W\left({\mathsf{\Lambda }}_2\right),\dots ,W\left({\mathsf{\Lambda }}_l\right)\}.$

A (discrete) Markov chain. A discrete-time Markov chain is a sequence of random variables $X_1,X_2,X_3$, … with the Markov property, namely that the probability of moving to the next state only depends on the present state and not on the previous states:

$$p(X_{t+1}=x_{t+1}\mid X_1=x_1,\dots ,X_t=x_t)=p(X_{t+1}=x_{t+1}\mid X_t=x_t)$$

if both conditional probabilities are well defined, that is, if $p(X_1=x_1,\dots ,X_t=x_t)>0.$ The possible values of $X_i$ form a countable set $\mathcal{S}$, called the state space of the chain. Time-homogeneous Markov chains are processes where $p(X_{t+1}=y\mid X_t=y)=p(X_t=y\mid X_{t-1}=y)$ for all t, i.e., the probability of the transition is independent of t (for more details, see Feller [13] and Hirano and Aki [32]).

An r-th order Markov chain. Let $\left\{X_t\right\}$ be a sequence of irreducible, aperiodic, and homogeneous r-th order Markov-dependent m-state random variables (trials) defined on the state space $\mathcal{S}=\{b_1,b_2,\dots b_m\}.$ (when $m=2,$ we have a Bernoulli trial). For $r\ge 1,$ the set of all possible r tuples ($m^r$ combinations) ${\mathcal{S}}^r$ is given by

$${\mathcal{S}}^r=\{\mathit{x}=(x_1\dots x_r):x_i\in \mathcal{S},i=1,\dots ,r\}.$$

The r-order transition probabilities of the Markov chain $\left\{X_t\right\}$ are defined by:

$$P(b\mid x_1,\dots ,x_r)=p(X_t=b\mid X_{t-r}=x_1,\dots ,X_{t-1}=x_r)=p_{\mathit{x},b},\mathit{x}={\mathcal{S}}^r,b\in \mathcal{S},$$

which is independent of t. The steady-state probability vector $\mathit{\pi }=({\pi }_{\mathit{x}}:$ $\mathit{x}\in {\mathcal{S}}^r)$ exists and satisfies $\mathit{\pi }\mathbb{A}=\mathit{\pi },\mathit{\pi }\mathit{e}=1,$ where $\mathbb{A}$ is the $(m^r\times m^r)$ transition probability matrix.

In this work, we consider Bernoulli trials with $m=2$.

Result 1. Let $\left\{X_t\right\}$ be an r-th order Markov chain. Fu and Lou [41] showed that there exists an embedded finite Markov chain $\left\{Y_t\right\}$ defined on a state space $\mathsf{\Omega }\left(\mathsf{\Lambda }\right)=\left(\mathsf{\Omega }\left(\mathsf{\Lambda }\right)-\left\{\alpha \right)\right)\cup \left\{\alpha \right\},$ where $\left\{\alpha \right\}$ is the absorbing state. Accordingly, the transition probability matrix has the form

$$\mathbb{M}=\begin{array}{c}\mathsf{\Omega }\left(\mathsf{\Lambda }\right)-\left\{\alpha \right)\\ \alpha \end{array}\left(\begin{array}{cc}\mathbb{N}& \mathit{C}\\ \mathbf{0}& \mathbf{1}\end{array}\right).$$

(Note $\mathbb{N}\mathit{e}+\mathit{C}=\mathit{e},$ i.e., the sum of the probabilities in each row of $\mathbb{M}$ is equal to 1). The waiting time of a pattern (simple or compound) $W\left(\mathsf{\Lambda }\right)$ has a general geometric distribution

$$p(W\left(\mathsf{\Lambda }\right)=n)=\mathit{\xi }{\mathbb{N}}^{n-r-1}(\mathbb{I}-\mathbb{N})\mathit{e}\mathrm{and}p(W\left(\mathsf{\Lambda }\right)\ge n)=\mathit{\xi }{\mathbb{N}}^{n-r-1}\mathit{e},n\ge r+1.$$

($\mathit{\xi }$ is the initial distribution, and $\mathbb{N}$ is known as the essential transition probability matrix). For more details, see Lemma 3.1, Theorems 3.1–4.2 of Fu and Lou [41].

Competing patterns. Let $\{{\mathsf{\Lambda }}^{\left(1\right)},\dots ,{\mathsf{\Lambda }}^{\left(c\right)}\},c\ge 1$, be the set of c compound patterns. Let $n_i$ denote the number of occurrences of the compound pattern ${\mathsf{\Lambda }}^{\left(i\right)}$ that terminates the experiment. The patterns $\{{\mathsf{\Lambda }}^{\left(1\right)},\dots ,{\mathsf{\Lambda }}^{\left(c\right)}\}$ are called competing patterns. We assume that no two competing compound patterns are identical.

Let $C_i\left(n\right),i=1,\dots ,c$ be the event that, by time n, the compound pattern ${\mathsf{\Lambda }}^{\left(i\right)}$ has occurred $n_i$ times. Then, the $p\left({\bigcup }_{i=1}^c{C}_i\left(n\right)\right)$ is the waiting time probability function of the competing patterns.

We further note that two distinct methods of counting patterns are considered in the literature (Inoue and Aki [53]). (1). Non-overlapping counting. In this case, when a pattern occurs, the counting restarts from that point, and any partially completed pattern cannot be finished. (2). Overlapping counting. In this case, partially completed patterns can be finished at any time, regardless of whether another pattern has been completed after the partially completed pattern starts but before it is completed.

Furthermore, we have two more definitions (Aston and Martin [43]): Ending blocks of a simple pattern $\mathsf{\Lambda }=\{b_i,\dots ,b_{i_k}\}$ are sub-patterns of the form $\left\{b_i,\dots ,b_{i_q}\right\}$, where $q\in \{1,2,\dots ,k-1\}$. Ending blocks always start at the beginning of a simple pattern but end at any point before its last symbol. Finishing blocks of the simple pattern $\mathsf{\Lambda }=\{b_{i_1},\dots ,b_{i_k}\}$ are sub-patterns of the form $\left\{b_{i_{\zeta }},\dots ,b_{i_k}\right\}$, where $\zeta \in \{1,2,\dots ,k\}.$ Finishing blocks may start at any point but always end with the last symbol. The finishing blocks of a compound pattern or competing patterns are formed by taking the union of the finishing blocks of their respective components.

Result 2. Consider the competing patterns $\{{\mathsf{\Lambda }}^{\left(1\right)},\dots ,{\mathsf{\Lambda }}^{\left(c\right)}\}$. Aston and Martin [43] showed that the waiting time distribution for competing compound patterns has a geometric form. Specifically, the competing pattern experiment ends with compound pattern ${\mathsf{\Lambda }}^{\left(i\right)}$ occurred $n_i$ times if and only if the Markov chain $\left\{Y_t\right)$ is absorbed in the corresponding absorbing state. Therefore, the probability function of the waiting time is given by

$$P\left(W\left(\mathsf{\Lambda }\right)=n\right)=p\left({\bigcup }_{i=1}^c{C}_i\left(n\right)\right)={\mathit{\psi }}_0{\mathbb{T}}_Y^n{\mathit{e}}_{\mathsf{\Lambda }},$$

where ${\mathit{\psi }}_0$ is the initial probability vector of $\left\{Y_t\right\},$ ${\mathbb{T}}_Y$ is the transition probability matrix, and ${\mathit{e}}_{\mathsf{\Lambda }}$ is a column vector with 1’s in the position corresponding to the absorbing states and 0’s elsewhere (see Section 3.2 of Aston and Martin [43]).

Probability Generating Function

The probability generating functions (pgf) is a useful technique for computing distributions (see, e.g., Feller [13], chapter XI). As a short background, for a non-negative discrete random variable Y, the pgf $G_Y\left(z\right)$ is defined as

$$G_Y\left(z\right)=E\left(z^Y\right)=\sum _{j=0}^{\infty }p(Y=j)z^j.$$

for all $z\in \mathbb{R}$ for which the sum converges. The pgf is then a power series and obeys all the rules obeyed by power series with non-negative coefficients. The probabilities $p(Y=j)$ are the coefficients of the power series, and may be recovered through series expansion or by taking derivatives of G with respect to z.

Pgfs are especially useful for computing the distributions of sums of random variables, as well as moments and factorial moments. We note that, for independent random variables $V_1\dots ,V_n$, the pgf of $Y=V_1+\cdots +V_n$ is given by

$$G_Y\left(z\right)=\prod _{i=1}^n{G}_{V_i}\left(z\right).$$

Along with the uniqueness of the pgf, it is helpful to determine the sampling distribution of interest.

3. Competing Patterns in High-Order Markov-Dependent Bernoulli Trials

The derivation of the probability generating function includes three steps. In the first step, we build the mathematical framework, the experiment design and settings of the experiment. The second step determines the paths that terminate the experiment. Applying tools from probability theory and Markov chain, the third step derives the pgf function of the number of trials (waiting time) for each path. All steps are illustrated by a continuous example.

3.1. Step 1. Experimental Design and Settings

Consider the r-th order Markov-dependent Bernoulli trials $\left\{X_i\right\},i=1,2,\dots$ on the state space (with $2^r$ combinations)

$${\mathcal{S}}^r=\{\mathit{x}=(x_1\dots x_r):x_i\in \{0,1\},i=1,\dots ,r\}.$$

The r-th order transition probabilities are given by:

$$p(X_t=y\mid X_{t-r}=x_1,\dots ,X_{t-1}=x_r)=p_{{\mathit{x},\mathit{x}}^{\prime }},\mathit{x}=(x_1,x_2,\dots ,x_r)\in {\mathcal{S}}^r,x_i,y\in \mathcal{S}=\{0,1\}$$

(1)

independent of t.

Clearly, ${\sum }_{y\in S}{p}_{\mathit{x},\mathit{y}}=1,$ for all $\mathit{x}\in S^r.$ In a matrix form, let $\mathbb{A}$ be the $\left({\mathcal{S}}^r\times {\mathcal{S}}^r\right)$ square matrix with elements ${\left[\mathbb{A}\right]}_{\mathit{x},{\mathit{x}}^{\prime }}$ given by

$${\left[\mathbb{A}\right]}_{\mathit{x},{\mathit{x}}^{\prime }}=\left\{\begin{array}{cc}{p}_{\mathit{x},\mathit{y}}& \mathit{x}=(x_1,x_2,\dots ,x_r),{\mathit{x}}^{\prime }=(x_2,\dots ,x_r,y)\\ 0& otherwise\end{array}\right\}.$$

(2)

Associated with the sequence are the initial probabilities

$$\mathit{\pi }=\pi (x_{-r+1},\dots ,x_0)=p(X_{-r+1}=x_{-r+1},\dots ,X_0=x_0).$$

(3)

(Note that $\mathbb{A}\mathit{e}={\mathit{e}}^T,$ and the vector $\mathit{\pi }=\{{\pi }_{\mathit{x}},\mathit{x}\in S^r\}$ satisfies the system of equations $\mathit{\pi }\mathbb{A}=\mathit{\pi },\mathit{\pi }\mathit{e}=1).$ Let ${\mathsf{\Lambda }}^{\left(i\right)}=\{{\mathsf{\Lambda }}_1^{\left(i\right)},\dots ,{\mathsf{\Lambda }}_{k_i}^{\left(i\right)}\},i=1,\dots ,c$ be a compound pattern that includes $k_i$ simple patterns, each of which has size $l_{i,k_i}=\left|{\mathsf{\Lambda }}_{k_i}^{\left(i\right)}\right|.$ We assume that $l_{i,k_i}\ge r.$ Let $n_i$ denote the number of non-overlapping occurrences of ${\mathsf{\Lambda }}^{\left(i\right)}$ needed for the termination of the experiment, and let $l_i={\displaystyle \underset{j=1,\dots ,k_i}{max}}\left\{l_{i,j}\right\},i=1,\dots ,c.$ Let $\mathsf{\Lambda }={\left\{{\mathsf{\Lambda }}^{\left(i\right)}\right\}}_{i=1}^c$ be the set of competing patterns, and let $W_{\mathsf{\Lambda }}$ denote the waiting time random variable (number of trials) until the experiment terminates given the steady-state environment. We assume non-overlapping counting.

Example 1.

Our base case considers second-order Markov-dependent Bernoulli trials, i.e., $r=2$ and $x_i\in \{0,1\}.$ The transition probabilities $p_{\mathit{x},\mathit{y}},\mathit{x}=(x_1,x_2),x_i,y\in \{0,1\}$ are:

$$\begin{array}{cc}{p}_{00,0}& =p_1,p_{00,1}=p_2,p_{10,0}=p_3,p_{10,1}=p_4,\hfill \\ p_{01,0}& =p_5,p_{01,1}=p_6,p_{11,0}=p_7,p_{11,1}=p_8.\hfill \end{array}$$

(4)

In a matrix form

$$\mathbb{A}=\begin{array}{cc}\begin{array}{c}00\\ 01\\ 10\\ 11\end{array}\hfill & \left(\begin{array}{cccc}{p}_1& p_2& 0& 0\\ 0& 0& p_5& p_6\\ p_3& p_4& 0& 0\\ 0& 0& p_7& p_8\end{array}\right)\end{array}.$$

(5)

(Note that $p_i+p_{i+1}=1,i=1,3,5,7$). Here, the steady-state probability vector $\mathit{\pi }=({\pi }_{00},{\pi }_{01},{\pi }_{10},{\pi }_{11})$ satisfying $\mathit{\pi }\mathbb{A}=\mathit{\pi },\mathit{\pi }\mathit{e}=1$ is given by:

$$\begin{array}{cc}{\pi }_{00}& ={\displaystyle \frac{p_3·p_7}{p_1·p_5-2p_1·p_7+p_3·p_7-p_1-p_5+2p_7+1}}\hfill \\ & \hfill \\ {\pi }_{01}& ={\displaystyle \frac{-(p_1-1)·p_7}{p_1·p_5-2p_1·p_7+p_3·p_7-p_1-p_5+2p_7+1}}\hfill \\ & \hfill \\ {\pi }_{10}& ={\displaystyle \frac{-(p_1-1)·p_7}{p_1·p_5-2p_1·p_7+p_3·p_7-p_1-p_5+2p_7+1}}\hfill \\ & \hfill \\ {\pi }_{11}& ={\displaystyle \frac{p_1·p_5-p_1-p_5+1}{p_1·p_5-2p_1·p_7+p_3·p_7-p_1-p_5+2p_7+1}}.\hfill \end{array}$$

(6)

We assume two competing patterns ($c=2$): ${\mathsf{\Lambda }}^{\left(1\right)}=\left\{00\right\}$ with $n_1=3$ ($l_1=2$), and ${\mathsf{\Lambda }}^{\left(2\right)}=\left\{111\right\},$ with $n_2=2$ ($l_2=3$). That is, the experiment terminates if either three occurrences of two consecutive $0^s$ (failures) or two occurrences of three consecutive 1s (successes) are observed.

3.2. Step 2. Stopping Paths

We start by applying a higher hierarchical point of view, focusing on paths of patterns rather than on individual trials. Clearly, several paths can terminate the experiment. Let $V_i,i=1,\dots ,c$ be the set of paths that terminate the experiment due to ${\mathsf{\Lambda }}^{\left(i\right)},i=1,\dots ,c.$ Concretely, the set $V_i$ includes all paths that have the following structure: the pattern ${\mathsf{\Lambda }}^{\left(i\right)}$ appears $n_i$ times in total, in which its last $n_i$-th occurrence (that terminates the experiment) is the last component of the path; other occurrences of ${\mathsf{\Lambda }}^{\left(j\right)},j\ne i$ appear no more than $n_j-1$ times each. Assume that there are $C_i$ such paths (i.e., $\left|V_i\right|=C_i$). Denote these paths by $V_{i,1,}{V}_{i,2},\dots ,V_{i,C_i},$ so that $V_i={\left\{V_{i,j}\right\}}_{j=1,\dots ,C_i}.$ In the following, each path will be referred to as a “stopping vector”. Let $V={\left\{V_i\right\}}_{i=1,\dots ,c}$ be the set of all stopping vectors that terminate the experiment.

Corollary 1.

It is easy to verify the following:

(i) 

The number of competing patterns in each path $V_{i,j}$ satisfies:

$$\underset{i=1\dots c}{min}\left(n_i\right)\le \left|V_{i,j}\right|\le \sum _{i=1}^c(n_i-1)+1,j=1,\dots ,C_i,,i=1,\dots ,c$$

(7)

(ii) 

The number of paths in the set $V_i,$ is given by (combinatorial considerations):

$$C_i=\sum _{k_1=0}^{n_1-1}\cdots \sum _{k_c=0}^{n_c-1}{\displaystyle \frac{\left(k_1+\cdots +k_c+n_i-1\right)!}{k_1!\dots k_c!(n_i-1)!}},k_1<k_2,\cdots <k_c,k_j\ne i.$$

(8)

(Note that the summing is over all patterns excluding ${\mathsf{\Lambda }}^{\left(i\right)}$).

(iii) 

Since ${\mathsf{\Lambda }}^{\left(i\right)},i=1\dots c,$ including distinct patterns, we have:

$$V=\bigcup _{i=1}^c{V}_i,\left|V\right|=CwhereC=\sum _{i=1}^c{C}_i.$$

(9)

Example 2.

We have ${\mathsf{\Lambda }}^{\left(1\right)}=00$ with $n_1=3$, and ${\mathsf{\Lambda }}^{\left(2\right)}=111,$ with $n_2=2;$ thus, the experiment terminates if either three occurrences of two consecutive $0^{\prime }s$ or two occurrences of three consecutive 1s appear. Let $V_1$ and $V_2$ be the sets of paths that terminate the experiment due to ${\mathsf{\Lambda }}^{\left(1\right)}$ and ${\mathsf{\Lambda }}^{\left(2\right)},$ respectively. Figure 1 illustrates the sequence (path) of possible patterns that terminate the experiment (denoted by $V_{i,j}$; see the red labels). We observe that $V_1$ contains four possible paths ($V_{11},\dots ,V_{14}$) and $V_2$ contains six possible paths ($V_{21},\dots ,V_{26}$). In summary, there are a total of 10 possible paths (stopping vectors) that end the experiment.

Figure 1. The possible paths of ${\mathsf{\Lambda }}^{\left(1\right)}$ and ${\mathsf{\Lambda }}^{\left(2\right)}$ that terminate the experiment.

Applying Corollary 1(i), each such stopping vector includes at least two patterns (due to $n_2$) and no more than four patterns ($(n_1-1)+(n_2-1)+1=4$). Applying Corollary 1(ii), the sets $V_1$ and $V_2$ include $C_1$ and $C_2$ stopping vectors, respectively. By Corollary 1(ii), $C_1$ and $C_2$ are given by:

$$\begin{array}{cc}{C}_1& ={\displaystyle \sum _{k_1=0}^{n_2-1}}{\displaystyle \frac{\left(k_1+n_1-1\right)!}{k_1!(n_1-1)!}}={\displaystyle \frac{(3-1)!}{0!(3-1)!}}+{\displaystyle \frac{(1+3-1)!}{1!(3-1)!}}=4,\hfill \\ C_2& ={\displaystyle \sum _{k_1=0}^{n_1-1}}{\displaystyle \frac{\left(k_1+n_2-1\right)!}{k_1!(n_2-1)!}}={\displaystyle \frac{(2-1)!}{0!(2-1)!}}+{\displaystyle \frac{(1+2-1)!}{1!(2-1)!}}+{\displaystyle \frac{(2+2-1)!}{2!(2-1)!}}=6,\hfill \end{array}$$

$V=V_1\cup V_2,$ and $\left|V\right|=C_1+C_2=10$ vectors (see also Figure 1),

$$\begin{array}{cc}{V}_1=\left\{\begin{array}{c}{V}_{1,1}=[{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(1\right)}],\\ V_{1,2}=[{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(1\right)}],\\ V_{1,3}=[{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(1\right)}],\\ V_{1,4}=[{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(1\right)}],\end{array}\right\}& V_2=\left\{\begin{array}{c}{V}_{2,1}=[{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(2\right)}],\\ V_{2,2}=[{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(2\right)}],\\ V_{2,3}=[{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(2\right)}],\\ V_{2,4}=[{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(2\right)}],\\ V_{2,5}=[{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(2\right)}],\\ V_{2,6}=[{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(2\right)}].\end{array}\right\}\end{array}$$

Remark 1.

Note that Figure 1 presents only the paths of competing patterns; intermediate trials that do not lead to a pattern are not presented.

We then derive the probability-generating function of the waiting time until the experiment terminates.

3.3. Step 3. The Waiting Time Distribution

Let $W_{\mathsf{\Lambda }}$ denote the waiting time until the experiment terminates with $G\left(z\right)$ its pgf. Applying the law of total expectation and noting that $\left\{V_{i,j}\right\},i=1,\dots ,c,j=1,\dots ,C_i,$ are disjoint vectors (each vector refers to a different path) leads to:

$$G\left(z\right)=E\left(s^{W_{\mathsf{\Lambda }}}\right)=\sum _{i,j}E(s^{W_{\mathsf{\Lambda }}}·1_{\left\{V_{i,j}\right\}}).$$

(10)

($1_{\left\{A\right\}}$ is the indicator function). Each stopping vector $V_{i,j}$ is composed of a numbered sequence of consecutive patterns $V_{i,j}=[V_{i,j}\left(1\right),V_{i,j}\left(2\right),\dots ]$, e.g., $V_{2,4}\left(1\right)={\mathsf{\Lambda }}^{\left(2\right)},V_{2,4}\left(2\right)={\mathsf{\Lambda }}^{\left(1\right)}.$ Let $W_{i,j}\left(k\right),k=2,3,\dots ,$ be the number of trials from pattern $V_{ij}(k-1)$ until the occurrence of the next pattern $V_{i,j}\left(k\right).$ Accordingly, for $k=1,$ we let $W_{i,j}\left(1\right)$ be the number of trials until the first pattern $V_{i,j}\left(1\right)$ occurs, given initial state $\mathit{x}\in {\mathcal{S}}^r$. Let $W_{i,j}$ be the waiting time due to path $V_{i,j}.$ It is easy to verify that:

$$W_{i,j}=\sum _{k=1}^{\left|V_{i,j}\right|}{W}_{i,j}\left(k\right),E\left(s^{W_{i,j}}\right)=E\left(s^{{\sum }_{k=1}^{\left|V_{i,j}\right|}{W}_{i,j}\left(k\right)}\right).$$

(11)

Substituting $W_{\mathsf{\Lambda }}={\sum }_{i,j}{W}_{i,j}·1_{\left\{V_{i,j}\right\}}$ and (11) into (10) yields:

$$E\left(s^{W_{\mathsf{\Lambda }}}\right)=\sum _{i,j}E(s^{W_{i,j}}·1_{\left\{V_{i,j}\right\}})=\sum _{i,j}E\left(s^{{\sum }_{k=1}^{\left|V_{i,j}\right|}{W}_{i,j}\left(k\right)}·1_{\left\{V_{i,j}\right\}}\right)=\sum _{i,j}E\left(s^{W_{i,j}\left(1\right)}·s^{W_{i,j}\left(2\right)}·\dots 1_{\left\{V_{i,j}\right\}}\right).$$

(12)

Assume that the pattern $V_{ij}(k$ − $1)$ occurs (for $k=1,$ we assume an initial state $\mathit{x}\in {\mathcal{S}}^r).$ Let $G_{V_{i,j}(k-1),V_{i,j}\left(k\right)}$ be the pgf of $W_{i,j}\left(k\right),$ i.e., the pgf of the number of trails from $V_{ij}(k$ − $1)$ until $V_{ij}\left(k\right)$,

$$G_{V_{i,j}(k-1),V_{i,j}\left(k\right)}=\left\{\begin{array}{cc}E(s^{W_{i,j}\left(k\right)}\mid V_{i,j}(k-1))\hfill & k=2,3,\dots \hfill \\ E(s^{W_{i,j}\left(1\right)}\mid \mathit{x}\in {\mathcal{S}}^r)\hfill & k=1.\hfill \end{array}\right\}$$

(13)

Note that, due to the Markovian property, the non-overlapping feature, and assuming $V_{i,j}(k-1),$ the waiting time $W_{i,j}\left(k\right)$ is independent of $V_{i,j}\left(1\right),\dots ,V_{i,j}(k-2).$ Therefore, we have:

$$G\left(z\right)=\sum _{i,j}{\displaystyle \prod _{k=1}^{\left|V_{i,j}\right|}}{G}_{V_{i,j}(k-1),V_{i,j}\left(k\right)}\left(z\right)·$$

(14)

We further note that the function $G_{V_{i,j}(k-1),V_{i,j}\left(k\right)}$ is homogeneous, depends only on the patterns $V_{i,j}(k-1),V_{i,j}\left(k\right)$ (and is independent of k). Thus, for simplicity, and without losing generality, we will use $W_{i,j},i,j=1,\dots ,c$ to denote the waiting time between two consecutive patterns ${\mathsf{\Lambda }}^{\left(i\right)}$ and ${\mathsf{\Lambda }}^{\left(j\right)},$ given that the pattern ${\mathsf{\Lambda }}^{\left(i\right)}$ occurs ($c^2$ permutations) and, respectively, $G_{i,j}\left(z\right)$ to be the pgf of $W_{i,j}.$ Accordingly, we let $G_i\left(z\right)$ be the pgf of the number of trials until the first appearance of ${\mathsf{\Lambda }}^{\left(i\right)},$ given the initial state $\mathit{x}\in {\mathcal{S}}^r.$

Example 3.

Two competing patterns yield four pgfs that differ in their starting and ending patterns, namely $G_{1,1}\left(z\right),G_{1,2}\left(z\right),G_{2,1}\left(z\right)$, and $G_{2,2}\left(z\right).$ In addition, we have two functions, $G_1\left(z\right)$ and $G_2\left(z\right),$ corresponding to the first occurrence of ${\mathsf{\Lambda }}^{\left(1\right)}$ and ${\mathsf{\Lambda }}^{\left(2\right)},$ respectively. The total pgf is the sum of the ten distinct stopping vectors in $V,$ each of which consists of the multiplication of the corresponding pgf along the path $V_{i,j}.$ Specifically, we have:

$$\begin{array}{c}G(z)=\underset{V_{11}}{\underbrace{G_1\left(z\right){\left[G_{1,1}\left(z\right)\right]}^2}}+\underset{V_{12}}{\underbrace{G_1\left(z\right)G_{1,1}\left(z\right)G_{1,2}\left(z\right)G_{2,1}\left(z\right)}}+\underset{V_{13}}{\underbrace{G_1\left(z\right)G_{1,2}\left(z\right)G_{2,1}\left(z\right)G_{1,1}\left(z\right)}}\hfill \\ +\underset{V_{14}}{\underbrace{G_2\left(z\right)G_{2,1}\left(z\right){\left[G_{1,1}\left(z\right)\right]}^2}}+\underset{V_{21}}{\underbrace{G_2\left(z\right)G_{2,2}\left(z\right)}}+\underset{V_{22}}{\underbrace{G_2\left(z\right)G_{2,1}\left(z\right)G_{1,2}\left(z\right)}}+\underset{V_{23}}{\underbrace{G_1\left(z\right)G_{1,2}\left(z\right)G_{2,2}\left(z\right)}}\hfill \\ +\underset{V_{24}}{\underbrace{G_2\left(z\right)G_{2,1}\left(z\right)G_{1,1}\left(z\right)G_{1,2}\left(z\right)}}+\underset{V_{25}}{\underbrace{G_1\left(z\right){\left[G_{1,2}\left(z\right)\right]}^2{G}_{2,1}\left(z\right)}}+\underset{V_{26}}{\underbrace{G_1\left(z\right)G_{1,1}\left(z\right)G_{1,2}\left(z\right)G_{2,2}\left(z\right)}}\hfill \end{array}$$

(15)

Our next step is to derive the probability-generating functions $G_{i,j}\left(z\right)$ and $G_i\left(z\right)$ for $i,j=1,\dots ,c.$ Recall that $l_i$ is the longest length of pattern in ${\mathsf{\Lambda }}^{\left(i\right)},$ and let $b={max}_{i=1,\dots ,c}\left\{l_i\right\}.$ Our aim is to apply the law of total expectation as a function of the number of trials between two consecutive patterns (or until the first pattern), distinguishing whether the number is less or equal to b or not. Thus, we use the decomposition:

$$\begin{array}{cc}\hfill G_{i,j}\left(z\right)& =G_{i,j}\left(z\right)1_{\left\{z\le b\right\}}+G_{i,j}\left(z\right)1_{\left\{z>b\right\}},\hfill \\ \hfill G_i\left(z\right)& =G_i\left(z\right)1_{\left\{z\le b\right\}}+G_i\left(z\right)1_{\left\{z>b\right\}}.\hfill \end{array}$$

(16)

In the left-hand side of (16), where the number of trials is no more than b, the derivation is relatively simpler and consists of a finite number of paths. Conversely, the derivation of the right-hand side of (16), where more than b trials are possible, is more challenging. We start with the relatively simpler derivation, $G_{i,j}\left(z\right)1_{\left\{z\le b\right\}}$ and $G_i\left(z\right)1_{\left\{z\le b\right\}}.$

3.3.1. The Functions $G_i\left(z\right)1_{\left\{z\le b\right\}},$ $G_{i,j}\left(z\right)1_{\left\{z\le b\right\}}$

We first highlight the main difference between $G_i\left(z\right)1_{\left\{z\le b\right\}}$ and $G_{i,j}\left(z\right)1_{\left\{z\le b\right\}}.$ The function $G_i\left(z\right)1_{\left\{z\le b\right\}}$ refers to the first b trials; here, we use the steady-state probability vector $\mathit{\pi }\left(\mathit{x}\right),\mathit{x}\in {\mathcal{S}}^r$, multiplied by the remaining $(b-r)$ probabilities. In contrast, the function $G_{i,j}\left(z\right)1_{\left\{z\le b\right\}}$ assumes that the ${\mathsf{\Lambda }}^{\left(i\right)}$ occurs, and continues with the multiplication of at most b transition probabilities leading to ${\mathsf{\Lambda }}^{\left(j\right)}$.

To derive $G_i\left(z\right)1_{\left\{z\le b\right\}},$ recall that $l_i\ge r.$ Let $p({\mathsf{\Lambda }}^{\left(i\right)},b)$ be the probability of hitting ${\mathsf{\Lambda }}^{\left(i\right)}$ by no more than b trials (and no other patterns are hit). Let $L_i\left(u\right)=\{\mathit{x}=x_1\dots x_u:$ $x_{u-l_i+1}\dots x_u={\mathsf{\Lambda }}^{\left(i\right)}\}$ be the set of sequences of u trials in which ${\mathsf{\Lambda }}^{\left(i\right)}$ appears in the last $l_i$ trials $(l_i\le u\le b)$ and no other pattern is hit; recall that $r\le l_i\le u\le b.$ Denote by $p\left({\mathit{x}}_i\left(u\right)\right)$ the ergodic probability of $L_i\left(u\right)$. Clearly, when $u=l_i=r,$ $L_i\left(u\right)$ includes only the one pattern ${\mathsf{\Lambda }}^{\left(i\right)}$ with probability $p\left(L_i\left(u\right)\right)={\pi }_{{\mathsf{\Lambda }}^{\left(i\right)}}.$ When $l_i<u\le b$, there may be few paths in $L_i\left(u\right);$ each denoted by ${\mathit{x}}_k\left(u\right),$ starts with $\tilde{\mathit{x}}\in {\mathcal{S}}^r$ (with probability $\pi \left(\tilde{\mathit{x}}\right)$), multiplying by a path of transition probabilities that leads to the ending pattern ${\mathsf{\Lambda }}^{\left(i\right)}.$ Thus, the ergodic probability distribution $p({\mathsf{\Lambda }}^{\left(i\right)},b)$ has the form of:

$$p({\mathsf{\Lambda }}^{\left(i\right)},b)={\pi }_{{\mathsf{\Lambda }}^{\left(i\right)}}{1}_{\{u=r\}}+{\displaystyle \sum _{u=l_i+1}^b}{\displaystyle \sum _k}p\left({\mathit{x}}_k\left(u\right)\right)1_{\{{\mathit{x}}_k\left(u\right)\in L_i\left(u\right)\}}.$$

(17)

The function $G_i\left(z\right)1_{\left\{z\le b\right\}}$ is then,

$$G_i\left(z\right)1_{\left\{z\le b\right\}}=z^u{\pi }_{{\mathsf{\Lambda }}^{\left(i\right)}}{1}_{\{u=r\}}+{\displaystyle \sum _{u=l_i+1}^b}{\displaystyle \sum _k}{z}^{u}p\left({\mathit{x}}_k\left(u\right)\right)1_{\{{\mathit{x}}_k\left(u\right)\in L_i\left(u\right)\}}.$$

(18)

To derive $G_{i,j}\left(z\right)1_{\left\{z\le b\right\}},$ we assume that, at time $t=0$, the state includes the last r trials of ${\mathsf{\Lambda }}^{\left(i\right)},$ and consider the set $L_i\left(u\right),$ $u\le b.$ Each path in the set $L_i\left(u\right)$ adds a component to $G_{i,j}\left(z\right)1_{\left\{z\le b\right\}},$ with the product of the corresponding transition probabilities multiplied by $z^u$. We next demonstrate (18) using our example.

Example 4.

Here, $r=2$ and $b=3$. To derive $G_1\left(z\right)1_{\left\{z\le 3\right\}},$ we consider the pattern ${\mathsf{\Lambda }}^{\left(1\right)}=00$ with $l_1=2=r.$ Here, $u=2$ or $u=3.$ When $u=2,$ we have $L_1\left(2\right)=\left\{00\right\}$ and ${\pi }_{{\mathsf{\Lambda }}^{\left(1\right)}}={\pi }_{00};$ when $u=3,$ we have $L_1\left(3\right)=\left\{100\right\}$ and $p\left(L_1\left(3\right)\right)={\pi }_{10}·p_3.$ Next, we derive $G_2\left(z\right)1_{\left\{z\le 3\right\}}.$ Here, ${\mathsf{\Lambda }}^{\left(2\right)}=111,$ $l_2=b=3>r;$ thus, $u=3$ with the only path $L_2\left(3\right)=\left\{111\right\}$ and $p\left(L_2\left(3\right)\right)={\pi }_{11}·p_8.$ Summarizing, we obtain:

$$G_1\left(z\right)1_{\left\{z\le 3\right\}}=z^2{\pi }_{00}{+z}^3\left({\pi }_{10}·p_3\right),G_2\left(z\right)1_{\left\{z\le 3\right\}}=z^3\left({\pi }_{11}·p_8\right).$$

(19)

Next, we derive $G_{i,j}\left(z\right)1_{\left\{z\le 3\right\}}.$ We assume that, at time $t=0$, the state is the last r trial of ${\mathsf{\Lambda }}^{\left(i\right)}$ (i.e., for ${\mathsf{\Lambda }}^{\left(1\right)}$ we assume $X_0=00,$ and for ${\mathsf{\Lambda }}^{\left(2\right)}$ we assume $X_0=11)$. The function $G_{i,j}\left(z\right)1_{\left\{z\le 3\right\}}$ is constructed by a path of at most three multiplications of transition probabilities (multiplied by the corresponding power of z) that lead to ${\mathsf{\Lambda }}^{\left(j\right)}$. Here, we obtain:

$$\begin{array}{cc}{G}_{1,1}\left(z\right)1_{\left\{z\le 3\right\}}& =z^2{\left(p_1\right)}^2{+z}^3\left(p_2·p_5·p_3\right),\hfill \\ G_{1,2}\left(z\right)1_{\left\{z\le 3\right\}}& =z^3\left(p_2·p_6·p_8\right),\hfill \\ G_{2,1}\left(z\right)1_{\left\{z\le 3\right\}}& =z^2\left(p_7·p_3\right){+z}^3\left(p_8·p_7·p_3\right),\hfill \\ G_{2,2}\left(z\right)1_{\left\{z\le 3\right\}}& =z^3{\left(p_8\right)}^3,\hfill \end{array}$$

(20)

i.e., when assuming ${\mathsf{\Lambda }}^{\left(1\right)}$ (with $X_0=00),$ the paths 00 and 100 (w.p. ${\left(p_1\right)}^2$ and $p_2·p_5·p_3,$ respectively) lead to ${\mathsf{\Lambda }}^{\left(1\right)},$ and the path 111 (w.p. $p_2·p_6·p_8$) leads to ${\mathsf{\Lambda }}^{\left(2\right)}.$ Similarly, when assuming ${\mathsf{\Lambda }}^{\left(2\right)}$ (with $X_0=11$), the paths 00 and 100 (w.p. $p_7·p_3$ and $p_8·p_7·p_3,$ respectively) lead to ${\mathsf{\Lambda }}^{\left(1\right)},$ and the path 111 (w.p. ${\left(p_8\right)}^3$) leads to ${\mathsf{\Lambda }}^{\left(2\right)}.$

3.3.2. The Functions $G_i\left(z\right)1_{\left\{z>b\right\}},$ $G_{i,j}\left(z\right)1_{\left\{z>b\right\}}$

Let us consider the first b trials $\mathit{x}=x_1\dots x_b=\mathit{x}\in {\mathcal{S}}^b.$ We group all $\mathit{x}\in {\mathcal{S}}^b$ that contain the pattern ${\mathsf{\Lambda }}^{\left(i\right)}$ into the set ${\xi }_b^i.$ Define the state ${\alpha }^i$ to be the absorbing state due to pattern ${\mathsf{\Lambda }}^{\left(i\right)},$ i.e., ${\alpha }^i$ groups all states in ${\xi }_b^i.$ Denote the absorbing vector by $\mathit{\alpha }=({\alpha }^1,{\alpha }^2,\dots ,{\alpha }^c).$ The set ${\mathsf{\Omega }}_Y-\{\mathit{\alpha }\}\in {\mathcal{S}}^b$ groups all states that do not include a pattern. Note that every state in $\left\{{\mathsf{\Omega }}_Y-\{\mathit{\alpha }\},{\alpha }^1,\dots ,{\alpha }^c\right\}$ has pattern of length b. Let $\mathbb{T}$ be the transition probability matrix among the states in ${\mathsf{\Omega }}_Y-\{\mathit{\alpha }\}.$ In addition, let ${\mathit{T}}_j^0,i=1,\dots ,c$ be the absorbing probability vector at state ${\alpha }^i,$ and define the absorbing matrix ${\mathbb{T}}^0=({\mathit{T}}_1^0,{\mathit{T}}_2^0,\dots ,{\mathit{T}}_c^0).$ We construct the embedded homogeneous Markov chain ${\left\{Y_t\right\}}_{b+1}^{\infty }$ on the state space ${\mathsf{\Omega }}_Y=\left\{{\mathcal{S}}^b\right\}.$ The transition probability matrix has the block form

$$\mathbb{M}=\left(p_{\mathit{x}\mathit{y}}\right)=\begin{array}{cc}\begin{array}{c}{\mathsf{\Omega }}_Y-\{\mathit{\alpha }\}\\ \mathit{\alpha }\end{array}\hfill & \left(\begin{array}{cc}\mathbb{T}& {\mathbb{T}}^0\\ \mathbf{0}& \mathbb{I}\end{array}\right)\hfill \end{array},$$

(21)

where

$$p_{\mathit{x}\mathit{y}}=\left\{\begin{array}{cc}{p}_{\mathit{x},x_{b+1}}& if \mathit{x}\in {\mathsf{\Omega }}_Y, \mathit{y}=\left(\mathit{x}{x}_{b+1}\right)\in {\mathsf{\Omega }}_Y,\\ 1& \mathrm{if} \mathit{x}=\mathit{y}\in \left\{{\mathit{\alpha }}^i\right\},\\ 0& \mathrm{otherwise}.\end{array}\right\}$$

(22)

Note that $\left(\mathbb{T}+{\sum }_{j=1}^c{\mathit{T}}_j^0\right)\mathit{e}={\mathit{e}}^T$. The initial probability of $\left\{Y_b\right\}$ is given by:

$${\mathit{\pi }}_{Y_b}=(p(Y_b=\mathit{x}):\mathit{x}\in {\mathsf{\Omega }}_Y)=\left\{\begin{array}{cc}{\pi }_{Y_b}\left(\mathit{x}\right)& \mathit{x}=x_1\dots x_b\in {\mathsf{\Omega }}_Y-\{\mathit{\alpha }\},\\ \sum _{\mathit{x}\in {\xi }_b^i}{\pi }_{Y_b}\left(\mathit{x}\right)& \mathit{x}\in {\alpha }^i,\\ 0& \mathrm{otherwise}.\end{array}\right\}$$

(23)

Since $b\ge r,$ the initial probability ${\pi }_{Y_b}\left(\mathit{x}\right)$ for $\mathit{x}=x_1\dots x_b$ starts with the corresponding steady-state probability ${\pi }_{x_1\dots x_r}$ multiplying by the transition probabilities along the path $x_{r+1},\dots ,x_b$. To complete the derivation, we need to define two row probability vectors of order $\left(1\times \left|{\mathsf{\Omega }}_Y-\{\mathit{\alpha }\}\right|\right)$, $\mathit{IP}$ and ${\mathit{IP}}_i,i=1,\dots ,c.$ Both vectors represent the probability of entering into the states in ${\mathsf{\Omega }}_Y-\{\mathit{\alpha }\}$ after b trials with no competing patterns; the difference arises from their initial conditions. The vector $\mathit{IP}=(\mathit{IP}\left(\mathit{x}\right):\mathit{x}\in {\mathsf{\Omega }}_Y-\{\mathit{\alpha }\left\}\right)$ assumes an initial state in ${\mathcal{S}}^r$ (not including a pattern), and calculates the probability of entering ${\mathsf{\Omega }}_Y-\{\mathit{\alpha }\}$ at time $t=b-r$; here, we use the steady-state probability vector $\mathit{\pi }$ of order r, and a multiplication of $(b-r)$ successive transition probabilities. The vector ${\mathit{IP}}_i=(I{P}_i\left(\mathit{x}\right):\mathit{x}\in {\mathsf{\Omega }}_Y-\{\mathit{\alpha }\left\}\right)$ assumes an initial pattern ${\mathsf{\Lambda }}^{\left(i\right)}$ and derives the probability of entering ${\mathsf{\Omega }}_Y-\{\mathit{\alpha }\}$ afterward; here, we use a multiplication of b successive transition probabilities.

Proposition 1.

The functions $G_i\left(z\right)1_{\left\{z>b\right\}}$ and $G_{i,j}\left(z\right)1_{\left\{z>b\right\}}$ have the general form of:

$$\begin{array}{cc}{G}_j\left(z\right)1_{\left\{z>b\right\}}& =\left(z^b\mathit{IP}\right)·{(\mathbb{I}-z\mathbb{T})}^{-1}·\left(z{\mathit{T}}_j^0\right),j=1,\dots ,c,\hfill \\ G_{i,j}\left(z\right)1_{\left\{z>b\right\}}& =\left(z^b{\mathit{IP}}_i\right)·{(\mathbb{I}-z\mathbb{T})}^{-1}·\left(z{\mathit{T}}_j^0\right),i,j=1,\dots ,c.\hfill \end{array}$$

(24)

Proof. 

The derivation of $G_i\left(z\right)1_{\left\{z>b\right\}}$ and $G_{i,j}\left(z\right)1_{\left\{z>b\right\}}$ is composed of three parts. In the first part, the experiment enters a state within the set ${\mathsf{\Omega }}_Y-\{\mathit{\alpha }\}$ after b trials where no hitting occurs. Thus, we multiply $IP$ and $I{P}_i$ by $z^b,$ respectively. The second part is the pgf of the number of trials to stay in the set ${\mathsf{\Omega }}_Y-\{\mathit{\alpha }\}$ until absorption. Here, following Fu and Lou [41], we have the term ${(\mathbb{I}-z\mathbb{T})}^{-1}.$ From that point, the third part is the probability of hitting ${\mathsf{\Lambda }}^{\left(j\right)}$ in the next trial, with pgf $z{\mathit{T}}_j^0.$ □

Remark 2.

An easy way to derive the vectors $\mathit{IP}$ and ${\mathit{IP}}_i,i=1,\dots ,c$ is as follows. Construct the transition probability matrices ${\mathbb{H}}_{r,b}=(p_{{\mathit{x},\mathit{x}}^{\prime }}:\mathit{x}\in {\mathcal{S}}^r,{\mathit{x}}^{\prime }\in {\mathcal{S}}^b)$ and ${\mathbb{H}}_{b,b}=(p_{{\mathit{x},\mathit{x}}^{\prime }}:\mathit{x}\in {\mathcal{S}}^b,{\mathit{x}}^{\prime }\in {\mathcal{S}}^b).$ The matrix ${\mathbb{H}}_{r,b}·{\left({\mathbb{H}}_{b,b}\right)}^{b-1}$ present the probability of each state in ${\mathsf{\Omega }}_Y$, given an initial r-state trial. The vectors $I{P}_i,i=1,\dots ,c$ can be extracted from ${\mathbb{H}}_{r,b}·{\left({\mathbb{H}}_{b,b}\right)}^{b-1}$ by taking the corresponding row to the r ending block of ${\mathsf{\Lambda }}^{\left(i\right)},$ and the corresponding columns ${\mathsf{\Omega }}_Y-\{\mathit{\alpha }\}$ with the appropriate eliminations.

Example 5.

Since $b=3,$ we have $\mathit{x}\in S^3$ and ${\mathsf{\Omega }}_Y=\{000,001,010,011,100,101,110,111\}.$ Here, ${\alpha }^1=\{000,001,100\},$ ${\alpha }^2=\left\{111\right\},$ $\alpha ={\alpha }^1\cup {\alpha }^2=\{000,001,100,111\},$ and ${\mathsf{\Omega }}_Y-\left\{\mathit{\alpha }\right\}=\{010,011,101,110\}.$ We construct the embedded homogeneous Markov chain ${\left\{Y_t\right\}}_4^{\infty }$ on the state space

$${\mathsf{\Omega }}_Y=\{010,011,101,110,{\alpha }^1,{\alpha }^2\}.$$

To clarify, Figure 2a–d illustrate the states and probabilities of $Y_3;$ states marked by light red, including a pattern and, thus, are marked by ${\alpha }^1$ or ${\alpha }^2.$

Figure 2. The states and probabilities of $Y_3(t=1)$, given (a) $\mathbf{x}(t=0)=\left(00\right)$, (b) $\mathbf{x}(t=0)=\left(01\right)$, (c) $\mathbf{x}(t=0)=\left(10\right)$, (d) $\mathbf{x}(t=0)=\left(11\right)$.

Summarizing, the initial distribution probability of $Y_3$ is given by (note that ${\mathit{\pi }}_{Y_3}{\mathit{e}}^T=1$):

$${\mathit{\pi }}_{Y_3}=(\underset{\mathit{x}\in {\mathsf{\Omega }}_Y-\{\mathit{\alpha }\}}{\underbrace{{\pi }_{01}{p}_5,{\pi }_{01}{p}_6,{\pi }_{10}{p}_4,{\pi }_{11}{p}_7}},\underset{\mathit{x}\in {\mathit{\alpha }}^1}{\underbrace{{\pi }_{00}{p}_1+{\pi }_{00}{p}_2+{\pi }_{10}{p}_3}},\underset{\mathit{x}\in {\mathit{\alpha }}^2}{\underbrace{{\pi }_{11}{p}_8}}).$$

(25)

The transition probability matrix has the form $\left(\begin{array}{cc}\mathbb{T}& {\mathbb{T}}^0\\ \mathbf{0}& \mathbb{I}\end{array}\right),$ and is given by (Table 1):

Table 1. The probability transition matrix of ${\mathsf{\Omega }}_Y$.

Px,y		ΩY
		010	011	101	110	α1	α2
ΩY	010	0	0	p4	0	p3	0
	011	0	0	0	p7	0	p8
	101	p5	p6	0	0	0	0
	110	0	0	p4	0	p3	0
	α1	0	0	0	0	1	0
	α2	0	0	0	0	0	1

To complete our derivation, we need to derive the vectors $\mathit{IP}, {\mathit{IP}}_1,$ and ${\mathit{IP}}_2.$ Note that the vector $\mathit{IP}$ is the sub-vector of ${\pi }_{Y_3},$ with the corresponding vectors in ${\mathsf{\Omega }}_Y-\left\{\mathit{\alpha }\right\};$ i.e.,

$$\mathit{IP}=(\underset{Y_3(t=1)=\left(010\right)}{\underbrace{{\pi }_{01}{p}_5}},\underset{Y_3(t=1)=\left(011\right)}{\underbrace{{\pi }_{01}{p}_6},}\underset{Y_3(t=1)=\left(101\right)}{\underbrace{{\pi }_{10}{p}_4}},\underset{Y_3(t=1)=\left(110\right)}{\underbrace{{\pi }_{11}{p}_7}}).$$

(26)

(Note that, since $\left(00\right)$ is a competing pattern, the vector $\mathit{IP}$ considers only the two-state trials {01,10,11}). In order to derive ${\mathit{IP}}_1$ and ${\mathit{IP}}_2,$ we apply Remark 2. Here, the matrices ${\mathbb{H}}_{2,3}$ and ${\mathbb{H}}_{3,3}$ are given by (Table 2):

Table 2. The probability transition matrix of ${\mathbb{H}}_{2,3}$.

${\mathbb{H}}_{\mathbf{2},\mathbf{3}}$		ΩY
		000	001	010	011	100	101	110	111
2-state trials	00	p1	p2
	01			p5	p6
	10					p3	p4
	11							p7	p8

And thus, ${\mathbb{H}}_{2,3}·{\left({\mathbb{H}}_{3,3}\right)}^2$ is given by:

The probability vectors ${\mathit{IP}}_1$ and ${\mathit{IP}}_2$ to be at state $x\in {\mathsf{\Omega }}_Y-\left\{\mathit{\alpha }\right\}$ in three trials without hitting, starting from ${\mathsf{\Lambda }}^{\left(1\right)}$ and ${\mathsf{\Lambda }}^{\left(2\right)}$ respectively, are highlighted by the red boxes; see the first and last rows in Table 3. Here, we obtain

$$\begin{array}{cc}{\mathit{IP}}_1& =(p_1{p}_2{p}_5,p_1{p}_2{p}_6,p_2{p}_5{p}_4,p_2{p}_6{p}_7),\hfill \\ {\mathit{IP}}_2& =(p_7{p}_4{p}_5,p_7{p}_4{p}_6,p_8{p}_7{p}_4,p_8{p}_8{p}_7).\hfill \end{array}$$

(27)

Table 3. The probability transition matrix of ${\mathbb{H}}_{3,3}$.

${\mathbb{H}}_{\mathbf{3},\mathbf{3}}$		ΩY
		000	001	010	011	100	101	110	111
ΩY	000	p1	p2
	001			p5	p6
	010					p3	p4
	011							p7	p8
	100	p1	p2
	101			p5	p6
	110					p3	p4
	111							p7	p8

(Note that $IP$ is also obtained by taking the columns corresponding to ${\mathsf{\Omega }}_Y-\left\{\mathit{\alpha }\right\}$ in the product $\pi ·{\tilde{\mathbb{H}}}_{2,3}$). Summarizing all, we have:

$$\begin{array}{cc}\hfill G_1\left(z\right)=& z^2·{\pi }_{00}+z^3·\left({\pi }_{10}·p_3\right)+z^3·\left(\begin{array}{cccc}{\pi }_{01}{p}_5& {\pi }_{01}{p}_6& {\pi }_{10}{p}_4& {\pi }_{11}{p}_7\end{array}\right)·{(\mathbb{I}-z\mathbb{T})}^{-1}z·\left(\genfrac{}{}{0.0pt}{}{\genfrac{}{}{0.0pt}{}{p_3}{0}}{\genfrac{}{}{0.0pt}{}{0}{p_3}}\right),\hfill \\ \hfill G_2\left(z\right)=& z^3·\left({\pi }_{11}·p_8\right)+z^3·\left(\begin{array}{cccc}{\pi }_{01}{p}_5& {\pi }_{01}{p}_6& {\pi }_{10}{p}_4& {\pi }_{11}{p}_7\end{array}\right)·{(\mathbb{I}-z\mathbb{T})}^{-1}z·\left(\genfrac{}{}{0.0pt}{}{\genfrac{}{}{0.0pt}{}{0}{p_8}}{\genfrac{}{}{0.0pt}{}{0}{0}}\right),\hfill \\ \hfill G_{1,1}\left(z\right)=& z^2·{\left(p_1\right)}^2+z^3·\left(p_2·p_5·p_3\right)\hfill \\ & +z^3·\left(\begin{array}{cccc}{p}_1{p}_2{p}_5& p_1{p}_2{p}_6& p_2{p}_5{p}_4& p_2{p}_6{p}_7\end{array}\right)·{(\mathbb{I}-z\mathbb{T})}^{-1}z·\left(\genfrac{}{}{0.0pt}{}{\genfrac{}{}{0.0pt}{}{p_3}{0}}{\genfrac{}{}{0.0pt}{}{0}{p_3}}\right),\hfill \\ \hfill G_{1,2}\left(z\right)=& z^3·\left(p_2·p_6·p_8\right)+z^3·\left(\begin{array}{cccc}{p}_1{p}_2{p}_5& p_1{p}_2{p}_6& p_2{p}_5{p}_4& p_2{p}_6{p}_7\end{array}\right)·{(\mathbb{I}-z\mathbb{T})}^{-1}z·\left(\genfrac{}{}{0.0pt}{}{\genfrac{}{}{0.0pt}{}{0}{p_8}}{\genfrac{}{}{0.0pt}{}{0}{0}}\right),\hfill \\ \hfill G_{2,1}\left(z\right)=& z^2·\left(p_7·p_3\right)+z^3·\left(p_8·p_7·p_3\right)\hfill \\ & +z^3·\left(\begin{array}{cccc}{p}_7{p}_4{p}_5& p_7{p}_4{p}_6& p_8{p}_7{p}_4& p_8{p}_8{p}_7\end{array}\right)·{(\mathbb{I}-z\mathbb{T})}^{-1}z·\left(\genfrac{}{}{0.0pt}{}{\genfrac{}{}{0.0pt}{}{p_3}{0}}{\genfrac{}{}{0.0pt}{}{0}{p_3}}\right),\hfill \\ \hfill G_{2,2}\left(z\right)=& z^3·{\left(p_8\right)}^3+z^3·\left(\begin{array}{cccc}{p}_7{p}_4{p}_5& p_7{p}_4{p}_6& p_8{p}_7{p}_4& p_8{p}_8{p}_7\end{array}\right)·{(\mathbb{I}-z\mathbb{T})}^{-1}z·\left(\genfrac{}{}{0.0pt}{}{\genfrac{}{}{0.0pt}{}{0}{p_8}}{\genfrac{}{}{0.0pt}{}{0}{0}}\right).\hfill \end{array}$$

(28)

Substituting (28) in (15) completes the derivation of G(z).

4. Algorithm and Examples

To summarize, we next present an algorithm with the main key steps; a detailed pseudocode is provided in Appendix A. The algorithm is then demonstrated by two additional examples.

4.1. The Algorithm

Step 1. Inputs and initialization

• The parameter $r,$ the space state ${\mathsf{\Omega }}_r=\left\{{\mathcal{S}}^r\right\},$ the $S^r$-square transition matrix $\mathbb{A}$.
• The competing patterns and their appearances: ${({\mathsf{\Lambda }}^{\left(i\right)},n_i)}_{i=1,\dots ,c}.$
• Calculate the steady-state probability vector $\mathit{\pi }=({\pi }_{\mathit{x}}:\mathit{x}\in S^r)$ satisfying $\mathit{\pi }\mathbb{A}=\mathit{\pi },\mathit{\pi }\mathit{e}=1.$

Step 2. Embedded Markov chain

• Calculate the stopping vectors ${\left\{V_i\right\}}_{i=1,\dots ,c};$ use (8).
• Set $b={max}_i\left\{\left|l_i\right|\right\},i=1,\dots ,c.$
• Build the state space ${\mathsf{\Omega }}_Y=\left\{S^b\right\},$ define the absorbing states $\left\{{\alpha }^i\right\},i=1,\dots ,c.$
• Construct the matrix $\left(\begin{array}{cc}\mathbb{T}& {\mathbb{T}}^0\\ \mathbf{0}& \mathbb{I}\end{array}\right),$ use (21) and (22).

Step 3. Probability generating function

• Derive $G_i\left(z\right)1_{\left\{z\le b\right\}}$ and $G_{i,j}\left(z\right)1_{\left\{z\le b\right\}};$ use (18).
• Calculate the vector $\mathit{IP},$ and the matrices ${\mathbb{H}}_{r,b}$ and ${\mathbb{H}}_{b,b}.$ Derive ${\left\{{\mathit{IP}}_i\right\}}_{i=1,\dots ,c}$ (the matrix ${\mathbb{H}}_{r,b}·{\left({\mathbb{H}}_{b,b}\right)}^{b-1}$ can be helpful with the appropriate eliminations).
• Apply (G14) to obtain $G_i\left(z\right)1_{\left\{z>b\right\}}$ and $G_{i,j}\left(z\right)1_{\left\{z>b\right\}}.$
• Apply (16) and (14) to derive $G\left(z\right).$

4.2. Example 2

Our next example is inspired by ReasonLabs. ReasonLabs Ltd. is a global pioneer in cybersecurity detection and prevention powered by machine learning (https://reasonlabs.com) (accessed on 10 January 2025). The Israeli team (called the Performance team) is responsible for projects marketing via a website. Their marketing occurs in several stages, in which a customer is expected to enter the website and choose the service that suits their needs. Basic services are free, while premium services are purchased. Usually, a customer (buyer) visits the website a few times for various projects. The Performance team’s purpose is to track customer acquisition and identify buyer patterns, especially those who are willing to purchase the premium service. To achieve this, they use Bernoulli trials, where each outcome represents a customer choice. The result ‘1’ represents purchasing a premium service, and ‘0’ represents choosing a free service. Two stopping rules are proposed to detect the customer behavior:

(1) Two consecutive purchases (1s) occurring twice (not necessarily consecutively). Here, we also allow at most one free service (‘0’) between them. This customer is considered the most serious buyer to track.

(2) Two consecutive free enters (0s) occurring twice (not necessarily consecutively). This customer is considered a less serious customer who can be ignored.

The above rules (1) and (2) can be modeled as a waiting time problem of competing patterns. We next demonstrate the algorithm on Example 2.

Step 1. Consider second-order Markov-dependent Bernoulli trials, with transition probabilities given in (4)–(6). Here, we have two competing patterns, ${\mathsf{\Lambda }}^{\left(1\right)}=\{101,$ $11\}$ with $n_1=2$ (associated with Rule 1), and ${\mathsf{\Lambda }}^{\left(2\right)}=\left\{00\right\},$ with $n_2=2$ (associated with Rule 2). Thus, the experiment terminates when one of the following rules occurs:

(i)

Two occurrences of the set of patterns $\{101,11\},$ i.e., either 101 occurs twice, or 11 occurs twice, or 101 and then $11,$ or vice versa, all occurrences are not necessarily consecutive.

(ii)

Two (not necessarily consecutive) occurrences of two consecutive $0^{\prime }s$.

Example of the sequences are, e.g., 010100011, 11100101 (Rule (i)), and 0100101100, 01011000100 (Rule (ii)).

Step 2. Accordingly, we have two sets $V_1$ and $V_2$ corresponding to ${\mathsf{\Lambda }}^{\left(1\right)}$ and ${\mathsf{\Lambda }}^{\left(2\right)},$ respectively. Figure 3 demonstrates the paths of possible patterns that terminate the experiment. We observe that $V_1$ and $V_2$ include three possible paths each ($V_{11},V_{12},V_{13}$), and ($V_{21},V_{22},V_{23}$). Thus, we have six possible paths in total (stopping vectors) that terminate the experiment.

Figure 3. The possible paths of ${\mathsf{\Lambda }}_1$ or ${\mathsf{\Lambda }}_2$ that terminate the experiment.

Corollary 1(i) yields that each such stopping vector includes at least two patterns and no more than three patterns ($(n_1-1)+(n_2-1)+1=3$). By Corollary 1(ii), $C_1$ and $C_2$ are given by:

$$C_i=\sum _{k_1=0}^1{\displaystyle \frac{\left(k_1+n_i-1\right)!}{k_1!(n_i-1)!}}={\displaystyle \frac{(2-1)!}{0!(2-1)!}}+{\displaystyle \frac{(1+2-1)!}{1!(2-1)!}}=3,\mathrm{for}i=1,2,$$

(29)

and $\left|V\right|=C_1+C_2=6$ vectors. Specifically, the stopping vectors are (see Figure 3):

$$\begin{array}{cc}{V}_1=& \left\{V_{1,1}=[{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(1\right)}],V_{1,2}=[{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(1\right)}],V_{1,3}=[{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(1\right)}]\right\},\hfill \\ V_2=& \left\{V_{2,1}=[{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(2\right)}],V_{2,2}=[{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(1\right)},{\mathsf{\Lambda }}^{\left(2\right)}],V_{2,3}=[{\mathsf{\Lambda }}^{\left(2\right)},{\mathsf{\Lambda }}^{\left(2\right)}]\right\}\hfill \\ V=& V_1\cup V_2.\hfill \end{array}$$

(30)

Since we have two competing patterns, ${\mathsf{\Lambda }}^{\left(1\right)}$ and ${\mathsf{\Lambda }}^{\left(2\right)},$ there are four pgfs, $G_{1,1}\left(z\right),$ $G_{1,2}\left(z\right),$ $G_{2,1}\left(z\right),$ $G_{2,2}\left(z\right),$ in addition to $G_1\left(z\right),$ $G_2\left(z\right)$ (the pgf of the first occurrence of ${\mathsf{\Lambda }}^{\left(1\right)}$ and ${\mathsf{\Lambda }}^{\left(2\right)},$ respectively). Furthermore, the pattern ${\mathsf{\Lambda }}^{\left(1\right)}$ includes two simple patterns $\{00,11\}.$ Thus, to distinguish between them, we mark $\left\{11\right\}$ by the sign “′”, and refer to $\left\{101\right\}$ by no sign. Hence, we need to derive $G_{1,1}\left(z\right),$ $G_{1,1^{\prime }}\left(z\right),$ $G_{1^{\prime },1}\left(z\right),$ $G_{1^{\prime },1^{\prime }}\left(z\right),$ $G_{1,2}\left(z\right),$ $G_{1^{\prime },2}\left(z\right),$ $G_{2,1}\left(z\right),$ $G_{2,1^{\prime }}\left(z\right),$ $G_{2,2}\left(z\right),$ $G_1\left(z\right),$ $G_{1^{\prime }}\left(z\right),$ and $G_2\left(z\right).$ The final pgf will be (to be shorter, the parameter z is omitted):

$$\begin{array}{cc}G\left(z\right)& =\underset{V_{11}}{\underbrace{G_1\left(G_{1,1}+G_{1,1^{\prime }}\right)+G_{1^{\prime }}\left(G_{1^{\prime },1}+G_{1^{\prime },1^{\prime }}\right)}}+\underset{V_{12}}{\underbrace{\left(G_1{G}_{1,2}+G_{1^{\prime }}{G}_{1^{\prime },2}\right)\left(G_{2,1}+G_{2,1^{\prime }}\right)}}\hfill \\ & +\underset{V_{13}}{\underbrace{G_2\left[G_{2,1}\left(G_{1,1}+G_{1,1^{\prime }})+G_{2,1^{\prime }}(G_{1^{\prime },1}+G_{1^{\prime },1^{\prime }}\right)\right]}}+\underset{V_{21}}{\underbrace{\left(G_1{G}_{1,2}+G_{1^{\prime }}{G}_{1^{\prime },2}\right)G_{2,2}}}\hfill \\ & +\underset{V_{22}}{\underbrace{G_2\left(G_{2,1}{G}_{1,2}+G_{2,1^{\prime }}{G}_{1^{\prime },2}\right)}}+\underset{V_{23}}{\underbrace{G_2{G}_{2,2}}}\hfill \end{array}$$

(31)

Here, the longest pattern is $\left\{101\right\}$, so $b=3,$ and ${\mathsf{\Omega }}_Y=\{000,001,010,011,100,101,110,111$}. We obtain ${\mathit{\alpha }}_2=\left\{101\right\},$ ${\mathit{\alpha }}_1^{\prime }=\{110,011,111\}$, and ${\mathit{\alpha }}_2=\{000,001,100\}.$ Thus, ${\mathsf{\Omega }}_Y-\{{\mathit{\alpha }}_1,{\mathit{\alpha }}_1^{\prime },{\mathit{\alpha }}_2\}=\{010\},$ and the $(1\times 1)$ vectors (in fact, scalars) are $IP={\pi }_{01}{p}_5,$ $I{P}_1=p_5{p}_4{p}_5,$ $I{P}_1^{\prime }=p_7{p}_4{p}_5,$ and $I{P}_2=p_1{p}_2{p}_5$ (see the red rectangles in Table 4). The absorbing vectors (scalar) are ${\mathit{T}}_1^0=p_4,$ ${\mathit{T}}_{1^{\prime }}^0=0$ and ${\mathit{T}}_2^0=p_3.$ Also, note that the $(1\times 1)$ transition matrix is $\mathbb{T}=0,$ and ${(\mathbb{I}-z\mathbb{T})}^{-1}$=$\mathbb{I}.$ This result is due to the fact that, when we have the sequence $010,$ either trial 0 or 1 immediately yields a hit. The immediate conclusion is that the waiting time for a single hit is no more than four trials.

Table 4. The probability transition matrix of ${\mathbb{H}}_{2,3}·{\left({\mathbb{H}}_{3,3}\right)}^2$.

${\mathbb{H}}_{\mathbf{2},\mathbf{3}}·{\left({\mathbb{H}}_{\mathbf{3},\mathbf{3}}\right)}^{\mathbf{2}}$		ΩY
		000	001	010	011	100	101	110	111
2-state trials	00	(p1)3	(p1)2p2	p1p2p5	p1p2p6	p2p5p3	p2p5p4	p2p6p7	p2p6p8
	01	p5p3p1	p2p5p3	(p5)2p4	p5p4p6	p6p7p3	p6p7p4	p6p8p7	p6(p8)2
	10	p3(p1)2	p3p1p2	p2p5p3	p3p2p6	p5p4p3	p5(p4)2	p6p7p4	p4p6p8
	11	p7p3p1	p7p3p2	p7p4p5	p7p4p6	p8p7p3	p8p7p4	(p8)2p7	(p8)3

Step 3. We next derive $G_i\left(z\right)1_{\left\{z\le 3\right\}},i=1,1^{\prime },2,$ and the pgfs $G_{i,j}\left(z\right)1_{\left\{z\le 3\right\}},i,j=1,1^{\prime },2.$

$$\begin{array}{cc}{G}_1\left(z\right)1_{\left\{z\le 3\right\}}& =z^3·{\pi }_{10}·p_4,\hfill \\ G_{1^{\prime }}\left(z\right)1_{\left\{z\le 3\right\}}& =z^2·{\pi }_{11}+z^3·{\pi }_{01}·p_6,\hfill \\ G_2\left(z\right)1_{\left\{z\le 3\right\}}& =z^2·{\pi }_{00}{+z}^3·{\pi }_{10}·p_3.\hfill \end{array}$$

(32)

The function $G_{i,j}\left(z\right)1_{\left\{z\le 3\right\}}$ is constructed by a path of at most three multiplications of transition probabilities:

$$\begin{array}{cc}{G}_{1,1}\left(z\right)1_{\left\{z\le 3\right\}}=z^3·p_8^3,\hfill & G_{1,1^{\prime }}\left(z\right)1_{\left\{z\le 3\right\}}=z^2·p_6·p_8{+z}^3·p_5·p_4·p_6,\hfill \\ G_{1^{\prime },1}\left(z\right)1_{\left\{z\le 3\right\}}=z^3·p_8·p_7·p_4,\hfill & G_{1^{\prime },1^{\prime }}\left(z\right)1_{\left\{z\le 3\right\}}=z^2·p_8^2+z^3·p_7·p_4·p_6,\hfill \\ G_{1,2}\left(z\right)1_{\left\{z\le 3\right\}}=z^2·p_5·p_3{+z}^3·p_6·p_7·p_3,\hfill & G_{1^{\prime },2}\left(z\right)1_{\left\{z\le 3\right\}}=z^2·p_7·p_3{+z}^3·p_8·p_7·p_3,\hfill \\ G_{2,1}\left(z\right)1_{\left\{z\le 3\right\}}=z^3·p_2·p_5·p_4,\hfill & G_{2,1^{\prime }}\left(z\right)1_{\left\{z\le 3\right\}}=z^2·p_2·p_6{+z}^3·p_1·p_2·p_6,\hfill \\ G_{2,2}\left(z\right)1_{\left\{z\le 3\right\}}=z^2·p_1^2+z^3·p_2·p_5·p_3.\hfill \end{array}$$

(33)

Summarizing everything, we have:

$$\begin{array}{ccc}\hfill G_1\left(z\right)& =& z^3·{\pi }_{10}·p_4+z^3·{\pi }_{01}·p_5·z·p_4,\hfill \\ \hfill G_{1^{\prime }}\left(z\right)& =& z^2·{\pi }_{11}+z^3·{\pi }_{01}·p_6,\hfill \\ \hfill G_2\left(z\right)& =& z^2·{\pi }_{00}+z^3·{\pi }_{10}·p_3+z^3·{\pi }_{01}·p_5·z·p_3,\hfill \\ \hfill G_{1,1}\left(z\right)& =& z^3·p_8^3+z^3·p_5·p_4·p_5·z·p_4,\hfill \\ \hfill G_{1,1^{\prime }}\left(z\right)& =& z^2·p_6·p_8+z^3·p_5·p_4·p_6,\hfill \\ \hfill G_{1^{\prime },1}\left(z\right)& =& z^3·p_8·p_7·p_4+z^3·p_7·p_4·p_5·z·p_4,\hfill \\ \hfill G_{1^{\prime },1^{\prime }}\left(z\right)& =& z^2·p_8^2+z^3·p_7·p_4·p_6,\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill G_{1,2}\left(z\right)& =z^2·p_5·p_3+z^3·p_6·p_7·p_3+z^3·p_5·p_4·p_5·z·p_3,\hfill \\ \hfill G_{1^{\prime },2}\left(z\right)& =z^2·p_7·p_3+z^3·p_8·p_7·p_3+z^3·p_7·p_4·p_5·z·p_3,\hfill \\ \hfill G_{2,1}\left(z\right)& =z^3·p_2·p_5·p_4+z^3·p_1·p_2·p_5·z·p_4,\hfill \\ \hfill G_{2,1^{\prime }}\left(z\right)& =z^2·p_2·p_6+z^3·p_1·p_2·p_6,\hfill \\ \hfill G_{2,2}\left(z\right)& =z^2·p_1^2+z^3·p_2·p_5·p_3+z^3·p_1·p_2·p_5·z·p_3.\hfill \end{array}$$

(35)

Substituting (34) and (35) in (31) completes the derivation of $G\left(z\right).$

To add real data, the ReasonLabs Performance team further conducted an estimation of daily user-choice probabilities for free vs. premium services, focusing on the two most recent days of user activity,

$$p_1=0.3,p_2=0.7,p_3=0.4,p_4=0.6,p_5=0.5,p_6=0.5,p_7=0.2,p_8=0.8.$$

The above probabilities show that those who switch from free to premium have a 50% chance of reverting and a 50% chance of continuing with premium, while in the opposite case, i.e., those who switch from premium to free have a 60% likelihood of switching back to premium. We also see that 80% of users that chose premium twice will stay premium, and 70% of free users will try premium. These probabilities suggest that users are likely to move toward premium.

Based on these estimates, the probability vector $\mathit{\pi }$ and the pgf of the waiting times are given by:

$$\begin{array}{cc}& \mathit{\pi }=(0.1126,0.1972,0.1972,0.4930),\hfill \\ & G\left(z\right)=0.319·z^4+0.203·z^5+0.164·z^6+0.125·z^7+0.088·z^8+0.059·z^9\hfill \\ & +0.028·z^{10}+0.009·z^{11}+0.0016·z^{12},\hfill \end{array}$$

with an average number of trials of 5.805 and a variance of 3.197. The information that it takes approximately 6 visits (with a minimum of 4 and a maximum of 12 visits) to classify a customer may help in the optimization of resource allocation and preventing early interventions. In addition, the information may also contribute to mapping decision-making processes about customers and identifying behavioral changes in users.

4.3. Example 3

Our third example is inspired by Gamida Ltd. (https://gamida.co.il), (accessed on 1 February 2025). Gamida Ltd. provides targeted and comprehensive first-class services to the medical, science, technology, and industrial community in Israel and is part of the international group of companies Gamida For Life B.V. Among others, Gamida is the exclusive representative in Israel of the following companies: Cardinal Health, Lohmann & Rauscher, Abbott, B.Braun, Flen Health, BD, Getinge, Philips, and Integra LifeSciences, among others. The group specializes in the import, development, production, marketing, and distribution of products from research, diagnostics, medicine, and the advanced industries. Gamida specializes in developing wearable bracelets designed to monitor potential heart rate irregularities, especially in the elderly. These bracelets continuously collect heart rate data and process the accumulated information at the end of each day to produce a daily binary result indicating whether the heart rate was normal or not. A result ‘0’ represents a normal heart rate, and ‘1’ represents an abnormal rate; these results are then transmitted to the medical team for further analysis. The empirical investigation shows that, as expected, there is a dependence between the results. Gamida Medical Ltd. recommends two situations whose occurrence requires further investigation:

(1)

Two occurrences (not necessarily consecutive) of an abnormal rate after a normal or abnormal rate. An abnormal rate that appears twice (even after a normal one) may indicate a cardiac problem and is worth checking. According to the company’s experience, the need to record two consecutive outcomes helps determine whether the result is part of an ongoing trend or a single event.

(2)

Three (not necessarily consecutively) sequences of an abnormal rate followed by two normal results. This situation may be caused by a malfunction of the device, or as a result of other factors that caused a positive change in the heart rate.

Both conditions suggest an irregular heartbeat and require medical examination and attention. Next, we will demonstrate the algorithm on Example 3.

Step 1. Consider second-order Markov-dependent Bernoulli trials, with arguments as in (4)–(6), with two competing patterns, ${\mathsf{\Lambda }}^{\left(1\right)}=\{01,$ $11\}$ with $n_1=2$, and ${\mathsf{\Lambda }}^{\left(2\right)}=\left\{100\right\},$ with $n_2=2.$ Thus, the experiment terminates when one of the following rules occurs:

(i)

Two occurrences of 11, or two occurrences of $01,$ or first 01 and then $11,$ or vice versa—all occurrences are not necessarily consecutive.

(ii)

Two (not necessarily consecutive) occurrences of the pattern $\left\{100\right\}$.

An example of a sequence is given as follows, e.g., 101101, 001100001 (Rule (i)), and 10011100, 101100100 for (Rule (ii)).

Step 2. Accordingly, we have two sets $V_1$ and $V_2$ corresponding to ${\mathsf{\Lambda }}^{\left(1\right)}$ and ${\mathsf{\Lambda }}^{\left(2\right)},$ respectively, which are the same paths as those of Example 2, and thus, are given by (30). Here, we also have four pgfs ($G_{1,1}\left(z\right),$ $G_{1,2}\left(z\right),$ $G_{2,1}\left(z\right),$ $G_{2,2}\left(z\right)$) in addition to $G_1\left(z\right),$ $G_2\left(z\right)$. We further distinguish between the patterns $\{01,11\}$ of ${\mathsf{\Lambda }}^{\left(1\right)}$ by adding the sign “′” to the terms referring to $\left\{11\right\}$, and using no sign for $\left\{01\right\}$. Hence, we need to derive 12 pgfs with the final pgf given by (31).

Here, the longest pattern is $\left\{100\right\}$, so $b=3,$ ${\mathsf{\Omega }}_Y=\{000,001,010,011,100,101,110,111\}$. The absorbing states are ${\mathit{\alpha }}_1=\{001,010,011,101\},$ ${\mathit{\alpha }}_1^{\prime }=\{110,111\}$ and ${\mathit{\alpha }}_2=\left\{100\right\}.$ Therefore, we have ${\mathsf{\Omega }}_Y-\{{\mathit{\alpha }}_1,{\mathit{\alpha }}_1^{\prime },{\mathit{\alpha }}_2\}=\{000\},$ and the $(1\times 1)$ vectors (in fact, scalars) are $IP={\pi }_{00}{p}_1,$ $I{P}_1=p_5{p}_3{p}_1,$ $I{P}_1^{\prime }=p_7{p}_3{p}_1,$ $I{P}_2={\left(p_1\right)}^3$ (highlighted by the red rectangles in Table 5 and Table 6). The absorbing vectors (scalar) are ${\mathit{T}}_1^0=p_2,$ ${\mathit{T}}_{1^{\prime }}^0=0$ and ${\mathit{T}}_2^0=0.$ Also note that the $(1\times 1)$ matrix $\mathbb{T}=p_1,$ thus, ${(\mathbb{I}-z\mathbb{T})}^{-1}$=${(1-z·p_1)}^{-1}.$

Table 5. The highlighted probabilities used in Example 2 from the matrix ${\mathbb{H}}_{2,3}·{\left({\mathbb{H}}_{3,3}\right)}^2$.

${\mathbb{H}}_{\mathbf{2},\mathbf{3}}·{\left({\mathbb{H}}_{\mathbf{3},\mathbf{3}}\right)}^{\mathbf{2}}$		ΩY
		000	001	010	011	100	101	110	111
2-state trials	00	(p1)3	(p1)2p2	p1p2p5	p1p2p6	p2p5p3	p2p5p4	p2p6p7	p2p6p8
	01	p5p3p1	p2p5p3	(p5)2p4	p5p4p6	p6p7p3	p6p7p4	p6p8p7	p6(p8)2
	10	p3(p1)2	p3p1p2	p2p5p3	p3p2p6	p5p4p3	p5(p4)2	p6p7p4	p4p6p8
	11	p7p3p1	p7p3p2	p7p4p5	p7p4p6	p8p7p3	p8p7p4	(p8)2p7	(p8)3

Table 6. The highlighted probabilities used in Example 3 from the matrix ${\mathbb{H}}_{2,3}·{\left({\mathbb{H}}_{3,3}\right)}^2$.

${\mathbb{H}}_{\mathbf{2},\mathbf{3}}·{\left({\mathbb{H}}_{\mathbf{3},\mathbf{3}}\right)}^{\mathbf{2}}$		ΩY
		000	001	010	011	100	101	110	111
2-state trials	00	(p1)3	(p1)2p2	p1p2p5	p1p2p6	p2p5p3	p2p5p4	p2p6p7	p2p6p8
	01	p5p3p1	p2p5p3	(p5)2p4	p5p4p6	p6p7p3	p6p7p4	p6p8p7	p6(p8)2
	10	p3(p1)2	p3p1p2	p2p5p3	p3p2p6	p5p4p3	p5(p4)2	p6p7p4	p4p6p8
	11	p7p3p1	p7p3p2	p7p4p5	p7p4p6	p8p7p3	p8p7p4	(p8)2p7	(p8)3

Step 3. We next derive $G_i\left(z\right)1_{\left\{z\le 3\right\}},i=1,1^{\prime },2,$ and the pgfs $G_{i,j}\left(z\right)1_{\left\{z\le 3\right\}},i,j=1,1^{\prime },2.$

$$\begin{array}{cc}\hfill G_1\left(z\right)1_{\left\{z\le 3\right\}}& =z^2·{\pi }_{01}{+z}^3·\left({\pi }_{00}·p_2+{\pi }_{10}·p_4\right),\hfill \\ \hfill G_{1^{\prime }}\left(z\right)1_{\left\{z\le 3\right\}}& =z^2·{\pi }_{11}+z^3·{\pi }_{01}·p_6,\hfill \\ \hfill G_2\left(z\right)1_{\left\{z\le 3\right\}}& =z^3·{\pi }_{10}·p_3.\hfill \end{array}$$

(36)

The function $G_{i,j}\left(z\right)1_{\left\{z\le 3\right\}}$ is constructed by a path of at most three multiplications of transition probabilities:

$$\begin{array}{cc}{G}_{1,1}\left(z\right)1_{\left\{z\le 3\right\}}=z^2·p_5{p}_4+z^3·\left(p_6·p_7·p_4+p_5·p_3·p_2\right),\hfill & G_{1,1^{\prime }}\left(z\right)1_{\left\{z\le 3\right\}}=z^2·p_6·p_8,\hfill \\ G_{1^{\prime },1}\left(z\right)1_{\left\{z\le 3\right\}}=z^2·p_7·p_4{+z}^3·\left(p_8·p_7·p_4+p_7·p_3·p_2\right),\hfill & G_{1^{\prime },1^{\prime }}\left(z\right)1_{\left\{z\le 3\right\}}=z^2·p_8^2,\hfill \\ G_{1,2}\left(z\right)1_{\left\{z\le 3\right\}}=z^3·p_6·p_7·p_3,\hfill & G_{1^{\prime },2}\left(z\right)1_{\left\{z\le 3\right\}}=z^3·p_8·p_7·p_3,\hfill \\ G_{2,1}\left(z\right)1_{\left\{z\le 3\right\}}=z^2·p_1·p_2{+z}^3·\left(p_1·p_1·p_2+p_2·p_5·p_4\right),\hfill & G_{2,1^{\prime }}\left(z\right)1_{\left\{z\le 3\right\}}=z^2·p_2·p_6,\hfill \\ G_{2,2}\left(z\right)1_{\left\{z\le 3\right\}}=z^3·p_2·p_5·p_3.\hfill \end{array}$$

(37)

Summarizing all, we have:

$$\begin{array}{cc}\hfill G_1\left(z\right)& =z^2·{\pi }_{01}{+z}^3·{\pi }_{00}·p_2+z^3·{\pi }_{10}·p_4+z^3·{\pi }_{00}·p_1·{(1-z·p_1)}^{-1}·z·p_2,\hfill \\ \hfill G_{1^{\prime }}\left(z\right)& =z^2·{\pi }_{11}+z^3·{\pi }_{01}·p_6+z^3·{\pi }_{00}·p_1·{(1-z·p_1)}^{-1}·z·0=z^2·{\pi }_{11}+z^3·{\pi }_{01}·p_6,\hfill \\ \hfill G_2\left(z\right)& =z^3·{\pi }_{10}·p_3+z^3·{\pi }_{00}·p_1·{(1-z·p_1)}^{-1}·z·0=z^3·{\pi }_{10}·p_3,\hfill \\ \hfill G_{1,1}\left(z\right)& =z^2·p_5·p_4+z^3·(p_6·p_7·p_4+p_5·p_3·p_2)+z^3·p_5·p_3·p_1·{(1-z·p_1)}^{-1}·z·p_2,\hfill \\ \hfill G_{1,1^{\prime }}\left(z\right)& =z^2·p_6·p_8+z^3·p_5·p_3·p_1·{(1-z·p_1)}^{-1}·z·0=z^2·p_6·p_8,\hfill \\ \hfill G_{1^{\prime },1}\left(z\right)& =z^2·p_7·p_4{+z}^3·(p_8·p_7·p_4+p_7·p_3·p_2)+z^3·p_7·p_3·p_1·{(1-z·p_1)}^{-1}·z·p_2,\hfill \\ \hfill G_{1^{\prime },1^{\prime }}\left(z\right)& =z^2·p_8^2+z^3·p_7·p_3·p_1·{(1-z·p_1)}^{-1}·z·0=z^2·p_8^2,\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill G_{1,2}\left(z\right)& =z^3·p_6·p_7·p_3+z^3·p_5·p_3·p_1·{(1-z·p_1)}^{-1}·z·0=z^3·p_6·p_7·p_3,\hfill \\ \hfill G_{1^{\prime },2}\left(z\right)& =z^3·p_8·p_7·p_3+z^3·p_7·p_3·p_1·{(1-z·p_1)}^{-1}·z·0=z^3·p_8·p_7·p_3,\hfill \\ \hfill G_{2,1}\left(z\right)& =z^2·p_1·p_2{+z}^3·\left(p_1·p_1·p_2+p_2·p_5·p_4\right)+z^3·p_1^3·{(1-z·p_1)}^{-1}·z·p_2,\hfill \\ \hfill G_{2,1^{\prime }}\left(z\right)& =z^2·p_2·p_6+z^3·p_1^3·{(1-z·p_1)}^{-1}·z·0=z^2·p_2·p_6,\hfill \\ \hfill G_{2,2}\left(z\right)& =z^3·p_2·p_5·p_3+z^3·p_1^3·{(1-z·p_1)}^{-1}z·0=z^3·p_2·p_5·p_3.\hfill \end{array}$$

(39)

Substituting (38) and (39) in (31) completes the derivation of $G\left(z\right).$

Equations (38) and (39) show some interesting results. For example, we observe that at most three trials are needed to hit 11 or 100 (see $G_{1^{\prime }}\left(z\right),G_2\left(z\right)$), exactly two trials are needed from $11/01/100$ to 11 (see $G_{1,1^{\prime }}\left(z\right),$ $G_{1^{\prime },1^{\prime }}\left(z\right),$ and $G_{2,1^{\prime }}\left(z\right)$), and exactly three trials are needed from $11/01/100$ to 100 (see $G_{1,2}\left(z\right),G_{1^{\prime },2}\left(z\right),$ and $G_{2,2}\left(z\right)$). To explain the results, assume that the pattern 11 occurs. From that point, any trial 0 followed by 1 immediately yields the hit 01; the only option that 11 occurs again (before other patterns are hit) is by the double trial $11.$ Similarly, other cases can be explained.

According to Gamida Ltd., the transition probabilities are estimated by:

$$p_1=0.65,p_2=0.35,p_3=0.6,p_4=0.4,p_5=0.7,p_6=0.3,p_7=0.2,p_8=0.8,$$

with a steady-state probability vector

$$\mathit{\pi }=(0.3288,0.1918,0.1918,0.2877).$$

Calculating the pgf of the waiting time yielded the following:

$$\begin{array}{cc}G\left(z\right)& =0.293·z^4+0.189·z^5+0.0529·z^6+0.032·z^7+0.037·z^8+0.0104·z^9+\hfill \\ & {\displaystyle \frac{0.0614·z^6+0.0312·z^7+0.0122·z^9+0.0068·z^{10}}{1-0.65·z}}+{\displaystyle \frac{0.00671·z^8+0.00123·z^{11}}{{\left(1-0.65·z\right)}^2}},\hfill \end{array}$$

with an average number of trials of 6.63 and a variance 8.99, whilst the average detection time for identifying critical heart rate patterns is 6.6 days. These findings indicate that approximately one week of continuous monitoring is generally required to reliably detect critical cardiac rate patterns. The results can contribute to determining a reasonable detection window while minimizing false alarms. In addition, they can be useful in developing patient care protocols and setting alert thresholds, leading to a more efficient approach to cardiac monitoring.

5. Summary and Future Research

This paper studies a Markov-dependent model with Bernoulli trials and competing patterns. Competing patterns are compound patterns that compete to be the first to occur a specified number of times. Using a finite Markov chain and tools from probability theory, we develop an algorithm to derive a closed-form expression for the pgf of the waiting time distribution. It must be noted that despite being simple to understand, its application requires preparatory work for calculating path probabilities, which is a function of the parameter b. However, we believe that integrating both traditional computing techniques and advanced AI and machine learning approaches may be useful in developing efficient solutions even for the most complex cases. In this vein, the methodology presented in this paper serves as the foundational framework for the computational solutions discussed.

Theoretical extensions of the model can be interpreted in several directions. Our model assumes non-overlapping counting. It would be an interesting extension to generalize the algorithm to overlapping counting, where partially completed patterns can be finished at any time, regardless of whether another pattern has been completed after the partially completed pattern starts but before it is completed. Another direction is to investigate other stopping time rules, such as the sooner or later models. Here, a sooner model captures the number of trials required for the first occurrence of one of two competing patterns, and conversely, a later model refers to the numbers of trials required for both patterns to occur.