Rate of Convergence and Periodicity of the Expected Population Structure of Markov Systems that Live in a General State Space

Abstract

In this article we study the asymptotic behaviour of the expected population structure of a Markov system that lives in a general state space (MSGS) and its rate of convergence. We continue with the study of the asymptotic periodicity of the expected population structure. We conclude with the study of total variability from the invariant measure in the periodic case for the expected population structure of an MSGS.

1. Introductory Notes

Ref [1] introduced the stochastic process non-homogeneous Markov system (NHMS) with countable state space. Ref [2] introduced the stochastic process non-homogeneous semi-Markov system (NHSMS) in countable spaces. The theory and applications of both processes have seen the realization of an interesting growth and have found many applications in fields with great diversity. The latest reviews on the theory are given in [3,4]. The motives of this theory go back in the homogeneous Markov chain models for manpower systems in [5,6] and the evolution of which, in a variety of populations, are well described in [7]. The stochastic process of an NHMS represents a general frame work for a large variety of applied probability models to be accommodated in. However, real motives were the works on non-homogeneous Markov models for manpower systems such as [8,9,10]. We could selectively only supply some applications of the theory due to their large numbers. We start with the evolution of the HIV virus within the human body of T-cells in [11,12,13]; asthma was studied in [14]; reliability applications exist in [15] for example; examples of biomedical studies exist in [16,17]; applications in gene sequences ([18]); in DNA and web navigation ([19]); in manpower systems in [20,21,22,23,24,25]; in Physical Chemistry ([26]); examples from ecological modelling ([27]). Finally, there is the work of the research school of Prof McClean in the health systems (see, for example, [28,29,30,31,32]). Note that health systems are large manpower systems and each member has many parameters that are used to categorize him/her into the different groups. This characteristic of the health systems is what makes it an area of potential application of the present results (see [33]).

The rigorous foundation of Markov systems in general spaces (MSGS) was introduced in [33]. Also, the problem of asymptotic behaviour or ergodicity of Markov systems was studied. Important theorems were proved on the ergodicity of the distribution of expected population structure of a Markov system, which lives in the general space $\left(X,\mathcal{B}\left(X\right)\right)$. In addition, the total variability from the invariant measure was studied given that the Markov system is ergodic. It is shown that the total variation is finite.

Markov chains in general spaces have very important applications in the areas of Markov models in time series, models in control and systems theory, Markov models with regenerations times, Moran dams, Markov chains Monte Carlo simulation algorithms, etc. For more details, see [34] Chapter 2. There is a belief that the introduction in the presence of a population which is modulated as a Markov chain with general state space will increase the dynamics for new applications considerably, as it was described above in the countable case. Hence, although the results in the present are purely theoretical, the prospects for interesting applications are present and promising.

In Section 2 and Section 2.1, we start with the formal definition of a Markov system in a general state space, as it was introduced in [33]. In Section 2.2, we provide an abstract image for an MSGS in an attempt to help the reader in understanding the present paper. In Section 2.3, we introduce some known concepts and results useful in the foundation of the novel results that follow from this point to the end. In Section 3, we study the rate of convergence of the expected population structure in an MSGS. Two theorems are provided where in the first we provide conditions under which the MSGS is uniformly ergodic, and in the second we provide conditions under which the MSGS is V-uniformly ergodic. In Section 4, we proceed to study the asymptotic periodicity of the expected population structure, for a population that lives in a general state space. A basic theorem is proved, where it is shown that if d is the period of the inherent Markov chain, then the sequence of the expected population structures splits into d converging subsequences. The asymptotic periodicity of non-homogeneous Markov systems in countable spaces was studied by [35,36]. In the present study, the basic tools and methodology are different since, among other reasons, the equations are of completely different nature. That is, we move from difference equations to integral equations for the expected population structure. In Section 5, we study the total variability from the invariant measure in the periodic case for the expected population structure of an MSGS. In the form of two theorems, we provide conditions under which the total variability in the periodic case is finite.

2. The Markov System in a General State Space

We will start with a formal definition of a Markov system in a general state space ([33]) and then we provide an explanatory example. The reader could choose his own suitable order of reading the two subsections.

2.1. The Foundation of an MSGS

Let $\left(X,\mathcal{B}\left(X\right)\right)$ be the state space where X is a general set and $\mathcal{B}\left(X\right)$ denote a countably generated $\sigma$-field on X. Assume that small letters are elements of X and $A,B,C$ elements of $\mathcal{B}\left(X\right)$. Let us denote by $T\left(t\right)$ the population of the system at time t, which lives in $\left(X,\mathcal{B}\left(X\right)\right)$. We assume that ${\left\{T\left(t\right)\right\}}_{t=0}^{\infty }$ is a known stochastic process with state space ${\mathbb{Z}}^{+}$ and in discrete time.

Let that $\lambda :X\to {\mathbb{R}}^{+}$ be a positive $\sigma$-finite measure $\lambda :X\to {\mathbb{R}}^{+}$, that is,

$$\lambda \left(A\right)\ge 0,\lambda \left(A\right)<\infty \mathrm{for}\mathrm{every}A\in \mathcal{B}\left(X\right),$$

for which we have that $\lambda \left(X\right)=1.$ We assume that $\lambda \left(.\right)$ represents the initial distribution of a member of the population in the space $\left(X,\mathcal{B}\left(X\right)\right)$. We also assume that there is a set $W\in B\left(X\right)$ for which $\lambda \left(W\right)=0$. We will refer to W as the “gate” of the space $\left(X,\mathcal{B}\left(X\right)\right)$.

In the space $\left(X,\mathcal{B}\left(X\right)\right)$ where the population lives, it is assumed that each member has a membership, and that the leavers are leaving their memberships at the “gate”. Therefore, at any time t at the “gate” W, the number of memberships are those left by leavers, and the necessary $\Delta T\left(t\right)=T\left(t\right)-T\left(t-1\right)$ memberships to complete the desired population $T\left(t\right)$. We will only work the cases for which $\Delta T\left(t\right)\ge 0$. New members of the population take their memberships at the gate “gate” W from which they then make a transition to any $A\in B\left(X\right)$. Now let

$$\left\{P\left(x,A\right),x\in X,A\in \mathcal{B}\left(X\right)\right\},$$

(1)

be the transition probability kernel from $x\in X$ to the set $A\in \mathcal{B}\left(X\right)$ in one time step for a member of the population. Denote by ${\mathsf{\Phi }}_p={\left\{{\Phi }_p^{\left(n\right)}\right\}}_{n\in {\mathbb{N}}^{+}}$ the Markov chain with state space $\left(X,\mathcal{B}\left(X\right)\right)$, which is defined uniquely by the probability kernel $\left(2.1\right)$. We now make the following:

Assumption 1.

For the Markov chain ${\mathsf{\Phi }}_p={\left\{{\Phi }_p^{\left(n\right)}\right\}}_{n\in {\mathbb{N}}^{+}}$ the set W is an atom.

That is, there exists a measure $\mu \left(.\right)$ on $\mathcal{B}\left(X\right)$ such that

$$P\left(x,A\right)=\mu \left(A\right)\mathrm{for}\mathrm{every}x\in W\mathrm{and}A\in \mathcal{B}\left(X\right),$$

(2)

with $\mu \left(X\right)=1$.

Now, define by

$$Q\left(x,A\right)=P\left(x,A\right)+P\left(x,W\right)\mu \left(A\right),\mathrm{for}\mathrm{every}x\in X,A\in \mathcal{B}\left(X\right),x\notin W,$$

(3)

$$Q\left(x,A\right)=P\left(x,A\right)=\mu \left(A\right),\mathrm{for}\mathrm{every}x\in W,A\in \mathcal{B}\left(X\right).$$

(4)

The transition probability kernel $Q\left(x,A\right)$ in $\left(3\right)$ is the probability of a membership to make a transition from point x into the set A in one time step, either by direct transition of the member holding the membership with probability $P\left(x,A\right),$ or by the member leaving the system through the gate W with probability $P\left(x,W\right)$ and the entrance of a new member, who gets the membership of the member from gate W, into set A with probability $\mu \left(A\right)$. The transition probability kernel $Q\left(x,A\right)$ in $\left(4\right)$ is the probability of new memberships entering the space $\left(X,\mathcal{B}\left(X\right)\right)$ at each time interval $\left(t-1,t\right].$ From the gate of the atom, W is then distributed according to the $\sigma$-finite probability measure $\mu \left(.\right)$: $\mathcal{B}\left(X\right)\to \left[0,1\right]$.

We will use the notation by ${\mathsf{\Phi }}_Q={\left\{{\Phi }_Q^{\left(n\right)}\right\}}_{n\in {\mathbb{N}}^{+}}$ for the Markov chain with state space $\left(X,\mathcal{B}\left(X\right)\right)$ defined uniquely by $Q\left(x,A\right)$ in $\left(3\right)$ and $\left(4\right)$. We also use the notation $Q^n\left(x,A\right)$ for probability that a membership in x will move in n time steps to the set $A\in \mathcal{B}\left(X\right)$.

Now, define:

$N\left(x,t,A\right):$ the number of memberships of the population that are in set $A\in \mathcal{B}\left(X\right)$ at time t given that were initially at $x\in X$.

Assume, as it is usual in many applications, a natural partition of the space $\left(X,\mathcal{B}\left(X\right)\right)$, that is,

$${\left\{A_i\right\}}_{i=1}^{k+1},\mathrm{with}{A}_i\in \mathcal{B}\left(X\right)\mathrm{and}{A}_i\cap A_j=⌀\mathrm{for}i\ne j\mathrm{and}\mathrm{such}\mathrm{that}$$

(5)

$${\cup }_{i=1}^{k+1}{A}_i=X\mathrm{and}\mathrm{with}\mathrm{no}\mathrm{loss}\mathrm{of}\mathrm{generality}{A}_{k+1}=W.$$

(6)

It is of interest the expected population structure defined as:

$${\mathbb{E}}_{\mathbb{P}}\left[\mathbf{N}\left(t,X\right)\right]=\left[{\mathbb{E}}_{\mathbb{P}}\left[N\left(x,t,A_1\right)\right],{\mathbb{E}}_{\mathbb{P}}\left[N\left(x,t,A_2\right)\right],\dots ,{\mathbb{E}}_{\mathbb{P}}\left[N\left(x,t,A_k\right)\right]\right],$$

(7)

where $\mathbf{N}\left(t,X\right)$ is a non-negative measurable function on X. We are intersted also for the relative expected population structure defined by

$${\mathbb{E}}_{\mathbb{P}}\left[\mathbf{q}\left(t,X\right)\right]={\mathbb{E}}_{\mathbb{P}}\left[\mathbf{N}\left(t,X\right)/T\left(t\right)\right].$$

(8)

It could be easily seen that $q:X\to {\mathbb{R}}^{+}$ is a positive $\sigma$-finite probability measure since

$$q\left(x,t,A_i\right)=\frac{N\left(x,t,A_i\right)}{T\left(t\right)}\ge 0;q\left(x,t,A_i\right)<\infty \forall i\in {\mathbb{N}}^{+}\mathrm{and}q\left(x,t,X\right)=1.$$

(9)

Of central importance is the evolution of the expected population structure ${\mathbb{E}}_{\mathbb{P}}\left[\mathbf{N}\left(t,X\right)\right]$ or the relative expected population structure ${\mathbb{E}}_{\mathbb{P}}\left[\mathbf{q}\left(t,X\right)\right]$ in the study of Markov systems in general state space. From Vassiliou $\left(2014\right)$ we get that

$${\mathbb{E}}_{\mathbb{P}}\left[N\left(x,t,A\right)\right]=T\left(0\right)\underset{X}{\int }\lambda \left(dx\right)Q^t\left(x,A\right)+{\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right)\underset{X}{\int }\mu \left(dx\right)Q^{t-n}\left(x,A\right).$$

(10)

Note that

$$T\left(0\right)\underset{X}{\int }\lambda \left(dx\right)Q^t\left(x,A\right),$$

is the expected number of memberships from the initial population $T\left(0\right)$ that survived up to time t and are in set A. The part

$${\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right)\underset{X}{\int }\mu \left(dx\right)Q^{t-n}\left(x,A\right),$$

is the expected number of memberships that entered the system in the interval $\left[0,t\right]$ and survived up to time t and are in set $A.$

We call the random process described above with state space $\left(X,\mathcal{B}\left(X\right)\right)$ a Markov system in a general state space. In addition, we will call the Markov chain ${\mathsf{\Phi }}_Q={\left\{{\Phi }_Q^{\left(n\right)}\right\}}_{n\in {\mathbb{N}}^{+}}$ the inherent Markov chain of the Markov system.

2.2. An Abstract Image for MSGS

In the present subsection, in order to understand the various concepts and results, we will use the analogy of the example of the frog who jumps on lilies in a pond. This example of an abstract image was firstly used in [33] and the reader is referred to that study for a more extensive description and an explanation of the various concepts. In summary, imagine a lily pond that is covered by the leaves of the lilies. A population of frogs is living on the pond and a number of them are in every lily at each point in time and hence we have an expected population of frogs in the lilies. There are also leavers and newcomers in the pond at every point in time due to the antagonistic mating situation that exists between the males.

2.3. Introducing Some Important Concepts and Known Results

Throughout the paper, we will assume that the Markov chain ${\mathsf{\Phi }}_Q$ is $\psi$-irreducible. That is, there is a $\sigma$-finite measure $\psi$ on $\left(X,\mathcal{B}\left(X\right)\right)$ such that, for any $A\in$$\mathcal{B}\left(X\right)$ with $\psi \left(A\right)>0,$ and any $x\in X$,

$$Q^n\left(x,A\right)>0,\mathrm{for}\mathrm{all}n\mathrm{sufficiently}\mathrm{large}.$$

(11)

Assume that $\psi$ is maximal with respect to $\psi$. We will also assume that the Markov chain is aperiodic, apart from the last section, where we will study periodicity. The concept of $\psi -$irreducibility for Markov chains on general state spaces was introduced by [37,38], and followed by many authors [39,40,41,42,43,44].

We will adopt the term ergodicity for the Markov chains for which the limit, ${lim}_{n\to \infty }{\mathbf{Q}}^n\left(x,A\right)$ exists and is equal with $\pi \left(A\right)$, where $\pi$ is the invariant measure or stationary distribution of ${\mathsf{\Phi }}_Q$. As a mode of convergence in the space $\left(X,\mathcal{B}\left(X\right)\right)$, the total variation norm will be used. For a a signed measure $\mu$ on $\left(X,\mathcal{B}\left(X\right)\right)$, the total variation norm $∥\mu ∥$ is given by

$$∥\mu ∥:=\underset{f:\left|f\right|\le 1}{sup}\left|\mu \left(f\right)\right|=\underset{A\in B\left(X\right)}{sup}\mu \left(A\right)-\underset{A\in B\left(X\right)}{inf}\mu \left(A\right).$$

(12)

The key limit of interest in the present paper will be of the form

$$\underset{n\to \infty }{lim}∥Q^n\left(x,.\right)-\pi \left(.\right)∥=2\underset{n\to \infty }{lim}\underset{A}{sup}\left|Q^n\left(x,.\right)-\pi \left(.\right)\right|.$$

(13)

For more on the spectral theory and rates of convergence for Markov chains in general spaces, see [34]. Research is motivated by elegant mathematics as well as a range of applications. Note that, for countable inhomogeneous Markov systems it was proved that the space of all possible expected population structures is a Hilbert space ([45]). Now, for a $\psi -$irreducible Markov chain who lives in $\left(X,\mathcal{B}\left(X\right)\right)$, denoted by

$${\mathcal{B}}^{+}\left(X\right)=\left\{A\in \mathcal{B}\left(X\right):\psi \left(A\right)>0\right\}.$$

(14)

Let $Q_1$ and $Q_2$ be transition kernels for Markov chains, then for a positive function $\infty >V\ge 1$, define the V-norm distance between $Q_1$ and $Q_2$ as

$${\left|\left|\left|Q_1-Q_2\right|\right|\right|}_V:=\underset{x\in X}{sup}\frac{{∥Q_1\left(x,.\right)-Q_2\left(x,.\right)∥}_V}{V\left(x\right)}.$$

(15)

We define the outer product of the function 1 and the invariant measure $\pi$ the kernel

$$\left[1\otimes \pi \right]\left(x,A\right)=\pi \left(A\right),x\in X,A\in \mathcal{B}\left(X\right).$$

(16)

In many applications, we consider the distance ${\left|\left|\left|Q^n-1\otimes \pi \right|\right|\right|}_V$ for large n. We provide the following definition from [34]:

Definition 1.

(V-uniform ergodicity). An ergodic chain Φ with transition kernel Q is called V-uniformly ergodic if

$${\left|\left|\left|Q^n-1\otimes \pi \right|\right|\right|}_V\to 0,\mathit{as}n\to \infty .$$

(17)

Of great interest is the special case when $V=1$. In this case, we provide the following definition:

Definition 2.

(Uniform ergodicity). A chain Φ with transition kernel Q is called uniformly ergodic if it is V-uniformly ergodic with $V=1,$ that is, if

$$\underset{x\in X}{sup}∥Q^n\left(x,.\right)-\pi ∥\to 0,\mathit{as}n\to \infty .$$

(18)

The concept of a small set is very useful in order to have a finite break up into cyclic sets for $\psi$-irreducible chains.

Definition 3.

For $C\in \mathcal{B}\left(X\right)$ we say that it is a small set if there exists an $m>0,$ and a non-trivial measure ${\nu }_m$ on $\mathcal{B}\left(X\right)$, such that for all $x\in C$, $B\in \mathcal{B}\left(X\right)$

$$P^m\left(x,B\right)\ge {\nu }_m\left(B\right).$$

(19)

In such a case we say that C is ${\nu }_m$-small.

From [34] p. 105 we get the following Theorem:

Theorem 1.

If Φ is ψ-irreducible, then for every $A\in {\mathcal{B}}^{+}\left(X\right)$, there exists $m\ge 1$ and a ${\nu }_m$-small set $C\subseteq A$ such that $C\in {\mathcal{B}}^{+}\left(X\right)$ and ${\nu }_m\left(C\right)>0.$

Given the existence of just one small set from the previous Theorem, the Proposition in [34] p. 106 states that it is further possible for set X to be covered by small sets in the $\psi$-irreducible case.

Assume that C is any ${\nu }_M$-small set, and ${\nu }_M\left(C\right)>0$, without loss of generality. With the use of the set C${\nu }_M$ we define a cycle for a general irreducible Markov chain. We will suppress the subscript of $\nu$ for simplicity.

We have that

$$P^M\left(x,.\right)\ge \nu \left(.\right),x\in C\mathrm{and}\nu \left(C\right)>0,$$

hence, when the chain starts in C, there is a positive probability that the chain will return to C at time M. Let

$$E_C=\left\{n\ge 1:\mathrm{the}\mathrm{set}C\mathrm{is}{\nu }_n={\delta }_n\nu \mathrm{small}\mathrm{for}\mathrm{some}{\delta }_n>0\right\}.$$

Then the “period” for the set C, is given by the greatest common divisor of $E_C.$ The following Theorem ([34] p. 113) will be useful in what follows:

Theorem 2.

Assume that Φ is a ψ-irreducible Markov chain on $X.$

If $C\in {\mathcal{B}}^{+}\left(X\right)$ is ${\nu }_M$-small and d the greatest common divisor of the set $E_C.$ Then there exist disjoint sets $D_0,\dots ,D_{d-1}\in \mathcal{B}\left(X\right)$ (a "d-cycle) such that

$\left(i\right)$ for $x\in D_i,$$P\left(x,D_{i+1}\right)=1,$$i=0,\dots ,d-1\left(\mathrm{mod}d\right).$

$\left(ii\right)$ the set $N={\left[{\cup }_{i=1}^d{D}_i\right]}^c$ is ψ-null.

The d-cycle $\left\{D_i\right\}$ is maximal in the sense that for any collection

$$\left\{d^{{}^{\prime }},D_k^{{}^{\prime }},k=0,\dots ,d^{{}^{\prime }}-1\right\}$$

satisfying $\left(i\right)$-$\left(ii\right)$, we have $d^{{}^{\prime }}$ dividing d; whilst if $d=d^{{}^{\prime }}$, then, by reordering the indices if necessary $D_i^{{}^{\prime }}=D_i$ a.e. ψ.

It is obvious from the above that any small set must be essentially contained inside one specific member of the d-cycle $\left\{D_i\right\}$ cyclic class. From [34] p. 115, we need the following Proposition:

Proposition 1.

Suppose Φ is a ψ-irreducible chain with period d and a D-cycle $\left\{D_i,i=0,\dots ,d-1\right\}$. Then each of the sets $D_i$ is an absorbing ψ-irreducible set for the chain ${\Phi }_d$ corresponding to the transition kernel $P^d$, and ${\Phi }_d$ on each $D_i$ is aperiodic.

For any set $A\in \mathcal{B}\left(X\right)$, we denote by $oc\left(A\right)$ the number of visits by $\Phi$ to A after time zero and it is given by

$$oc\left(A\right)={\displaystyle \sum _{n=1}^{\infty }}\mathbf{1}\left\{{\Phi }^{\left(n\right)}\in A\right\}.$$

(20)

We define the kernel

$$U\left(x,A\right):={\displaystyle \sum _{n=1}^{\infty }}{Q}^n\left(x,A\right)={\mathbb{E}}_{\mathbb{P}}\left[o{c}_x\left(A\right)\right].$$

(21)

The chain $\Phi$ is called recurrent if it is $\psi$-irreducible and $U\left(x,A\right)=\infty$ for every $x\in X$ and every $A\in {\mathcal{B}}^{+}\left(X\right)$. If the chain $\Phi$ is irreducible and admits an atom $a\in {\mathcal{B}}^{+}\left(X\right)$ then if a is recurrent, then every set in ${\mathcal{B}}^{+}\left(X\right)$ is recurrent.

Definition 4.

The set $A\in {\mathcal{B}}^{+}\left(X\right)$ is called Harris recurrent if

$$P\left[o{c}_x\left(A\right)=\infty \right]=1.$$

(22)

A chain Φ is called Harris recurrent if it is ψ-irreducible and every set in ${\mathcal{B}}^{+}\left(X\right)$ is Harris recurrent.

Definition 5.

A σ-finite measure π on $\mathcal{B}\left(X\right)$ with the property

$$\pi \left(A\right)={\int }_X\pi \left(dx\right)P\left(x,A\right),A\in \mathcal{B}\left(X\right).$$

(23)

will be called invariant.

Suppose that $\Phi$ is irreducible, and admits an invariant probability measure $\pi$. Then $\Phi$ is called a positive chain. If $\Phi$ is Harris recurrent and positive, then $\Phi$ is called a positive Harris chain. From [34] p. 328 and p. 204, we get the following two results:

Theorem 3.

If Φ is positive Harris and aperiodic, then for any initial distribution λ

$$∥\underset{X}{\int }\lambda \left(dx\right)P^n\left(x,.\right)-\pi ∥\to 0\mathit{as}n\to \infty .$$

(24)

For any Harris recurrent set H, we write $H^{\infty }=\left\{y:L\left(y,H\right)=1\right\}$, where $L\left(y,H\right)$ is the probability of $\Phi$ of ever entering H from y, so that $H\subseteq H^{\infty }$ and $H^{\infty }$ is absorbing. We will call H a maximal absorbing set if $H=H^{\infty }$.

Theorem 4.

If Φ is recurrent, then we can write

$$X=H\cup N$$

where H is a non-empty maximal Harris set and N is transient.

3. Rate of Convergence of MSGS

In the present section, we will study the rate of convergence of Markov systems in general spaces. We will provide conditions under which the rate of convergence of the expected population structure of an ergodic Markov system is uniformly ergodic and V-uniformly ergodic. From [34] p. 393, we get the following two Theorems:

Theorem 5.

Suppose that Φ is ψ-irreducible and aperiodic with transition kernel Q. Then the following are equivalent for $V\ge 1:$

$\left(i\right)$Φ is V-uniformly ergodic.

$\left(ii\right)$ There exists $r>1$ and $R<\infty$ such that for all $n\in {\mathbb{N}}^{+}$

$${\left|\left|\left|Q^n-1\otimes \pi \right|\right|\right|}_V\le R{r}^{-n}.$$

$\left(iii\right)$ There exists some $n>0$ such that ${\left|\left|\left|Q^i-1\otimes \pi \right|\right|\right|}_V<\infty$ for $i\le n$ and

$${\left|\left|\left|Q^i-1\otimes \pi \right|\right|\right|}_V<1.$$

Theorem 6.

For any Markov chain Φ with transition kernel Q, the following are equivalent:

$\left(i\right)$Φ is uniformly ergodic.

$\left(ii\right)$ There exists $r>1$ and $R<\infty$ such that for all x

$$∥Q^n\left(x,.\right)-\pi \left(.\right)∥\le R{r}^{-n};$$

(25)

that is, the convergence takes place at a uniform geometric rate.

$\left(iii\right)$ For some $n\in {\mathbb{N}}^{+}$

$$\underset{x\in X}{sup}∥Q^n\left(x,.\right)-\pi \left(.\right)∥<1.$$

$\left(iv\right)$ The chain is aperiodic and Doeblin condition holds: that is, there is a probability measure ϕ on $\mathcal{B}\left(X\right)$ and $\epsilon <1$, $\delta >0,$$n\in {\mathbb{N}}^{+}$ such that when $\varphi \left(A\right)>\epsilon$

$$\underset{x\in X}{inf}{Q}^n\left(x,A\right)>\delta .$$

$\left(v\right)$ The set X for some m is ${\nu }_m$-small.

$\left(vi\right)$ For every set $A\in {\mathcal{B}}^{+}\left(X\right)$ and an aperiodic Markov chain there is a petit set C with

$$\underset{x\in X}{sup}{\mathbb{E}}_x\left[{\tau }_C\right]<\infty ,$$

and in this case ${sup}_{x\in X}{\mathbb{E}}_x\left[{\tau }_A\right]<\infty .$

$\left(vii\right)$ For every set $A\in {\mathcal{B}}^{+}\left(X\right)$ and an aperiodic Markov chain there is a petite set C and $\kappa >1$ with

$$\underset{x\in X}{sup}{\mathbb{E}}_x\left[{\kappa }^{{\tau }_C}\right]<\infty ,$$

and we have for some ${\kappa }_A>1,$

$$\underset{x\in X}{sup}{\mathbb{E}}_x\left[{\kappa }_A^{{\tau }_A}\right]<\infty ,$$

$\left(viii\right)$ For an aperiodic Markov chain there is a bounded solution $V\ge 1$ to

$$\Delta V\left(x\right)\le -\beta V\left(x\right)+b{\mathbb{I}}_C\left(x\right),x\in X.$$

for some $\beta >0,b<\infty$, and some petit set $C,$

Under $\left(v\right),$ we get that for any $x,$

$$∥Q^n\left(x,.\right)-\pi \left(.\right)∥\le 2{\rho }^{\left[n/m\right]},$$

where $\rho =1-{\nu }_m\left(X\right).$

Now, from [46], we will borrow the following theorem adapted for an MSGS:

Theorem 7.

Let a Markov system in a general state space $\left(X,\mathcal{B}\left(X\right)\right)$, which is expanding $\left(\Delta T\left(t\right)=T\left(t\right)-T\left(t-1\right)\ge 0\right)$. The following two statements are equivalent

$\left(i\right)$ The sequence ${\left\{T\left(t\right)\right\}}_{t=0}^{\infty }$ converges, that is, ${lim}_{t\to \infty }T\left(t\right)=T$ geometrically.

$\left(ii\right)$ The non-negative sequence ${\left\{\Delta T\left(t\right)\right\}}_{t=0}^{\infty }$ tends to zero geometrically.

We will start by studying Uniform ergodicity for an MSGS for simplicity reasons and then proceed to study V-uniform ergodicity.

Definition 6.

(Uniform ergodicity for an MSGS). Let a Markov system in a general state space $\left(X,\mathcal{B}\left(X\right)\right).$ Also let ${\left\{T\left(t\right)\right\}}_{t=0}^{\infty }$ be the total population of memberships with ${lim}_{t\to \infty }T\left(t\right)=T;$$Q\left(x,A\right)$ with $x\in X$ and $A\in \mathcal{B}\left(X\right)$ the transition kernel of the inherent Markov chain of memberships and ${\mathbb{E}}_Q\left[\mathbf{N}\left(x,A\right)\right]$ the expected population structure. We say that the MSGS is uniformly ergodic if and only if there exists a $C<\infty$ and an $0<a<1$ such that

$$∥{\mathbb{E}}_Q\left[N\left(x,t,A\right)]-T\pi \left(A\right)\right]∥\le C{a}^t\mathit{for}\mathit{every}x\in X,A\in \mathcal{B}\left(X\right).$$

(26)

We now provide the following theorem concerning uniform ergodicity for an MSGS.

Theorem 8.

Let a Markov system that lives in $\left(X,\mathcal{B}\left(X\right)\right)$. Assume in addition that

$$\underset{t\to \infty }{lim}T\left(t\right)=T\mathit{at}a\mathit{geometric}\mathit{rate},\Delta T\left(t\right)\ge 0\mathit{for}\mathit{every}t\in {\mathbb{N}}_{+},$$

and that the inherent Markov chain with transition kernel $Q\left(x,A\right)$ is uniformly ergodic. Then the MSGS is uniformly ergodic.

Proof. 

We have that

$$\begin{array}{c}∥{\mathbb{E}}_Q\left[N\left(x,t,A\right)]-T\pi \left(A\right)\right]∥=∥{\mathbb{E}}_Q\left[N\left(x,t,A\right)]-T\left(0\right)\pi \left(A\right)-\left(T-T\left(0\right)\pi \left(A\right)\right)\right]∥\\ =\left(\mathrm{From}\mathrm{Equation}\left(10\right)\right)\\ =\Vert T\left(0\right)\left[\underset{X}{\int }\lambda \left(dx\right)Q^t\left(x,A\right)-\pi \left(A\right)\right]+{\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right){\int }_X\mu \left(dx\right)Q^{t-n}\left(x,A\right)-{\displaystyle \sum _{n=1}^{\infty }}\Delta T\left(n\right)\pi \left(A\right)\Vert \\ \le \Vert T\left(0\right)\left[\underset{X}{\int }\lambda \left(dx\right)Q^t\left(x,A\right)-\pi \left(A\right)\right]+{\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right)\left[{\int }_X\mu \left(dx\right)Q^{t-n}\left(x,A\right)-\pi \left(A\right)\right]-{\displaystyle \sum _{n=t}^{\infty }}\Delta T\left(n\right)\pi \left(A\right)\Vert \\ \le T\left(0\right)\Vert \underset{X}{\int }\lambda \left(dx\right)Q^t\left(x,A\right)-\pi \left(A\right)\Vert \\ +\Vert {\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right)\left[{\int }_X\mu \left(dx\right)Q^{t-n}\left(x,A\right)-\pi \left(A\right)\right]\Vert +\Vert {\displaystyle \sum _{n=t}^{\infty }}\Delta T\left(n\right)\pi \left(A\right)\Vert .\end{array}$$

(27)

From $\left(27\right)$ and since ${lim}_{t\to \infty }T\left(t\right)=T$ is geometrically fast and $\Delta T\left(t\right)\ge 0$ for every $t\in {\mathbb{N}}_{+}$ from Theorem 6 we get that there exist a $c>0$ and a $1>b>0$ such that

$$\Vert {\displaystyle \sum _{n=t}^{\infty }}\Delta T\left(n\right)\pi \left(A\right)\Vert \le ∥\pi \left(A\right)∥{\displaystyle \sum _{n=t}^{\infty }}\Delta T\left(n\right)\le c{\left(1-b\right)}^{-1}{b}^t.$$

(28)

Also from $\left(27\right)$, since ${\int }_X\lambda \left(dx\right)=1$ and since the inherent Markov chain with kernel $Q\left(x,A\right)$ is uniformly ergodic, from Theorem 6 (ii) there exists $R<\infty$ and $r>1$ such that

$$\begin{array}{c}T\left(0\right)\Vert \underset{X}{\int }\lambda \left(dx\right)Q^t\left(x,A\right)-\pi \left(A\right)\Vert =T\left(0\right)\Vert \underset{X}{\int }\lambda \left(dx\right)\left[Q^t\left(x,A\right)-\pi \left(A\right)\right]\Vert \\ \le T\left(0\right)\underset{X}{\int }\lambda \left(dx\right)∥Q^t\left(x,A\right)-\pi \left(A\right)∥\le T\left(0\right)\underset{X}{\int }\lambda \left(dx\right)R{a}^t\le T\left(0\right)R{a}^t,\end{array}$$

(29)

with $a=r^{-1},$$0<a<1.$

Now, again since ${lim}_{t\to \infty }T\left(t\right)=T$ is geometrically fast and $\Delta T\left(t\right)\ge 0$ for every $t\in {\mathbb{N}}_{+}$ from Theorem 6 there exist a $c>0$ and a $1>b>0$ with $\Delta T\left(t\right)\le c{b}^t$; in addition since the inherent Markov chain with kernel $Q\left(x,A\right)$ is uniformly ergodic, from Theorem 6 (ii) there exists $R<\infty$ and $r>1$, that is, $0<a=r^{-1}<1$ such that

$$\begin{array}{c}\Vert {\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right)\left[{\int }_X\mu \left(dx\right)Q^{t-n}\left(x,A\right)-\pi \left(A\right)\right]\Vert \\ \le {\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right)∥{\int }_X\mu \left(dx\right)Q^{t-n}\left(x,A\right)-\pi \left(A\right)∥\\ \le {\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right){\int }_X\mu \left(dx\right)∥Q^{t-n}\left(x,A\right)-\pi \left(A\right)∥\le {\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right){\int }_X\mu \left(dx\right)R{a}^{t-n}\\ \le {\displaystyle \sum _{n=1}^{t-1}}c{b}^{t}R{a}^{t-n}\le cR{\left(1-b/a\right)}^{-1}\left(1-{\left(b/a\right)}^t\right)a^t=R^{*}{a}^t.\end{array}$$

(30)

Now, from (27)–(30) we have that

$$∥{\mathbb{E}}_Q\left[N\left(x,t,A\right)]-T\pi \left(A\right)\right]∥\le T\left(0\right)R{a}^t+R^{*}{a}^t+c{\left(1-b\right)}^{-1}{b}^t.$$

(31)

Let $a_1=max\left\{a,b\right\}$ then $0<a_1<1$ and

$$\begin{array}{rr}∥{\mathbb{E}}_Q\left[N\left(x,t,A\right)]-T\pi \left(A\right)\right]∥& \le T\left(0\right)R{a}_1^t+R^{*}{a}_1^t+c{\left(1-b\right)}^{-1}{a}_1^t\\ & \le \left[T\left(0\right)R+R^{*}+c{\left(1-b\right)}^{-1}\right]a_1^t\\ & \le c^{*}{a}^t,\end{array}$$

which concludes the proof that MSGS is uniformly ergodic. □

Generalizing the above results to $V-$iniform ergodicity is now straight forward. We start with the definition

Definition 7.

(V-Uniform ergodicity for an MSGS). Let a Markov system in a general state space $\left(X,\mathcal{B}\left(X\right)\right).$ Also let ${\left\{T\left(t\right)\right\}}_{t=0}^{\infty }$ be the total population of memberships with ${lim}_{t\to \infty }T\left(t\right)=T;$$Q\left(x,A\right)$ with $x\in X$ and $A\in \mathcal{B}\left(X\right)$ the transition kernel of the inherent Markov chain of memberships and ${\mathbb{E}}_Q\left[\mathbf{N}\left(x,A\right)\right]$ the expected population structure. We say that the MSGS is $V-$ uniformly ergodic if and only if there exists a $C<\infty$ and an $0<a<1$ such that

$$\mid \mid \mid {\mathbb{E}}_Q\left[N\left(x,t,A\right)]-T\pi \left(A\right)\right]\mid \mid {\mid }_V\le C{a}^t\mathit{for}\mathit{every}x\in X,A\in \mathcal{B}\left(X\right).$$

(32)

Following exactly the proof of Theorem 8, we arrive at the following result:

Theorem 9.

Let a Markov system that leaves in $\left(X,\mathcal{B}\left(X\right)\right)$. Assume in addition that

$$\underset{t\to \infty }{lim}T\left(t\right)=T\mathit{at}a\mathit{geometric}\mathit{rate},\Delta T\left(t\right)\ge 0\mathit{for}\mathit{every}t\in {\mathbb{N}}_{+},$$

and that the inherent Markov chain with transition kernel $Q\left(x,A\right)$ is $V-$uniformly ergodic. Then the MSGS is $V-$uniformly ergodic.

4. Asymptotic Periodicity of an MSGS

The study of the asymptotic behaviour of Markov chains has been very important for finite or countable spaces from the start of their history. The general state space results are due to [40,41,47]. In the non-homogeneous Markov chains when the state space is countable, the initial important results on asymptotic behaviour were achieved by [48,49,50,51,52]. For the semi Markov process analogue, a wealth of results exist in the books by [53,54]. For NHMS and NHSMS on countable spaces the asymptotic behaviour has also been a problem of central importance see for example [1,36,55,56,57]. In the present section, we will assume that the Markov chain is periodic and study the asymptotic behaviour of the expected population structure, which apparently is an important problem for a population that lives in $\left(X,\mathcal{B}\left(X\right)\right)$. Let it be that the inherent Markov chain of the MSGS with kernel Q is positive Harris with period $d\ge 1$, and let the d-cycle described in Theorem 2 be the set $\left\{D_0,D_1,\dots ,D_{d-1}\right\}$, where $D_i\in \mathcal{B}\left(X\right)$ for $i=0,1,\dots ,d-1.$ We have assumed that $\lambda \left(x\right)$ is the initial distribution on X. Then the distribution on the set $D_i$ is

$${\lambda }_i\left(x\right)=\frac{\lambda \left(x\right)}{\lambda \left(D_i\right)}\mathrm{where}x\in D_i.$$

(33)

From Theorem 2, we have that if $x\in D_0$ then $Q\left(x,D_1\right)=1$ and consequently $Q^2\left(x,D_2\right)={\int }_{D_1}Q\left(x,dy\right)Q\left(y,D_2\right)$. However, for $y\in D_1$ we have $Q\left(y,D_2\right)=1,$ hence $Q^2\left(x,D_2\right)={\int }_{D_1}Q\left(x,dy\right)=Q\left(x,D_1\right)=1.$ Similarly, since the period is d we have

$$\mathrm{If}x\in D_i\mathrm{then}{Q}^{nd}\left(x,D_i\right)=1,Q^{nd}\left(x,D_j\right)=0\mathrm{for}i\ne j.$$

(34)

Now, in general for $x\in D_i$ and for $d>i+\upsilon$ we have

$$\begin{array}{c}{Q}^{nd+\upsilon }\left(x,D_{i+\upsilon }\right)=\underset{D_i}{\int }{Q}^{nd}\left(x,dy\right)Q^{\upsilon }\left(y,D_{i+\upsilon }\right)\\ =\left(\mathrm{for}y\in D_i\mathrm{and}d>i+\upsilon ,Q^{\upsilon }\left(y,D_{i+\upsilon }\right)=1\right)\\ ={\int }_{D_i}{Q}^{nd}\left(x,dy\right)=1.\end{array}$$

(35)

It is apparent from $\left(35\right)$ that

$$\mathrm{If}x\in D_i\mathrm{and}\mathrm{for}d>i+\upsilon \mathrm{we}\mathrm{have}{Q}^{nd+\upsilon }\left(x,D_j\right)=0\mathrm{for}j\ne i+\upsilon .$$

(36)

If $x\in D_i$ and for $d\le i+\upsilon$ we have

$$\begin{array}{rr}{Q}^{nd+\upsilon }\left(x,D_{i+\upsilon -d}\right)& =\underset{D_i}{\int }{Q}^{nd}\left(x,dy\right)Q^{\upsilon }\left(y,D_{i+\upsilon -d}\right)\\ & =\left(\mathrm{for}y\in D_i\mathrm{and}d\le i+\upsilon ,Q^{\upsilon }\left(y,D_{i+\upsilon -d}\right)=1\right)\\ & =\underset{D_i}{\int }{Q}^{nd}\left(x,dy\right)=1\end{array}$$

From the fact that the inherent Markov chain with kernel Q is assumed to be positive Harris with period d we know that for each cyclic set is positive Harris on the d-skeleton and aperiodic, and by Theorem 3 it is straightforward to check the following Theorem:

Theorem 10.

If the inherent Markov chain of the MSGS is positive Harris and periodic with period d, then for the sets $\left\{D_0,D_1,\dots ,D_{d-1}\right\}$ with $D_i\in \mathcal{B}\left(X\right)$ we have that

$$\underset{n\to \infty }{lim}∥\underset{D_i}{\int }{\lambda }_i\left(dx\right)P^{nd+\upsilon }\left(x,A\right)-{\pi }_{i+\upsilon }\left(A\right)∥=0\mathit{if}d>i+\upsilon \mathit{for}i=0,1,\dots ,d-1,$$

for any set $A\subseteq D_{i+\upsilon },$ and

$$\underset{n\to \infty }{lim}∥\underset{D_i}{\int }{\lambda }_i\left(x\right)P^{nd+\upsilon }\left(x,A\right)-{\pi }_{i+\upsilon -d}\left(A\right)∥=0\mathit{if}d\le i+\upsilon \mathit{for}i=0,1,\dots ,d-1,$$

for any set $A\subseteq D_{i+\upsilon -d}.$

Now, from the above Theorem and the decomposition Theorem 4, we immediately get the following:

Theorem 11.

If the inherent Markov chain of the MSGS is positive recurrent and periodic with period d, then for the sets $\left\{D_0,D_1,\dots ,D_{d-1}\right\}$ with $D_i\in \mathcal{B}\left(X\right)$, then for each i there exist a ${\pi }_i-$null set $N_i$ which for every initial distribution ${\lambda }_i$ with ${\lambda }_i\left(N_i\right)=0$

$$\underset{n\to \infty }{lim}∥\underset{D_i}{\int }{\lambda }_i\left(dx\right)P^{nd+\upsilon }\left(x,A\right)-{\pi }_{i+\upsilon }\left(A\right)∥=0\mathit{if}d>i+\upsilon \mathit{for}i=0,1,\dots ,d-1,$$

for any set $A\subseteq D_{i+\upsilon },$ and

$$\underset{n\to \infty }{lim}∥\underset{D_i}{\int }{\lambda }_i\left(dx\right)P^{nd+\upsilon }\left(x,A\right)-{\pi }_{i+\upsilon -d}\left(A\right)∥=0\mathit{if}d\le i+\upsilon \mathit{for}i=0,1,\dots ,d-1,$$

for any set $A\subseteq D_{i+\upsilon -d}.$

We will now start building the proof for the periodicity Theorem for an MSGS and we will state the theorem in the end. The important question is the periodicity of the expected population structure

$${\mathbb{E}}_{\mathbb{P}}\left[\mathbf{N}\left(t,X\right)\right]=\left[{\mathbb{E}}_{\mathbb{P}}\left[N\left(x,t,A_1\right)\right],{\mathbb{E}}_{\mathbb{P}}\left[N\left(x,t,A_2\right)\right],\dots ,{\mathbb{E}}_{\mathbb{P}}\left[N\left(x,t,A_k\right)\right]\right],$$

when the inherent Markov chain with kernel Q is positive Harris and periodic with period d or is positive recurrent and periodic with period d. With no loss of generality and for simplicity reasons, in what follows we will use the set A instead of any of the sets $A_i,\left(i=1,2,\dots ,k\right).$ From Equation $\left(10\right)$ we have that

$${\mathbb{E}}_{\mathbb{P}}\left[N\left(x,t,A\right)\right]=T\left(0\right)\underset{X}{\int }\lambda \left(dx\right)Q^t\left(x,A\right)+{\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right)\underset{X}{\int }\mu \left(dx\right)Q^{t-n}\left(x,A\right).$$

(37)

We will work with each part of the right hand side of Equation $\left(37\right)$ separately. We have that

$$\begin{array}{rr}\underset{X}{\int }\lambda \left(dx\right)Q^t\left(x,A\right)=& \lambda \left(D_0\right)\underset{D_0}{\int }{\lambda }_0\left(dx\right)Q^t\left(x,A\right)+\lambda \left(D_1\right)\underset{D_1}{\int }{\lambda }_1\left(dx\right)Q^t\left(x,A\right)\\ & +\dots +\lambda \left(D_{d-1}\right)\underset{D_{d-1}}{\int }{\lambda }_{d-1}\left(dx\right)Q^t\left(x,A\right)\end{array}$$

(38)

We could write the set A in general as follows

$$A=A\cap D_0\cup A\cap D_1\cup \dots \cup A\cap D_{d-1},\mathrm{with}\left(A\cap D_i\right)\cap \left(A\cap D_j\right)=⌀,\mathrm{for}i\ne j.$$

(39)

Let without loss of generality that $t=\pi d+\upsilon$ and consider in what follows that for all $i=0,1,\dots ,d-1$, $d>i+\upsilon$. Then we have that

$$Q^t\left(x,A\right)=Q^{\pi d+\upsilon }\left(x,A\right)=Q^{\pi d+\upsilon }\left(x,A\cap D_0\right)+Q^{\pi d+\upsilon }\left(x,A\cap D_1\right)+\dots +Q^{\pi d+\upsilon }\left(x,A\cap D_{d-1}\right).$$

(40)

From $\left(38\right)$ we have that

$$\begin{array}{c}\underset{t\to \infty }{lim}∥\underset{X}{\int }\lambda \left(dx\right)Q^t\left(x,A\right)-{\displaystyle \sum _{i=0}^{d-1}}\lambda \left(D_i\right){\pi }_{i+\upsilon }\left(A\cap D_{i+\upsilon }\right)∥=\\ =\underset{\pi \to \infty }{lim}∥{\displaystyle \sum _{i=0}^{d-1}}\lambda \left(D_i\right)\underset{D_i}{\int }{\lambda }_i\left(dx\right)Q^{\pi d+\upsilon }\left(x,A\right)-{\displaystyle \sum _{i=0}^{d-1}}\lambda \left(D_i\right){\pi }_{i+\upsilon }\left(A\cap D_{i+\upsilon }\right)∥\\ \le \underset{\pi \to \infty }{lim}{\displaystyle \sum _{i=0}^{d-1}}\lambda \left(D_i\right)∥\underset{D_i}{\int }{\lambda }_i\left(dx\right)Q^{\pi d+\upsilon }\left(x,A\right)-{\pi }_{i+\upsilon }\left(A\cap D_{i+\upsilon }\right)∥\end{array}$$

(41)

Now, from $\left(40\right)$ and $\left(41\right)$ we get that

$$\begin{array}{c}\underset{t\to \infty }{lim}∥\underset{X}{\int }\lambda \left(dx\right)Q^t\left(x,A\right)-{\displaystyle \sum _{i=0}^{d-1}}\lambda \left(D_i\right){\pi }_{i+\upsilon }\left(A\cap D_{i+\upsilon }\right)∥\\ \le \underset{\pi \to \infty }{lim}{\displaystyle \sum _{i=0}^{d-1}}\lambda \left(D_i\right)∥{\displaystyle \sum _{j=0}^{d-1}}\underset{D_i}{\int }{\lambda }_i\left(dx\right)Q^{\pi d+\upsilon }\left(x,A\cap D_j\right)-{\pi }_{i+\upsilon }\left(A\cap D_{i+\upsilon }\right)∥\\ \le \underset{\pi \to \infty }{lim}{\displaystyle \sum _{i=0}^{d-1}}\lambda \left(D_i\right)∥\underset{D_i}{\int }{\lambda }_i\left(dx\right)Q^{\pi d+\upsilon }\left(x,A\cap D_{i+\upsilon }\right)-{\pi }_{i+\upsilon }\left(A\cap D_{i+\upsilon }\right)∥\\ +\underset{\pi \to \infty }{lim}{\displaystyle \sum _{i=0}^{d-1}}\lambda \left(D_i\right){\displaystyle \sum _{j=0,j\ne i+\upsilon }^{d-1}}∥\underset{D_i}{\int }{\lambda }_i\left(dx\right)Q^{\pi d+\upsilon }\left(x,A\cap D_j\right)∥\\ \left(\mathrm{from}\mathrm{Theorem}10\mathrm{and}\mathrm{relation}\left(36\right)\mathrm{we}\mathrm{get}\mathrm{that}\right)\\ \le \in .\end{array}$$

(42)

Hence, we proved that when $t=\pi d+\upsilon$ and $d>i+\upsilon$ then the limit of $\underset{X}{\int }\lambda \left(dx\right)Q^t\left(x,A\right)$ as $t\to \infty$ is

$${\displaystyle \sum _{i=0}^{d-1}}\lambda \left(D_i\right){\pi }_{i+\upsilon }\left(A\cap D_{i+\upsilon }\right).$$

(43)

Similarly, following the same steps we could prove that for $t=\pi d+\upsilon$ and $d\le i+\upsilon$ then

$$\underset{t\to \infty }{lim}∥\underset{X}{\int }\lambda \left(dx\right)Q^t\left(x,A\right)-{\displaystyle \sum _{i=0}^{d-1}}\lambda \left(D_i\right){\pi }_{i+\upsilon -d}\left(A\cap D_{i+\upsilon -d}\right)∥=0.$$

(44)

From $\left(42\right)$ and $\left(43\right)$ it is not difficult to check that

$$\begin{array}{c}\underset{t\to \infty }{lim}\Vert \underset{X}{\int }\lambda \left(dx\right)Q^t\left(x,A\right)-{\displaystyle \sum _{i=0}^{d-\upsilon -1}}\lambda \left(D_i\right){\pi }_{i+\upsilon }\left(A\cap D_{i+\upsilon }\right)\\ -{\displaystyle \sum _{i=d-\upsilon }^{d-1}}\lambda \left(D_i\right){\pi }_{i+\upsilon -d}\left(A\cap D_{i+\upsilon -d}\right)\Vert =0.\end{array}$$

(45)

Assume now that ${lim}_{t\to \infty }T\left(t\right)=T$ and $\Delta T\left(t\right)\ge 0$ for every t. Let again that $t=\pi d+\upsilon$ then

$$\begin{array}{c}U\left(t\right)={\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right)\underset{X}{\int }\mu \left(dx\right)Q^{t-n}\left(x,A\right)={\displaystyle \sum _{n=1}^{\pi d+\upsilon -1}}\Delta T\left(n\right)\underset{X}{\int }\mu \left(dx\right)Q^{\pi d+\upsilon -n}\left(x,A\right)\\ ={\displaystyle \sum _{m=0}^{d-1}}{\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right)\underset{X}{\int }\mu \left(dx\right)Q^{\pi d+\upsilon -\left(\tau d+m\right)}\left(x,A\right)\\ +{\displaystyle \sum _{m=0}^{\upsilon -1}}\Delta T\left(\pi d+m\right)\underset{X}{\int }\mu \left(dx\right)Q^{\pi d+\upsilon -\left(\pi d+m\right)}\left(x,A\right).\end{array}$$

(46)

Now, since ${lim}_{t\to \infty }\Delta T\left(t\right)=0$ we have

$$\begin{array}{c}\underset{\pi \to \infty }{lim}∥{\displaystyle \sum _{m=0}^{\upsilon -1}}\Delta T\left(\pi d+m\right)\underset{X}{\int }\mu \left(dx\right)Q^{\pi d+\upsilon -\left(\pi d+m\right)}\left(x,A\right)∥\\ \le {\displaystyle \sum _{m=0}^{\upsilon -1}}\underset{\pi \to \infty }{lim}\Delta T\left(\pi d+m\right)\underset{\pi \to \infty }{lim}∥\underset{X}{\int }\mu \left(dx\right)Q^{\pi d+\upsilon -\left(\pi d+m\right)}\left(x,A\right)∥=0.\end{array}$$

(47)

Hence, from $\left(46\right)$ and $\left(47\right)$ we get that

$$\begin{array}{rr}\underset{t\to \infty }{lim}U\left(t\right)& =\underset{\pi \to \infty }{lim}{\displaystyle \sum _{m=0}^{d-1}}{\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right)\underset{X}{\int }\mu \left(dx\right)Q^{\pi d+\upsilon -\left(\tau d+m\right)}\left(x,A\right)\\ & =\underset{\pi \to \infty }{lim}{\displaystyle \sum _{m=0}^{\upsilon }}{\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right)\underset{X}{\int }\mu \left(dx\right)Q^{\left(\pi -\tau \right)d+\upsilon -m}\left(x,A\right)\\ & +\underset{\pi \to \infty }{lim}{\displaystyle \sum _{m=\upsilon +1}^{d-1}}{\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right)\underset{X}{\int }\mu \left(dx\right)Q^{\left(\pi -\tau -1\right)d+\left(d+\upsilon -m\right)}\left(x,A\right).\end{array}$$

(48)

It is apparent that since $\Delta T\left(t\right)\ge 0$

$${\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right)\le {\displaystyle \sum _{t=0}^{\pi d+r}}\Delta T\left(t\right)=T\left(\pi d+r\right)-T\left(0\right),$$

and thus the series

$${\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right),$$

is bounded by $T-T\left(0\right)$, its elements are non-negative and thus it converges; so let

$$w_m=\frac{{lim}_{\pi \to \infty }{\displaystyle {\displaystyle \sum _{\tau =0}^{\pi -1}}}\Delta T\left(\tau d+m\right)}{{lim}_{t\to \infty }{\displaystyle {\displaystyle \sum _{\tau =0}^t}}\Delta T\left(t\right)}.$$

(49)

Now, let us define by

$$\underset{\pi \to \infty }{lim}{U}_1\left(t\right)={\displaystyle \sum _{m=0}^{\upsilon }}\underset{\pi \to \infty }{lim}{\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right)\underset{X}{\int }\mu \left(dx\right)Q^{\left(\pi -\tau \right)d+\upsilon -m}\left(x,A\right).$$

(50)

Now from $\left(49\right)$ we get that

$$\underset{\pi \to \infty }{lim}{\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right)=w_m\underset{t\to \infty }{lim}{\displaystyle \sum _{\tau =0}^t}\Delta T\left(t\right)=w_m\left(T-T\left(0\right)\right).$$

(51)

Therefore $\left(50\right)$ becomes

$$\underset{\pi \to \infty }{lim}{U}_1\left(t\right)=\left(T-T\left(0\right)\right){\displaystyle \sum _{m=0}^{\upsilon }}{w}_m$$

(52)

$$\begin{array}{c}\underset{\pi \to \infty }{lim}{\displaystyle \sum _{\tau =0}^{\pi -1}}\frac{\Delta T\left(\tau d+m\right)}{{\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right)}\underset{X}{\int }\mu \left(dx\right)Q^{\left(\pi -\tau \right)d+\upsilon -m}\left(x,A\right)\times \hfill \\ \underset{\pi \to \infty }{lim}{\displaystyle \sum _{\tau =0}^{\pi -1}}\frac{\Delta T\left(\tau d+m\right)}{{\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right)}\underset{X}{\int }\mu \left(dx\right)Q^{\left(\pi -\tau \right)d+\upsilon -m}\left(x,A\right).\hfill \end{array}$$

We now provide the following useful Lemma:

Lemma 1.

If $a_n/{\displaystyle {\displaystyle \sum _{k=0}^n}}{a}_k\to 0$ as $n\to \infty$ and $\left\{a_n\right\}$ non-negative, and $\left\{b_n\right\}\to b$ then

$$\underset{n\to \infty }{lim}\frac{b_0{a}_n+b_1{a}_{n-1}+\dots +b_n{a}_0}{{\displaystyle \sum _{k=0}^n}{a}_k}=b.$$

(53)

From Lemma 1, $\left(52\right)$ and the result in $\left(45\right)$ where we may replace $\lambda \left(dx\right)$ by $\mu \left(dx\right)$ it is not difficult to check that the following holds

$$\begin{array}{c}\underset{\pi \to \infty }{lim}\Vert U_1\left(t\right)-\left(T-T\left(0\right)\right){\displaystyle \sum _{m=0}^{\upsilon }}{\displaystyle \sum _{i=0}^{d-\left(\upsilon -m\right)-1}}{w}_m\mu \left(D_i\right){\pi }_{i+\left(\upsilon -m\right)}\left(A\cap D_{i+\left(\upsilon -m\right)}\right)\\ -\left(T-T\left(0\right)\right){\displaystyle \sum _{m=0}^{\upsilon }}{\displaystyle \sum _{\begin{array}{c}i=d-\left(\upsilon -m\right)\\ i\le d-1\end{array}}^{d-1}}{w}_m\mu \left(D_i\right){\pi }_{i-d+\left(\upsilon -m\right)}\left(A\cap D_{i-d+\left(\upsilon -m\right)}\right)\Vert =0.\end{array}$$

(54)

Define by

$$\underset{t\to \infty }{lim}{U}_2\left(t\right)=\underset{\pi \to \infty }{lim}{\displaystyle \sum _{m=\upsilon +1}^{d-1}}{\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right)\underset{X}{\int }\mu \left(dx\right)Q^{\left(\pi -\tau -1\right)d+\left(d+\upsilon -m\right)}\left(x,A\right).$$

(55)

Then similarly we could prove that

$$\begin{array}{c}\underset{\pi \to \infty }{lim}\Vert U_2\left(t\right)-(T-T\left(0\right))\times \\ {\displaystyle \sum _{m=\upsilon +1}^{d-1}}{\displaystyle \sum _{i=0}^{d-\left(d+\upsilon -m\right)-1}}{w}_m\mu \left(D_i\right){\pi }_{i+d+\left(\upsilon -m\right)}\left(A\cap D_{i+d+\left(\upsilon -m\right)}\right)\\ -(T-T\left(0\right)){\displaystyle \sum _{m=\upsilon +1}^{d-1}}{\displaystyle \sum _{\begin{array}{c}i=d-\left(d+m-\upsilon \right)\\ i\le d-1\end{array}}^{d-1}}{w}_m\mu \left(D_i\right)\times \\ {\pi }_{i-d+\left(d+\upsilon -m\right)}\left(A\cap D_{i-d+\left(d+\upsilon -m\right)}\right)\Vert =0.\end{array}$$

(56)

Therefore from $\left(37\right),\left(45\right),\left(48\right),\left(54\right)$ and $\left(56\right)$ for $t=\pi d+\upsilon$ we get that

$$\begin{array}{c}\underset{t\to \infty }{lim}\Vert {\mathbb{E}}_{\mathbb{P}}\left[N\left(x,t,A\right)\right]-T\left(0\right){\displaystyle \sum _{i=0}^{d-\upsilon -1}}\lambda \left(D_i\right){\pi }_{i+\upsilon }\left(A\cap D_{i+\upsilon }\right)\\ -T\left(0\right){\displaystyle \sum _{i=d-\upsilon }^{d-1}}\lambda \left(D_i\right){\pi }_{i+\upsilon -d}\left(A\cap D_{i+\upsilon -d}\right)\\ -\left(T-T\left(0\right)\right){\displaystyle \sum _{m=0}^{\upsilon }}{\displaystyle \sum _{i=0}^{d-\left(\upsilon -m\right)-1}}{w}_m\mu \left(D_i\right){\pi }_{i+\left(\upsilon -m\right)}\left(A\cap D_{i+\left(\upsilon -m\right)}\right)\\ -\left(T-T\left(0\right)\right){\displaystyle \sum _{m=0}^{\upsilon }}{\displaystyle \sum _{\begin{array}{c}i=d-\left(\upsilon -m\right)\\ i\le d-1\end{array}}^{d-1}}{w}_m\mu \left(D_i\right){\pi }_{i-d+\left(\upsilon -m\right)}\left(A\cap D_{i-d+\left(\upsilon -m\right)}\right)\\ -(T-T\left(0\right)){\displaystyle \sum _{m=\upsilon +1}^{d-1}}{\displaystyle \sum _{i=0}^{\left(m-\upsilon \right)-1}}{w}_m\mu \left(D_i\right){\pi }_{i+d+\left(\upsilon -m\right)}\left(A\cap D_{i+d+\left(\upsilon -m\right)}\right)\\ -(T-T\left(0\right)){\displaystyle \sum _{m=\upsilon +1}^{d-1}}{\displaystyle \sum _{\begin{array}{c}i=\left(m-\upsilon \right)\\ i\le d-1\end{array}}^{d-1}}{w}_m\mu \left(D_i\right){\pi }_{i+\left(\upsilon -m\right)}\left(A\cap D_{i+\left(\upsilon -m\right)}\right)\Vert =0,\end{array}$$

(57)

for $\upsilon =0,1,\dots ,d-1$. Hence, we have proved the following Theorem:

Theorem 12.

Let an MSGS and let that the inherent Markov chain of the MSGS be positive Harris and periodic with period d.

OR let it be that the inherent Markov chain of the MSGS is positive recurrent and periodic with period d, and let the sets $\left\{D_0,D_1,\dots ,D_{d-1}\right\}$ with $D_i\in \mathcal{B}\left(X\right)$. Then the expected population structure splits into d converging subsequences, that is,

$$\underset{t\to \infty }{lim}\Vert {\mathbb{E}}_{\mathbb{P}}\left[N\left(x,t,A\right)\right]-T\left(0\right){\displaystyle \sum _{i=0}^{d-\upsilon -1}}\lambda \left(D_i\right){\pi }_{i+\upsilon }\left(A\cap D_{i+\upsilon }\right)$$

$$-T\left(0\right){\displaystyle \sum _{i=d-\upsilon }^{d-1}}\lambda \left(D_i\right){\pi }_{i+\upsilon -d}\left(A\cap D_{i+\upsilon -d}\right)$$

$$-\left(T-T\left(0\right)\right){\displaystyle \sum _{m=0}^{\upsilon }}{\displaystyle \sum _{i=0}^{d-\left(\upsilon -m\right)-1}}{w}_m\mu \left(D_i\right){\pi }_{i+\left(\upsilon -m\right)}\left(A\cap D_{i+\left(\upsilon -m\right)}\right)$$

$$-\left(T-T\left(0\right)\right){\displaystyle \sum _{m=0}^{\upsilon }}{\displaystyle \sum _{\begin{array}{c}i=d-\left(\upsilon -m\right)\\ i\le d-1\end{array}}^{d-1}}{w}_m\mu \left(D_i\right){\pi }_{i-d+\left(\upsilon -m\right)}\left(A\cap D_{i-d+\left(\upsilon -m\right)}\right)$$

$$-(T-T\left(0\right)){\displaystyle \sum _{m=\upsilon +1}^{d-1}}{\displaystyle \sum _{i=0}^{\left(m-\upsilon \right)-1}}{w}_m\mu \left(D_i\right){\pi }_{i+d+\left(\upsilon -m\right)}\left(A\cap D_{i+d+\left(\upsilon -m\right)}\right)$$

$$-(T-T\left(0\right)){\displaystyle \sum _{m=\upsilon +1}^{d-1}}{\displaystyle \sum _{\begin{array}{c}i=\left(m-\upsilon \right)\\ i\le d-1\end{array}}^{d-1}}{w}_m\mu \left(D_i\right){\pi }_{i+\left(\upsilon -m\right)}\left(A\cap D_{i+\left(\upsilon -m\right)}\right)\Vert =0,$$

for $\upsilon =0,1,2,\dots ,d-1.$

5. Total Variability from the Invariant Measures in the Periodic Case. Coupling Theorems

In this section, we study the total variability from the invariant measures in the periodic case for an MSGS. We show that the total variation in the periodic case is finite. This is also known as the coupling problem (see [58]). From [34] (p. 332) we get the following:

Theorem 13.

Suppose that Φ is positive Harris and aperiodic chain and assume that the chain has an atom $a\in {\mathcal{B}}^{+}\left(X\right)$. Then for any $\lambda ,\mu$ regular initial distributions

$${\displaystyle \sum _{n=1}^{\infty }}\underset{X}{\int }\underset{X}{\int }\lambda \left(dx\right)\mu \left(dy\right)∥P^n\left(x,.\right)-P^n\left(y,.\right)∥<\infty ,$$

and in the case that Φ is regular, then for any $x,y\in X$

$${\displaystyle \sum _{n=1}^{\infty }}∥P^n\left(x,.\right)-P^n\left(y,.\right)∥<\infty .$$

Theorem 14.

Suppose Φ is a positive Harris and aperiodic chain. For any $\lambda ,\mu$ regular initial distributions

$${\displaystyle \sum _{n=1}^{\infty }}\underset{X}{\int }\underset{X}{\int }\lambda \left(dx\right)\mu \left(dy\right)∥P^n\left(x,.\right)-P^n\left(y,.\right)∥<\infty .$$

Theorem 15.

Suppose that Φ is a positive Harris and aperiodic chain and then let α be an accessible atom. If ${\mathbb{E}}_{\alpha }\left[{\tau }_{\alpha }^2\right]<\infty$, then for any regular initial distribution

$${\displaystyle \sum _{n=1}^{\infty }}∥\lambda P^n-\pi ∥<\infty .$$

Now, if we assume that the inherent Markov chain ${\Phi }_Q$ with kernel Q of the MSGS is positive Harris with period d then by Proposition 1 when restricted to each cyclic set is aperiodic and positive Harris on the d-skeleton, and by Theorems 13–15 it is straightforward to check the following theorem:

Theorem 16.

If the inherent Markov chain ${\Phi }_Q$ with kernel Q of the MSGS is positive Harris with period $d,$ then for the sets $\left\{D_0,D_1,\dots ,D_{d-1}\right\}$ with $D_i\in \mathcal{B}\left(X\right)$ we have that

(a) For any regular initial distributions $\lambda ,\mu$ and for $d>i+\upsilon ,$

$${\displaystyle \sum _{n=1}^{\infty }}\underset{D_i}{\int }\underset{D_i}{\int }\lambda \left(dx\right)\mu \left(dy\right)∥Q^{nd+\upsilon }\left(x,A\cap D_{i+\upsilon }\right)-Q^{nd+\upsilon }\left(y,A\cap D_{i+\upsilon }\right)∥$$

$$<\infty ,x,y\in D_i,\mathit{for}i,\upsilon =0,1,\dots ,d-1.$$

Also, for $d\le i+\upsilon$

$${\displaystyle \sum _{n=1}^{\infty }}\underset{D_i}{\int }\underset{D_i}{\int }\lambda \left(dx\right)\mu \left(dy\right)∥Q^{nd+\upsilon }\left(x,A\cap D_{i+\upsilon -d}\right)-Q^{nd+\upsilon }\left(y,A\cap D_{i+\upsilon -d}\right)∥$$

$$<\infty ,x,y\in D_i,\mathit{for}i,\upsilon =0,1,\dots ,d-1.$$

for $i,\upsilon =0,1,\dots ,d-1.$

(b) If for the accessible atom W we have that ${\mathbb{E}}_w\left[{\tau }_w^2\right]<\infty ,$ then for any regular initial distribution λ we have for $d>i+\upsilon ,$$x\in D_i$ and $i,\upsilon =0,1,\dots ,d-1$

$${\displaystyle \sum _{n=1}^{\infty }}∥\lambda \left(dx\right)Q^{nd+\upsilon }\left(x,A\cap D_{i+\upsilon }\right)-{\pi }_{i+\upsilon }\left(A\cap D_{i+\upsilon }\right)∥<\infty ,$$

and for $d\le i+\upsilon ,$$x\in D_i$ and $i,\upsilon =0,1,\dots ,d-1$

$${\displaystyle \sum _{n=1}^{\infty }}∥\lambda \left(dx\right)Q^{nd+\upsilon }\left(x,A\cap D_{i+\upsilon -d}\right)-{\pi }_{i+\upsilon -d}\left(A\cap D_{i+\upsilon -d}\right)∥<\infty .$$

We will now prove the following coupling theorem for an MSGS with inherent Markov chain ${\Phi }_Q$, which is assumed to be positive Harris and periodic.

Theorem 17.

Assume that the inherent Markov chain ${\Phi }_Q$ of a Markov system that lives in $\left(X,\mathcal{B}\left(X\right)\right)$ with kernel $Q\left(x,.\right)$ is positive Harris with period $d.$ Also, let it be that

$$\underset{t\to \infty }{lim}T\left(t\right)=T,\Delta T\left(t\right)\ge 0\mathit{for}\mathit{every}t\in {\mathbb{Z}}^{+}.$$

Then for any regular distributions ${\lambda }_1,{\mu }_1,{\lambda }_2,{\mu }_2$ we have

$${\displaystyle \sum _{t=1}^{\infty }}∥{\mathbb{E}}_{P}N\left(x,t,A\right)-{\mathbb{E}}_P\left[N\left(y,t,A\right)\right]∥<\infty .$$

Proof. 

We have that

$$\begin{array}{c}{\displaystyle \sum _{t=1}^{\infty }}∥{\mathbb{E}}_{P}N\left(x,t,A\right)-{\mathbb{E}}_P\left[N\left(y,t,A\right)\right]∥={\displaystyle \sum _{t=1}^{\infty }}\Vert T\left(0\right)\underset{X}{\int }{\lambda }_1\left(dx\right)Q^t\left(x,A\right)+\\ {\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right)\underset{X}{\int }{\mu }_1\left(dx\right)Q^{t-n}\left(x,A\right)-T\left(0\right)\underset{X}{\int }{\lambda }_2\left(dy\right)Q^t\left(y,A\right)-\\ -{\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right)\underset{X}{\int }{\mu }_2\left(dy\right)Q^{t-n}\left(y,A\right)\Vert \\ \le {\displaystyle \sum _{t=1}^{\infty }}T\left(0\right)\Vert \underset{X}{\int }{\lambda }_1\left(dx\right)Q^t\left(x,A\right)-\underset{X}{\int }{\lambda }_2\left(dy\right)Q^t\left(y,A\right)]\Vert +\\ {\displaystyle \sum _{t=1}^{\infty }}{\displaystyle \sum _{n=1}^{t-1}}\Delta T\left(n\right)\Vert \underset{X}{\int }{\mu }_1\left(dx\right)Q^{t-n}\left(x,A\right)-\underset{X}{\int }{\mu }_2\left(dy\right)Q^{t-n}\left(y,A\right)\Vert =U_{\lambda }\left(t\right)+U_{\mu }\left(t\right)\end{array}$$

(58)

With no loss of generality assume that $t=\pi d+\upsilon$. Denoted by

$${\lambda }_{1i}\left(dx\right)=\frac{{\lambda }_1\left(dx\right)}{{\lambda }_1\left(D_i\right)}\mathrm{and}{\lambda }_{2i}\left(dx\right)=\frac{{\lambda }_2\left(dx\right)}{{\lambda }_2\left(D_i\right)}.$$

(59)

From $\left(58\right)$ and $\left(59\right)$ we have that

$$U_{\lambda }\left(t\right)\le {\displaystyle \sum _{\pi =0}^{\infty }}{\displaystyle \sum _{i=0}^{d-1}}T\left(0\right)\Vert {\lambda }_1\left(D_i\right)\underset{D_i}{\int }{\lambda }_{1i}\left(dx\right)Q^{\pi d+\upsilon }\left(x,A\right)-{\lambda }_2\left(D_i\right)\underset{D_i}{\int }{\lambda }_{2i}\left(dy\right)Q^{\pi d+\upsilon }\left(y,A\right)\Vert .$$

(60)

We have that

$${\displaystyle \sum _{i=0}^{d-1}}{\lambda }_1\left(D_i\right)=1\mathrm{and}{\displaystyle \sum _{i=0}^{d-1}}{\lambda }_2\left(D_i\right)=1.$$

(61)

From $\left(61\right)$ it is apparent that with no loss of generality we may assume that

$${\lambda }_1\left(D_i\right)\le {\lambda }_2\left(D_i\right)\mathrm{for}i=0,1,\dots j\mathrm{and}{\lambda }_1\left(D_i\right)\ge {\lambda }_2\left(D_i\right)\mathrm{for}j=j+1,\dots ,d-1.$$

(62)

Also, without loss of generality again we may assume that $d>j+\upsilon$, then from $\left(60\right)$ and $\left(62\right)$

$$\begin{array}{c}{U}_{\lambda }\left(t\right)\le {\displaystyle \sum _{\pi =0}^{\infty }}{\displaystyle \sum _{i=0}^j}T\left(0\right){\lambda }_2\left(D_i\right)\Vert \underset{D_i}{\int }{\lambda }_{1i}\left(dx\right)Q^{\pi d+\upsilon }\left(x,A\cap D_{i+\upsilon }\right)\\ -\underset{D_i}{\int }{\lambda }_{2i}\left(dy\right)Q^{\pi d+\upsilon }\left(y,A\cap D_{i+\upsilon }\right)\Vert \\ +{\displaystyle \sum _{\pi =0}^{\infty }}{\displaystyle \sum _{i=j+1}^{d-\upsilon -1}}T\left(0\right){\lambda }_1\left(D_i\right)\Vert \underset{D_i}{\int }{\lambda }_{1i}\left(dx\right)Q^{\pi d+\upsilon }\left(x,A\cap D_{i+\upsilon }\right)\\ -\underset{D_i}{\int }{\lambda }_{2i}\left(dy\right)Q^{\pi d+\upsilon }\left(y,A\cap D_{i+\upsilon }\right)\Vert \\ +{\displaystyle \sum _{\pi =0}^{\infty }}{\displaystyle \sum _{i=d-\upsilon }^{d-1}}T\left(0\right){\lambda }_1\left(D_i\right)\Vert \underset{D_i}{\int }{\lambda }_{1i}\left(dx\right)Q^{\pi d+\upsilon }\left(x,A\cap D_{i+\upsilon -d}\right)\\ -\underset{D_i}{\int }{\lambda }_{2i}\left(dy\right)Q^{\pi d+\upsilon }\left(x,A\cap D_{i+\upsilon -d}\right)\Vert =U_{\lambda }^{\left(1\right)}\left(t\right)+U_{\lambda }^{\left(2\right)}\left(t\right)+U_{\lambda }^{\left(3\right)}\left(t\right).\end{array}$$

(63)

Now, it is not difficult to check that

$$\underset{D_i}{\int }{\lambda }_{1i}\left(dx\right)Q^{\pi d+\upsilon }\left(x,A\cap D_{i+\upsilon }\right)=\underset{D_i}{\int }\underset{D_i}{\int }{\lambda }_{1i}\left(dx\right){\lambda }_{2i}\left(dy\right)Q^{\pi d+\upsilon }\left(x,A\cap D_{i+\upsilon }\right),$$

(64)

and

$$\underset{D_i}{\int }{\lambda }_{2i}\left(dy\right)Q^{\pi d+\upsilon }\left(y,A\cap D_{i+\upsilon }\right)=\underset{D_i}{\int }\underset{D_i}{\int }{\lambda }_{1i}\left(dx\right){\lambda }_{2i}\left(dy\right)Q^{\pi d+\upsilon }\left(y,A\cap D_{i+\upsilon }\right).$$

(65)

Therefore from (63)–(65) we get that

$$\begin{array}{c}{U}_{\lambda }^{\left(1\right)}\left(t\right)\le {\displaystyle \sum _{i=0}^j}T\left(0\right){\lambda }_2\left(D_i\right){\displaystyle \sum _{\pi =0}^{\infty }}\underset{D_i}{\int }\underset{D_i}{\int }{\lambda }_{1i}\left(dx\right){\lambda }_{2i}\left(dy\right)\\ \times \Vert Q^{\pi d+\upsilon }\left(x,A\cap D_{i+\upsilon }\right)-Q^{\pi d+\upsilon }\left(y,A\cap D_{i+\upsilon }\right)\Vert .\end{array}$$

(66)

From Theorem 16 and $\left(66\right)$ we get that

$$U_{\lambda }^{\left(1\right)}\left(t\right)\le {\displaystyle \sum _{i=0}^j}T\left(0\right){\lambda }_2\left(D_i\right)K_1<\infty ,$$

(67)

In a similar way we get that $U_{\lambda }^{\left(2\right)}\left(t\right)<\infty$ and $U_{\lambda }^{\left(3\right)}\left(t\right)<\infty$ and consequently from $\left(63\right)$ we get that

$$U_{\lambda }\left(t\right)<\infty .$$

(68)

The case for $d\le j+\upsilon$ is proved in an analogous way.

We have that

$$\begin{array}{c}{U}_{\mu }\left(t\right)={\displaystyle \sum _{t=1}^{\infty }}[{\displaystyle \sum _{n=0}^{t-1}}\Delta T\left(n\right)\underset{X}{\int }{\mu }_1\left(dx\right)Q^{t-n}\left(x,A\right)\\ -{\displaystyle \sum _{n=0}^{t-1}}\Delta T\left(n\right)\underset{X}{\int }{\mu }_2\left(dx\right)Q^{t-n}\left(x,A\right)]\\ ={\displaystyle \sum _{\pi =0}^{\infty }}{\displaystyle \sum _{\upsilon =0}^{d-1}}\left[{\displaystyle \sum _{n=0}^{\pi d+\upsilon -1}}\Delta T\left(n\right)\left\{\underset{X}{\int }{\mu }_1\left(dx\right)Q^{t-n}\left(x,A\right)-\underset{X}{\int }{\mu }_2\left(dy\right)Q^{t-n}\left(y,A\right)\right\}\right]\\ ={\displaystyle \sum _{\pi =0}^{\infty }}{\displaystyle \sum _{\upsilon =0}^{d-1}}[{\displaystyle \sum _{m=0}^{d-1}}{\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right)\times \\ \{\underset{X}{\int }{\mu }_1\left(dx\right)Q^{\left(\pi d+\upsilon \right)-\left(\tau d+m\right)}\left(x,A\right)-\underset{X}{\int }{\mu }_2\left(dy\right)Q^{\left(\pi d+\upsilon \right)-\left(\tau d+m\right)}\left(y,A\right)\}]\\ +{\displaystyle \sum _{\pi =0}^{\infty }}[{\displaystyle \sum _{m=0}^{\upsilon -1}}\Delta T\left(\tau d+m\right)\{\underset{X}{\int }{\mu }_1\left(dx\right)Q^{\left(\pi d+\upsilon \right)-\left(\pi d+m\right)}\left(x,A\right)\\ -\underset{X}{\int }{\mu }_2\left(dy\right)Q^{\left(\pi d+\upsilon \right)-\left(\pi d+m\right)}\left(y,A\right)\left\}\right]=U_{\mu }^{\left(\pi \right)}\left(t\right)+U_{\mu }^{\left(\upsilon \right)}\left(t\right).\end{array}$$

(69)

We start with $U_{\mu }^{\left(\upsilon \right)}\left(t\right)$ and we have that

$$\begin{array}{c}∥U_{\mu }^{\left(\upsilon \right)}\left(t\right)∥\le {\displaystyle \sum _{\pi =0}^{\infty }}[{\displaystyle \sum _{m=0}^{\upsilon -1}}\Delta T\left(\tau d+m\right)\\ \times ∥\underset{X}{\int }{\mu }_1\left(dx\right)Q^{\upsilon -m}\left(x,A\right)-\underset{X}{\int }{\mu }_2\left(dy\right)Q^{\upsilon -m}\left(y,A\right)\left\}\right]∥\\ \le {\displaystyle \sum _{\pi =0}^{\infty }}[{\displaystyle \sum _{m=0}^{\upsilon -1}}\Delta T\left(\tau d+m\right){\displaystyle \sum _{i=0}^{d-1}}\Vert \underset{D_i}{\int }{\mu }_1\left(dx\right)Q^{\upsilon -m}\left(x,A\cap D_{i+\upsilon -m}\right)\\ -\underset{D_i}{\int }{\mu }_2\left(dy\right)Q^{\upsilon -m}\left(y,A\cap D_{i+\upsilon -m}\right)\left\}\right]\Vert .\end{array}$$

(70)

Denote by

$${\mu }_{1i}\left(dx\right)=\frac{{\mu }_1\left(dx\right)}{{\mu }_1\left(D_i\right)}\mathrm{and}{\mu }_{2i}\left(dx\right)=\frac{{\mu }_2\left(dx\right)}{{\mu }_2\left(D_i\right)}.$$

(71)

Then with no loss of generality we may assume that there is an r such that

$${\mu }_1\left(D_i\right)\le {\mu }_2\left(D_i\right)\mathrm{for}i=0,1,\dots ,r\mathrm{and}{\mu }_1\left(D_i\right)>{\mu }_2\left(D_i\right)\mathrm{for}i=r+1,\dots ,d-1.$$

(72)

From (70)–(72) and using Theorem 16 we get that

$$∥U_{\mu }^{\left(\upsilon \right)}\left(t\right)∥\le {\displaystyle \sum _{\pi =0}^{\infty }}[{\displaystyle \sum _{m=0}^{\upsilon -1}}\Delta T\left(\tau d+m\right){\displaystyle \sum _{i=0}^r}{\mu }_2\left(D_i\right)\times$$

$$\begin{array}{c}\Vert \underset{D_i}{\int }\underset{D_i}{\int }{\mu }_{1i}\left(dx\right){\mu }_{2i}\left(dy\right)Q^{\upsilon -m}\left(x,A\cap D_{i+\upsilon -m}\right)\\ -\underset{D_i}{\int }\underset{D_i}{\int }{\mu }_{1i}\left(dx\right){\mu }_{2i}\left(dy\right)Q^{\upsilon -m}\left(y,A\cap D_{i+\upsilon -m}\right)\left\}\right]\Vert \\ +{\displaystyle \sum _{\pi =0}^{\infty }}[{\displaystyle \sum _{m=0}^{\upsilon -1}}\Delta T\left(\tau d+m\right){\displaystyle \sum _{j=r+1}^{d-1}}{\mu }_1\left(D_i\right)\times \\ \Vert \underset{D_i}{\int }\underset{D_i}{\int }{\mu }_{1i}\left(dx\right){\mu }_{2i}\left(dy\right)Q^{\upsilon -m}\left(x,A\cap D_{i+\upsilon -m}\right)\\ -\underset{D_i}{\int }\underset{D_i}{\int }{\mu }_{1i}\left(dx\right){\mu }_{2i}\left(dy\right)Q^{\upsilon -m}\left(y,A\cap D_{i+\upsilon -m}\right)\left\}\right]\Vert \\ \le {\displaystyle \sum _{\pi =0}^{\infty }}{\displaystyle \sum _{m=0}^{\upsilon -1}}\Delta T\left(\tau d+m\right)[{\displaystyle \sum _{j=0}^r}{\mu }_2\left(D_i\right)K_1\left(\upsilon -m\right)+{\displaystyle \sum _{j=r+1}^{d-1}}{\mu }_1\left(D_i\right)K_2\left(\upsilon -m\right)]<\infty .\end{array}$$

(73)

We now move to $U_{\mu }^{\left(\pi \right)}\left(t\right)$ and we have that

$$\begin{array}{c}∥U_{\mu }^{\left(\pi \right)}\left(t\right)∥\le {\displaystyle \sum _{\pi =0}^{\infty }}{\displaystyle \sum _{\upsilon =0}^{d-1}}[{\displaystyle \sum _{m=0}^{\upsilon }}{\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right)\Vert \underset{X}{\int }{\mu }_1\left(dx\right)Q^{\left(\pi -\tau \right)d+\upsilon -m}\left(x,A\right)\\ -\underset{X}{\int }{\mu }_2\left(dy\right)Q^{\left(\pi -\tau \right)d+\upsilon -m}\left(y,A\right)\Vert \\ {\displaystyle \sum _{\pi =0}^{\infty }}{\displaystyle \sum _{\upsilon =0}^{d-1}}[{\displaystyle \sum _{m=\upsilon +1}^{d-1}}{\displaystyle \sum _{\tau =0}^{\pi -1}}\Delta T\left(\tau d+m\right)\Vert \underset{X}{\int }{\mu }_1\left(dx\right)Q^{\left(\pi -\tau -1\right)d+d+\upsilon -m}\left(x,A\right)\\ -\underset{X}{\int }{\mu }_2\left(dy\right)Q^{\left(\pi -\tau -1\right)d+d+\upsilon -m}\left(y,A\right)\Vert =∥U_{\mu }^{\left(\pi ,1\right)}\left(t\right)∥+∥U_{\mu }^{\left(\pi ,2\right)}\left(t\right)∥.\end{array}$$

(74)

We start first with $U_{\mu }^{\left(\pi ,1\right)}\left(t\right)$ and we get that

$$\begin{array}{c}∥U_{\mu }^{\left(\pi ,1\right)}\left(t\right)∥\le {\displaystyle \sum _{\tau =0}^{\infty }}{\displaystyle \sum _{\upsilon =0}^{d-1}}{\displaystyle \sum _{m=0}^{\upsilon }}\Delta T\left(\tau d+m\right){\displaystyle \sum _{\pi -\tau =1}^{\infty }}{\displaystyle \sum _{i=0}^{d-1}}\Vert \underset{D_i}{\int }{\mu }_1\left(dx\right)Q^{\left(\pi -\tau \right)d+\upsilon -m}\left(x,A\right)\\ -\underset{D_i}{\int }{\mu }_2\left(dy\right)Q^{\left(\pi -\tau \right)d+\upsilon -m}\left(y,A\right)\Vert \\ \le {\displaystyle \sum _{\tau =0}^{\infty }}{\displaystyle \sum _{\upsilon =0}^{d-1}}{\displaystyle \sum _{m=0}^{\upsilon }}\Delta T\left(\tau d+m\right){\displaystyle \sum _{\pi -\tau =1}^{\infty }}{\displaystyle \sum _{i=0}^r}{\mu }_2\left(D_i\right)\times \\ \Vert \underset{D_i}{\int }\underset{D_i}{\int }{\mu }_{1i}\left(dx\right){\mu }_{2i}\left(dy\right)Q^{\left(\pi -\tau \right)d+\upsilon -m}\left(x,A\right)\\ -\underset{D_i}{\int }\underset{D_i}{\int }{\mu }_{1i}\left(dx\right){\mu }_{2i}\left(dy\right)Q^{\left(\pi -\tau \right)d+\upsilon -m}\left(x,A\right)\Vert \\ +{\displaystyle \sum _{\tau =0}^{\infty }}{\displaystyle \sum _{\upsilon =0}^{d-1}}{\displaystyle \sum _{m=0}^{\upsilon }}\Delta T\left(\tau d+m\right){\displaystyle \sum _{\pi -\tau =1}^{\infty }}{\displaystyle \sum _{i=r+1}^{d-1}}{\mu }_1\left(D_i\right)\times \\ \Vert \underset{D_i}{\int }\underset{D_i}{\int }{\mu }_{1i}\left(dx\right){\mu }_{2i}\left(dy\right)Q^{\left(\pi -\tau \right)d+\upsilon -m}\left(x,A\right)\\ -\underset{D_i}{\int }\underset{D_i}{\int }{\mu }_{1i}\left(dx\right){\mu }_{2i}\left(dy\right)Q^{\left(\pi -\tau \right)d+\upsilon -m}\left(x,A\right)\Vert =\Vert U_{\mu }^{\left(\pi ,1,1\right)}\left(t\right)\Vert +\Vert U_{\mu }^{\left(\pi ,1,2\right)}\left(t\right)\Vert .\end{array}$$

(75)

Now, using Theorem 17 and from $\left(75\right)$ we have that

$$\Vert U_{\mu }^{\left(\pi ,1,1\right)}\left(t\right)\Vert \le {\displaystyle \sum _{\tau =0}^{\infty }}{\displaystyle \sum _{\upsilon =0}^{d-1}}{\displaystyle \sum _{m=0}^{\upsilon }}\Delta T\left(\tau d+m\right){\displaystyle \sum _{\pi -\tau =1}^{\infty }}{\displaystyle \sum _{i=0}^r}{\mu }_2\left(D_i\right)C\left(\upsilon -m\right)<\infty .$$

(76)

Similarly, we get also that

$$\Vert U_{\mu }^{\left(\pi ,1,2\right)}\left(t\right)\Vert <\infty \mathrm{and}\Vert U_{\mu }^{\left(\pi ,2\right)}\left(t\right)\Vert <\infty .$$

(77)

From $\left(68\right)$, $\left(69\right)$, $\left(76\right)$, and $\left(77\right)$, we conclude the proof of the Theorem. □

Similarly, and with the use of Theorem 16.b we could prove the following Theorem

Theorem 18.

Assume that the inherent Markov chain ${\Phi }_Q$ of a Markov system that lives in $\left(X,\mathcal{B}\left(X\right)\right)$ with kernel $Q\left(x,.\right)$ is positive Harris with period d. Assume, in addition, that for the accessible atom W, we have ${\mathbb{E}}_{\omega }\left[{\tau }_{\omega }^2\right]<\infty$ and that

$$\underset{t\to \infty }{lim}T\left(t\right)=T,\Delta T\left(t\right)\ge 0\mathit{for}\mathit{every}t\in {\mathbb{Z}}^{+}.$$

Then for any regular distribution λ we have

$${\displaystyle \sum _{t=1}^{\infty }}\Vert \mathbb{E}\left[N\left(x,t,A\right)\right]-T\left(0\right){\displaystyle \sum _{\upsilon =0}^{d-1}}{\displaystyle \sum _{i=0}^{d-\upsilon -1}}\lambda \left(D_i\right){\pi }_{i+\upsilon }\left(A\cap D_{i+\upsilon }\right)$$

$$-T\left(0\right){\displaystyle \sum _{\upsilon =0}^{d-1}}{\displaystyle \sum _{i=d-\upsilon }^{d-1}}\lambda \left(D_i\right){\pi }_{i+\upsilon -d}\left(A\cap D_{i+\upsilon -d}\right)$$

$$-\left(T-T\left(0\right)\right){\displaystyle \sum _{\upsilon =0}^{d-1}}{\displaystyle \sum _{m=0}^{\upsilon }}{\displaystyle \sum _{i=0}^{d-\left(\upsilon -m\right)-1}}{w}_m\mu \left(D_i\right){\pi }_{i+\left(\upsilon -m\right)}\left(A\cap D_{i+\left(\upsilon -m\right)}\right)$$

$$-\left(T-T\left(0\right)\right){\displaystyle \sum _{\upsilon =0}^{d-1}}{\displaystyle \sum _{m=0}^{\upsilon }}{\displaystyle \sum _{\begin{array}{c}i=d-\left(\upsilon -m\right)\\ i\le d-1\end{array}}^{d-1}}{w}_m\mu \left(D_i\right){\pi }_{i-d+\left(\upsilon -m\right)}\left(A\cap D_{i-d+\left(\upsilon -m\right)}\right)$$

$$-(T-T\left(0\right)){\displaystyle \sum _{\upsilon =0}^{d-1}}{\displaystyle \sum _{m=\upsilon +1}^{d-1}}{\displaystyle \sum _{i=0}^{\left(m-\upsilon \right)-1}}{w}_m\mu \left(D_i\right){\pi }_{i+d+\left(\upsilon -m\right)}\left(A\cap D_{i+d+\left(\upsilon -m\right)}\right)$$

$$-(T-T\left(0\right)){\displaystyle \sum _{\upsilon =0}^{d-1}}{\displaystyle \sum _{m=\upsilon +1}^{d-1}}{\displaystyle \sum _{\begin{array}{c}i=\left(m-\upsilon \right)\\ i\le d-1\end{array}}^{d-1}}{w}_m\mu \left(D_i\right){\pi }_{i+\left(\upsilon -m\right)}\left(A\cap D_{i+\left(\upsilon -m\right)}\right)\Vert =0.$$

6. Conclusions and Further Research

We studied a population that lives in a general state space that at every instant has leavers and new entrants, which evolves under the Markov property. We proved that under certain conditions the expected population structure converges with geometrical rate to an invariant population structure with loss of memory. We also provided the analogous result for the V-uniformly ergodicity of MSGS. Moreover, under the assumption that the inherent Markov chain is periodic with period d, we proved that the sequence of the expected population structure splits into d convergent subsequences and provided their limits in closed forms. Finally, we proved that the total variability from the invariant measure in the periodic case is finite. We concluded the novel results by proving two coupling theorems under the assumption that the inherent Markov chain is positive Harris with period d using different additional conditions in each case. The above theoretical results have important applications both in the classical areas where Markov chains in general spaces have already been applied but especially to the important area of the health systems that, nowadays, are in crisis all over the world.

The results in the present paper together with the ones introduced in [33] provide the nucleus for a new path for extensive research in the area. Immediate further research of interest is to establish Laws of Large numbers for MSGS as a possible extension of the Laws of Large numbers for populations that live in countable spaces [59].