Non-Homogeneous Markov Set Systems

Abstract

A more realistic way to describe a model is the use of intervals which contain the required values of the parameters. In practice we estimate the parameters from a set of data and it is natural that they will be in confidence intervals. In the present study, we study Non-Homogeneous Markov Systems (NHMS) processes for which the required basic parameters are in intervals. We call such processes Non-Homogeneous Markov Set Systems (NHMSS). First we study the set of the relative expected population structure of memberships and we prove that under certain conditions of convexity of the intervals of the parameters the set is compact and convex. Next, we establish that if the NHMSS starts with two different initial distributions sets and allocation probability sets under certain conditions, asymptotically the two expected relative population structures coincide geometrically fast. We continue proving a series of theorems on the asymptotic behavior of the expected relative population structure of a NHMSS and the properties of their limit set. Finally, we present an application for geriatric and stroke patients in a hospital and through it we solve problems that surface in an application.

1. Introduction

The class of stochastic processes called Non-Homogeneous Markov Systems (NHMS) was first defined in [1] The class of NHMS provided a general framework for many applied probabilities models used to model populations of a wide diversity of entities. The primary motive was to provide a general framework for a wide class of stochastic models in social processes ([2]). They also include as special cases non-homogeneous Markov chain models in manpower systems such as [3,4,5]. The literature on NHMS has flourished since then to a large extent and presently exist a large volume of theoretical results as well a variety of applications. In Section 2 of the present we provide a definition and a concise description of a NHMS. As we will discuss in Section 2, it is important for the reader to have in mind that actually the well-known non-homogeneous Markov chain is a special case of a NHMS.

In many stochastic processes and so naturally in Markov chains and specifically in non-homogenous Markov chains and NHMS, the values of the various parameters are assumed to be exact while in practice these are estimated from the data. Therefore, actually the values of the parameters is more realistic to be viewed as being contained in intervals with the desired probability confidence. This approach has been used in systems of linear equations and in this case the solutions are given as the set of all possible solutions. Two books have been written on this topic by [6,7]. For the analogous problem for differential equations a book was written by [8]. For homogeneous Markov chains with this approach a book was written see [9].

In Section 3 of the present we will now add some additional assumptions on a NHMS in our way to define a non-homogeneous Markov set system (NHMSS). In this way now a NHMSS will be a NHMS whose basic parameters will be assumed to be in compact convex intervals.

The NHMSS is a stochastic system which has a population of members which increases at every point in time. I addition the initial members need not to be the same entities at different time points since there is wastage from the system. The members of the population move among the different states, exit from the system (population) and new members are coming into the population (system) as replacements or to expand the system. In the case of non-homogeneous Markov set chains we have only one particle in the population, which never leaves the system and no procedure to replace this particle exists. Mathematically now the NHMSS has more elements in the one step in time equation with more parameters introduced in the stochastic difference equation. As the equation is applied recursively we end up with series of components which interact together with more parameters being in an interval. Hence, the problems to be solved are a lot harder, and new strategies and tools must be used, than the simple case of the Markov set chain. The introduction of the concept of a membership is crucial in dealing with the different individual members as time progress. The tool of Minkowski sum of vectors and its properties for convex combination of compact sets will play a vital role which was not needed in the case of Markov set chains. One of the hard problems which we encounter which does not exist in the Markov set chains is finding the range of infinite series. The Hausdorff metric for compact sets and the coefficient of ergodicity together with properties of appropriate norms introduced and the manipulation of infinite series will help to provide the following:

In Section 4, we establish in the form of a theorem, using the Minkowski sum of two sets, under which conditions in a NHMSS the set of all possible expected relative population structures at a certain point in time is a convex set. Also, we establish a Theorem where we provide conditions under which the set of all expected relative population structures at a certain point in time is a convex polygon.

In Section 5 we study the asymptotic behavior of an NHMSS, a problem that has been of central importance for homogeneous Markov chains, non-homogeneous Markov chains, NHMS and homogeneous Markov set chains. In Theorem 4, with the use of the coefficient of ergodicity and the Hausdorff metric we prove the following: Let that in an NHMSS the sets of initial structures are different but compact and convex; also, the sets of allocation probabilities of the memberships are different but convex and compact; the inherent non-homogeneous Markov set chain is common; then the Hausdorff metric of the two different sets of all possible expected relative structures asymptotically goes to zero geometrically fast. This is equivalent with concluding that the two sets asymptotically coincide geometrically fast. In Theorem 5 we prove that in an NHMSS if the total population of memberships converges in a finite number geometrically fast, and the sets of initial structures and allocation probabilities of memberships are compact and convex, then the set of all possible expected relative population structure converges to a limit set geometrically fast. These two theorems have important consequences for a NHMS process also. The first is Theorem 6 which relaxes important assumptions of the basic asymptotic theorem for NHMS which is provided as Theorem 7. The second labeled as Theorem 8 answers a novel question for NHMS, i.e., provides conditions under which two different NHMS, with the same number of states and population, but different initial states and different allocation probabilities of memberships if they have the same transition probabilities sequence of memberships, they converge in the same relative population structure geometrically fast.

In Section 6 we study properties of the limit set of expected relative population structures. In Theorem 10 we prove the first property, that under some mild conditions the limit set of the expected relative population structures of an NHMSS remains invariant if any selected transition probability matrix of the inherent non-homogeneous Markov chain from the respective interval is multiplied by it from the right. We also prove that the limit set is the only set with this property if the interval of selection of transition probabilities of the inherent non-homogeneous Markov chain is product scrambling. In Theorem 11 the second property is established, i.e., let two different NHMSS in the sense that they have different sets of selecting initial distributions, different sets of selecting allocation probabilities and different intervals of selecting the transition probabilities of the inherent non-homogeneous Markov chains. What they have in common is that their respective intervals are uniformly scrambling with a common bound and they have the same total population of memberships. We prove that the Hausdorff metric of the limit sets of the expected relative population structures of the two NHMSS is bounded by the multiplication of a function of the Hausdorff metric of the two tight intervals of selection of the stochastic matrices of the inherent non-homogeneous Markov set chains and the bound of their uniform coefficients of ergodicity.

In Section 7 we present a representative application for geriatric and stroke patients in a hospital. Through this application we provide solutions in problems arising in an application by providing respective Lemmas and a general Algorithm with computational geometry procedures which are applicable to any population system.

2. The Non-Homogeneous Markov System

Consider a population which has $T\left(t\right)$ memberships at time t. These memberships could be held by any kind of entities, i.e., human beings, animals, T-cells in a biological entity, fish in an organized area in the sea, cars on a highway etc. We assume that $T\left(t\right)$ is known for every t, for example in a hospital the memberships are the beds for patients and from the management of the hospital’s planning the number of beds are known. Let that the population is stratified into classes which we call states and let that there are a finite number of states, i.e., the state space is $\mathbb{S}=\left\{1,2,\cdots ,k\right\}$. We assume that the evolution of the population is in discrete time, i.e., $t=1,2,\dots$ and we call the vector of random variables $\mathit{N}\left(t\right)=\left[N_1\left(t\right),N_2\left(t\right),\dots ,N_k\left(t\right)\right]$ where $N_i\left(t\right)$ is the number of memberships in state i at time t, the population structure of the NHMS. Define by $\mathit{q}\left(t\right)=\mathit{N}\left(t\right)/T\left(t\right)$ to be the relative population structure. At every time instant $t=1,2,\dots$, we have internal transitions of members among the states in $\mathbb{S}$ with probabilities which we collect in the $k\times k$ matrix $\mathit{P}\left(t\right)$; we have wastage from all the states with probabilities which we collect in the $1\times k$ vector ${\mathit{p}}_{k+1}\left(t\right)$$=\left[p_{1,k+1}\left(t\right),p_{2,k+1}\left(t\right),\dots ,p_{k,k+1}\left(t\right)\right]$; finally, we have recruitment or allocation probabilities of replacements or new entrants in the various states at time t which we collect in the $1\times k$ stochastic vector ${\mathit{p}}_0\left(t\right)=\left[p_{01}\left(t\right),p_{02}\left(t\right),\dots ,p_{0k}\left(t\right)\right]$. We assume that the system is expanding, i.e., $\Delta T\left(t\right)=T\left(t\right)-T\left(t-1\right)\ge 0$. During the time interval $\left(t-1,t\right]$ a member of the system in state i either moves internally to another state j of the system with probability $p_{ij}\left(t\right)$ or leave the system and his membership remains at the exit of the system. New entrants to the system are of two types, those to replace leavers and those needed to be added in the system to meet the target of $T\left(t\right)$ total memberships. The new entrant gets his membership at the entrance and he is being allocated or recruited at state j with probability $p_{0j}\left(t\right)$. Hence, the probability of movement of a membership from state i to state j at time t is $q_{ij}\left(t\right)=p_{ij}\left(t\right)+p_{0j}\left(t\right)p_{i,k+1}\left(t\right)$. We collect these probabilities in the $k\times k$ matrix $\mathit{Q}\left(t\right)=\mathit{P}\left(t\right)+{\mathit{p}}_{k+1}^{\top }\left(t\right){\mathit{p}}_0\left(t\right)$ which apparently is a stochastic matrix. We call the Markov chain defined by the sequence of matrices ${\left\{\mathit{Q}\left(t\right)\right\}}_{t=0}^{\infty }$ the imbedded or inherent non-homogeneous Markov chain of the NHMS.

It is of interest the expected relative population structure. Let $X_t$ be the random variable representing the state that a membership of the system is at time t. Define by

$$\mathit{q}\left(s,t\right)=\left[q_1\left(s,t\right),q_2\left(s,t\right),\dots ,q_2\left(s,t\right)\right],$$

where

$$q_j\left(s,t\right)=\mathbb{P}\left[X_t=j\mid \mathit{q}\left(s\right)\right]\mathrm{for}s\le t,$$

(1)

then from ([10] p. 140) we get that

$$\mathbb{E}\left[\mathit{q}\left(s,t\right)\right]=a\left(t-1\right)\mathbb{E}\left[\mathit{q}\left(t-1\right)\right]\mathit{Q}\left(t\right)+b\left(t-1\right){\mathit{p}}_0\left(t\right),$$

(2)

where

$$a\left(t-1\right)=\frac{T\left(t-1\right)}{T\left(t\right)}\mathrm{and}b\left(t-1\right)=\frac{T\left(t\right)-T\left(t-1\right)}{T\left(t\right)},$$

(3)

from which we get that

$$\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]=\frac{T\left(0\right)}{T\left(t\right)}\mathit{q}\left(0\right)\mathit{Q}\left(0,t\right)+$$

$$\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right){\mathit{p}}_0\left(\tau \right)\mathit{Q}\left(\tau ,t\right),$$

(4)

where $\mathit{Q}\left(s,t\right)=\mathit{Q}\left(s+1\right)\mathit{Q}\left(s+2\right)\dots \mathit{Q}\left(t\right)$ for $\left(s\le t\right)$. We set $\mathit{Q}\left(s,t\right)=\mathit{I}$ the identity matrix for $s\ge t$. Please note that we set $\mathit{q}\left(s,t\right)=\mathbf{0}$ for $s>t$. We call any such process as described above a Non-homogeneous Markov process in discrete time and discrete state space. It is important for the reader to realize that the well-known ordinary Markov chain is a very special case of a NHMS with $T\left(t\right)=1$, ${\mathit{p}}_0\left(t\right)=\mathbf{0}$, ${\mathit{p}}_{k+1}\left(t\right)=0$ and $\mathit{Q}\left(t\right)=\mathit{P}$.

As we mentioned in the Introduction the stochastic process NHMS was first introduced in [1] as a discrete time, discrete state space stochastic process with motives which have their roots in actual applications in manpower systems see for example [1,2,11], and also the review papers [12,13]. Since then, a large literature on theoretical developments on many aspects of a NHMS were published which also included the developments in [14,15] of NHMS’s in a general state space. In [16] there appeared the link between the theory of NHMS and martingale theory. Lately, also another area of large interest has been the Law of Large Numbers in NHMS ([17]) which has its roots as a motive the study of Laws of Large numbers on homogeneous Markov chains by Markov himself. Also, many applications in areas with great diversity have also appeared in the literature. For example we could selectively refer to some of them. Let as start with [18,19,20] which are applications in the evolution of the HIV virus on the T-cells of the human body; population consisting with patients with asthma was studied in [21]; reliability studies were presented in [22]; applications in biomedical research appeared in [23,24]; various applications for human populations [25,26,27,28,29]; interesting application to consumption credit [30] infections of populations [31]; a very interesting application in DNA and web navigation [32]; interesting ecological applications [33]; results in Physical Chemistry [34]. Finally, there are a large number of publications by the research school of Prof McClean in hospital systems which are large manpower systems [35,36,37,38,39].

3. Non-Homogeneous Markov Set System

In Section 2 we defined the NHMS process and we will now define for the first time ever the non-homogeneous Markov set system. So far in the well developed theory of NHMS’s the various perimeters are assumed to be exact while in practice they are naturally estimated by the data. Therefore, it is more realistic to be viewed as being contained in intervals with the desired probability confidence. In summary as we will see bellow a NHMSS is a NHMS for which its parameters are defined in intervals. In addition the study of NHMSS provides a new area of theoretical research with different mathematical tools in many instances than the corresponding theory of NHMS and a potential to be applied in other stochastic processes.

The practical advantages of NHMSS’s are rather apparent since the assumptions on the parameters are less restrictive. The assumption of the parameters being in appropriate intervals absorbs in a way the errors of point estimates which increase their variability. In addition it provides the tools to study NHMS’s whose parameters will be in “desired” intervals which increases considerably the control of the system since we could choose policies of the systems in intervals with desired outcomes for the expected relative population structures or to avoid trouble some situations.

We will start with the definition of an interval for a stochastic vector following [9] who first defined Markov set chains. Denote by $M_n\left(\mathbb{R}\right)$ or simply $M_n$ the set of all $n\times n$ matrices with elements from the field $\mathbb{R}$.

Definition 1.

Let$S{M}_{1n}$the set of all$1\times n$stochastic vectors. Also let λ and μ be non-negative$1\times n$vectors with λ≤μ componentwise. Then define the corresponding interval in$S{M}_{1n}$by

$$\left[\lambda ,\mu \right]=\left\{\mathit{p}\mathbf{:}\mathit{p}\in S{M}_{1,n}with\lambda \le \mathit{p}\le \mu \right\},$$

where λ,μ are chosen such that$\left[\lambda ,\mu \right]\ne ⌀$.

Example 1.

It is sometimes helpful to view mathematics geometrically. Let$S{M}_{1,3}$the set of all$1\times 3$stochastic vectors, then it is easy to see that this is the convex hull of the vectors${\mathit{e}}_1=\left(\begin{array}{ccc}1& 0& 0\end{array}\right)$, ${\mathit{e}}_2=\left(\begin{array}{ccc}0& 1& 0\end{array}\right)$and${\mathit{e}}_3=\left(\begin{array}{ccc}0& 0& 1\end{array}\right)$in${\mathbb{R}}^3$. Now, all the non-negative vectors$\mathit{x}$such that$\lambda \le \mathit{x}$$\le \mu$are within and the surface of a rectangle the coordinates of which are determined by$\lambda ,\mu$. The interval$\left[\lambda ,\mu \right]$will be the intersection of the two above described spaces. We can visualize this more easily if we consider the triangle${\mathit{e}}_1{\mathit{e}}_2{\mathit{e}}_3$in${\mathbb{R}}^2$. Let$\lambda =\left(\begin{array}{ccc}0.1& 0.2& 0.3\end{array}\right)$and$\mu =\left(\begin{array}{ccc}0.5& 0.7& 0.8\end{array}\right)$then the interval$\left[\lambda ,\mu \right]$could be easily designed in the following way. Draw two parallel lines to the line${\mathit{e}}_2{\mathit{e}}_3$at the points${\lambda }_1=0.1$and${\mu }_1=0.5;$also draw two parallel lines to the line${\mathit{e}}_1{\mathit{e}}_3$at the points${\lambda }_2=0.2$and${\mu }_2=0.7$; finally draw two parallel lines to the line${\mathit{e}}_1{\mathit{e}}_2$at the points${\lambda }_3=0.3$and${\mu }_3=0.8$. Then the interval$\left[\lambda ,\mu \right]$is the common area between these lines.

Tight intervals are important in what follows:

Definition 2.

Let$\left[\lambda ,\mu \right]$be an interval, then if

$${\lambda }_i=\underset{\mathit{x}\in \left[\lambda ,\mu \right]}{min}{x}_{i}and{\mu }_i=\underset{\mathit{x}\in \left[\lambda ,\mu \right]}{max}{x}_i,$$

then${\lambda }_i$, ${\mu }_i$are called tight, respectively. If ${\lambda }_i$, ${\mu }_i$are tight for all i, then the interval$\left[\lambda ,\mu \right]$is called tight.

Intervals can be tested for tightness using the following Lemma ([9]). Also, with the use of this Lemma an interval which is not tight, we can tighten it up using an algorithm without actually changing it. That is the new interval, the tightened one will contain the same stochastic vectors.

Lemma 1. 

([9]). Let$\left[\lambda ,\mu \right]$be an interval. Then for each coordinate i

$$\left(i\right){\lambda }_{i}istightifandonlyif{\lambda }_i+\sum _{k\ne i}{\mu }_k\ge 1.$$

$$\left(ii\right){\mu }_{i}istightifandonlyif{\mu }_i+\sum _{k\ne i}{\lambda }_k\ge 1.$$

We now need the following definition of when in a tight interval a vector is called free.

Definition 3.

Let$\left[\lambda ,\mu \right]$be a tight interval and$\mathit{p}\in \left[\lambda ,\mu \right]$. Then if${\lambda }_i<p_i<{\mu }_i$for some coordinate i, then the coordinate$p_i$in$\mathit{p}$is called free.

Tight intervals and convex sets are well linked and play an important role in the preservation of many properties. In this respect, the following Lemma is very useful.

Lemma 2.

 ([9]). Let$\left[\lambda ,\mu \right]$be a tight an interval. Then$\left[\lambda ,\mu \right]$is a convex polytope. A vector$\mathit{p}\in \left[\lambda ,\mu \right]$is a vertex of$\left[\lambda ,\mu \right]$if and only if$\mathit{p}$has at most one free component.

We will now extend the definition of an interval of a vector to an interval of a matrix and to a tight interval of a matrix.

Definition 4.

Let$S{M}_n$the set of all$1\times n$stochastic matrices. Also let Λ and $\mathit{M}$ be non-negative $n\times n$ matrices with $\Lambda \le \mathit{M}$ componentwise. Then define the corresponding interval in $S{M}_n$ by

$$\left[\Lambda ,\mathit{M}\right]=\left\{\mathit{P}\mathbf{:}\mathit{P}\in S{M}_{n}with\Lambda \le \mathit{P}\le \mathit{M}\right\},$$

where $\Lambda ,\mathit{M}$ are chosen such that $\left[\Lambda ,\mathit{M}\right]\ne ⌀$.

We now proceed to define a tight interval of matrices:

Definition 5.

Let$\left[\Lambda ,\mathit{M}\right]$be an interval of matrices. If

$${\lambda }_{ij}=\underset{\mathit{P}\in \left[\Lambda ,\mathit{M}\right]}{min}{p}_{ij}and{\mu }_{ij}=\underset{\mathit{P}\in \left[\Lambda ,\mathit{M}\right]}{max}{p}_{ij},$$

for all i and j, then$\left[\Lambda ,\mathit{M}\right]$is called tight.

The interval $\left[\Lambda ,\mathit{M}\right]$ can be constructed also by rows, i.e.,

$$\left[\Lambda ,\mathit{M}\right]=\left\{\begin{array}{c}\mathit{P}:{\mathit{p}}_i\in \left[{\lambda }_i,{\mu }_i\right]\mathrm{for}\mathrm{all}i,\mathrm{with}{\mathit{p}}_i,{\lambda }_i,{\mu }_i\\ \mathrm{being}\mathrm{the}\mathrm{rows}\mathrm{of}\mathrm{the}\mathrm{respective}\mathrm{matrices}\mathit{P},\Lambda ,\mathit{M}\end{array}\right\}.$$

In what follows we will define a non-homogeneous Markov set system. We will keep the entire notation introduced in Section 2 for a NHMS and we will build on that.

Let $\left[\mathbb{M}\right]=\left[\check{\mathit{Q}},\widehat{\mathit{Q}}\right]$ be an interval of $k\times k$ stochastic matrices with $\check{\mathit{Q}}\le \mathit{Q}\left(t\right)\le \widehat{\mathit{Q}}$ for every $t\in \mathbb{N}$ which is tight, i.e.,

$$\left[\mathbb{M}\right]=\left\{\mathit{Q}\left(t\right):\mathrm{is}\mathrm{an}k\times k\mathrm{stochastic}\mathrm{matrix}\mathrm{with}\check{\mathit{Q}}\le \mathit{Q}\left(t\right)\le \widehat{\mathit{Q}}\right\}$$

with

$${\check{q}}_{ij}=\underset{\mathit{Q}\left(t\right)\in \left[\check{\mathit{Q}},\widehat{\mathit{Q}}\right]}{min}{q}_{ij}\left(t\right)\mathrm{for}\mathrm{every}t\in \mathbb{N},$$

$${\widehat{q}}_{ij}=\underset{\mathit{Q}\left(t\right)\in \left[\check{\mathit{Q}},\widehat{\mathit{Q}}\right]}{max}{q}_{ij}\left(t\right)\mathrm{for}\mathrm{every}t\in \mathbb{N},$$

and the notation $\left[\check{\mathit{Q}},\widehat{\mathit{Q}}\right]$ will be taken to imply that $\left[\check{\mathit{Q}},\widehat{\mathit{Q}}\right]\ne ⌀.$

We will make now the following basic assumptions:

Assumption 1.

Let that the imbedded non-homogeneous Markov chain of the NHMS has all its probability matrices in$\left[\mathbb{M}\right]$.

We call $\left[\mathbb{M}\right]$ the probability transition matrix set (PTMS) of the imbedded non-homogeneous Markov chain.

Now define by

$$\left[{\mathbb{M}}^2\right]=\left\{\mathit{Q}\left(0\right)\mathit{Q}\left(1\right):\mathit{Q}\left(0\right),\mathit{Q}\left(1\right)\in \left[\mathbb{M}\right]\right\},$$

$$\begin{array}{ccc}& & ...\hfill \\ & & ...\hfill \\ & & ...\hfill \end{array}$$

$$\left[{\mathbb{M}}^n\right]=\left\{\mathit{Q}\left(0\right)\mathit{Q}\left(1\right)\dots \mathit{Q}\left(n-1\right):\mathit{Q}\left(0\right),\dots ,\mathit{Q}\left(n-1\right)\in \left[\mathbb{M}\right]\right\}.$$

We call the sequence ${\left\{\left[{\mathbb{M}}^n\right]\right\}}_{n=1}^{\infty }$ the inherent or imbedded non-homogeneous Markov set chain.

Assumption 2.

Let$\left[{\mathbb{S}}_0\right]$be the set of$1\times k$stochastic vectors from which the initial distribution$\mathit{q}\left(0\right)$is chosen.

$$\left[{\mathbb{S}}_0\right]=\left[{\check{\mathit{q}}}_0,{\widehat{\mathit{q}}}_0\right]=\left\{\mathit{q}\left(0\right):\mathit{is}\mathit{a}\mathit{stochastic}\mathit{vector}\mathit{with}\mathit{q}\left(0\right)\in \left[{\mathbb{S}}_0\right]\right\}.$$

Assumption 3.

Let$\left[{\mathbb{R}}_0\right]$be the set of$1\times k$stochastic vectors from which the allocation probabilities${\mathit{p}}_0\left(t\right)$are being selected. That is

$$\left[{\mathbb{R}}_0\right]=\left[{\check{\mathit{p}}}_0,{\widehat{\mathit{p}}}_0\right]=\left\{{\mathit{p}}_0\left(t\right):\mathit{is}\mathit{a}\mathit{stochastic}\mathit{vector}\mathit{with}{\mathit{p}}_0\left(t\right)\in \left[{\mathbb{R}}_0\right]\mathit{for}\mathit{every}t\right\}.$$

We call a NHMS whose parameters are assumed to be in intervals as in Assumptions 1–3 a Non-homogeneous Markov Set System (NHMSS).

Note that it is apparent by now that Markov set chains that were initiated by [9,40,41,42,43] are special cases of a NHMSS.

4. The Set of the Expected Relative Population Structures of a NHMSS

In geometry the Minkowski sum (also known as dilation) of two sets of position vectors $\mathbb{A}$ and $\mathbb{B}$ in Euclidean space is formed by adding each vector in $\mathbb{A}$ to each vector in $\mathbb{B}$. That is

$$\mathbb{A}+\mathbb{B}=\left\{a+b:a\in \mathbb{A},b\in \mathbb{B}\right\}.$$

Example 2.

If we have two sets$\mathbb{A}$and$\mathbb{B}$consisting of three position vectors (informally, three points) representing the vertices of two triangles in${\mathbb{R}}^2$with coordinates

$$\mathbb{A}=\left\{\left(1,0\right),\left(0,1\right),\left(0,-1\right)\right\}and\mathbb{B}=\left\{\left(0,0\right),\left(1,1\right),\left(1,-1\right)\right\},$$

then their Minkowski sum is

$$\mathbb{A}+\mathbb{B}=\left\{\left(1,0\right),\left(2,1\right),\left(2,-1\right),\left(0,1\right),\left(1,2\right),\left(0,-1\right),\left(1,-2\right)\right\},$$

which comprises the vertices of a hexagon.

For Minkowski addition, the zero set containing only the zero vector $\mathbf{0}$, is an identity element for every subset $\mathbb{V}$ of a vector space, i.e., $\mathbb{V}+\left\{0\right\}=$$\mathbb{V}$.

The empty set is important in Minkowski addition because the empty set annihilates every other subset for every subset $\mathbb{V}$ of a vector space, its sum with the empty set is empty, i.e., $\mathbb{V}+⌀=$⌀.

We are now in a position to state the following Lemma ([44])

Lemma 3.

If$\mathbb{V}$is a convex set then$\mu \mathbb{V}+\cancel{\geqq }\mathbb{V}$is also a convex set and furthermore$\mu \mathbb{V}+\cancel{\geqq }\mathbb{V}=\left(\mu +\lambda \right)\mathbb{V}$for every$\lambda ,\mu >0$. Conversely, if this “distributive property” holds for all non-negative real numbers$\lambda ,\mu >0$then the set is convex.

Remark 1.

For two convex polygons${\mathbb{V}}_1$and${\mathbb{V}}_2$in the plane with m and n vertices, their Minkowski sum is a convex polygon with at most$m+n$vertices and may be computed in time$O\left(m+n\right)$by a very simple procedure.

We need the following sets for the Lemma that follows

$$Rng\left(\mathit{q}\right)=\left\{\mathit{y}\mathbf{:}\mathit{y}=\mathit{qQ}\mathrm{for}\mathrm{some}\mathit{Q}\in \left[\mathbb{M}\right]\mathrm{and}\mathrm{any}\mathit{q}\in S{M}_{1,n}\right\}.$$

Also

$$Rng\left(\mathbb{S}\right)={\cup }_{\mathit{q}\in \mathbb{S}}Rng\left(\mathit{q}\right)=\left\{\mathit{y}\mathbf{:}\mathit{y}=\mathit{qQ}\mathrm{for}\mathrm{some}\mathit{Q}\in \left[\mathbb{M}\right]\mathrm{and}\mathrm{some}\mathit{q}\in \mathbb{S}\right\}.$$

Lemma 4.

Let an NHMSS with$T\left(t\right)\ge 0$and finite and which is expanding$\left(\Delta T\left(t\right)\ge 0\right)$. Also let$\left[{\mathbb{S}}_0\right]$the set from which the initial distribution of memberships is drawn,$\left[{\mathbb{R}}_0\right]$the set from which the allocation probabilities in the various states are chosen at every time step and finally let$\left[\mathbb{M}\right]$the set to which all the transition probability matrices of the inherent Markov chain of memberships belong. Then the set$Rn{g}_t\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)$of all possible expected relative population structures at time t is given by

$$Rn{g}_t\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)=\{\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]:\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]\in \frac{T\left(0\right)}{T\left(t\right)}{\cup }_{\mathit{q}\left(0\right)\in \left[{\mathbb{S}}_0\right]}\mathit{q}\left(0\right)\left[{\mathbb{M}}^t\right]$$

(5)

$$+\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right){\cup }_{{\mathit{p}}_0\left(\tau \right)\in \left[{\mathbb{R}}_0\right]}{\mathit{p}}_0\left(\tau \right)\left[{\mathbb{M}}^{t-\tau }\right]\},$$

with$\left[{\mathbb{M}}^0\right]=\left[\mathbb{M}\right]$and$\left[{\mathbb{M}}^{-1}\right]=\left\{\mathit{I}\right\}$.

Proof. 

Following the relevant proofs that lead to Equations $\left(2.1\right)$, $\left(2.2\right)$ and $\left(3.4\right)$ in [10] or Equations $\left(2\right)$ and $\left(4\right)$ in the present, we could easily prove $\left(5\right)$. □

We will need the following Lemma from ([9] p. 39):

Lemma 5.

In a non-homogeneous Markov set chain let the set of initial distributions$\left[{\mathbb{S}}_0\right]$be convex and let$\left[\mathbb{M}\right]$be a tight interval from which the transition probability sequence of matrices is being selected. Then the set$\left[{\mathbb{S}}_t\right]$of all possible probability distributions at the various states at time t is a convex set.

In the next theorem we show under which conditions the set of all possible expected relative population structures in a NHMSS is a convex set.

Theorem 1.

Let an NHMSS with$T\left(t\right)$finite and which is expanding$\left(\Delta T\left(t\right)\ge 0\right)$. If $\left[{\mathbb{S}}_0\right],\left[{\mathbb{R}}_0\right]$are convex sets and$\left[\mathbb{M}\right]$a tight interval then the set of all possible expected relative population structures is a convex set.

Proof. 

Define $\left\{\mathbb{E}\left[\mathring{\mathit{q}}\left(0,t\right)\right]\right\}$ be the set of all possible expected relative structures of the initial memberships then from Lemma 4 we have

$$\left\{\mathbb{E}\left[\mathring{\mathit{q}}\left(0,t\right)\right]\right\}=\{\mathbb{E}\left[\mathring{\mathit{q}}\left(0,t\right)\right]:\mathbb{E}\left[\mathring{\mathit{q}}\left(0,t\right)\right]\in \frac{T\left(0\right)}{T\left(t\right)}{\cup }_{\mathit{q}\left(0\right)\in \left[{\mathbb{S}}_0\right]}\mathit{q}\left(0\right)\left[{\mathbb{M}}^t\right]\},$$

(6)

since $\frac{T\left(0\right)}{T\left(t\right)}\ge 0$, $\left[{\mathbb{S}}_0\right]$ is a convex set and $\left[\mathbb{M}\right]$ a tight interval, from Lemma 4 and 5 we get that $\left\{\mathbb{E}\left[\mathring{q}\left(0,t\right)\right]\right\}$ is a convex set. Also, the set

$$\left\{\mathbb{E}\left[{\mathit{r}}_{\tau }\right]\right\}=\{\mathbb{E}\left[{\mathit{r}}_{\tau }\right]:\mathbb{E}\left[{\mathit{r}}_{\tau }\right]\in \Delta T\left(\tau \right){\cup }_{\mathit{p}\left(\tau \right)\in \left[{\mathbb{R}}_0\right]}{\mathit{p}}_0\left(\tau \right)\left[{\mathbb{M}}^{t-\tau }\right]\},$$

is the set of all possible expected structures of new memberships at time t which entered in the system at time $\tau$. Now, since the system is expanding, i.e., $\Delta T\left(\tau \right)\ge 0$, $\left[{\mathbb{R}}_0\right]$ is a convex set and $\left[\mathbb{M}\right]$ a tight interval, then with the same reasoning as in $\left(6\right)$, we get that $\left\{\mathbb{E}\left[{\mathit{r}}_{\tau }\right]\right\}$ is a convex set. Also, from Remark 1 we get that the Minkowski sum of sets

$$\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\left\{\mathbb{E}\left[{\mathit{r}}_{\tau }\right]\right\},$$

is a convex set. Hence, since the two sets in the right-hand side of Equation $\left(5\right)$ are convex and then according to Remark 1 their Minkowski sum $Rn{g}_t\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)$ is a convex set. □

We will now borrow the following Theorem from ([9] p. 40).

Theorem 2.

In a non-homogeneous Markov set chain let the set of initial distributions$\left[{\mathbb{S}}_0\right]$be a convex polytope and let$\left[\mathbb{M}\right]$be a tight interval from which the transition probability sequence of matrices is being selected. Then the set$\left[{\mathbb{S}}_t\right]$of all possible probability distributions at the various states at time t is a convex polytope with vertices of the form${\mathcal{E}}_i{E}_{i_1}\dots E_{i_k}$for some vertices${\mathcal{E}}_i$of$\left[{\mathbb{S}}_0\right]$and some vertices$E_{i_j}$of$\left[\mathbb{M}\right]$.

In the next theorem we show under which conditions the set of all possible expected relative population structures in a NHMSS is a convex polytope.

Theorem 3.

Let an NHMSS with$T\left(t\right)$finite and which is expanding$\left(\Delta T\left(t\right)\ge 0\right)$. If$\left[{\mathbb{S}}_0\right],\left[{\mathbb{R}}_0\right]$are convex polytopes and$\left[\mathbb{M}\right]$a tight interval then the set of all possible expected relative population structures is a convex polytope.

Proof. 

The proof follows the steps of the proof of Theorem 1 using Theorem 2. □

5. Asymptotic Behavior of NHMSS

The problem of asymptotic behavior has been one of central importance for, homogeneous Markov chains, non-homogeneous Markov chains, NHMS, and non-homogeneous Markov set chains. In the present section we will prove a series of theorems with which we establish the asymptotic behavior of NHMSS.

Since Markov himself and his student Dobrushin the coefficient of ergodicity $\mathcal{T}\left(\mathit{Q}\right)$ of a $k\times k$ stochastic matrix $\mathit{Q}={\left\{q_{ij}\right\}}_{i,j=1}^k$, has been a fundamental tool in the study of Markov chains. We have

$$\mathcal{T}\left(\mathit{Q}\right)=\frac{1}{2}\underset{ij}{max}\sum _{l=1}^k\left|q_{il}-q_{jl}\right|,$$

(7)

thus $0\le \mathcal{T}\left(\mathit{Q}\right)\le 1$. We clarify in here that if $\mathcal{T}\left(\mathit{Q}\right)<1$ the stochastic matrix $\mathit{Q}$ is called scrambling. Scrambling matrices are regular, but not all regular matrices are scrambling. Yet if $\mathit{Q}$ is a regular stochastic matrix then some power of $\mathit{Q}$, say ${\mathit{Q}}^n$ is scrambling. We define by

$$\overline{\mathcal{T}}\left(\left[\mathbb{M}\right]\right)=\underset{\mathit{Q}\in \left[\check{\mathit{Q}},\widehat{\mathit{Q}}\right]}{max}\mathcal{T}\left(\mathit{Q}\right),$$

(8)

if $\overline{\mathcal{T}}\left(\left[\mathbb{M}\right]\right)<1$ we say that $\left[\mathbb{M}\right]$ is uniformly scrambling. More on uniform scrambling and the interpretation of the coefficient of ergodicity of a matrix $\mathit{A}\in {\mathbb{R}}^{n\times n}$ as a matrix norm when the norm is restricted to a specified subspace could be found, in [45]. For explicit forms for ergodicity coefficients and properties see also [46,47]. In what follows we will use the following norm for a matrix $\mathit{A}\in M_n\left(\mathbb{R}\right)$

$$∥\mathit{A}∥=\underset{i}{max}\sum _{j=1}^k\left|a_{ij}\right|$$

We will use the concept of $\overline{\mathcal{T}}\left(\left[\mathbb{M}\right]\right)<1$ to study asymptotic behavior in a NHMSS. It is important to note that if we consider as $\left(\left[{\mathbb{M}}^C\right],∥.∥\right)$ the space of non-empty compact subsets of $\left(\left[\mathbb{M}\right],∥.∥\right)$ then $\left(\left[{\mathbb{M}}^C\right],∥.∥\right)$ is a metric space ([48]). This space can be topologized using the Hausdorff metric $d\left(.,.\right)$ defined by

$$d\left({\mathbb{S}}_1,{\mathbb{S}}_2\right)=max\left\{\delta \left({\mathbb{S}}_1,{\mathbb{S}}_2\right),\delta \left({\mathbb{S}}_2,{\mathbb{S}}_1\right)\right\},$$

(9)

where

$$\delta \left({\mathbb{S}}_1,{\mathbb{S}}_2\right)=\underset{{\mathit{Q}}_1\in S_1}{max}\underset{{\mathit{Q}}_2\in S_2}{min}∥{\mathit{Q}}_1-{\mathit{Q}}_2∥\mathrm{and}{\mathbb{S}}_1,{\mathbb{S}}_2\in \left[{\mathbb{M}}^C\right],$$

(10)

From [48] we get also that $\left(\left[{\mathbb{M}}^C\right],d\left(.,.\right)\right)$ is a metric space.

We will need the following Lemmas

Lemma 6

([49]). The following statements are equivalent:

(i)

Sequence${\left\{\Delta T\left(t\right)\right\}}_{t=0}^{\infty }$converges to zero with geometrical rate;

(ii)

Sequence${\left\{T\left(t\right)\right\}}_{t=0}^{\infty }$converges to T with geometrical rate,

Lemma 7

([50] p. 541). Suppose that

$${\left\{\frac{\Delta T\left(t\right)}{T\left(t\right)}\right\}}_{t=0}^{\infty }$$

converges to zero as $t\to \infty$ geometrically fast with $T\left(t\right)\ge T\left(t-1\right).$ Then ${\left\{T\left(t\right)\right\}}_{t=0}^{\infty }$ converges geometrically fast.

Remark 2.

The restriction

$$\underset{t\to \infty }{lim}\frac{\Delta T\left(t\right)}{T\left(t\right)}=0,$$

is a general assumption with the physical interpretation that the proportional growth rate vanishes in the limit. This assumption allows lim${}_{t\to \infty }T\left(t\right)=\infty$.

We will now study what happens asymptotically to the two sets of the expected relative population structures when the initial structures belong to two different sets as well as the allocation probabilities in the respective cases.

Theorem 4.

Let an NHMSS for which$\left[\mathbb{M}\right]$Lis a tight interval which is uniformly scrambling. Assume that

$${\left\{\frac{\Delta T\left(t\right)}{T\left(t\right)}\right\}}_{t=0}^{\infty }\mathit{converges}\mathit{to}\mathit{zero}\mathit{geometrically}\mathit{fast}.$$

Also, assume that$\left[\mathbb{S}\right],\left[{\mathbb{R}}_0\right]$are compact and convex. Let${\mathbb{S}}_1,{\mathbb{S}}_2\subseteq \left[{\mathbb{S}}^C\right]$and${\mathbb{R}}_1,{\mathbb{R}}_2\subseteq \left[{\mathbb{R}}_0^C\right]$then

$$d\left(Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1\right),Rn{g}_t\left({\mathbb{S}}_2,{\mathbb{R}}_2\right)\right){\to }_{t\to \infty }0\mathit{geometrically}\mathit{fast}.$$

Proof. 

From $\left(5\right)$ we get that

$$\begin{array}{ccc}\hfill Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1\right)& =& \{\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]:\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]\in \frac{T\left(0\right)}{T\left(t\right)}{\cup }_{\mathit{q}\left(0\right)\in \left[{\mathbb{S}}_1\right]}\mathit{q}\left(0\right)\left[{\mathbb{M}}^t\right]+\hfill \\ \hfill & & \frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right){\cup }_{{\mathit{p}}_0\left(\tau \right)\in \left[{\mathbb{R}}_1\right]}{\mathit{p}}_0\left(\tau \right)\left[{\mathbb{M}}^{t-\tau }\right]\},\hfill \end{array}$$

(11)

and from $\left(11\right)$ if we replace $\mathit{q}\left(0\right)$ with $\overline{\mathit{q}}\left(0\right)$, $\left[{\mathbb{S}}_1\right]$ with $\left[{\mathbb{S}}_2\right]$; ${\mathit{p}}_0\left(\tau \right)$ with ${\overline{\mathit{p}}}_0\left(\tau \right);$$\left[{\mathbb{R}}_1\right]$ with $\left[{\mathbb{R}}_2\right]$ and $\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]$ with $\mathbb{E}\left[\overline{\mathit{q}}\left(0,t\right)\right]$ we get $Rn{g}_t\left({\mathbb{S}}_2,{\mathbb{R}}_2\right)$. Since $\left[{\mathbb{S}}_1\right],\left[{\mathbb{R}}_1\right]$ are compact and $\left[\mathbb{M}\right]$ is a tight interval, it is not difficult to show that $Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1\right)$ is compact. The same applies for $Rn{g}_t\left({\mathbb{S}}_2,{\mathbb{R}}_2\right)$ since $\left[{\mathbb{S}}_2\right],\left[{\mathbb{R}}_2\right]$ are also compact. Hence, we may take their Hausdorff metric. Now, we have that

$$\begin{array}{ccc}& & \delta \left(Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1\right),Rn{g}_t\left({\mathbb{S}}_2,{\mathbb{R}}_2\right)\right)\hfill \\ \hfill & =& {\displaystyle \underset{\underset{\mathit{Q}\left(t\right)\in \left[\mathbb{M}\right]\mathrm{for}\mathrm{all}\tau ,t}{\mathit{q}\left(0\right)\in \left[{\mathbb{S}}_1\right],{\mathit{p}}_0\left(\tau \right)\in \left[{\mathbb{R}}_1\right],}}{max}}{\displaystyle \underset{\underset{\overline{\mathit{Q}}\left(t\right)\in \left[\mathbb{M}\right]\mathrm{for}\mathrm{all}\tau ,t}{\overline{\mathit{q}}\left(0\right)\in \left[{\mathbb{S}}_2\right],{\overline{\mathit{p}}}_0\left(\tau \right)\in \left[{\mathbb{R}}_2\right]}}{min}}∥\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]-\mathbb{E}\left[\overline{\mathit{q}}\left(0,t\right)\right]∥.\hfill \end{array}$$

(12)

by continuity of

$$f\left(\mathbb{E}\left[\overline{q}\left(0,t\right)\right]\right)=min∥\mathbb{E}\left[q\left(0,t\right)\right]-\mathbb{E}\left[\overline{q}\left(0,t\right)\right]∥$$

then for some ${\mathit{q}}^{*}\left(0\right)\in \left[{\mathbb{S}}_1\right]$, ${\mathit{p}}_0^{*}\left(\tau \right)\in \left[{\mathbb{R}}_1\right]$ for every $\tau \in \left[1,t\right]$, ${\mathit{Q}}^{*}\left(t\right)\in \left[\mathbb{M}\right]$ for every $t\in \left[0,t\right]$ and by denoting with

$$\mathbb{E}\left[{\mathit{q}}^{*}\left(0,t\right)\right]=\frac{T\left(0\right)}{T\left(t\right)}{\mathit{q}}^{*}\left(0\right){\mathit{Q}}^{*}\left(0,t\right)+\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right){\mathit{p}}_0^{*}\left(\tau \right){\mathit{Q}}^{*}\left(\tau ,t\right),$$

(13)

we get

$$\delta \left(Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1\right),Rn{g}_t\left({\mathbb{S}}_2,{\mathbb{R}}_2\right)\right)=$$

(14)

$$\underset{\underset{\overline{\mathit{Q}}\left(t\right)\in \left[\mathbb{M}\right]\mathrm{for}\mathrm{all}\tau ,t}{\overline{\mathit{q}}\left(0\right)\in \left[{\mathbb{S}}_2\right],{\overline{\mathit{p}}}_0\left(\tau \right)\in \left[{\mathbb{R}}_2\right]}}{min}∥\mathbb{E}\left[{\mathit{q}}^{*}\left(0,t\right)\right]-\mathbb{E}\left[\overline{\mathit{q}}\left(0,t\right)\right]∥$$

$$\le ∥\mathbb{E}\left[{\mathit{q}}^{*}\left(0,t\right)\right]-\mathbb{E}\left[{\overline{\mathit{q}}}^{*}\left(0,t\right)\right]∥,$$

where $\mathbb{E}\left[{\overline{\mathit{q}}}^{*}\left(0,t\right)\right]$ is given by $\left(13\right)$ if we replace ${\mathit{q}}^{*}\left(0\right)$ with any ${\overline{\mathit{q}}}^{*}\left(0\right)\in \left[{\mathbb{S}}_2\right]$, ${\mathit{p}}_0^{*}\left(\tau \right)$ with any ${\overline{\mathit{p}}}_0^{*}\left(\tau \right)\in \left[{\mathbb{R}}_2\right]$ for every $\tau \in \left[1,t\right]$. Now, from $\left(13\right)$ and $\left(14\right)$ we get that

$$\delta \left(Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1\right),Rn{g}_t\left({\mathbb{S}}_2,{\mathbb{R}}_2\right)\right)\le \frac{T\left(0\right)}{T\left(t\right)}∥\left[{\mathit{q}}^{*}\left(0\right)-{\overline{\mathit{q}}}^{*}\left(0\right)\right]{\mathit{Q}}^{*}\left(0,t\right)∥$$

$$+\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right)∥{\mathit{p}}_0^{*}\left(\tau \right){\mathit{Q}}^{*}\left(\tau ,t\right)-{\overline{\mathit{p}}}_0^{*}\left(\tau \right){\mathit{Q}}^{*}\left(\tau ,t\right)∥$$

$$\le \frac{T\left(0\right)}{T\left(t\right)}\mathcal{T}\left({\mathit{Q}}^{*}\left(0,t\right)\right)∥\left[{\mathit{q}}^{*}\left(0\right)-{\overline{\mathit{q}}}^{*}\left(0\right)\right]∥$$

$$+\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right)\mathcal{T}\left({\mathit{Q}}^{*}\left(\tau ,t\right)\right)∥{\mathit{p}}_0^{*}\left(\tau \right)-{\overline{\mathit{p}}}_0^{*}\left(\tau \right)∥$$

$$(\mathrm{since}\left[\mathbb{M}\right]\mathrm{is}\mathrm{uniformly}\mathrm{scrambling}\tilde{\mathcal{T}}=\underset{{\mathit{Q}}^{*}\left(t\right)\in \left[\mathbb{M}\right]}{max}\mathcal{T}\left({\mathit{Q}}^{*}\left(t\right)\right)<1)$$

$$\le \frac{T\left(0\right)}{T\left(t\right)}{\tilde{\mathcal{T}}}^t∥\left[{\mathit{q}}^{*}\left(0\right)-{\overline{\mathit{q}}}^{*}\left(0\right)\right]∥$$

$$+\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right){\tilde{\mathcal{T}}}^{t-\tau }∥{\mathit{p}}_0^{*}\left(\tau \right)-{\overline{\mathit{p}}}_0^{*}\left(\tau \right)∥$$

$$\le \frac{T\left(0\right)}{T\left(t\right)}{\tilde{\mathcal{T}}}^t\delta \left({\mathbb{S}}_1,{\mathbb{S}}_2\right)+\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right){\tilde{\mathcal{T}}}^{t-\tau }\delta \left({\mathbb{R}}_1,{\mathbb{R}}_2\right).$$

(15)

Since ${\left\{\frac{\Delta T\left(t\right)}{T\left(t\right)}\right\}}_{t=0}^{\infty }$ converges to zero geometrically fast, from Lemmas 6 and 7 we get that there are $c_1$, $c_2>0$ and $0<b_1,b_2<1$ such that

$$\left|\frac{1}{T\left(t\right)}-\frac{1}{T}\right|\le c_1{b}_1^t\mathrm{and}\left|\frac{\Delta T\left(t\right)}{T\left(t\right)}\right|\le c_2{b}_2^t.$$

(16)

From $\left(15\right),\left(16\right)$ we get that

$$\delta \left(Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1\right),Rn{g}_t\left({\mathbb{S}}_2,{\mathbb{R}}_2\right)\right)\le T\left(0\right)\left(\frac{1}{T}+c_1{b}_1^t\right){\tilde{\mathcal{T}}}^t\delta \left({\mathbb{S}}_1,{\mathbb{S}}_2\right)$$

$$+\delta \left({\mathbb{R}}_1,{\mathbb{R}}_2\right)\left(\frac{1}{T}+c_1{b}_1^t\right)\sum _{\tau =1}^t{c}_2{b}_2^{\tau }{\tilde{\mathcal{T}}}^{t-\tau },$$

$$(\mathrm{assuming}\tilde{\mathcal{T}}<b_2)$$

$$\le T\left(0\right)\left(\frac{1}{T}+c_1{b}_1^t\right){\tilde{\mathcal{T}}}^t\delta \left({\mathbb{S}}_1,{\mathbb{S}}_2\right)+\delta \left({\mathbb{R}}_1,{\mathbb{R}}_2\right)\left(\frac{1}{T}+c_1{b}_1^t\right)c_2{b}_2^{t-1}\sum _{\tau =1}^t{\left(\frac{\tilde{\mathcal{T}}}{b_2}\right)}^{t-\tau }$$

$$\le T\left(0\right)\left(\frac{1}{T}+c_1{b}_1^t\right){\tilde{\mathcal{T}}}^t\delta \left({\mathbb{S}}_1,{\mathbb{S}}_2\right)+\delta \left({\mathbb{R}}_1,{\mathbb{R}}_2\right)\left(\frac{1}{T}+c_1{b}_1^t\right)c_2{b}_2^{t-1}{\left(1-\frac{\tilde{\mathcal{T}}}{b_2}\right)}^{-1}.$$

(17)

From $\left(17\right)$ we conclude that $\delta \left(Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1\right),Rn{g}_t\left({\mathbb{S}}_2,{\mathbb{R}}_2\right)\right)$ converges to zero geometrically fast. Similarly, we get the same conclusion if $\tilde{\mathcal{T}}>b_2.$

Now, in a similar way we may prove that $\delta \left(Rn{g}_t\left({\mathbb{S}}_2,{\mathbb{R}}_2\right),Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1\right)\right)$ converges to zero geometrically fast. Thus, we arrive at the desired conclusion. □

We will now establish under what conditions the convergence to the limiting set in a NHMSS is geometrically fast.

Theorem 5.

Let an NHMSS for which$\left[\mathbb{M}\right]$is a tight interval which is uniformly scrambling. Assume that

$${\left\{\frac{\Delta T\left(t\right)}{T\left(t\right)}\right\}}_{t=0}^{\infty }\mathit{converges}\mathit{to}\mathit{zero}\mathit{geometrically}\mathit{fast}.$$

Also, let that$\left[\mathbb{S}\right],\left[{\mathbb{R}}_0\right]$are compact and convex. Then we have that

$$d(Rn{g}_t\left(\mathbb{S},{\mathbb{R}}_0\right),Rn{g}_{\infty }\left(\mathbb{S},{\mathbb{R}}_0\right)\to 0,\mathit{geometrically}\mathit{fast}.$$

(18)

Proof. 

We will first show that

$$Rng\left(Rn{g}_t\left(\mathbb{S},{\mathbb{R}}_0\right)\right)=Rn{g}_{t+1}\left(\mathbb{S},{\mathbb{R}}_0\right).$$

(19)

By the evaluation of the function $Rng\left(.,.\right)$ given in Lemma 4 we get that

$$Rng\left(Rn{g}_t\left(\mathbb{S},{\mathbb{R}}_0\right)\right)=\frac{T\left(t\right)}{T\left(t+1\right)}{\cup }_{\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]\in Rn{g}_t\left(\mathbb{S},{\mathbb{R}}_0\right)}\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]\left[\mathbb{M}\right]$$

$$+\frac{1}{T\left(t+1\right)}\Delta T\left(t+1\right){\cup }_{{\mathit{p}}_0\left(t\right)\in \left[{\mathbb{R}}_0\right]}{\mathit{p}}_0\left(t\right)=$$

$$\frac{T\left(t\right)}{T\left(t+1\right)}\frac{T\left(0\right)}{T\left(t\right)}{\cup }_{\mathit{q}\left(0\right)\in \left[\mathbb{S}\right]}\mathit{q}\left(0\right)\left[{\mathbb{M}}^t\right]\left[\mathbb{M}\right]+$$

$$\frac{T\left(t\right)}{T\left(t+1\right)}\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right){\cup }_{{\mathit{p}}_0\left(\tau \right)\in \left[{\mathbb{R}}_0\right]}{\mathit{p}}_0\left(\tau \right)\left[{\mathbb{M}}^{t-\tau }\right]\left[\mathbb{M}\right]$$

$$+\frac{1}{T\left(t+1\right)}\Delta T\left(t+1\right){\cup }_{{\mathit{p}}_0\left(t\right)\in \left[{\mathbb{R}}_0\right]}{\mathit{p}}_0\left(t\right)$$

$$=\frac{T\left(0\right)}{T\left(t+1\right)}{\cup }_{\mathit{q}\left(0\right)\in \left[\mathbb{S}\right]}\mathit{q}\left(0\right)\left[{\mathbb{M}}^{t+1}\right]+$$

$$\frac{1}{T\left(t+1\right)}\sum _{\tau =1}^{t+1}\Delta T\left(\tau \right){\cup }_{{\mathit{p}}_0\left(\tau \right)\in \left[{\mathbb{R}}_0\right]}{\mathit{p}}_0\left(\tau \right)\left[{\mathbb{M}}^{t-\tau +1}\right]=Rn{g}_{t+1}\left(\mathbb{S},{\mathbb{R}}_0\right).$$

Now, we will show that

$$Rn{g}_t\left(Rn{g}_{\infty }\left(\mathbb{S},{\mathbb{R}}_0\right)\right)=Rn{g}_{\infty }\left(\mathbb{S},{\mathbb{R}}_0\right).$$

(20)

Assume that $\mathbb{E}\left[\mathit{q}\left(\infty \right)\right]$ is an element of $Rn{g}_{\infty }\left(\mathbb{S},{\mathbb{R}}_0\right)$ then from Lemma 4 we get that

$$Rng\left(Rn{g}_{\infty }\left(\mathbb{S},{\mathbb{R}}_0\right)\right)=\frac{T}{T}{\cup }_{\mathbb{E}\left[\mathit{q}\left(\infty \right)\right]\in Rn{g}_{\infty }\left(\mathbb{S},{\mathbb{R}}_0\right)}\mathbb{E}\left[\mathit{q}\left(\infty \right)\right]\left[\mathbb{M}\right]+$$

$$\frac{1}{T}\left(T-T\right){\cup }_{{\mathit{p}}_0\left(\infty \right)\in \left[{\mathbb{R}}_0\right]}{\mathit{p}}_0\left(\infty \right)=Rn{g}_{\infty }\left(\mathbb{S},{\mathbb{R}}_0\right)\left[\mathbb{M}\right]=Rn{g}_{\infty }\left(\mathbb{S},{\mathbb{R}}_0\right),$$

(21)

where the last equality will be proved in Theorem 10. From $\left(21\right)$ we recursively get $\left(20\right)$.

Since the conditions of Theorem 4 hold in the present theorem we get that

$$d\left(Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1\right),Rn{g}_t\left({\mathbb{S}}_2,{\mathbb{R}}_2\right)\right)\to 0\mathrm{geometrically}\mathrm{fast}.$$

(22)

Now, in $\left(22\right)$ replace $Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1\right)$ with $Rn{g}_t\left(Rn{g}_{\infty }\left(\mathbb{S},{\mathbb{R}}_0\right)\right)$ and $Rn{g}_t\left({\mathbb{S}}_2,{\mathbb{R}}_2\right)$ with $Rn{g}_t\left(\mathbb{S},{\mathbb{R}}_0\right)$, then we get that

$$d\left(Rn{g}_t\left(Rn{g}_{\infty }\left(\mathbb{S},{\mathbb{R}}_0\right)\right),Rn{g}_t\left(\mathbb{S},{\mathbb{R}}_0\right)\right)\to 0,\mathrm{geometrically}\mathrm{fast}.$$

(23)

From $\left(20\right),\left(23\right)$ we arrive at

$$d(Rn{g}_t\left(\mathbb{S},{\mathbb{R}}_0\right),Rn{g}_{\infty }\left(\mathbb{S},{\mathbb{R}}_0\right)\to 0,\mathrm{geometrically}\mathrm{fast}.$$

 □

The above Theorems 4 and 5 have an important consequence since it provides a generalization, by relaxing important assumptions, of the basic asymptotic theory for an NHMS. The theorem is the following and could be proved by just following the analogous steps as in the proofs of Theorems 4 and 5 and Theorem 3.3 in [11]:

Theorem 6.

Consider an NHMS and assume that

$${\left\{\frac{\Delta T\left(t\right)}{T\left(t\right)}\right\}}_{t=0}^{\infty }\mathit{converges}\mathit{geometrically}\mathit{fast}.$$

Let${\left\{\mathit{Q}\left(t\right)\right\}}_{t=0}^{\infty }$be the sequence of transition matrices and${\left\{{\mathit{p}}_0\left(t\right)\right\}}_{t=0}^{\infty }$be the sequence of allocation probabilities. If${sup}_t$$\tilde{\mathcal{T}}\left(\mathit{Q}\left(t\right)\right)<1$then

$$\underset{t\to \infty }{lim}\mathbb{E}\left(\mathit{q}\left(0,t\right)\right)=\mathit{q}\left(\infty \right)\mathit{geometrically}\mathit{fast},$$

where$\mathit{q}\left(\infty \right)$is the row of the stable matrix$\mathit{Q}\left(\infty \right)={lim}_{t\to \infty }\mathit{Q}\left(0,t\right)$.

The basic asymptotic theorem for an NHMS and which has been used in many papers to provide further results is:

Theorem 7.

Let an NHMS and let that$\left(a\right)$${lim}_{t\to \infty }∥\mathit{Q}\left(t\right)-\mathit{Q}∥=0$and$\mathit{Q}$a regular stochastic matrix;$\left(b\right)$ lim${}_{t\to \infty }∥{\mathit{p}}_0\left(t\right)-{\mathit{p}}_0∥=0;$ and $\left(c\right)$${lim}_{t\to \infty }\left[\Delta T\left(t\right)/T\left(t\right)\right]=0$. Then

$$\underset{t\to \infty }{lim}∥\mathbb{E}\left(\mathit{q}\left(0,t\right)\right)-\mathit{q}\left(\infty \right)∥=0$$

where $\mathit{q}\left(\infty \right)$ is the row of the stable matrix $\mathit{Q}\left(\infty \right)={lim}_{t\to \infty }{\mathit{Q}}^t.$

Theorem 7 has been used extensively in the theory of NHMS to produce further results see for example [10,11,12,17,51], and the relaxation of the necessary conditions in Theorem 6 is apparent.

Another consequence is that Theorem 4 provides conditions under which two different NHMS, with the same number of states and population, but different initial states and different allocation probabilities of memberships if they have the same transition probabilities sequence of memberships (PTMS of the embedded Markov chains), they converge in the same expected relative population structure geometrically fast. This is stated in detail in the following theorem which could be proved by just following the analogous steps as in the proof of Theorems 4 and 5.

Theorem 8.

Let two NHMS$\mathit{a}$and$\mathit{b}$which have the same number of states, a common sequence of transition probability matrices of memberships${\left\{\mathit{Q}\left(t\right)\right\}}_{t=0}^{\infty }$and the same total population of memberships. Assume that

$${\left\{\frac{\Delta T\left(t\right)}{T\left(t\right)}\right\}}_{t=0}^{\infty }\mathit{converges}\mathit{geometrically}\mathit{fast}.$$

Let${\mathit{q}}_{\mathit{a}}\left(0\right)$and${\mathit{q}}_{\mathit{b}}\left(0\right)$be their initial relative population structures respectively, and${\left\{{\mathit{p}}_{\mathit{a}}\left(t\right)\right\}}_{t=0}^{\infty }$and${\left\{{\mathit{p}}_{\mathit{b}}\left(t\right)\right\}}_{t=0}^{\infty }$their sequences of allocation probabilities. If${sup}_t$$\tilde{\mathcal{T}}\left(\mathit{Q}\left(t\right)\right)<1$then

$$\underset{t\to \infty }{lim}\mathbb{E}\left({\mathit{q}}_{\mathit{a}}\left(0,t\right)\right)=\underset{t\to \infty }{lim}\mathbb{E}\left({\mathit{q}}_{\mathit{b}}\left(0,t\right)\right)=\mathit{q}\left(\infty \right)\mathit{geometrically}\mathit{fast},$$

where$\mathit{q}\left(\infty \right)$is the row of the stable matrix$\mathit{Q}\left(\infty \right)={lim}_{t\to \infty }\mathit{Q}\left(0,t\right).$

6. Properties of the Limit Set

In the present section we establish some important properties of the limit set $Rn{g}_{\infty }\left(\mathbb{S},{\mathbb{R}}_0\right)$. We start with the following definition:

Definition 6.

Let n be an integer such that$\mathcal{T}\left({\mathit{Q}}_1{\mathit{Q}}_2\dots {\mathit{Q}}_n\right)<1$for all${\mathit{Q}}_1,{\mathit{Q}}_2,\dots ,{\mathit{Q}}_n\in \left[\mathbb{M}\right]$. Then$\left[\mathbb{M}\right]$is said to be product scrambling and n its scrambling integer.

We will make use of the following Theorem 3.3 in [9]:

Theorem 9.

 ([9]). Let $\mathit{x}$, $\mathit{y}$ be non-compact subsets of $S{M}_{1,n}$. Then using the Hausdorff metric we have

$$d\left(\mathit{x}\mathbb{M},\mathit{y}\mathbb{M}\right)\le \mathcal{T}\left(\mathbb{M}\right)d\left(\mathit{x},\mathit{y}\right).$$

We will now establish with a Theorem some important properties of the limit set.

Theorem 10.

Consider an NHMSS for which$\left[\mathbb{M}\right]$is an interval which is uniformly scrambling. Assume that$T\left(t\right)\ge T\left(t-1\right)>0$for all$t=1,2,\dots$and

$${\left\{\frac{\Delta T\left(t\right)}{T\left(t\right)}\right\}}_{t=0}^{\infty }\mathit{converges}\mathit{to}\mathit{zero}\mathit{geometrically}\mathit{fast}.$$

Also, assume that$\left[{\mathbb{S}}_0\right],\left[{\mathbb{R}}_0\right]$are compact and convex. Then if we define by

$$Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)=\underset{t\to \infty }{lim}Rn{g}_t\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)={\cap }_{t=1}^{\infty }Rn{g}_t\left({\mathbb{S}}_0,{\mathbb{R}}_0\right),$$

the limit set$Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)$satisfies

$$Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)=Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\left[\mathbb{M}\right].$$

If in addition$\left[\mathbb{M}\right]$is product scrambling with integer n then it is the unique set that has this property.

Proof. 

For the first part of the Theorem 10 since $Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)$ is compact, it is sufficient to show that

$$d\left(Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right),Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\left[\mathbb{M}\right]\right)=0.$$

In this respect

$$\delta \left(Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right),Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\left[\mathbb{M}\right]\right)=$$

$$\underset{\begin{array}{c}\mathit{q}\left(0\right)\in \left[{\mathbb{S}}_0\right],\\ {\mathit{p}}_0\left(\tau \right)\in \left[{\mathbb{R}}_0\right]\\ \mathit{Q}\left(t\right)\in \left[\mathbb{M}\right]\mathrm{for}\mathrm{all}\tau ,t\end{array}}{max}\underset{{}_{\begin{array}{c}\mathit{q}\left(0\right)\in \left[{\mathbb{S}}_0\right],\\ {\mathit{p}}_0\left(\tau \right)\in \left[{\mathbb{R}}_0\right]\\ \mathit{Q}\left(t\right)\in \left[\mathbb{M}\right]\mathrm{for}\mathrm{all}\tau ,t\end{array}}}{min}∥\underset{t\to \infty }{lim}\left\{\mathbb{E}\left[\mathit{q}\left(0,t+1\right)\right]\right\}-\underset{t\to \infty }{lim}\left\{\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]\right\}\left[\mathbb{M}\right]∥,$$

where

$$\left\{\mathbb{E}\left[\mathit{q}\left(0,t+1\right)\right]\right\}={\cup }_{\mathit{q}\left(0\right)\in \left[{\mathbb{S}}_0\right]}\frac{T\left(0\right)}{T\left(t+1\right)}\mathit{q}\left(0\right)\left[{\mathbb{M}}^{t+1}\right]$$

$$+\frac{1}{T\left(t+1\right)}\sum _{\tau =1}^{t+1}\Delta T\left(\tau \right){\cup }_{{\mathit{p}}_0\left(\tau \right)\in \left[{\mathbb{R}}_0\right]}{\mathit{p}}_0\left(\tau \right)\left[{\mathbb{M}}^{t-\tau +1}\right].$$

Now there exists a ${\mathit{q}}^{*}\left(0\right)\in$$\left[{\mathbb{S}}_0\right],{\mathit{Q}}^{*}\left(t\right)\in \left[\mathbb{M}\right]$ for $t\in \left[0,t\right]$, ${\mathit{p}}_0\left(\tau \right)\in \left[{\mathbb{R}}_0\right]$ for $\tau \in \left[1,t\right]$, i.e.,

$$\mathbb{E}\left[\mathit{q}\left(0,t+1\right)\right]=\frac{T\left(0\right)}{T\left(t+1\right)}{\mathit{q}}^{*}\left(0\right){\mathit{Q}}^{*}\left(0,t+1\right)+$$

$$\frac{1}{T\left(t+1\right)}\sum _{\tau =1}^{t+1}\Delta T\left(\tau \right){\mathit{p}}_0^{*}\left(\tau \right){\mathit{Q}}^{*}\left(\tau ,t+1\right),$$

such that

$$\delta \left(Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right),Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\left[\mathbb{M}\right]\right)=$$

$$\underset{\underset{\mathit{Q}\left(t\right)\in \left[\mathbb{M}\right]\mathrm{for}\mathrm{all}\tau ,t}{\mathit{q}\left(0\right)\in \left[{\mathbb{S}}_0\right],{\mathit{p}}_0\left(\tau \right)\in \left[{\mathbb{R}}_0\right],}}{min}∥\underset{t\to \infty }{lim}\mathbb{E}\left[{\mathit{q}}^{*}\left(0,t+1\right)\right]-\underset{t\to \infty }{lim}\left\{\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]\right\}\left[\mathbb{M}\right]∥.$$

(24)

Now, for any values of the parameters of $\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]$ that does not maximize it, the difference above is still greater or equal than the ones which minimize $\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]$. Thus, we are free to choose the parameters ${\mathit{q}}^{*}\left(0\right)\in \left[{\mathbb{S}}_0\right]$, ${\mathit{Q}}^{*}\left(0,t\right)\in \left[{\mathbb{M}}^t\right]$ and ${\mathit{p}}_0^{*}\left(\tau \right)\in \left[{\mathbb{R}}_0\right]$ for every $\tau \in \left[0,t\right]$. With the same reasoning we could choose ${\mathit{Q}}^{*}\left(t\right)$ in the place of $\left[\mathbb{M}\right]$. Thus, we get that

$$\delta \left(Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right),Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\left[\mathbb{M}\right]\right)=$$

$$\le \underset{t\to \infty }{lim}∥\frac{T\left(0\right)}{T\left(t+1\right)}{\mathit{q}}^{*}\left(0\right){\mathit{Q}}^{*}\left(0,t+1\right)-\frac{T\left(0\right)}{T\left(t\right)}{\mathit{q}}^{*}\left(0\right){\mathit{Q}}^{*}\left(0,t+1\right)∥$$

$$+\underset{t\to \infty }{lim}\Vert \frac{1}{T\left(t+1\right)}\sum _{\tau =1}^{t+1}\Delta T\left(\tau \right){\mathit{p}}_0^{*}\left(\tau \right){\mathit{Q}}^{*}\left(\tau ,t+1\right)-$$

$$\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right){\mathit{p}}_0^{*}\left(\tau \right){\mathit{Q}}^{*}\left(\tau ,t+1\right)\Vert$$

$$\le \underset{t\to \infty }{lim}\left|\frac{T\left(0\right)}{T\left(t+1\right)}-\frac{T\left(0\right)}{T\left(t\right)}\right|∥{\mathit{q}}^{*}\left(0\right)∥1∥{\mathit{Q}}^{*}\left(0,t+1\right)∥+$$

$$\underset{t\to \infty }{lim}\left|\frac{1}{T\left(t+1\right)}-\frac{1}{T\left(t\right)}\right|∥\sum _{\tau =1}^t\Delta T\left(\tau \right){\mathit{p}}_0^{*}\left(\tau \right){\mathit{Q}}^{*}\left(\tau ,t+1\right)∥$$

$$+\underset{t\to \infty }{lim}\left|\frac{\Delta T\left(t\right)}{T\left(t+1\right)}\right|∥{\mathit{p}}_0^{*}\left(t\right)∥.$$

(25)

Since $\left[\mathbb{M}\right]$ is uniformly scrambling we have that $\overline{\mathcal{T}}=ma{x}_{t\in \mathbb{N}}\mathcal{T}\left({\mathit{Q}}^{*}\left(t\right)\right)<1$ and thus $∥{\mathit{Q}}^{*}\left(\tau ,t\right)∥<{\overline{\mathcal{T}}}^{t-\tau }$ and

$$∥\sum _{\tau =1}^t\Delta T\left(\tau \right){\mathit{p}}_0^{*}\left(\tau \right){\mathit{Q}}^{*}\left(\tau ,t+1\right)∥\le \sum _{\tau =1}^t\Delta T\left(\tau \right){\overline{\mathcal{T}}}^{t-\tau +1},$$

which goes to zero as $t\to \infty$. Hence

$$\delta \left(Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right),Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\left[\mathbb{M}\right]\right)=0.$$

In a similar way we could prove that

$$\delta \left(Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\left[\mathbb{M}\right],Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\right)=0,$$

which lead us to the conclusion

$$d\left(Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\left[\mathbb{M}\right],Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\right)=0.$$

For the second part of the theorem in addition we have that $\left[\mathbb{M}\right]$ is product scrambling with index $n.$ Assume that there is a second $Rn{g}_{\infty }^{*}\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)$ which is compact for which

$$Rn{g}_{\infty }^{*}\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\left[\mathbb{M}\right]=Rn{g}_{\infty }^{*}\left({\mathbb{S}}_0,{\mathbb{R}}_0\right).$$

Then we get that

$$d\left(Rn{g}_{\infty }^{*}\left({\mathbb{S}}_0,{\mathbb{R}}_0\right),Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\right)=d\left(Rn{g}_{\infty }^{*}\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\left[{\mathbb{M}}^n\right],Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\left[{\mathbb{M}}^n\right]\right)$$

$$\le \left(\mathrm{by}\mathrm{Theorem}9\right)$$

$$\le \mathcal{T}\left[{\mathbb{M}}^n\right]d\left(Rn{g}_{\infty }^{*}\left({\mathbb{S}}_0,{\mathbb{R}}_0\right),Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\right),$$

and since $\mathcal{T}\left[{\mathbb{M}}^n\right]<1$ we get

$$d\left(Rn{g}_{\infty }^{*}\left({\mathbb{S}}_0,{\mathbb{R}}_0\right),Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)\right)=0$$

from which we conclude that $Rn{g}_{\infty }^{*}\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)$ and $Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)$ are the same set. □

We will now establish an interesting result, that under certain conditions if we have two different inherent Markov set chains for two NHMSS, then the Hausdorff metric of the two sets of all possible expected relative structures of the NHMSS is less than the multiplication of a function of the common bound of the two uniform coefficients of ergodicity of the two intervals and the Hausdorff metric of the two intervals. We will need the following Lemma from ([9] p. 70), or [52].

Lemma 8.

Let$\mathit{q}\left(0\right),$$\tilde{\mathit{q}}\left(0\right)$be stochastic vectors and$\mathit{Q}\left(t\right)\in \left[\mathbb{M}\right]$; $\tilde{\mathit{Q}}\left(t\right)\in [\tilde{\mathbb{M}}]$for$t=1,2,\dots$for$\mathcal{T}\left(\mathbb{M}\right)\le \mathcal{T}<1$and$\mathcal{T}(\tilde{\mathbb{M}})\le \mathcal{T}<1$. Then

$$∥\mathit{q}\left(0\right)\mathit{Q}\left(1,t\right)-\tilde{\mathit{q}}\left(0\right)\tilde{\mathit{Q}}\left(1,t\right)∥\le {\mathcal{T}}^t∥\mathit{q}\left(0\right)-\tilde{\mathit{q}}\left(0\right)∥1+\left({\mathcal{T}}^{t-1}+\dots +1\right)\mathit{D}$$

where$\mathit{D}={max}_n∥\mathit{Q}\left(n\right)-\tilde{\mathit{Q}}\left(n\right)∥.$

Theorem 11.

Let two NHMSS with inherent Markov set chains with two different tight intervals$\left[\mathbb{M}\right]$and$[$which have a common bound, i.e.,$\mathcal{T}\left(\mathbb{M}\right)\le \mathcal{T}<1$and$\mathcal{T}(\tilde{\mathbb{M}})\le \mathcal{T}<1$. Let that for the first NHMSS we have$\mathit{q}\left(0\right)\in \left[{\mathbb{S}}_1\right]$and${\mathit{p}}_0\left(\tau \right)\in \left[{\mathbb{R}}_1\right]$for$\tau =1,\dots ,t$where$\left[{\mathbb{S}}_1\right]$, $\left[{\mathbb{R}}_1\right]$are convex and compact. For the second NHMSS we assume that$\tilde{\mathit{q}}\left(0\right)\in \left[{\mathbb{S}}_2\right]$and${\tilde{\mathit{p}}}_0\left(\tau \right)\in \left[{\mathbb{R}}_2\right]$for$\tau =1,\dots ,t$where$\left[{\mathbb{S}}_2\right]$, $\left[{\mathbb{R}}_2\right]$are convex and compact. Let that the two NHMSS have common total population of memberships${\left\{T\left(t\right)\right\}}_{t=0}^{\infty }$which is known. Assume that$T\left(t\right)\ge T\left(t-1\right)>0$and

$${\left\{\frac{\Delta T\left(t\right)}{T\left(t\right)}\right\}}_{t=0}^{\infty }\mathit{converges}\mathit{to}\mathit{zero}\mathit{geometrically}\mathit{fast}.$$

Denote by$Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1,\mathbb{M}\right)$the set of all possible expected relative population structures for the first NHMSS and$Rn{g}_t({\mathbb{S}}_2,{\mathbb{R}}_2,\tilde{\mathbb{M}})$for the second one, respectively. Then the Hausdorff metric of the two limit sets of expected relative population structures is bounded by

$$d(Rn{g}_{\infty }\left({\mathbb{S}}_1,{\mathbb{R}}_1,\mathbb{M}\right),Rn{g}_{\infty }({\mathbb{S}}_2,{\mathbb{R}}_2,\tilde{\mathbb{M}}))\le {\left(1-\mathcal{T}\right)}^{-1}d(\mathbb{M},\tilde{\mathbb{M}}).$$

Proof. 

From Lemma 4 we get that

$$Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1,\mathbb{M}\right)=\{\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]:\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]\in \frac{T\left(0\right)}{T\left(t\right)}{\cup }_{\mathit{q}\left(0\right)\in \left[{\mathbb{S}}_1\right]}\mathit{q}\left(0\right)\left[{\mathbb{M}}^t\right]+$$

$$+\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right){\cup }_{{\mathit{p}}_0\left(\tau \right)\in \left[{\mathbb{R}}_1\right]}{\mathit{p}}_0\left(\tau \right)\left[{\mathbb{M}}^{t-\tau }\right]\},$$

(26)

and an analogous description is valid for the set $Rn{g}_t({\mathbb{S}}_2,{\mathbb{R}}_2,\tilde{\mathbb{M}}).$ Since $\left[{\mathbb{S}}_1\right]$, $\left[{\mathbb{R}}_1\right]$, $\left[{\mathbb{S}}_2\right]$, $\left[{\mathbb{R}}_2\right]$ are convex and compact and $\left[\mathbb{M}\right]$ and $[$ are tight intervals from Theorem 1 we get that the sets $Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1,\mathbb{M}\right)$ and $Rn{g}_t({\mathbb{S}}_2,{\mathbb{R}}_2,\tilde{\mathbb{M}})$ are convex and compact, hence there is a meaning to get their Hausdorff metric

$$d(Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1,\mathbb{M}\right),Rn{g}_t({\mathbb{S}}_2,{\mathbb{R}}_2,\tilde{\mathbb{M}}))=$$

$$max\{\delta (Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1,\mathbb{M}\right),Rn{g}_t({\mathbb{S}}_2,{\mathbb{R}}_2,\tilde{\mathbb{M}})),$$

(27)

$$\delta (Rn{g}_t({\mathbb{S}}_2,{\mathbb{R}}_2,\tilde{\mathbb{M}}),Rn{g}_t({\mathbb{S}}_1,{\mathbb{R}}_1,\mathbb{M}))\}.$$

Now, we have that

$$\delta (Rn{g}_t({\mathbb{S}}_1,{\mathbb{R}}_1,\mathbb{M}),Rn{g}_t\left({\mathbb{S}}_2,{\mathbb{R}}_2,\tilde{\mathbb{M}}\right))=$$

$$\underset{\underset{\mathit{Q}\left(t\right)\in \left[\mathbb{M}\right]\mathrm{for}\mathrm{all}\tau ,t}{\mathit{q}\left(0\right)\in \left[{\mathbb{S}}_1\right],{\mathit{p}}_0\left(\tau \right)\in \left[{\mathbb{R}}_1\right],}}{max}\underset{\underset{\tilde{\mathit{Q}}\left(t\right)\in \left[\tilde{\mathbb{M}}\right]\mathrm{for}\mathrm{all}\tau ,t}{\tilde{\mathit{q}}\left(0\right)\in \left[{\mathbb{S}}_2\right],{\tilde{\mathit{p}}}_0\left(\tau \right)\in \left[{\mathbb{R}}_2\right],}}{min}∥\mathbb{E}\left[\mathit{q}\left(0,t\right)\right]-\mathbb{E}\left[\tilde{\mathit{q}}\left(0,t\right)\right]∥.$$

(28)

There exists some ${\mathit{q}}^{*}\left(0\right)\in \left[{\mathbb{S}}_1\right],{\mathit{p}}_0^{*}\left(\tau \right)$ for every $\tau \in \left[1,t\right]$, ${\mathit{Q}}^{*}\left(t\right)\in \left[\mathbb{M}\right]$ for every $t\in \left[0,t\right]$ which determines a $\mathbb{E}\left[{\mathit{q}}^{*}\left(0,t\right)\right];$ also for any ${\tilde{\mathit{q}}}^{*}\left(0\right)\in \left[{\mathbb{S}}_2\right]$, any ${\tilde{\mathit{p}}}_0^{*}\left(\tau \right)$ for every $\tau \in \left[1,t\right]$, and finally any ${\tilde{\mathit{Q}}}^{*}\left(t\right)\in \left[\tilde{\mathbb{M}}\right]$ for every $t\in \left[0,t\right]$ which determines a $\mathbb{E}\left[{\tilde{\mathit{q}}}^{*}\left(0,t\right)\right]$ for which we have

$$\delta (Rn{g}_t\left({\mathbb{S}}_1,{\mathbb{R}}_1,\mathbb{M}\right),Rn{g}_t({\mathbb{S}}_2,{\mathbb{R}}_2,\tilde{\mathbb{M}}))\le ∥\mathbb{E}\left[{\mathit{q}}^{*}\left(0,t\right)\right]-\mathbb{E}\left[{\tilde{\mathit{q}}}^{*}\left(0,t\right)\right]∥$$

$$\le \frac{T\left(0\right)}{T\left(t\right)}∥{\mathit{q}}^{*}\left(0\right){\mathit{Q}}^{*}\left(0,t\right)-{\tilde{\mathit{q}}}^{*}\left(0\right){\overline{\mathit{Q}}}^{*}\left(0,t\right)∥$$

(29)

$$+\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right)∥{\mathit{p}}_0^{*}\left(\tau \right){\mathit{Q}}^{*}\left(\tau ,t\right)-{\tilde{\mathit{p}}}_0^{*}\left(\tau \right){\tilde{\mathit{Q}}}^{*}\left(\tau ,t\right)∥.$$

Using Lemma 8 we get that

$$\frac{T\left(0\right)}{T\left(t\right)}∥{\mathit{q}}^{*}\left(0\right){\mathit{Q}}^{*}\left(0,t\right)-{\tilde{\mathit{q}}}^{*}\left(0\right){\tilde{\mathit{Q}}}^{*}\left(0,t\right)∥\le$$

$$\frac{T\left(0\right)}{T\left(t\right)}{\mathcal{T}}^t∥{\mathit{q}}^{*}\left(0\right)-{\tilde{\mathit{q}}}^{*}\left(0\right)∥+\frac{T\left(0\right)}{T\left(t\right)}∥{\mathcal{T}}^{t-1}+\dots +1∥\underset{t}{max}∥{\mathit{Q}}^{*}\left(t\right)-{\tilde{\mathit{Q}}}^{*}\left(t\right)∥$$

$$\le \frac{T\left(0\right)}{T\left(t\right)}{\mathcal{T}}^t\delta \left({\mathbb{S}}_1,{\mathbb{S}}_2\right)+\frac{T\left(0\right)}{T\left(t\right)}∥{\mathcal{T}}^{t-1}+\dots +1∥\delta \left(\mathbb{M},\tilde{\mathbb{M}}\right).$$

(30)

Similarly, we have

$$\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right)∥{\mathit{p}}_0^{*}\left(\tau \right){\mathit{Q}}^{*}\left(\tau ,t\right)-{\tilde{\mathit{p}}}_0^{*}\left(\tau \right){\tilde{\mathit{Q}}}^{*}\left(\tau ,t\right)∥$$

$$\le \frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right)\delta \left({\mathbb{R}}_1,{\mathbb{R}}_2\right){\mathcal{T}}^{t-\tau }+$$

$$\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right)\left({\mathcal{T}}^{t-1}+\dots +1\right)\delta \left(\mathbb{M},\tilde{\mathbb{M}}\right).$$

(31)

From $\left(29\right),\left(30\right)$ and $\left(31\right)$ as $t\to \infty$ we get that

$$\delta (Rn{g}_{\infty }\left({\mathbb{S}}_1,{\mathbb{R}}_1,\mathbb{M}\right),Rn{g}_{\infty }({\mathbb{S}}_2,{\mathbb{R}}_2,\tilde{\mathbb{M}}))\le {\left(1-\mathcal{T}\right)}^{-1}\delta (\mathbb{M},\tilde{\mathbb{M}}),$$

(32)

since

$$\underset{t\to \infty }{lim}\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right)\left({\mathcal{T}}^{t-1}+\dots +1\right)\delta \left(\mathbb{M},\tilde{\mathbb{M}}\right)$$

$$=\underset{t\to \infty }{lim}\frac{1}{T\left(t\right)}\delta \left(\mathbb{M},\tilde{\mathbb{M}}\right)\sum _{\tau =1}^t\Delta T\left(\tau \right)\sum _{k=\tau }^{t-1}{\mathcal{T}}^{t-k}\le$$

$$\underset{t\to \infty }{lim}\frac{1}{T\left(t\right)}{\left(1-\mathcal{T}\right)}^{-1}\delta \left(\mathbb{M},\tilde{\mathbb{M}}\right)\sum _{\tau =1}^t\Delta T\left(\tau \right)=\left[1-\frac{T\left(0\right)}{T}\right]{\left(1-\mathcal{T}\right)}^{-1}\delta \left(\mathbb{M},\tilde{\mathbb{M}}\right),$$

also

$$\underset{t\to \infty }{lim}\frac{T\left(0\right)}{T\left(t\right)}{\mathcal{T}}^t\delta \left({\mathbb{S}}_1,{\mathbb{S}}_2\right)+\underset{t\to \infty }{lim}\frac{T\left(0\right)}{T\left(t\right)}∥{\mathcal{T}}^{t-1}+\dots +1∥1\delta \left(\mathbb{M},\tilde{\mathbb{M}}\right)$$

$$=\frac{T\left(0\right)}{T}{\left(1-\mathcal{T}\right)}^{-1}\delta \left(\mathbb{M},\tilde{\mathbb{M}}\right)$$

and finally

$$\underset{t\to \infty }{lim}\frac{1}{T\left(t\right)}\sum _{\tau =1}^t\Delta T\left(\tau \right)\delta \left({\mathbb{R}}_1,{\mathbb{R}}_2\right){\mathcal{T}}^{t-\tau +1}=0,$$

as we have seen in the proof of Theorem 8. Similarly, as we proved $\left(32\right)$ we could prove that

$$\delta (Rn{g}_{\infty }({\mathbb{S}}_2,{\mathbb{R}}_2,\tilde{\mathbb{M}}),Rn{g}_{\infty }\left({\mathbb{S}}_1,{\mathbb{R}}_1,\mathbb{M}\right))\le {\left(1-\mathcal{T}\right)}^{-1}\delta (\tilde{\mathbb{M}},\mathbb{M}).$$

(33)

From $\left(32\right)$ and $\left(33\right)$ we arrive at the conclusion of the Theorem. □

7. An Illustrative Representative Example

In the present section we present an illustrative representative example to a Geriatric and Stroke Patients system and through it we will present the methodology in terms of computational geometry algorithms needed for an application to any population system. The NHMS model used is a general Coxian phase type model, special forms of which has been used by the school of research by McClean and her co-authors [38,53,54,55,56,57,58,59]. We distinguish three states which are called hospital pathways. For the system of Geriatric and Stroke Patients these stages are labeled as “Acute Care”, the “Rehabilitative” and the “Long Stay”. From each stay we have movements outside the hospital due to discharge or death. Also, geriatric patients may be thought of as progressing through stages of acute care, rehabilitation and long-stay care, where most patients are eventually rehabilitated and discharged. Geriatric medical services are an important asset in the care of elderly and their quality is certainly an indication of the level of civilization in a society. At the same time their funding could be easily reduced due to political pressure on savings in health care expenditure.

It is apparent that the number of pathways could be increased as much as it is needed to accommodate any important characteristics of any patients systems. However, there is no need to consider in here a larger number of states due to the restriction of space. Also, the internal movements in a population of patients could be of any number to accommodate any important characteristics.

Consider a hospital which starts with $T\left(0\right)=400$ patients and in a very short time reaches its full capacity of 435 patients, i.e., $T\left(1\right)=420$, $T\left(2\right)=430$, $T\left(3\right)=435$. Assume three hospital pathways and let that the initial relative population structure be any stochastic vector which lies in the set

$${\mathbb{S}}_0=\left\{\left[000\right],\left[111\right]\right\}.$$

The physical meaning of selecting ${\mathbb{S}}_0$ as above is that the initial relative structure could be any stochastic vector, i.e., ${\mathbb{S}}_0$ contains all possible initial structures. For example $\mathit{q}\left(0\right)=\left[\begin{array}{ccc}0.2& 0.3& 0.5\end{array}\right]$ means that $20\%$ of the patients are in pathway 1, $30\%$ are in pathway 2, and $50\%$ are in pathway 3. Now, there are some initial structures which might not be acceptable for the management of the hospital, such as for example let say $\mathit{q}\left(0\right)=\left[\begin{array}{ccc}0& 0& 1.0\end{array}\right]$. In this case in the initial design of the hospital measures should be taken that such a situation will be avoided in cooperation with other nearby hospitals. Then ${\mathbb{S}}_0$ could be chosen to be

$${\mathbb{S}}_0=\left\{\left[000\right],\left[0.110.9\right]\right\}.$$

This is a convex set which excludes the initial relative structure $\mathit{q}\left(0\right)=\left[\begin{array}{ccc}0& 0& 1.0\end{array}\right]$. However, it also needs to be tight. How to make it tight is explained below with the use of the Algorithm 1. Naturally, we could exclude more than one relative structure from the chosen initial relative structure but the procedure will be the same.

Most new patients enter the system in hospital pathway one, either by taking an empty place or as a virtual replacement of a discharged patient. Hence, let ${\mathbb{R}}_0$ be the convex set from which allocation probabilities are drawn and let it be the interval

$${\mathbb{R}}_0=\left\{\left[0.50.20.1\right],\left[0.70.40.1\right]\right\}.$$

The set of stochastic vectors of allocation probabilities that are in the above interval is a convex set with vertices $r_1=\left[0.70.20.1\right]$ and $r_2=\left[0.50.40.1\right]$. How to find the vertices will be explained below. The physical meaning of the above interval is that any stochastic vector that belongs in the interval ${\mathbb{R}}_0$ is a candidate to represent a registration policy for the hospital. Naturally, we can restrict our interval ${\mathbb{R}}_0$ to an interval which will contain the desired recruitment policies of the hospital management and find in this way using the results of the present paper the consequences of these policies. The recruitment vectors are the best control variables for human populations ([2]), as are the hospitals in this case. Methods of control by recruitment could be found in [50,60,61,62]. However, the interval ${\mathbb{R}}_0$ needs to be tight and how to make it tight is explained below with the use of the Algorithm 1.

Now, by observing past data it is not difficult to determine an interval of matrices $\left[\mathbb{M}\right]=\left[\check{\mathit{Q}},\widehat{\mathit{Q}}\right]$ where all stochastic transition matrices of the movements of memberships lie. Please note that the fact that the matrices $\check{\mathit{Q}},\widehat{\mathit{Q}}$ are not necessarily stochastic matrices makes this task easy. We need that $\left[\mathbb{M}\right]=\left[\check{\mathit{Q}},\widehat{\mathit{Q}}\right]$ should be tight. In order to test that interval $\left[\mathbb{M}\right]$ is tight we use Lemma 1 and Definition 5

We chose $\left[\mathbb{M}\right]$ to be

$$\check{\mathit{Q}}=\left(\begin{array}{ccc}0.5& 0.2& 0.1\\ 0.2& 0.6& 0.2\\ 0.3& 0.2& 0.3\end{array}\right)\mathrm{and}\widehat{\mathit{Q}}=\left(\begin{array}{ccc}0.7& 0.4& 0.1\\ 0.2& 0.6& 0.2\\ 0.5& 0.5& 0.3\end{array}\right),$$

in order that $\left[\mathbb{M}\right]$ is tight every row should be tight hence using Lemma 1 we could see that the interval is tight. Hence, the stochastic matrices that belong to $\left[\mathbb{M}\right]$ is a convex polytope and we need to find its vertices. The same applies for ${\mathbb{S}}_0$ and ${\mathbb{R}}_0$ which are tight, and we need to find the vertices of the convex polytope on which all stochastic vectors of the interval lie.

Applying Lemma 1 we find that the set ${\mathbb{S}}_0$ is tight and applying Algorithm 1 we get that ${\mathbb{S}}_0$ is convex with vertices ${\upsilon }_1=\left(100\right)$, ${\upsilon }_2=\left(010\right)$ and ${\upsilon }_3=\left(001\right)$. The physical meaning of the set ${\mathbb{S}}_0$ in here is that at the start of our study at time 0 we allow that the hospital could have any relative population structure. Applying Lemma 1 we find that the set ${\mathbb{R}}_0$ is tight and applying Algorithm 1 we get that ${\mathbb{R}}_0$ is convex with vertices ${\mathit{r}}_1=\left(0.70.20.1\right)$, ${\mathit{r}}_2=\left(0.50.40.1\right)$.

Algorithm 1 ([9]) Finding the vertices of a tight interval$\left[\mathit{p},\mathit{q}\right].$

For each $i=1,2,3$ construct the vectors:   For $i=1$ and the vector $\mathit{p}$ $$\left[p_1{p}_2{p}_3\right],\left[p_1{p}_2{q}_3\right],\left[p_1{q}_2{p}_3\right],\left[p_1{q}_2{q}_3\right],$$   replace $p_1$ with ${\tilde{p}}_1$ such that the above vectors will become stochastic.   If any of the four resulting stochastic vectors $$\left[{\tilde{p}}_1{p}_2{p}_3\right],\left[{\tilde{p}}_1{p}_2{q}_3\right],\left[{\tilde{p}}_1{q}_2{p}_3\right],\left[{\tilde{p}}_1{q}_2{q}_3\right],$$   belongs in the interval $\left[\mathit{p},\mathit{q}\right]$ then it is a vertex. Do the same for the vector $\mathit{q}$. END.

To find the vertices of the convex set of stochastic matrices that belong to the tight interval $\left[\mathbb{M}\right]$ we apply Algorithm 1 for each row vector in $\left[\check{\mathit{Q}},\widehat{\mathit{Q}}\right]$ and we find that the vertices are

$${\mathit{V}}_1=\left(\begin{array}{ccc}0.7& 0.2& 0.1\\ 0.2& 0.6& 0.2\\ 0.5& 0.2& 0.3\end{array}\right),{\mathit{V}}_2=\left(\begin{array}{ccc}0.7& 0.2& 0.1\\ 0.2& 0.6& 0.2\\ 0.3& 0.4& 0.3\end{array}\right),$$

$${\mathit{V}}_3=\left(\begin{array}{ccc}0.5& 0.4& 0.1\\ 0.2& 0.6& 0.2\\ 0.5& 0.2& 0.3\end{array}\right),{\mathit{V}}_4=\left(\begin{array}{ccc}0.5& 0.4& 0.1\\ 0.2& 0.6& 0.2\\ 0.3& 0.4& 0.3\end{array}\right).$$

Now we compute all the row vectors ${\upsilon }_i{\mathit{V}}_j$ for $i=1,2,3$ and $j=1,2,3,4:$

$$\left(\begin{array}{ccc}0.7& 0.2& 0.1\end{array}\right),\left(\begin{array}{ccc}0.7& 0.2& 0.1\end{array}\right),\left(\begin{array}{ccc}0.5& 0.4& 0.1\end{array}\right),\left(\begin{array}{ccc}0.5& 0.4& 0.1\end{array}\right),$$

$$\left(\begin{array}{ccc}0.2& 0.6& 0.2\end{array}\right),\left(\begin{array}{ccc}0.2& 0.6& 0.2\end{array}\right),\left(\begin{array}{ccc}0.2& 0.6& 0.2\end{array}\right),\left(\begin{array}{ccc}0.2& 0.6& 0.2\end{array}\right)$$

$$\left(\begin{array}{ccc}0.5& 0.2& 0.3\end{array}\right),\left(\begin{array}{ccc}0.3& 0.4& 0.5\end{array}\right),\left(\begin{array}{ccc}0.5& 0.2& 0.3\end{array}\right),\left(\begin{array}{ccc}0.3& 0.4& 0.3\end{array}\right).$$

These 12 stochastic vectors belong to a convex set, hence using any of the computational geometry methods in [61] we find that the vertices of the convex hull of these vectors are

$${\omega }_1=\left(\begin{array}{ccc}0.7& 0.2& 0.1\end{array}\right),{\omega }_2=\left(\begin{array}{ccc}0.5& 0.4& 0.1\end{array}\right),{\omega }_1=\left(\begin{array}{ccc}0.2& 0.6& 0.2\end{array}\right),$$

$${\omega }_4=\left(\begin{array}{ccc}0.5& 0.2& 0.3\end{array}\right),{\omega }_5=\left(\begin{array}{ccc}0.3& 0.4& 0.3\end{array}\right).$$

Hence

$$Rn{g}_1\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)=\frac{T\left(0\right)}{T\left(1\right)}conv\left\{{\omega }_1,{\omega }_2,{\omega }_3,{\omega }_4,{\omega }_5\right\}+$$

$$\frac{1}{T\left(1\right)}\left[T\left(1\right)-T\left(0\right)\right]conv\left\{r_1,r_2\right\},$$

from which we get that $Rn{g}_1\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)$ is the convex set with vertices given by the above Minkowski sum thus

$$\left(\begin{array}{ccc}0.7& 0.2& 0.1\end{array}\right),\left(\begin{array}{ccc}0.51& 0.39& 0.1\end{array}\right),\left(\begin{array}{ccc}0.22& 0.58& 0.2\end{array}\right),\left(\begin{array}{ccc}0.51& 0.2& 0.29\end{array}\right),$$

$$\left(\begin{array}{ccc}0.32& 0.39& 0.29\end{array}\right),\left(\begin{array}{ccc}0.69& 0.21& 0.1\end{array}\right),\left(\begin{array}{ccc}0.5& 0.4& 0.1\end{array}\right),$$

$$\left(\begin{array}{ccc}0.21& 0.59& 0.2\end{array}\right),\left(\begin{array}{ccc}0.5& 0.21& 0.29\end{array}\right),\left(\begin{array}{ccc}0.31& 0.4& 0.29\end{array}\right),$$

$$\left(\begin{array}{ccc}0.51& 0.2& 0.29\end{array}\right).$$

Taking into account the rounding errors done with all the multiplications and additions we compute the vertices with one decimal point of accuracy and we get that $Rn{g}_1\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)$ is the convex hull of the vertices

$$\left(\begin{array}{ccc}0.7& 0.2& 0.1\end{array}\right),\left(\begin{array}{ccc}0.5& 0.4& 0.1\end{array}\right),\left(\begin{array}{ccc}0.2& 0.6& 0.2\end{array}\right),\left(\begin{array}{ccc}0.5& 0.2& 0.3\end{array}\right),$$

$$\mathrm{and}\left(\begin{array}{ccc}0.3& 0.4& 0.3\end{array}\right).$$

This result verifies Theorem 1. With now the vertices of the convex set $Rn{g}_1\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)$ we repeat the previous process to find $Rn{g}_2\left({\mathbb{S}}_0,{\mathbb{R}}_0\right).$

Now, in this way at every point in time we have the convex compact space $Rn{g}_t\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)$ of all possible expected population structures. If any of these are problematic in some way then apparently the hospital has a lead time to adapt new policies and an instrument to visualize their consequences.

To verify that a tight interval of transition probability matrices is uniformly scrambling we need a sufficient condition as a criterion. This is given in the following easily proved Lemma.

Lemma 9.

Let$\left[\mathbb{M}\right]=\left[\check{\mathit{Q}},\widehat{\mathit{Q}}\right]$then$\left[\mathbb{M}\right]$is uniformly scrambling if the following holds$\mathcal{T}(\widehat{\mathit{Q}}-\check{\mathit{Q}})<1.$

Let $\left[\mathbb{M}\right]$ be the interval of the application we are working so far, then it is easy to check that $\mathcal{T}(\widehat{\mathit{Q}}-\check{\mathit{Q}})<1$ and hence any stochastic matrix selected from $\left[\mathbb{M}\right]$ will be scrambling. We select as $\mathit{Q}\left(1\right)$, $\mathit{Q}\left(2\right)$, $\mathit{Q}\left(3\right),$$\mathit{Q}\left(4\right)$ the four vertices of the convex set of stochastic matrices in $\left[\mathbb{M}\right]$ and as $\mathit{Q}\left(6\right)$, $\mathit{Q}\left(7\right)$ any convex combination of them. We also select as $\mathit{q}\left(0\right)=\left(\begin{array}{ccc}0.5& 0.25& 0.25\end{array}\right)$ and as vectors of allocation probabilities we select ${\mathit{p}}_0\left(1\right)=\left(\begin{array}{ccc}0.6& 0.3& 0.1\end{array}\right)$ and ${\mathit{p}}_0\left(2\right)=\left(\begin{array}{ccc}0.5& 0.4& 0.1\end{array}\right)$ which both belong to ${\mathbb{R}}_0$. Then we compute

$$\mathbb{E}\left[\mathit{q}\left(0,1\right)\right]=\left(\begin{array}{ccc}0.5& 0.3& 0.2\end{array}\right)$$

$$\mathbb{E}\left[\mathit{q}\left(0,2\right)\right]=\left(\begin{array}{ccc}0.48& 0.36& 0.16\end{array}\right)$$

$$\mathbb{E}\left[\mathit{q}\left(0,3\right)\right]=\left(\begin{array}{ccc}0.4& 0.43& 0.17\end{array}\right)$$

$$\mathbb{E}\left[\mathit{q}\left(0,4\right)\right]=\left(\begin{array}{ccc}0.34& 0.48& 0.18\end{array}\right)$$

$$\mathbb{E}\left[\mathit{q}\left(0,5\right)\right]=\left(\begin{array}{ccc}0.35& 0.46& 0.19\end{array}\right)$$

$$\mathbb{E}\left[\mathit{q}\left(\infty \right)\right]=\left(\begin{array}{ccc}0.4& 0.4& 0.2\end{array}\right).$$

Hence the expected relative population structure converges in six steps, that is geometrically fast which verifies Theorem 6. Also, it is easy to see that

$$\mathbb{E}\left[\mathit{q}\left(\infty \right)\right]{\mathit{V}}_1=\left(\begin{array}{ccc}0.4& 0.4& 0.2\end{array}\right)\left(\begin{array}{ccc}0.7& 0.2& 0.1\\ 0.2& 0.6& 0.2\\ 0.5& 0.2& 0.3\end{array}\right)=\left(\begin{array}{ccc}0.4& 0.4& 0.2\end{array}\right),$$

and the same happens with all the vertices of $\left[\mathbb{M}\right]$ which was proved in Theorem 10.

The hospital now has beforehand knowledge with a good lead time where its policies and tendencies of the hospital system will converge in terms of relative expected population structure. Hence it is able to decide if this is a desirable situation; to find out if it can cope with the resources available in doctors, nurses and medical material; if its medical facilities are adequate; it can also have an estimate of the cost of the system see [62,63].

Consider now that the previous NHMS is system $\mathit{a}$ and let $\mathit{b}$ be a second NHMS with initial population structure $\mathit{q}\left(0\right)=\left(\begin{array}{ccc}0.6& 0.2& 0.2\end{array}\right)$; allocation probabilities ${\mathit{p}}_0\left(1\right)=\left(\begin{array}{ccc}0.5& 0.4& 0.1\end{array}\right)$ and ${\mathit{p}}_0\left(2\right)=\left(\begin{array}{ccc}0.7& 0.2& 0.1\end{array}\right)$ and the remaining parameters the same. Then the asymptotically relative population structure is again $\mathbb{E}\left[\mathit{q}\left(\infty \right)\right]=\left(\begin{array}{ccc}0.4& 0.4& 0.2\end{array}\right)$ which verifies Theorem 8. The physical meaning of the previous result is that when the hospital is at full capacity for some time, then with different initial structures and allocation probabilities the expected relative population structure remains unchanged under the condition that the maximum ergodicity coefficient of the transition probability matrices is less than one.

To be able to use for the benefit of the hospital the last theorem, that is Theorem 11, we need a way to find a numerical value that will replace $d(\mathbb{M},\tilde{\mathbb{M}})$. The following Lemma, which is not difficult to be proved, provides a solution to the problem.

Lemma 10.

Let$\mathbb{M},\tilde{\mathbb{M}}$be two tight intervals of transition probability matrices for the memberships. Let${\mathit{V}}_1,{\mathit{V}}_2,\dots ,{\mathit{V}}_m$be the vertices of the convex set$\mathbb{M}$, and${\mathit{U}}_1,{\mathit{U}}_2,\dots ,{\mathit{U}}_n$be the vertices of the convex set$\tilde{\mathbb{M}}$. Then

$$\delta (\mathbb{M},\tilde{\mathbb{M}})=∥a_1^{*}{\mathit{V}}_1+a_2^{*}{\mathit{V}}_2+\dots +a_m^{*}{\mathit{V}}_m-\left[b_1^{*}{\mathit{U}}_1+b_2^{*}{\mathit{U}}_2+\dots +b_n^{*}{\mathit{U}}_n\right]∥,$$

where$a_1^{*},a_2^{*},\dots ,a_m^{*}$is the solution of the optimization problem

$$max\left[a_1{\mathit{V}}_1+a_2{\mathit{V}}_2+\dots +a_m{\mathit{V}}_m\right]\mathit{with}$$

$$a_1+a_2+\dots +a_m=1,a_1\ge 0,a_2\ge 0,\dots ,a_m\ge 0.$$

and$b_1^{*},b_2^{*},\dots ,b_n^{*}$is the solution of the optimization problem

$$min\left[b_1{\mathit{U}}_1+b_2{\mathit{U}}_2+\dots +b_n{\mathit{U}}_n\right]\mathit{with}$$

$$b_1+b_2+\dots +b_n=1,b_1\ge 0,b_2\ge 0,\dots ,b_m\ge 0.$$

In what follows we summarize in Algorithm 2 for convenience of the interest readers the previous steps which are necessary for using the present results in any population system.

Algorithm 2

Use Lemma 1 to check that $\left[{\mathbb{S}}_0\right]$ is tight. Apply Algorithm 1 to find the vertices of the convex set $\left[{\mathbb{S}}_0\right]:$ $${\upsilon }_1,{\upsilon }_2,\dots ,{\upsilon }_s.$$ Use Lemma 1 to check that $\left[{\mathbb{R}}_0\right]$ is tight. Apply Algorithm 1 to find the vertices of the convex set $\left[{\mathbb{R}}_0\right]:$ $${\mathit{r}}_1,{\mathit{r}}_2,\dots ,{\mathit{r}}_{\pi }.$$ Use Lemma 1 to check that each row in $\left[\check{\mathit{Q}},\widehat{\mathit{Q}}\right]$ is tight. If yes then $\left[\mathbb{M}\right]$ is tight. Apply Algorithm 1 for each row vector in $\left[\check{\mathit{Q}},\widehat{\mathit{Q}}\right]$ to find the vertices of $\left[\mathbb{M}\right]:$ $${\mathit{V}}_1,{\mathit{V}}_2,\dots ,{\mathit{V}}_m.$$ Compute all the raw vectors $${\upsilon }_i{\mathit{V}}_j\mathrm{for}\mathrm{all}i=1,2,\dots ,s;j=1,2,\dots ,m.$$ Use any of the computational geometry methods in [64]. to find the vertices of the convex hull of the vectors ${\upsilon }_i{\mathit{V}}_j$ for all $i=1,2,\dots ,s;j=1,2,\dots ,m.$ Let that $${\omega }_1,{\omega }_2,\dots ,{\omega }_{\nu },$$ the vertices found. Compute using properties of the Minkowski sum of vectors $$Rn{g}_1\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)=\frac{T\left(0\right)}{T\left(1\right)}conv\left\{{\omega }_1,{\omega }_2,{\omega }_3,{\omega }_4,{\omega }_5\right\}+$$ $$\frac{1}{T\left(1\right)}\left[T\left(1\right)-T\left(0\right)\right]conv\left\{r_1,r_2\right\}.$$ Repeat the process until $Rn{g}_{\infty }\left({\mathbb{S}}_0,{\mathbb{R}}_0\right)$ geometrically fast, i.e., in 6 to 8 steps. Use Lemma 9 to verify that $\left[\mathbb{M}\right]=\left[\check{\mathit{Q}},\widehat{\mathit{Q}}\right]$ is uniformly scrambling, i.e., $\mathcal{T}(\widehat{\mathit{Q}}-\check{\mathit{Q}})<1$. Use Lemma 10 to find bounds for $$\delta (\mathbb{M},\tilde{\mathbb{M}})<\mu \mathrm{and}\delta (\tilde{\mathbb{M}},\mathbb{M})<\tilde{\mu }.$$ Set $$d(\mathbb{M},\tilde{\mathbb{M}})<max(\mu ,\tilde{\mu }).$$

8. Conclusions

The concept of the non-homogeneous Markov set system was introduced which is a NHMS with its parameters in an interval. It was established under which conditions in a NHMSS the set of all possible expected relative population structures at a certain point in time is a convex set and a convex polygon. Then it was founded that if in an NHMSS the sets of initial structures are different but compact and convex; also, the sets of allocation probabilities of the memberships are different but convex and compact; the inherent non-homogeneous Markov set chain is common; then the Hausdorff metric of the two different sets of all possible expected relative structures asymptotically goes to zero geometrically fast, i.e., asymptotically they coincide geometrically fast. Then we established that in an NHMSS if the total population of memberships converge in a finite number geometrically fast, and the sets of initial structures and allocation probabilities of memberships are compact and convex, then the set of all possible expected relative population structure converge to a limit set geometrically fast. Then it was proved that these results generalize certain well-known results for NHMS’s. Then it was proved that under some mild conditions the limit set of the expected relative population structures of an NHMSS remains invariant if any selected transition probability matrix of the inherent non-homogeneous Markov chain from the respective interval is multiplied by it from the right. It was also proved that the limit set is the only set with this property if the interval of selection of transition probabilities of the inherent non-homogeneous Markov chain is product scrambling. Finally it was assumed that two different NHMSS in the sense that they have different sets of selecting initial distributions, different sets of selecting allocation probabilities and different intervals of selecting the transition probabilities of the inherent non-homogeneous Markov chains, while they have in common that their respective intervals are uniformly scrambling with a common bound and they have the same total population of memberships. Then it was proved that the Hausdorff metric of the limit sets of the expected relative population structures of the two NHMSS is bounded by the multiplication of a function of the Hausdorff metric of the two tight intervals of selection of the stochastic matrices of the inherent non-homogeneous Markov set chains and the bound of their uniform coefficients of ergodicity.