Non-Homogeneous Markov Set Systems

: A more realistic way to describe a model is the use of intervals which contain the required values of the parameters. In practice we estimate the parameters from a set of data and it is natural that they will be in conﬁdence intervals. In the present study, we study Non-Homogeneous Markov Systems (NHMS) processes for which the required basic parameters are in intervals. We call such processes Non-Homogeneous Markov Set Systems (NHMSS). First we study the set of the relative expected population structure of memberships and we prove that under certain conditions of convexity of the intervals of the parameters the set is compact and convex. Next, we establish that if the NHMSS starts with two different initial distributions sets and allocation probability sets under certain conditions, asymptotically the two expected relative population structures coincide geometrically fast. We continue proving a series of theorems on the asymptotic behavior of the expected relative population structure of a NHMSS and the properties of their limit set. Finally, we present an application for geriatric and stroke patients in a hospital and through it we solve problems that surface in an application.


Introduction
The class of stochastic processes called Non-Homogeneous Markov Systems (NHMS) was first defined in [1] The class of NHMS provided a general framework for many applied probabilities models used to model populations of a wide diversity of entities. The primary motive was to provide a general framework for a wide class of stochastic models in social processes ( [2]). They also include as special cases non-homogeneous Markov chain models in manpower systems such as [3][4][5]. The literature on NHMS has flourished since then to a large extent and presently exist a large volume of theoretical results as well a variety of applications. In Section 2 of the present we provide a definition and a concise description of a NHMS. As we will discuss in Section 2, it is important for the reader to have in mind that actually the well-known non-homogeneous Markov chain is a special case of a NHMS.
In many stochastic processes and so naturally in Markov chains and specifically in non-homogenous Markov chains and NHMS, the values of the various parameters are assumed to be exact while in practice these are estimated from the data. Therefore, actually the values of the parameters is more realistic to be viewed as being contained in intervals with the desired probability confidence. This approach has been used in systems of linear equations and in this case the solutions are given as the set of all possible solutions. Two books have been written on this topic by [6,7]. For the analogous problem for differential equations a book was written by [8]. For homogeneous Markov chains with this approach a book was written see [9].
In Section 3 of the present we will now add some additional assumptions on a NHMS in our way to define a non-homogeneous Markov set system (NHMSS). In this way now a NHMSS will be a NHMS whose basic parameters will be assumed to be in compact convex intervals.
The NHMSS is a stochastic system which has a population of members which increases at every point in time. I addition the initial members need not to be the same entities at different time points since there is wastage from the system. The members of the population move among the different states, exit from the system (population) and new members are coming into the population (system) as replacements or to expand the system. In the case of non-homogeneous Markov set chains we have only one particle in the population, which never leaves the system and no procedure to replace this particle exists. Mathematically now the NHMSS has more elements in the one step in time equation with more parameters introduced in the stochastic difference equation. As the equation is applied recursively we end up with series of components which interact together with more parameters being in an interval. Hence, the problems to be solved are a lot harder, and new strategies and tools must be used, than the simple case of the Markov set chain. The introduction of the concept of a membership is crucial in dealing with the different individual members as time progress. The tool of Minkowski sum of vectors and its properties for convex combination of compact sets will play a vital role which was not needed in the case of Markov set chains. One of the hard problems which we encounter which does not exist in the Markov set chains is finding the range of infinite series. The Hausdorff metric for compact sets and the coefficient of ergodicity together with properties of appropriate norms introduced and the manipulation of infinite series will help to provide the following: In Section 4, we establish in the form of a theorem, using the Minkowski sum of two sets, under which conditions in a NHMSS the set of all possible expected relative population structures at a certain point in time is a convex set. Also, we establish a Theorem where we provide conditions under which the set of all expected relative population structures at a certain point in time is a convex polygon.
In Section 5 we study the asymptotic behavior of an NHMSS, a problem that has been of central importance for homogeneous Markov chains, non-homogeneous Markov chains, NHMS and homogeneous Markov set chains. In Theorem 4, with the use of the coefficient of ergodicity and the Hausdorff metric we prove the following: Let that in an NHMSS the sets of initial structures are different but compact and convex; also, the sets of allocation probabilities of the memberships are different but convex and compact; the inherent non-homogeneous Markov set chain is common; then the Hausdorff metric of the two different sets of all possible expected relative structures asymptotically goes to zero geometrically fast. This is equivalent with concluding that the two sets asymptotically coincide geometrically fast. In Theorem 5 we prove that in an NHMSS if the total population of memberships converges in a finite number geometrically fast, and the sets of initial structures and allocation probabilities of memberships are compact and convex, then the set of all possible expected relative population structure converges to a limit set geometrically fast. These two theorems have important consequences for a NHMS process also. The first is Theorem 6 which relaxes important assumptions of the basic asymptotic theorem for NHMS which is provided as Theorem 7. The second labeled as Theorem 8 answers a novel question for NHMS, i.e., provides conditions under which two different NHMS, with the same number of states and population, but different initial states and different allocation probabilities of memberships if they have the same transition probabilities sequence of memberships, they converge in the same relative population structure geometrically fast.
In Section 6 we study properties of the limit set of expected relative population structures. In Theorem 10 we prove the first property, that under some mild conditions the limit set of the expected relative population structures of an NHMSS remains invariant if any selected transition probability matrix of the inherent non-homogeneous Markov chain from the respective interval is multiplied by it from the right. We also prove that the limit set is the only set with this property if the interval of selection of transition probabilities of the inherent non-homogeneous Markov chain is product scrambling. In Theorem 11 the second property is established, i.e., let two different NHMSS in the sense that they have different sets of selecting initial distributions, different sets of selecting allocation probabilities and different intervals of selecting the transition probabilities of the inherent non-homogeneous Markov chains. What they have in common is that their respective intervals are uniformly scrambling with a common bound and they have the same total population of memberships. We prove that the Hausdorff metric of the limit sets of the expected relative population structures of the two NHMSS is bounded by the multiplication of a function of the Hausdorff metric of the two tight intervals of selection of the stochastic matrices of the inherent non-homogeneous Markov set chains and the bound of their uniform coefficients of ergodicity.
In Section 7 we present a representative application for geriatric and stroke patients in a hospital. Through this application we provide solutions in problems arising in an application by providing respective Lemmas and a general Algorithm with computational geometry procedures which are applicable to any population system.

The Non-Homogeneous Markov System
Consider a population which has T(t) memberships at time t. These memberships could be held by any kind of entities, i.e., human beings, animals, T-cells in a biological entity, fish in an organized area in the sea, cars on a highway etc. We assume that T(t) is known for every t, for example in a hospital the memberships are the beds for patients and from the management of the hospital's planning the number of beds are known. Let that the population is stratified into classes which we call states and let that there are a finite number of states, i.e., the state space is S ={1, 2, . . . , k}. We assume that the evolution of the population is in discrete time, i.e., t = 1, 2, ... and we call the vector of random variables is the number of memberships in state i at time t, the population structure of the NHMS. Define by q(t) = N(t)/T(t) to be the relative population structure. At every time instant t = 1, 2, ..., we have internal transitions of members among the states in S with probabilities which we collect in the k × k matrix P(t); we have wastage from all the states with probabilities which we collect in the 1 × k vector p k+1 (t) = [p 1,k+1 (t), p 2,k+1 (t), . . . , p k,k+1 (t)]; finally, we have recruitment or allocation probabilities of replacements or new entrants in the various states at time t which we collect in the 1 × k stochastic vector p 0 (t) = [p 01 (t), p 02 (t), . . . , p 0k (t)]. We assume that the system is expanding, i.e., ∆T(t) = T(t) − T(t − 1) ≥ 0. During the time interval (t − 1, t] a member of the system in state i either moves internally to another state j of the system with probability p ij (t) or leave the system and his membership remains at the exit of the system. New entrants to the system are of two types, those to replace leavers and those needed to be added in the system to meet the target of T(t) total memberships. The new entrant gets his membership at the entrance and he is being allocated or recruited at state j with probability p 0j (t). Hence, the probability of movement of a membership from state i to state j at time t is q ij (t) = p ij (t) + p 0j (t)p i,k+1 (t). We collect these probabilities in the k × k matrix Q(t) = P(t) + p k+1 (t)p 0 (t) which apparently is a stochastic matrix. We call the Markov chain defined by the sequence of matrices {Q(t)} ∞ t=0 the imbedded or inherent non-homogeneous Markov chain of the NHMS.
It is of interest the expected relative population structure. Let X t be the random variable representing the state that a membership of the system is at time t. Define by where from which we get that where Q(s, t) = Q(s + 1)Q(s + 2) . . . Q(t) for (s ≤ t). We set Q(s, t) = I the identity matrix for s ≥ t. Please note that we set q(s, t) = 0 for s > t. We call any such process as described above a Non-homogeneous Markov process in discrete time and discrete state space. It is important for the reader to realize that the well-known ordinary Markov chain is a very special case of a NHMS with T(t) = 1, p 0 (t) = 0, p k+1 (t) = 0 and Q(t) = P.
As we mentioned in the Introduction the stochastic process NHMS was first introduced in [1] as a discrete time, discrete state space stochastic process with motives which have their roots in actual applications in manpower systems see for example [1,2,11], and also the review papers [12,13]. Since then, a large literature on theoretical developments on many aspects of a NHMS were published which also included the developments in [14,15] of NHMS's in a general state space. In [16] there appeared the link between the theory of NHMS and martingale theory. Lately, also another area of large interest has been the Law of Large Numbers in NHMS ( [17]) which has its roots as a motive the study of Laws of Large numbers on homogeneous Markov chains by Markov himself. Also, many applications in areas with great diversity have also appeared in the literature. For example we could selectively refer to some of them. Let as start with [18][19][20] which are applications in the evolution of the HIV virus on the T-cells of the human body; population consisting with patients with asthma was studied in [21]; reliability studies were presented in [22]; applications in biomedical research appeared in [23,24]; various applications for human populations [25][26][27][28][29]; interesting application to consumption credit [30] infections of populations [31]; a very interesting application in DNA and web navigation [32]; interesting ecological applications [33]; results in Physical Chemistry [34]. Finally, there are a large number of publications by the research school of Prof McClean in hospital systems which are large manpower systems [35][36][37][38][39].

Non-Homogeneous Markov Set System
In Section 2 we defined the NHMS process and we will now define for the first time ever the non-homogeneous Markov set system. So far in the well developed theory of NHMS's the various perimeters are assumed to be exact while in practice they are naturally estimated by the data. Therefore, it is more realistic to be viewed as being contained in intervals with the desired probability confidence. In summary as we will see bellow a NHMSS is a NHMS for which its parameters are defined in intervals. In addition the study of NHMSS provides a new area of theoretical research with different mathematical tools in many instances than the corresponding theory of NHMS and a potential to be applied in other stochastic processes.
The practical advantages of NHMSS's are rather apparent since the assumptions on the parameters are less restrictive. The assumption of the parameters being in appropriate intervals absorbs in a way the errors of point estimates which increase their variability. In addition it provides the tools to study NHMS's whose parameters will be in "desired" intervals which increases considerably the control of the system since we could choose policies of the systems in intervals with desired outcomes for the expected relative population structures or to avoid trouble some situations.
We will start with the definition of an interval for a stochastic vector following [9] who first defined Markov set chains. Denote by M n (R) or simply M n the set of all n × n matrices with elements from the field R. Definition 1. Let SM 1n the set of all 1 × n stochastic vectors. Also let λ and µ be non-negative 1 × n vectors with λ ≤ µ componentwise. Then define the corresponding interval in SM 1n by where λ,µ are chosen such that [λ, µ] = ∅. Example 1. It is sometimes helpful to view mathematics geometrically. Let SM 1,3 the set of all 1 × 3 stochastic vectors, then it is easy to see that this is the convex hull of the vectors e 1 = 1 0 0 , e 2 = 0 1 0 and e 3 = 0 0 1 in R 3 . Now, all the non-negative vectors x such that λ ≤ x ≤ µ are within and the surface of a rectangle the coordinates of which are determined by λ, µ. The interval [λ, µ] will be the intersection of the two above described spaces. We can visualize this more easily if we consider the triangle e 1 e 2 e 3 in R 2 . Let λ = 0.1 0.2 0.3 and µ = 0.5 0.7 0.8 then the interval [λ, µ] could be easily designed in the following way. Draw two parallel lines to the line e 2 e 3 at the points λ 1 = 0.1 and µ 1 = 0.5; also draw two parallel lines to the line e 1 e 3 at the points λ 2 = 0.2 and µ 2 = 0.7; finally draw two parallel lines to the line e 1 e 2 at the points λ 3 = 0.3 and µ 3 = 0.8. Then the interval [λ, µ] is the common area between these lines.

Tight intervals are important in what follows:
Intervals can be tested for tightness using the following Lemma ( [9]). Also, with the use of this Lemma an interval which is not tight, we can tighten it up using an algorithm without actually changing it. That is the new interval, the tightened one will contain the same stochastic vectors.

Lemma 1. ([9]
). Let [λ, µ] be an interval. Then for each coordinate i We now need the following definition of when in a tight interval a vector is called free.
Then if λ i < p i < µ i for some coordinate i, then the coordinate p i in p is called free.
Tight intervals and convex sets are well linked and play an important role in the preservation of many properties. In this respect, the following Lemma is very useful.
if and only if p has at most one free component.
We will now extend the definition of an interval of a vector to an interval of a matrix and to a tight interval of a matrix. The interval [Λ, M] can be constructed also by rows, i.e., In what follows we will define a non-homogeneous Markov set system. We will keep the entire notation introduced in Section 2 for a NHMS and we will build on that.
q ij (t) for every t ∈ N, and the notation Q ,Q will be taken to imply that Q ,Q = ∅.
We will make now the following basic assumptions: Assumption 3. Let [R 0 ] be the set of 1 × k stochastic vectors from which the allocation probabilities p 0 (t) are being selected. That is We call a NHMS whose parameters are assumed to be in intervals as in Assumptions 1-3 a Non-homogeneous Markov Set System (NHMSS).
Note that it is apparent by now that Markov set chains that were initiated by [9,[40][41][42][43] are special cases of a NHMSS.

The Set of the Expected Relative Population Structures of a NHMSS
In geometry the Minkowski sum (also known as dilation) of two sets of position vectors A and B in Euclidean space is formed by adding each vector in A to each vector in B. That is Example 2. If we have two sets A and B consisting of three position vectors (informally, three points) representing the vertices of two triangles in R 2 with coordinates which comprises the vertices of a hexagon.
For Minkowski addition, the zero set containing only the zero vector 0, is an identity element for every subset V of a vector space, i.e., V+{0} = V.
The empty set is important in Minkowski addition because the empty set annihilates every other subset for every subset V of a vector space, its sum with the empty set is empty, i.e., V+∅ = ∅.
We are now in a position to state the following Lemma ( [44]) Lemma 3. If V is a convex set then µV + V is also a convex set and furthermore µV + V =(µ + λ)V for every λ, µ > 0. Conversely, if this "distributive property" holds for all non-negative real numbers λ, µ > 0 then the set is convex.

Remark 1.
For two convex polygons V 1 and V 2 in the plane with m and n vertices, their Minkowski sum is a convex polygon with at most m + n vertices and may be computed in time O(m + n) by a very simple procedure.
We need the following sets for the Lemma that follows Rng(q) = {y : y = qQ for some Q ∈ [M] and any q ∈ SM 1,n }.
We will need the following Lemma from ( [9] p. 39): Lemma 5. In a non-homogeneous Markov set chain let the set of initial distributions [S 0 ] be convex and let [M] be a tight interval from which the transition probability sequence of matrices is being selected. Then the set [S t ] of all possible probability distributions at the various states at time t is a convex set.
In the next theorem we show under which conditions the set of all possible expected relative population structures in a NHMSS is a convex set. since is the set of all possible expected structures of new memberships at time t which entered in the system at time τ. Now, since the system is expanding, i.e., ∆T(τ) ≥ 0, [R 0 ] is a convex set and [M] a tight interval, then with the same reasoning as in (6), we get that {E[r τ ]} is a convex set. Also, from Remark 1 we get that the Minkowski sum of sets is a convex set. Hence, since the two sets in the right-hand side of Equation (5) are convex and then according to Remark 1 their Minkowski sum Rng t (S 0 , R 0 ) is a convex set.
We will now borrow the following Theorem from ( [9] p. 40).

Theorem 2.
In a non-homogeneous Markov set chain let the set of initial distributions [S 0 ] be a convex polytope and let [M] be a tight interval from which the transition probability sequence of matrices is being selected. Then the set [S t ] of all possible probability distributions at the various states at time t is a convex polytope with vertices of the form In the next theorem we show under which conditions the set of all possible expected relative population structures in a NHMSS is a convex polytope. Proof. The proof follows the steps of the proof of Theorem 1 using Theorem 2.

Asymptotic Behavior of NHMSS
The problem of asymptotic behavior has been one of central importance for, homogeneous Markov chains, non-homogeneous Markov chains, NHMS, and non-homogeneous Markov set chains. In the present section we will prove a series of theorems with which we establish the asymptotic behavior of NHMSS.
Since Markov himself and his student Dobrushin the coefficient of ergodicity T (Q) of a k × k stochastic matrix Q = q ij k i,j=1 , has been a fundamental tool in the study of Markov chains. We have thus 0 ≤ T (Q) ≤ 1. We clarify in here that if T (Q) < 1 the stochastic matrix Q is called scrambling. Scrambling matrices are regular, but not all regular matrices are scrambling. Yet if Q is a regular stochastic matrix then some power of Q, say Q n is scrambling. We define byT ifT ([M]) < 1 we say that [M] is uniformly scrambling. More on uniform scrambling and the interpretation of the coefficient of ergodicity of a matrix A ∈ R n×n as a matrix norm when the norm is restricted to a specified subspace could be found, in [45]. For explicit forms for ergodicity coefficients and properties see also [46,47]. In what follows we will use the following norm for a matrix A ∈ M n (R) We will use the concept ofT ([M]) < 1 to study asymptotic behavior in a NHMSS. It is important to note that if we consider as M C , . the space of non-empty compact subsets of ([M], . ) then M C , . is a metric space ( [48]). This space can be topologized using the Hausdorff metric d(., .) defined by where δ(S 1 , S 2 ) = max From [48] we get also that M C , d(., .) is a metric space. We will need the following Lemmas converges to zero as t → ∞ geometrically fast with T(t) ≥ T(t − 1). Then {T(t)} ∞ t=0 converges geometrically fast.

Remark 2. The restriction
is a general assumption with the physical interpretation that the proportional growth rate vanishes in the limit. This assumption allows lim t→∞ T(t) = ∞.
We will now study what happens asymptotically to the two sets of the expected relative population structures when the initial structures belong to two different sets as well as the allocation probabilities in the respective cases. Also, assume that [S], [R 0 ] are compact and convex. Let S 1 , S 2 ⊆ S C and R 1 , R 2 ⊆ R C 0 then d(Rng t (S 1 , R 1 ), Rng t (S 2 , R 2 )) → t→∞ 0 geometrically fast.
We will now establish under what conditions the convergence to the limiting set in a NHMSS is geometrically fast.
By the evaluation of the function Rng(., .) given in Lemma 4 we get that
The above Theorems 4 and 5 have an important consequence since it provides a generalization, by relaxing important assumptions, of the basic asymptotic theory for an NHMS. The theorem is the following and could be proved by just following the analogous steps as in the proofs of Theorems 4 and 5 and Theorem 3. The basic asymptotic theorem for an NHMS and which has been used in many papers to provide further results is: Theorem 7 has been used extensively in the theory of NHMS to produce further results see for example [10][11][12]17,51], and the relaxation of the necessary conditions in Theorem 6 is apparent.
Another consequence is that Theorem 4 provides conditions under which two different NHMS, with the same number of states and population, but different initial states and different allocation probabilities of memberships if they have the same transition probabilities sequence of memberships (PTMS of the embedded Markov chains), they converge in the same expected relative population structure geometrically fast. This is stated in detail in the following theorem which could be proved by just following the analogous steps as in the proof of Theorems 4 and 5. Let q a (0) and q b (0) be their initial relative population structures respectively, and {p a (t)} ∞ t=0 and {p b (t)} ∞ t=0 their sequences of allocation probabilities. If sup tT (Q(t)) < 1 then where q(∞) is the row of the stable matrix Q(∞) = lim t→∞ Q(0, t).

Properties of the Limit Set
In the present section we establish some important properties of the limit set Rng ∞ (S, R 0 ). We start with the following definition: Definition 6. Let n be an integer such that T (Q 1 Q 2 ...Q n ) < 1 for all Q 1 , Q 2 , ..., Q n ∈ [M]. Then [M] is said to be product scrambling and n its scrambling integer.
We will make use of the following Theorem 3.3 in [9]: We will now establish with a Theorem some important properties of the limit set.
Also, assume that [S 0 ], [R 0 ] are compact and convex. Then if we define by .

If in addition [M]
is product scrambling with integer n then it is the unique set that has this property. T(0) T(t + 1) q(0) M t+1 Now, for any values of the parameters of E[q(0, t)] that does not maximize it, the difference above is still greater or equal than the ones which minimize E[q(0, t)]. Thus, we are free to choose the parameters q Since [M] is uniformly scrambling we have thatT =max t∈N T (Q * (t)) < 1 and thus Q * (τ, t) <T t−τ and which goes to zero as t → ∞ . Hence In a similar way we could prove that which lead us to the conclusion For the second part of the theorem in addition we have that [M] is product scrambling with index n. Assume that there is a second Rng * ∞ (S 0 , R 0 ) which is compact for which Then we get that from which we conclude that Rng * ∞ (S 0 , R 0 ) and Rng ∞ (S 0 , R 0 ) are the same set.
We will now establish an interesting result, that under certain conditions if we have two different inherent Markov set chains for two NHMSS, then the Hausdorff metric of the two sets of all possible expected relative structures of the NHMSS is less than the multiplication of a function of the common bound of the two uniform coefficients of ergodicity of the two intervals and the Hausdorff metric of the two intervals. We will need the following Lemma from ( [9] p. 70), or [52].

Proof. From Lemma 4 we get that
and an analogous description is valid for the set Rng t S 2 , R 2 ,M .
Now, we have that There exists some q ; also for anyq * (0) ∈ [S 2 ], anyp * 0 (τ) for every τ ∈ [1, t], and finally anyQ for which we have Using Lemma 8 we get that Similarly, we have From (29), (30) and (31) as t → ∞ we get that as we have seen in the proof of Theorem 8. Similarly, as we proved (32) From (32) and (33) we arrive at the conclusion of the Theorem.

An Illustrative Representative Example
In the present section we present an illustrative representative example to a Geriatric and Stroke Patients system and through it we will present the methodology in terms of computational geometry algorithms needed for an application to any population system. The NHMS model used is a general Coxian phase type model, special forms of which has been used by the school of research by McClean and her co-authors [38,[53][54][55][56][57][58][59]. We distinguish three states which are called hospital pathways. For the system of Geriatric and Stroke Patients these stages are labeled as "Acute Care", the "Rehabilitative" and the "Long Stay". From each stay we have movements outside the hospital due to discharge or death. Also, geriatric patients may be thought of as progressing through stages of acute care, rehabilitation and long-stay care, where most patients are eventually rehabilitated and discharged. Geriatric medical services are an important asset in the care of elderly and their quality is certainly an indication of the level of civilization in a society. At the same time their funding could be easily reduced due to political pressure on savings in health care expenditure.
It is apparent that the number of pathways could be increased as much as it is needed to accommodate any important characteristics of any patients systems. However, there is no need to consider in here a larger number of states due to the restriction of space. Also, the internal movements in a population of patients could be of any number to accommodate any important characteristics.
Consider a hospital which starts with T(0) = 400 patients and in a very short time reaches its full capacity of 435 patients, i.e., T(1) = 420, T(2) = 430, T(3) = 435. Assume three hospital pathways and let that the initial relative population structure be any stochastic vector which lies in the set The physical meaning of selecting S 0 as above is that the initial relative structure could be any stochastic vector, i.e., S 0 contains all possible initial structures. For example q(0) = 0.2 0.3 0.5 means that 20% of the patients are in pathway 1, 30% are in pathway 2, and 50% are in pathway 3. Now, there are some initial structures which might not be acceptable for the management of the hospital, such as for example let say q(0) = 0 0 1.0 . In this case in the initial design of the hospital measures should be taken that such a situation will be avoided in cooperation with other nearby hospitals. Then S 0 could be chosen to be This is a convex set which excludes the initial relative structure q(0) = 0 0 1.0 . However, it also needs to be tight . How to make it tight is explained below with the use of the Algorithm 1. Naturally, we could exclude more than one relative structure from the chosen initial relative structure but the procedure will be the same.
Most new patients enter the system in hospital pathway one, either by taking an empty place or as a virtual replacement of a discharged patient. Hence, let R 0 be the convex set from which allocation probabilities are drawn and let it be the interval The set of stochastic vectors of allocation probabilities that are in the above interval is a convex set with vertices r 1 = [0.7 0.2 0.1] and r 2 = [0.5 0.4 0.1]. How to find the vertices will be explained below. The physical meaning of the above interval is that any stochastic vector that belongs in the interval R 0 is a candidate to represent a registration policy for the hospital. Naturally, we can restrict our interval R 0 to an interval which will contain the desired recruitment policies of the hospital management and find in this way using the results of the present paper the consequences of these policies. The recruitment vectors are the best control variables for human populations ( [2]), as are the hospitals in this case. Methods of control by recruitment could be found in [50,[60][61][62]. However, the interval R 0 needs to be tight and how to make it tight is explained below with the use of the Algorithm 1. in order that [M] is tight every row should be tight hence using Lemma 1 we could see that the interval is tight. Hence, the stochastic matrices that belong to [M] is a convex polytope and we need to find its vertices. The same applies for S 0 and R 0 which are tight, and we need to find the vertices of the convex polytope on which all stochastic vectors of the interval lie. Applying Lemma 1 we find that the set S 0 is tight and applying Algorithm 1 we get that S 0 is convex with vertices υ 1 = (1 0 0), υ 2 = (0 1 0) and υ 3 = (0 0 1). The physical meaning of the set S 0 in here is that at the start of our study at time 0 we allow that the hospital could have any relative population structure. Applying Lemma 1 we find that the set R 0 is tight and applying Algorithm 1 we get that R 0 is convex with vertices r 1 = (0.7 0.2 0.1), r 2 = (0.5 0.4 0.1).

Algorithm 1 ([9]) Finding the vertices of a tight interval [p, q].
For each i = 1, 2, 3 construct the vectors: For i = 1 and the vector p replace p 1 withp 1 such that the above vectors will become stochastic. If any of the four resulting stochastic vectors belongs in the interval [p, q] then it is a vertex. Do the same for the vector q. END.
To find the vertices of the convex set of stochastic matrices that belong to the tight interval [M] we apply Algorithm 1 for each row vector in Q ,Q and we find that the vertices are These 12 stochastic vectors belong to a convex set, hence using any of the computational geometry methods in [61] we find that the vertices of the convex hull of these vectors are ω 1  Hence from which we get that Rng 1 (S 0 , R 0 ) is the convex set with vertices given by the above This result verifies Theorem 1. With now the vertices of the convex set Rng 1 (S 0 , R 0 ) we repeat the previous process to find Rng 2 (S 0 , R 0 ). Now, in this way at every point in time we have the convex compact space Rng t (S 0 , R 0 ) of all possible expected population structures. If any of these are problematic in some way then apparently the hospital has a lead time to adapt new policies and an instrument to visualize their consequences.
To verify that a tight interval of transition probability matrices is uniformly scrambling we need a sufficient condition as a criterion. This is given in the following easily proved Lemma. Hence the expected relative population structure converges in six steps, that is geometrically fast which verifies Theorem 6. Also, it is easy to see that The hospital now has beforehand knowledge with a good lead time where its policies and tendencies of the hospital system will converge in terms of relative expected population structure. Hence it is able to decide if this is a desirable situation; to find out if it can cope with the resources available in doctors, nurses and medical material; if its medical facilities are adequate; it can also have an estimate of the cost of the system see [62,63].
Consider now that the previous NHMS is system a and let b be a second NHMS with initial population structure q(0) = 0.6 0.2 0.2 ; allocation probabilities p 0 (1) = 0.5 0.4 0.1 and p 0 (2) = 0.7 0.2 0.1 and the remaining parameters the same. Then the asymptotically relative population structure is again E[q(∞)] = 0.4 0.4 0.2 which verifies Theorem 8. The physical meaning of the previous result is that when the hospital is at full capacity for some time, then with different initial structures and allocation probabilities the expected relative population structure remains unchanged under the condition that the maximum ergodicity coefficient of the transition probability matrices is less than one.
To be able to use for the benefit of the hospital the last theorem, that is Theorem 11, we need a way to find a numerical value that will replace d M,M . The following Lemma, which is not difficult to be proved, provides a solution to the problem. Lemma 10. Let M,M be two tight intervals of transition probability matrices for the memberships. Let V 1 , V 2 , . . . , V m be the vertices of the convex set M, and U 1 , U 2 , . . . , U n be the vertices of the convex setM. Then , where a * 1 , a * 2 , . . . , a * m is the solution of the optimization problem max[a 1 V 1 + a 2 V 2 + ... + a m V m ] with a 1 + a 2 + ... + a m = 1, a 1 ≥ 0, a 2 ≥ 0, . . . , a m ≥ 0. and b * 1 ,b * 2 ,..., b * n is the solution of the optimization problem In what follows we summarize in Algorithm 2 for convenience of the interest readers the previous steps which are necessary for using the present results in any population system.

Conclusions
The concept of the non-homogeneous Markov set system was introduced which is a NHMS with its parameters in an interval. It was established under which conditions in a NHMSS the set of all possible expected relative population structures at a certain point in time is a convex set and a convex polygon. Then it was founded that if in an NHMSS the sets of initial structures are different but compact and convex; also, the sets of allocation probabilities of the memberships are different but convex and compact; the inherent non-homogeneous Markov set chain is common; then the Hausdorff metric of the two different sets of all possible expected relative structures asymptotically goes to zero geometrically fast, i.e., asymptotically they coincide geometrically fast. Then we established that in an NHMSS if the total population of memberships converge in a finite number geometrically fast, and the sets of initial structures and allocation probabilities of memberships are compact and convex, then the set of all possible expected relative population structure converge to a limit set geometrically fast. Then it was proved that these results generalize certain well-known results for NHMS's. Then it was proved that under some mild conditions the limit set of the expected relative population structures of an NHMSS remains invariant if any selected transition probability matrix of the inherent non-homogeneous Markov chain from the respective interval is multiplied by it from the right. It was also proved that the limit set is the only set with this property if the interval of selection of transition probabilities of the inherent non-homogeneous Markov chain is product scrambling. Finally it was assumed that two different NHMSS in the sense that they have different sets of selecting initial distributions, different sets of selecting allocation probabilities and different intervals of selecting the transition probabilities of the inherent non-homogeneous Markov chains, while they have in common that their respective intervals are uniformly scrambling with a common bound and they have the same total population of memberships. Then it was proved that the Hausdorff metric of the limit sets of the expected relative population structures of the two NHMSS is bounded by the multiplication of a function of the Hausdorff metric of the two tight intervals of selection of the stochastic matrices of the inherent non-homogeneous Markov set chains and the bound of their uniform coefficients of ergodicity.