Discrete-Time Semi-Markov Random Evolutions in Asymptotic Reduced Random Media with Applications

Abstract

This paper deals with discrete-time semi-Markov random evolutions (DTSMRE) in reduced random media. The reduction can be done for ergodic and non ergodic media. Asymptotic approximations of random evolutions living in reducible random media (random environment) are obtained. Namely, averaging, diffusion approximation and normal deviation or diffusion approximation with equilibrium by martingale weak convergence method are obtained. Applications of the above results to the additive functionals and dynamical systems in discrete-time produce the above tree types of asymptotic results.

1. Introduction

In order to simplify the analysis of complex systems, we consider stochastic approximation methods where we not only simplify the system but also the random media. These address the fact that some subsets of state space are weakly connected with the other subsets, that is, the transition probabilities are very small compared to transition probabilities inside the considered subsets. This fact allows one to proceed into an asymptotic reduction of the state space of system and also of the random medium.

In fact, the random medium, which is coming to perturb the considered system, or equivalently the random evolution, can explore some subsets of its state space in a fast time, while some other subsets of states are explored in a slow time. On the one hand, in the scale of fast time, the slow time explored subsets take place as rare events. On the other hand, in the slow time scale, the fast time can be considered as a unique merged state, since states into the fast subsets are undistinguishable from the point of view of slow time scale.

Of course, the different kind of stochastic approximations of the random evolutions give us different kind of results. Namely, the average approximation leave the same structure of the system but with a simpler state space of the random medium and structure. In the diffusion approximation the structure of the system is simplified to a switched diffusion process, and the switching random medium in a simpler state space and structure. In the normal deviation, or equivalently in the merging with equilibrium, the considered process is the difference of the initial process by the mean process obtained in the averaging scheme and the limit is a switched diffusion process.

Concerning the state space of random media we may consider a finite or even uncountable factor space of the state space on which we consider a supporting Markov chain and the original process is considered as a perturbation of the above supporting Markov chain by a signed transition kernel.

Results of this kind in continuous time have been presented in several works including those by the authors of the present paper. This kind of results in semi-Markov setting were first presented by V.S. Koroliuk and his collaborators [1,2,3,4,5]. Asymptotic merging called consolidation is also studied by Anisimov [6]. See also results by Yin and Zhang [7]. Some recent papers are also dedicated to the above problems. For the Markov switching models, see, for example, References [8,9], and for non equilibrium Markov processes, see, for example, References [10,11].

Discrete-time semi-Markov random evolution have already been studied as the embedded Markov process of semi-Markov processes, where in fact it turns out to be a Markov chain random evolution, see, for example, References [1,2]. Discrete calendar time Markov evolution were introduced first by Keepler in Reference [12]. In semi-Markov setting they have been introduced in Reference [13], and studied in depth in Reference [14]. This paper presents new results as continuation of those presented in Reference [14]; they are different from the fact that the random media there were on fixed state space and not reducible as in the present case. Nevertheless, for the first part, the merging of semi-Markov chain, we are using here a different technique to obtain merging of the state space and asymptotic results for stochastic systems, which is based on the compensating operator of the semi-Markov chain, see, for example, References [1,15].

The limit results obtained here are in the weak functional sense in the Skorohod topology, see, for example, References [15,16,17,18,19,20,21]. For works on random evolution see, for example, in References [1,2,3,5,12,22,23], and references therein. Reference works for Markov chains see, for example, References [24,25,26,27,28]. For semi-Markov processes see, for example, References [29,30,31,32]; and for discrete-time see, for example, References [13,14,33]. Useful results in Banach space can be found in, for example, References [14,34,35,36,37,38,39]. For applications to real problems of the type of results presented here see, for example, References [4,16,40,41,42].

The paper is organized as follows. Section 2 includes the semi-Markov chain setting needed in the sequel. Section 3 includes merging state space definition and results of asymptotic merging in the ergodic and non ergodic cases. Section 4, includes discrete-time semi-Markov random evolution (DTSMRE) definition and preliminary results. Section 5 presents the main results of this paper, that is average, diffusion and diffusion with equilibrium approximation or normal deviation results for DTSMRE with merging. Section 6 presents average, diffusion and diffusion with equilibrium approximation results for particular systems: integral functionals and dynamical systems. Section 7 presents proofs of the theorems. Finally, Section 8 contains concluding remarks.

2. Semi-Markov Chains with Merging

Let $(E,\mathcal{E})$ be a measurable space with countably generated $\sigma$-algebra and $(\mathsf{\Omega },\mathcal{F},{({\mathcal{F}}_n)}_{n\in \mathrm{N}},\mathbf{P})$ be a stochastic basis on which we consider a Markov renewal process (MRP), $(x_n,{\tau }_n,n\in \mathrm{N})$, in discrete time $k\in \mathrm{N}$, with state space $(E,\mathcal{E})$. It is worth noticing that k is the calendar time, while n is the number of jumps, both are $\mathrm{N}$-valued. Notice that $\mathrm{N}$ is the set of non-negative integer numbers. The semi-Markov kernel q is defined by (see, e.g., Reference [33]),

$$\begin{array}{c}\hfill q(x,B,k):=\mathbf{P}(x_{n+1}\in B,{\tau }_{n+1}-{\tau }_n=k\mid x_n=x),x\in E,B\in \mathcal{E},k,n\in \mathrm{N}.\end{array}$$

(1)

We will also denote $q(x,B,\mathsf{\Gamma })={\sum }_{k\in \mathsf{\Gamma }}q(x,B,k)$, where $\mathsf{\Gamma }\subset \mathrm{N}$. The process $(x_n)$ is the embedded Markov chain (EMC) of the MRP $(x_n,{\tau }_n)$ with transition kernel $P(x,dy)$ on the state space $(E,\mathcal{E})$. The semi-Markov kernel q is written as

$$q(x,dy,k)=P(x,dy)f_{xy}(k),$$

where $f_{xy}(k):=\mathbf{P}({\tau }_{n+1}-{\tau }_n=k\mid x_n=x,x_{n+1}=y)$, the conditional distribution of the sojourn time in state x given that the next visited state is y. We set $q(·,·,0)\equiv 0$.

Here for simplicity we do not consider dependence of the function $f_{xy}$ on the second state y, that is, the sojourn time distribution in state x is independent of the arrival state y, and we will denote it as $f_x$. In fact, any semi-Markov process with x and y dependence can be transformed to one with dependence only on x, see, for example, Reference [29]. So, there is no restriction to the generality.

Let ${\nu }_k=\max \{n:{\tau }_n\le k\}$ be the process which counts the jumps of the EMC $x_n$, in the time interval $[0,k]\subset \mathrm{N}$, and the discrete-time semi-Markov chain $z_k$ by $z_k=x_{{\nu }_k}$, for $k\in \mathrm{N}$. Define now the backward recurrence time process ${\gamma }_k:=k-{\tau }_{{\nu }_k}$, $k\ge 0$, and the filtration ${\mathcal{F}}_k:=\sigma (z_{\ell },{\gamma }_{\ell };\ell \le k)$, $k\ge 0$.

The Markov chain $(z_k,{\gamma }_k),k\ge 0$, has the following transition probability operator on the real bounded measurable functions defined on $E\times \mathrm{N}$,

$$\begin{array}{c}\hfill P^{♯}\phi (x,k)=\frac{1}{{\overline{F}}_x(k)}{{\displaystyle \int }}_{E\backslash \left\{x\right\}}q(x,dy,k+1)\phi (y,0)+\frac{{\overline{F}}_x(k+1)}{{\overline{F}}_x(k)}\phi (x,k+1).\end{array}$$

(2)

It is worth noticing that the above relation (2) can be written also in another interesting form as follows

$$\begin{array}{c}\hfill P^{♯}\phi (x,k)=\phi (x,k+1)+{\lambda }_x(k+1)[P\phi (x,0)-\phi (x,k+1)],\end{array}$$

where ${\lambda }_x(k+1)$ is the exit rate of the SMC from the state $x\in E$ and time $k+1$, given by ${\mathbf{P}}_x({\tau }_1=k+1\mid {\tau }_1>k)$. Of course, as usually, the transition rate in discrete-time is a probability and not a positive real-valued function as is the case in continuous-time. The above relation is similar to the generator of the process $(z_t,{\gamma }_t)$ in the continuous-time, see, for example, References [1,2,3].

The stationary distribution of the process $(z_k,{\gamma }_k)$, if there exists, is given by

$$\begin{array}{c}\hfill {\pi }^{♯}(dx\times \left\{k\right\})=\rho (dx){\overline{F}}_x(k)/m,\end{array}$$

where $m(x)$ is the mean sojourn time in state $x\in E$, and

$$\begin{array}{c}\hfill m:={{\displaystyle \int }}_E\rho (dx)m(x),m(x)={\displaystyle \sum _{k\ge 0}}{\overline{F}}_x(k),\end{array}$$

and $\rho (dx)$ is the stationary distribution of the EMC $(x_n)$, $F_x(k):=q(x,E,[0,k])$, and ${\overline{F}}_x(k):=1-F_x(k)=q(x,E,[k+1,\infty ))$. The probability measure $\pi$ defined by $\pi (B)={\pi }^{♯}(B\times \mathrm{N})$ is the stationary probability of the SMC $(z_k)$. From the above equality we get the following useful equality

$$\pi (dx)=\rho (dx)m(x)/m,$$

(3)

which connect the stationary distribution of the semi-Markov chain, with the stationary distribution of the embedded Markov chain, when they exist.

Define also the r-th moment of holding time in state $x\in E$,

$$m_r(x):={\displaystyle \sum _{k\ge 1}}{k}^{r}q(x,E,k),r=1,2,\dots$$

Of course, $m(x)=m_1(x)$, for any $x\in E$.

Define now the uniform integrability of the r-th moments of the sojourn time in states by

$${\displaystyle \underset{M\to \infty }{\lim }}{\displaystyle \underset{x\in E}{\sup }}{\displaystyle \sum _{k\ge M}}{k}^r{f}_x(k)=0,$$

(4)

for any $n\ge 1$.

3. Merging of Semi-Markov Chains

We present here the two cases: ergodic and non ergodic of the semi-Markov chain in merging scheme.

3.1. The Ergodic Case

Let us consider a family of ergodic semi-Markov chains $z_k^{\epsilon },k\ge 0$, $\epsilon >0$, with semi-Markov kernel $q^{\epsilon }$ and a fixed state space $(E,\mathcal{E})$, a measurable space.

Let us consider the following partition (split) of the state space.

$$\begin{array}{c}\hfill E={\cup }_{j=1}^d{E}_j,E_i\cap E_j=\varnothing ,i\ne j.\end{array}$$

(5)

Let us also consider the trace of $\sigma -$ algebra $\mathcal{E}$ on $E_j$, denoted by ${\mathcal{E}}_j$, for $j=1,\dots ,d$.

The semi-Markov kernels have the following representation

$$\begin{array}{c}\hfill q^{\epsilon }(x,B,k)=P^{\epsilon }(x,B)f_x(k),\end{array}$$

(6)

where the transition kernel of the EMC $x_n^{\epsilon },n\ge 0$, has the representation

$$\begin{array}{c}\hfill P^{\epsilon }(x,B)=P(x,B)+\epsilon P_1(x,B).\end{array}$$

(7)

The transition kernel P determines a support Markov chain, say $x_n^0$, $n\ge 0$, and satisfies the following relations

$$\begin{array}{c}\hfill P(x,E_j)={\mathbf{1}}_j(x)\equiv {\mathbf{1}}_{E_j}(x)=\{\begin{array}{ccc}1\hfill & \mathrm{if}\hfill & x\in E_j\hfill \\ 0\hfill & \mathrm{if}\hfill & x\notin E_j,\hfill \end{array}\end{array}$$

(8)

for $j=1,\dots ,d$. Of course, the signed perturbing kernel $P_1$ satisfies the relation $P_1(x,E)=0$, and $P^{\epsilon }(x,E)=P(x,E)=1$.

The perturbing signed transition kernel, $P_1$, provides transition probabilities between merged states.

Let $v:E\to \widehat{E}$ be the merging onto function defined by $v(j)=\ell$, if $j\in E_{\ell }$, $\ell \in \widehat{E}=\{1,\dots ,d\}$.

Set $k:=[t/\epsilon ]$, where $[x]$ is the integer part of the positive real number x, and define the split family of processes

$${\widehat{x}}_t^{\epsilon }:=v(z_{[t/\epsilon ]}^{\epsilon }),t\ge 0,\epsilon >0.$$

(9)

Define also the projector operator $\mathsf{\Pi }$ onto the null space, $\mathcal{N}(Q)$, of the operator $Q:=P-I$ by

$$\mathsf{\Pi }\phi (x)=\widehat{\phi }(v(x)),\mathrm{where}\widehat{\phi }(j):={{\displaystyle \int }}_{E_j}{\rho }_j(dx)\phi (x).$$

(10)

This operator satisfies the equations

$$\mathsf{\Pi }Q=Q\mathsf{\Pi }=0.$$

The potential operator of Q, denoted by $R_0$, is defined by

$$\begin{array}{c}\hfill R_0:={(Q+\mathsf{\Pi })}^{-1}-\mathsf{\Pi }={\displaystyle \sum _{k\ge 0}}[P^k-\mathsf{\Pi }].\end{array}$$

Let us now consider the following assumptions needed in the sequel.

C1:

The transition kernel $P^{\epsilon }(x,B)$ of the embedded Markov chain $x_n^{\epsilon }$ has the representation (7).

C2:

The supporting Markov chain $(x_n^0)$ with transition kernel P is uniformly ergodic in each class $E_j$, with stationary distribution ${\rho }_j(dx)$, $j\in \widehat{E}$, that is,

$${\rho }_j(B)={{\displaystyle \int }}_{E_j}{\rho }_j(dx)P(x,B),\mathrm{and}{\rho }_j(E_j)=1,B\in {\mathcal{E}}_j$$

C3:

The average exit probabilities of the initial embedded Markov chain $(x_n^{\epsilon }$) are positive, that is,

$${\widehat{p}}_j:={{\displaystyle \int }}_{E_j}{\rho }_j(dx)P_1(x,E\backslash E_j)>0.$$

C4:

The mean merged values are positive and bounded, that is,

$$0<m_j:={{\displaystyle \int }}_{E_j}{\rho }_j(dx)m(x)<\infty .$$

From relation (3), we get directly,

$${\pi }_j(dx)q(x)=q_j{\rho }_j(dx),$$

(11)

where $q(x):=1/m(x)$ and $q_j:=1/m_j$ with $m_j:={\int }_{E_j}{\rho }_j(dx)m(x)$.

Theorem 1.

Under assumptions C1-C4, the following weak convergence takes place

$${\widehat{x}}_t^{\epsilon }\Rightarrow {\widehat{x}}_{t}as\epsilon \to 0,$$

(12)

where the limit merged process${\widehat{x}}_t$is a continuous-time Markov process determined on the state space$\widehat{E}=\{1,\dots ,d\}$, by the intensity matrix

$$\widehat{Q}=({\widehat{q}}_{ij};i,j\in \widehat{E}),$$

where:

$${\widehat{q}}_{ij}=\left\{\begin{array}{cc}{q}_i{\widehat{p}}_{ij},& j\ne i\\ -q_i{\widehat{p}}_{ii},& j=i.\end{array}\right.$$

and${\widehat{p}}_{ij}:={\int }_{E_i}{\rho }_i(dx)P_1(x,E_j)$, with$i,j\in \widehat{E}$, and$q_i:={\int }_{E_i}{\pi }_i(dx)q(x)$.

3.2. The Non-Ergodic Case

Let us consider a family of semi-Markov chains $z_k^{\epsilon },k\ge 0$, $\epsilon >0$, with semi-Markov kernels $q^{\epsilon }$ and a fixed state space $(E^{\prime },{\mathcal{E}}^{\prime })$, a measurable space, which includes an absorbing state, say 0. Of course, here state 0 can represent a final class, say $E_0$, and the analysis presented here is the same.

Let us consider the following partition of the state space.

$$\begin{array}{c}\hfill E^{\prime }=E\cup \left\{0\right\},E={\cup }_{j=1}^d{E}_j,E_i\cap E_j=\varnothing ,i\ne j.\end{array}$$

(13)

Let $v:E^{\prime }\to {\widehat{E}}_0$ be the merging onto function defined by $v(j)=\ell$, if $j\in E_{\ell }$, $\ell \in {\widehat{E}}_0=\{0,1,\dots ,d\}$, and $v(0)=0$.

We now need the following condition.

C5:

The average transition probabilities of the initial embedded Markov chain $(x_n^{\epsilon }$) to state 0, satisfy the following,

$${\widehat{p}}_{j0}:=-{{\displaystyle \int }}_{E_j}{\rho }_j(dx)P_1(x,E)>0$$

with partition of the state space as defined by (13).

Let us also define the absorption time to state 0, ${\zeta }^{\epsilon }$, for any $\epsilon >0$,

$${\zeta }^{\epsilon }:=\inf \{t\ge 0:z_{[t/\epsilon ]}^{\epsilon }=0\}.$$

(14)

Theorem 2.

Under assumptions C1-C4, and C5 the following weak convergence takes place

$${\widehat{x}}_t^{\epsilon }\Rightarrow {\widehat{x}}_{t}as\epsilon \to 0,$$

(15)

where the limit merged process${\widehat{x}}_t$,$0\le t\le \widehat{\zeta }$, is a continuous-time Markov process determined, on the state space$\widehat{E}=\{0,1,\dots ,d\}$, by the intensity matrix

$$\widehat{Q}=({\widehat{q}}_{ij};i,j\in {\widehat{E}}_0),$$

where:

$${\widehat{q}}_{ij}=\left\{\begin{array}{cc}{q}_i{\widehat{p}}_{ij},& j\ne i,i\ne 0\\ -q_i{\widehat{p}}_{ii},& j=i,i\ne 0\\ 0,& i=0\end{array}\right.$$

and$\widehat{\zeta }:=\inf \{t\ge 0:\widehat{x}(t)=0\}$.

4. Semi-Markov Random Evolution

Let us consider a separable Banach space B of real-valued measurable functions defined on E, endowed with the sup norm $∥·∥$ and denote by $\mathcal{B}$ its Borel $\sigma$-algebra. Let us given a family of bounded contraction operators $D(x),x\in E$, defined on B, where the maps $D(x)\phi :E\to B$ are $\mathcal{E}$-measurable, $\phi \in B$. Denote by I the identity operator on B. For a discrete generator Q, on B, let $\mathsf{\Pi }B=\mathcal{N}(Q)$ be the null space, and $(I-\mathsf{\Pi })B=\mathcal{R}(Q)$ be the range values space of operator Q. We will suppose here that the Markov chain $(x_n,n\in \mathrm{N})$, with discrete operator $Q=P-I$, is uniformly ergodic, that is, $∥(P^n-\mathsf{\Pi })\phi ∥\to 0$, as $n\to \infty$, for any $\phi \in B$. In that case, the transition operator is reducible-invertible on B. Thus, we have $B=\mathcal{N}(Q)\oplus \mathcal{R}(Q)$, the direct sum of the two subspaces. The domain of an operator A on B is $\mathcal{D}(A):=\{\phi \in B:A\phi \in B\}$.

Let us define now a discrete-time semi-Markov random evolution (DTSMRE).

Let us define a (forward) discrete-time semi-Markov random evolution ${\mathsf{\Phi }}_k,k\in \mathrm{N}$, on B, by ([13,14]):

$$\begin{array}{c}\hfill {\mathsf{\Phi }}_k\phi =D(z_k)D(z_{k-1})\cdots D(z_2)D(z_1)\phi ,k\ge 1,\mathrm{and}{\mathsf{\Phi }}_0=I.\end{array}$$

(16)

for any $\phi \in B_0:={\cap }_{x\in E}\mathcal{D}(D(x))$. Thus we have ${\mathsf{\Phi }}_k=D(z_k){\mathsf{\Phi }}_{k-1}$.

For example, consider an additive functional of the SMC $(z_k)$, that is, ${\alpha }_k:=u+{\sum }_{\ell =1}^{k}a(z_{\ell })$, for $k\ge 1$, and ${\alpha }_0=u$. Define now a family of operators $D(x),x\in E$, defined on B by $D(x)\phi (u)=\phi (u+a(x))$. Then we can write ${\mathsf{\Phi }}_k\phi (u)={\prod }_{\ell =1}^{k}D(z_{\ell })\phi (u)=\phi (u+{\sum }_{\ell =1}^{k}a(z_{\ell }))=\phi (z_k)$.

The process $M_k$ defined by

$$\begin{array}{c}\hfill M_k:={\mathsf{\Phi }}_k-I-{\displaystyle \sum _{\ell =0}^{k-1}}\mathbf{E}[{\mathsf{\Phi }}_{\ell +1}-{\mathsf{\Phi }}_{\ell }\mid {\mathcal{F}}_{\ell }],\end{array}$$

(17)

on B, is an ${\mathcal{F}}_k$-martingale. The random evolution ${\mathsf{\Phi }}_k$ can be written as follows

$$\begin{array}{c}\hfill {\mathsf{\Phi }}_k:=I+{\displaystyle \sum _{\ell =0}^{k-1}}[D(z_{\ell +1})-I]{\mathsf{\Phi }}_{\ell },\end{array}$$

and then, the martingale (17) can be written as follows

$$\begin{array}{c}\hfill M_k:={\mathsf{\Phi }}_k-I-{\displaystyle \sum _{\ell =0}^{k-1}}\mathbf{E}[(D(z_{\ell +1})-I){\mathsf{\Phi }}_{\ell }\mid {\mathcal{F}}_{\ell }],\end{array}$$

or

$$\begin{array}{c}\hfill M_k:={\mathsf{\Phi }}_k-I-{\displaystyle \sum _{\ell =0}^{k-1}}[\mathbf{E}(D(z_{\ell +1})\mid {\mathcal{F}}_{\ell })-I]{\mathsf{\Phi }}_{\ell }.\end{array}$$

Finally, as $\mathbf{E}[(D(z_{\ell +1}){\mathsf{\Phi }}_{\ell }\phi \mid {\mathcal{F}}_{\ell })]=(P^{♯}D(·){\mathsf{\Phi }}_{\ell }\phi )(z_{\ell },u_{\ell })$, one takes

$$\begin{array}{c}\hfill M_k:={\mathsf{\Phi }}_k-I-{\displaystyle \sum _{\ell =0}^{k-1}}[P^{♯}D(·)-I]{\mathsf{\Phi }}_{\ell }.\end{array}$$

Let us now define the average random evolution $u_k(x)$, $x\in E$, $k\in \mathrm{N}$, by

$$u_k(x):={\mathbf{E}}_x[{\mathsf{\Phi }}_k\phi (z_k)].$$

(18)

Theorem 3.

The random evolution$u_k(x)$satisfy the following Markov renewal equation

$$u_k(x)={\overline{F}}_x(k)D(x)\phi (x)+{\displaystyle \sum _{l=0}^k}{{\displaystyle \int }}_{E}q(x,dy,l)D(y)u_{k-l}(y).$$

(19)

5. Average and Diffusion Approximation with Merging

In this section we present average and diffusion approximation results for the discrete-time semi-Markov random evolution, as well as diffusion approximation with equilibrium or normal deviation.

5.1. Averaging

Let us consider the continuous time process $M_t^{\epsilon }$

$$\begin{array}{c}\hfill M_t^{\epsilon }:=M_{[t/\epsilon ]}={\mathsf{\Phi }}_{[t/\epsilon ]}^{\epsilon }-I-{\displaystyle \sum _{\ell =0}^{[t/\epsilon ]-1}}[P^{♯}{D}^{\epsilon }(·)-I]{\mathsf{\Phi }}_{\ell }^{\epsilon }.\end{array}$$

We will prove here asymptotic results for this process as $\epsilon \to 0$.

The following assumptions are needed for averaging.

A1:

The MC $(z_k,{\gamma }_k,k\in \mathrm{N})$ is uniformly ergodic in each class $E_j$, with ergodic distribution ${\pi }_j^{♯}(B\times \left\{k\right\}),B\in \mathcal{E}\cap E_j,k\in \mathrm{N}$, and the projector operator $\mathsf{\Pi }$ is defined by relation (10).

A2:

The moments $m_2(x),x\in E$, are uniformly integrable, that is, relation (4) holds for $r=2$.

A3:

Let us assume that the perturbed operator $D^{\epsilon }(x)$ has the following representation in B

$$\begin{array}{c}\hfill D^{\epsilon }(x)=I+\epsilon D_1(x)+\epsilon D_0^{\epsilon }(x),\end{array}$$

(20)

where operators $D_1(x)$ on B are closed and $B_0:={\cap }_{x\in E}\mathcal{D}(D_1(x))$ is dense in B, ${\overline{B}}_0=B$. Operators $D_0^{\epsilon }(x)$ are negligible, that is, ${\lim }_{\epsilon \to 0}∥D_0^{\epsilon }(x)\phi ∥=0$ for $\phi \in B_0$.

A4:

We have: ${\int }_E\pi (dx){∥D_1(x)\phi ∥}^2<\infty$.

A5:

There exists Hilbert spaces H and $H^{\ast }$ such that compactly embedded in Banach spaces B and $B^{\ast },$ respectively, where $B^{\ast }$ is a dual space to $B.$

A6:

Operators $D^{\epsilon }(z)$ and ${(D^{\epsilon })}^{\ast }(z)$ are contractive on Hilbert spaces H and $H^{\ast },$ respectively.

We note that if $B=C_0(\mathrm{R})$, the space of continuous function on $\mathrm{R}$ vanishing at infinity, then $H=W^{l,2}(\mathrm{R})$ is a Sobolev space, and $W^{l,2}(\mathrm{R})\subset C_0(\mathrm{R})$ and this embedding is compact (see References [34,43]). For the spaces $B=L_2(\mathrm{R})$ and $H=W^{l,2}(\mathrm{R})$ the situation is the same.

Theorem 4.

Under assumptions A1-A6 and C1-C4 the following weak convergence takes place

$${\mathsf{\Phi }}_{[t/\epsilon ]}^{\epsilon }⟹\widehat{\mathsf{\Phi }}(t),\epsilon \downarrow 0,$$

where the limit random evolution$\widehat{\mathsf{\Phi }}(t)$is determined by the following equation

$$\begin{array}{c}\hfill \widehat{\mathsf{\Phi }}(t)\widehat{\phi }({\widehat{x}}_t)-\widehat{\phi }(u)-{{\displaystyle \int }}_0^t\widehat{\mathrm{L}}\widehat{\mathsf{\Phi }}(s)\widehat{\phi }({\widehat{x}}_s)ds=0,0\le t\le T,\phi \in B_0,\end{array}$$

(21)

with generator$\widehat{\mathrm{L}}$, defined by

$$\begin{array}{c}\hfill \widehat{\mathrm{L}}\mathsf{\Pi }=\mathsf{\Pi }{D}_1\mathsf{\Pi }+\mathsf{\Pi }{Q}_1\mathsf{\Pi }\end{array}$$

(22)

and acting on test functions$\phi (x,v(x))$. The operator$Q_1$is defined by

$$Q_1\phi (x,k)={\lambda }_x(k+1){{\displaystyle \int }}_{E}P(x,dy)\phi (y,0).$$

(23)

Let us consider the average random evolution defined as ${\mathsf{\Lambda }}_x(t):={\mathbf{E}}_x[\widehat{\mathsf{\Phi }}(t)\widehat{\varphi }(u)]$, $x\in E$. Set ${\widehat{D}}_1\mathsf{\Pi }=\mathsf{\Pi }{D}_1\mathsf{\Pi }$ and $\widehat{Q}\mathsf{\Pi }=\mathsf{\Pi }{Q}_1\mathsf{\Pi }$. For detailed description of operator $\widehat{Q}$ see Theorem 1.Then we have the following straightforward result.

Corollary 1.

The average random evolution${\mathsf{\Lambda }}_x(t)$satisfy the following Cauchy problem:

$$\left\{\begin{array}{c}\frac{d{\mathsf{\Lambda }}_x}{dt}(t)=(\widehat{Q}+{\widehat{D}}_1){\mathsf{\Lambda }}_x(t)\hfill \\ {\mathsf{\Lambda }}_x(0)=\widehat{\phi }(u).\hfill \end{array}\right.$$

5.2. Diffusion Approximation

For the diffusion approximation we will consider a different time-scaling and some additional assumptions. In this case, we replace relation (7) by the following one

$$\begin{array}{c}\hfill P^{\epsilon }(x,B)=P(x,B)+{\epsilon }^2{P}_1(x,B).\end{array}$$

(24)

D1:

Let us assume that the perturbed operators $D^{\epsilon }(x)$ have the following representation in B

$$\begin{array}{c}\hfill D^{\epsilon }(x)=I+\epsilon D_1(x)+{\epsilon }^2{D}_2(x)+{\epsilon }^2{D}_0^{\epsilon }(x),\end{array}$$

(25)

where operators $D_2(x)$ on B are closed and $B_0:={\cap }_{x\in E}\mathcal{D}(D_2(x))$ is dense in B, ${\overline{B}}_0=B$; operators $D_0^{\epsilon }(x)$ are a negligible operator, that is, ${\lim }_{\epsilon \downarrow 0}∥D_0^{\epsilon }(x)\phi ∥=0$.

D2:

The following balance condition holds

$$\begin{array}{c}\hfill \mathsf{\Pi }{D}_1(x)\mathsf{\Pi }=0.\end{array}$$

(26)

D3:

The moments $m_3(x),x\in E$, are uniformly integrable, that is, relation (4) holds for $r=3$.

Theorem 5.

Under Assumptions A1, A5-A6 and D1-D3, the following weak convergence takes place

$${\mathsf{\Phi }}_{[t/{\epsilon }^2]}^{\epsilon }⟹{\mathsf{\Phi }}_0(t),\epsilon \downarrow 0,$$

where the limit random evolution${\mathsf{\Phi }}_0(t)$is a diffusion random evolution determined by the following generator$\widehat{\mathrm{L}}$,

$$\begin{array}{c}\hfill \widehat{\mathrm{L}}\mathsf{\Pi }=\mathsf{\Pi }{D}_2\mathsf{\Pi }+\mathsf{\Pi }{D}_1{R}_0{D}_1\mathsf{\Pi }-\mathsf{\Pi }{D}_1^2\mathsf{\Pi }+\widehat{Q}\mathsf{\Pi },\end{array}$$

where the operator$\widehat{Q}$is defined in Theorem 1.

5.3. Normal Deviation with Merging

We note, that averaged semi-Markov random evolutions can be considered as the first approximation to the initial evolutions. The diffusion approximation of the semi-Markov random evolutions determine the second approximation to the initial evolution, since the first approximation under balance condition appears to be trivial.

Here we consider the algorithms of construction of the first and the second approximation in the case when the balance condition in the diffusion approximation scheme is not fulfilled. We introduce the deviated semi-Markov random evolution as the normalized difference between the initial and averaged evolutions. In the limit, we obtain the diffusion approximation with equilibrium of the initial evolution from the averaged one.

Let us consider the discrete-time semi-Markov random evolution ${\mathsf{\Phi }}_{[t/\epsilon ]}^{\epsilon },$ averaged evolution $\widehat{\mathsf{\Phi }}(t)$ (see Section 5.1) and the deviated evolution

$$\begin{array}{c}\hfill {\mathsf{\Psi }}_t^{\epsilon }:={\epsilon }^{-1/2}[{\mathsf{\Phi }}_{[t/\epsilon ]}^{\epsilon }-\widehat{\mathsf{\Phi }}(t)].\end{array}$$

(27)

Theorem 6.

Under Assumptions A1, A5-A6 and D3, with operators$D^{\epsilon }(x)$in A3, instead of D1, the deviated semi-Markov random evolution${\mathsf{\Psi }}_t^{\epsilon }$weakly convergence when$\epsilon \to 0$to the diffusion random evolution${\mathsf{\Psi }}_t^0$defined by the following generator

$$\begin{array}{c}\hfill \widehat{\mathrm{L}}\mathsf{\Pi }=\mathsf{\Pi }(D_1-{\widehat{D}}_1)R_0(D_1-{\widehat{D}}_1)\mathsf{\Pi }+\widehat{Q}\mathsf{\Pi },\end{array}$$

where the operator$\widehat{Q}$is defined in Theorem 1.

6. Application to Particular Systems

In this section we will apply the above Theorems 4–6 to obtain limit results for particular stochastic systems, namely, additive functionals of semi-Markov chains and discrete-time dynamical systems perturbed by semi-Markov chains.

6.1. Integral Functionals

The integral functional of a semi-Markov chain considered here is defined by

$$y_k:={\displaystyle \sum _{l=0}^k}a(z_l),k\ge 0,y_0=u,$$

where a is a real-valued measurable function defined on the state space E.

The perturbing operator $D^{\epsilon }(x)$ is defined as follows

$$D^{\epsilon }(x)\phi (u)=\phi (u+\epsilon a(x)).$$

The perturbed operator $D^{\epsilon }(x)$ has the asymptotic expansions (20), for averaging, and (25), for diffusion approximation, with $D_1(x)\phi (u)=a(x){\phi }^{\prime }(u)$ and $D_2(x)\phi (u)=\frac{1}{2}a(x){\phi }^{\prime \prime }(u)$.

6.1.1. Average Approximation

In the averaging scheme the additive functional has the representation

$$y_t^{\epsilon }:=\epsilon {\displaystyle \sum _{l=0}^{[t/\epsilon ]}}a(z_l),k\ge 0,\epsilon >0,y_0^{\epsilon }=u.$$

(28)

Theorem 7.

Under conditions C1-C4 and A1-A2 the following weak convergence holds

$$y_t^{\epsilon }⟹{\widehat{y}}_t,as\epsilon \to 0,$$

where the limit process is an integral functional, defined by

$${\widehat{y}}_t:=u+{{\displaystyle \int }}_0^t\widehat{a}({\widehat{x}}_s)ds,$$

with$\widehat{a}(j)={\int }_{E_j}{\pi }_j(dx)a(x)$. The Markov process${\widehat{x}}_t$, is defined on the state space$\widehat{E}$as in the previous section by the generator$\widehat{Q}$, defined in Theorem 1

It is worth noticing here that the initial processes (28) are switched by a SMC, and the limit process by a continuous-time Markov process on a finite state space, which is much simpler.

6.1.2. Diffusion Approximation

In the diffusion approximation the additive functional is time-rescaled as follows

$${\xi }_t^{\epsilon }:=\epsilon {\displaystyle \sum _{l=0}^{[t/{\epsilon }^2]}}a(z_l),t\ge 0,\epsilon >0,{\xi }_0^{\epsilon }=u.$$

(29)

Then we have the following result.

Theorem 8.

Under conditions C1-C4, A1-A6 and D1-D2 the following weak convergence holds

$${\xi }_t^{\epsilon }⟹{\xi }_t^0,as\epsilon \to 0,$$

where the limit process is a switched diffusion process,

$$d{\xi }_t^0=b({\widehat{x}}_t)d{W}_t,and{\xi }_0^0=u.$$

The process$W_t,t\ge 0$, is a standard Brownian motion, and$b^2(j):={\widehat{a}}_0(j)-\frac{1}{2}{\widehat{a}}_2(j).$The coefficients are

$${\widehat{a}}_0(j):={{\displaystyle \int }}_{E_j}{\pi }_j(dx)a(x)R_{0}a(x),and{\widehat{a}}_2(j):={{\displaystyle \int }}_{E_j}{\pi }_j(dx)a^2(x).$$

It is worth noticing that the generator of the diffusion ${\xi }_0(t)$ can be written as follows

$$\mathrm{L}\phi (u,x)=\frac{1}{2}{\displaystyle \sum _{j=1}^d}{b}^2(j){\widehat{\phi }}_j^{\prime \prime }(u){\mathbf{1}}_j(x).$$

(30)

In fact the limit process is switched by the merged Markov process ${\widehat{x}}_t$, defined in Theorem 1.

6.1.3. Normal Deviation

The diffusion approximation with equilibrium will be realized without balance condition $D2$. Let us consider the stochastic processes ${\zeta }_t^{\epsilon },t\ge 0,\epsilon >0$,

$${\zeta }_t^{\epsilon }:={\epsilon }^{-1/2}(y_t^{\epsilon }-{\widehat{y}}_t).$$

The process ${\widehat{y}}_t$ is the limit process in the averaging scheme.

Then we have the following weak convergence result.

Theorem 9.

Under conditions C1-C4 and A1-A2 the following weak convergence holds

$${\zeta }_t^{\epsilon }⟹{\zeta }_t^0,as\epsilon \to 0,$$

where the limit process is a switched diffusion process,

$$d{\zeta }_0(t)=c({\widehat{x}}_t)d{W}_t,and{\zeta }_0^0=0.$$

The process$W_t,t\ge 0$, is a standard Brownian motion, and

$$c^2(j):={{\displaystyle \int }}_{E_j}{\pi }_j(dx)(\widehat{a}(j)-a(x))R_0(\widehat{a}(j)-a(x)),1\le j\le d.$$

The limit process here is also switched by the merged Markov process ${\widehat{x}}_t$.

6.2. Discrete Dynamical Systems

Let us consider the family of difference equations

$$\begin{array}{c}\hfill y_{k+1}^{\epsilon }=y_k^{\epsilon }+\epsilon C(y_k^{\epsilon };z_{k+1}),k\ge 0,\mathrm{and}{y}_0^{\epsilon }=u,\end{array}$$

(31)

switched by the SMC $(z_k)$.

The perturbed operators $D^{\epsilon }(x),x\in E$, are defined now by

$$D^{\epsilon }(x)\phi (u)=\phi (u+\epsilon C(u,x)).$$

The perturbed operator $D^{\epsilon }(x)$ has the asymptotic expansions (20), for averaging, and (25), for diffusion approximation, with $D_1(x)\phi (u)=C(u,x){\phi }^{\prime }(u)$ and $D_2(x)\phi (u,x)=\frac{1}{2}C(u,x){\phi }^{\prime \prime }(u)$.

6.2.1. Average Approximation

The time-scaled system considered here is

$$\begin{array}{c}\hfill y_t^{\epsilon }:=y_{[t/\epsilon ]+1}^{\epsilon }=y_{[t/\epsilon ]}^{\epsilon }+\epsilon C(y_{[t/\epsilon ]}^{\epsilon };z_{[t/\epsilon ]+1}),t\ge 0,\mathrm{and}{y}_0^{\epsilon }=u.\end{array}$$

(32)

Theorem 10.

Under conditions C1-C4 and A1-A2 the following weak convergence holds

$$y_t^{\epsilon }⟹{\widehat{y}}_t,as\epsilon \to 0,$$

where the limit process is a continuous-time dynamical system, defined by

$$\frac{d}{dt}{\widehat{y}}_t:=\widehat{C}({\widehat{y}}_t;{\widehat{x}}_t),and{\widehat{y}}_0^0=u$$

with$\widehat{C}(u,j)={\int }_{E_j}{\pi }_j(dx)C(u,x)$.

6.2.2. Diffusion Approximation

The time-scaled dynamical system considered here is

$$\begin{array}{c}\hfill {\xi }_t^{\epsilon }:=y_{[t/{\epsilon }^2]+1}^{\epsilon }=y_{[t/{\epsilon }^2]}^{\epsilon }+\epsilon C(y_{[t/{\epsilon }^2]}^{\epsilon };z_{[t/{\epsilon }^2]+1}),t\ge 0,\mathrm{and}{y}_0^{\epsilon }=u.\end{array}$$

(33)

Theorem 11.

Under conditions C1-C4, A1-A2 and D2 the following weak convergence holds

$${\xi }_t^{\epsilon }⟹{\xi }_t^0,as\epsilon \to 0,$$

where the limit process is a switched diffusion process,

$$d{\xi }_t^0=c({\widehat{x}}_t)d{W}_t,and{\xi }_0^0=u.$$

The process$W_t,t\ge 0$, is a standard Brownian motion, and$c^2(j):=\frac{1}{2}\widehat{C}(u,j)+{\widehat{C}}_0(u,j)-{\widehat{C}}_2(u,j).$

The coefficients are

$$\widehat{C}(u,j):={{\displaystyle \int }}_{E_j}{\pi }_j(dx)C(u,x),{\widehat{C}}_0(u,j):={{\displaystyle \int }}_{E_j}{\pi }_j(dx)C(u,x)R_{0}C(u,x),$$

$$and{\widehat{C}}_2(u,j):={{\displaystyle \int }}_{E_j}{\pi }_j(dx)C^2(u,x).$$

6.2.3. Normal Deviation

The time-scaled system considered here is

$$\begin{array}{c}\hfill {\zeta }_t^{\epsilon }:={\epsilon }^{-1/2}(y_{[t/\epsilon ]}^{\epsilon }-{\widehat{y}}_t),t\ge 0,\mathrm{and}{\zeta }_0^{\epsilon }=u.\end{array}$$

(34)

Theorem 12.

Under conditions C1-C4 and A1-A2 the following weak convergence holds

$${\zeta }_t^{\epsilon }⟹{\zeta }_t^0,as\epsilon \to 0,$$

where the limit process is a switched diffusion process,

$$d{\zeta }_t^0=c_e({\widehat{x}}_t)d{W}_t,and{\zeta }_0^0=u.$$

The process$W_t,t\ge 0$, is a standard Brownian motion, and

$$c_e^2(j):={{\displaystyle \int }}_{E_j}{\pi }_j(dx)[C(u,x)-\widehat{C}(u,j)]R_0[C(u,x)-\widehat{C}(u,j)].$$

7. Proofs

As the state space $\widehat{E}$ of the switching process $z_k$ is a finite set, we do not consider the new component $v(z_k)$, and the proofs of tightness in Reference [14] are also valuable here. So, we will only prove here the finite dimensional distributions convergence concerned by the transition kernels.

7.1. Proof of Theorem 1

Proof. 

Let us consider the extended Markov renewal process

$$x_n^{\epsilon },v(x_n^{\epsilon }),{\tau }_n^{\epsilon },t\ge 0,\epsilon >0,$$

(35)

where $n:=[t/\epsilon ]$, $x_n^{\epsilon }=z^{\epsilon }({\tau }_n^{\epsilon })$ and ${\tau }_{n+1}^{\epsilon }={\tau }_n^{\epsilon }+[\epsilon {\theta }_{n+1}^{\epsilon }]$.

The compensating operator of this process is defined by the following relation, (see Reference [1]),

$${\mathrm{L}}^{\epsilon }\phi (x,v(x),k)={\epsilon }^{-1}q(x)\left\{\mathbf{E}[\phi (x_1^{\epsilon },v(x_1^{\epsilon }),{\tau }_1^{\epsilon })\mid x_0^{\epsilon }=x,v(x_0^{\epsilon })=j,{\tau }_0^{\epsilon }=k]-\phi (x,j,k)\right\}.$$

(36)

The compensating operator ${\mathrm{L}}^{\epsilon }$ acting on test functions $\phi (x,v(x))$, $x\in E$, can be written as follows

$${\mathrm{L}}^{\epsilon }\phi (x,v(x))={\epsilon }^{-1}q(x){{\displaystyle \int }}_E{P}^{\epsilon }(x,dy)[\phi (y,v(y))-\phi (x,v(x))].$$

(37)

And now from (7), the operator ${\mathrm{L}}^{\epsilon }$, can be written as follows

$${\mathrm{L}}^{\epsilon }={\epsilon }^{-1}Q+Q_1,$$

(38)

where

$$Q\phi (x,v(x))=q(x){{\displaystyle \int }}_{E}P(x,dy)[\phi (y,v(y))-\phi (x,v(x))],$$

(39)

and

$$Q_1\phi (x,v(x))=q(x){{\displaystyle \int }}_E{P}_1(x,dy)\phi (y,v(y)).$$

(40)

Now, by the following singular perturbation problem, on test functions ${\phi }^{\epsilon }(x,v(x))=\widehat{\phi }(v(x))+\epsilon {\phi }_1(x,v(x))$,

$${\mathrm{L}}^{\epsilon }{\phi }^{\epsilon }(x,v(x))=\mathrm{L}\phi (x)+\epsilon {\theta }^{\epsilon }(x)$$

(41)

and from Proposition 5.1 in Reference [1], we get the limit operator $\mathrm{L}$, whose the contracting form, $\widehat{\mathrm{L}}$, defined by the relation

$$\mathsf{\Pi }{Q}_1\mathsf{\Pi }=\widehat{\mathrm{L}}\mathsf{\Pi }$$

provide us directly the generator $\widehat{Q}\equiv \widehat{\mathrm{L}}$ of the limit process $\widehat{x}(t)$, and the prove is achieved. □

Proof of Theorem 2 is the same as the previous one.

7.2. Proof of Theorem 3

Proof. 

We have ${\mathbf{E}}_x[{\mathsf{\Phi }}_k\phi (z_k){\mathbf{1}}_{\{{\tau }_1>k\}}]={\overline{F}}_x(k)D(x)\phi (x)$, and ${\mathbf{E}}_x[{\mathsf{\Phi }}_k\phi (z_k){\mathbf{1}}_{\{{\tau }_1\le k\}}]={\sum }_{l=0}^k{\int }_{E}q(x,dy,l){\mathbf{E}}_x[{\mathsf{\Phi }}_k\phi (z_k)\mid x_1=y,{\tau }_1=l]={\sum }_{l=0}^k{\int }_{E}q(x,dy,l)D(y){\mathbf{E}}_y[{\mathsf{\Phi }}_{k-l}\phi (z_{k-l})]$, and the result follows. □

7.3. Proof of Theorem 4

Proof. 

The perturbed semi-Markov kernel $q^{\epsilon }$, has the representation $q^{\epsilon }(x,B,k)=q(x,B,k)+\epsilon q_1(x,B,k)$, where $q_1(x,dy,k):=P_1(x,dy)f_x(k).$

The discrete generators of the four component family of processes ${\mathsf{\Phi }}_{[t/\epsilon ]}^{\epsilon }\phi$, $z_{[t/\epsilon ]}^{\epsilon }$, $v(z_{[t/\epsilon ]}^{\epsilon })$, ${\gamma }_{[t/\epsilon ]}^{\epsilon }$, $t\ge 0$, $\epsilon >0$, are

$${\mathrm{L}}^{\epsilon }\phi (u,x,v(x),k)={\epsilon }^{-1}{{\displaystyle \int }}_E{P}^{♯}(x,k;dy,k+1)[D^{\epsilon }(y)\phi (u,y;v(y),k+1)-\phi (u,x,v(x),k)].$$

(42)

The asymptotic representation of the above operator acting on test functions $\phi (u,x,v(x),k)$ is given by

$${\mathrm{L}}^{\epsilon }\phi (u,x,v(x),k)=[{\epsilon }^{-1}{Q}^{♯,\epsilon }+P^{♯,\epsilon }{D}_1(·)+P^{♯,\epsilon }{D}_0^{\epsilon }(·)]\phi (u,x,v(x),k),$$

(43)

where $Q^{♯,\epsilon }:=P^{♯,\epsilon }-I$.

Now from (2) the transition operator $P^{♯,\epsilon }$ can be written as follows $P^{♯,\epsilon }=P^{♯}+\epsilon Q_1$, where the operator $Q_1$ is defined by relation (23).

Finally, the asymptotic representation of the operator ${\mathrm{L}}^{\epsilon }$ can be written

$${\mathrm{L}}^{\epsilon }(x)={\epsilon }^{-1}{Q}^{♯}+Q_1+P^{♯}{D}_1(x)+{\theta }^{\epsilon }(x)$$

(44)

where $Q^{♯}:=P^{♯}-I$, and the negligible operator ${\theta }^{\epsilon }(x)$ is given by ${\theta }^{\epsilon }(x):=P^{♯,\epsilon }{D}_0^{\epsilon }(x)+\epsilon Q_1(D_1(x)+D_0^{\epsilon }(x)]$.

From the singular perturbation problem ${\mathrm{L}}^{\epsilon }\phi (u,x)=\mathrm{L}\phi (v(x))+{\theta }^{\epsilon }(x)$, with test functions ${\phi }^{\epsilon }=\phi +\epsilon {\phi }_1$, we get the limit operator defined by $\widehat{\mathrm{L}}\mathsf{\Pi }=\mathsf{\Pi }(P^{♯}{D}_1(x)+Q_1)\mathsf{\Pi }$, see, for example, Proposition 5.1 in Reference [1]. From this representation, we get

$$\widehat{\mathrm{L}}={\widehat{D}}_1+\widehat{Q}$$

where ${\widehat{D}}_1\mathsf{\Pi }=\mathsf{\Pi }{D}_1(x)\mathsf{\Pi }$ and $\widehat{Q}\mathsf{\Pi }=\mathsf{\Pi }{Q}_1\mathsf{\Pi }$. □

7.4. Proof of Theorem 5

Proof. 

The discrete generators of the four component family of processes ${\mathsf{\Phi }}_{[t/{\epsilon }^2]}^{\epsilon }\phi$, $z_{[t/{\epsilon }^2]}^{\epsilon }$, $v(z_{[t/{\epsilon }^2]}^{\epsilon })$, ${\gamma }_{[t/{\epsilon }^2]}^{\epsilon }$, $t\ge 0$, $\epsilon >0$, are

$${\mathrm{L}}^{\epsilon }(x)={\epsilon }^{-2}{Q}^{♯}+{\epsilon }^{-1}{P}^{♯}{D}_1(x)+Q_1+P^{♯}{D}_2(x){\mathsf{\Theta }}^{\epsilon }(x).$$

(45)

And solving singular perturbation problem ${\mathrm{L}}^{\epsilon }(x)\phi (x,k)=\mathrm{L}\phi (v(x))+{\theta }^{\epsilon }(x,k)$, with test functions ${\phi }^{\epsilon }=\phi +\epsilon {\phi }_1+{\epsilon }^2{\phi }_2$, see, for example, Proposition 5.2 in Reference [1], we get the desired result. □

The proof of Theorem 6 is similar to the previous ones.

Finally, the proofs of Theorems 7 to 12 are obtained directly as corollaries from Theorems 4 to 6.

8. Concluding Remarks

In this paper, we presented the semi-Markov random evolution in reduced random media, where the main results were given in Section 4 and Section 5. And their applications for integral functionals of semi-Markov chains and dynamical systems perturbed by semi-Markov chains were given in Section 6. These kind of results have to be extended to many other stochastic systems, as in hidden semi-Markov chains, controlled systems, epidemiology systems, and so forth.

For simplicity, we consider fixed initial conditions of processes, that is, independent of $\epsilon$. This is not a loss of generality since we can add dependent on $\epsilon$ initial conditions without any problem, see, for example, References [1,2].

It is worth noticing that the discrete-time semi-Markov chains have to be developed, in parallel with the continuous-time semi-Markov processes, as is the case with discrete-time Markov chains and continuous-time Markov processes.