Controlled Discrete-Time Semi-Markov Random Evolutions and Their Applications

: In this paper, we introduced controlled discrete-time semi-Markov random evolutions. These processes are random evolutions of discrete-time semi-Markov processes where we consider a control. applied to the values of random evolution. The main results concern time-rescaled weak convergence limit theorems in a Banach space of the above stochastic systems as averaging and diffusion approximation. The applications are given to the controlled additive functionals, controlled geometric Markov renewal processes, and controlled dynamical systems. We provide dynamical principles for discrete-time dynamical systems such as controlled additive functionals and controlled geometric Markov renewal processes. We also produce dynamic programming equations (Hamilton– Jacobi–Bellman equations) for the limiting processes in diffusion approximation such as controlled additive functionals, controlled geometric Markov renewal processes and controlled dynamical systems. As an example, we consider the solution of portfolio optimization problem by Merton for the limiting controlled geometric Markov renewal processes in diffusion approximation scheme. The rates of convergence in the limit theorems are also presented.

Discrete-time semi-Markov chains (SMC) have only recently been used in applications. Especially, in DNA analysis, image and speech processing, reliability theory, etc., see in [19] and references therein. These applications have stimulated a research effort in this area. While the literature in discrete-time Markov chains theory and applications is quite extensive, there is only a small amount of the literature on SMC and most of them are related to hidden semi-Markov models for estimation.
The present article is a continuation of our previous work [20]. Thus, we keep all our notation and definitions the same as in the latter paper. Compared with our previous work [20], where we studied random evolutions of semi-Markov chains, here we considered additionally a control on the random evolution, which we call controlled discrete-time semi-Markov random evolution (CDTSMRE) in a Banach space, and we presented timerescaled convergence theorems. In particular, we get weak convergence theorems in Skorokhod space D[0, ∞) for càdlàg stochastic processes, see, e.g., in [21]. The limit theorems include averaging, diffusion approximation, and diffusion approximation with equilibrium. For the above limit theorems we also presented rates of convergence results. Finally, we give some applications regarding the above mentioned results, especially to controlled additive functionals (CAF), CGMRP, and controlled dynamical systems (CDS), and optimization problems.
Regarding the optimization problems, we provide dynamical principles for discretetime dynamical systems such as CAF and CGMRPs (see Section 2.4), see, e.g., [22][23][24]. We also produce dynamic programming equations (Hamilton-Jacobi-Bellman equations) for the limiting processes in diffusion approximation such as CAF, CGMRP, and CDS. As an example, we consider the solution of portfolio optimization problem by Merton for the limiting CGMRP in DA (see Section 4.4). Merton problem, or Merton portfolio's problem, is a problem in continuous-time finance associated with portfolio choice. In (B, S)-security market, which consists of a stock and a risk-free asset, an investor must choose how much to consume, and must allocate his wealth between the stock and the risk-free asset in a such way that maximizes expected utility. The problem was formulated and first solved by Robert Merton in 1969, and published in 1971 [25].
Results presented here are new and deals with CDTSMRE on Banach spaces. This paper contains new and original results on dynamical principle for CDTSMRE and DPE (HJB equations) for the limiting processes in DA. One of the new remarkable results is the solution of Merton portfolio problem for the limiting CGMRP in DA. The method of proofs was based on the martingale approach together with convergence of transition operators of the extended semi-Markov chain via a solution of a singular perturbation problem [3,4,26]. As in our previous work [20], the tightness of these processes is proved via Sobolev's embedding theorems [27][28][29]. It is worth mentioning that, as in the Markov case, the results presented here cannot be deduced directly from the continuous-time case. We should also note that that DTSMREs have been completely studied in [20]. For semi-Markov processes see, e.g., [30][31][32][33]. For Markov chains and additive functionals see, e.g., [34][35][36][37][38].
The paper is organized as follows. Definition and properties of discrete-time semi-Markov random evolutions and Controlled DTSMREs, as well as particular stochastic systems as applications, are introduced in Section 2. The main results of this paper, limit theorems of CDTSMRE, as averaging, diffusion approximation and diffusion approximation with equilibrium of controlled DTSMREs are considered in Section 3. In Section 4, we provide three applications of averaging, diffusion approximation, and diffusion approximation with equilibrium of controlled DTSMREs: controlled additive functionals, controlled GMRP, and controlled dynamical systems. Section 5 deals with the analysis of the rates of convergence in the limit theorems, presented in the previous sections, for controlled DTSMREs and for CAF and CGMRP. In Section 6, we give the proofs of theorems presented in the previous sections. The last section concludes the paper and indicates some future works.

Semi-Markov Chains
The aim of this section is to present some notation and to make this paper as autonomous as possible. The reader may refer to our article in [20] for more details.
Let (E, E ) be a measurable space with countably generated σ-algebra and (Ω, F , (F n ) n∈I N , P) be a stochastic basis on which we consider a Markov renewal process (x n , τ n , n ∈ IN) in discrete time k ∈ IN, with state space (E, E ). Notice that IN is the set of non-negative integer numbers. The semi-Markov kernel q is defined by (see, e.g., in [9,19]), We will denote also q(x, B, Γ) = ∑ n∈Γ q(x, B, n), where Γ ⊂ IN. The process (x n ) is the embedded Markov chain of the MRP (x n , τ n ) with transition kernel P(x, dy). The semi-Markov kernel q is written as where f xy (k) := P(τ n+1 − τ n = k | x n = x, x n+1 = y), the conditional distribution of the sojourn time in state x given that the next visited state is y.
Define also the counting process of jumps ν k = max{n : τ n ≤ k}, and the discrete-time semi-Markov chain z k by z k = x ν k , for k ∈ IN. Define now the backward recurrence time process γ k := k − τ ν k , k ≥ 0, and the filtration F k := σ(z , γ ; ≤ k), k ≥ 0.
Let us consider a separable Banach space B of real-valued measurable functions defined on E × IN, endowed with the sup norm · and denote by B its Borel σ-algebra. The Markov chain (z k , γ k ), k ≥ 0, has the following transition probability operator P on B where ϕ ∈ B, and its stationary distribution, if there exist, is given by The probability measure π defined by π(B) = π (B × IN) is the stationary probability of the SMC (z k ). Define also the r-th moment of holding time in state x ∈ E, Define now the stationary projection operator Π on the null space of the (discrete) generating operator Q := P − I, where 1(x, s) = 1 for any x ∈ E, and s ∈ IN. This operator satisfies the equations The potential operator of Q , denoted by R 0 , is defined by

General Definition and Properties of DTSMREs
We define here controlled discrete-time semi-Markov random evolutions. Let U denote a compact Polish space representing the control, and let u k be U-valued control process and we suppose that it is a Markov chain. We note that we could also define the process u ν k which is a semi-Markov control process, considered in many papers (see, e.g., in [39,40]). We suppose that homogeneous Markov chain u k is independent of z k , and transition probability kernel P u = P(u k+1 ∈ dy | u k = u) = Q(u, dy).
Let us consider a family of bounded contraction operators D(z, u), z ∈ E, u ∈ U, defined on B, where the maps D(z, u)ϕ : E × U → B are E × U -measurable, ϕ ∈ B. Denote by I the identity operator on B. Let ΠB = N (Q ) be the null space, and (I − Π)B = R(Q ) be the range values space of operator Q . We will suppose here that the Markov chain (z k , γ k , k ∈ IN) is uniformly ergodic, that is, ((P ) n − Π)ϕ → 0, as n → ∞, for any ϕ ∈ B. In that case, the transition operator is reducible-invertible on B. Thus, we have B = N (Q ) ⊕ R(Q ), the direct sum of the two subspaces. The domain of an operator A on B is denoted by D(A) := {ϕ ∈ B : Aϕ ∈ B}.
The process M u k defined by on B, is an F u k -martingale. The random evolution Φ u k can be written as follows and then, the martingale (5) can be written as follows,

Example 1. Controlled Additive Functional or Markov Decision Process.
Let define the following controlled additive functional, If we define the operator D(z, u) on C 0 (IR) in the following way, D(z, u)ϕ(y) := ϕ(y + a(z, u)), then the controlled discrete-time semi-Markov random evolution Φ k ϕ has the following presentation, Process y u k is usually called in the literature the Markov decision process (see, e.g., in [41][42][43][44]).

Example 2. Controlled geometric Markov renewal process.
The CGMRP is defined in the following way, We suppose that ∏ 0 k=1 = 1. If we define the operator D(z, u) on C 0 (IR) in the following way, D(z, u)ϕ(s) := ϕ(s(1 + a(z, u))), then the controlled discrete-time semi-Markov random evolution Φ u k ϕ can be given as follows, To the authors opinion, this process is defined for the first time in the literature and the notion of controlled GMRP is a new one as well.

Dynamic Programming for Controlled Models
Here, we present dynamic programming for controlled models given in Examples in previous section. Let us consider a Markov control model (see in [45] Here, E is the state space; A is the control or action set; Q is the transition kernel, i.e., a stochastic kernel on E given K, where K := {(z, u)|z ∈ E, u ∈ A(z)}; and c : K → R is a measurable function called the cost-per-stage function.
We are interested in is to minimize the finite-horizon performance criterion either (see Example 1) or (see Example 2) where a N (z N ) is the terminal cost function, π ∈ Π is the set of control policies. In this way, denoting by J * the value function Then, the deterministic Markov policy π * = { f 0 , . . . , f N−1 } is optimal, and the value function J * equals J 0 , i.e., J * 2 (z) = J 0 (z) = J 2 (π * , z), z ∈ E.

Limit Theorems for Controlled Semi-Markov Random Evolutions
In this section, we present averaging, diffusion approximation, and diffusion approximation with equilibrium results for the controlled discrete-time semi-Markov random evolutions. It is worth noticing that the main scheme of results are almost the same as in our previous works in particular [20]. Nevertheless, the additional component of the control allows us to study more interesting problems.

Averaging of CDTSMREs
We consider here CDTSMREs defined in Section 2. Let us now set k := [t/ε] and consider the continuous time process We will prove here asymptotic results for this process as ε → 0.
The following assumptions are needed for averaging.
where operators D 1 (x, u) on B are closed and B 0 : A5: There exists Hilbert spaces H and H * such that compactly embedded in Banach spaces B and B * , respectively, where B * is a dual space to B. A6: Operators D ε (x) and (D ε ) * (x) are contractive on Hilbert spaces H and H * , respectively. A7: The MC (u k , k ∈ IN), is independent of (z k ), and is uniformly ergodic with stationary distribution π 1 (du), k ∈ IN.
We note that if B = C 0 (IR), then H = W l,2 (IR) is a Sobolev space, and W l,2 (IR) ⊂ C 0 (IR) and this embedding is compact (see [29]). For the spaces B = L 2 (IR) and H = W l,2 (IR) the situation is the same.
We also note, that semi-Markov chain (z k , u k ) is uniformly ergodic on E × U with stationary probabilities π(dx)π 1 (du), which follows from conditions A1 and A7. Theorem 1. Under Assumptions A1-A7, the following weak convergence takes place, where the limit random evolution Φ(t) is determined by the following equation, or, equivalently, where the limit contracted operator is then given by This result generalize the classical Krylov-Bogolyubov averaging principle [46] on a Banach and a controlled spaces.

Diffusion Approximation of DTSMREs
For the diffusion approximation of CDTSMREs, we will consider a different timescaling and some additional assumptions.
D1: Let us assume that the perturbed operators D ε (x, u) have the following representation in B, The following balance condition holds, where D3: The moments m 3 (x), x ∈ E, are uniformly integrable.

Diffusion Approximation with Equilibrium
The diffusion approximation with equilibrium or the normal deviation is obtained by considering the difference between the rescaled initial processes and the averaging limit process. This is of great interest when we have no balance condition as previously in the standard diffusion approximation scheme.
Consider now the controlled discrete-time semi-Markov random evolution Φ ε [t/ε,u] , averaged evolution Φ(t) (see Section 3.1) and the deviated evolution Theorem 3. Under Assumptions A1, A5-A6 (see Section 3.1), and D3, with operators D ε (x) in A3, instead of D1, the deviated controlled semi-Markov random evolution W ε,u t weakly convergence, when ε → 0, to the diffusion random evolution W 0 t defined by the following generator where Π is defined in (9).

Applications to Stochastic Systems
In this section, we give two applications in connection with the above results: additive functionals that has many application, e.g., in storage, reliability, and risk theories (see, e.g., in [3,4,19,47]), and to geometric Markov renewal processes, that also have many application including finance (see [15][16][17][18]). Our main goal here is to get the limiting processes and apply optimal control methods to receive the solutions of optimization problems. The limiting results for MC such as LLN and CLT were considered in [11,12].

Controlled Additive Functionals
Let us consider here the CAF, (y u k ), described previously in Example 1. Averaging of CAF. Now, if we define the continuous time process then from Theorem 1 it follows that this process has the following limit y 0 (t) = lim ε→0 y ε t y 0 (t) = y +ât, whereâ = E U π(dz)π 1 (du)a(z, u). We suppose that E U π(dz)π 1 (du)|a(z, u)| < +∞. Diffusion Approximation of CAF. If we consider the continuous time process ξ ε,u t as follows then under balance condition E U π(dz)π 1 (du)a(z, u) = 0 and E U π(dz)π 1 (du)|a(z, u)| 2 < +∞ we get that the limit process ξ 0 (t) = lim ε→0 ξ ε t has the following form, and w t is a standard Wiener process. Diffusion Approximation with Equilibrium of CAF. Let us consider the following normalized additive functional, Then, this process converges to the following process, σw t , where and w t is a standard Wiener process. In this way, the AF y ε t may be presented in the following approximated form,
Then, under averaging conditions the limit processS t has the following form, whereâ = E U π(dz)π 1 (du)a(z, u).
Diffusion Approximation of CGMRP. If we define the following sequence of processes, then, in the diffusion approximation scheme, we have the following limit process, S 0 (t) S 0 (t) = S 0 e −tâ 2 /2 e σ a w(t) , It means that S 0 (t) satisfies the following stochastic differential equation, where w t is a standard Wiener process.
Diffusion Approximation with Equilibrium of CGMRP. Let us consider the following normalized GMRP: w ε,u t := ε −1/2 [ln(S ε,u t /S 0 ) −ât]. It is worth noticing that in finance the expression ln(S ε,u t /S 0 ) represents the log-return of the underlying asset (e.g., stock) S ε,u t . Then, this process converges to the following process, σw t , where and w t is a standard Wiener process.
In this way, the GMRP S ε t may be presented in the following approximated form,

Controlled Dynamical Systems
We consider here discrete-time CDS and their asymptotic behaviour in series scheme: average and diffusion approximation ( [9]).
Define the measurable function C on IR × E × U. Let us consider the difference equation switched by the SMC (z k ). The perturbed operators D ε (z, u), x ∈ E, are defined now by

Averaging of CDS.
Under averaging assumptions the following weak convergence takes place, where y(t), t ≥ 0 is the solution of the following (deterministic) differential equation, d dt y(t) = C(y(t)), and y(0) = u, where C(z) = E U π(dx)π 1 (du)C(z, x, u). Diffusion Approximation of CDS. Under diffusion approximation conditions the following weak convergence takes place where x t , t ≥ 0, is a diffusion processes, with initial value x 0 = u, determined by the operator provided that b 2 (z) > 0, and drift and diffusion coefficients are defined as follows, with: C 0 (z) := E U π(dx)π 1 (du)C 0 (z, x, u), C 0 (z, x, u) := C(z, x, u)R 0 C(z, x, u),

The Dynamic Programming Equations for Limiting Models in Diffusion Approximation
In this section, we consider the DPE, i.e., HJB Equations, for the limiting models in DA from Sections 4.1-4.3. As long as all limiting processes in DA in Sections 4.1-4.3 are diffusion processes, then we will set up a general approach to control for diffusion processes, see in [48]. Let x u t be a diffusion process satisfying the following stochastic differential equation, where u t is the control process, w t is a standard Wiener process. Let us also introduce the following performance criterion function, J u (t, x) where G(x) : R → R is a terminal reward function (uniformly bounded), F(t, x, u) : R + × R 2 → R is a running penalty/reward function (uniformly bounded), 0 ≤ t ≤ T. The problem is to maximize this performance criteria, i.e., to find the value function where U t,T is the admissible set of strategies/controls which are F -predictable, nonnegative, and bounded. The Dynamic Programming Principle (DPP) for diffusions states that the value function J(t, x) satisfies the DPP Moreover, the value function J(t, x) above satisfies the Dynamic Programming Equation (DPE) or Hamilton-Jacobi-Bellman (HJB) Equation: where L u t is an infinitesimal generator of the diffusion process x u t above, i.e.,

• DPE/HJB Equation for the Limiting CDS in DA (see Section 4.3)
We remind that in the diffusion approximation the limiting process is a diffusion process x t with a generator provided that b 2 (z) > 0, and drift and diffusion coefficients are defined as follows, where C * means transpose of the vector C, C 01 (z) := E U π(dx)π 1 (du)C 01 (z, x, u), C 01 (z, x, u) := C(z, x, u)R 0 C z (z, x, u), C 1 (z) := E U π(dx)π 1 (du)C 1 (z, x, u), C 1 (z, x, u) := C(z, x, u)C z (z, x, u).
In this case the DPE or HJB Equation (16) reads with the generator

Remark 1.
Our construction here is equivalent to some extend to "Recurrent Processes of a semi-Markov type (RPSM)" studied first in [13,14] including limit theorems. Those results were described in more detail in [11,12]. In particular, "RPSM with Markov switching" reflects the case of independent Markov components z k and u k , and "General case of RPSM" reflects the case when u k is dependent on z k .

• The Merton Problem
This is an example of solution of DPE/HJB equation for the limiting CGMRP in DA. Let us consider the portfolio optimization problem proposed by Merton (1971), see in [25]. We will apply this approach to the limiting CGMRP in DA above. In this problem, the agent seeks to maximize expected wealth by trading in a risky asset and the risk-free bonds (or bank account). She/he places $π t for a total wealth X t in the risky asset S 0 (t) and looks to obtain the value function (performance criterion) which depends on the current wealth x and asset price S, and the optimal trading strategy π, U(x) is the agent's utility function (e.g., exponential (−e −γx ) or power x γ ). We suppose that the asset price S 0 (t) satisfies the following SDE Here, µ represents the expected continuously compounded rate of growth of the traded asset, r is the continuously compounded rate of return of the risk-free asset (bond or bank account).
The wealth process X π t follows the following SDE, From the SDEs for S 0 (t) and for X π t above we conclude that the infinitesimal generator for the pair (S 0 (t), X π t ) is From HJB equation for the limiting CGRMP in DA it follows that the value function should satisfy the equation The explicit solution of this PDE depends on the explicit form of the utility function U(x). Let us take the exponential utility function In this case we can find that the optimal amount to invest in the risky asset is a deterministic function of time

Rates of Convergence in Averaging and Diffusion Approximations
The rate of convergence in a limit theorem is important in several ways, both theoretical and practical. We present here the rates of convergence of CDTSMRE in the averaging, diffusion approximation and diffusion approximation with equilibrium schemes and, as corollaries, we give the rates of convergence for CAF and CGMRP in the corresponding limits.
The proof of this proposition is given in Section 6.4.
The proofs of the above Propositions 2 and 3 are similar as the proof of Proposition 1. We give in what follows some rate of convergence results (Corollaries 1 and 2 where a(T, ||R 0 ||, ||a||) is a constant, and 0 ≤ t ≤ T.

Proofs
The proofs here have almost the same general construction scheme as in our paper [20] except that we consider also the control process. Let C B [0, ∞) be the space of B-valued continuous functions defined on [0, ∞).

Proof of Theorem 1
The proof of the relative compactness of CDTSMRE in the average approximation is based on the following four lemmas.
Proof. Assumptions A5-A6 imply that the discrete-time semi-Markov random evolution Φ u k ϕ is a contractive operator in H and, therefore, ||Φ u k ϕ|| H is a supermartingale for any ϕ ∈ H, where || · || H is a norm in Hilbert space H ( [4,9]) Obviously, the same properties satisfy the following family Φ ε,u [t/ε] . Using Doob's inequality for the supermartingale ||Φ ε,u where K ∆ is a compact set in B and ∆ is any small number. It means that sequence Φ ε,u [t/ε] is tight in B. Taking into account conditions A1-A6, we obtain that discrete-time semi-Markov [t/ε] ϕ , and let K ∆ be a compact set from compact containment condition ∆ > 0. It is sufficient to show that J ε,u t weakly converges to zero. This is equivalent to the convergence of J ε,u t in probability as ε → 0. From the very definition of J ε,u t and A3, we obtain where 1 K ∆ is the indicator of the set K ∆ , and S ∆ is the finite δ-set for K ∆ . Then, for δ < ∆, we have , and D i := {ω : D k contains the maximum for the first time on the variable D i }. It is worth noticing that the operator P k is bounded when k → ∞. So is the case for Taking both ε and δ go to 0 we obtain the proof of the this lemma.
Proof. It is worth noticing that the martingale M ε,u [t/ε] can be represented in the form of the martingale differences Then, using the equality for any η > 0. Now, from the above, we get which proves the lemma. Now the proof of Theorem 1 is achieved as follows. From Lemmas 2-4 and the representation (17) it follows that the family (Φ ε,u [t/ε] ϕ) is relatively compact for any ∈ B * 0 , and any ϕ ∈ B 0 . Moreover, let IL ε (x), x ∈ E, be a family of perturbed operators defined on B as follows, Then, the process is an F ε,u t -martingale. The following singular perturbation problem, for the non-negligible part of compensating operator, IL ε,u , denoted by IL ε,u on the test functions ϕ ε (z, x) = ϕ(z) + εϕ 1 (z, x), has the solution (see [3] Proposition 5.1): The limit operator is then given by form which we get the contracted limit operator We note that martingale M ε,u t has the following asymptotic representation, where ||O ϕ (ε)|| → 0, as ε → 0. The families l(M [t/ε] ) and l(∑ [(P × P u )D ε (·, u) − I]Φ ε,u ) are weakly compact for all l ∈ B * 0 in a dense subset B * 0 ⊂ B. It means that family l(Φ ε,u [t/ε] ) is also weakly compact. In this way, the sum ε ∑ ILΦ ε,u ϕ converges, as ε → 0, to the integral t 0 ILΦ(s)ϕds. The quadratic variation of the martingale l(M ε,u t ϕ) tends to zero when ε → 0, thus, M ε,u t ϕ → 0 when ε → 0, for any f ∈ B 0 and for any l ∈ B * 0 . Passing to the limit in (23) (6). The quadratic variation of the martingale M ε,u t , in the average approximation, is where ϕ ε (x) = ϕ(x) + εϕ 1 (x). Hence Therefore, Now, from (24) and (26) and from boundedness of all operators in (26) with respect to E π×π 1 , it follows that (M ε,u [t/ε] goes to 0 when ε → 0, and the quadratic variation of limit process M 0,u t , for the martingale M ε,u t , is equals to 0. In this case, the limit martingale M 0 t equals to 0. Therefore, the limit equation for M ε,u t has the form (6). As long as the solution of the martingale problem for operator IL is unique, then it follows that the solution of the Equation (6) is unique as well [49,50]. It is worth noticing that operatorÎ L is a first order operator ( D 1 , see (22)). Finally, the operator IL generates a semigroup, then Φ(t)ϕ = exp[ ILt]ϕ and the latter representation is unique.

Proof of Theorem 2
We can prove the relative compactness of the family Φ ε,u [t/ε 2 ] exactly on the same way, and following the same steps as above. However, in the case of diffusion approximation the limit continuous martingale M 0 (t) for the martingale M ε t has quadratic variation that is not zero, that is, and so (M 0 ) = 0, for ∈ B * 0 . Moreover, operator IL defined in Theorem 2 is a second-order kind operator as it contains operator D 2 and ΠD 1 R 0 PD 1 Π, compare with the first-order operator IL in (22).

Proof of Theorem 3
We note that W ε,u t in (12) has the following presentation, As the balance condition Π(D 1 − D 1 ) = 0, holds, then we apply the diffusion approximation algorithm (see Section 3.2), i.e., to the right-hand side of (36) with the following operators, D 2 = 0 and (D 1 (z) − D 1 ) instead of D 1 (z). It is worth mentioning that the family W ε,u t is weakly compact and the result is proved (see Sections 6.1 and 6.2).
Let us introduce the following martingale,

Concluding Remarks and Future Work
In this paper, we introduced controlled semi-Markov random evolutions in discretetime in Banach space. The main results concerned time-rescaled limit theorems, namely, averaging, diffusion approximation, and diffusion approximation with equilibrium by martingale weak convergence method. We applied these results to various important families of stochastic systems, i.e., the controlled additive functionals, controlled geometric Markov renewal processes, and controlled dynamical systems. We provided dynamical principles for discrete-time dynamical systems such as controlled additive functionals and controlled geometric Markov renewal processes. We also produced dynamic programming equations (Hamilton-Jacobi-Bellman equations) for the limiting processes in diffusion approximation such as CAF, CGMRP, and CDS. As an example, we considered the solution of portfolio optimization problem by Merton for the limiting CGMRP in DA. We also point out the importance of convergence rates and obtained them in the limit theorems for CDTSMRE and CAF, CGMRP, and CDS.
The future work will be associated with the study of optimal control for the initial, not limiting models, such as CAF in Section 4.1, CGMRP in Section 4.2, and CDS in Section 4.3.
Other optimal control problems would be also interesting to consider for diffusion models with equilibrium, e.g., CAF in Section 4.1 and CGMRP in Section 4.2. In our future work, the latter models will be considered for solutions of Merton portfolio's problems as well. We will also consider in our future research the case of dependent SMC z k and the MC u k .