Stabilization of Stochastic Dynamic Systems with Markov Parameters and Concentration Point

Abstract

This paper addresses the problem of optimal stabilization for stochastic dynamical systems characterized by Markov switches and concentration points of jumps, which is a scenario not adequately covered by classical stability conditions. Unlike traditional approaches requiring a strictly positive minimal interval between jumps, we allow jump moments to accumulate at a finite point. Utilizing Lyapunov function methods, we derive sufficient conditions for exponential stability in the mean square and asymptotic stability in probability. We provide explicit constructions of Lyapunov functions adapted to scenarios with jump concentration points and develop conditions under which these functions ensure system stability. For linear stochastic differential equations, the stabilization problem is further simplified to solving a system of Riccati-type matrix equations. This work provides essential theoretical foundations and practical methodologies for stabilizing complex stochastic systems that feature concentration points, expanding the applicability of optimal control theory.

1. Introduction

One of the prerequisites for the physical realization of a process is its stability. Hence, ensuring stability is an essential task known as the stabilization problem.

The stabilization problem for stochastic dynamical systems with random structures was first solved by I.Ya. Kats in [1]. For stochastic dynamical systems with random structure and Markov switches that lead to jumps of the phase vector, the problem of optimal stabilization was solved by the authors in [2]. In that work, it was assumed that the moments of Markov switches are known. This assumption allowed a relatively straightforward transfer of basic properties from stochastic differential equations (SDEs) with continuous trajectories to systems with jumps. This global problem includes sub-problems related to the Markov property for solution $x\left(t\right),t\ge 0$, the martingale properties for ${\parallel x\left(t\right)\parallel }^2,t\ge 0$, and other local characteristics [3,4,5]. Similar problems for stochastic differential equations with delays have been studied in [6,7]. Stochastic games [7,8] have become widely used, in which it is assumed that two players have different objectives, and their strategies are described by stochastic differential equations. More general approaches to analyzing random fields and stochastic partial differential equations can be found in [9,10,11,12,13].

The inclusion of an integral term with respect to a Poisson measure also allowed cases with random moments of finite jumps in the phase vector to be addressed. For such systems, an explicit control form that stabilizes linear systems to asymptotic stochastic stability was obtained in [5], along with justification of exact and approximate methods for control calculation. A system of Riccati-type matrix equations has been derived to find a general solution to the stabilization problem.

In the works mentioned above, as well as in most studies involving trajectory jumps, i.e., distance between jumps satisfy the following condition: $|t_k-t_{k-1}|>\delta >0$. However, in catastrophe theory or resonant systems, cases often arise where jumps concentrate at a point, leading to the relation:

$$\underset{k\to \infty }{lim}{t}_k=t_{\infty }<\infty .$$

In this scenario, as previously indicated in [14], the cumulative effect of jumps can result in the loss of system stability. Consider the simple example which illustrates problems with the existence of a concentration point:

$$dx\left(t\right)=-x\left(t\right)dt$$

with jumps defined by

$$x\left(t_k\right)=x(t_k-)(1+k^2)$$

at points

$$t_k={\displaystyle \frac{\alpha }{k}},\alpha >0.$$

One can easily conclude that

$$\underset{t\to \alpha -}{lim}\left|x\left(t\right)\right|=\infty$$

provided that $x\left(0\right)\ne 0$. This straightforward example highlights the critical role of jump magnitudes in systems with concentration points.

In Section 2, we introduce the mathematical model for dynamical systems with jumps, described by a system of stochastic differential equations with Markov parameters and switches, providing sufficient conditions for the existence and uniqueness of solutions. Section 3 establishes sufficient conditions for exponential stability of the solution $x\left(t\right),t\ge 0$ (Theorem 1), which simultaneously define the class of admissible controls. Sufficient conditions for the existence of solutions to the optimal stabilization problem are established in Section 4 (Theorem 2). The synthesis of optimal control in explicit form for a linear system with a quadratic quality functional is presented in Section 5.

2. Task Definition

On the probabilistic basis $(\mathsf{\Omega },\mathfrak{F},F,\mathbf{P})$ [3,4,15], consider a controlled stochastic dynamical system of random structure given by a stochastic differential equation (SDE)

$$dx\left(t\right)=a(t,\xi \left(t\right),x\left(t\right),u\left(t\right))dt+b(t,\xi \left(t\right),x\left(t\right),u\left(t\right))dw\left(t\right),t\in {\mathbb{R}}_{+}\setminus K,$$

(1)

with Markov switches

$$\mathsf{\Delta }x\left(t\right)=g(t_k-,\xi (t_k-),{\eta }_k,x(t_k-)),t_k\in K=\{t_n\Uparrow \},$$

(2)

and initial conditions

$$x\left(0\right)=x_0\in {\mathbb{R}}^m,\xi \left(0\right)=y\in \mathbf{Y},{\eta }_0=h\in \mathbf{H}.$$

(3)

Here, $\xi \left(t\right),t\ge 0$ is a Markov chain with a finite number of states $\mathbf{Y}=\{1,2,\dots ,\overline{N}\}$ and generator $Q=\left\{q_{ij}\right\},i,j=1,\dots ,\overline{N}$; $\{{\eta }_k,k\ge 0\}$ is a Markov chain with values in the space $\mathbf{H}$ and transition probability matrix ${\mathbb{P}}_H$; $x:[0,+\infty )\times \mathsf{\Omega }\to {\mathbb{R}}^m$; $u\left(t\right)\in {\mathbb{R}}^m$ is the control; $w\left(t\right),t\ge 0,$ is an m-dimensional standard Wiener process; the processes $w,\xi$, and $\eta$ are independent [3,4,15].

Define

$${\mathfrak{F}}_{t_k}=\sigma (\xi \left(s\right),w\left(s\right),{\eta }_e,s\le t_k,t_e\le t_k)$$

as the minimal $\sigma$-algebra with respect to which $\xi \left(t\right)$ for $t\in [0,t_k]$ and ${\eta }_n,n\le k$ are measurable.

Measurable functions $a:{\mathbb{R}}_{+}\times \mathbf{Y}\times {\mathbb{R}}^m\to {\mathbb{R}}^m$, $b:{\mathbb{R}}_{+}\times \mathbf{Y}\times {\mathbb{R}}^m\to {\mathbb{R}}^m\times {\mathbb{R}}^m$ and $g:{\mathbb{R}}_{+}\times \mathbf{Y}\times \mathbf{H}\times {\mathbb{R}}^m\to {\mathbb{R}}^m$ satisfy the boundedness and global Lipschitz conditions:

Coefficients of stochastic differential equation are measurable maps: $a:{\mathbb{R}}_{+}\times \mathbf{Y}\times {\mathbb{R}}^m\to {\mathbb{R}}^m$, $b:{\mathbb{R}}_{+}\times \mathbf{Y}\times {\mathbb{R}}^m\to {\mathbb{R}}^m\times {\mathbb{R}}^m$, $g:{\mathbb{R}}_{+}\times \mathbf{Y}\times \mathbf{H}\times {\mathbb{R}}^m\to {\mathbb{R}}^m$ satisfy the boundedness condition and the global Lipschitz condition

$${\left|a(t,y,x,u)\right|}^2+{\left|b(t,y,x,u)\right|}^2+{\left|g(t,y,h,x)\right|}^2\le C(1+{\left|x\right|}^2);$$

(4)

$${|a(t,y,x_1,u)-a(t,y,x_2,u)|}^2+{|b(t,y,x_1,u)-b(t,y,x_2,u)|}^2\le L{|x_1-x_2|}^2,x_1,x_2\in {\mathbb{R}}^m;$$

(5)

$${|g(t_k,y,h,x_1)-g(t_k,y,h,x_2)|}^2\le L_k{|x_1-x_2|}^2,x_1,x_2\in {\mathbb{R}}^m,\sum _{k=1}^{\infty }{L}_k<\infty .$$

(6)

Consider the scenario with a concentration point of jumps, i.e.,

$$\underset{n\to \infty }{lim}{t}_n=t^{\ast }<\infty .$$

Assume the following conditions are satisfied:

$$\sum _{k=1}^{\infty }{\gamma }_k<\infty ,{\gamma }_k=\underset{x\in {\mathbb{R}}^m,y\in \mathbf{Y},h\in \mathbf{H}}{sup}\left|g(t_k,y,h,x)\right|$$

(7)

and

$$\underset{\epsilon \downarrow 0}{lim}\left(ln\epsilon +N_{\epsilon }\sum _{k=1}^{N_{\epsilon }}{L}_k\right)=-\infty ,N_{\epsilon }:=inf\left\{k\ge 1:\sum _{m=k}^{\infty }{\gamma }_m<\epsilon \right\}.$$

(8)

Conditions (4)–(8) in fact are the sufficient conditions of existence and unique for a strong solution to the Cauchy problem (1)–(3) [16].

Define the transition probability of the Markov chain $(\xi \left(t_k\right),{\eta }_k,x\left(t_k\right))$ that determines the solution of the problem (1)–(3) at step k as

$${\mathbf{P}}_k((y,h,x),\mathsf{\Gamma }\times G\times \mathbf{C}):=$$

$$:=\mathbf{P}((\xi \left(t_{k+1}\right),{\eta }_{k+1},x\left(t_{k+1}\right))\in \mathsf{\Gamma }\times G\times \mathbf{C}|(\xi \left(t_k\right),{\eta }_k,x\left(t_k\right))=(y,h,x)).$$

Definition 1

([1]). The discrete Lyapunov operator $\left(l{v}_k\right)(y,h,x)$ on a sequence of measurable scalar functions $v_k(y,h,x):\mathbf{Y}\times \mathbf{H}\times {\mathbb{R}}^m\to {\mathbb{R}}^1,k\in \mathbb{N}\cup \left\{0\right\},$ for the SDE (1) with Markov switches (2) is defined as

$$\left(l{v}_k\right)(y,h,x):=$$

$$:=\underset{\mathbf{Y}\times \mathbf{H}\times {\mathbb{R}}^m}{\int }{\mathbf{P}}_k(y,h,x)(du\times dz\times dl)v_{k+1}(u,z,l)-v_k(y,h,x).$$

(9)

Here, $v_k(y,h,x),k\in \mathbb{N}$ is a Lyapunov function defined by the following definition.

Definition 2

([1,2]). A Lyapunov function for the system (1)–(3) is a sequence of nonnegative non-decreasing functions $\left\{v_k(y,h,x),k\ge 0\right\},$ satisfying the following conditions:

1. 

for all $k\ge 0,y\in \mathbf{Y},h\in \mathbf{H},x\in {\mathbb{R}}^m$ the discrete Lyapunov operator $\left(l{v}_k\right)(y,h,x)$ (9) is defined;

2. 

$v_k(y,h,x)\le v_{k+1}(y,h,x)$ for all $k\ge 0,y\in \mathbf{Y},h\in \mathbf{H},x\in {\mathbb{R}}^m$;

3. 

if $r\to \infty$

$$\overline{v}\left(r\right)\equiv \underset{k\in \mathbb{N},\mathit{y}\in \mathbf{Y},h\in \mathbf{H},\left|x\right|\ge r}{inf}{v}_k(y,h,x)\to +\infty ;$$

4. 

if $r\to 0$

$$\underline{v}\left(r\right)\equiv \underset{k\in \mathbb{N},y\in \mathbf{Y},h\in \mathbf{H},\left|x\right|\le r}{sup}{v}_k(y,h,x)\to 0;$$

and $\overline{v}\left(r\right)$ and $\underline{v}\left(r\right)$ are continuous and monotonous.

Definition 3

([17,18]). The stochastic system (1)–(3) is called:

—stable in probability if for $\forall {\epsilon }_1>0,{\epsilon }_2>0$ exist $\delta =\delta ({\epsilon }_1,{\epsilon }_2)>0$, such that $\left|x\right|<\delta$ implies

$$\mathbf{P}\left\{\underset{t\ge 0}{sup}\left|x\left(t\right)\right|>{\epsilon }_1\right\}<{\epsilon }_2$$

(10)

for all $y\in \mathbf{Y},h\in \mathbf{H}$;

—asymptotically stochastically stable if it is stable in probability and for any $\epsilon >0$ there exists ${\delta }_2>0$, such that

$$\underset{T\to \infty }{lim}\mathbf{P}\left\{\underset{t\ge T}{sup}\left|x\left(t\right)\right|>\epsilon \right\}=0$$

(11)

for all $\left|x\right|<{\delta }_2$, $y\in \mathbf{Y},h\in \mathbf{H}$ and $T\ge 0$.

Definition 4

([17,18,19]). The system (1)–(3) is called exponentially stable in the mean square if $\forall x_0\in {\mathbb{R}}^m$, $\xi \left(0\right)$ and ${\eta }_0$, there exist constants $\alpha >0,\beta >0$, such that

$$\mathbf{E}{\left|x\left(t\right)\right|}^2\le \alpha {\left|x_0\right|}^2{e}^{-\beta t},\forall t\ge 0.$$

(12)

In general, these two types of convergences are not related to each other [19], but in specific cases one type of convergence can be used to infer the other. A remark to Theorem 1 allows us to state that, provided that the Lyapunov function exists, the exponential stability in the mean square implies it is asymptotically stochastically stable. Thus, Theorem 1 allows us to draw conclusions, not only about the moment convergence of the solution to 0 but also about the probabilistic properties of the solution for large T.

3. Stability

One common approach to establishing sufficient conditions for exponential stability involves imposing a constraint on the switching moments of the type

$$|t_{k+1}-t_k|>\mathsf{\Delta },\mathsf{\Delta }>0=\mathrm{const},$$

(13)

which excludes the possibility of concentration points of jumps [1,20,21]. Clearly, in the case considered here, condition (13) is not fulfilled. Therefore, it is essential to identify conditions under which the solution to the system (1)–(3) is exponentially stable in the mean square.

Theorem 1.

Suppose that, for the system (1)–(3), there exist Lyapunov functions $v_k(x,y,h),k\ge 0,$ and strictly increasing on $[0,\infty )$, positive and continuous functions $c,f$ and z, $c\left(0\right)=f\left(0\right)=z\left(0\right)=0,$ such that, under the condition,

$$f\left({\left|x\left(t\right)\right|}^2\right)\le v_k(x\left(t\right),y,h)\le z\left({\left|x\left(t\right)\right|}^2\right)$$

(14)

holds, along with the inequality

$$l{v}_k(x\left(t\right),y,h)\le -c\left({\left|x\left(t\right)\right|}^2\right),$$

(15)

for $t\in [t_k,t_{k+1}),k\ge 0$, and

$$\sum _{j=k{N}_T}^{k{N}_T+n-1}\mathbf{E}\left\{v_j(x,\xi \left(t_j\right),{\eta }_j)\right\}\le {\chi }_k\left(v_k(x(t_k-),\xi (t_k-),{\eta }_k)\right),$$

(16)

for some integer $N_T\ge 0$, $n=1,2,\dots ,N_T$, where ${\chi }_k:{\mathbb{R}}_{+}\to {\mathbb{R}}_{+}$ is a non-decreasing function satisfying ${\chi }_k\left(s\right)\le s$. Assume also that

$$\underset{x\in (0,\infty )}{inf}{\displaystyle \frac{c\left(x\right)}{z\left(x\right)}}>0.$$

Then, the system (1)–(3) is exponentially stable in the mean square.

Proof of Theorem 1.

On the interval $[t_k,t_{k+1}),k\ge 0,$ consider the weak infinitesimal operator acting on the Lyapunov function $v_k(x,y,h)$. From (15), we have

$$l{v}_k(x,y,h)\le -c\left(\right|x\left(t\right)\left|\right)=-{\displaystyle \frac{c\left(\right|x\left(t\right)\left|\right)}{v_k(x,y,h)}}·v_k(x,y,h)\le -\alpha v_k(x,y,h),$$

where the scalar $\alpha >0$ is defined as

$$\alpha =\underset{x\in (0,\infty )}{inf}{\displaystyle \frac{c\left(x\right)}{z\left(x\right)}}.$$

By Dynkin’s formula [4], for any $t\in [t_{\overline{k}},t_{\overline{k}+1}),$ and some $\overline{k}\ge 0$,

$$\mathbf{E}\left\{\sum _{j=0}^{\overline{k}-1}\underset{t_j}{\overset{t_{j+1}}{\int }}l{v}_j(x\left(s\right),y,h)ds+\underset{t_{\overline{k}}}{\overset{t}{\int }}l{v}_{\overline{k}}(x\left(s\right),y,h)ds\right\}=$$

$$=\sum _{j=0}^{\overline{k}-1}\mathbf{E}\left\{v_{j+1}(x(t_{j+1}-),\xi (t_{j+1}-),\eta (t_{j+1}-))\right\}-v_j(x\left(t_j\right),\xi \left(t_j\right),\eta \left(t_j\right))+$$

$$+\mathbf{E}{v}_{\overline{k}}(x\left(t_{\overline{k}}\right),\xi \left(t_{\overline{k}}\right),\eta \left(t_{\overline{k}}\right))-v_{\overline{k}}(x,y,h)=$$

$$\mathbf{E}\left\{v_{\overline{k}}(x\left(t_{\overline{k}}\right),\xi \left(t_{\overline{k}}\right),\eta \left(t_{\overline{k}}\right))\right\}-v_0(x_0,y,h)+$$

$$\sum _{k=0}^{\left[{\displaystyle \frac{\overline{k}}{N_T}}\right]-1}\sum _{j=k{N}_T}^{(k+1)N_T-1}\mathbf{E}\left\{v_j(x(t_j-),\xi (t_j-),\eta (t_j-))\right\}-v_j(x\left(t_j\right),\xi \left(t_j\right),\eta \left(t_j\right))+$$

$$+\sum _{j=\left[{\displaystyle \frac{\overline{k}}{N_T}}\right]N_T}^{\overline{k}}\mathbf{E}\left\{v_j(x(t_j-),\xi (t_j-),\eta (t_j-))\right\}-v_j(x\left(t_j\right),\xi \left(t_j\right),\eta \left(t_j\right)).$$

Using (16), it follows that

$$\mathbf{E}\left\{v_{\overline{k}}(x\left(t_{\overline{k}}\right),\xi \left(t_{\overline{k}}\right),\eta \left(t_{\overline{k}}\right))\right\}-v_0(x_0,y,h)\le$$

$$\le \mathbf{E}\left\{\sum _{j=0}^{\overline{k}-1}\underset{t_j}{\overset{t_{j+1}}{\int }}l{v}_j(x\left(s\right),\xi \left(t_j\right),\eta \left(t_j\right))ds+\underset{t_{\overline{k}}}{\overset{t}{\int }}l{v}_{\overline{k}}(x\left(s\right),\xi \left(t_{\overline{k}}\right),\eta \left(t_{\overline{k}}\right))ds\right\}\le$$

$$\le -\alpha \mathbf{E}\left\{\sum _{j=0}^{\overline{k}-1}\underset{t_j}{\overset{t_{j+1}}{\int }}{v}_j(x\left(s\right),\xi \left(t_j\right),\eta \left(t_j\right))ds+\underset{t_{\overline{k}}}{\overset{t}{\int }}{v}_{\overline{k}}(x\left(s\right),\xi \left(t_{\overline{k}}\right),\eta \left(t_{\overline{k}}\right))ds\right\}=$$

$$=-\alpha \mathbf{E}\left\{\underset{0}{\overset{t}{\int }}{v}_{\overline{k}}(x\left(s\right),\xi \left(t_{\overline{k}}\right),\eta \left(t_{\overline{k}}\right))ds\right\}.$$

The last inequality implies that

$${\displaystyle \frac{d}{dt}}\mathbf{E}\left\{v_k(x,\xi \left(t_k\right),{\eta }_k)\right\}\le -\alpha {\displaystyle \frac{d}{dt}}\underset{0}{\overset{t}{\int }}\mathbf{E}\left\{v_k(x\left(s\right),\xi \left(t_k\right),{\eta }_k)\right\}ds=-\alpha \mathbf{E}\left\{v_k(x,y,h)\right\},$$

which, by the Gronwall–Bellman lemma, implies

$$\mathbf{E}\left\{v_k(x\left(t\right),y,h)\right\}\le v_k(x_0,y,h)e^{-\alpha t},t\in [0,t_k].$$

This estimate and (14) imply exponential stability in the mean square of the system (1)–(3). Indeed, based on the estimate of $\mathbf{E}\left\{v_k(x,y,h)\right\}$, the event ${lim}_{t\to \infty }\left|x\left(t\right)\right|=0$ is equivalent to ${lim}_{t_k\ge t,t\to \infty }\mathbf{E}\left\{v_k(x\left(t\right),y,h)\right\}=0$, proving the theorem. □

Remark 1.

Since the inequality (15) holds, the solution of (1)–(3) is asymptotically stable in probability [14].

4. Stabilization

The problem of optimal stabilization for the system (1)–(3) consists of determining a control $u(t,x(t\left)\right)$, such that the trivial solution $x\left(t\right)\equiv 0$ of the system becomes asymptotically stable in probability.

It is assumed that the control u is based on the full feedback principle and is continuous in t for $t\ge 0,x\in {\mathbb{R}}^m$, for all fixed $\xi \left(t\right)=y\in \mathbf{Y}$ and ${\eta }_k=h\in \mathbf{H}$. Specifically, in the case of continuous dynamics (1) and (2), the control is defined by the relation

$$u\left(t\right)=u(t,x(t-\left)\right),$$

and the left-hand side boundary is considered precisely due to the presence of jumps (2).

The set of admissible controls consists of those controls for which the system is exponentially stable [1,22], namely

$$U=\left\{u\left(t\right)=u(t,x(t-))\left|E\right|x\left(t\right)|\le |x_0|e^{-\alpha \left(u\right)t},\alpha \left(u\right)>0\right\}.$$

In the previous section, we established sufficient conditions under which exponential stability in the mean square is equivalent to asymptotic stability in probability. Therefore, if these conditions are met, every admissible control will be optimal with respect to the stabilization problem, resulting in an infinite set of such controls. The optimal control must then be selected based on the best quality of the transient process, expressed through minimizing the quality functional

$${\mathit{I}}_u(y,h,x_0):=$$

$$:=\sum _{k=0}^{\infty }{\int }_{t_k}^{t_{k+1}}\mathit{E}\left\{W(t,x\left(t\right),u\left(t\right))/\xi \left(0\right)=y,{\eta }_0=h,x\left(0\right)=x_0\right\}dt,$$

(17)

where $W(t,x,u)\ge 0$ is a measurable function defined for $t\ge 0,x\in {\mathbb{R}}^m$, $u\in {\mathbb{R}}^r$.

The functional (17) can by calculated as follows:

• Compute the trajectory $x\left(t\right)$ of the SDE (1) for a given control $u(t,y,h,x)$, e.g., using the Euler–Maruyama method [23].
• Put $x\left(t\right)$, $\xi \left(t\right)$, and $u\left(t\right)=u(t,x(t\left)\right)$ into the functional (17).
• Estimate the value of (17) through statistical simulation (Monte Carlo method).
• The choice of the functional $W(t,x,u)$, determining the functional estimate $I_u$ and the quality of the process $x\left(t\right)$ as a strong solution of the SDE (1), must satisfy the following criteria:

  (a)

  Minimization conditions of (17) must ensure that the strong solution $x\left(t\right)$ of the SDE (1) converges to zero rapidly on average, with high probability;

  (b)

  The integral’s value should reasonably estimate the computational cost for generating the control $u\left(t\right)$;

  (c)

  The value of the quality functional should adequately reflect the computational effort required to determine the control $u\left(t\right)$;

  (d)

  The chosen functional $W(t,x,u)$ must permit explicit or constructive solutions to the stabilization problem.

Definition 5.

A control $u^0\left(t\right)$ satisfying

$${\mathit{I}}_{u^0}(y,h,x_0)=min{\mathit{I}}_u(y,h,x_0),$$

where the minimum is taken over all controls continuous in variables t and x for each $\xi \left(0\right)=y\in \mathbf{Y}$, and ${\eta }_0=h\in \mathbf{H}$ is called optimal with respect to the optimal stabilization of the strong solution $x\in {\mathbb{R}}^m$ of the system (1)–(3).

Theorem 2.

Let, for the system (1)–(3), $v^0(t_k,y,h,x)$ exists, and the r-vector function $u^0(t,y,h,x)\in {\mathbb{R}}^r$ exists, such that:

1. 

The sequence of functions $v_k^0(y,h,x)\equiv v^0(t_k,y,h,x)$ is a Lyapunov functional, satisfying the conditions of Theorem 1;

2. 

The sequence of r-dimensional control functions

$$u_k^0(y,h,x)\equiv u^0(t_k,y,h,x)\in {\mathbb{R}}^{\mathit{r}}$$

is measurable in all arguments, where $0\le t_k<t_{k+1},k\ge 0$;

3. 

The sequence of functions appearing in the criterion (17) by $x\in {\mathbb{R}}^m$ is positive definite, i.e., for $\forall t\in [t_k,t_{k+1}),k\ge 0$,

$$W(t,x,u_k^0(y,h,x))>0;$$

4. 

The sequence of infinitesimal operators $\left(l{v}_k^0\right)u_k^0$, calculated at $u_k^0\equiv u^0(y,h,x)$, satisfies the condition for $\forall t\in [t_k,t_{k+1})$

$$\left(l{v}_k^0\right)u_k^0=-W(t,x,u_k^0);$$

5. 

The expression $\left(l{v}_k^0\right)+W(t,x,u)$ reaches its minimum at $u=u^0,k\ge 0$, i.e.,

$$\left(l{v}_k^0\right)u_k^0+W(t,x,u_k^0)=$$

$$=\underset{u\in {\mathbb{R}}^r}{inf}\{\left(l{v}_k^0\right){|}_u+W(t,x,u)\}=0.$$

(18)

6. 

The series

$$\sum _{k=0}^{\infty }{\int }_{t_k}^{t_{k+1}}\mathbf{E}\left\{W(t,x\left(t\right),u\left(t\right))/x\left(t_k\right)\right\}dt<\infty$$

converges.

Then, the control $u_k^0\equiv u^0(t_k,y,h,x),k\ge 0$ stabilizes the solution of the problem (1)–(3). Moreover, the following equality holds:

$$v^0(y,h,x_0)\equiv$$

$$\equiv \sum _{k=0}^{\infty }{\int }_{t_k}^{t_{k+1}}\mathbf{E}\left\{W(t,x\left(t\right),u\left(t\right))|x\left(t_k\right)\right\}dt=$$

$$=\underset{u\in {\mathbb{R}}^r}{min}\sum _{k=0}^{\infty }{\int }_{t_k}^{t_{k+1}}\mathbf{E}\left\{W(t,x\left(t\right),u\left(t\right))|x\left(t_k\right)\right\}dt\equiv {\mathit{I}}_{u^0}(y,h,x_0).$$

Proof of Theorem 2.

The proof follows exactly the argument provided for Theorem 2 in [5]. □

Since $\xi \left(t_k\right)$ is a Markov process with a finite number of states, then transition probability can be defined as follows:

$$\mathbf{P}\{\omega :\xi (t+\mathsf{\Delta }t)=y_j\mid \xi \left(t\right)=y_i,y_i\ne y_j\}=-q_{ij}\left(t\right)\mathsf{\Delta }t+o\left(\mathsf{\Delta }t\right),i,j=\overline{1,N}.$$

(19)

According to this assumption, we obtain an equation that must be satisfied by the optimal Lyapunov functions $v_k^0(y,h,x)$ and the optimal control $u_k^0(t,x)\forall t\in [t_k,t_{k+1})$.

Following [14,24], the weak infinitesimal operator (WIO) (9) has the form

$$\left(l{v}_k\right)(y,h,x)={\displaystyle \frac{\partial v_k(y,h,x)}{\partial t}}+(\nabla v_k(y,h,x),a(t,y,x,u))+$$

$$+{\displaystyle \frac{1}{2}}Sp(b^T(t,y,x,u)·{\nabla }^2{v}_k(y,h,x)·b(t,y,x,u))+$$

$$+\sum _{j\ne i}^N[\underset{{\mathbb{R}}^m}{\int }{v}_j(t,x)p_{ij}(t,z/x)dz-v_i(t,x)]q_{ij},$$

(20)

where $(·,·)$ is a scalar product, $\nabla v_k={\left({\displaystyle \frac{\partial v_k}{\partial x_1}},\dots ,{\displaystyle \frac{\partial v_k}{\partial x_m}}\right)}^T$, ${\nabla }^2{v}_k={\left[{\displaystyle \frac{{\partial }^2{v}_k}{\partial x_i\partial x_j}}\right]}_{i,j=1}^m,k\ge 0$, “T” denotes transposition, $Sp$ is a trace of the matrix, and $p_{ij}(t,z/x)$ is a conditional probability density

$$P\left\{x\left(\tau \right)\in [z,z+dz]/x(\tau -0)=x\right\}=p_{ij}(\tau ,z/x)dz+o\left(dz\right)$$

assuming $\xi (\tau -0)=y_i,\xi \left(\tau \right)=y_j$.

Using (20), we derive the first equation for $v^0$ by substituting the averaged infinitesimal operator $\left(l{v}_k^0\right){|}_{u^{\ast }}$ [1] into the left-hand side of (18). The resulting equation at the points $(t_k,y_j,{\eta }_k,x)$ is:

$${\displaystyle \frac{\partial v_k^0}{\partial t}}+\left({\left({\displaystyle \frac{\partial v_k^0}{\partial x}}\right)}^T·a(t,y,x,u)\right)+{\displaystyle \frac{1}{2}}Sp\left(\left(b^T(t,y_i,x)·{\displaystyle \frac{{\partial }^2{v}_k^0}{\partial x^2}}·b(t,y_i,x)\right)\right)+$$

$$+\sum _{j\ne i}^l\left[{\int }_{-\infty }^{+\infty }{v}_j^0(y_j,h,x_j)p_{ij}\left(t,z/x\right)dz-v_i^0(y_i,h,x)\right]q_{ij}\left(t\right)dt+$$

$$+W(t,x,u)=0.$$

(21)

For defining optimal control $u_k^0(t,y,h,x)$ we differentiate (21) with respect to the variable u:

$${\left.\left[{\left({\displaystyle \frac{\partial v^0}{\partial x}}\right)}^T·\left({\displaystyle \frac{\partial a}{\partial u}}\right)+{\left({\displaystyle \frac{\partial W}{\partial u}}\right)}^T\right]\right|}_{u=u_k^0}=0,$$

(22)

where ${\displaystyle \frac{\partial a}{\partial u}}$ – $m\times r$-matrix of Jacobi, stacked with elements $\left\{{\displaystyle \frac{\partial a_n}{\partial u_s}},n=\overline{1,m},s=\overline{1,r}\right\};$$\left({\displaystyle \frac{\partial W}{\partial u}}\right)\equiv \left({\displaystyle \frac{\partial W}{\partial u_1}},\dots ,{\displaystyle \frac{\partial W}{\partial u_r}}\right),k\ge 0$.

Thus, according to Theorem 2, the problem of optimal stabilization reduces to solving a complex system of the nonlinear Equation (18), involving partial derivatives, to find the unknown Lyapunov functions $v_{ik}^0\equiv v_k^0(y,h,x)$, where $i=\overline{1,l}$ and $k\ge 0$.

It is important to note that this nonlinear system is derived by eliminating the control $u_k^0=u^0(t,y,h,x)$ from Equations (21) and (22).

Given the inherent difficulty of solving such a nonlinear system directly, we will subsequently focus on linear stochastic systems, for which more tractable solution schemes can be constructed.

5. Stabilization of Linear Systems

Consider a linear case:

$$dx\left(t\right)=\left[A\right(t-,\xi (t-)\left)x\right(t-)+B(t-,\xi (t-)\left)u\right(t-\left)\right]dt+$$

$$+\sigma (t-,\xi (t-))x(t-)dw\left(t\right),t\in {\mathbb{R}}_{+}\setminus K,$$

(23)

with Markov switching given by

$$\mathsf{\Delta }x\left(t\right){|}_{t=t_k}=g(t_k-,\xi (t_k-),{\eta }_k,x(t_k-)),t_k\in K=\{t_n\Uparrow \}$$

(24)

where $\underset{n\to +\infty }{lim}{t}_n=+\infty$, and initial conditions are

$$x\left(0\right)=x_0\in {\mathbb{R}}^m,\xi \left(0\right)=y\in \mathbf{Y},{\eta }_0=h\in \mathbf{H}.$$

(25)

Here, $A,B,\sigma$ are piecewise continuous integrable matrix functions of appropriate dimensions.

We assume that the jump conditions for the state vector $x\in {\mathbb{R}}^m$ at a switching instant $t=t^{\ast }$, corresponding to the change in the structure of the system due to the transition $\xi (t^{\ast }-)=y_i$ to $\xi \left(t^{\ast }\right)=y_j\ne y_i$, are linear and expressed as

$$x\left(t^{\ast }\right)=K_{ij}x(t^{\ast }-)+\sum _{s=1}^N{\xi }_s{Q}_{s}x(t^{\ast }-),$$

(26)

where ${\xi }_s:={\xi }_s\left(\omega \right)$ – are independent random variables satisfying $\mathbf{E}{\xi }_s=0,\mathbf{E}{\xi }_s^2=1$, $K_{ij}$ and $Q_s$ are given $(m\times m)$-matrices.

Note that Equation (26) can replace the general jump conditions under the following circumstances [21]:

If jumps are deterministic, then $Q_s=0$ and Expression (26) reduces to

$$x\left(t^{\ast }\right)=K_{ij}x(t^{\ast }-);$$

Continuous changes in the phase vector correspond to $Q_s=0$ and $K_{ij}=I$—the identity matrix of size $(m\times m)$.

The quality of the transient process is evaluated through the quadratic functional

$${\mathit{I}}_u(y,h,x_0):=$$

(27)

$$:=\sum _{k=0}^{\infty }{\int }_{t_k}^{t_{k+1}}\mathbf{E}\left\{x^T\left(t\right)M\left(t\right)x\left(t\right)+u^T\left(t\right)D\left(t\right)u\left(t\right)\right./\left.y,h,x_0\right\}dt,$$

where $M\left(t\right)\ge 0$ and $D\left(t\right)>0$ are symmetric matrices of dimensions $(m\times m)$ and $(r\times r)$, respectively.

The optimal Lyapunov functions are assumed in the quadratic form:

$$v_k^0(y,h,x)=x^{T}G(t,y,h)x,$$

(28)

where $G(t,y,h)$ is a positive-definite symmetric matrix of dimension $(m\times m)$.

Throughout this section, we assume that $\xi \left(t\right)$ is a Markov chain with a finite state space $\mathbf{Y}=\{y_1,y_2,\dots ,y_l\}$, and ${\eta }_k,k\ge 0$ is a Markov chain with states $h_k$ in a metric space $\mathbf{H}$ and transition probabilities ${\mathbf{P}}_k(h,G)$ at step k. We introduce the following notations:

$$A_i\left(t\right):=A(t,y_i),B_i\left(t\right):=B(t,y_i),{\sigma }_i\left(t\right):=\sigma (t,y_i),$$

$$G_{ik}\left(t\right):=G(t,y_i,h_k),v_{ik}:=v(y_i,h_k,x).$$

Substituting the functional (28) into Equations (21) and (22), we derive equations for determining the optimal Lyapunov function $v_k^0(y,h,x)$ and optimal control $u_k^0(t,x)$ for $\forall t\in [t_k,t_{k+1})$. Using WIO form (20), we find that:

$$x^T\left(t\right){\displaystyle \frac{d{G}_{ik}\left(t\right)}{dt}}x\left(t\right)+2[A_i\left(t\right)x\left(t\right)+B_i\left(t\right)u\left(t\right)]G_{ik}\left(t\right)x\left(t\right)+$$

$$+Sp\left(x^T\left(t\right){\sigma }_i^T\left(t\right)G_{ik}\left(t\right){\sigma }_i\left(t\right)x\left(t\right)\right)+$$

$$+x^T\left(t\right)\sum _{j\ne i}^N\left[K_{ij}^T{G}_{ik}\left(t\right)K_{ij}+\sum _{s=0}^l{Q}_s^T{G}_{ik}\left(t\right)Q_s-G_{ik}\left(t\right)\right]q_{ij}x\left(t\right)+$$

$$+x^T\left(t\right)M_{ik}\left(t\right)x\left(t\right)+u^T\left(t\right)D_{ik}\left(t\right)u\left(t\right)=0,$$

(29)

$$2x^T\left(t\right)G_{ik}\left(t\right)B_i\left(t\right)+2u^T\left(t\right)D_{ik}\left(t\right)=0.$$

(30)

Using (30), we can derive optimal control for $\xi \left(t\right)=y_i$ and ${\eta }_k=h_k,k\ge 0$:

$$u_{ik}^0(t,x)=-D_{ik}^{-1}\left(t\right)B_i^T\left(t\right)G_{ik}\left(t\right)x\left(t\right).$$

(31)

Given the matrix equality

$$2x^T\left(t\right)G_{ik}\left(t\right)A_i\left(t\right)x=x^T\left(t\right)(G_{ik}\left(t\right)A_i\left(t\right)+A_i^T\left(t\right)G_{ik}\left(t\right))x\left(t\right),$$

eliminating $u_{ik}^0$ from (29) and setting to zero a quadratic form, a system of matrix Riccati-type differential equations for determining the matrices $G_{ik}\left(t\right)$, where $i=1,2,\dots ,l$, $k\ge 0$, corresponding to the interval $[t_k,t_{k+1})$, are obtained:

$${\displaystyle \frac{d{G}_{ik}\left(t\right)}{dt}}+G_{ik}\left(t\right)A_i\left(t\right)-B_i\left(t\right)D_{ik}^{-1}\left(t\right)B_i^T\left(t\right)G_{ik}\left(t\right)+$$

$$+Sp\left({\sigma }_i^T\left(t\right)G_{ik}\left(t\right){\sigma }_i\left(t\right)\right)+$$

$$+\sum _{j\ne i}^N\left[K_{ij}^T{G}_{ik}\left(t\right)K_{ij}+\sum _{s=0}^l{Q}_s^T{G}_{ik}\left(t\right)Q_s-G_{ik}\left(t\right)\right]q_{ij}+M_{ik}\left(t\right)=0,$$

(32)

$$\underset{t\to \infty }{lim}{G}_{ik}\left(t\right)=0,i=\overline{1,N},k\ge 0.$$

(33)

Theorem 3.

Suppose the system of matrix Equations (32) and (33) has positive-definite solutions of the order $(m\times m)$:

$$G_{1k}\left(t\right)>0,G_{2k}\left(t\right)>0,\dots ,G_{lk}\left(t\right)>0.$$

Then, the control defined by (31) provides a solution to the optimal stabilization problem for the linear stochastic system (23)–(25) with jump conditions (26) and optimality criterion (27).

Remark 2.

Sufficient conditions of resolvability of Ricatti-type Equations (32) and (33) given in the work [25].

6. Model Example

For comparison results, consider example from [14]. For this example, define the linear autonomous stochastic differential equation

$$dx\left(t\right)=\left(a\right(\xi \left(t\right)\left)x\right(t)+b(\xi \left(t\right)\left)u\right(t\left)\right)dt+\sigma \left(\xi \right(t\left)\right)x\left(t\right)dw\left(t\right),t\ge 0,$$

with perturbations

$$x\left(t_k\right)=x\left(t_k-\right)+e^{-\alpha k{\eta }_k}\left(x\left(t_k-\right)\wedge 1\right),$$

where breakpoints $t_k$ are defined as

$$t_k=2-{\displaystyle \frac{1}{k}},k\ge 1$$

with concentration point $t_{\infty }={lim}_{k\to \infty }=2$. Also define the non-random initial condition

$$x\left(0\right)=10,\xi \left(0\right)=y_0\in \mathbf{Y},{\eta }_0=1.$$

In this autonomous case, the system (32) has the next form [5]:

$$G_{ik}{A}_i+A_i^T{G}_{ik}-B_i{D}_{ik}^{-1}{B}_i^T{G}_{ik}+{\sigma }_i^T{G}_{ik}{\sigma }_i+$$

$$+\sum _{j\ne i}^N\left[K_{ij}^T{G}_{ik}{K}_{ij}+\sum _{s=0}^l{Q}_s^T{G}_{ik}{Q}_s-G_{ik}\right]q_{ij}+M_{ik}=0,i=\overline{1,N},k\ge 0.$$

The three cases of the parameters are considered as in [14].

Case 1. Unstable system for $b\equiv 0$:

-

if $\xi =1$: $a=1,\sigma =0.3,b=1$;

-

if $\xi =2$: $a=-0.5,\sigma =2.1,b=1$;

Case 2. Stable system for $b\equiv 0$:

-

if $\xi =1$: $a=-1,\sigma =0.3,b=1$;

-

if $\xi =2$: $a=0.5,\sigma =2,b=1$;

Case 3. Unstable system with values of parameters from Case 2 and impulse action

$$x\left(t_k\right)=x\left(t_k-\right)+e^{\alpha k{\eta }_k}\left(x\left(t_k-\right)\wedge 1\right).$$

The results of synthesis of the optimal control (31) are visualized in Figure 1.

Figure 1. Examples of solution trajectories estimated by the Euler–Maruyama method (previously shown in [14]): (a) Case 1—unstable, (b) Case 2—stable, and (c) Case 3—unstable with an extreme growth at t = 2. Uncontrolled (red lines) and controlled (green lines) solutions with control given by (31). Blue marks indicate moments of impulse actions. Optimal control stabilizes the system’s trajectory.

As we can see, the optimal control stabilizes the unstable system in Case 1 and makes the decay of stable solutions faster in Cases 2 and 3.

7. Discussion

Optimal control theory relies on several fundamental methods, one of the most prominent being the Lyapunov function method. This method, along with its various modifications, is extensively employed to address practical problems in numerous mathematical models, including stochastic differential equations. In this study, particular emphasis is placed on applying Lyapunov functions to stochastic differential equations with Markov switches, specifically addressing scenarios involving concentration points. This approach could be extended by incorporating additional assumptions about the switching mechanism, such as semi-Markov processes, where state durations do not necessarily follow an exponential distribution.

The paper also considers a model example based on a similar example from [14]. As can be seen from the simulation results, unstable systems can be stabilized; however, this is not possible in all cases, as illustrated in Case 3 of the model example. Thus, it remains an important issue to study the conditions for unconditional boundedness of solutions of the system (1)–(3).

Future research in this field will explore broader characteristics of the switching process $\xi \left(t\right)$ and validate the theoretical results derived here through practical applications. Furthermore, the computational complexity of the algorithms proposed in Theorems 2 and 3 remains an area requiring further investigation, particularly in comparison to heuristic algorithms for optimal control estimation. Hence, subsequent research will include comparative analyses between the algorithms developed in this paper and heuristic methods. Further studies will primarily focus on linear systems, exploring necessary and sufficient conditions for stability and the existence of optimal controls.

8. Conclusions

In this paper, we have established sufficient conditions for ensuring stability in stochastic differential equations characterized by jump concentration points. Unlike most classical assumptions, which impose a strict minimal interval between jumps (i.e., $|t_k-t_{k-1}|>\mathsf{\Delta }$), our study deliberately omits this condition, thus allowing for jump concentration scenarios.

The stability analysis performed leverages a sequence of Lyapunov functions $v_k(y,h,x),k\ge 0$, whose properties guarantee the stability of the solutions to Equations (1)–(3). Under assumption (7), these Lyapunov functions can explicitly be constructed as

$$v_k(y,h,x)=d_k{v}_0(y,h,x),$$

where constants $d_k=1+{\sum }_{m=1}^k{\gamma }_m<\infty$. Additionally, assumption (7) significantly relaxes the previously stringent condition (8) used in earlier studies [16]. Thus, the derived stability conditions for stochastic differential equations with jump concentration points combine conditions from systems without jumps ($g=0$) and constraints on jump magnitudes.

In the special case of linear stochastic differential equations, the stability conditions simplify to the existence of positive-definite solutions to Riccati-type matrix equations, similar to the classical cases. These conditions, derived from Equation (32), are sufficient but do not fully characterize all stable systems, as demonstrated by the examples in [5].

Future research directions will focus specifically on linear systems, aiming to define both necessary and sufficient stability conditions and determine the existence of optimal control solutions.