The Linear Quadratic Optimal Control Problem for Stochastic Systems Controlled by Impulses

Abstract

This paper focuses on addressing the linear quadratic (LQ) optimal control problem on an infinite time horizon for stochastic systems controlled by impulses. No constraint regarding the sign of the quadratic functional is applied. That is why our first concern is to find conditions which guarantee that the considered optimal control problem is well posed. Then, when the optimal control problem is well posed, it is natural to look for conditions which guarantee the attainability of the optimal control problem that is being evaluated. The main tool involved in the solution of the problems stated before is a backward jump matrix linear differential equation (BJMLDE) with a Riccati-type jumping operator. This is formulated using the matrix coefficients of the controlled system and the weight matrices of the performance criterion. We show that the well posedness of the optimal control problem under investigation is guaranteed by the existence of the maximal and bounded solution of the associated BJMLDE with a Riccati-type jumping operator. Further, we show that when the associated BJMLDE with a Riccati-type jumping operator has a maximal solution which satisfies a suitable sign condition, then the optimal control problem is attainable if and only if it has an optimal control in a state feedback form, or if and only if the maximal solution of the BJMLDE with a Riccati-type jumping operator is a stabilizing solution. In order to make the paper more self-contained, we present a set of conditions that correspond to the existence of the maximal solution of the BJMLDE satisfying the desired sign condition.

1. Introduction

Studying impulsive systems holds significant theoretical and practical significance. In more formal terms, impulsive systems belong to a category of dynamic systems, where state evolution follows continuous time dynamics, except for a number of instances where the state undergoes instantaneous changes. Numerous examples of such systems exist as referenced in [1,2,3]. These systems serve as valuable models for diverse real-world applications across engineering [4,5,6], environmental science [7], and mathematical finance [8]. For the latest insights into the subject matter, readers can consult the review article [9].

LQ control theory and its utilization of Riccati equations has grown substantially. This initial expansion includes investigations into both deterministic systems [10,11] and stochastic systems [12,13], marking it as a significant area within control theory. In 2000, Rami, Zhou, and Moore [14] examined the attainability of the LQ problem; specifically, the existence of an optimal control is established using the generalized algebraic Riccati equation. Numerous developments have been made in this area, and we will highlight only a few that are particularly relevant: in [15], the examination revolves around studying LQ optimal control problems in an infinite horizon characterized by stochastic coefficients. In [16], the investigation centers on exploring optimal quadratic control for an affine equation in the infinite horizon driven by Levy processes. In [17], the authors examine an LQ optimal control problem in the infinite horizon case pertaining to mean-field stochastic differential equations. Also worth considering are references [18,19,20,21].

In [22,23], several criteria for exponential stability in the mean square and mean square stabilizability of linear stochastic systems controlled by impulses are provided.

In the present work, we consider a linear quadratic (LQ) optimal control problem for a dynamical system described by the Ito differential equation controlled by impulses. Often, such an optimal control problem by impulses is obtained as an equivalent version of an optimal control problem requiring the minimization of a quadratic criterion along the trajectories of a stochastic dynamical system controlled by piece-wise constant controls; see [24]. No constraint regarding the sign of the quadratic functional is applied. That is why our first concern is to find conditions which guarantee that the considered optimal control problem is well posed. Then, when the optimal control problem is well posed, it is natural to look for conditions which guarantee the attainability of the optimal control problem that is being evaluated. The main tool involved in the solution of the problems stated before is a backward jump matrix linear differential equation (BJMLDE) with a Riccati-type jumping operator. This is formulated using the matrix coefficients of the controlled system and the weight matrices of the performance criterion.

It is worth mentioning that in the case of an LQ control problem for a stochastic system controlled by impulses, the BJMLDE with a Riccati-type jumping operator plays the role which the Riccati differential equation or the Riccati algebraic equation, respectively, has in the case of an LQ control problem, where the control laws do not act by impulses. That is why it is natural to introduce the concepts of maximal solution and stabilizing solution of a BJMLDE with a Riccati-type jumping operator.

The principal contributions of the present paper encompass the following:

• We introduce the concepts of maximal solution and stabilizing solution of the BJMLDE with a Riccati-type jumping operator and provide necessary and sufficient conditions which guarantee the existence of these solutions.
• We show that the existence of the maximal solution of the BJMLDE with a Riccati-type jumping operator satisfying a suitable sign condition guarantees the well posedness of the optimal control problem under consideration. We introduce the value function $\mathbf{v}(t_0,x_0)$ associated to the optimal control problem and compute its value for all initial pairs $(t_0,x_0)\in [0,\infty )\times {\mathbb{R}}^n$.
• Under the assumption that the corresponding BJMLDE with a Riccati-type jumping operator has a maximal solution which satisfies a suitable sign condition, the optimal control problem under consideration is attainable if and only if it admits a unique optimal control which is in a state feedback form, or equivalently, if and only if the maximal solution of the BJMLDE with a Riccati-type jumping operator is a stabilizing solution too.

The derivations use the equivalence of the exponential stability in the mean square of a continuous time jump linear stochastic system with the exponential stability of the companion discrete-type linear equation. That is why we do not need to refer to the criteria for the exponential stability in the mean square or to the criteria for stabilizability in the mean square of the linear stochastic systems controlled by impulses existing in the literature, such as those from [22,23].

The rest of the paper is structured as follows: In Section 2, we introduce the problem and provide necessary preliminary concepts. In Section 3, we introduce the backward jump matrix linear differential equation (BJMLDE) with a Riccati-type jumping operator and provide a necessary and sufficient condition for the existence of the maximal and bounded solution for this (see Theorems 1 and 2). Section 4 is centered on the the well posedness (see Theorem 3) and attainability (see Theorem 4) of the optimal control problem. Finally, Section 5 offers the concluding remarks for the paper.

2. The Problem

Consider the impulsive controlled linear stochastic system (ICLSS) described by

$$\begin{array}{cc}\hfill & dx\left(t\right)=A_{0}x\left(t\right)dt+A_{1}x\left(t\right)dw\left(t\right),kh<t\le (k+1)h,\hfill \\ \hfill & x\left(k{h}^{+}\right)={\mathcal{A}}_{0}x\left(kh\right)+{\mathcal{B}}_{0}u\left(k\right)+w_d\left(k\right)({\mathcal{A}}_{1}x\left(kh\right)+{\mathcal{B}}_{1}u\left(k\right)),\hfill \end{array}$$

(1a)

$$\begin{array}{cc}\hfill & h>0\mathrm{is}\mathrm{given},k\in {\mathbb{Z}}_{+}=\{0,1,\dots \},\hfill \end{array}$$

(1b)

$$\begin{array}{cc}\hfill & x\left(0\right)=x_0,\hfill \end{array}$$

(1c)

where we denote by $x\left(t\right)\in {\mathbb{R}}^n$ the state vector of the system and by $u\left(k\right)\in {\mathbb{R}}^m$ the control parameters. In (1a), ${\left\{w\left(t\right)\right\}}_{t\ge 0}$ is a one-dimensional standard Wiener process (or process of the Brownian motion). This is defined on a given probability space $(\mathsf{\Omega },\mathfrak{F},\mathcal{P})$, and in (1b), ${\left\{w_d\left(k\right)\right\}}_{k\in {\mathbb{Z}}_{+}}$ is a sequence of independent random variables with zero mean and variance one. For further information on the definition and characteristics of the Wiener process, we can refer to [25,26].

Throughout the work, we assume that ${\left\{w\left(t\right)\right\}}_{t\ge 0}$ and ${\left\{w_d\left(k\right)\right\}}_{k\in {\mathbb{Z}}_{+}}$ are independent stochastic processes. For each $t\ge 0$, ${\mathcal{F}}_t\subset \mathfrak{F}$ represents the $\sigma -$algebra generated by the random variables $w\left(s\right)$, $w_d\left(k\right)$, $t\ge s\ge 0,$ and ${\displaystyle \frac{t}{h}}>k\ge 0$, augmented with all subsets $\mathfrak{A}\in \mathfrak{F}$ with $\mathcal{P}\left(\mathfrak{A}\right)=0.$

In (1) the controls $u\left(k\right)$ act at impulsive time instances ${\tau }_k=kh$, $k=0,1,\dots .$ That is why this kind of system is called impulsive controlled systems.

Based on Theorem 5.2.1 from [27] used on each interval $[kh,(k+1\left)h\right]$, we can demonstrate the following.

Proposition 1.

For each initial pair $(t_0,x_0)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n$ and for any random vectors $u\left(k\right):\mathsf{\Omega }\to {\mathbb{R}}^m$ which are ${\mathcal{F}}_{kh}-$measurable with

$$\mathbb{E}\left[\right|u\left(k\right){|}^2]<\infty ,$$

the ICLSS (1) has a unique solution $x\left(t\right)=x(t;t_0,x_0,\mathbf{u})$, $t\ge 0,$ with the following properties:

(a)

$x(·;t_0,x_0,\mathbf{u})$ is continuous (with probability 1) in any $t\ne kh$, left continuous in $kh,$ for $kh>t_0;$

(b)

$x(·;t_0,x_0,\mathbf{u})\in {\mathcal{L}}_{\mathcal{F}}^2\{[t_0,T],{\mathbb{R}}^n\}$ for all $T>t_0;$

(c)

$x(t_0;t_0,x_0,\mathbf{u})=x_0$ and $\underset{\begin{array}{c}t\to kh\\ t>kh\end{array}}{lim}x(t;t_0,x_0,\mathbf{u})=x\left(k{h}^{+}\right)$, where $x\left(k{h}^{+}\right)$ is provided by (1b).

We recall that ${\mathbb{R}}_{+}=[0,\infty ),$ and $\mathbb{E}[·]$ is used for the mathematical expectation. If $\mathcal{I}\subset {\mathbb{R}}_{+}$ is an interval, then ${\mathcal{L}}_{\mathcal{F}}^2\{\mathcal{I},{\mathbb{R}}^p\}$ denotes the Hilbert space of the stochastic processes $\mathbf{Y}={\left\{y\left(t\right)\right\}}_{t\in \mathcal{I}}$, which are non-anticipative with respect to the filtration $\mathcal{F}={\left\{{\mathcal{F}}_t\right\}}_{t\in \mathcal{I}}$ and satisfies

$$\mathbb{E}\left[\underset{\mathcal{I}}{\int }{\left|y\left(t\right)\right|}^{2}dt\right]<\infty .$$

For each initial pair $(t_0,x_0)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n$, we denote ${\mathcal{U}}_{ad}(t_0,x_0)$ the set of controls $\mathbf{u}={\left\{u\left(k\right)\right\}}_{k\ge \kappa \left(t_0\right)}$ satisfying the following properties:

(a)

$u\left(k\right):\mathsf{\Omega }\to {\mathbb{R}}^m$ are ${\mathcal{F}}_{kh}-$measurable and

$$\sum _{k=\kappa \left(t_0\right)}^{\infty }{\mathbb{E}\left[\right|u\left(k\right)|}^2]<\infty ;$$

(b)

$$\begin{array}{cc}\hfill & \mathbb{E}\left[{\int }_{t_0}^{\infty }{\left|x(t;t_0,x_0,\mathbf{u})\right|}^{2}dt\right]+\sum _{k=\kappa \left(t_0\right)}^{\infty }\mathbb{E}\left[\right|x(kh;t_0,x_0,\mathbf{u}){|}^2]<\infty ,\hfill \end{array}$$

(2a)

$$\begin{array}{cc}\hfill & \underset{t\to \infty }{lim}\mathbb{E}\left[\right|x(t;t_0,x_0,\mathbf{u}){|}^2]=0.\hfill \end{array}$$

(2b)

Here and in the sequel, $\kappa \left(t_0\right)$ is the integer with the property that

$$(\kappa \left(t_0\right)-1)h<t_0\le \kappa \left(t_0\right)h.$$

(3)

In (2), $x(·;t_0,x_0,\mathbf{u})$, $t\ge t_0$ is the trajectory of ICLSS (1) starting from $x_0$ at the initial time $t_0$, and it is determined by the input $\mathbf{u}={\left\{u\left(k\right)\right\}}_{k\ge \kappa \left(t_0\right)}.$

It is worth noticing that if the ICLSS (1) is mean square stabilizable by the linear state feedback in the sense of Definition 2 from [28], then the sets ${\mathcal{U}}_{ad}(t_0,x_0)$ are not empty for any initial pair $(t_0,x_0)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n.$ In the sequel, ${\mathcal{U}}_{ad}(t_0,x_0)$ will be called the set of admissible controls associated to the initial pair $(t_0,x_0).$

To the pair formed by the ICLSS (1) and the admissible controls ${\mathcal{U}}_{ad}(t_0,x_0),$ we associate the following quadratic functional:

$$\begin{array}{cc}\hfill & J(t_0,\infty ,x_0;\mathbf{u})=\underset{t_0}{\overset{\infty }{\int }}\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(t\right)M{x}_{\mathbf{u}}\left(t\right)\right]dt\hfill \\ \hfill & +\sum _{k=\kappa \left(t_0\right)}^{\infty }\mathbb{E}[x_{\mathbf{u}}^{\top }\left(kh\right)M_d{x}_{\mathbf{u}}\left(kh\right)+2x_{\mathbf{u}}^{\top }\left(kh\right)L_{d}u\left(k\right)+u^{\top }\left(k\right)R_{d}u\left(k\right)],\hfill \end{array}$$

(4)

where $x_{\mathbf{u}}(·)=x(·;t_0,x_0,\mathbf{u}).$

Regarding the matrix coefficients of (1) and the weight matrices from (4), we assume that $A_j$, ${\mathcal{A}}_j\in {\mathbb{R}}^{n\times n}$, ${\mathcal{B}}_j\in {\mathbb{R}}^{n\times m}$, $j=0,1$, $M,M_d\in {\mathcal{S}}_n$, $R_d\in {\mathcal{S}}_m$, $L_d\in {\mathbb{R}}^{n\times m}$ are known matrices. Here and in the sequel, ${\mathcal{S}}_q$ represents the linear space of real symmetric matrices of size $q\times q.$

The optimal control problem we aim to solve in this work involves finding conditions that ensure the existence of an admissible control $\tilde{\mathbf{u}}={\left\{\tilde{u}\right\}}_{k\ge \kappa \left(t_0\right)}$ which is satisfying the optimality condition

$$-\infty <J(t_0,\infty ,x_0;\tilde{\mathbf{u}})\le J(t_0,\infty ,x_0;\mathbf{u}),$$

(5)

for any $\mathbf{u}\in {\mathcal{U}}_{ad}(t_0,x_0).$

Since we do not impose any condition regarding the sign of the considered weighting matrices from (4), it is necessary to find first conditions which guarantee that the functional (4) is bounded below across the set of the admissible controls ${\mathcal{U}}_{ad}(t_0,x_0).$

To this end, we introduce the so-called value function $\mathbf{v}(·,·)$ associated to the optimal control problem under consideration. We set

$$\mathbf{v}(t_0,x_0)\stackrel{\Delta }{=}\underset{\mathbf{u}\in {\mathcal{U}}_{ad}(t_0,x_0)}{inf}J(t_0,\infty ,x_0;\mathbf{u}).$$

(6)

Definition 1.

We say that the optimal control problem described by the performance criterion (4), the controlled system (1), and the set of the admissible controls ${\mathcal{U}}_{ad}(t_0,x_0)$ is as follows:

(a)

Well posed if

$$\mathbf{v}(t_0,x_0)>-\infty ,$$

for any initial pair $(t_0,x_0)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n;$

(b)

Attainable, if for each $(t_0,x_0)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n$, there exists the control $\tilde{\mathbf{u}}\in {\mathcal{U}}_{ad}(t_0,x_0)$ with

$$J(t_0,\infty ,x_0;\tilde{\mathbf{u}})=\mathbf{v}(t_0,x_0)>-\infty .$$

(7)

In Section 3, we examine the issue of whether a maximal solution and a stabilizing solution exist for this type of backward jump matrix linear differential equation.

In Section 4, we shall develop conditions which guarantee that the optimal control problem under consideration is well posed and attainable. The primary instrument is a BJMLDE with a Riccati-type jumping operator.

3. BJMLDE with Riccati-Type Jumping Operators

Based on the matrix coefficients of the ICLSS (1) and the weighting matrices provided by the performance criterion (4), we introduce the following BJMLDE with a Riccati-type jumping operator:

$$\begin{array}{cc}\hfill & \dot{X}\left(t\right)+A_0^{\top }X\left(t\right)+X\left(t\right)A_0+A_1^{\top }X\left(t\right)A_1+M=0,kh\le t<(k+1)h,\hfill \end{array}$$

(8a)

$$\begin{array}{cc}\hfill & X\left(k{h}^{-}\right)=\sum _{j=0}^1{\mathcal{A}}_j^{\top }X\left(kh\right){\mathcal{A}}_j-\left(\sum _{j=0}^1{\mathcal{A}}_j^{\top }X\left(kh\right){\mathcal{B}}_j+L_d\right)\hfill \\ \hfill & ·{\left(\sum _{j=0}^1{\mathcal{B}}_j^{\top }X\left(kh\right){\mathcal{B}}_j+R_d\right)}^{-1}\left(\sum _{j=0}^1{\mathcal{B}}_j^{\top }X\left(kh\right){\mathcal{A}}_j+L_d^{\top }\right)\hfill \\ \hfill & +M_d,k\in {\mathbb{Z}}_{+}.\hfill \end{array}$$

(8b)

${\mathsf{\Gamma }}^{\mathsf{\Sigma }}$ is the set of all bounded functions $Z:{\mathbb{R}}_{+}\to {\mathcal{S}}_n$ satisfying the following:

(a)

$Z(·)$ is a differentiable function on each interval $(kh,(k+1\left)h\right),$ right continuous in $t_k=kh$, $k\in {\mathbb{Z}}_{+};$

(b)

$Z(·)$ verify the following jump matrix linear differential inequalities:

$$\begin{array}{cc}\hfill & \dot{Z}\left(t\right)+A_0^{\top }Z\left(t\right)+Z\left(t\right)A_0+A_1^{\top }Z\left(t\right)A_1+M\ge 0,kh\le t<(k+1)h,\hfill \end{array}$$

(9a)

$$\begin{array}{cc}\hfill & \left(\begin{array}{cc}-Z\left(k{h}^{-}\right)+M_d& L_d\\ L_d^{\top }& R_d\end{array}\right)+\sum _{j=0}^1{\left(\begin{array}{cc}{\mathcal{A}}_j& {\mathcal{B}}_j\end{array}\right)}^{\top }Z\left(kh\right)\left(\begin{array}{cc}{\mathcal{A}}_j& {\mathcal{B}}_j\end{array}\right)\ge 0,\hfill \end{array}$$

(9b)

$$\begin{array}{cc}\hfill & \sum _{j=0}^1{\mathcal{B}}_j^{\top }Z\left(kh\right){\mathcal{B}}_j+R_d\ge \gamma I_m>0,\hfill \end{array}$$

(9c)

for all $k\in {\mathbb{Z}}_{+}$, $\gamma >0.$

Remark 1.

Employing the Schur complement technique (Theorem 1 from [29]), one may show that the set ${\mathsf{\Gamma }}^{\mathsf{\Sigma }}$ contains all the bounded solutions $t\to X\left(t\right):{\mathbb{R}}_{+}\to {\mathcal{S}}_n$ of the BJMLDE (8) which are satisfying the sign conditions

$$\sum _{j=0}^1{\mathcal{B}}_j^{\top }X\left(kh\right){\mathcal{B}}_j+R_d\ge \gamma I_m>0,$$

for all $k\in {\mathbb{Z}}_{+}.$

A significant role will be performed by the subset ${\tilde{\mathsf{\Gamma }}}^{\mathsf{\Sigma }}$ which contains all bounded functions $Z(·)\in {\mathsf{\Gamma }}^{\mathsf{\Sigma }}$ satisfying the stronger condition:

$$\left(\begin{array}{cc}-Z\left(k{h}^{-}\right)+M_d& L_d\\ L_d^{\top }& R_d\end{array}\right)+\sum _{j=0}^1{\left(\begin{array}{cc}{\mathcal{A}}_j& {\mathcal{B}}_j\end{array}\right)}^{\top }Z\left(kh\right)\left(\begin{array}{cc}{\mathcal{A}}_j& {\mathcal{B}}_j\end{array}\right)\ge \widehat{\gamma }{I}_{n+m}>0,$$

(10)

for all $k\in {\mathbb{Z}}_{+}.$

Definition 2.

A globally defined solution $t\to X_{max}\left(t\right):{\mathbb{R}}_{+}\to {\mathcal{S}}_n$ of the BJMLDE (8) is named a maximal solution if

$$X_{max}\left(t\right)\ge Z\left(t\right),$$

for all $t\in {\mathbb{R}}_{+}$ for any $Z(·)\in {\mathsf{\Gamma }}^{\mathsf{\Sigma }}.$

Definition 3.

A globally defined solution $t\to X_s\left(t\right):{\mathbb{R}}_{+}\to {\mathcal{S}}_n$ of the BJMLDE (8) is called the stabilizing solution if the closed-loop jump stochastic linear differential equation, JSLDE,

$$\begin{array}{cc}\hfill & dx\left(t\right)=A_{0}x\left(t\right)dt+A_{1}x\left(t\right)dw\left(t\right),kh<t\le (k+1)h,\hfill \end{array}$$

(11a)

$$\begin{array}{cc}\hfill & x\left(k{h}^{+}\right)=({\mathcal{A}}_0+{\mathcal{B}}_0{F}_s\left(kh\right))x\left(kh\right)+w_d\left(k\right)({\mathcal{A}}_1+{\mathcal{B}}_1{F}_s\left(kh\right))x\left(kh\right),\hfill \end{array}$$

(11b)

$k\in {\mathbb{Z}}_{+}$ is exponentially stable in mean square (ESMS), where

$$F_s\left(kh\right)=-{\left(\sum _{j=0}^1{\mathcal{B}}_j^{\top }{X}_s\left(kh\right){\mathcal{B}}_j+R_d\right)}^{-1}\left(\sum _{j=0}^1{\mathcal{B}}_j^{\top }{X}_s\left(kh\right){\mathcal{A}}_j+L_d^{\top }\right).$$

(12)

Definition 4.

The ICLSS is mean square stabilizable by the linear state feedback if there exists a matrix $F\in {\mathbb{R}}^{m\times n}$ such that the following closed-loop JSLDE

$$\begin{array}{cc}\hfill & dx\left(t\right)=A_{0}x\left(t\right)dt+A_{1}x\left(t\right)dw\left(t\right),kh<t\le (k+1)h,\hfill \end{array}$$

(13a)

$$\begin{array}{cc}\hfill & x\left(k{h}^{+}\right)=({\mathcal{A}}_0+{\mathcal{B}}_{0}F)x\left(kh\right)+w_d\left(k\right)({\mathcal{A}}_1+{\mathcal{B}}_{1}F)x\left(kh\right),\hfill \end{array}$$

(13b)

$k\in {\mathbb{Z}}_{+}$, is ESMS.

A set of necessary and sufficient criteria for the mean square stabilizability by the linear state feedback of an ICLSS of type (1) is provided by Theorem 6 in [28].

In this section, we shall highlight the necessary and sufficient conditions for the existence of the maximal solution and the stabilizing solution of a BJMLDE of type (8). To this end, we will appeal to the properties of the linear and positive operators and of the linear operators which are defining a positive evolution on an ordered Hilbert space.

We recall that ${\mathcal{S}}_n$ is a finite dimensional real Hilbert space with the inner product:

$$⟨X,Y⟩:=\mathrm{Tr}\left[XY\right],$$

(14)

for all $X,Y\in {\mathcal{S}}_n$, here, $\mathrm{Tr}[·]$ stands for the trace operator. Furthermore, ${\mathcal{S}}_n$ becomes an ordered Hilbert space with the order relation ⪰ provided by the convex cone

$${\mathcal{S}}_n^{+}:=\{X\in {\mathcal{S}}_n|X\mathrm{is}\mathrm{a}\mathrm{positive}\mathrm{semidefinite}\mathrm{matrix}\}.$$

Consider $X\to \mathcal{L}\left[X\right]:{\mathcal{S}}_n\to {\mathcal{S}}_n$ the linear operator defined as

$$\mathcal{L}\left[X\right]=A_{0}X+X{A}_0^{\top }+A_{1}X{A}_1^{\top },$$

(15)

for all $X\in {\mathcal{S}}_n.$ Based on the inner product (14), we consider the adjoint operator ${\mathcal{L}}^{*}[·]$ which is described by

$${\mathcal{L}}^{*}\left[X\right]=A_0^{\top }X+X{A}_0+A_1^{\top }X{A}_1,$$

(16)

for all $X\in {\mathcal{S}}_n.$ With these notations, the differential Equation (8a) can be expressed in a compact form as

$$\dot{X}\left(t\right)+{\mathcal{L}}^{*}\left[X\left(t\right)\right]+M=0,kh<t\le (k+1)h.$$

(17)

Let $t\to X\left(t\right):{\mathbb{R}}_{+}\to {\mathcal{S}}_n$ be a globally defined solution of (8). From (17), we obtain that

$$\begin{array}{cc}\hfill X\left(t\right)=& e^{{\mathcal{L}}^{*}((k+1)h-t)}\left[X\left((k+1)h^{-}\right)\right]+\underset{t}{\overset{(k+1)h}{\int }}{e}^{{\mathcal{L}}^{*}(s-t)}\left[M\right]ds,\hfill \\ \hfill & t\in (kh,(k+1)h),k\in {\mathbb{Z}}_{+}.\hfill \end{array}$$

(18)

Letting $t\to kh$, $t>kh$ in (18) and substituting the result in (8b), one obtains that the sequence ${\left\{X\left(k{h}^{-}\right)\right\}}_{k\in {\mathbb{Z}}_{+}}$ solves the next generalized discrete-time Riccati-type equation (GDTRE):

$$\begin{array}{cc}\hfill Y\left(k\right)=& {\mathsf{\Pi }}_1\left[Y(k+1)\right]+\tilde{M}-({\mathsf{\Pi }}_2\left[Y(k+1)\right]+\tilde{L})\hfill \\ \hfill & ·{({\mathsf{\Pi }}_3\left[Y(k+1)\right]+\tilde{R})}^{-1}{({\mathsf{\Pi }}_2\left[Y(k+1)\right]+\tilde{L})}^{\top },\hfill \end{array}$$

(19)

where

$$\begin{array}{cc}\hfill & {\mathsf{\Pi }}_1\left[Y\right]:=\sum _{j=0}^1{\mathcal{A}}_j^{\top }{e}^{{\mathcal{L}}^{*}h}\left[Y\right]{\mathcal{A}}_j,\hfill \end{array}$$

(20a)

$$\begin{array}{cc}\hfill & {\mathsf{\Pi }}_2\left[Y\right]:=\sum _{j=0}^1{\mathcal{A}}_j^{\top }{e}^{{\mathcal{L}}^{*}h}\left[Y\right]{\mathcal{B}}_j,\hfill \end{array}$$

(20b)

$$\begin{array}{cc}\hfill & {\mathsf{\Pi }}_3\left[Y\right]:=\sum _{j=0}^1{\mathcal{B}}_j^{\top }{e}^{{\mathcal{L}}^{*}h}\left[Y\right]{\mathcal{B}}_j,\hfill \end{array}$$

(20c)

$$\begin{array}{cc}\hfill & \tilde{M}:=M_d+\sum _{j=0}^1{\mathcal{A}}_j^{\top }\underset{0}{\overset{h}{\int }}{e}^{{\mathcal{L}}^{*}s}\left[M\right]ds{\mathcal{A}}_j,\hfill \end{array}$$

(21a)

$$\begin{array}{cc}\hfill & \tilde{L}:=L_d+\sum _{j=0}^1{\mathcal{A}}_j^{\top }\underset{0}{\overset{h}{\int }}{e}^{{\mathcal{L}}^{*}s}\left[M\right]ds{\mathcal{B}}_j,\hfill \end{array}$$

(21b)

$$\begin{array}{cc}\hfill & \tilde{R}:=R_d+\sum _{j=0}^1{\mathcal{B}}_j^{\top }\underset{0}{\overset{h}{\int }}{e}^{{\mathcal{L}}^{*}s}\left[M\right]ds{\mathcal{B}}_j.\hfill \end{array}$$

(21c)

Employing Theorem 2.6.1 from [30], we obtain the following.

Corollary 1.

If $\mathcal{L}$ and ${\mathcal{L}}^{*}$ are the linear operators, defined by (15) and (16), then both $e^{\mathcal{L}t}$ as well as $e^{{\mathcal{L}}^{*}t}$, $t\ge 0,$ are positive operators defined on the ordered Hilbert space $({\mathcal{S}}_n,{\mathcal{S}}_n^{+}),$ that is

$$e^{\mathcal{L}t}\left[Y\right]⪰0,$$

and

$$e^{{\mathcal{L}}^{*}t}\left[Y\right]⪰0,$$

for all $t\ge 0$ if $Y⪰0.$

Based on (20), we introduce the following linear operator $Y\to \mathsf{\Pi }\left[Y\right]:{\mathcal{S}}_n\to {\mathcal{S}}_{n+m}$ defined by

$$\mathsf{\Pi }\left[Y\right]:=\left(\begin{array}{cc}{\mathsf{\Pi }}_1\left[Y\right]& {\mathsf{\Pi }}_2\left[Y\right]\\ {\mathsf{\Pi }}_2^{\top }\left[Y\right]& {\mathsf{\Pi }}_3\left[Y\right]\end{array}\right),$$

(22)

for all $Y\in {\mathcal{S}}_n.$ From (20) and (22), one obtains via Corollary 1 that

$$\mathsf{\Pi }\left[Y\right]⪰0\mathrm{if}Y⪰0.$$

This allows us to conclude that the GDTRE (19) is a special case of the relation 5.8 from [31].

The GDTRE (19) is determined by the pair ${\mathsf{\Sigma }}_d=(\mathsf{\Pi },\mathcal{Q}),$ where $\mathsf{\Pi }:{\mathcal{S}}_n\to {\mathcal{S}}_{n+m}$ is the linear operator defined in (22), while

$$\mathcal{Q}=\left(\begin{array}{cc}\tilde{M}& \tilde{L}\\ {\tilde{L}}^{\top }& \tilde{R}\end{array}\right),$$

with $\tilde{M}$, $\tilde{L},$ and $\tilde{R}$ being described in (21). To the pair ${\mathsf{\Sigma }}_d$, we associate the following sets of sequences of symmetric matrices:

$$\begin{array}{c}\hfill {\mathsf{\Gamma }}^{{\mathsf{\Sigma }}_d}:=\{\mathbf{Z}={\left\{Z\left(k\right)\right\}}_{k\ge 0}|\mathcal{D}\left[\mathbf{Z}\right]\left(k\right)⪰0,{\mathsf{\Pi }}_3\left[Z(k+1)\right]+\tilde{R}⪰{\gamma }_d{I}_m>0,k\ge 0\},\end{array}$$

(23)

$${\tilde{\mathsf{\Gamma }}}^{{\mathsf{\Sigma }}_d}:=\{\mathbf{Z}={\left\{Z\left(k\right)\right\}}_{k\ge 0}|\mathcal{D}\left[\mathbf{Z}\right]\left(k\right)⪰{\widehat{\gamma }}_d{I}_{n+m}>0,\mathrm{for}\mathrm{all}k\ge 0\},$$

(24)

where

$$\mathcal{D}\left[\mathbf{Z}\right]\left(k\right):=\left(\begin{array}{cc}-Z\left(k\right)& 0\\ 0& 0\end{array}\right)+\mathsf{\Pi }\left[Z(k+1)\right]+\mathcal{Q}.$$

(25)

Lemma 1.

(a)

There exists a one-to-one correspondence between the set ${\mathsf{\Gamma }}^{\mathsf{\Sigma }}$ of the matrix valued functions $Z(·)$ which are satisfying the linear matrix differential inequalities with jumps (9) and the set ${\mathsf{\Gamma }}^{{\mathsf{\Sigma }}_d}$ of the sequences of symmetric matrices $\mathbf{Z}={\left\{Z\left(k\right)\right\}}_{k\ge 0}$ which are satisfying the linear matrix inequalities (23);

(b)

There exists a one-to-one correspondence between the set ${\tilde{\mathsf{\Gamma }}}^{\mathsf{\Sigma }}$ of the matrix valued functions which satisfy the linear matrix inequalities (9a) with jumps described by (10) and the set ${\tilde{\mathsf{\Gamma }}}^{{\mathsf{\Sigma }}_d}$ of the matrix valued sequences $\mathbf{Z}={\left\{Z\left(k\right)\right\}}_{k\ge 0}$ which satisfy the linear matrix inequalities (24).

Proof. 

$\left(a\right)$ If $Z(·)$ is satisfying (9a), we define

$$\Delta \left(t\right):=\dot{Z}\left(t\right)+{\mathcal{L}}^{*}\left[Z\left(t\right)\right]+M.$$

So (9a) can be expressed as

$$-\dot{Z}\left(t\right)={\mathcal{L}}^{*}\left[Z\left(t\right)\right]+M-\Delta \left(t\right),kh\le t<(k+1)h,k\in {\mathbb{Z}}_{+},$$

where

$$\Delta \left(t\right)\ge 0,\mathrm{for}\mathrm{all}t\in {\mathbb{R}}_{+}.$$

Hence, if $Z(·)$ is satisfying (9a), it also satisfies

$$\begin{array}{cc}\hfill Z\left(t\right)=& e^{{\mathcal{L}}^{*}((k+1)h-t)}\left[Z\left((k+1)h^{-}\right)\right]+\underset{t}{\overset{(k+1)h}{\int }}{e}^{{\mathcal{L}}^{*}(s-t)}\left[M\right]ds\hfill \\ \hfill & -\underset{t}{\overset{(k+1)h}{\int }}{e}^{{\mathcal{L}}^{*}(s-t)}[\Delta \left(s\right)]ds,kh\le t<(k+1)h,k\in {\mathbb{Z}}_{+}.\hfill \end{array}$$

(26)

Letting $t\to kh$, $t>kh$ in (26) and substituting the result in (9b) and in (9c), we obtain via (20) and (21) that the sequence ${\left\{Z\left(k{h}^{-}\right)\right\}}_{k\in {\mathbb{Z}}_{+}}$ lies in ${\mathsf{\Gamma }}^{{\mathsf{\Sigma }}_d}$ because

$$\underset{t}{\overset{(k+1)h}{\int }}{e}^{{\mathcal{L}}^{*}(s-t)}[\Delta \left(s\right)]ds\ge 0,\mathrm{for}\mathrm{all}k\in {\mathbb{Z}}_{+}.$$

Conversely, if $\mathbf{Y}={\left\{Y\left(k\right)\right\}}_{k\ge 0}$ lies in ${\mathsf{\Gamma }}^{{\mathsf{\Sigma }}_d}$, we define

$$Z\left(t\right):=e^{{\mathcal{L}}^{*}((k+1)h-t)}\left[Y(k+1)\right]+\underset{t}{\overset{(k+1)h}{\int }}{e}^{{\mathcal{L}}^{*}(s-t)}\left[M\right]ds,$$

(27)

for all $kh\le t<(k+1)h,$ for any $k\in {\mathbb{Z}}_{+}.$ Letting $t\to (k+1)h$, $t<(k+1)h$ in (27), we obtain that

$$Z\left((k+1)h^{-}\right)=Y(k+1),\mathrm{for}\mathrm{all}k\in {\mathbb{Z}}_{+}.$$

(28)

Differentiating (27) with respect to the parameter t, we obtain

$$\dot{Z}\left(t\right)+{\mathcal{L}}^{*}\left[Z\left(t\right)\right]+M=0,kh\le t<(k+1)h.$$

Employing (20), (23), (25), and (27) written for $t=kh$ with (28) written for k replaced by $k-1,$ we obtain that $Z(·)$ satisfies (9b) and (9c). Hence, $Z(·)$ defined in (27) lies in ${\mathsf{\Gamma }}^{\mathsf{\Sigma }}.$ Thus, the proof of $\left(a\right)$ is complete.

Part $\left(b\right)$ may be proved in a similar manner. □

Particularizing the Definition 4.7 from [31] for the stabilizability of a sequence of linear operators to the case of the linear operator $\mathsf{\Pi }$ defined by (20) and (22), we obtain the following.

Definition 5.

The linear operator $\mathsf{\Pi }:{\mathcal{S}}_n\to {\mathcal{S}}_{n+m}$ is stabilizable if there exists a matrix $F\in {\mathbb{R}}^{m\times n}$ with the property that the discrete time linear equation on the Hilbert space ${\mathcal{S}}_n$

$$S(k+1)={\mathsf{\Pi }}_F^{*}\left[S\left(k\right)\right],$$

(29)

is exponentially stable. ${\mathsf{\Pi }}_F^{*}[·]$ is the adjoint of the linear operator $X\to {\mathsf{\Pi }}_F\left[X\right]:{\mathcal{S}}_n\to {\mathcal{S}}_n$ with respect to the inner product (14), where

$${\mathsf{\Pi }}_F\left[X\right]:=\left(\begin{array}{cc}{I}_n& F^{\top }\end{array}\right)\mathsf{\Pi }\left[X\right]{\left(\begin{array}{cc}{I}_n& F^{\top }\end{array}\right)}^{\top }.$$

(30)

By direct calculation, involving (20), (22), (29), and (30), we obtain the following.

Corollary 2.

On the linear operator specified by (20) and (22), we have the following equivalences:

(i)

The operator Π is stabilizable (in the framework of Definition 5);

(ii)

There exists a matrix $F\in {\mathbb{R}}^{m\times n}$ with the property that the discrete-time linear equation on the Hilbert space ${\mathcal{S}}_n$

$$S(k+1)=\sum _{j=0}^1{e}^{\mathcal{L}h}\left[({\mathcal{A}}_j+{\mathcal{B}}_{j}F)S\left(k\right){({\mathcal{A}}_j+{\mathcal{B}}_{j}F)}^{\top }\right],$$

(31)

is exponentially stable.

Invoking Proposition 3.1 and Proposition 3.3 from [28] in the case of the JSLDE (13), we obtain via Corollary 2 Definitions 4 and 5.

Corollary 3.

The ICLSS (1) is mean square stabilizable by the linear state feedback if and only if the accompanying linear operator Π associated via (20) and (22) is stabilizable.

The following result establishes the conditions that are both necessary and sufficient conditions for the existence of the maximal and bounded solution of the BJMLDE (8).

Theorem 1.

If the ICLSS (1) is mean square stabilizable by the linear state feedback, then we have the following equivalences:

(i)

The set ${\mathsf{\Gamma }}^{\mathsf{\Sigma }}$ is not empty;

(ii)

The BJMLDE (8) with a Riccati-type jumping operator has a unique maximal and bounded solution $X_{max}:{\mathbb{R}}_{+}\to {\mathcal{S}}_n$ which satisfies the sign condition

$$R_d+\sum _{j=0}^1{\mathcal{B}}_j^{\top }{X}_{max}\left(kj\right){\mathcal{B}}_j\ge \nu I_m>0,$$

(32)

for all $k\in {\mathbb{Z}}_{+}.$ Furthermore, $X_{max}(·)$ is a periodic function of period $h.$

Proof. 

The implication $\left(ii\right)\Rightarrow \left(i\right)$ follows immediately because, according to Remark 1, the maximal solution, if any, lies in ${\mathsf{\Gamma }}^{\mathsf{\Sigma }}.$

In order to prove the implication $\left(i\right)\Rightarrow \left(ii\right),$ let us remark that Lemma 1 $\left(a\right)$ together with Corollary 3 guarantees that in the case of GDTRE (19), the assumptions from Theorem 5.3 in [31] are fulfilled. Hence, under the considered assumptions, the GDTRE (19) has a unique maximal solution $Y_{max}\in {\mathcal{S}}_n$ which satisfies the following sign condition:

$${\mathsf{\Pi }}_3\left[Y_{max}\right]+\tilde{R}>0.$$

(33)

For each $k\ge 0$ and $t\in [kh,(k+1\left)h\right)$, we define

$$X_{max}\left(t\right):=e^{{\mathcal{L}}^{*}((k+1)h-t)}\left[Y_{max}\right]+\underset{t}{\overset{(k+1)h}{\int }}{e}^{{\mathcal{L}}^{*}(s-t)}\left[M\right]ds.$$

(34)

So,

$$X_{max}\left((k+1)h^{-}\right)=\underset{\begin{array}{c}t\to (k+1)h\\ t<(k+1)h\end{array}}{lim}{X}_{max}\left(t\right)=Y_{max},\mathrm{for}\mathrm{all}k=0,1,\dots .$$

One shows that (34) defines a periodic function of period h, and (33) becomes (32).

Finally, employing (20), (21), and (34), one see that $X_{max}(·)$ is a solution of the BJMLDE (8). Employing again Lemma 1 $\left(a\right)$, one obtains that $X_{max}(·)$ defined in (34) is just the unique bounded and maximal solution of the BJMLDE (8). Thus, the proof ends. □

Theorem 2.

The following are equivalent:

(i)

The ICLSS (1) is mean square stabilizable by the linear state feedback, and the set ${\tilde{\mathsf{\Gamma }}}^{\mathsf{\Sigma }}$ is not empty;

(ii)

The BJMLDE with a Riccati-type jumping operator (8) has a unique bounded and stabilizing solution $X_s(·):{\mathbb{R}}_{+}\to {\mathcal{S}}_n$ which satisfies the following sign condition:

$$R_d+\sum _{j=0}^1{\mathcal{B}}_j^{\top }{X}_s\left(kh\right){\mathcal{B}}_j\ge \tilde{\nu }{I}_m>0,$$

(35)

for all $k\in {\mathbb{Z}}_{+}.$ Further, $X_s(·)$ is a periodic function of period h, and it coincides with the maximal bounded solution $X_{max}(·)$ of (8).

Proof. 

Invoking Lemma 1 $\left(b\right)$ together with Corollary 3, we see that we can apply the implication $\left(i\right)\Rightarrow \left(ii\right)$ of Theorem 5.5 and Theorem 5.6 from [31] to deduce that the GDTRE (19) has a stabilizing solution $Y_s$ which satisfies the following sign condition:

$$\tilde{R}+{\mathsf{\Pi }}_3\left[Y_s\right]>0.$$

(36)

We define

$$X_s\left(t\right):=e^{{\mathcal{L}}^{*}((k+1)h-t)}\left[Y_s\right]+\underset{t}{\overset{(k+1)h}{\int }}{e}^{{\mathcal{L}}^{*}(s-t)}\left[M\right]ds,$$

(37)

for any $k=0,1,\dots ,$ and for all $t\in [kh,(k+1\left)h\right)$

One shows that $X_s(·)$ defined as before is just the stabilizing solution of (8). Thus, the proof of the implication $\left(i\right)\Rightarrow \left(ii\right)$ ends.

To prove $\left(ii\right)\Rightarrow \left(i\right)$, one takes into account that if $X_s(·)$ is the stabilizing and bounded solution of (8), then the sequence ${\left\{X_s\left(kh\right)\right\}}_{k\ge 0}$ is the stabilizing solution of the GDTRE (19). Reasoning from $\left(ii\right)\Rightarrow \left(i\right)$ from Theorem 5.6 of [31], we obtain that the operator $\mathsf{\Pi }$ is stabilizable (in the framework of Definition 5), and the set ${\tilde{\mathsf{\Gamma }}}^{\mathsf{\Sigma }}$ is not empty.

Invoking again Lemma 1 $\left(b\right)$ together with Corollary 3, we may conclude that the set ${\tilde{\mathsf{\Gamma }}}^{\mathsf{\Sigma }}$ is not empty, and the ICLSS (1) is mean square stabilizable by the linear state feedback. □

Remark 2.

According to the equivalence $\left(i\right)\iff \left(iii\right)$ from Proposition 5.2 from [31] applied in the particular case of the GDTRE (19), if the set ${\tilde{\mathsf{\Gamma }}}^{\mathsf{\Sigma }}$ is not empty, it necessarily contains constant sequences. That is why in view of Lemma 1 $\left(b\right),$ in order to test if the set ${\tilde{\mathsf{\Gamma }}}^{\mathsf{\Sigma }}$ is not empty, it is sufficient to check if the linear matrix inequality (LMI) (24)–(25) has a solution which is not dependent upon $k\in {\mathbb{Z}}_{+}.$

4. Well Posedness and Attainability of the Optimal Control Problem

First, we present an auxiliary result.

Lemma 2.

Let $X(·):{\mathcal{I}}^X\subset {\mathbb{R}}_{+}\to {\mathcal{S}}_n$ be a solution of the BJMLDE (8). For each $[t_0,T]\subset {\mathcal{I}}^X$, we have

$$\begin{array}{cc}\hfill & \underset{t_0}{\overset{T}{\int }}\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(t\right)M{x}_{\mathbf{u}}\left(t\right)\right]+\sum _{k=\kappa \left(t_0\right)}^{\kappa \left(T\right)-1}\mathbb{E}[x_{\mathbf{u}}^{\top }\left(kh\right)M_d{x}_{\mathbf{u}}\left(kh\right)+2x_{\mathbf{u}}^{\top }\left(kh\right)L_{d}u\left(k\right)\hfill \\ \hfill & +u^{\top }\left(k\right)R_{d}u\left(k\right)]=x_0^{\top }X\left(t_0^{-}\right)x_0-\mathbf{E}\left[x_{\mathbf{u}}^{\top }\left(T\right)X\left(T^{-}\right)x_{\mathbf{u}}\left(T\right)\right]\hfill \\ \hfill & +\sum _{k=\kappa \left(t_0\right)}^{\kappa T-1}\mathbb{E}\left[{(u\left(k\right)-F^X\left(kh\right)x_{\mathbf{u}}\left(kh\right))}^{\top }\mathbb{R}\left(X\left(kh\right)\right)(u\left(k\right)-F^X\left(kh\right)x_{\mathbf{u}}\left(kh\right))\right],\hfill \end{array}$$

(38)

where $\kappa \left(T\right)$ follows the definition provided in (3) but with $t_0$ replaced by T and

$$\mathbb{R}\left(X\left(kh\right)\right):=R_d+\sum _{j=0}^1{\mathcal{B}}_j^{\top }X\left(kh\right){\mathcal{B}}_j,$$

(39)

and

$$F^X\left(kh\right):=-{\left(\mathbb{R}\left(X\left(kh\right)\right)\right)}^{-1}\left(\sum _{j=0}^1{\mathcal{B}}_j^{\top }X\left(kh\right){\mathcal{A}}_j+L_d^{\top }\right).$$

(40)

In (38), $x_0^{\top }X\left(t_0^{-}\right)x_0$ is replaced by $x_0^{\top }X\left(t_0\right)x_0$ whenever $t_0<\kappa \left(t_0\right)h.$

Proof. 

Employing the Itô formula (see [27]) to $V(t,x)=x^{\top }X\left(t\right)x$ and to the stochastic process $x_{\mathbf{u}}\left(t\right)$ which is satisfying (1a) on the interval of the form $[a_p,b_p]\subset [kh,(k+1)h]$ if $\kappa \left(t_0\right)\le k\le \kappa \left(T\right)-2$ or $[a_p,b_p]\subset [(\kappa \left(T\right)-1)h,T]$ and letting $a+p\to kh$, $b_p\to (k+1)h,$ or $a_p\to (\kappa \left(T\right)-1)h$ and $b_p\to T,$ we obtain via (8a) that

$$\begin{array}{cc}\hfill \underset{kh}{\overset{(k+1)h}{\int }}\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(t\right)M{x}_{\mathbf{u}}\left(t\right)\right]dt& =\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(k{h}^{+}\right)X\left(kh\right)x_{\mathbf{u}}\left(k{h}^{+}\right)\right]\hfill \\ \hfill & -\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left((k+1)h\right)X\left((k+1)h^{-}\right)x_{\mathbf{u}}\left((k+1)h\right)\right],\hfill \end{array}$$

(41)

for $\kappa \left(t_0\right)\le k\le \kappa \left(T\right)-2$ and

$$\begin{array}{cc}\hfill \underset{\left(\kappa \right(T)-1)h}{\overset{T}{\int }}\mathbb{E}[x_{\mathbf{u}}^{\top }\left(t\right)M& x_{\mathbf{u}}\left(t\right)]dt=\mathbb{E}[x_{\mathbf{u}}^{\top }\left((\kappa \left(T\right)-1)h^{+}\right)X\left((\kappa \left(T\right)-1)h\right)\hfill \\ \hfill & ·x_{\mathbf{u}}\left((\kappa \left(T\right)-1)h^{+}\right)]-\mathbb{E}[x_{\mathbf{u}}^{\top }(\left(T\right)X\left(T^{-}\right)x_{\mathbf{u}}\left(\left(T\right)\right].\hfill \end{array}$$

(42)

From (1b), we obtain that

$$\begin{array}{cc}\hfill & \mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(k{h}^{+}\right)X\left(kh\right)x_{\mathbf{u}}\left(k{h}^{+}\right)\right]\hfill \\ \hfill & =\sum _{j=0}^1\mathbb{E}\left[{\left({\mathcal{A}}_{j}X\left(kh\right){\mathcal{B}}_{j}u\left(k\right)\right)}^{\top }X\left(kh\right)({\mathcal{A}}_{j}X\left(kh\right)+{\mathcal{B}}_{j}u\left(k\right))\right].\hfill \end{array}$$

After some computations involving (8b) and the technique of square computations, we obtain

$$\begin{array}{cc}\hfill & \mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(k{h}^{+}\right)X\left(kh\right)x_{\mathbf{u}}\left(k{h}^{+}\right)\right]=\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(kh\right)X\left(k{h}^{-}\right)x_{\mathbf{u}}\left(kh\right)\right]\hfill \\ \hfill & +\mathbb{E}\left[{(u\left(k\right)-F^X\left(kh\right)x_{\mathbf{u}}\left(kh\right))}^{\top }\mathbb{R}\left(X\left(kh\right)\right)(u\left(k\right)-F^X\left(kh\right)x_{\mathbf{u}}\left(kh\right))\right]\hfill \\ \hfill & -\mathbb{E}[x_{\mathbf{u}}^{\top }\left(kh\right)M_d{x}_{\mathbf{u}}\left(kh\right)+2x_{\mathbf{u}}^{\top }\left(kh\right)L_{d}u\left(k\right)+u^{\top }\left(k\right)R_{d}u\left(k\right)],\hfill \end{array}$$

(43)

$\kappa \left(t_0\right)\le k\le \kappa \left(T\right)-1.$ Substituting (43) in (41) and (42) and summing from $k=\kappa \left(t_0\right)$ to $k=\kappa \left(T\right)-1,$ we obtain

$$\begin{array}{cc}\hfill & \underset{\kappa \left(t_0\right)}{\overset{T}{\int }}\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(t\right)M{x}_{\mathbf{u}}\left(t\right)\right]dt\hfill \\ \hfill & +\sum _{k=\kappa \left(t_0\right)}^{\kappa \left(T\right)-1}\mathbb{E}[x_{\mathbf{u}}^{\top }\left(kh\right)M_d{x}_{\mathbf{u}}\left(kh\right)+2x_{\mathbf{u}}\left(kh\right)L_{d}u\left(k\right)u^{\top }\left(k\right)R_{d}u\left(k\right)]\hfill \\ \hfill & =\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(\kappa \left(t_0\right)h\right)X\left(\kappa \left(t_0\right)h^{-}\right)x_{\mathbf{u}}\left(\kappa \left(t_0\right)h\right)\right]-\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(T\right)X\left(T^{-}\right)x_{\mathbf{u}}\left(T\right)\right]\hfill \\ \hfill & +\sum _{k=\kappa \left(t_0\right)}^{\kappa \left(T\right)-1}\mathbb{E}[{(u\left(k\right)-F^X\left(kh\right)x_{\mathbf{u}}\left(kh\right))}^{\top }\mathbb{R}\left(X\left(kh\right)\right)(u\left(k\right)-F^X\left(kh\right)x_{\mathbf{u}}\left(kh\right)].\hfill \end{array}$$

(44)

If $t_0=\kappa \left(t_0\right)h$, then (44) is just (38). In this case, the proof would be complete.

If $(\kappa \left(t_0\right)-1)h<t_0<\kappa \left(t_0\right)h$, we use Itô’s formula to $V(t,x)=x^{\top }X\left(t\right)x$ and to the stochastic process $x_{\mathbf{u}}\left(t\right)$, $t\in [t_0,\kappa \left(t_0\right)h]$, and we obtain via (8a) that

$$\begin{array}{c}\hfill \underset{t_0}{\overset{\kappa \left(t_0\right)h}{\int }}\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(t\right)M{x}_{\mathbf{u}}\left(t\right)\right]dt=x_0^{\top }X\left(t_0\right)x_0-\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(\kappa \left(t_0\right)h\right)X\left(\kappa \left(t_0\right)h^{-}\right)x_{\mathbf{u}}\left(\kappa \left(t_0\right)h\right)\right].\end{array}$$

Adding this equality to (44), we obtain (38). Thus, the proof ends. □

The next result provides a sufficient condition for the attainability of the optimal control problem under consideration.

Proposition 2.

Assume that the BJMLDE with a Riccati-type jumping operator (8) has a bounded and stabilizing solution $X_s(·):{\mathbb{R}}_{+}\to {\mathcal{S}}_n$ satisfying the sign condition (35).

Let us explore the control law $\tilde{\mathbf{u}}={\left\{\tilde{u}\left(k\right)\right\}}_{k\ge \kappa \left(t_0\right)}$ defined by

$$\tilde{u}\left(k\right):=F_s\left(kh\right)\tilde{u}\left(kh\right),k\ge \kappa \left(t_0\right),$$

(45)

where $F_s\left(kh\right)$ are the gain matrices associated to the stabilizing solution $X_s(·)$ via (12), and $\tilde{x}\left(kh\right)$, $kh\ge t_0$ are the value of the solution of the IVP (11).

Under these conditions, the control defined by (45) lies in ${\mathcal{U}}_{ad}(t_0,x_0)$ and satisfies the optimality condition (5). The minimal value of the cost function (4) is:

$$J(t_0,\infty ,x_0;\tilde{\mathbf{u}})=x_0^{\top }{X}_s\left(t_0^{-}\right)x_0,$$

with the remark that $X_s\left(t_0^{-}\right)=X_s\left(t_0\right)$ whenever $t_0<\kappa \left(t_0\right)h.$

Proof. 

The fact that the control (45) satisfies the constraints (2) follows immediately from Definition 3 of the stabilizing solution. Since $X_s(·)$ is a bounded function, then there exists $\tilde{c}>0$ with the property that

$$|X_s\left(T\right)|\le \tilde{c},$$

and

$$|X_s\left(T^{-}\right)|\le \tilde{c},$$

for all $T\ge 0.$ Employing (2b), we may infer that

$$\underset{T\to \infty }{lim}\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(T\right)X_s\left(T^{-}\right)x_{\mathbf{u}}\left(T\right)\right]=0,$$

(46)

for all $\mathbf{u}\in {\mathcal{U}}_{ad}(t_0,x_0)$ for any $(t_0,x_0)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n.$

Writing (38) for $X(·)$ replaced by $X_s(·)$, we obtain via (2a) and (46) that

$$\begin{array}{cc}\hfill & J(t_0,\infty ,x_0;\mathbf{u})=x_0^{\top }{X}_s\left(t_0^{-}\right)x_0\hfill \\ \hfill & +\sum _{k=\kappa \left(t_0\right)}^{\infty }\mathbb{E}\left[{(u\left(k\right)-F_s\left(kh\right)x_{\mathbf{u}}\left(kh\right))}^{\top }\mathbb{R}\left(X_s\left(kh\right)\right)(u\left(k\right)-F_s\left(kh\right)x_{\mathbf{u}}\left(kh\right))\right],\hfill \end{array}$$

(47)

for all $\mathbf{u}\in {\mathcal{U}}_{ad}(t_0,x_0)$, $(t_0,x_0)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n.$ For the unicity of the solution of the IVP (1) directed by a control $\mathbf{u}\in {\mathcal{U}}_{ad}(t_0,x_0)$, we deduce that in the case of the control (45), we have

$$x_{\tilde{\mathbf{u}}}\left(t\right)=\tilde{x}\left(t\right),$$

for all $t\ge t_0$, $\tilde{x}(·)$ being the solution of (11) which satisfies

$$\tilde{x}\left(t_0\right)=x_0.$$

This allows us to conclude that, in the case of the control (45), the equality (47) gives

$$J(t_0,\infty ,x_0;\tilde{\mathbf{u}})=x_0^{\top }{X}_s\left(t_0^{-}\right)x_0.$$

(48)

From (35), (47) and (48), we conclude that the control (45) satisfies the optimality condition (5), and (48) provides the maximal value of the cost function (4) over the set of the admissible controls. Thus, the proof ends. □

The next theorem provides a sufficient condition for the well posedness of the optimal control problem under consideration.

Theorem 3.

Assume the following:

(a)

The ICLSS (1) is mean square stabilizable by the linear state feedback;

(b)

The set ${\mathsf{\Gamma }}^{\mathsf{\Sigma }}$ of the functions $Z(·)$ which satisfy the constraints (9) is not empty.

Under these conditions, the optimal control problem described by the performance criterion (4), the controlled system (1), and the set of the admissible controls ${\mathcal{U}}_{ad}(t_0,x_0)$ is well posed. Furthermore,

$$\mathbf{v}(t_0,x_0)=x_0^{\top }{X}_{max}\left(t_0^{-}\right)x_0,$$

(49)

for all $(t_0,x_0)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n$. $X_{max}(·)$ is the unique maximal and bounded solution of the BJMLDE (8) which satisfies the sign condition (32). In (49), $X_{max}\left(t_0^{-}\right)$ coincides to $X_{max}\left(t_0\right)$ whenever $(\kappa \left(t_0\right)-1)h<t_0<\kappa \left(t_0\right)h.$

Proof. 

If the assumption $\left(a\right)$ holds, then the set ${\mathcal{U}}_{ad}(t_0,x_0)$ is not empty for any initial pair $(t_0,x_0).$ Further, Theorem 1 guarantees the existence of the maximal and bounded solution $X_{max}(·)$ which satisfies the sign condition (32), and it is a periodic function with period $h.$ Applying Lemma 2, taking $X_{max}(·)$ instead of $X(·)$, we obtain

$$\begin{array}{cc}\hfill & \underset{t_0}{\overset{T}{\int }}\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(t\right)M{x}_{\mathbf{u}}\left(t\right)\right]dt\hfill \\ \hfill & +\sum _{k=\kappa \left(t_0\right)}^{\kappa \left(T\right)-1}\mathbb{E}[x_{\mathbf{u}}^{\top }\left(kh\right)M_d{x}_{\mathbf{u}}\left(kh\right)+2x_{\mathbf{u}}^{\top }\left(kh\right)L_{d}u\left(k\right)+u^{\top }\left(k\right)R_{d}u\left(k\right)]\hfill \\ \hfill & =x_0^{\top }{X}_{max}\left(t_0^{-}\right)x_0-\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(T\right)X_{max}\left(T^{-}\right)x_{\mathbf{u}}\left(T\right)\right]\hfill \\ \hfill & +\sum _{k=\kappa \left(t_0\right)}^{\kappa \left(T\right)-1}\mathbb{E}[{(u\left(k\right)-F_{max}\left(kh\right)x_{\mathbf{u}}\left(kh\right))}^{\top }\mathbb{R}\left(X_{max}\left(kh\right)\right)(u\left(k\right)-F_{max}\left(kh\right)x_{\mathbf{u}}\left(kh\right)],\hfill \end{array}$$

(50)

for all $T>\kappa \left(t_0\right)h$ and any $\mathbf{u}\in {\mathcal{U}}_{ad}(t_0,x_0)$ for any $(t_0,x_0)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n.$ Here, $F_{max}\left(kh\right)$ is computed as in (40) with $X_{max}(·)$ instead of $X(·).$ Since $X_{max}(·)$ is a bounded function, then there exists $\widehat{c}>0$ such that

$$|X_{max}\left(T\right)|\le \widehat{c},$$

and

$$|X_{max}\left(T^{-}\right)|\le \widehat{c},$$

for all $T\ge 0.$ Based on (2b), we may infer that

$$\underset{T\to \infty }{lim}\mathbb{E}\left[x_{\mathbf{u}}^{\top }\left(T\right)X_{max}\left(T^{-}\right)x_{\mathbf{u}}\left(T\right)\right]=0,$$

(51)

for all $\mathbf{u}\in {\mathcal{U}}_{ad}(t_0,x_0)$ for an arbitrary initial pair $(t_0,x_0)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n.$ Further, taking the limit for $T\to \infty$ in (50), we obtain via (2a) and (51) that

$$\begin{array}{cc}\hfill & J(t_0,\infty ,x_0;\mathbf{u})=x_0^{\top }{X}_{max}\left(t_0^{-}\right)x_0\hfill \\ \hfill & +\sum _{k=\kappa \left(t_0\right)}^{\infty }\mathbb{E}\left[{(u\left(k\right)-F_{max}\left(kh\right)x_{\mathbf{u}}\left(kh\right))}^{\top }\mathbb{R}\left(X_{max}\left(kh\right)\right)(u\left(k\right)-F_{max}\left(kh\right)x_{\mathbf{u}}\left(kh\right))\right],\hfill \end{array}$$

(52)

for all $u={\left\{u\left(k\right)\right\}}_{k\ge \kappa \left(t_0\right)}\in {\mathcal{U}}_{ad}(t_0,x_0)$ for arbitrary $(t_0,x_0)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n.$ Bearing in mind (32), we may deduce from (52) that

$$J(t_0,\infty ,x_0;\mathbf{u})\ge x_0^{\top }{X}_{max}\left(t_0^{-}\right)x_0,$$

(53)

for all $\mathbf{u}\in {\mathcal{U}}_{ad}(t_0,x_0).$ From (53) and (6), we may conclude that

$$\mathbf{v}(t_0,x_0)\ge x_0^{\top }{X}_{max}\left(t_0^{-}\right)x_0,$$

(54)

for all $(t_0,x_0)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n.$ Thus, from Definition 1 $\left(a\right)$, it follows that the optimal control problem under consideration is well posed.

To end the proof, it remains to show that in (54), we have equality. To accomplish this, we consider a sequence ${\left\{{\delta }_q\right\}}_{g\ge 1}$ with the following properties:

$${\delta }_q\ge {\delta }_{q+1}>0,$$

and

$$\underset{q\to \infty }{lim}{\delta }_q=0.$$

We consider the following perturbation of (4):

$$J_{{\delta }_q}(t_0,\infty ,x_0;\mathbf{u})=J(t_0,\infty ,x_0;\mathbf{u})+{\delta }_q\sum _{k=\kappa \left(t_0\right)}^{\infty }\mathbb{E}\left[\right|x_{\mathbf{u}}\left(kh\right){|}^2],$$

(55)

for all $\mathbf{u}={\left\{u\left(k\right)\right\}}_{k\ge \kappa \left(t_0\right)}\in {\mathcal{U}}_{ad}(t_0,x_0).$

The BJMLDE of type (8) associated to the pair formed by the ICLSS (1) and the quadratic functional (55) is

$$\begin{array}{cc}\hfill & \dot{X}\left(t\right)+{\mathcal{L}}^{*}\left[X\left(t\right)\right]+M=0,kh\le t<(k+1)h,\hfill \end{array}$$

(56a)

$$\begin{array}{cc}\hfill & X\left(k{h}^{-}\right)={\mathcal{R}}_d\left(X\left(kh\right)\right)+{\delta }_q{I}_n,k\in {\mathbb{Z}}_{+},\hfill \end{array}$$

(56b)

where ${\mathcal{L}}^{*}(·)$ is introduced by (16), while ${\mathcal{R}}_d\left(X\left(kh\right)\right)$ is described by the right-hand side of (8b).

The accompanying GDTRE of the form (19) associated to (56) is

$$\begin{array}{cc}\hfill Y\left(k\right)=& {\mathsf{\Pi }}_1\left[Y(k+1)\right]+\tilde{M}+{\delta }_q{I}_n-({\mathsf{\Pi }}_2\left[Y(k+1)\right]+\tilde{L})\hfill \\ \hfill & ·{({\mathsf{\Pi }}_3\left[Y(k+1)\right]+\tilde{R})}^{-1}{({\mathsf{\Pi }}_2\left[Y(k+1)\right]+\tilde{L})}^{\top },\hfill \end{array}$$

(57)

where ${\mathsf{\Pi }}_k[·]$, $j=1,2,3$ are defined as in (20) and $\tilde{M}$, $\tilde{L}$, $\tilde{R}$ are those computed in (21). Hence, the GDTRE (57) is described by the pair

$${\sigma }_d^{{\delta }_q}=(\mathsf{\Pi },{\mathcal{Q}}_{{\delta }_q}),$$

where $\mathsf{\Pi }:{\mathcal{S}}_n\to {\mathcal{S}}_{n+m}$ is the operator defined in (22) and

$${\mathcal{Q}}_{{\delta }_q}=\left(\begin{array}{cc}\tilde{M}+{\delta }_q{I}_n& \tilde{L}\\ {\tilde{L}}^{\top }& \tilde{R}\end{array}\right),q\ge 1.$$

(58)

Let ${\tilde{\mathsf{\Gamma }}}^{{\mathsf{\Sigma }}_d^{{\delta }_q}}$ be the set of the sequences of symmetric matrices $\mathbf{Z}={\left\{Z\left(k\right)\right\}}_{k\ge 0}$ which satisfy

$${\mathcal{D}}^{{\delta }_q}\left(\mathbf{Z}\right)\left(k\right)\ge \gamma I_{n+m}>0,$$

(59)

for all $k\ge 0,$ where

$${\mathcal{D}}^{{\delta }_q}\left(\mathbf{Z}\right)\left(k\right):=\left(\begin{array}{cc}-Z\left(k\right)& 0\\ 0& 0\end{array}\right)+\mathsf{\Pi }\left[Z(k+1)\right]+{\mathcal{Q}}^{{\delta }_q}.$$

(60)

From Lemma 1 $\left(a\right)$ together with (23), (59) and (60) we may infer that if the set ${\mathsf{\Gamma }}^{\mathsf{\Sigma }}$ is not empty, then the set ${\tilde{\mathsf{\Gamma }}}^{{\mathsf{\Sigma }}_d^{{\delta }_q}}$ associated to the BJMLDE (56) is not empty. Invoking the implication $\left(i\right)\Rightarrow \left(ii\right)$ from Theorem 2, we may conclude that the BJMLDE (56) has a stabilizing solution $X_s^{{\delta }_q}(·)$ which satisfies the sign conditions

$$R_d+\sum _{j=0}^1{\mathcal{B}}_j^{\top }{X}_s^{{\delta }_q}\left(kh\right){\mathcal{B}}_j\ge {\gamma }_1{I}_m>0,$$

for all $k\in {\mathbb{Z}}_{+}.$ One proves via Theorem 5.4 from [31] that

$$X_s^{{\delta }_q}\left(t\right)\ge X_s^{{\delta }_{q+1}}\left(t\right)\ge X_{max}\left(t\right),$$

for all $q\ge 1$ and

$$\underset{q\to \infty }{lim}{X}_s^{{\delta }_q}\left(t\right)=X_{max}\left(t\right),t\in {\mathbb{R}}_{+}.$$

(61)

Let $F_s^{{\delta }_q}\left(t\right)$ be the stabilizing gain matrix associated to the solution $X_s^{{\delta }_q}(·)$ as in (12). We consider the control law

$${\tilde{u}}_{{\delta }_q}\left(k\right):=F_s^{{\delta }_q}\left(kh\right){\tilde{x}}_{{\delta }_q}\left(kh\right),$$

(62)

${\tilde{x}}_{{\delta }_q}(·)$ being the solution of the IVP obtained when the control (62) is plugged in (1). Applying Proposition 2 in the case of the optimal control problem described by the cost function (55), we obtain via (52) and (55) that

$$\begin{array}{cc}\hfill & x_0^{\top }{X}_{max}\left(t_0^{-}\right)x_0\le \mathbf{v}(t_0,x_0)\le J(t_0,\infty ,x_0;{\tilde{\mathbf{u}}}_{{\delta }_q})\hfill \\ \hfill & \le J_{{\delta }_q}(t_0,\infty ,x_0;{\tilde{\mathbf{u}}}_{{\delta }_q})=x_0^{\top }{X}_s^{{\delta }_q}\left(t_0^{-}\right)x_0,\hfill \end{array}$$

(63)

for all $q\ge 1.$ Taking the limit for $q\ge \infty$ in (63), we obtain via (61) that

$$\mathbf{v}(t_0,x_0)=x_0^{\top }{X}_{max}\left(t_0^{-}\right)x_0,$$

which complete the proof. □

The next result provides a necessary and sufficient condition for the attainability of the optimal control problem under consideration.

Theorem 4.

Under the assumptions of Theorem 3, the following are equivalent:

(i)

The optimal control problem described by the cost functional (4), the controlled system (1), and the set of the admissible controls ${\mathcal{U}}_{ad}(t_0,x_0)$ is attainable.

(ii)

The unique bounded and maximal solution of the BJMLDE (8) is also a stabilizing solution of the BJMLDE (8).

Proof. 

The implication $\left(ii\right)\Rightarrow \left(i\right)$ follows directly from Proposition 2. It remains to prove the implication $\left(i\right)\Rightarrow \left(ii\right).$ If the assumptions from Theorem 3 hold, then the BJMLDE (8) has a maximal and bounded solution $X_s(·)$ which satisfies the sign condition (32).

Let $\tilde{\mathbf{u}}\in {\mathcal{U}}_{ad}(t_0,x_0)$ satisfy (7). Then, from (7), (49), and (52), we deduce that

$$\begin{array}{c}\hfill \sum _{k=\kappa \left(t_0\right)}^{\infty }\mathbb{E}\left[{(\tilde{u}\left(k\right)-F_{max}\left(kh\right)x_{\tilde{\mathbf{u}}}\left(kh\right))}^{\top }\mathbb{R}\left(X_{max}\left(kh\right)\right)(\tilde{u}\left(k\right)-F_{max}\left(kh\right)x_{\tilde{\mathbf{u}}}\left(kh\right))\right]=0.\end{array}$$

(64)

Bearing in mind (32), we may infer that (64) holds if and only if

$$\tilde{u}\left(k\right)=F_{max}\left(kh\right)x_{\tilde{\mathbf{u}}}\left(kh\right)a.s.,$$

(65)

for all $k\ge \kappa \left(t_0\right).$ Moreover,

$$\underset{t\to \infty }{lim}\mathbb{E}\left[\right|x_{\tilde{\mathbf{u}}}\left(t\right){|}^2]=0,$$

(66)

because $\tilde{\mathbf{u}}\in {\mathcal{U}}_{ad}(t_0,x_0).$ Taking into account that $X_{max}(·)$ is a periodic function of period $h,$ it follows that

$$F_{max}\left(kh\right)=F_{max}\left(0\right),$$

for all $k\in {\mathbb{Z}}_{+}.$ Hence, (65) becomes

$$\tilde{u}\left(k\right)=F_{max}\left(0\right)x_{\tilde{\mathbf{u}}}\left(kh\right)a.s.,$$

(67)

for all $k\in {\mathbb{Z}}_{+}.$ Because of the unicity of the solution of an IVP, we deduce that $x_{\tilde{\mathbf{u}}}(·)$ coincides to $x(·;t_0,x_0)$, where $x(·;t_0,x_0)$ is the solution of the jump linear stochastic system (JLSS):

$$\begin{array}{cc}\hfill & dx\left(t\right)=A_{0}x\left(t\right)dt+A_{1}x\left(t\right)dw\left(t\right),kh<t\le (k+1)h,\hfill \end{array}$$

(68a)

$$\begin{array}{cc}\hfill & x\left(k{h}^{+}\right)=({\mathcal{A}}_0+{\mathcal{B}}_0{F}_{max}\left(0\right))x\left(kh\right)+w_d\left(k\right)({\mathcal{A}}_1+{\mathcal{B}}_1{F}_{max}\left(0\right))x\left(kh\right),\hfill \end{array}$$

(68b)

$$\begin{array}{cc}\hfill & x\left(t_0\right)=x_0,\hfill \end{array}$$

(68c)

obtained when (67) is substituted in (1). Thus, (66) yields

$$\underset{t\to \infty }{lim}\mathbb{E}\left[x(t;t_0,x_0){|}^2\right]=0,$$

for all $(t_0,x_0)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n.$ This means that the JLSS (68) is asymptotic stable in mean square (ASMS).

Reasoning as in the proofs of Propositions 2 and Proposition 3 from [28], one may prove that the JLSS (68) is ASMS if and only if the companion DTLE

$$Z(k+1)={\mathcal{L}}_d\left[Z\left(k\right)\right],$$

(69)

is asymptotic stable, where

$${\mathcal{L}}_d\left[Z\right]:=\sum _{j=0}^1{e}^{\mathcal{L}h}\left[({\mathcal{A}}_j+{\mathcal{B}}_j{F}_{max}\left(0\right))Z{({\mathcal{A}}_j+{\mathcal{B}}_j{F}_{max}\left(0\right))}^{\top }\right].$$

Since the DTLE (69) is a linear time-invariant equation, it follows that it is asymptotic stable if and only if it is exponentially stable. According to the Remark 3 from [28], we deduce that (69) is exponentially stable if and only if the JLSS (68) is ESMS. Thus, the proof ends. □

5. Conclusions

This paper has contributed to the further development of the LQ control problem for stochastic systems controlled by impulses. A sufficient condition for the well posedness of the LQ control problem under consideration is expressed in terms of the existence of a maximal solution satisfying a suitable sign condition of the BJMLDE with a Riccati-type jumping operator constructed based on the matrix coefficients of the controlled system and the weight matrices of the performance criterion. When the condition for the well posedness is fulfilled, the LQ control problem is attainable if and only if the maximal solution of the BJMLDE with a Riccati-type jumping operator is also a stabilizing solution.

According to the terminology introduced in [32] for the deterministic framework, the problem analyzed in the present work is a generalization to the stochastic case of a fixed-endpoint LQ control problem. A remaining challenge for future research is the analysis of the well-posedness and attainability of a free-endpoint LQ control problem for a stochastic system controlled by impulses.

Another challenge for future research could be the investigation of the well-posedness and attainability of an LQ optimal control problem for a stochastic system controlled by impulses in which the length of the intervals between two impulse instances is not constant, it being time varying or just driven by a stochastic process.

The derivation of some efficient numerical procedures for the computation of the maximal solution and the stabilizing solution (if any) of the BJMLDE with a Riccati-type jumping operator associated to the control problems under consideration is necessary in order to be able to illustrate the applicability of the theoretical developments both in the constant dwell time case as well as in the time-varying dwell time case.