Decentralized Control for Interrelated Systems with Asymmetric Information Architecture

Abstract

This paper focuses on finite-horizon optimum state feedback control problems for interconnected systems of two players involved with asymmetric one-step delay information. For the finite horizon optimum decentralized control problem, a crucial and adequate condition is derived by using Pontryagin’s maximum principle. Under this framework, player 1 transmits its state and control input data with a one-step delay to the controller of player 2, while player 1’s controller does not have access to the real-time or delayed states and control inputs of player 2, resulting in an asymmetric information structure characterized by a one-step delay Then, the solutions to the forward and backward stochastic difference equations are derived. A target tracking system is given in numerical examples to verify the proposed algorithm.

1. Introduction

The traditional stochastic optimal control problem was first formulated by [1] in the 1960s, with subsequent in-depth investigations conducted by [2,3,4,5,6,7,8,9,10,11,12,13,14]. Specifically, Ref. [7] proposed an outlined Riccati equation to establish the solvability of the stochastic optimal control problem. For delay-free stochastic systems, Ref. [11] addressed the infinite-horizon optimal control problem where the status and control weighting matrices are indefinite. Regarding stochastic linear systems with time delays, Ref. [14] solved the optimal control and equalization problem by proposing a novel Riccati-ZXL difference equation. In summary, the traditional stochastic optimal control problem has been well resolved.

With the growing demand for practical employments—such as networked control systems, formation flight, and target tracking—decentralized control has attracted considerable attention [15,16,17,18,19,20]. A key distinction between traditional stochastic optimal control and decentralized control lies in the feedback information (i.e., state or observation information) available to different controllers: while the feedback information is uniform for all controllers in traditional stochastic optimal control, it varies across controllers in decentralized control, leading to asymmetric information. Generally, for traditional stochastic optimal control, linear controllers are globally optimal, with control gains computable via Riccati equations and the feedback form expressible as a linear function of the conditional expectation of the status [10]. In contrast, for decentralized control, nonlinear strategies may achieve better performance than linear ones [17], and the optimal decentralized control problem may not be convex [18]. Consequently, the methodologies developed for traditional stochastic optimal control cannot be directly applied to decentralized control scenarios.

While the optimal decentralized control problem is intractable, many studies have been devoted to it; see [18,21,22,23,24,25,26,27] and the references therein. Ref. [22] showed that the optimal control strategies were linear by proposing the partial nestedness structure. Ref. [23] introduced a general decentralized model named partial historical sharing structure. Under this structure, assuming that controllers satisfied some linear forms, Ref. [24] presented the optimal controllers in virtue of the common information. Refs. [25,26,27,28] extended this work to networked control systems with special structure. By using dynamic programming and common information, Refs. [26,28] solved the optimal control and equalization problem for systems with a special unreliable communication channel. Under the same special structure, Ref. [27] presented the optimal controllers and stabilization condition with Pontryagin’s maximum principle and algebraic Riccati equations. Recently, Ref. [29] extended the above special structure to the general case of a standard observation equation and obtained linear optimal controllers.

However, the decentralized control problem with time delay is not considered in the above references. Meanwhile, in the context of reliable control under component failures, Ref. [30] proposed a guaranteed-cost LQ control based on algebraic Riccati equations for discrete-time systems with actuator failures, ensuring the system’s asymptotic stability and bounded quadratic cost. Ref. [31] developed a reliable LQG control method that addresses sensor failures via coupled Riccati equations and LMIs, guaranteeing robust stability for systems with partial degradation or outage. These works inspire our research to resolve information asymmetry induced by asymmetric delays while enhancing tolerance to model uncertainties. With the identical combination of delay and sparsity constraints, Ref. [32] presented solutions to the decentralized state feedback control case. Ref. [33] studied applications of heavy-duty vehicle platooning and solved the optimal control problem for chain structures with delayed information. For the finite-horizon case, Ref. [34] presented an optimal state feedback controller for large-scale delayed systems. By requiring plant dynamics to exhibit lower block triangular characteristics, Ref. [35] put forward a decentralized system model with two players that has an asymmetric one-step delayed information pattern. Employing the dynamic programming approach, Ref. [35] obtained the solution of optimal control strategies for the finite horizon. Ref. [5] studied the asymmetric information control involved in different delayed state information, and optimal controllers were derived for the finite-horizon case. Ref. [36] addressed control problem formulations for discrete-time stochastic systems equipped with multiple input paths and input time delays. However, merely one state is involved in [36]. To the best of our knowledge, for general plant dynamics, the optimal control of the asymmetric delayed decentralized problem has not been solved yet.

In this paper, we investigate the finite-horizon optimal state feedback control problem for two-player decentralized interconnected systems with asymmetric one-step delay information—a critical gap in existing research that has not been fully addressed for general coupled dynamics. Unlike previous studies that impose special structural constraints on plant dynamics or assume decoupled control inputs, our work completely removes structural restrictions on system dynamics, enabling the handling of fully coupled state and control input relationships between the two players, which is more aligned with practical interconnected system scenarios such as multi-agent coordination. The main contributions are summarized as follows:

(1) Applying Pontryagin’s maximum principle, we propose an innovative framework specifically designed to derive coupled costate–state backward stochastic difference equations and inter-player equilibrium equations. This framework can explicitly capture the impact of asymmetric delay on information asymmetry, overcoming the limitation of traditional methods that struggle to quantify the correlation between delay and information coupling, and laying a solid theoretical foundation for the subsequent derivation of optimal control strategies.

(2) Building on these equations, we further derive the analytical solution to the forward and backward stochastic difference equations (FBSDEs) and rigorously establish finite-horizon optimal decentralized control strategies through two sets of Riccati equations. We clarify the mild condition for the unique solvability of the strategy, namely the invertibility of relevant matrices, which ensures the theoretical rigor of the method and its feasibility in engineering applications, thereby addressing the issue regarding whether optimal control solutions exist uniquely for coupled systems with asymmetric delays.

(3) A practical unmanned aerial vehicle (UAV) target tracking system is adopted to serve as a case study for confirming the suggested method, fully demonstrating its effectiveness in real-world applications. Experimental results show that the method exhibits stable performance in terms of tracking accuracy and energy efficiency, capable of meeting the dual requirements of practical engineering for control effects and resource consumption, and providing an implementable solution for the optimal control of interrelated systems with asymmetric delays.

Subsequent parts of the paper are arranged in the manner described below. Section 2 presents a study on the finite-horizon optimum decentralized control case. In Section 3, numerical examples are shown on the target tracking system. Relevant proofs are shown in Appendix A in detail.

2. Optimum Control

2.1. Problem Formulation

We study the linear system with a coupled two-player structure:

$$\begin{array}{cc}\hfill \left[\begin{array}{c}{\psi }_1(\varrho +1)\\ {\psi }_2(\varrho +1)\end{array}\right] =& \left[\begin{array}{cc}{D}_{11}& D_{12}\\ D_{21}& D_{22}\end{array}\right]\left[\begin{array}{c}{\psi }_1\left(\varrho \right)\\ {\psi }_2\left(\varrho \right)\end{array}\right]+\left[\begin{array}{cc}{G}_{11}& G_{12}\\ G_{21}& G_{22}\end{array}\right]\left[\begin{array}{c}{u}_1\left(\varrho \right)\\ u_2\left(\varrho \right)\end{array}\right]\hfill \\ \hfill & + \left[\begin{array}{c}{\lambda }_1\left(\varrho \right)\\ {\lambda }_2\left(\varrho \right)\end{array}\right],\hfill \end{array}$$

(1)

where ${\psi }_1\left(\varrho \right)$, $u_1\left(\varrho \right)$ and ${\lambda }_1\left(\varrho \right)$ are the status, control input and system noise of player 1, respectively. ${\psi }_2\left(\varrho \right)$, $u_2\left(\varrho \right)$ and ${\lambda }_2\left(\varrho \right)$ are the status, control input and system noise of player 2, respectively. ${\psi }_1\left(0\right)$, ${\psi }_2\left(0\right)$, ${\lambda }_1\left(\varrho \right)$ and ${\lambda }_2\left(\varrho \right)$ are Gaussian white noises and are featured to not influence each other, with a mean of (${\overline{\psi }}_1\left(0\right),{\overline{\psi }}_2\left(0\right),0,0$) and a covariance of (${\sigma }_1,{\sigma }_2,Q_{{\lambda }_1},Q_{{\lambda }_2}$), respectively. The objective for both players is to minimize the following quadratic cost function related to the system (1):

$$\begin{array}{cc}\hfill \Omega \left(\varkappa \right) =& E\{\sum _{\varrho =0}^{\varkappa }\sum _{i=1}^2[{\psi }_i{\left(\varrho \right)}^{\prime }{Q}_i^t{\psi }_i\left(\varrho \right)+u_i{\left(\varrho \right)}^{\prime }{R}_i^t{u}_i\left(\varrho \right)\hfill \\ \hfill & + {\psi }_i{(\varkappa +1)}^{\prime }{H}_i(\varkappa +1){\psi }_i(\varkappa +1)\left]\right\},\hfill \end{array}$$

(2)

where $Q_i^t$, $R_i^t$ and $H_i(\varkappa +1)$ are positive semi-definite matrices for $i=1,2$.

At each time instant $\varrho$, the control actions for the two players obey the system structure information, i.e., the available information for $u_1\left(\varrho \right)$ are $\{{\psi }_1\left(\varrho \right),{\psi }_1(\varrho -1),\dots ,{\psi }_1\left(0\right),u_1(\varrho -1),\dots ,u_1\left(0\right)\}$. Accordingly, $u_1\left(\varrho \right)$ is ${\mathcal{F}}_1\left(\varrho \right)$-measurable. The accessible information for $u_2\left(\varrho \right)$ is $\{{\psi }_1(\varrho -1),\dots ,{\psi }_1\left(0\right),u_1(\varrho -1),\dots ,u_1\left(0\right),{\psi }_2\left(\varrho \right),{\psi }_2(\varrho -1),\dots ,{\psi }_2\left(0\right),u_2(\varrho -1),\dots ,$$u_2\left(0\right)\}$. Accordingly, $u_2\left(\varrho \right)$ is ${\mathcal{F}}_2\left(\varrho \right)$-measurable. Obviously, the common information for the two players are ${\mathcal{F}}_c\left(\varrho \right)=\{{\psi }_1(\varrho -1),\dots ,{\psi }_1\left(0\right),u_1(\varrho -1),\dots ,u_1\left(0\right)\}$. The whole information for the system is denoted by $\mathcal{F}\left(\varrho \right)=\{{\mathcal{F}}_1\left(\varrho \right),{\mathcal{F}}_2\left(\varrho \right)\}$.

To make the context much clearer, (1) and (2) are rewritten as

$$\begin{array}{c}\psi (\varrho +1) =D\psi \left(\varrho \right)+{\overline{G}}_1{u}_1\left(\varrho \right)+{\overline{G}}_2{u}_2\left(\varrho \right)+e\left(\varrho \right),\end{array}$$

(3)

$$\begin{array}{cc}\hfill \Omega \left(\varkappa \right) =& E\{\sum _{\varrho =0}^{\varkappa }\sum _{i=1}^2{[\psi \left(\varrho \right)}^{\prime }{Q}^t\psi \left(\varrho \right)+u_i{\left(\varrho \right)}^{\prime }{R}_i^t{u}_i\left(\varrho \right)\hfill \\ \hfill & + \psi {(\varkappa +1)}^{\prime }H(\varkappa +1)\psi (\varkappa +1)\left]\right\},\hfill \end{array}$$

(4)

where $\psi \left(\varrho \right)=\left[\begin{array}{c}{\psi }_1\left(\varrho \right)\\ {\psi }_2\left(\varrho \right)\end{array}\right]$, $D=\left[\begin{array}{cc}{D}_{11}& D_{12}\\ D_{21}& D_{22}\end{array}\right]$, ${\overline{G}}_1=\left[\begin{array}{c}{G}_{11}\\ G_{21}\end{array}\right]$, ${\overline{G}}_2=\left[\begin{array}{c}{G}_{12}\\ G_{22}\end{array}\right]$, $e\left(\varrho \right)=\left[\begin{array}{c}{\lambda }_1\left(\varrho \right)\\ {\lambda }_2\left(\varrho \right)\end{array}\right]$, $Q^t=\left[\begin{array}{cc}{Q}_1^t& 0\\ 0& Q_2^t\end{array}\right]$.

The issue to be dealt with in this section is presented as follows.

Problem 1.

Find the ${\mathcal{F}}_1\left(\varrho \right)$-measurable $u_1\left(\varrho \right)$ and ${\mathcal{F}}_2\left(\varrho \right)$-measurable $u_2\left(\varrho \right)$ to minimize (4) subject to (3).

2.2. Strategy to Solve Problem 1

Similarly to the procedure of [14], Pontryagin’s maximum principle is applied to (3) and (4) to derive the equations listed below:

$$\begin{array}{c}\eta (\varrho -1)=E\left[D^{\prime }\eta \left(\varrho \right)\right|\mathcal{F}\left(\varrho \right)]+Q^t\psi \left(\varrho \right),\end{array}$$

(5)

$$\begin{array}{c}0=E[{\overline{G}}_1^{\prime }\eta \left(\varrho \right)|{\mathcal{F}}_1\left(\varrho \right)]+R_1^t{u}_1\left(\varrho \right),\end{array}$$

(6)

$$\begin{array}{c}0=E[{\overline{G}}_2^{\prime }\eta \left(\varrho \right)|{\mathcal{F}}_2\left(\varrho \right)]+R_2^t{u}_2\left(\varrho \right),\end{array}$$

(7)

$$\begin{array}{c}\eta \left(\varkappa \right)=H(\varkappa +1)\psi (\varkappa +1),\end{array}$$

(8)

where $\eta \left(\varrho \right)$ represents the costate variable.

Noting the available information for the controllers $u_1\left(\varrho \right)$ and $u_2\left(\varrho \right)$, it can be found that $u_1\left(\varrho \right)$ is not ${\mathcal{F}}_2\left(\varrho \right)$-measurable and $u_2\left(\varrho \right)$ is not ${\mathcal{F}}_1\left(\varrho \right)$-measurable. With the above costate Equations (5)–(7), this leads to an unsolvable mathematical problem where five unknown variables exist in four Equations, (3), (5)–(7). To this end, in view of the common information of $u_1\left(\varrho \right)$ and $u_2\left(\varrho \right)$, we define

$$\begin{array}{c}{u}_1^c\left(\varrho \right)=E\left[u_1\left(\varrho \right)\right|{\mathcal{F}}_c\left(\varrho \right)],u_1^p\left(\varrho \right)=u_1\left(\varrho \right)-u_1^c\left(\varrho \right),\end{array}$$

(9)

$$\begin{array}{c}{u}_2^c\left(\varrho \right)=E\left[u_2\left(\varrho \right)\right|{\mathcal{F}}_c\left(\varrho \right)],u_2^p\left(\varrho \right)=u_2\left(\varrho \right)-u_2^c\left(\varrho \right).\end{array}$$

(10)

The following relationship can be obtained:

$$\begin{array}{c}\hfill E\left[u_1^p\left(\varrho \right)\right|{\mathcal{F}}_c\left(\varrho \right)]=0,E\left[u_2^p\left(\varrho \right)\right|{\mathcal{F}}_c\left(\varrho \right)]=0.\end{array}$$

(11)

In virtue of (9) and (10), we rewrite (3) and (4) as

$$\begin{array}{cc}\hfill & \psi (\varrho +1)=D\psi \left(\varrho \right)+\overline{G}{u}^c\left(\varrho \right)+{\overline{G}}_1{u}_1^p\left(\varrho \right)+{\overline{G}}_2{u}_2^p\left(\varrho \right)+e\left(\varrho \right),\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill & \Omega \left(\varkappa \right)=E\{\sum _{\varrho =0}^{\varkappa }\sum _{i=1}^2{[\psi \left(\varrho \right)}^{\prime }{Q}^t\psi \left(\varrho \right)+u^c{\left(\varrho \right)}^{\prime }{R}^t{u}^c\left(\varrho \right)\hfill \\ \hfill & + u_i^p{\left(\varrho \right)}^{\prime }{R}_i^t{u}_i^p\left(\varrho \right)+\psi {(\varkappa +1)}^{\prime }H(\varkappa +1)\psi (\varkappa +1)\left]\right\},\hfill \end{array}$$

(13)

where $\overline{G}=\left[\begin{array}{cc}{\overline{G}}_1& {\overline{G}}_2\end{array}\right],u^c\left(\varrho \right)=\left[\begin{array}{c}{u}_1^c\left(\varrho \right)\\ u_2^c\left(\varrho \right)\end{array}\right],R^t=\left[\begin{array}{cc}{R}_1^t& 0\\ 0& R_2^t\end{array}\right]$.

As a result of the preceding, we give the next lemma.

Lemma 1.

The costate Equations (5)–(8) can be expressed as

$$\begin{array}{cc}\hfill & \eta (\varrho -1)=E\left[D^{\prime }\eta \left(\varrho \right)\right|\mathcal{F}\left(\varrho \right)]+Q^t\psi \left(\varrho \right),\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill & 0=E[{\overline{G}}^{\prime }\eta \left(\varrho \right)|{\mathcal{F}}_c\left(\varrho \right)]+R^t{u}^c\left(\varrho \right),\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill & 0=E[{\overline{G}}_1^{\prime }\eta \left(\varrho \right)|{\mathcal{F}}_1\left(\varrho \right)]-E[{\overline{G}}_1^{\prime }\eta \left(\varrho \right)|{\mathcal{F}}_c\left(\varrho \right)]+R_1^t{u}_1^p\left(\varrho \right),\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill & 0=E[{\overline{G}}_2^{\prime }\eta \left(\varrho \right)|{\mathcal{F}}_2\left(\varrho \right)]-E[{\overline{G}}_2^{\prime }\eta \left(\varrho \right)|{\mathcal{F}}_c\left(\varrho \right)]+R_2^t{u}_2^p\left(\varrho \right),\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill & \eta \left(\varkappa \right)=H(\varkappa +1)\psi (\varkappa +1).\hfill \end{array}$$

(18)

Proof. 

By virtue of (9) and (10), applying expectation to both sides of (6) and (7) with ${\mathcal{F}}_c\left(\varrho \right)$, respectively, we can obtain the following:

$$\begin{array}{c}0=E[{\overline{G}}_1^{\prime }\eta \left(\varrho \right)|{\mathcal{F}}_c\left(\varrho \right)]+R_1^t{u}_1^c\left(\varrho \right),\hfill \end{array}$$

(19)

$$\begin{array}{c}0=E[{\overline{G}}_2^{\prime }\eta \left(\varrho \right)|{\mathcal{F}}_c\left(\varrho \right)]+R_2^t{u}_2^c\left(\varrho \right).\hfill \end{array}$$

(20)

Noting (19) and (20) and observing (12) and (13), (15) can be readily obtained.

Subtracting (19) from (6) and using (9), we have

$$\begin{array}{c}0=E[{\overline{G}}_1^{\prime }\eta \left(\varrho \right)|{\mathcal{F}}_1\left(\varrho \right)]-E[{\overline{G}}_1^{\prime }\eta \left(\varrho \right)|{\mathcal{F}}_c\left(\varrho \right)]+R_1^t{u}_1\left(\varrho \right)-R_1^t{u}_1^c\left(\varrho \right),\hfill \\ 0=E[{\overline{G}}_1^{\prime }\eta \left(\varrho \right)|{\mathcal{F}}_1\left(\varrho \right)]-E[{\overline{G}}_1^{\prime }\eta \left(\varrho \right)|{\mathcal{F}}_c\left(\varrho \right)]+R_1^t{u}_1^p\left(\varrho \right),\hfill \end{array}$$

which implies that (16) holds. Similarly, subtracting (20) from (7) and applying (10), (17) is valid. This ends the proof. □

It is noted that through the transformation in lemma 1 and the definitions of (9) and (10), the FBSDEs (12) and (14) can be successfully solved as 4 unknown variables exist in 5 Equations: (12), (14)–(17).

Now the following Riccati equations are introduced:

$$\begin{array}{c}{H}^c\left(\varrho \right)=D^{\prime }{H}^c(\varrho +1)D-M^c{\left(\varrho \right)}^{\prime }{{\rm Y}}^c{\left(\varrho \right)}^{-1}{M}^c\left(\varrho \right)+Q^t,\hfill \end{array}$$

(21)

$$\begin{array}{c}{H}^1\left(\varrho \right)=D^{\prime }{H}^c(\varrho +1)D-M^1{\left(\varrho \right)}^{\prime }{{\rm Y}}^1{\left(\varrho \right)}^{-1}{M}^1\left(\varrho \right)+Q^t,\hfill \end{array}$$

(22)

$$\begin{array}{c}{H}^2\left(\varrho \right)=D^{\prime }{H}^2(\varrho +1)D-M^2{\left(\varrho \right)}^{\prime }{{\rm Y}}^2{\left(\varrho \right)}^{-1}{M}^2\left(\varrho \right)+Q^t,\hfill \end{array}$$

(23)

where

$$\begin{array}{cc}\hfill & {{\rm Y}}^c\left(\varrho \right)={\overline{G}}^{\prime }{H}^c(\varrho +1)\overline{G}+R,M^c\left(\varrho \right)={\overline{G}}^{\prime }{H}^c(\varrho +1)D,\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill & {{\rm Y}}^1\left(\varrho \right)={\overline{G}}_1^{\prime }{H}^c(\varrho +1){\overline{G}}_1+R_1^t,M^1\left(\varrho \right)={\overline{G}}_1^{\prime }{H}^c(\varrho +1)D,\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill & {{\rm Y}}^2\left(\varrho \right)={\overline{G}}_2^{\prime }{H}^2(\varrho +1)\overline{G_2}+R_2^t,M^2\left(\varrho \right)={\overline{G}}_2^{\prime }{H}^2(\varrho +1)D.\hfill \end{array}$$

(26)

with terminal values $H^c(\varkappa +1)=H^1(\varkappa +1)=H^2(\varkappa +1)=H(\varkappa +1)$.

The main results are presented now.

Theorem 1.

A unique solution to Problem 1 exists precisely when ${{\rm Y}}^c\left(\varrho \right)$, ${{\rm Y}}^1\left(\varrho \right)$ and ${{\rm Y}}^2\left(\varrho \right)$ are invertible for $\varrho =0,\dots ,\varkappa$. And the optimal controllers are

$$\begin{array}{c}{u}^c\left(\varrho \right)=-{{\rm Y}}^c{\left(\varrho \right)}^{-1}{M}^c\left(\varrho \right){\widehat{\psi }}^c\left(\varrho \right|\varrho ),\hfill \end{array}$$

(27)

$$\begin{array}{c}{u}_1^p\left(\varrho \right)=-{{\rm Y}}^1{\left(\varrho \right)}^{-1}{M}^1\left(\varrho \right)\left[\begin{array}{c}{\tilde{\psi }}_1\left(\varrho \right|\varrho )\\ 0\end{array}\right],\hfill \end{array}$$

(28)

$$\begin{array}{c}{u}_2^p\left(\varrho \right)=-{{\rm Y}}^2{\left(\varrho \right)}^{-1}{M}^2\left(\varrho \right)\left[\begin{array}{c}0\\ {\tilde{\psi }}_2\left(\varrho \right|\varrho )\end{array}\right],\hfill \end{array}$$

(29)

where

$$\begin{array}{cc}\hfill & {\widehat{\psi }}^c\left(\varrho \right|\varrho )=E\left[\psi \left(\varrho \right)\right|{\mathcal{F}}_c\left(\varrho \right)],\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill & {\tilde{\psi }}_1\left(\varrho \right|\varrho )={\psi }_1\left(\varrho \right)-E\left[{\psi }_1\left(\varrho \right)\right|{\mathcal{F}}_c\left(\varrho \right)]={\psi }_1\left(\varrho \right)-{\widehat{\psi }}_1^c\left(\varrho \right|\varrho ),\hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill & {\tilde{\psi }}_2\left(\varrho \right|\varrho )={\psi }_2\left(\varrho \right)-E\left[{\psi }_2\left(\varrho \right)\right|{\mathcal{F}}_c\left(\varrho \right)]={\psi }_2\left(\varrho \right)-{\widehat{\psi }}_2^c\left(\varrho \right|\varrho ).\hfill \end{array}$$

(32)

Then, the optimum cost is

$$\begin{array}{cc}\hfill {\Omega }^{*}\left(\varkappa \right)& = E\left[\psi {\left(0\right)}^{\prime }{H}^c\left(0\right)\overline{\psi }\left(0\right)+\psi {\left(0\right)}^{\prime }{H}^2\left(0\right)\left[\begin{array}{c}0\\ {\tilde{\psi }}_2\left(0\right|0)\end{array}\right]\right]\hfill \\ \hfill & + \sum _{\varrho =0}^{\varkappa }\left\{tr\right[(H^1\left(\varrho \right)+H^1(\varrho +1))\left[\begin{array}{cc}{Q}_{{\lambda }_1}& 0\\ 0& 0\end{array}\right]\hfill \\ \hfill & + H^2(\varrho +1)\left[\begin{array}{cc}0& 0\\ 0& Q_{{\lambda }_2}\end{array}\right]\left]\right\},\hfill \end{array}$$

(33)

where $\overline{\psi }\left(0\right)=\left[\begin{array}{c}{\overline{\psi }}_1\left(0\right)\\ {\overline{\psi }}_2\left(0\right)\end{array}\right]$. Furthermore, the result of the FBSDEs (12) and (14) is

$$\begin{array}{cc}\hfill \eta (\varrho -1) =& H^c\left(\varrho \right){\widehat{\psi }}^c\left(\varrho \right|\varrho )+H^1\left(\varrho \right)\left[\begin{array}{c}{\tilde{\psi }}_1\left(\varrho \right|\varrho )\\ 0\end{array}\right]\hfill \\ \hfill & + H^2\left(\varrho \right)\left[\begin{array}{c}0\\ {\tilde{\psi }}_2\left(\varrho \right|\varrho )\end{array}\right].\hfill \end{array}$$

(34)

Proof. 

Please view Appendix A. □

3. Numerical Examples

Target tracking (TT) systems have drawn significant attention due to their vast applications, such as navigation, collision avoidance, monitoring missions and so on [37].

Consider a simple TT system of two unmanned aerial vehicles (UAVs), which is depicted as in Figure 1. The two unmanned aerial vehicles are named UAV1 and UAV2. In the TTS, UAV1 is the target aerial vehicle with inferior equipments; UAV2 with superior equipments tracks UAV1 within a safe distance and monitors UAV1’s information. But UAV1 cannot observe the information of UAV2 due to its poor equipments when the distances between UAV1 and UAV2 are far enough. To avoid being detected, UAV2 should keep a safe distance from UAV1. Thus, when UAV2 monitors the information of UAV1, delay is unavoidable. The location and velocity for UAV1 and UVA2 are defined as ${ϵ}_1\left(k\right),{\lambda }_1\left(k\right)$ and ${ϵ}_2\left(k\right),{\lambda }_2\left(k\right)$, respectively (for simplicity, we suppose that UAVs fly in the straight route; the variables are scalars and the delay is a unit).

Correspondingly, the dynamic equations for UAV1 and UAV2 can be obtained as follows:

$$\begin{array}{cc}\hfill {ϵ}_1(\varrho +1)& = d_{11}{ϵ}_1\left(\varrho \right)+g_{11}{v}_1\left(\varrho \right)+{\lambda }_1\left(\varrho \right),\hfill \\ \hfill {ϵ}_2(\varrho +1)& = {ϵ}_2\left(\varrho \right)+d_{21}{ϵ}_1\left(\varrho \right)+g_{22}{v}_2\left(\varrho \right)+g_{21}{v}_1\left(\varrho \right)+{\lambda }_2\left(\varrho \right),\hfill \end{array}$$

where ${\lambda }_1\left(\varrho \right)$ and ${\lambda }_2\left(\varrho \right)$ present the uncertain flight condition, and $d_{11},d_{21},g_{11},g_{21},g_{22}$ are scalar constants. The initial locations ${ϵ}_1\left(0\right)$, ${ϵ}_2\left(0\right)$, ${\lambda }_1\left(0\right)$ and ${\lambda }_2\left(0\right)$ are Gaussian white noises and non-interacting with one another, with a mean of $({\mu }_1,{\mu }_2,0,0)$ and a covariance of $({\sigma }_1,{\sigma }_2,Q_{{\lambda }_1},Q_{{\lambda }_2})$, respectively.

From Figure 1, at time $\varrho$, UAV2 can observe the locations $\{{ϵ}_1(\varrho -1),\dots ,{ϵ}_1\left(0\right)\}$, the velocities $\{v_1(\varrho -1),\dots ,v_1\left(0\right)\}$ of UAV1, and its own locations $\{{ϵ}_2\left(\varrho \right),{ϵ}_2(\varrho -1),\dots ,{ϵ}_1\left(0\right)\}$ and velocities $\{v_2\left(\varrho \right),v_2(\varrho -1),\dots ,v_2\left(0\right)\}$. Obviously, the communication channel from UAV1 to UAV2 presents a unit delay. UVA1 can only obtain its own locations $\{{ϵ}_1\left(\varrho \right),{ϵ}_1(\varrho -1),\dots ,{ϵ}_1\left(0\right)\}$ and velocities $\{v_1\left(\varrho \right),v_1(\varrho -1),\dots ,v_1\left(0\right)\}$. In this scenario, the aim of the TT systems is to keep the distance of UAV1 and UAV2 to the safe distance l and keep the energy cost of UAV1 and UAV2 to the relative minimum. Accordingly, the cost function is of the form

$$\begin{array}{cc}\hfill \Omega \left(\varkappa \right)& =\sum _{\varrho =0}^{\varkappa }E\{{[{ϵ}_2\left(\varrho \right)-{ϵ}_1\left(\varrho \right)-l]}^2+R_1^t{v}_1{\left(\varrho \right)}^2+R_2^t{v}_2{\left(\varrho \right)}^2\},\hfill \end{array}$$

where $R_1^t>0$ and $R_2^t>0$ are the weighting matrices of the energy cost for UAV1 and UAV2, respectively.

This problem can be resolved by using the results obtained in this paper. Firstly, we define ${\psi }_1\left(\varrho \right)={ϵ}_1\left(\varrho \right)$, ${\psi }_2\left(\varrho \right)={ϵ}_2\left(\varrho \right)-l$, $u_1\left(\varrho \right)=v_1\left(\varrho \right)$ and $u_2\left(\varrho \right)=v_2\left(\varrho \right)$. Then, the corresponding linear system and the finite-horizon cost function for the can be defined as

$$\begin{array}{c}\psi (\varrho +1)=D\psi \left(\varrho \right)+{\overline{G}}_1{u}_1\left(\varrho \right)+{\overline{G}}_2{u}_2\left(\varrho \right)+\lambda \left(\varrho \right),\end{array}$$

(35)

$$\begin{array}{c}\Omega \left(\varkappa \right)= E\left\{\sum _{\varrho =0}^{\varkappa }\sum _{i=1}^2[\psi {\left(\varrho \right)}^{\prime }{Q}^t\psi \left(\varrho \right)+R_i^t{u}_i^2\left(\varrho \right)]\right\},\end{array}$$

(36)

where $\psi \left(\varrho \right)=\left[\begin{array}{c}{\psi }_1\left(\varrho \right)\\ {\psi }_2\left(\varrho \right)\end{array}\right],D=\left[\begin{array}{cc}{d}_{11}& 0\\ d_{21}& 1\end{array}\right],{\overline{G}}_1=\left[\begin{array}{c}{g}_{11}\\ g_{21}\end{array}\right],{\overline{G}}_2=\left[\begin{array}{c}0\\ g_{22}\end{array}\right],e\left(\varrho \right)=\left[\begin{array}{c}{\lambda }_1\left(\varrho \right)\\ {\lambda }_2\left(\varrho \right)\end{array}\right],Q^t=\left[\begin{array}{cc}1& -1\\ -1& 1\end{array}\right]$.

By virtue of Theorem 1, we give the following corollary.

Corollary 1.

The optimal controllers $u_1^{*}\left(\varrho \right)$ and $u_2^{*}\left(\varrho \right)$ are of the forms

$$\begin{array}{cc}\hfill u_1^{*}\left(\varrho \right)=& -\left[\begin{array}{cc}1& 0\end{array}\right]{{\rm Y}}^c{\left(\varrho \right)}^{-1}{M}^c\left(\varrho \right){\widehat{\psi }}^c\left(\varrho \right|\varrho )\hfill \\ \hfill & - {{\rm Y}}^1{\left(\varrho \right)}^{-1}{M}^1\left(\varrho \right)\left[\begin{array}{c}{\tilde{\psi }}_1\left(\varrho \right|\varrho )\\ 0\end{array}\right],\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill u_2^{*}\left(\varrho \right)=& -\left[\begin{array}{cc}0& 1\end{array}\right]{{\rm Y}}^c{\left(\varrho \right)}^{-1}{M}^c\left(\varrho \right){\widehat{\psi }}^c\left(\varrho \right|\varrho )\hfill \\ \hfill & - {{\rm Y}}^2{\left(\varrho \right)}^{-1}{M}^2\left(\varrho \right)\left[\begin{array}{c}0\\ {\tilde{\psi }}_2\left(\varrho \right|\varrho )\end{array}\right],\hfill \end{array}$$

(38)

where ${\widehat{\psi }}^c\left(\varrho \right|\varrho )$ is the estimation of $\psi \left(\varrho \right)$ based on $\{{\psi }_1(\varrho -1),\dots ,{\psi }_1\left(0\right)\}$, ${\tilde{\psi }}_1\left(\varrho \right|\varrho )={\psi }_1\left(\varrho \right)-{\widehat{\psi }}_1^c\left(\varrho \right|\varrho )$ is the estimation error of ${\psi }_1\left(\varrho \right)$, ${\tilde{\psi }}_2\left(\varrho \right|\varrho )={\psi }_2\left(\varrho \right)-{\widehat{\psi }}_2^c\left(\varrho \right|\varrho )$ is the estimation error of ${\psi }_2\left(\varrho \right)$, and ${{\rm Y}}^c\left(\varrho \right),M^c\left(\varrho \right),{{\rm Y}}^1\left(\varrho \right),M^1\left(\varrho \right),{{\rm Y}}^2\left(\varrho \right),M^2\left(\varrho \right)$ are as in (24)–(26).

For similarity, we set the safe distance $l=3$, $d_{11}=d_{21}=g_{11}=g_{21}=g_{22}=1,$${\mu }_1={\mu }_2=0$, ${\sigma }_1={\sigma }_2=Q_{{\lambda }_1}=Q_{{\lambda }_2}=1$, $R_1^t=1.5$, $R_2^t=2$ and $N=100$.

Figure 2 shows the distances between UAV1 and UAV2. It can be observed that the distances between the two UAVs are around the safe distance $l=3$. Figure 3 indicates the velocities of the two UAVs. It can be seen that the velocities of UAV2 are close to those of UAV1.

4. Conclusions

This paper handles the problems of optimal control and stability for decentralized system with one-step asymmetric delay information. Employing Pontryagin’s maximum principle, costate equation and equilibrium equations are given. By virtue of these equations, we obtain the solution to the FBSDEs. Based on this solution, the finite-horizon optimization problem is solved. In view of the finite-horizon optimal cost, the Lyapunov function is defined and the mean-square stabilization conditions for the system are derived in terms of two algebraic Riccati equations. We generalize the proposed algorithm to the examples of a target tracking system to verify the effectiveness of the given results. Future research will focus on extending the proposed framework to multi-player interconnected systems, relaxing the one-step delay constraint to multi-step delays, investigating infinite-horizon scenarios, weakening the matrix invertibility assumption, and accommodating more general non-Gaussian noise distributions to enhance the method’s generality and applicability.