A Cooperative Decision-Making and Control Algorithm for UAV Formation Based on Non-Cooperative Game Theory

Abstract

The formation control problem of distributed fixed-wing Unmanned Aerial Vehicles (UAVs) is investigated in this paper. By utilizing the theoretical foundations of non-cooperative game theory, a novel control strategy is introduced, which allows UAVs to autonomously determine the optimal flight trajectory without relying on centralized coordination while concurrently mitigating conflicts with other UAVs. By transforming the UAV model into a double integrator form, the control complexity is reduced. Additionally, the incorporation of a homogeneous differential disturbance observer enhances the UAV’s resilience against disturbances during the control process. Through the development and validation of a Nash equilibrium-based algorithm, it is demonstrated that UAVs can sustain a predefined formation flight and autonomously adapt their trajectories in complex environments. Simulations are presented to confirm the efficiency of the proposed method.

1. Introduction

Recently, the utilization of Unmanned Aerial Vehicles (UAVs) has become increasingly prevalent. Specifically, fixed-wing UAVs are prized for their extended endurance and superior speed. Formation flight serves as a pivotal technology for enhancing the efficiency of UAV missions, allowing multiple UAVs to collaborate in tasks such as search, surveillance, mapping, and target localization [1,2,3,4,5,6,7]. However, with the escalating complexity of mission environments and the proliferating number of UAVs, traditional centralized control approaches encounter obstacles, including constraints in communication bandwidth, risks of single-point failures, and limitations in real-time processing [8,9,10]. In contrast, the distributed formation control method rooted in graph theory exhibits strong stability and robustness [11,12,13,14,15].

Unlike cooperative games, non-cooperative participants optimize independently, which is suitable for scenarios where UAVs cannot or will not share information. Assuming each UAV minimizes its cost function based on its own and others’ actions, an appropriate game-theoretic model and Nash equilibrium ensure stable equilibrium without external enforcement [16,17]. The communication and computational costs of Nash equilibrium-seeking strategies in non-cooperative games were reduced by introducing an interference graph to describe the interactions between agents within each coalition and by designing Nash equilibrium-seeking strategies that consider the interference graph [18]. A projection algorithm based on continuous-time distributed gradient was proposed to address the problem where both the cost function and the feasible action set depend on the actions of all participants, and where each participant can only access the information of its neighbors. This algorithm utilizes a leader–follower consensus algorithm to estimate the behavior of each participant [19].

The Disturbance Observer was proposed by Japanese scholar K. Ohnishi in 1987 [20]. It is primarily used to handle external disturbances and uncertainties arising from changes in model parameters in control systems. By observing and compensating for these disturbances, it achieves optimization and stabilization of system performance. A time-varying dynamic modeling method for space double flexible telescopic manipulators, coupled with a fuzzy PI real-time control strategy integrated with a fractional order disturbance observer, was proposed to address the challenges posed by their nonlinear and time-varying characteristics, aiming to enhance operational stability and precision [21]. A finite-time control method is proposed for unmanned ground vehicle trajectory tracking, considering uncertainty and disturbance compensation. Employing a Frenet-coordinate system and a novel coordinate transformation, a finite-time disturbance observer and controller based on an integral-type Lyapunov function are designed [22]. An improved discrete super-twisting control algorithm is proposed, leveraging homogeneous system theory to precisely estimate and compensate for system disturbances, enhancing robustness. An adjustable fractional power parameter is introduced to address sampling constraints, ensuring higher control accuracy in closed-loop systems [23].

In the context of multi-UAV formation control, backstepping can be used to develop controllers that ensure the formation shape and spacing between the UAVs are maintained during flight [24,25]. The effectiveness of backstepping for this complex and challenging control problem has been demonstrated [26]. An adaptive fuzzy tracking control strategy for pure feedback nonlinear multi-agent systems, developed through backstepping, has been proposed [27]. An approach combining backstepping technology with neural networks, introducing a radial basis function neural network with a gradient descent algorithm, has been proposed to address the unknown complex nonlinear problems of UAV systems caused by external disturbances [28]. A distributed formation controller is designed using the backstepping method based on the asymmetric barrier Lyapunov function [29].

This paper introduces a distributed formation control strategy for fixed-wing UAVs based on the non-cooperative game theory. Compared to the aforementioned literature, this paper makes the following innovations. A double integrator model was used to reduce the design complexity of the controller [30]. We innovatively combine the principles of non-cooperative game theory with the backstepping control method to design a distributed formation controller, enabling the states of the multi-UAV system to converge to a Nash equilibrium point. Additionally, a homogeneous differential disturbance observer is employed to observe and mitigate the disturbances generated in the system. The effectiveness of this method has been confirmed by simulations.

2. Preliminaries

2.1. Graph Theory

In a multi-agent communication topology, the graph $G=\{V,E\}$ embodies the connectivity among various agents. Here, $V=\{V_1,V_2,\dots ,V_N\}$ denotes the set of nodes, representing individual agents, and $E=\{E_1,E_2,\dots ,E_M\}$ signifies the set of edges, denoting communication links between these nodes. The presence of an edge from node i to node j is symbolically indicated by the pair $(i,j)\in E$. Furthermore, if there exists a sequence of edges $(i,j_1),(j_1,j_2),\dots ,(j_m,j)\in E$, this implies that there is a directed path enabling communication from node i to node j.

A directed graph G qualifies as strongly connected if, regardless of the pair of nodes chosen, a directed path facilitating communication between them can be identified. The adjacency matrix associated with this directed graph is denoted as A, where each element $a_{ij}$ corresponds to the entry in the i-th row and j-th column. Specifically, $a_{ij}=1$ when there exists an edge from node i to node j, i.e., $(i,j)\in E$, while $a_{ij}=0$ in all other cases. It is important to note that the graph G explicitly disallows self-loops, meaning that $a_{ii}=0$ for all nodes.

The in-degree matrix, represented as D, is constructed as a diagonal matrix with the degrees of the nodes along its diagonal. Mathematically, it is expressed as $D=\mathrm{diag}\{\mathrm{deg}\left(V_1\right),\mathrm{deg}\left(V_2\right),\dots ,\mathrm{deg}\left(V_N\right)\}$, where the degree $\mathrm{deg}\left(V_j\right)$ of a node $V_j$ is calculated as the sum of the entries in the j-th column of the adjacency matrix A, or $\mathrm{deg}\left(V_j\right)={\sum }_{i=1}^N{a}_{ij}$.

The Laplacian matrix L of the graph G is then defined as the difference between the in-degree matrix D and the adjacency matrix A, formulated as $L=D-A$. In the context of a strongly connected graph $G=\{V,E\}$, the eigenvalue spectrum of the Laplacian matrix L exhibits a unique characteristic: the value 0 emerges as the sole eigenvalue, accompanied by the vector $1_N$ as the corresponding eigenvector [31].

2.2. Non-Cooperative Game

Defining V as the set of N players, $o_i\in {\mathbb{R}}^n$ represents the operation of the i-th player, $o={[o_1^T,\dots ,o_N^T]}^T$ is the joint action vector, and $f_i\left(o\right)$ the cost function associated with the i-th player. In a non-cooperative game, the cost function for each player is determined by the joint actions of all players involved in the game. Each player aims to minimize their own individual cost function, taking into account the actions of the other players. This results in a strategic interaction where the optimal strategy for each player depends on the strategies chosen by the others.

If there exists a joint action vector $o^{\ast }={[o_1^{\ast T},\dots ,o_N^{\ast T}]}^T$, which means a collection of strategies chosen by each of the N players, such that for any player i within the set $\{1,\dots ,N\}$ and for any alternative strategy $o_i$ that this player might consider, the following condition holds:

$$f_i(o_i,o_{-i}^{\ast })\ge f_i(o_i^{\ast },o_{-i}^{\ast })$$

(1)

where $o^{\ast }$ is referred to as a Nash equilibrium solution of the non-cooperative game [32]. Here, $o_{-i}={[o_1^T,\dots ,o_{i-1}^T,o_{i+1}^T,\dots ,o_N^T]}^T$ represents the actions of all players except the i-th player, highlighting the interdependence of players’ decisions in the game.

The aforementioned inequality signifies a crucial property of the Nash equilibrium: it ensures that once the system reaches this state, no individual player has an incentive to deviate from their current strategy, as doing so would not lead to a reduction in their cost function, or equivalently, would not improve their payoff. This principle underlines the stability of the Nash equilibrium as a solution concept in non-cooperative games.

For an unconstrained non-cooperative game among players, the Nash equilibrium is mathematically characterized by the first-order condition ${\displaystyle \frac{\partial f_i\left(o^{\ast }\right)}{\partial o_i}}=0$. This condition implies that at the equilibrium point, the marginal change in the cost (or payoff) with respect to the player’s own action is zero, indicating a local optimum for each player given the actions of others.

Furthermore, if the cost function $f_i\left(o\right)$ is continuous in the overall action vector o and exhibits convexity in the individual action $o_i$ while the actions of other players $o_{-i}$ are held constant, the existence of a Nash equilibrium in the non-cooperative game is assured. This assurance stems from the fact that under these conditions, the game’s structure allows for a unique solution where no player can benefit from a unilateral deviation, thus ensuring the stability and predictability of the game’s outcome. Meanwhile, the convexity of the function lays the foundation for the effectiveness of the gradient descent method, which will be partially utilized later on.

Lemma 1 

([33]). If Q is a non-singular M-matrix, then $PQ+Q^{T}P>0$ for $P=\mathrm{diag}(p_1,\dots ,p_N)>0$.

Lemma 2 

([34]). If $c,d>0$ and $\gamma >0$, and $x,y\in \mathbb{R}$, the following inequalities hold:

$${\left|x\right|}^c{\left|y\right|}^d⩽{\displaystyle \frac{c}{c+d}}{\gamma \left|x\right|}^{c+d}+{\displaystyle \frac{d}{c+d}}{\gamma }^{-\frac{c}{d}}{\left|y\right|}^{c+d}$$

(2)

3. Modeling of Fixed-Wing Unmanned Aerial Vehicles

Assume that the UAVs are labeled as $i=1,2,3,\dots ,N$, where N represents the number of UAVs in the formation, and all the fixed-wing UAVs are flying at the same altitude. The typical kinematic model for the i-th UAV can be expressed as follows:

$$\begin{array}{cc}\hfill \dot{x_i}=v_i& cos{\theta }_i,\dot{y_i}=v_{i}sin{\theta }_i,\dot{{\theta }_i}={\omega }_i\hfill \\ \hfill \dot{v_i}& =u_{i,1},\dot{{\omega }_i}=u_{i,2}\hfill \end{array}$$

(3)

The variables $x_i$, $y_i$, and ${\theta }_i$ denote the horizontal coordinate, vertical coordinate, and heading angle of the i-th UAV, respectively. The linear velocity and angular velocity of the UAV are represented by $v_i$ and ${\omega }_i$, respectively. The control inputs for these dynamics are given by $u_{i,1}$ and $u_{i,2}$.

As described in the paper [30] and illustrated in Figure 1, we define $x_{hi}=x_i+l_{i}cos{\theta }_i$ and $y_{hi}=y_i+l_{i}sin{\theta }_i$, where $(x_i,y_i)$, indicated by the red dot, is the centroid between the head and tail of the i-th UAV, and $l_i$ represents the distance from the centroid to the head of the i-th UAV. As a result, it follows from (3) that

$$\begin{array}{cc}\hfill {\dot{x}}_{hi}& ={\dot{x}}_i-l_i{\omega }_{i}sin\theta =v_{i}cos{\theta }_i-l_i{\omega }_{i}sin{\theta }_i\hfill \\ \hfill {\ddot{x}}_{hi}& =u_{i,1}cos{\theta }_i-v_i{\omega }_{i}sin{\theta }_i-l_i{\omega }_i^{2}cos{\theta }_i-l_i{u}_{i,2}sin{\theta }_i\hfill \\ \hfill {\dot{y}}_{hi}& ={\dot{y}}_i+l_i{\omega }_{i}cos{\theta }_i=v_{i}sin{\theta }_i+l_i{\omega }_{i}cos{\theta }_i\hfill \\ \hfill {\ddot{y}}_{hi}& =u_{i,1}sin{\theta }_i+v_i{\omega }_{i}cos{\theta }_i-l_i{\omega }_i^{2}sin{\theta }_i+l_i{u}_{i,2}cos{\theta }_i\hfill \end{array}$$

(4)

Defining ${\ddot{x}}_{hi}=u_{x,i}$ and ${\ddot{y}}_{hi}=u_{y,i}$, one obtains

$$\begin{array}{cc}\hfill \left[\begin{array}{c}{\ddot{x}}_{hi}\\ {\ddot{y}}_{hi}\end{array}\right]=& \left[\begin{array}{cc}cos{\theta }_i& -l_{i}sin{\theta }_i\\ sin{\theta }_i& l_{i}cos{\theta }_i\end{array}\right]\left[\begin{array}{c}{u}_{i,1}\\ u_{i,2}\end{array}\right]+\left[\begin{array}{c}-v_i{\omega }_{i}sin{\theta }_i-l_i{\omega }_i^{2}cos{\theta }_i\\ v_i{\omega }_{i}cos{\theta }_i-l_i{\omega }_i^{2}sin{\theta }_i\end{array}\right]\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill \left[\begin{array}{c}{u}_{i,1}\\ u_{i,2}\end{array}\right]=& {\left[\begin{array}{cc}cos{\theta }_i& -l_{i}sin{\theta }_i\\ sin{\theta }_i& l_{i}cos{\theta }_i\end{array}\right]}^{-1}\left[\begin{array}{c}{u}_{x,i}+v_i{\omega }_{i}sin{\theta }_i+l_i{\omega }_i^{2}cos{\theta }_i\\ u_{y,i}-v_i{\omega }_{i}cos{\theta }_i+l_i{\omega }_i^{2}sin{\theta }_i\end{array}\right]\hfill \end{array}$$

(6)

At this point, the following relationships are established:

$$\begin{array}{c}\hfill \left[\begin{array}{c}{\dot{x}}_{hi}\hfill \\ {\dot{y}}_{hi}\hfill \end{array}\right]=\left[\begin{array}{c}{v}_{xi}\hfill \\ v_{yi}\hfill \end{array}\right],\left[\begin{array}{c}{\dot{v}}_{xi}\hfill \\ {\dot{v}}_{yi}\hfill \end{array}\right]=\left[\begin{array}{c}{u}_{x,i}\hfill \\ u_{y,i}\hfill \end{array}\right]\end{array}$$

(7)

As a result, ${\xi }_i={[x_{hi},y_{hi}]}^T$ represents the head coordinates of the UAVs, ${\nu }_i={[v_{x,i},v_{y,i}]}^T$ represents the velocities of the UAVs, and $u_i={[u_{x,i},u_{y,i}]}^T$ represents the control signals. Consequently, the following relationships hold:

$$\begin{array}{c}\hfill {\dot{\xi }}_i={\nu }_i,{\dot{\nu }}_i=u_i\end{array}$$

(8)

4. Non-Cooperative Game and Nash Equilibrium

4.1. Cost Function

In a non-cooperative game, players aim to minimize their individual cost functions by underscoring the importance of accurately determining these functions.

In the context of UAV formation flying, each UAV pursues two primary objectives. The first objective is to maintain the formation by keeping a specified relative distance from other UAVs, constituting an individual objective. The second objective is to follow the overall reference path ${\xi }_0$, which involves maintaining a specified distance relative to ${\xi }_0$, fulfilling a collective objective. Here, ${\xi }_0={[x_{h0}\left(t\right),y_{h0}\left(t\right)]}^T$ represents a time-dependent function.

To maintain formation, we expect each UAV to maintain a fixed relative position with respect to the others. We defined $d_{ij}$ represents the desired coordinate difference between the head of the i-th UAV and the head of the j-th UAV. To achieve this objective, we aim to minimize

$$\sum _{j=1}^N{a}_{ij}\left|\right|{\xi }_i-{\xi }_j-d_{ij}{\left|\right|}^2$$

(9)

where $a_{ij}$ is the element of the adjacency matrix A and it represents the interaction between the i-th and j-th UAVs.

In order for the UAV formation to track a given mission trajectory as a whole, we expect the UAVs with individual targets to maintain a fixed relative position with respect to the mission trajectory point ${\xi }_0\left(t\right)$. Let $p_i$ be the desired coordinate difference between the head of the i-th UAV and ${\xi }_0$. To achieve this goal for agent $V_i$ with independent objectives, we aim to minimize

$$\sum _{j=1}^N{\alpha }_i\left|\right|{\xi }_i-{\xi }_0-p_i{\left|\right|}^2$$

(10)

where ${\alpha }_i=1$ when $V_i\in V_s$ and ${\alpha }_i=0$ otherwise. Integrating these two objectives, the cost function $J_i\left(\xi \right)$ for the i-th UAV is crafted as a balanced trade-off between formation maintenance and path tracking. The first component ensures that each UAV contributes to the formation’s cohesion by minimizing deviations from the desired inter-UAV distances. The second component focuses on the collective objective of tracking the reference path, with select UAVs assuming primary responsibility for this task. Consequently, the cost function $J_i\left(\xi \right)$ of the i-th UAV can be designed as follows:

$$J_i\left(\xi \right)={\displaystyle \frac{1}{2}}\sum _{j=1}^N{a}_{ij}\left|\right|{\xi }_i-{\xi }_j-d_{ij}{\left|\right|}^2+{\displaystyle \frac{{\alpha }_i}{2}}\left|\right|{\xi }_i-{\xi }_0-p_i{\left|\right|}^2$$

(11)

where $\xi ={[{\xi }_1^T,\dots ,{\xi }_N^T]}^T$, $V_s$ denotes the set of UAVs with independent objectives, and ${\xi }_0$ is defined as the given trajectory. By optimizing this cost function, the UAVs within the formation are able to dynamically adjust their positions in response to changes in the environment, ensuring that the formation moves in harmony towards its designated destination with precision and efficiency.

4.2. Nash Equilibrium

Previously, we concluded that in an unconstrained non-cooperative game among players, at the Nash equilibrium, ${\displaystyle \frac{\partial f_i\left(o^{\ast }\right)}{\partial o_i}}=0$ holds. If $f_i\left(o\right)$ is continuous in o and convex in $o_i$ with fixed $o_{-i}$, then a Nash equilibrium exists for the non-cooperative game.

Currently, N UAVs are interconnected as represented by the graph G, with A as the adjacency matrix and L as the Laplacian matrix of the graph. Taking the partial derivative of $J_i\left(\xi \right)$ with respect to ${\xi }_i$, we derive that

$${\Psi }_i\left(\xi \right)={\displaystyle \frac{\partial J_i\left(\xi \right)}{\partial {\xi }_i}}=\sum _{j=1}^N{a}_{ij}({\xi }_i-{\xi }_j-d_{ij})+{\alpha }_i({\xi }_i-{\xi }_0-p_i).$$

(12)

We define $\Psi \left(\xi \right)={[{\Psi }_1{\left(\xi \right)}^T,\dots ,{\Psi }_N{\left(\xi \right)}^T]}^T$, $\Lambda =\mathrm{diag}({\alpha }_1,\dots ,{\alpha }_N)$, and $Q=L+\Lambda$. Let $\alpha ={[{\alpha }_1,\dots ,{\alpha }_N]}^T$. Consequently, we have the following:

$$\Psi \left(\xi \right)=(Q\otimes I_3)\xi -d-\alpha \otimes {\xi }_0,$$

(13)

where $\xi ={[{\xi }_1^T,\dots ,{\xi }_N^T]}^T$, $d={[d_1^T,\dots ,d_N^T]}^T$, and $d_i={\sum }_{j=1}^N{a}_{ij}{d}_{ij}+{\alpha }_i{p}_i$. Given that Q is non-singular, when $\Psi \left(\xi \right)=0_{3N}$, a unique Nash equilibrium state exists, which is denoted as ${\xi }^{\ast }={[{\xi }_1^{\ast T},\dots ,{\xi }_N^T]}^T$. This state can be expressed as follows:

$$\begin{array}{cc}\hfill {\xi }^{\ast }& ={(Q\otimes I_3)}^{-1}(d+\alpha \otimes {\xi }_0)\hfill \\ \hfill & ={(Q\otimes I_3)}^{-1}d+1_N\otimes {\xi }_0\hfill \end{array}$$

(14)

5. Design of Disturbance Observer Based on Homogeneous Differentiation

The Disturbance Observer (DOB) represents an advanced control strategy specifically tailored to estimate and subsequently compensate for external disturbances that may arise within a system. We will design a DOB based on homogeneous differentiation to observe the system disturbances and compensate for them within the controller. Additionally, we will prove the convergence of this disturbance observer to ensure its stability and effectiveness.

Definition 1. 

We define

$$si{g}^{\alpha }\left(x\right)=sign\left(x\right){\left|x\right|}^{\alpha }$$

(15)

where $0\le \alpha \le 1$, and $sign$ denote the sign function.

We assume that the disturbances in the system are denoted as $\kappa =[{\kappa }_1;{\kappa }_2]$, and under these conditions, for the i-th UAV:

$$\begin{array}{cc}\hfill \dot{x_i}=v_i& cos{\theta }_i,\dot{y_i}=v_{i}sin{\theta }_i,\dot{{\theta }_i}={\omega }_i\hfill \\ \hfill \dot{v_i}& =u_{i,1}+{\kappa }_1,\dot{{\omega }_i}=u_{i,2}+{\kappa }_2\hfill \end{array}$$

(16)

Assumption A1. 

The disturbance κ acting on the system is bounded, meaning that there exists an $\iota >0$ such that $\left|\kappa \right|<\iota$.

Theorem 1 

([23]). The exponent power terms of the homogeneous differential observer possess a degree of flexibility, allowing them to be tuned within a predefined range. This adaptability enhances the performance and versatility of the observer. The expression below exemplifies a specific form of the homogeneous differential observer, showcasing its structural characteristics:

$$\begin{array}{cc}\hfill {\dot{\widehat{x}}}_1& ={\widehat{x}}_2-m{\mathrm{sig}}^{1+\alpha }\left({\widehat{x}}_1-g\right)\hfill \\ \hfill {\dot{\widehat{x}}}_2& =-n{\mathrm{sig}}^{1+2\alpha }\left({\widehat{x}}_1-g\right)\hfill \end{array}$$

(17)

where $-{\displaystyle \frac{1}{2}}<\alpha <0$, $m>0$, $n>0$, ${\widehat{x}}_1$ and ${\widehat{x}}_2$ are, respectively, the estimated values of g and $\dot{g}$. If

$$\begin{array}{cc}\hfill \left|\ddot{g}\left(t\right)\right|& <C\hfill \\ \hfill m& \ge {(2^{\frac{1}{1+\alpha }-1}(1+(2{\lambda }_1+2){\lambda }_2+{\lambda }_3)n)}^{\frac{1}{2}}\hfill \\ \hfill n& >C\hfill \end{array}$$

(18)

where C is a constant, ${\lambda }_1={\displaystyle \frac{1}{1+\alpha }}\left({\displaystyle \frac{1-2\alpha }{2+\alpha }}+{\displaystyle \frac{3+4\alpha }{2+\alpha }}{2}^{\frac{1}{1+\alpha }}\right)$, ${\lambda }_2={\displaystyle \frac{1+2\alpha }{2+\alpha }}{2}^{\frac{-\alpha }{1+\alpha }}{(2^{\frac{1}{1+\alpha }}{\displaystyle \frac{1-\alpha }{2+\alpha }})}^{\frac{1-\alpha }{1+2\alpha }}$, ${\lambda }_3={\displaystyle \frac{1}{1+\alpha }}{\displaystyle \frac{3+4\alpha }{2+\alpha }}{2}^{\frac{1}{1+\alpha }}$.

Then, the aforementioned homogeneous differential observer exhibits a property of finite-time convergence. This signifies that the estimation error generated by the observer rapidly diminishes and converges precisely to zero within a predetermined, finite time interval. In contrast, it does not follow the traditional asymptotic convergence pattern where the error tends to zero as time approaches infinity. The finite-time convergence feature ensures that the observer’s estimates accurately reflect the actual values in a timely and efficient manner.

Lemma 3. 

By using Theorem 1, we can draw the following conclusion: A homogeneous differential disturbance observer in the following form

$$\begin{array}{cc}\hfill \dot{\widehat{v}}& =-msi{g}^{(}1+\alpha )(\widehat{v}-v)+\widehat{\kappa }-\kappa +u\hfill \\ \hfill \dot{\widehat{\kappa }}& =-nsi{g}^{(}1+2\alpha )(\widehat{v}-v),\hfill \end{array}$$

(19)

can track the internal disturbances within the system within a finite time. That is, $\widehat{\kappa }$ will converge to κ within a finite time. Where $-{\displaystyle \frac{1}{2}}<\alpha <0$, $m>0$, $n>0$, $\widehat{\kappa }$ is the output of the disturbance observer.

Proof. 

Define the Estimated value $x_1$ and $x_2$ of this homogeneous differential disturbance observer as

$$\begin{array}{c}\hfill x_1=\widehat{v}\\ \hfill x_2=\widehat{\kappa }\end{array}$$

(20)

Therefore, we can obtain the dynamic equation of the estimation error as follows:

$$\begin{array}{c}\hfill \dot{x_1}=-msi{g}^{1+\alpha }(x_1-v)+x_2\\ \hfill \dot{x_2}=-nsi{g}^{1+2\alpha }(x_1-v)\end{array}$$

(21)

If the homogeneous differential observer is transformed into this particular form, it becomes readily apparent that it fulfills the necessary criteria stipulated by Theorem 1. Consequently, the estimation $\widehat{\kappa }$ of the disturbance $\kappa$ generated by this tailored observer exhibits the property of finite-time convergence. This implies that the estimation error converges precisely to zero within a finite time interval, ensuring the timely and accurate reflection of the actual disturbance value.

After offsetting the estimated value of the disturbance against the actual disturbance for the i-th UAV is as follows:

$$\begin{array}{cc}\hfill \dot{x_i}& =v_{i}cos{\theta }_i,\dot{y_i}=v_{i}sin{\theta }_i,\dot{{\theta }_i}={\omega }_i\hfill \\ \hfill \dot{v_i}& =(u_{i,1}-\widehat{{\kappa }_1})+{\kappa }_1,\dot{{\omega }_i}=(u_{i,2}-\widehat{{\kappa }_2})+{\kappa }_2\hfill \end{array}$$

(22)

6. Implementation of Fixed-Wing UAV Formation Control Based on Non-Cooperative Game

We have presented the designed controller and proven its feasibility.

Theorem 2. 

Assuming that the objective of the UAVs is to minimize the cost function $J_i\left(\xi \right)$, if a gradient control strategy is designed as follows:

$$u_i={\ddot{\xi }}_0-{\dot{\Psi }}_i\left(\xi \right)-c({\Psi }_i\left(\xi \right)+{\nu }_i-{\dot{\xi }}_0)$$

(23)

then the UAVs will gradually converge to the Nash equilibrium state, where c is a coefficient to be determined.

Proof. 

Let $\nu ={[{\nu }_1^T,\dots ,{\nu }_N^T]}^T$ and $u={[u_1^T,\dots ,u_N^T]}^T$. Then, the following relationships hold:

$$\begin{array}{c}\hfill \dot{\xi }=\nu ,\dot{\nu }=u\end{array}$$

(24)

Feasibility is proved using backstepping theory [35]. Initially, let ${\epsilon }_1=\xi -{\xi }^{\ast }$, and choose the Lyapunov function as follows:

$$V_1={\epsilon }_1^T(P\otimes I_3){\epsilon }_1$$

(25)

where P is a positive definite matrix. It follows that

$$\begin{array}{cc}\hfill \dot{V_1}=& {(\xi -{\xi }^{\ast })}^T(P\otimes I_3)(\nu -1_N\otimes {\dot{\xi }}_0)\hfill \\ \hfill & +{(\nu -1_N\otimes {\dot{\xi }}_0)}^T(P\otimes I_3)(\xi -{\xi }^{\ast })\hfill \end{array}$$

(26)

Using the fact that $\Psi \left({\xi }^{\ast }\right)=0_{3N}$, we have

$$\begin{array}{cc}\hfill {\dot{V}}_1=& {(\xi -{\xi }^{\ast })}^T(P\otimes I_3)(\Psi \left(\xi \right)+\nu -1_N\otimes {\dot{\xi }}_0)\hfill \\ \hfill & -{(\xi -{\xi }^{\ast })}^T(P\otimes I_3)(\Psi \left(\xi \right)-\Psi \left({\xi }^{\ast }\right))\hfill \\ \hfill & +{(\Psi \left(\xi \right)+\nu -1_N\otimes {\dot{\xi }}_0)}^T(P\otimes I_3)(\xi -{\xi }^{\ast })\hfill \\ \hfill & -{(\Psi \left(\xi \right)-\Psi \left({\xi }^{\ast }\right))}^T(P\otimes I_3)(\xi -{\xi }^{\ast })\hfill \end{array}$$

(27)

From $\Psi \left(\xi \right)-\Psi \left(\xi \right)=(Q\otimes I_3)(\xi -\xi )$, it can be derived that

$$\begin{array}{cc}\hfill {\dot{V}}_1=& {(\xi -{\xi }^{\ast })}^T(P\otimes I_3)(\Psi \left(\xi \right)+\nu -1_N\otimes {\dot{\xi }}_0)\hfill \\ \hfill & -{(\xi -{\xi }^{\ast })}^T(PQ\otimes I_3)(\xi -{\xi }^{\ast })\hfill \\ \hfill & +{(\Psi \left(\xi \right)+\nu -1_N\otimes {\dot{\xi }}_0)}^T(P\otimes I_3)(\xi -{\xi }^{\ast })\hfill \\ \hfill & -{(\xi -{\xi }^{\ast })}^T(Q^{T}P\otimes I_3)(\xi -{\xi }^{\ast })\hfill \end{array}$$

(28)

Given that $-PQ-Q^{T}P<0$, we define $\lambda ={\lambda }_{min}(PQ+Q^{T}P)$, $p_{max}=max(p_1,\dots ,p_N)$, and $p_{min}=min(p_1,\dots ,p_N)$, which leads to the following inequality:

$$\begin{array}{cc}\hfill {\dot{V}}_1\le & -\lambda \left|\right|\xi -{\xi }^{\ast }{\left|\right|}^2\hfill \\ \hfill & +2p_{max}\left|\right|\xi -{\xi }^{\ast }\left|\right|·\left|\right|\Psi \left(\xi \right)+\nu -1_N\otimes {\dot{\xi }}_0\left|\right|\hfill \end{array}$$

(29)

Let ${\nu }^{\ast }=1_N\otimes {\dot{\xi }}_0-\Psi \left(\xi \right)$. We can obtain the following:

$$\begin{array}{cc}\hfill {\dot{V}}_1& \le -\lambda \parallel \xi -{\xi }^{\ast }{\parallel }^2\hfill \\ \hfill & +2p_{max}\parallel \xi -{\xi }^{\ast }\parallel ·\parallel \Gamma \left(\xi \right)+\nu +{\nu }^{\ast }-1_N\otimes {\dot{\xi }}_0\parallel \hfill \\ \hfill & \le -\lambda \parallel \xi -{\xi }^{\ast }{\parallel }^2+2p_{max}\parallel \xi -{\xi }^{\ast }\parallel ·\parallel \nu -{\nu }^{\ast }\parallel \hfill \\ \hfill & \le -\lambda \parallel \xi -{\xi }^{\ast }{\parallel }^2+{\parallel \xi -{\xi }^{\ast }\parallel }^2\hfill \\ \hfill & +p_{max}^2\parallel \xi -{\xi }^{\ast }\parallel ·\parallel \nu -{\nu }^{\ast }\parallel \hfill \end{array}$$

(30)

Defining ${\epsilon }_2=\nu -{\nu }_d=\nu -{\nu }^{\ast }$, it can be deduced that

$${\dot{\epsilon }}_2=(Q\otimes I_3)\nu -\alpha \otimes {\dot{\xi }}_0+\dot{\nu }-1_N\otimes {\ddot{\xi }}_0$$

(31)

We propose a Lyapunov function for the second part of the system:

$$V_2={\displaystyle \frac{1}{2}}{\epsilon }_2^T{\epsilon }_2$$

(32)

The derivative of $V_2$ is

$$\begin{array}{cc}\hfill {\dot{V}}_2& ={(\nu -{\nu }^{\ast })}^T(u-{\dot{\nu }}^{\ast })\hfill \end{array}$$

(33)

If

$$\begin{array}{cc}\hfill u& ={\dot{\nu }}^{\ast }-c(\nu -{\nu }^{\ast })\hfill \\ \hfill & =1_N\otimes {\ddot{\xi }}_0-\dot{\Psi }\left(\xi \right)-k(\Psi \left(\xi \right)+\nu -1_N\otimes {\dot{\xi }}_0)\hfill \end{array}$$

(34)

then

$$\begin{array}{cc}\hfill {\dot{V}}_2& =-c{(\nu -{\nu }^{\ast })}^T(\nu -{\nu }^{\ast })\le 0\hfill \end{array}$$

(35)

where c is a positive real number. Choose the total Lyapunov function

$$V_1+V_2={\epsilon }_1^T(P\otimes I_3){\epsilon }_1+{\displaystyle \frac{1}{2}}{\epsilon }_2^T{\epsilon }_2\ge 0$$

(36)

Since $P\otimes I_3$ is a positive definite matrix, then ${\lambda }_{max}(P\otimes I_3)>0$, which implies that

$$\begin{array}{c}\hfill {\epsilon }_1^T(P\otimes I_3){\epsilon }_1\le {\lambda }_{max}(P\otimes I_3){\epsilon }_1^T{\epsilon }_1\end{array}$$

(37)

As a result

$$\begin{array}{cc}\hfill V_1+V_2& \le {\lambda }_{max}(P\otimes I_3){\epsilon }_1^T{\epsilon }_1+{\displaystyle \frac{1}{2}}{\epsilon }_2^T{\epsilon }_2\hfill \\ \hfill & \le {\lambda }_1({\epsilon }_1^T{\epsilon }_1+{\epsilon }_2^T{\epsilon }_2)\hfill \end{array}$$

(38)

where ${\lambda }_1=max\{{\lambda }_{max}(P\otimes I_3),{\displaystyle \frac{1}{2}}\}$. At the same time, we have

$$\begin{array}{cc}\hfill {\dot{V}}_1+{\dot{V}}_2& ={(\xi -{\xi }^{\ast })}^T(P\otimes I_3)(\nu -1_N\otimes {\dot{\xi }}_0)\hfill \\ \hfill & +{(\nu -1_N\otimes {\dot{\xi }}_0)}^T(P\otimes I_3)(\xi -{\xi }^{\ast })\hfill \\ \hfill & +{(\nu -{\nu }^{\ast })}^T(u-{\dot{\nu }}^{\ast })\hfill \\ \hfill & \le -\lambda \parallel \xi -{\xi }^{\ast }{\parallel }^2+{\parallel \xi -{\xi }^{\ast }\parallel }^2\hfill \\ \hfill & +p_{max}^2\parallel \xi -{\xi }^{\ast }\parallel ·\parallel \nu -{\nu }^{\ast }\parallel \hfill \\ \hfill & -c{(\nu -{\nu }^{\ast })}^T(\nu -{\nu }^{\ast })\hfill \end{array}$$

(39)

By Lemma 2, $p_{max}^2\parallel \xi -{\xi }^{\ast }\parallel ·\parallel \nu -{\nu }^{\ast }\parallel$ can be calculated by the following:

$$\begin{array}{cc}\hfill p_{max}^2\left|\right|\xi -{\xi }^{\ast }\left|\right|·& \left|\right|\nu -{\nu }^{\ast }\left|\right|\hfill \\ & \le {\displaystyle \frac{p_{max}^2\gamma }{2}}\left|\right|\xi -{\xi }^{\ast }{\left|\right|}^2+{\displaystyle \frac{{\gamma }^{-1}{p}_{max}^2}{2}}\left|\right|\nu -{\nu }^{\ast }{\left|\right|}^2\hfill \end{array}$$

(40)

where $\gamma$ is a positive number. By substituting (40) into (39), we obtain

$$\begin{array}{cc}\hfill {\dot{V}}_1+{\dot{V}}_2\le & -(\lambda -1-{\displaystyle \frac{p_{max}^2\gamma }{2}})\left|\right|\xi -{\xi }^{\ast }{\left|\right|}^2\hfill \\ \hfill & -(c-{\displaystyle \frac{{\gamma }^{-1}{p}_{max}^2}{2}})\left|\right|\nu -{\nu }^{\ast }{\left|\right|}^2\hfill \end{array}$$

(41)

Letting $\gamma ={\displaystyle \frac{\lambda }{p_{max}^2}}>0$, we obtain that

$$\begin{array}{c}\hfill {\dot{V}}_1+{\dot{V}}_2\le -({\displaystyle \frac{\lambda }{2}}-1)\left({\epsilon }_1^T{\epsilon }_1\right)-(c-{\displaystyle \frac{p_{max}^4}{2\lambda }})\left({\epsilon }_2^T{\epsilon }_2\right)\end{array}$$

(42)

Since the parameters c and $\lambda$ can be adjusted by modifying the controller gains and the matrix P, if selecting appropriate values to satisfy $c\ge {\displaystyle \frac{p_{max}^4}{2\lambda }}$ and $\lambda \ge 2$, it can be concluded that

$$\begin{array}{c}\hfill {\dot{V}}_1+{\dot{V}}_2\le -{\lambda }_2({\epsilon }_1^T{\epsilon }_1+{\epsilon }_2^T{\epsilon }_2)\end{array}$$

(43)

where ${\lambda }_2=min\left\{{\displaystyle \frac{\lambda }{2}}-1,c-{\displaystyle \frac{p_{max}^4}{2\lambda }}\right\}>0$. This, combined with (38), leads to the following:

$$\begin{array}{c}\hfill {\dot{V}}_1+{\dot{V}}_2\le -{\lambda }_2({\epsilon }_1^T{\epsilon }_1+{\epsilon }_2^T{\epsilon }_2)\le -{\displaystyle \frac{{\lambda }_2}{{\lambda }_1}}(V_1+V_2)\end{array}$$

(44)

Defining ${\lambda }_3={\displaystyle \frac{{\lambda }_2}{{\lambda }_1}}>0$, we arrive at the conclusion:

$$\begin{array}{c}\hfill {\dot{V}}_1+{\dot{V}}_2\le -{\lambda }_3(V_1+V_2)\end{array}$$

(45)

Since $V_1+V_2$ is positive definite and ${\dot{V}}_1+{\dot{V}}_2$ is negative definite, we determine that the system is exponentially stable. It follows that ${\epsilon }_1,{\epsilon }_2\to 0$ as $t\to \infty$, $\xi \to {\xi }^{\ast }$ while $t\to \infty$. Through the above derivation, it can be concluded that under the given control law, the UAV cluster will gradually converge to the Nash equilibrium state. □

7. Simulation and Validation

We utilize MATLAB R2023b as our simulation platform to validate the effectiveness of the proposed control algorithm.

In our simulation, we set $N=4$, representing a cluster of four UAVs. These UAVs are designed to operate in a coordinated manner, following a predetermined overall path. The desired path ${\xi }_0$ is configured as a circular trajectory. This choice of trajectory allows us to evaluate the algorithm’s performance in a continuous and repetitive pattern.

To visualize the formation and coordination of the four UAVs, we present a schematic diagram (Figure 2), which depicts their relative positions and orientations within the formation. 1, 2, 3, and 4 are their respective identification numbers, corresponding to the first, second, third, and fourth UAVs. The UAVs are arranged in a specific pattern, which ensures optimal coverage and coordination during flight.

When the network connectivity relationships between the UAVs are established, the values of the adjacency matrix for the UAV connection graph can be determined. This matrix serves as a fundamental mathematical representation of the connectivity and information flow between the UAVs.

The adjacency matrix for the UAV connection graph is defined as follows:

$$A=\left[\begin{array}{cccc}0& 1& 0& 1\\ 1& 0& 1& 0\\ 0& 1& 0& 1\\ 1& 0& 1& 0\end{array}\right]$$

(46)

To accurately quantify the pose differences among the UAVs within a network, we introduce a matrix representation called the pose difference matrix, which denoted by $D^{\ast }$, provides a structured view of the variations in position and orientation among the UAVs. The matrix $D^{\ast }$ is structured as follows:

$$D^{\ast }=\left[\begin{array}{cccc}{d}_{11}& d_{12}& d_{13}& d_{14}\\ d_{21}& d_{22}& d_{23}& d_{24}\\ d_{31}& d_{32}& d_{33}& d_{34}\\ d_{41}& d_{42}& d_{43}& d_{44}\end{array}\right]=\left[\begin{array}{cccc}0& 10& 0& 10\\ 0& 0& 10& 10\\ -10& 0& -10& 0\\ 0& 0& 10& 10\\ 0& 10& 0& 10\\ -10& -10& 0& 0\\ -10& 0& -10& 0\\ -10& -10& 0& 0\end{array}\right]$$

(47)

To accurately determine the pose differences between the UAVs and the reference path points, we first establish a matrix representation of these differences. Specifically, we define the matrix $P^{\ast }={[p_1^T,p_2^T,p_3^T,p_4^T]}^T$, where $p_i^T$ represents the transpose of the pose difference vector for the i-th UAV with respect to the corresponding reference path point. In this example, $P^{\ast }={[-5,-5,5,-5,-5,5,5,5]}^T$.

We utilize the mathematical formulation represented by Equation (14) to calculate the Nash equilibrium trajectory for a formation of four UAVs, which symbolizes a state where the actions of each UAV are balanced with those of the others, resulting in a stable formation configuration.

By applying Equation (14), we determine the optimal trajectories for each UAV in the formation. The results of this calculation are illustrated in Figure 3, where the Nash equilibrium trajectories of the four UAVs are displayed, showing how they coordinate their movements to follow the desired path while maintaining the required formation.

In Figure 3, the prominent solid black line serves as a representation of the intended path that the UAV formation is required to follow. Consequently, the calculated Nash equilibrium trajectories for each individual UAV within the formation are precisely depicted. Specifically, the Nash equilibrium trajectory for UAV 1 is illustrated in red, UAV 2 in pink, UAV 3 in green, and UAV 4 in blue.

Additionally, we introduce the disturbance $\kappa =[{\kappa }_1;{\kappa }_2]=[4 \times sin(t+1);3 \times cos\left(2t\right)]$, which represents a significant disturbance with relatively large amplitude and varying speed. It can be observed that during the period when the disturbance is acting, the estimated value $\widehat{\kappa }$ from the homogeneous differential disturbance observer rapidly converges to the actual value $\kappa$. By incorporating this estimated disturbance value $\widehat{\kappa }$ into the control strategy by subtracting it from the control input $u_i=[u_{i,1};u_{i,2}]$, we effectively counteract the disturbance’s detrimental effects on the system. As evident from the subsequent graphical representation Figure 4, the homogeneous differential disturbance observer plays a pivotal role in minimizing the disturbance’s impact, rendering its influence on the system virtually negligible.

Upon activation of the control signal (23), it begins to influence the four UAVs. After a brief period of adjustment and alignment, the UAVs successfully achieve a cohesive formation. This unified entity then proceeds to follow the predefined desired trajectory. As depicted in Figure 5 and Figure 6, the UAVs maintain their relative positions within the formation, ensuring synchronized movement along the intended path.

The control system diagram is shown in Figure 7.

After careful analysis of the collected data, we conclude that a fixed-wing UAV formation will reach the desired Nash equilibrium of the specified non-cooperative game within a set time, assuming its initial state is reasonable. Once the Nash equilibrium is achieved, the formation will maintain its structure and adhere to the planned trajectory, ensuring smooth and coordinated flight.

8. Conclusions

Given the widespread application of fixed-wing UAV formations and the communication and control challenges encountered in the process, this paper proposes a distributed fixed-wing UAV formation control method based on non-cooperative game theory. Firstly, we assumed a planar task environment and established a UAV model based on this assumption. We further transformed this model into a double integrator form, which reduces the complexity of controller design. Next, we implemented a homogeneous differential disturbance observer based on the UAV model to enhance the UAV’s robustness against disturbances generated by the system itself during operation. Considering the two objectives in UAV formation tasks: maintaining formation and following the mission trajectory, we designed a cost function that balances both aspects. The UAV’s goal is to minimize its own cost function. Therefore, utilizing non-cooperative game theory, we identified the Nash equilibrium solution. Subsequently, we employed the backstepping method to design an appropriate controller, which enables the states of the multi-UAV system to ultimately converge to the Nash equilibrium point. Finally, simulations were conducted to demonstrate the feasibility of the proposed algorithm.