A Situation Assessment Method with an Improved Fuzzy Deep Neural Network for Multiple UAVs

: To improve the intelligence and accuracy of the Situation Assessment (SA) in complex scenes, this work develops an improved fuzzy deep neural network approach to the situation assessment for multiple Unmanned Aerial Vehicle(UAV)s. Firstly, this work normalizes the scene data based on time series and use the normalized data as the input for an improved fuzzy deep neural network. Secondly, adaptive momentum and Elastic SGD (Elastic Stochastic Gradient Descent) are introduced into the training process of the neural network, to improve the learning performance. Lastly, in the real-time situation assessment task for multiple UAVs, conventional methods often bring inaccurate results for the situation assessment because these methods don’t consider the fuzziness of task situations. This work uses an improved fuzzy deep neural network to calculate the results of situation assessment and normalizes these results. Then, the degree of trust of the current result, relative to each situation label, is calculated with the normalized results using fuzzy logic. Simulation results show that the proposed method outperforms competitors.


Introduction
As a new type of advanced equipment, the unmanned aerial vehicle (UAV) can achieve independent flight and investigation tasks and it has attracted the attention of many scholars [1][2][3].For some military missions, UAV has been proved to have potential value due to its excellent performance.Recently, UAV technology has achieved great progress due to the development of computer science and communication technology [4].It is widely known that multiple UAVs can finish some cooperative tasks or military confrontation by making a decision under uncertain environment [5].Meanwhile, previous works have indicated that the most critical premise of decision-making is situation assessment [6,7].
The intelligent decision-making technology for the intelligent robot system mainly includes three components: environmental perception module, situation assessment module, and behavior planning module.The framework of intelligent decision-making of the intelligent robot systems is shown in Figure 1.The perception module can acquire and represent environment information.The situation assessment module analyzes the environmental information and makes the high-level behavior strategy.Finally, the behavior planning module produces specific actions.Therefore, situation assessment is the basis of intelligent decision-making for robots, and it is an essential component for robots to conduct collaborative tasks [8].The situation assessment system can give a complete high-level description for the current situation according to the objective model and the task scenario, which is usually used to describe the relationship between entities and events.In addition, the situation assessment system can be extended to achieve the effective prediction of scene situation information.The existing theories of situation assessment include multi-attribute decision-making [9], fuzzy set [10], neural network [11] and Bayesian network [12].
Information 2020, 11,194 2 of 20 complete high-level description for the current situation according to the objective model and the task scenario, which is usually used to describe the relationship between entities and events.In addition, the situation assessment system can be extended to achieve the effective prediction of scene situation information.The existing theories of situation assessment include multi-attribute decisionmaking [9], fuzzy set [10], neural network [11] and Bayesian network [12].The situation assessment of multiple UAVs system refers to the classification of the superiority level for situation assessment according to current multiple factors, including the combat effectiveness of both sides and the degree of superiority.The situation assessment for multiple UAVs is the premise of intelligent decision-making and collaborative confrontation for a UAV group, so scholars have done a lot of research on this.The two renowned methods are the fuzzy logic method [13] and BP neural network method [14].There are some problems with the situation assessment method based on fuzzy logic.First of all, artificial subjective factors have great influence, which may lead to poor adaptability and flexibility of a decision-making system.Secondly, the assessment deviation is large.Due to too many calculation steps including subjective consciousness, the accuracy of an assessment for a new task is not enough.For the situation assessment method based on neural network, the conventional neural network model focuses on the layer closing to the output layer to correct the error, which will lead to the insufficient assessment for a real-time task.
The latest development of machine learning technology makes it possible to achieve the more advanced decision-making system using the perceptual data, which provides an opportunity for multiple UAVs to improve the performance of the situation assessment in a dynamic environment.Recently, deep reinforcement learning combines the superiorities of reinforcement learning and deep learning, so it can help a learning agent to perform well in high-dimensional task space, with an endto-end approach to achieve a complex task combining perception and decision together [15].AlphaGo robot masters the game of Go to finish an advanced situation assessment and real-time decision-making by self-play without expert experience [16].Genetic Fuzzy Trees uses genetic methods to optimize the multiple cascading fuzzy systems and this method has been proved to have excellent performance in the complex spatiotemporal environments [17].These fuzzy controllers based on the Genetic Fuzzy Trees can even accomplish some new and different combat tasks efficiently.[18] proposed a Deep Q-Network method for multi-agent confrontation and cooperative tasks in video games.The game agent uses a multi-agent DQN algorithm with a classical rewarding scheme, to learn a policy mapping a state to competitive or collaborative behavior.
The deep neural network is based on an artificial neural network [19].Its most obvious feature is that it contains multiple hidden layers and addresses the problem of the sparse correction gradient of an artificial neural network model.The deep neural network can better mine the distributed features and implicit abstract features of the data sample by combining the shallow features into more abstract high-level features [20].Recently, the deep neural network method has been widely used in situation assessment scenarios.For example, in [21], a CNN model is proposed to fit the advanced comprehension process of the battlefield situation, so as to achieve the simulation of the commander's The situation assessment of multiple UAVs system refers to the classification of the superiority level for situation assessment according to current multiple factors, including the combat effectiveness of both sides and the degree of superiority.The situation assessment for multiple UAVs is the premise of intelligent decision-making and collaborative confrontation for a UAV group, so scholars have done a lot of research on this.The two renowned methods are the fuzzy logic method [13] and BP neural network method [14].There are some problems with the situation assessment method based on fuzzy logic.First of all, artificial subjective factors have great influence, which may lead to poor adaptability and flexibility of a decision-making system.Secondly, the assessment deviation is large.Due to too many calculation steps including subjective consciousness, the accuracy of an assessment for a new task is not enough.For the situation assessment method based on neural network, the conventional neural network model focuses on the layer closing to the output layer to correct the error, which will lead to the insufficient assessment for a real-time task.

Environment
The latest development of machine learning technology makes it possible to achieve the more advanced decision-making system using the perceptual data, which provides an opportunity for multiple UAVs to improve the performance of the situation assessment in a dynamic environment.Recently, deep reinforcement learning combines the superiorities of reinforcement learning and deep learning, so it can help a learning agent to perform well in high-dimensional task space, with an end-to-end approach to achieve a complex task combining perception and decision together [15].AlphaGo robot masters the game of Go to finish an advanced situation assessment and real-time decision-making by self-play without expert experience [16].Genetic Fuzzy Trees uses genetic methods to optimize the multiple cascading fuzzy systems and this method has been proved to have excellent performance in the complex spatiotemporal environments [17].These fuzzy controllers based on the Genetic Fuzzy Trees can even accomplish some new and different combat tasks efficiently.In [18] proposed a Deep Q-Network method for multi-agent confrontation and cooperative tasks in video games.The game agent uses a multi-agent DQN algorithm with a classical rewarding scheme, to learn a policy mapping a state to competitive or collaborative behavior.
The deep neural network is based on an artificial neural network [19].Its most obvious feature is that it contains multiple hidden layers and addresses the problem of the sparse correction gradient of an artificial neural network model.The deep neural network can better mine the distributed features and implicit abstract features of the data sample by combining the shallow features into more abstract high-level features [20].Recently, the deep neural network method has been widely used in situation assessment scenarios.For example, in [21], a CNN model is proposed to fit the advanced comprehension process of the battlefield situation, so as to achieve the simulation of the commander's thinking process.However, this situation assessment method based on a CNN model only focuses on the situation comprehension module, and the situation prediction stage is achieved using the expert experience.This method will limit the situation assessment process to expert experience.In [22], an Auto-encoder model is proposed to describe the relationship between the performance metrics of the situation assessment system for multiple UAVs.However, this method requires a large number of training data, and training data need to be preprocessed using the expert experience.The preprocessing process will lead to a lot of time costs.
In summary, two critical issues have to be considered in situation assessment problems.Firstly, in real life, for the results of the situation assessment of a task, there is often no definite situation assessment.For example, to complete a situation assessment of one team in the football field, it is too rough to give the current situation with "strengths" or "weaknesses", and rough assessment will lead to an inaccurate real-time evaluation of the scene.Secondly, the conventional deep neural network model for situation assessment often has poor learning performance [23].The optimization method based on the training process may be a solution to improve the performance of deep neural networks.
Previous works have the following problems.First of all, the previous situation assessment technology for multiple UAVs often does not consider that the multi-objective decision-making environment is a dynamic and uncertain environment, so the rough results of conventional methods cannot get accurate situation assessment results.Secondly, the performance of the deep neural network for the situation assessment of multiple UAVs is often limited by the training process of the neural networks.Moreover, conventional methods often focus on situation comprehension or situation awareness, while for situation prediction, the method based on expert experience is used to get results.Two conventional situation assessment methods for end-to-end prediction are the fuzzy logic method and BP neural network method.However, these two methods have some disadvantages.A more practical situation assessment method is lacking.
To address these problems, this paper develops a situation assessment method for multiple UAVs, using an improved fuzzy deep neural network.The proposed situation assessment method achieves an end-to-end mapping from environmental information to assessment results.The normalized situation value is the input of the improved fuzzy deep neural network, and the output of the network is the degree of trust for each situation label.The advantage of the fuzzy deep neural network is that the output situation assessment result has a certain fuzziness, which is more consistent with the uncertainty of the real-time system.At the same time, the deep neural network realizes a deeper feature representation, which can achieve more accurate assessment results.Secondly, in order to accelerate the convergence rate of the deep neural networks, adaptive momentum [24] and Elastic SGD (Elastic Stochastic Gradient Descent) [25] are introduced into the training process of the neural networks.The simulation results verify the effectiveness of the proposed method.
The contributions of this paper are as follows: In this paper, an improved fuzzy deep neural network method for situation assessment of multiple UAVs is proposed.An end-to-end mapping is achieved without requiring too much expert experience in the situation prediction process.Firstly, this method uses the advantages of deep feature representation of the deep neural network.Meanwhile, we use fuzzy logic to calculate the degree of trust in the output assessment results.Secondly, we introduce adaptive momentum and elastic SGD to accelerate the training speed of the deep neural network.Efficient learning can be more in line with the real-time situation assessment system.
The remainder of this work is organized as follows: Section 2 introduces the background of this work, the general process of situation assessment and the neural network model.Section 3 gives a kinematics model of UAV, which provides the UAV motion rules for the multiple UAVs situation assessment experiment.Section 4 introduces two baseline methods for the situation assessment of multiple UAVs, which are the fuzzy evaluation method and the classical BP neural network method.Section 5 introduces the proposed situation assessment method with an improved fuzzy deep neural network for multiple UAVs.Section 6 gives the simulation results calculated by the proposed method.Conclusions are drawn in the last section.

Situation Assessment
Generally speaking, in a complete decision-making system, a situation comprehension is an important component of the situation evaluation.The situation assessment method estimates the situation effectively and then determines different strategies for different situations using the previous experience, such as attack or defense [26].
Previous work divides situation assessment into three phases: situation awareness, situation comprehension, and situation forecast, which are progressive [27], as shown in Figure 2. Firstly, according to the environmental information, the scene data is recorded by sensors, which is called situational awareness.Then, in the situation understanding phase, the current situation is evaluated after the processing of the real-time data for the current scene.Finally, an analysis of the current situation for the situation information with continuous scene data has been conducted to provide a more accurate decision-making result for the subsequent situation assessment, that is, the situation prediction process.Generally, the basic procedure for the situation assessment is shown in Algorithm 1.

Situation Assessment
Generally speaking, in a complete decision-making system, a situation comprehension is an important component of the situation evaluation.The situation assessment method estimates the situation effectively and then determines different strategies for different situations using the previous experience, such as attack or defense [26].
Previous work divides situation assessment into three phases: situation awareness, situation comprehension, and situation forecast, which are progressive [27], as shown in Figure 2. Firstly, according to the environmental information, the scene data is recorded by sensors, which is called situational awareness.Then, in the situation understanding phase, the current situation is evaluated after the processing of the real-time data for the current scene.Finally, an analysis of the current situation for the situation information with continuous scene data has been conducted to provide a more accurate decision-making result for the subsequent situation assessment, that is, the situation prediction process.Generally, the basic procedure for the situation assessment is shown in Algorithm 1.

Algorithm 1. The basic procedure for situation assessment
Step 1: Deduce the knowledge base of situation assessment using the expert experience in situation assessment; Step 2: Obtain the complete knowledge map, that is, the knowledge representation via the classification and analysis of domain knowledge base; Step 3: Record the real-time data for the real scene using the knowledge acquisition module and recorded information which is stored in the constructed knowledge base; Step 4: Describe the rule knowledge as rules that can be recognized by the system and stored in the knowledge base according to the rule editor provided by the knowledge acquisition module, such as the data format given by the data acquisition module; Step 5: Answer the real-time feedback question in the knowledge module using the expert experience.The answer to the feedback question is directly sent to the knowledge acquisition module or processed and then sent to the knowledge acquisition module, waiting for a certain behavior decision.

Situation forecast decision making execution
Constraint conditions Algorithm 1.The basic procedure for situation assessment Step 1: Deduce the knowledge base of situation assessment using the expert experience in situation assessment; Step 2: Obtain the complete knowledge map, that is, the knowledge representation via the classification and analysis of domain knowledge base; Step 3: Record the real-time data for the real scene using the knowledge acquisition module and recorded information which is stored in the constructed knowledge base; Step 4: Describe the rule knowledge as rules that can be recognized by the system and stored in the knowledge base according to the rule editor provided by the knowledge acquisition module, such as the data format given by the data acquisition module; Step 5: Answer the real-time feedback question in the knowledge module using the expert experience.
The answer to the feedback question is directly sent to the knowledge acquisition module or processed and then sent to the knowledge acquisition module, waiting for a certain behavior decision.

Neural Network Model
Figure 3 gives the basic structure for the neural network.The neural network is a multilayer feed-forward neural network model, which consists of three parts: an input layer, a hidden layer, and an output layer.Each part contains an indefinite number of layers, and each layer contains one or more nonlinear neural units.

Neural Network Model
Figure 3 gives the basic structure for the neural network.The neural network is a multilayer feed-forward neural network model, which consists of three parts: an input layer, a hidden layer, and an output layer.Each part contains an indefinite number of layers, and each layer contains one or more nonlinear neural units.The connection weights of the BP neural network are updated by the back-propagation algorithm [28], which includes two processes: the signal forward transmission and the error backpropagation.In the process of the forward transmission, the input signal is input from the input layer, and then processed layer by layer through the hidden layer until it reaches the output layer, and finally, the output result is calculated in the output layer.The state of each neuron of a layer in the deep network is only affected by the state of the neurons of the previous layer, but it won't be affected by the state of the neurons of other layers.If there is a gap between the desired result and the actual result for the neural network, the error signal will be sent back to each layer via back-propagation to modify the network weights.Previous works have demonstrated that the neural network model can estimate any nonlinear function by learning a large number of learning samples [29].
Deep Neural Network (DNN) uses multiple layers of neuron units to achieve feature extraction, which is a very common machine learning method.Recently, the research of deep neural networks comes from artificial neural networks [30].Deep Neural Network can combine the low-level features into high-level features, that represent features at a more abstract level.The similarities between deep neural networks and artificial neural networks are that they both adopt a hierarchical structure.The system consists of three parts: the input layer, an output layer, and a hidden layer.The nodes between adjacent layers are connected, and the nodes of the same layer or cross-layer are not connected.Each layer can be regarded as a logistic regression model [31].Figure 4 shows the basic structure for the deep neural network, which is similar to the working principle of the human brain.It is a significant function for the DNN to learn multi-level representation corresponding to different levels of abstraction.The connection weights of the BP neural network are updated by the back-propagation algorithm [28], which includes two processes: the signal forward transmission and the error back-propagation.In the process of the forward transmission, the input signal is input from the input layer, and then processed layer by layer through the hidden layer until it reaches the output layer, and finally, the output result is calculated in the output layer.The state of each neuron of a layer in the deep network is only affected by the state of the neurons of the previous layer, but it won't be affected by the state of the neurons of other layers.If there is a gap between the desired result and the actual result for the neural network, the error signal will be sent back to each layer via back-propagation to modify the network weights.Previous works have demonstrated that the neural network model can estimate any nonlinear function by learning a large number of learning samples [29].

Output layer
Deep Neural Network (DNN) uses multiple layers of neuron units to achieve feature extraction, which is a very common machine learning method.Recently, the research of deep neural networks comes from artificial neural networks [30].Deep Neural Network can combine the low-level features into high-level features, that represent features at a more abstract level.The similarities between deep neural networks and artificial neural networks are that they both adopt a hierarchical structure.The system consists of three parts: the input layer, an output layer, and a hidden layer.The nodes between adjacent layers are connected, and the nodes of the same layer or cross-layer are not connected.Each layer can be regarded as a logistic regression model [31].Figure 4 shows the basic structure for the deep neural network, which is similar to the working principle of the human brain.It is a significant function for the DNN to learn multi-level representation corresponding to different levels of abstraction.

The Kinematic Model for UAV
In this mission, the flight altitude of each UAV is constant, that is, each UAV has been flying at a fixed altitude.In the process of flight for each UAV, the ground radar should be taken as the reference position.The movement model of each UAV is shown below.
The coordinate for radar is ( ) . i r is the distance from the i-th UAV to the radar.For the i-th UAV, i θ is the pitch angle, i ϕ is the azimuth angle, i β is the heading angle, and i h is the flight altitude.The motion relationship of UAV in the Cartesian Coordinate System is shown in Figure 5.The kinematic model of UAV flying at a fixed altitude is shown in Equation (1).sin cos 0

The Kinematic Model for UAV
In this mission, the flight altitude of each UAV is constant, that is, each UAV has been flying at a fixed altitude.In the process of flight for each UAV, the ground radar should be taken as the reference position.The movement model of each UAV is shown below.
The coordinate for radar is (x r , y r , z r ).r i is the distance from the i-th UAV to the radar.For the i-th UAV, θ i is the pitch angle, ϕ i is the azimuth angle, β i is the heading angle, and h i is the flight altitude.The motion relationship of UAV in the Cartesian Coordinate System is shown in Figure 5.The kinematic model of UAV flying at a fixed altitude is shown in Equation (1).

The Kinematic Model for UAV
In this mission, the flight altitude of each UAV is constant, that is, each UAV has been flying at a fixed altitude.In the process of flight for each UAV, the ground radar should be taken as the reference position.The movement model of each UAV is shown below.
The coordinate for radar is ( ) . i r is the distance from the i-th UAV to the radar.For the i-th UAV, i θ is the pitch angle, i ϕ is the azimuth angle, i β is the heading angle, and i h is the flight altitude.The motion relationship of UAV in the Cartesian Coordinate System is shown in Figure 5.The kinematic model of UAV flying at a fixed altitude is shown in Equation (1).sin cos 0 The derivative of Equation ( 2) is given by, The expression for Equation (3) using the matrix is given by, According to Equations ( 2)-( 4), with radar as the coordinate origin, the kinematics model of UAV in the Polar Coordinate System is shown in Equation (5).
According to Equation (1) and Equations ( 5) and ( 6) is given by, The kinematic parameters of the UAV are given by, Figure 6 shows the flight paths of two different UAVs.The UAV used in this experiment is based on the kinematics model of this section.

A Situation Assessment with BP Neural Network (SA-BP)
A situation assessment method with BP neural network is shown in Figure 7.

A Situation Assessment with BP Neural Network (SA-BP)
A situation assessment method with BP neural network is shown in Figure 7.

A Situation Assessment with BP Neural Network (SA-BP)
A situation assessment method with BP neural network is shown in Figure 7.

Algorithm 2. The conventional method with SA-BP
Step 1: Define a BP neural network model.Choose a reasonable activation function and define a neural network model from the input layer to the output layer.Meanwhile, the final assessment model can be obtained by combining it with a certain amount of data training.
Step 2: Define the input layer for the neural network.All the information obtained by the situational awareness module is used as a set for input neurons, which includes all scene information data in a certain period pushed forward from the current time.The scene data is recorded according to the evaluation factors.IN = {I i |i = 1, 2, . . ., m} represents n pieces of scene data which have been obtained from the task scene.
Step 3: Define a situation assessment set.In order to describe the scene situation reasonably, it is necessary to set the situation label in combination with the real-time situation.n situation labels are set and the set for situation label is given by, Step 4: Normalization of input data.By recording the data within a certain period, an approximate range of each evaluation factor can be obtained.If the minimum value in the recorded data of the evaluation factor I j is I min j and the maximum value is I max j , then the given value range I bo j , I to j is given by Equation ( 10) and the normalized data Îj for the evaluation factor I j is given by Equation (11).
Step 5: Achieve a real-time situation assessment.When the new data enters into the assessment network model, the result calculated by the neural network is a result vector for each situation label.Then, we take the output assessment corresponding to the maximum value as the final situation assessment result.

A Fuzzy Evaluation Method for Situation Assessment (SA-F)
Fuzzy evaluation is a method based on the theories of fuzzy mathematics to provide a reasonable evaluation of the actual tasks.A fuzzy evaluation based situation assessment method is to apply the fuzzy evaluation method to the assessment of specific complex scene situations.The procedure is shown in Algorithm 3.

Algorithm 3. The conventional method with SA-F
Step 1: Define the evaluation factor.Considering specific tasks and m evaluation factors obtained from the scene, the set for the evaluation factor is A = {a i |i = 1, 2, . . ., m}.
Step 2: Define the situation label.In order to describe the situation of the scene reasonably, we need to set the final situation label.The number of situation labels is n, which is B = {b i |i = 1, 2, . . ., n}.
Step 3: Define a fuzzy priority relation matrix.Firstly, the relative importance of each evaluation factor to the final situation assessment needs to be determined.Setting the expression of the relative importance of two factors a i and a j to d i j , we can get the Equation (12).
Then, the priority relationship matrix is given by D = d i j m×m .
Step 4: Define a fuzzy weight set.Combining the fuzzy analytic hierarchy process to transform the fuzzy matrix into the fuzzy consistent matrix D ∈ m×m .
Set an evaluation factor U in the in U -th row in the fuzzy consistent matrix D, then the fuzzy weight value of the evaluation factor relative to the final situation assessment is given by, where k ≥ 0.5 × (n − 1) is a constant and its specific value is given according to the actual task.Therefore, the final set of weights for the m evaluation factors can be obtained according to Equation ( 14) and it is recorded as ω = (ω 1 , ω 2 , . . ., ω m ) T (15) Step 5: Achieve a fuzzy situation assessment.If the current time is t, the value for the degree of trust of each evaluation factor relative to the situation described in the first T time slices is calculated using the fuzzy method, which is given by η : → f d j ⇒ ϕ j , as shown in: The final fuzzy situation assessment result using the "maximum-minimum operation" [29] is calculated, which is represented by a fuzzy vector M = ω × η = m j ∈ 1×n as shown in: where ∧ represents the smallest value in the set.∨ represents the largest value in the set.
Step 6: Achieve a comprehensive assessment result.According to the maximum operation, a comprehensive assessment result of the current situation which is represented as Z, can be obtained, which is given by,

The Framework for the Proposed Method
The basic framework of the proposed method is shown in Figure 8. Multi-UAVs system obtains current scene data in the environment using the situation awareness technology.In Figure 8, scene data is input from the input layer.Then, we normalize the input data in the normalization layer.
An improved deep neural network is used to calculate the output results, that is, the results of the situation assessment.In the fuzzy layer, we achieve the fuzzification of the output value and calculate the degree of truth for the situation assessment results.The output is the degree of truth of each situation label for multiple UAVs.In the training process for the deep neural network, we use the adaptive momentum and Elastic SGD to improve the performance of the neural network and this method can accelerate the convergence rate of the deep neural network.
Information 2020, 11,194 11 of 20 situation assessment.In the fuzzy layer, we achieve the fuzzification of the output value and calculate the degree of truth for the situation assessment results.The output is the degree of truth of each situation label for multiple UAVs.In the training process for the deep neural network, we use the adaptive momentum and Elastic SGD to improve the performance of the neural network and this method can accelerate the convergence rate of the deep neural network.

An Improved Deep Neural Network Model with Adaptive Momentum and Elastic SGD
The input of the neural network is a data vector with m samples: , and the labels of output are: . The error between the actual output of the network and the label output vector is used to update these weights so that the current output O is close to the desired value O ˆ.According to the decline of the slope of the error function, the weight and the deviation are determined, which makes the error function change towards the direction of the desired output, and transmit it to other layers in this way.The basic procedure for the neural network model with adaptive momentum and Elastic SGD is as follows.
The output n y of the hidden layer is given by,

(
) ( ) The output n O of the output layer is given by,
The error function is given by,

An Improved Deep Neural Network Model with Adaptive Momentum and Elastic SGD
The input of the neural network is a data vector with m samples: I = {I i |i = 1, 2, . . ., m}, and the labels of output are: Ô = Ôi i = 1, 2, . . ., n .The error between the actual output O = {O i |i = 1, 2, . . ., n} of the network and the label output vector Ô = Ôi i = 1, 2, . . ., n is used to update these weights so that the current output O is close to the desired value Ô.According to the decline of the slope of the error function, the weight and the deviation are determined, which makes the error function change towards the direction of the desired output, and transmit it to other layers in this way.The basic procedure for the neural network model with adaptive momentum and Elastic SGD is as follows.
The output y n of the hidden layer is given by, The output O n of the output layer is given by, where f (x) = 2E 1+e −Fx − F is the activation function.E = 1.812 and F = 0.676 are constant.The error function is given by, Step 2: Back-propagation.
We use the gradient descent method to calculate the error for the weights of the output layer.The updating law with adaptive momentum for the weights from the i-th input to the k-th output is given by, where δ ki = Ôk − O t k f .So, Equation ( 22) can be deduced.
We use the gradient descent method to calculate the error for the weights of the hidden layer.The updating law with adaptive momentum for the weights from the j-th input to the i-th output is given by, where In practice, the weight update is related to t, and the weight update is determined by its trend of the gradient change to avoid the slow convergence of the neural network.The updating law with adaptive momentum and Elastic SGD for the weight update is obtained by Equation (25).
Then, the weight update at the time t + 1 is given by ω t+1 = ∆ t ω + ω t .The updating rule for ∆ t ω is given by, where 0 < ε < 1 < µ.When two update processes are in the same direction, the value ∆ t ω is increased.But when the two update processes are in different directions, the value ∆ t ω is reduced.Other situations do not change the value ∆ t ω.

The Whole Algorithm Using An Improved Fuzzy Deep Neural Network for Situation Assessment of Multiple UAVs
The procedure for the proposed method is shown below.
Step 1: Define an improved deep neural network model.The initial weights of the hidden layer are assigned randomly.The output layer is a fuzzy layer.
Step 2: Define the input layer for the improved deep neural network.The current time is t and the length for a certain period is T. If the scene data at each time has n values, the scene data gave by Equation ( 27) from t − T + 1 to t is selected as the input.
round.In each round, we use two models to classify signals and record the results.We selected ten rounds of classification results for performance comparison in the test phase.The simulation results are shown in Figure 10.In Figure 10, the unit of ordinate is 10

Experiment on Multiple UAVs
In this paper, a confrontation scenario for multiple UAVs is set up to verify the effectiveness of the proposed method and the confrontation scene is shown in Figure 11.There are two sides: the red side and the blue side, with four UAVs respectively.The kinematic rules of motion for each UAV are described in the third section.The confrontation rules are as follows: Each UAV is given 100 health points before the start of the round.This article defines health points as life values, similar to agents in real-time strategy games.When the health point (HP) is 0, the UAV is eliminated.Each UAV attacks the opponent with a different energy, which will consume the HP of the UAV itself.One score is given when it hits the other side.

Experiment on Multiple UAVs
In this paper, a confrontation scenario for multiple UAVs is set up to verify the effectiveness of the proposed method and the confrontation scene is shown in Figure 11.There are two sides: the red side and the blue side, with four UAVs respectively.The kinematic rules of motion for each UAV are described in the third section.The confrontation rules are as follows: Each UAV is given 100 health points before the start of the round.This article defines health points as life values, similar to agents in real-time strategy games.When the health point (HP) is 0, the UAV is eliminated.Each UAV attacks the opponent with a different energy, which will consume the HP of the UAV itself.One score is given when it hits the other side.
side and the blue side, with four UAVs respectively.The kinematic rules of motion for each UAV are described in the third section.The confrontation rules are as follows: Each UAV is given 100 health points before the start of the round.This article defines health points as life values, similar to agents in real-time strategy games.When the health point (HP) is 0, the UAV is eliminated.Each UAV attacks the opponent with a different energy, which will consume the HP of the UAV itself.One score is given when it hits the other side.The labels of situation assessment for multiple UAVs are as follows: the difference between the average HP of the two sides, the difference between the remaining numbers of the two sides, the difference between the minimum remaining HP of the two sides and the difference between the scores of the two sides.The situation category is described by five descriptions for situation results, which are very unfavorable to us, relatively unfavorable to us, no effect on both sides, relatively favorable to us and very favorable to us.Firstly, a certain number of training data are collected from the experiment for training, and then the effectiveness of different situation assessment methods are verified by using the validation data.50 groups of scene data are used as validation data to verify the effectiveness of the method.Situation labels are set empirically as the standard result for comparison.Three methods of situation assessment (SA) are compared for multiple UAVs, which are the SA with The labels of situation assessment for multiple UAVs are as follows: the difference between the average HP of the two sides, the difference between the remaining numbers of the two sides, the difference between the minimum remaining HP of the two sides and the difference between the scores of the two sides.The situation category is described by five descriptions for situation results, which are very unfavorable to us, relatively unfavorable to us, no effect both sides, relatively favorable to us and very favorable to us.Firstly, a certain number of training data are collected from the experiment for training, and then the effectiveness of different situation assessment methods are verified by using the validation data.50 groups of scene data are used as validation data to verify the effectiveness of the method.Situation labels are set empirically as the standard result for comparison.Three methods of situation assessment (SA) are compared for multiple UAVs, which are the SA with the proposed method (SA-P), the SA with the BP model (SA-BP) and the SA with the Fuzzy method (SA-F).
The experimental results are shown in Figures 12-14.In the experiment, the horizontal axis represents the number of groups, and the vertical axis represents the situation assessment results.The three methods use an end-to-end approach to map environmental information to situation assessment results.However, the results of the situation assessment are different.From the experimental results, for the proposed method, the difference between the label results and the actual output results is relatively small.Meanwhile, the results of the situation assessment for the baseline methods are relatively low.In the experiment, the accuracy of the proposed method is 92%, that of method SA-BP is 84%, and that of method SA-F is 61%.The proposed method has the highest accuracy.SA-F method relies on expert experience, so it can not deal with different task scenarios well.SA-BP method is limited by the performance of the conventional neural network, and the immutable situation label cannot adapt to the continuously changing task environment.The proposed method uses the deep neural network to achieve a deeper feature representation and employs a fuzzy layer to make the output results more suitable for the real-time system.methods are relatively low.In the experiment, the accuracy of the proposed method is 92%, that of method SA-BP is 84%, and that of method SA-F is 61%.The proposed method has the highest accuracy.SA-F method relies on expert experience, so it can not deal with different task scenarios well.SA-BP method is limited by the performance of the conventional neural network, and the immutable situation label cannot adapt to the continuously changing task environment.The proposed method uses the deep neural network to achieve a deeper feature representation and employs a fuzzy layer to make the output results more suitable for the real-time system.method SA-BP is 84%, and that of method SA-F is 61%.The proposed method has the highest accuracy.SA-F method relies on expert experience, so it can not deal with different task scenarios well.SA-BP method is limited by the performance of the conventional neural network, and the immutable situation label cannot adapt to the continuously changing task environment.The proposed method uses the deep neural network to achieve a deeper feature representation and employs a fuzzy layer to make the output results more suitable for the real-time system.

Conclusions
In this paper, a situation assessment method for multiple UAVs using an improved fuzzy deep neural network is developed to improve the performance of the real-time situation assessment system.The conventional rule-based approach needs to focus on different tasks to set different situation assessment rules, which not only costs a lot of time but also is limited by specific tasks.Different from the traditional deep learning methods, the proposed method realizes the end-to-end situation assessment.In the process of situation assessment, the proposed method does not require

Conclusions
In this paper, a situation assessment method for multiple UAVs using an improved fuzzy deep neural network is developed to improve the performance of the real-time situation assessment system.The conventional rule-based approach needs to focus on different tasks to set different situation assessment rules, which not only costs a lot of time but also is limited by specific tasks.Different from the traditional deep learning methods, the proposed method realizes the end-to-end situation assessment.In the process of situation assessment, the proposed method does not require much expert experience, and will not be limited by the inflexible situation assessment results.Firstly, the normalized scene data is input into the neural network, and the corresponding output results are obtained through the cascade calculation of the neural network.Then, in the output layer, the output situation assessment results are fuzzed to get the degree of truth relative to each situation assessment label.In the training process of the neural network, adaptive momentum and elastic SGD are introduced into the back-propagation process to accelerate the learning speed of the assessment network.The proposed method is more suitable for a real-time situation assessment system because of the fuzziness and uncertainty of a real-time system.Simulation results show the effectiveness of the proposed method.
A learning agent with reinforcement learning modifies its behavior policy using the collected rewards while performing a task in the environment [32,33].The situation assessment method, based on reinforcement learning may be able to calculate the continuous situation assessment results using the predefined state and action space.In the future, we will investigate the application of reinforcement learning in a situation assessment system for multiple UAVs [34][35][36].More realistic task scenarios will be considered for execution in order to test the practicality of the proposed method.In the actual environment, we need to focus on the UAV's visual perception algorithm [37,38], flight planning method [39] and some emergency measures [40] in the face of some uncertain factors.This paper will also investigate a control system integrating the proposed situation assessment method to achieve a more advanced intelligent navigation strategy [41].

Figure 1 .
Figure 1.The basic structure for intelligent decision-making.

Figure 1 .
Figure 1.The basic structure for intelligent decision-making.

Figure 2 .
Figure 2. The basic structure for the situation assessment.

Figure 2 .
Figure 2. The basic structure for the situation assessment.

Figure 3 .
Figure 3.The basic structure for the neural network.

Figure 3 .
Figure 3.The basic structure for the neural network.

Figure 4 .
Figure 4.The basic structure for the deep neural network.

Figure 5 .
Figure 5.The kinematic model of Unmanned Aerial Vehicle (UAV).According to the transformation relationship between the Polar Coordinate System and the Cartesian Coordinate System, the kinematic model of UAV is transformed into a Polar Coordinate System.

Figure 4 .
Figure 4.The basic structure for the deep neural network.

Figure 4 .
Figure 4.The basic structure for the deep neural network.

Figure 5 .
Figure 5.The kinematic model of Unmanned Aerial Vehicle (UAV).According to the transformation relationship between the Polar Coordinate System and the Cartesian Coordinate System, the kinematic model of UAV is transformed into a Polar Coordinate System.

Figure 5 .
Figure 5.The kinematic model of Unmanned Aerial Vehicle (UAV).According to the transformation relationship between the Polar Coordinate System and the Cartesian Coordinate System, the kinematic model of UAV is transformed into a Polar Coordinate System.

Figure 6 .
Figure 6.The flight paths for two different UAVs.

Figure 7 .
Figure 7.The situation assessment method with Back Propagation (BP )neural networks.

Figure 6 .
Figure 6.The flight paths for two different UAVs.

Figure 7 .
Figure 7.The situation assessment method with Back Propagation (BP )neural networks.

Figure 7 .
Figure 7.The situation assessment method with Back Propagation (BP) neural networks.

Figure 8 .
Figure 8.The framework for the proposed method.

Figure 8 .
Figure 8.The framework for the proposed method.

Figure 9 .
Figure 9.The convergence time for different models.Figure 9.The convergence time for different models.

Figure 9 .
Figure 9.The convergence time for different models.Figure 9.The convergence time for different models.Information 2020, 11, 194 16 of 20

Figure 10 .
Figure 10.The classification error for different models.

Figure 10 .
Figure 10.The classification error for different models.

Figure 11 .
Figure 11.The confrontation scenario for multiple UAVs.

Figure 11 .
Figure 11.The confrontation scenario for multiple UAVs.

Figure 12 .
Figure 12.The results for the situation assessment using SA-P.

Figure 13 .
Figure 13.The results for the situation assessment using SA-BP.

Figure 12 .
Figure 12.The results for the situation assessment using SA-P.

Figure 12 .
Figure 12.The results for the situation assessment using SA-P.

Figure 13 .
Figure13.The results for the situation assessment using SA-BP.Figure13.The results for the situation assessment using SA-BP.

Figure 13 . 20 Figure 14 .
Figure 13.The results for the situation assessment using SA-BP.Figure 13.The results for the situation assessment using SA-BP.Information 2020, 11, 194 18 of 20

Figure 14 .
Figure 14.The results for the situation assessment using SA-F.
−4.From the results of the simulation, we can see that the classification error of the INN method is lower than that of the BP method.The INN model has higher accuracy than the BP method.The deep neural network can achieve a deep feature representation of the signal, so the accuracy of classification is higher.The simulation results show that the INN model has better learning performance than the classical BP model, which provides a new solution to improve the performance of the situation assessment for multiple UAVs.the training results for the two different models.The experimental results show that the convergence time of the proposed INN model is less than that of the BP method for ten training processes.The convergence time of the proposed INN model is about 2000 rounds, while that of the original BP method is about 8000 rounds.Adaptive momentum and Elastic SGD can accelerate the convergence rate of the neural network, which makes the proposed INN model need less convergence time.After the training, the classification performance of the two models is compared to the test data set.Each test is defined as a round.In each round, we use two models to classify signals and record the results.We selected ten rounds of classification results for performance comparison in the test phase.The simulation results are shown in Figure10.In Figure10, the unit of ordinate is 4 10 − .From the results of the simulation, we can see that the classification error of the INN method is lower than that of the BP method.The INN model has higher accuracy than the BP method.The deep neural network can achieve a deep feature representation of the signal, so the accuracy of classification is higher.The simulation results show that the INN model has better learning performance than the classical BP model, which provides a new solution to improve the performance of the situation assessment for multiple UAVs.