Research on Safety Prediction of Sector Traffic Operation Based on a Long Short Term Memory Model

Abstract

Traffic safety has been thought of as a basic feature of transportation, recent developments in civil aviation have emphasized the need for risk identification and safety prediction. This study aims to increase en-route flight safety through the development of prediction models for flight conflicts. Firstly, flight conflicts time series and traffic parameters are extracted from historical ADS-B data. In the second step, a Long Short-Term Memory (LSTM) model is trained to make a one-step-ahead prediction on the flight conflict time series. The results show that the LSTM model has the greatest prediction effect (MAE 0.3901) with comparison to other models. Based on that, we add traffic parameters (volume, density, velocity) into the LSTM model as new input variables and issue a comprehensive analysis of the relative predictive power of traffic parameters. The accuracy of prediction model is validated with a mean error of less than 3%. Based on the improvements of model performance brought by traffic parameters, LSTM models with a single traffic parameter are proposed for further discussion. The results illustrate that volume is the most important factor in promoting prediction accuracy and density has an advantage of improvement in the aspect of model stability.

1. Introduction

For the civil aviation transportation industry, safety is the basis cornerstone for the sustainable development, which are always highly valued. In order to significantly improve safety, the aviation industry has been moving towards a proactive approach to safety problems identification, which could increase awareness of emerging risks among the operators. With the development of data recording techniques in commercial aircrafts, more and more safety related data can be applied to investigate safety issues and recognize risks, which is collected from different data sources, such as safety reports, on-board recorders, and flight-data records.

Numerous applications of machine learning based on different data have gained popularity in the aviation safety domain recently. Generally, these data can be divided into two different types according to theirs formats, textual data or numerical data. Safety data are collected from proactive or reactive safety-related activities, such as safety reporting [1]. Aviation Safety Reporting System (ASRS) was first built in the U.S in the 1970s, and the Civil Aviation Administration of China (CAAC) has established the Sino Confidential Aviation Safety Reporting System (SCASS), which are both aimed at collecting voluntarily submitted incident reports from pilots, air traffic controllers, dispatchers, and others. Processing these texts can benefit identification, analysis, and evaluation of risks [2]. There has been recent progress in establishing theories and tools for performing largely automated and relatively rapid analysis of a corpus [3]. Many researchers process safety information in large text corpora by using artificial intelligence algorithms, such as latent Dirichlet allocation (LDA), which could be applied to obtain the probability distribution of words and topics in unsafe incident reports, and draw out the hazard source classification [4]. The structural topic modeling also helps in identifying known issues and evaluating the usefulness of a novel method based on ASRS data [3].

On the other side, safety issues are also found in numerical data, such as safety assessment data or other time-series data. On the basis of the numerical data, artificial intelligence can provide insights into identifying safety related factors and providing predictive capabilities, so as to move toward proactive safety management. The Random Forest regression algorithm is used to predict the critical landing parameters, which aid in risk assessment during the approach and landing phases of flight [5]. Based on the data from the flight data recorder, a cluster-based anomaly detection method to detect abnormal flights, which can support in detecting anomalies and associated risks from routine operations [6]. The Human Factors Analysis and Classification System (HFACS) framework and Bayesian Network Model (BNM) are proposed to provide a systematic method for aviation safety research [7]. The LSTM model trained with historical flight data are used to predict the flight safety level based on percentile velocity bounds of historical flights [8]. A Multi-Head Convolutional Neural Networks-Recurrent Neural Networks (MHCNN-RNN) architecture is proposed, which enables the prediction of different adverse events for any given flight [9]. The Extreme Learning Model (ELM) is used to predict the evolution of time series, and the prediction results of different algorithms at different dimensional levels and periods were compared [10]. Based on the safety evaluation, performance assessment about delay and predictability has been put forward in recent research [11]. Besides the safety related issues that have been mentioned, such as incidents, landing parameters, and human workload, conflict issues are directly relevant to unsafe events, which means that the flight separation violates the minimum separation standard. Air Traffic Control (ATC) plays an important role in Air Traffic Management (ATM), and ATC service does hold the purpose of preventing collision between aircrafts and obstacles and expediting and maintaining an orderly flow of air traffic [12]. There are a lot of discussion about collision risk in traffic safety assessment [13,14]. ICAO uses collision risk estimates safety in a given en-route airspace [13,15], and the level of risk that is deemed acceptable is termed the Target Level of Safety (TLS), which is 2.5 × 10⁻⁹ collisions per flight hour [13]. A combination of LSTM model and DNN model is used to predict the en-route flight safety, in which probabilistic separation distance is used as a quantitative safety metric [16]. However, the uncertainty in air traffic (such as weather uncertainty) increases the difficulty in recognizing conflicts issues, and the nonlinear characteristic of flight conflict has been verified, which means that the changes in flight conflict are complex and hard to understand and predict [17].

On the air traffic flow theory side, which can help in explaining traffic phenomenon from the perspective of intrinsic traffic properties, so as to provide a theoretical reference for transportation management. Since the Euler mathematical model of air traffic flow was established [18], research about air traffic flow theory largely concentrated in the field of theoretical modeling and simulation. With traffic parameters, the car-following model of high-speed flight [19] and the correlation between traffic flow parameters has been verified [20]. There are a lot of discussion about air traffic parameters without a flow model, many researchers focus the nonlinear characteristics of parameters [10,17,21,22] and specific variant on basic parameters, such as dynamic density [23,24]. With the development of surveillance services, high quality data can be collected easily. More and more researchers apply air traffic parameters to describe flight operation, which makes air traffic flow theory more practical. The distribution of traffic parameters in the terminal area [25] and time-headway of nose in en-route airspace have been proposed [26]. On the basis of traffic flow model and traffic data, optimization research about controlling traffic flow in terminal airspace and en-route airspace also attract researchers’ attention [27,28].

In the field of transportation, volume, density, and velocity are the basic parameters that could be used to describe traffic flow, which can effectively characterize traffic status in some way. Therefore, this research believes that some information that related to air traffic safety issues do exist in traffic flow parameters, and the relationship between traffic parameters and traffic safety should be studied precisely, so as to realize connection between air traffic flow theory and flight operation theoretically and practically.

Aiming at promoting safety problems identification and safety prediction, this paper proposes prediction models for forecasting flight conflicts. Based on the model with highest accuracy, adding traffic parameters as new input variable into current prediction model to realize performance improvements and further observation. The research framework of this paper is shown in Figure 1. The remaining part of the paper proceeds as follows: Section 2 introduces the definition and calculation process of required parameters. Section 3 proposes time series forecasting based on LSTM. Section 4 presents a case study based on a real data set from Guangzhou Area Control Center (GACC) and compares model performance under different input conditions (without traffic parameters, with all traffic parameters and with single traffic parameter), and results and discussion are included. Conclusions are subsequently discussed in Section 5.

2. Air Traffic Parameters in Sectors

2.1. Definition of Traffic Parameters

(1) Volume: the number of aircraft operating in this sector during statistical period. We calculate this parameter using Equation (1), where l represents the number of segments in current sector, T represents observation time, and $n_i$ represents the number of aircrafts in each segment:

$$volume=\sum _{i=1}^l\frac{n_i}{T}$$

(1)

(2) Density: the number of aircraft per unit length of each segment in sector during statistical period. We calculate this parameter using Equation (2), where l represents the number of flight segments in the sector, $q_i$ represents the number of aircraft in each segment during observation period, and $lengt{h}_i$ represents the length of each flight segment (unit: km):

$$density=\sum _{i=1}^l\frac{q_i}{lengt{h}_i}$$

(2)

(3) Velocity: the average velocity of aircrafts which are in operation during statistical period. We calculate this parameter using Equation (3), where m represents the number of filtered data, $q_i^{*}$ represents number of aircrafts in sector, and $v_m$ represents each aircraft speed which recorded in related data:

$$velocity=\frac{1}{q_i^{*}}\sum _{i=1}^{q_i^{*}}\frac{\sum _{i=1}^m{v}_m}{m}$$

(3)

2.2. Definition of Safety Parameters

From the perspective of deterministic conflict detection, conflict events are identified by an aircraft safety zone [29]. The safety zone and relative position between aircrafts are shown in Figure 2.

The conflict event is defined by the separation index in Equation (4). According to the separation standard from the Chinese rules of air traffic control, $d_0$ = 10 km and $h_0$ = 300 m are assigned:

$$index=max(\frac{\left|\Delta h\right|}{h_0},\frac{d}{d_0})-1$$

(4)

On the basis of index, the calculation process is shown in Figure 3. The specific definitions are:

(1) Number of flight conflicts: the number of conflict events during statistical period, where $num$ represents the total number of aircraft pairs, and $is\text{\_}conflic{t}_i$ represents whether there is a conflict event between pair of aircrafts:

$$is\text{\_}conflic{t}_i=\left\{\begin{array}{c}1\mathrm{if}index<0\\ 0\mathrm{if}index\ge 0\end{array}\right.$$

(5)

$$count=\sum _{i=1}^{num}is\text{\_}conflic{t}_i$$

(6)

(2) Minimum safety index: the minimum value of index during statistical period with respect to the pair of aircrafts which hold the closest distance, where $min\text{\_}index$ represents the minimum index itself, $num$ represents the total number of aircraft pairs, and $inde{x}_i$ represents the index between the pair of aircrafts:

$$min\text{\_}index=min(inde{x}_1,\cdots ,inde{x}_i,\cdots ,inde{x}_{num})$$

(7)

3. Time Series Forecasting Model

3.1. Time Series Forecasting

A time series is a sequence S of historical measurements $y_t$ of an observable variable y at equal time intervals [30]. Through a sequence of time, the method of time series forecasting predicts future events by analyzing the trends of the past, on the assumption that future trends will hold similar to historical trends. Forecasting future values of an observed time series plays an important role in nearly all fields of science and engineering, such as economics, finance, business intelligence, meteorology, and telecommunication [31]. It is also an important area of Machine Learning (ML) and Deep Learning (DL), which can be cast as a supervised learning problem. Methods, such as Support Vector Machines (SVM), Ridge Regression (RR), Random Forests (RF), and Long-Short Term Memory (LSTM), can be applied to it. These data-driven models with time series are examples of non-parametric nonlinear models which use only historical data to learn the stochastic dependency between the past and the future [30].

Based on the flight conflict time series, which is the data sequence of flight conflict amount obtained from equal time intervals according to time sequence. We establish different prediction models, in which the required parameters in the last time instant $t-1$ are introduced as input variables and the number of conflicts at the current time instant t is introduced as an output variable.

3.2. LSTM Model

LSTM is a special kind of the recurrent neural network (RNN), which is a class of artificial neural network where connections between units form a directed cycle [32]. In most RNNs, the hidden layer function is an elementwise application of sigmoid function. However, LSTM uses purpose-built memory cells to store information, which is better at finding and exploiting long range dependencies in the data [33].

The basic network structure is shown in Figure 4, where $C_t$ represents the cell state, $tanh$ is the hyperbolic tangent activation function, and $\sigma$ is the sigmoid activation function. The LSTM cell takes input $x_i$ and the hidden vector from the last time step and presents ${\widehat{y}}_i$ as an estimated output with a new hidden vector and memory vector.

For the current time step, ${\widehat{y}}_i=\mathrm{LSTM}\left(x_i,y_i\right)$ is established. Firstly, the forget gate determines the cell state by checking the input value, which is the number of conflict events in the last time instant $i-1$:

$$f_i=\sigma (W_f·\left[y_{i-1},x_i\right]+b_f)$$

(8)

Secondly, the input gate decides how much input information enters the current cell, which determines cell state within the cell:

$${\tilde{C}}_i=tanh(W_c·\left[y_{i-1},x_i\right]+b_C)$$

(9)

$$C_i=f_i\ast C_{i-1}+i_i\ast {\tilde{C}}_i$$

(10)

Finally, the output gate determines the current output which is the number of conflicts at the current time instant i.

$$o_i=\sigma (W_o·\left[y_{i-1},x_i\right]+b_o)$$

(11)

$$y_i=o_i\ast tanh\left(C_i\right)$$

(12)

In order to test and verify the forecasting performance of model, prediction models based on SVR, LSTM, and RR are established, respectively. SVR is the most common application form of SVMs, which could achieve good results in many domains [34], while RR is one of a family of penalized methods for the regression task [35]. The training and verification processes are shown in Figure 5. The results of another two prediction models, i.e., SVR and RR, are given for the purpose of comparison. By doing this, we forecast the flight conflicts for the current time instant t. The prediction can enable us to estimate the number of conflicts in a defined sector, which could provide insights in traffic safety.

3.3. Performance Indicators

In this paper, Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) are adopted as the evaluation metric, which are used to quantity the effectiveness of different prediction models. The definitions are shown in Equations (13) and (14):

$$E_{RMSE}=\sqrt{\frac{1}{N}{\sum _{i=1}^N(y_i-{\widehat{y}}_i)}^2}$$

(13)

$$E_{MAE}=\frac{1}{N}\sum _{i=1}^N\left|y_i-{\widehat{y}}_i\right|$$

(14)

4. Case Study

The experiment selects Reduced Vertical Separation Minimum (RVSM) airspace in Guangzhou during the period from 1 September 2020 to 7 September 2020, which belongs to CAAC Central and Southern Regional Administration. This RVSM airspace ranges vertically from 29,000 feet to 41,000 feet, which is preferred by airlines for its fuel-efficient feature.

The research data include sector data and Automatic Dependent Surveillance-Broadcast (ADS-B) data. The sector data includes geographical boundaries, flight level, routes information, and separation rules. ADS-B data include flight callsign, operating altitude, latitude, longitude, speed, et al. The specific format of ADS-B data is shown in

Table 1. The format of ADS-B data.

CallSign	Longitude	Latitude	Height	Speed	...	Time
SKW4310	−104.685	38.826	2324.10	333.35	...	22:25:09
SKW4310	−104.516	38.862	3581.39	629.67	...	22:27:39
SKW4310	−104.474	38.592	5577.83	685.23	...	22:30:21
SKW4310	−104.500	38.323	7132.31	740.79	...	22:32:57

.

ADS-B data are filtered by geographical boundaries information and flight level information firstly, and the airspace structure and trajectory information are shown in Figure 6. Then, we fill in missing records by averaging ones in the data cleaning process, and we take 1 min as the unit time. Finally, 261,231 pieces of ADS-B data are processed in this paper, which are in 10,080 time slices.

Taking the traffic parameters on 1 September as an example, the statistical distribution of traffic parameters is shown in Figure 7.

4.1. Time Series Forecasting without Traffic Parameters

As introduced in Section 3, we establish three prediction models based on different methods with the purpose of comparison. Here, the number of conflict events at the last moment is taken as the only input variable in these prediction models. The experimental steps are as follows:

(1) The division of training set and test set. For flight conflict time series, it does hold a time order dependence between observations. The common methods of cross validation would cause the crossover of temporal features. In order to test model performance, rolling origin is used, which is an evaluation technique according to which the forecasting origin is updated successively and the forecasts are produced from each origin [36]. The original data will be divided into the training set and test set in 4:3, 5:2, and 6:1, and we name it as case 1, case 2, and case 3 respectively, which is a rolling origin with a non-constant holdout sample size.

(2) The training and verification process.

The training set is used as input variable and the test set is used to verify model output. The grid search method is used to optimize parameters among all models. Taking the LSTM model as an example, the key parameters in model includes the number of neurons, batch size, and learning rate. The optimization process of parameters is shown in

Table 2. The optimization process of parameters.

Batch Size	Neurons = 10			Neurons = 20			Neurons = 30
	Learning Rate	MAE	RMSE	Learning Rate	MAE	RMSE	Learning Rate	MAE	RMSE
120	0.1	0.6698	1.4344	0.1	0.7808	1.6602	0.1	1.1558	2.2456
	0.01	0.6122	1.4699	0.01	0.5919	1.3938	0.01	0.5756	1.3461
	0.001	0.5561	1.4566	0.001	0.6583	1.5871	0.001	0.5559	1.4510
180	0.1	0.7416	1.6152	0.1	1.2039	2.3464	0.1	1.8482	3.3518
	0.01	0.5524	1.3160	0.01	0.5756	1.3461	0.01	0.5756	1.3461
	0.001	0.4520	1.0235	0.001	0.6903	1.7057	0.001	0.4783	1.0996
240	0.1	0.6405	1.5370	0.1	1.7343	3.1770	0.1	1.8482	3.3518
	0.01	0.6475	1.5399	0.01	0.5929	1.3945	0.01	0.5756	1.3461
	0.001	0.4380	0.9157	0.001	0.6572	1.5843	0.001	0.4147	0.9183
300	0.1	0.6393	1.5341	0.1	1.8482	3.3518	0.1	1.8482	3.3518
	0.01	0.6666	1.5906	0.01	0.6655	1.5899	0.01	0.6655	1.5899
	0.001	0.4130	0.8311	0.001	0.5005	1.2870	0.001	0.3901	0.8037
360	0.1	0.6851	1.6164	0.1	1.8482	3.3518	0.1	1.8482	3.3518
	0.01	0.6460	1.5276	0.01	0.6564	1.5527	0.01	0.6655	1.5899
	0.001	0.4492	0.8721	0.001	0.4521	1.0801	0.001	0.4102	0.8308

.

Due to the randomness of thresholds and weights in the neural network, different prediction methods were tested 10 times under different data division methods, and the prediction performance of each model is shown in Figure 8.

The average error of model prediction is reported in

Table 3. Average error of models.

Model	Case 1		Case 2		Case 3
	MAE	RMSE	MAE	RMSE	MAE	RMSE
SVR	1.7500	2.5814	0.7050	1.7709	0.0500	0.2328
LSTM	1.9062	3.0783	0.3901	0.8037	0.0493	0.2252
RR	1.7877	3.0092	0.6259	1.5346	0.0500	0.2328

.

As the result of case 1 shows, the prediction errors of three models are pretty large, and the performance of SVR model is slightly better than the other two models. As the proportion of training data increases in case 2, the performance of these prediction models has been greatly improved, and the average decrease in mean square error is 68.39 %. Moreover, the model performances start showing significant differences. In case 2, the average MAE of SVR model and RR model is 0.7050 and 0.6259 respectively, while the figure for LSTM model is 0.3901. The advantage of LSTM model emerged in a significant way. In case 3, the average decrease in mean square error is 91.32% and the performance of LSTM model is still better than other models. Among these prediction models, LSTM model presents a robust performance in predicting flight conflicts with comparison to other models.

4.2. Time Series Forecasting with Traffic Parameters

The results in Section 4.1 show that the LSTM model does have comparatively higher prediction accuracy for conflict prediction. In order to achieve the potential improvements for model performance and issue a comprehensive analysis of the relative predictive power of traffic parameter, traffic volume, density, and velocity are added into prediction model as input variables for further observation. The experiment process is shown in Figure 9.

For the LSTM model with new input variables, its training loss and verification loss in 10 trials are shown in Figure 10, which includes the value of loss in 10 trials and the average value of loss. At the beginning of training, the value of loss dropped significantly, and convergence was met within 20 generations. Then, the value of loss gradually became stabilized. In the end, the gap between the verification loss and training loss was pretty narrow and the task of model training was completed.

As shown in Figure 11, the values of performance indicators in 10 repeated experiments are in comparison with the results in Section 4.1, where the solid line represents prediction error of each trial, the dashed line represents average prediction error, and the box chart shows the distributions of prediction error.

From the results, we observe that the LSTM model with all traffic parameters has presented better performance in predicting flight conflicts with error reduced by 3%. Moreover, the performance of prediction model without traffic parameters is more unstable and fluctuating greatly, which could be seen from the box chart. The results shows that the time series of traffic parameters do hold information related to the flight conflicts, which is of positive significance for conflict forecasting.

Next, we establish the model with single traffic parameters to analyze the predictive power of each parameter in predicting flight conflicts, which could give insight into the relationship between traffic flow parameters and conflict. In a similar way, we compare the performance of the LSTM model with single parameter. The RMSE and MAE between predicted values and actual values are used as an evaluation metric to compare model performance, as shown in Figure 12, where the solid line represents prediction error of each trial, and the dashed line represents average prediction error.

It is clear that the model with traffic volume shows a much better prediction accuracy, i.e., the smallest average RMSE and MAE, as the blue dashed line shows. The order ranking from high to low according to prediction accuracy is the model with volume, the model with density, and the model with velocity. As the box chart shows, which describes the values of average error as they spread across the entire range, we observe that the model with density is of best stability, the model with volume comes second and the model with velocity takes the final place in the aspect of model stability. We believe that volume is the key factor that could give help in improving the prediction accuracy among all traffic parameters, followed by density and velocity, while the improvement of model stability which is brought by density should be given adequate attention.

4.3. Correlation Analysis

In order to verify the relationship between traffic parameters and flight conflicts, traffic and safety parameters are selected for correlation analysis, considering that the number of conflict events are integer values, and there are a lot of cases where the actual value is 0 in low activity periods, which is not conducive to analyze correlation. The separation index is selected for comparison, which is calculated under the deterministic conflict detection model in Section 2. Taking 1 September as an example, as shown in Figure 13a.

4.3.1. Coefficient Calculation

The time series of traffic parameters between 400 and 650 min are shown in Figure 13b, and it can be obviously seen that the trend of index holds similar to the trend of volume and density, which is synchronization characteristics. However, such similar trend cannot be found in traffic velocity time series, due to the slight change of aircraft velocity in the cruise phrase, which is relatively stable during most of the time. In order to quantify this similarity, calculations of correlation coefficient between different time series are shown in Figure 14.

According to the intensity of correlation, the order ranking from high to low is traffic volume, traffic density, and traffic velocity, which is consistent with the results ranking in Section 4.3. In conclusion, the correlation between velocity and index is the weakest, while there is a significant negative correlation in volume-index and density-index.

4.3.2. Coefficient Calculation under Time Delay

To ensure the synchronous correlation is stable and effective, we present a further discussion about how the index and volume is correlated with, in which volume is the most important factor in promoting prediction accuracy. The asynchrony is embodied as the evolution law of a sequence lags behind or leads another sequence by certain time steps. In the experiment, the sequence of volume is processed with time delay and the correlation coefficient between volume and index is calculated. The results are shown in Figure 15.

The results show that the most significant correlation occurs when there is no time delay between this two parameters. Therefore, it is believed that the negative correlation which is revealed in current experimental time scale is relatively stable, and there is no such asynchronous law with more significant correlation.

5. Conclusions

In this paper, we address the problem of conflict prediction in a sector. Different prediction models are proposed based on the data set of conflicts that is extracted from historical ADS-B data.

This paper makes two contributions to the field of traffic safety prediction in aviation. Firstly, LSTM prediction models are proposed for forecasting the number of conflicts, and SVR prediction model and RR prediction model are given for the purpose of comparison. The results show that LSTM model does have comparatively higher practical value for conflict prediction, which could capture the variation and the uncertainty of the traffic conflicts in the en-route airspace. Secondly, traffic basic parameters are added into the LSTM prediction model as new input variables with the purpose of the potential improvements for model performance and comprehensive analysis of the relative predictive power of traffic parameters. Thus, a more accurate prediction model for conflicts forecasting is achieved with the addition of all traffic parameters, and the ranking results of traffic parameters according to their contributions in conflicts forecasting are presented.

All proposed methods are applied to a real data set of AR01 airspace in Guangzhou. The prediction results show that the LSTM model has the highest prediction accuracy under limited input conditions, with an MAE of 0.3901. With the addition of traffic parameters, the improved prediction model shows an acceptable result with error reduced by 3% and stronger stability. Among all the models with single traffic parameter, we found that volume is a key factor in improving prediction accuracy and the model with density shows a great advantage in improving model stability. Further steps will include detailed analysis for the relationship between other traffic parameters and safety issues under different time scales, as a variation in time scales will change the values of traffic parameter and traffic status.