Assessment of Potential Conflict Detection by the ATCo

Abstract

The main goal of this article is to analyse the probability of detecting potential conflicts by the Air Traffic Controller (ATCo). The ATCo ensures the safety of aircraft and one of its main functions is collision avoidance. Collision avoidance is known as separation provision and this term means assuring the safe distance between each aircraft by sides, vertical and longitudinal minimums of separation. The air traffic controller must ensure a high level of airspace capacity. The work performance is related to high demands on individual characteristics, knowledge, skills and, of course, air traffic characteristics. In addition to analysing the probability of detecting potential conflicts, the study of the most influential factors on this safety event is considered of special relevance since the ATCo represents the last executive section of the air traffic control system and failure to detect potential conflicts could lead to a possible infringement of the minimum separation distances between aircraft or even a collision. In order to carry out this approach, Bayesian Networks will be used due to their high predictive capacity. In addition, a dual approach based on knowledge and real operational data provided by an ANSP will be used. These data are one of the great advantages of this study compared to those included in the current literature.

1. Introduction

Current knowledge of the determinants of the occurrence of aircraft-to-aircraft safety incidents is limited, as is the industry’s current ability to predict them [1].

Studies and models analysing or determining the safety of aircraft operations in airspace, and the loss of separation between aircraft, have so far focused on human intervention and human error as the main cause of unsafe situations resulting from loss of separation between aircraft [2,3,4].

Loss of separation incidents between aircraft generally have very low frequencies of occurrence, which makes it very difficult to have a sufficient volume of data to allow relevant statistical analysis. As the SESAR safety reference material points out [5]: “Due to the high level of safety achieved in ATM, relatively little data is available on negative safety outcomes; consequently, it is increasingly difficult to achieve more safety benefits using standard safety approaches”.

Because of this, most studies have been devoted to ex-post analysis and the reconstruction of events leading to loss of separation between aircraft in the air [3]. Some studies have developed analytical models, in which the probability of incidents is predicted from the expected trajectories of future aircraft, calculated from the respective flight plans [4]. However, there are very few references that exploit operational data on a massive scale, i.e., the large volume of data from all those airspace operations that have not resulted in loss of separation [6,7]; nor that allow the incorporation of expert information into predictive models or use data from scenarios or situations with similar characteristics [8]. It is important to report and assess less serious security incidents. Less serious incident data and statistics can provide comprehensive and valuable information, considering that such data are more frequent and the reports generated are simpler and more accessible.

To address all these shortcomings, a combination of reactive, proactive and predictive analytics is required to exploit the potential of safety-related data to provide information on potential hazards and operational risks. In this context, the statistical models used and the safety analyses performed must evolve from the analysis of reactive methods towards a more predictive perspective, which allows predicting the likelihood of future safety events.

It is for this reason that it was decided to develop the model presented here. To do this, it was first necessary to carry out a bibliographical analysis of the state of the art of the statistical models used to date and their predictive capacity.

In recent decades, significant research has been carried out on the use of modern statistical techniques to predict incidents and precursors of safety in other modes of transport, particularly road transport. One such technique that has proven to be highly effective is Bayesian Networks (BN). The relevant academic literature provides interesting examples of the types of aviation-related problems to which Bayesian Networks can be applied. Some examples are [9,10,11,12].

Still in the context of aviation, research is available on predictive modelling or BN to predict safety events, or to help predict safety performance. Some recent studies that provide relevant results are [13,14,15,16,17].

All of these examples of methodological advances in research take an innovative statistical approach. They use Bayesian Networks to develop statistical models for estimating and predicting safety events based on the intrinsic characteristics of the route and sector scenario, traffic and its evolution, and airspace management. These models allow the assessment, determination and prediction of the safety performance of a given airspace and once developed, calibrated and validated, could be extended to other key performance areas in air traffic management, such as capacity, efficiency, etc.

Therefore, one of the most comprehensive alternatives is to employ BN-based models. These models address two common problems associated with predictive safety models: (1) the consideration of regression to the mean (RTM) on the one hand; and (2) the lack of data when there is an insufficient historical period or a very low number of occurrences on the other hand [18]. Regression to the mean is a common bias when assessing a network in terms of accident or safety, as a point or element of the network may have a high number of occurrences in a year and yet be compatible with an acceptable probability distribution for the occurrences. The BN model will allow better estimation of the safety of a part of the air transport system, taking into account not only the number of safety-related occurrences at that location, but also the occurrences observed in similar environments, naturally incorporating expert knowledge of the causes that could have produced them [18].

In all the cases analysed, the ATCo behaviour is difficult to predict or model. In fact, most accidents are the result of human error caused by poor performance or distraction. This is why special relevance is attributed to the ATCo and his performance in detecting potential conflicts in order to avoid the progression of a possible loss of separation [19].

Furthermore, given that these factors and their effects can be very diverse depending on the different airspace considered, it is recommended to calibrate the existing models for each airspace sector, with the aim of developing new, more accurate tools that predict more reliably the safety levels of the airspace considered [14].

1.1. Principles of Bayesian Network Analysis

Bayesian Networks (BN) are statistical tools that represent a set of associated uncertainties on the basis of conditional independence relationships between them.

A BN is a directed acyclic graph in which each node represents a random variable that has an associated conditional probability function. The structure of the BN provides information about the conditional dependence and independence relationships between the variables. These relationships simplify the representation of the joint probability function as the product of the conditional probability functions of each variable [20].

Let U = {X1, X2,..., Xn} be a set of random variables. Formally, a Bayesian Network for U is a pair B = <G, T> in which:

-

G is a directed acyclic graph in which each node represents one of the variables X1, X2, …, Xn, and each arc represents direct dependence relationships between the variables. The direction of the arcs indicates that the variable “pointed to” by the arc depends on the variable at its origin.

-

T is a set of parameters that quantify the network. It contains the probabilities PB(xi|pxi) for each possible value xi of each variable Xi and each possible value pxi of PXi, where the latter denotes the set of parents of Xi in G. Thus, a Bayesian network B defines a unique joint probability distribution over U.

It is important to note that the topology or structure of the network not only provides information on the probabilistic dependencies between variables, but also on the conditional independencies of a variable or set of variables given one or more other variables. Each variable is independent of variables that are not its descendants in the network, given the state of its parent variables.

The inclusion of independence relations in the graph structure itself makes Bayesian networks a good tool for compact knowledge representation (the number of required parameters is reduced). In addition, they provide flexible methods of reasoning based on the propagation of probabilities along the network according to the laws of probability theory.

The Main Reasons for Selecting Bayesian Network Methods

The main reasons why Bayesian Networks have been selected are explained below:

-

Bayesian networks are very useful for capturing and analysing causality and influence relationships. They are very effective for diffusing uncertainty and updating systems with new data. They are also applicable when the structure of the system is too complex. They provide an intuitive and efficient way to represent a large field, making it feasible to model complex systems.

-

Bayesian networks are primarily used to update the probability distribution of the states of hypothetical variables (variables that cannot be directly observed). This probability distribution helps decision-makers determine the appropriate course of action.

-

Bayesian networks provide a convenient and consistent way of expressing uncertainty in uncertainty models and are increasingly used to express knowledge. They are used for qualitative and quantitative modelling of uncertainty and its causes.

-

Due to the conditional dependence of the variables in the network, BNs provide the ability to predict or diagnose (i.e., they can determine impact and causes). BNs are used to model multidirectional forward and backward uncertainties.

BNs can perform qualitative cause-and-effect assessments and can quantitatively update the probability distribution of unobservable variables.

-

Qualitative analysis: Given a scene, the BN graphically represents the causal relationship between the different elements of the scene.

-

Quantitative analysis: Updating the probability distribution. Given the hypothetical variables representing the possible actions and the prior probability distribution, the BN provides the function of updating this probability distribution when new data and information are acquired.

2. Methodology

The proposed methodology aims to obtain a model capable of characterising and predicting the detection of potential conflicts between en-route aircraft (level flight as well as climb and descent) by ATCo. The content of this study can be structured in three blocks:

-

Development of the Bayesian Network Model (BNM). The development of the BNM starts with a characterisation of the safety event in terms of safety dimensions (precursors) and their aggregation. The result of this descriptive analysis, together with prior statistical knowledge, serves to structure the model that could provide statistical representations of the frequency and severity of the safety event.

-

Calibration, tuning and sensitivity analysis of the Bayesian Network Model (BNM). The model proposed in the previous task is fitted and calibrated on real data. The explanatory power of the model and/or of each independent variable and mixed effects is quantified. The sensitivity analysis considering mixed effects allows characterising the safety performance in terms of not only the independent dimensions, but also of their combinations, identifying the prior thresholds of these dimensions that would reduce the frequency of a safety event.

-

Most influential factors in the Bayesian Network Model (BNM). Application of the model to the defined case study or scenario to quantify the most influential factors.

3. Development of the Bayesian Network Model (BNM)

This section will focus on the development of the Bayesian Network aimed and assess the probability of conflict detection by the ATCo.

3.1. Construction of the BN

This network models the ATCo’s ability to detect potential conflicts as a result of short- and medium-term effects. The high-level structure of the network is presented in Figure 1. For this purpose, mainly two concepts are considered:

ATCo alert status. This alert status is assessed on the basis of the workload of the air traffic controller during the hour in which the Closest Point of Approach (CPA) of the aircraft pair occurs. The workload shall consider the traffic in that hour, the trajectories and conditions of this traffic, as well as the actions taken by the ATCo during that period of time [21]. This concept takes into account the medium-term effects that influence the ATCo’s performance. This idea is represented by the blue box at the top left of Figure 1 with the label “Hourly ATCo workload & Situational awareness”. It should be noted that as these are real operational data collected by an ANSP, psychometric factors relating to ATCo are not available. However, there are other variables of great relevance to its behaviour, such as the ATC actions it performed on the traffic.

Scenario and traffic conditions at the time of the event. This concept considers the short-term effects that influence ATCo performance. They are derived from the relative situation of each pair of aircraft at the time at which ATCo searches for potential conflicts. This time t is considered to be the 5 min period in which the event occurs. To account for short-term effects, each hour will be divided into 12 periods of 5 min, and the variables representing the situation of the aircraft pairs will be calculated for each of the 5 min periods. This idea is represented by the blue box at the bottom left of Figure 1 labelled “Traffic complexity at the time of conflict”. The scenario conditions are subdivided into three variables:

o

Traffic conditions.

o

Aircraft performance.

o

Not expected aircraft in the sector.

Thus, the probability of detecting the conflict will depend on the ATCo alert status throughout the hour, and on the complexity of the overall situation in short periods of 5 min, when aircraft pair events occur.

Finally, the probability of the conflict being detected by the ATCo shall be assessed for each aircraft pair and marked with a label “YES/NO–1/0” in the corresponding file. The two possible branches of results are foreseen, as shown in Figure 1.

The outputs of this network could lead to further analysis, e.g., in case the potential conflict is not detected by the controller, the final vertical and horizontal distance in the CPA of the aircraft pair could be assessed [22]. On the other hand, if the ATCo has detected the conflict, the conflict resolution could be assessed [23,24].

Thus, the construction of the network is based on:

-

Network entry: It shall consist of the four inputs already mentioned, composed of the ATCo workload plus the three variables characterising the complexity of the scenario (traffic conditions, aircraft performance and unexpected aircraft).

-

Training data: The network is trained with the conditions of each record.

-

Network exit: The output will be the probability that the ATCo detects the potential conflict. Note that the conflict will be considered detected whenever the air traffic controller takes any action on any of the two aircraft involved.

This high-level approach is incorporated into a single network (Figure 1) in which, ultimately, four main variables directly feed into the probability of a potential conflict being detected by the ATCo. These variables are represented by the four areas of analysis that are considered to be of most relevance in determining whether a controller successfully detects the conflict. Each of the above areas is discussed separately below:

3.1.1. ATCo Alert Status Analysis Area

The state of alertness will depend directly on the workload of the controller during the hour of operation. In this sense, both “saturation” and “relaxation” can be a problem.

Therefore, it is a matter of having an assessment of the workload during the hour in which the event occurs, estimated by the actions performed by the ATCo during that time.

This area of analysis could be treated as a separate network, the construction of which has been based on the following elements:

-

Network entry: Global parameters of the hour, referring to traffic density and ATCo actions during the hour.

-

Training data: Considering the sector conditions, it is trained to identify the ATCo actions in the hour, and in particular the authorisations to resolve a conflict (flight level change, direct to, reroute).

-

Network exit: The workload level, defined in four states:

o

Level 1: Number of high-resolution actions, and high total number of actions.

o

Level 2: Number of medium resolution actions, and total number of actions high.

o

Level 3: Number of medium resolution actions and medium total number of actions.

o

Level 4: Low number of total actions.

3.1.2. Scenario Conditions Analysis Area

This is the assessment of the scenario at the moment when the second aircraft of the pair enters the sector, as this would be the moment from which a conflict is assessed and detected.

This is implemented through three areas of analysis where each output variable would constitute a direct input to determine the probability of the ATCo detecting the conflict. However, the type of input data for each of these can be described in a general way:

-

Network entry: Average of the scenario parameters in the 5 min period identified in time.

-

Parameter calculation: Considering the scenario conditions, training is carried out to evaluate the three parameters that characterise the scenario and which are described below:

o

Traffic conditions: defined by parameters such as instantaneous demand (count of number of aircraft entering the sector), the percentage of regulated aircraft and the distribution of flight levels. All these variables are evaluated at 5 min time intervals.

o

Aircraft performance: Defined by vertical and horizontal speed distributions.

o

Unexpected aircraft in the sector: Both out-of-flow and unexpected aircraft are considered; however, as it was not possible to obtain data related to these flows, only the first variable will be used.

The detailed calculation of the scenario parameters is discussed below.

Traffic conditions: It considers the situation of the demand.

-

Instantaneous demand: Ratio of instantaneous demand to average maximum instantaneous demand on the day in question (maximum shall be 1).

-

% Regulated aircraft: % of regulated aircraft with respect to the total in the period of time in question on a scale of 0–1. The fact that the aircraft are regulated means that the flow will be continuous.

-

% Aircraft changing FL in the period: % of aircraft changing FL relative to the total in the time period in question on a scale of 0–1.

Performances: Considers the difference in aircraft performances at time t.

-

Speed distribution: Complexity occurs when aircraft have differences in speed. Therefore, the speed difference between the two aircraft is compared:

o

% Aircraft with a speed difference greater than 50 knots: % on a scale of 0 to 1.

o

Vertical speed distribution: Complexity occurs when ACs climb or descend with different speeds. Therefore, the difference in speeds between the two aircraft is compared.

o

% of AC with non-zero vertical speed difference: % on a scale of 0–1.

Unplanned aircraft: Considers the complexity contributed by unexpected aircraft or aircraft operating outside the standard flows in the 5 min period.

o

% Unexpected AC: % of aircraft that have changed EOBT time against IOBT.

Figure 2 shows the detailed schematic of the proposed Bayesian network. In this case, it is a three-level distributed network. It represents the selected input variables and the causal relationships between them. The causal relationships, i.e., the arrows connecting the variables, have been established directly by the model and reviewed and completed by expert judgement.

4. Calibration, Fitting and Sensitivity Analysis of the Bayesian Network Model (BNM)

One of the objectives of this project is to try to identify the cause-effect relationship that exists between certain variables or situations, which directly or indirectly affect safety and the detection of potential conflicts by the ATCo.

Therefore, a wide range of factors that may be related to the occurrence of a loss of separation between aircraft have been identified, regardless of whether or not the data are available in later phases of the project.

This study has been developed in three hierarchical levels as the level of detail of the possible cause is deepened [25].

Thus, the different areas of analysis are placed at the first level. These areas try to group the different precursors or potential causal factors according to their nature and, in turn, each causal factor is divided into different parameters that characterise the event.

However, in this section we intend to develop the third level parameters, i.e., the input, training and output variables, in more depth.

Therefore,

Table 1. Model variables, description and discretisation criterion.

Parameter	Description	Discretisation Criteria	Discretisation Example
Occupancy	Occupation Sector data at the AC entry hour	It is divided into 4 intervals that represent the percentage of occupation in the sector at the AC entry hour respect to the maximum value detected in the sector. The values of the intervals depend on the sector data.	Value < 40% 40% < value < 65% 65% < value < 85% 85% < value
Entry	Entries Sector data at the AC entry hour	It is divided into 4 intervals that represent the percentage of entries in the sector respect to the sector entry declared value at the AC entry hour. The values of the intervals depend on the sector data.	Value < 55% 55% < value < 65% 65% < value < 80% 80% < value
Instant demand	Ac entry data in the sector in the 5 min period	It is divided into 4 intervals that represent the percentage respect to the maximum instantaneous demand value. The values of the intervals depend on the sector data.	Value < 60% 60% < value < 90% 90% < value < 120% 120% < value
AC above capacity	Percentage of aircraft exceeding declared capacity per hour	It is divided into two states, when the percentage of AC above capacity is equal to 0 and when it is greater than 0	Value = 0 Value > 0
Entry FL Distribution	Calculation of the median flight level with which AC enter the sector at the entry hour	It is divided into three intervals that represent the entry FL distribution in the sector	Value < 350 350 < Value < 360 Value > 360
Horizontal Speeds Differences Distribution	Calculation of the median horizontal speeds with which AC enter the sector at the entry hour	It is divided into three intervals that represent the entry Horizontal Speed Difference Distribution in the sector	Value < 35 35 < Value < 55 Value > 55
Entry Vertical Speeds Distribution	Calculation of the median vertical speeds with which AC enter the sector at the entry hour. It is calculated in absolute value	It is divided into two states, when the median of entry Vertical speed is 0 and when it is greater than 0. This discretisation is intended to represent the complexity for the controller, modelling the AC that are ascending or descending.	Value = 0 Value > 0
Changes in FL distribution	Calculation of the median percentage of AC that change their FL in the sector at the entry hour	It is divided into three intervals, that depends on the sector data.	Value = 0 0% < Value < 15% Value > 15%
Delay Distribution	Calculation of the median time delayed of AC at the entry hour into the sector	It is divided into four intervals that represent the delay time (seconds) in the sector. These intervals depend on the sector data	Value = 0 0 < Value < 240 240 < Value < 420 420 < Value
% Regulated AC	Calculation of the percentage of AC that have some regulation at the entry hour	It is divided into four intervals. These intervals depend on the sector data	Value < 15% 15% < Value < 25% 25% < Value < 35% 35% < Value
% Delayed AC	Calculation of the percentage of AC that have some delay at the entry hour	It is divided into four intervals. These intervals depend on the sector data	Value < 35% 35% < Value < 50% 50% < Value < 65% 65% < Value
ATCo actions hour	Calculation of the total actions that the controller has given in one hour in the sector	It is divided into three intervals, which are intended to represent the workload of ATCo according to the action they have undertaken. These intervals depend on the sector data	Value < 75 75 < Value < 100 Value > 100
Action MIX	Sum of resolution actions of any kind. This variable considers flight level changes, vectors and directs.	Resolution actions are added, and the same three categories are defined	Value < 0.5 0.5 < Value < 1 Value > 1
% AC that change FL in 5 min	Distribution of aircraft by number of flight levels that change	It is divided into three intervals	Value = 0% 0% < Value < 70% 70% < Value
Proportion Vertical Speed Difference	Calculation of the percentage of AC with vertical speed difference other than 0, on a scale of 0–1	It is divided into three intervals that represent the complexity when ACs are ascending or descending with different speeds.	Value < 0.35 0.35 < Value < 0.7 0.7 < Value
Proportion Horizontal Speed Difference	Calculation of the percentage of AC with horizontal speed difference greater than 50 knots: on a scale of 0–1	It is divided into three intervals that represent the complexity when ACs have speed difference	Value < 0.35 0.35 < Value < 0.7 0.7 < Value
ATC Workload	Combination of total actions with resolution actions	This output node is divided into four states, as defined in the first sections	Level 1 Level 2 Level 3 Level 4
Traffic Conditions	Relative situation of each aircraft pair	This output node is divided into three states, in order to model the traffic conditions at the conflict moment	Value < 0.5 0.5 < Value < 0.75 0.75 < Value
No Planned AC	Unexpected aircraft or aircraft operating outside standard flows in the 5 min period	This output node is divided into three states from 0 to 1	Value < 0.28 0.28 < Value < 0.35 0.35 < Value
Performance	Considers the difference in aircraft performances at time t	This output node is divided into three states from 0 to 1	Value < 0.35 0.35 < Value < 0.7 0.7 < Value
Conflict identification	It is considered that there is a detected conflict according to the conditions set out in the first section of this node	It is divided into 2 states, 0 (when there is not a detection of the conflict) and 1 (when the conflict is detected)	Value = 0 Value = 1

includes a description of the variables of the model as well as an explanation of the discretisation criterion followed for each of them. Discretisation consists of converting the continuous variables into variables grouped by intervals. This step is necessary since most algorithms are optimised for discrete variables.

The discretisation of the variables can be based on statistical characterisation or expert knowledge. The discretisation must ensure that no information is lost or too many states are considered [26].

A possible example of discretisation for each variable is also included for information purposes, as it is considered more visual. However, this discretisation could vary for the same variable depending on the scenario in which it is found.

4.1. Parametric Learning

Figure 3 shows the probabilities observed directly from the data for the network parameters, with particular reference to the Santiago Sector (LECMSAN).

After looking at the directly observed probabilities of the Santiago Sector data, it is worth noting that at the exit node, 95% of the records are in State 1, i.e., they are conflicts that have been detected by the controller. Even so, it should be noted that the ATM system has a considerable number of safety barriers between this detection by the ATCo of a potential conflict and a possible loss of separation.

Another aspect to highlight is that most of the cases of the ATC Workload variable are in State 3 and 4, specifically 82% of them, which indicates that in this sector there is a high workload for the ATCs.

Finally, with regard to the distribution of vertical entry speeds in the sector, it should be noted that in 99% of the cases, the median of this variable per hour is 0, which indicates that in this sector practically all flights are stabilised.

4.2. Sensitivity Analysis

This section shows the results after performing a sensitivity analysis [27] on the Bayesian network. This analysis is carried out with the intention of identifying which variables have a greater influence on the output variable, i.e., conflict detection.

For the sensitivity analysis, the variable “Conflict detected” has been selected as the target variable. Figure 4 shows the results of the sensitivity analysis carried out on the Santiago Sector network.

The variables closest to the target node (on the right-hand side of Figure 4) that are most sensitive to the target node are: “ATC workload”, followed by “No planned AC”, “Performance” and “Traffic conditions”, respectively.

As for the input variables most sensitive to this node, the most important are the percentage of regulated aircraft (“% AC regulated”) and the vertical and horizontal speed distributions at sector entry time (“Proportion Vertical Speed Diff”, “Proportion Horizontal Speed Diff”). The relationship with the “Proportion Vertical Speed Diff” has a great impact due to the “Proportion Horizontal Speed Diff”. Vertical Speed Diff” has a great impact due to the reasons explained in the previous section.

On the other hand, it should also be noted that less sensitive variables are the hourly distribution of flight levels (“Entry FL Distribution”), aircraft over declared capacity (“AC over capacity”), and the mix of actions performed by the ATCo (“Action Mix”). Therefore, it could be considered to remove these variables from the Bayesian network created in order to calibrate it.

5. Discussion: Most Influential Factors in the Bayesian Network Model (BNM)

Once the sensitivity analysis had been carried out, it was considered interesting to evaluate the most influential factors from another perspective [28].

Tornado Analysis measures the impact of each variable or combination of variables and states of the model on a target forecast.

Thus, Figure 5 shows the tornado diagram of network 3 of the LECMSAN sector. It shows the ten variables or combinations of variables that are most sensitive to a change in the output node “Conflict_detected”.

Firstly, it is observed that the combination of variable states most sensitive to a change in the exit node is given by a low aircraft vertical speed difference (“Proportion_Vertical_Speed_Difference” = s1_below_0_35) and that the vertical speed distribution at the sector entry is negative (“Entry_Vertical_Speeds_Distribution” = s1_below_0”).

The number of unplanned aircraft in the sector (“No_Planned_AC” = s3_0_35_up) is also very sensitive to change. Moreover, for the first three cases, an inverse relationship with the change in the output variable is observed. That is, an increase in the output variable implies a decrease in any of these three combinations of variable states.

To summarise, there are variables and states that are repeated throughout the graph, thus establishing themselves as highly relevant variables vis à vis the output node (“Conflict_detected”). These variables refer to the vertical (“Proportion_Vertical_Speed_Difference”) and horizontal (“Proportion_Horizontal_Speed_Difference”) speeds of the pair of aircraft, their performances (“Performances”) and, finally, the number of unplanned aircraft in the sector (“No_Planned_AC”).

As the sensitivity of the variables decreases, the symmetry is lost; in this case, it reflects that a 10% increase in the output variable will have a greater influence than a decrease in the output variable.

Forward Analysis

The last of the analyses carried out on this potential conflict detection probability network is the Forward Analysis. Forward analysis is used to predict the effects, i.e., the level of uncertainty (output-node-child), by establishing the probability distribution of the input nodes.

The forward analyses that have been carried out are based on changing the configuration of the parent nodes of each network. This new configuration consists of establishing specific conditions for two different scenarios. The first scenario taken into account is the case in which the value of the hourly AC inputs is less than 20% of the sector capacity, and the second case for a value of the hourly inputs higher than 90%.

Through data processing, we have obtained the configurations that the parent nodes of the networks would have if these scenarios were to occur. By changing these configurations, the forward analysis allows us to see how these changes propagate to all the nodes in the network.

Comparing Figure 6 and Figure 7, it can be seen that the intermediate nodes have undergone notable changes, as has happened, for example, for the variable representing the entry of aircraft into the sector every 5 min (“Instantaneous_Demand”). Logically, in the case of inflows above 90%, the instantaneous demand has increased in states 3 and 4 compared to the case of inflows below 20%.

Other variables that have also modified the configuration of their states are the distribution of flight levels at the sector entry (“Entry_FL_Distribution”), or the actions that the ATCo performs per hour (“ATC_actions_hour”), for the case of 90% these actions have increased.

However, no variation is observed in the output node (“Conflict_detected”). This is because it has had a variation in its states of 2 per thousand, but the graph shown is not able to capture this variation due to the rounding of the percentages. This order of magnitude in the output node variation is consistent with the rate of error variation for a skilled person performing a complex task. This rate is on the order of one per thousand.

6. Validation

The basic concept in the validation is the goodness of fit of the model. The proposed model is, in effect, a BN structure.

Inspired by classical validation techniques, two different use cases were defined to test the goodness-of-fit of the network. Each use case refers to a specific sector of the Spanish airspace (LECMSAN and LECBCCC, respectively). Furthermore, each use case was divided into two scenarios; these scenarios represent extreme conditions of sector occupancy rates, therefore both high levels of sector occupancy (90%, 80% and 70% occupancy) and low levels (40%, 30% and 20% occupancy) will be taken into account.

For each scenario, a representative dataset was separated from the learning data and subsequently used to test the predictive ability of the BN.

The predicted results of the BN (predicted success probability of the ATM barrier) are compared with the actual ones to determine the validity of the statistical approach (

Table 2. Bayesian Network validation.

		Use Case 1			Use Case 2
	Occupancy	Actual Data	Model Outputs	Error	Actual Data	Model Outputs	Error
Scenary 1-High Occupancy	90%	93.527%	93.860%	0.333%	97.198%	95.915%	1.283%
	80%	95.165%	93.856%	1.309%	96.706%	95.815%	0.891%
	70%	92.374%	93.845%	1.471%	96.642%	95.824%	0.818%
Scenary 2-Low Occupancy	40%	94.584%	93.959%	0.625%	96.381%	95.917%	0.464%
	30%	93.434%	93.948%	0.514%	95.946%	95.913%	0.033%
	20%	95.235%	94.153%	1.082%	95.879%	95.918%	0.039%

).

The ATM safety barrier concerning the potential conflict identification by the ATCO is divided into three columns. The first refers to the percentage value provided by the data, the second to the value predicted by the BN model, and the third reflects the difference between the two values or the error.

Thus, the most important way of determining whether a model is good is ensuring that the difference between the actual and the predicted is as small as possible, i.e., the error is as small as possible.

In addition, the box plot has been selected as appropriate for this purpose, as this diagram is suitable for comparing the range and distribution of numerical data sets, illustrated by a box and a central line in the middle. The chart is suitable for comparing the range and distribution in numerical data sets.

The first thing to note, which can already be seen in Figure 8, is that the errors are very small (less than 1.5%), considering that they are expressed as a percentage.

Furthermore, the plot reflects little dispersion of the data, there are no outliers and the extreme values are close to the mean. It is for this reason that the model is considered tested and valid.

7. Conclusions

This article has dealt with the highly relevant issue of analysing the probability that the ATCo will detect a potential conflict and thus prevent it from becoming a possible Separation Minima Infringement.

For this purpose, a model based on data and knowledge was developed using Bayesian Inference techniques due to its high statistical capacity in low probability events. In this way, the Bayesian network is constructed by defining all the variables considered influential for the defined safety event.

The differentiating element of this study in comparison with previous analyses is the large number of variables identified for which real data have been obtained to support the study, specifically the variables related to ATCo performance, since they are not easily accessible.

Thus, after carrying out the parametric analysis, it was decided to perform a sensitivity analysis on the potential conflict estimation network by the ATCo in order to clarify which precursors have a greater explanatory capacity for this safety event.

In this way, conclusions can be drawn, among which the high influence of the ATCo’s workload on the detection of potential conflicts stands out. In other words, in more saturated sectors or with a higher traffic complexity [28], it is less likely that the ATCo has the capacity to detect all potential conflicts, thus having to resort to other ATM safety barriers to avoid the progression of an accident.

As explained above, the complexity of the sector is another aspect to be taken into account. This is why there are variables such as the % of regulated ACs or the differences in vertical and horizontal speeds of the ACs at the entrance of the sector that also show a highly explainability on the target node.

It was also intended to evaluate the influence of combinations of variables on the influence of potential conflict detection. For this purpose, tornado analysis was used. This analysis shows, for example, the high influence of the difference between the difference in vertical velocities and their distribution of aircraft at the entrance to the sector in question.

Finally, a forward analysis is developed to assess the change in the distribution of the exit node that would result from a change in the entry nodes. To this end, two scenarios are selected, one with high occupancy and one with low occupancy, in order to be able to compare the output distribution of the safety event. Thus, it is observed that although there are significant changes in the intermediate variables of the model, these are not transmitted to the distribution of the output variable.

It is considered that a very complete model has been developed, which incorporates variables that had not been taken into account until now and that allow clarifying some of the most notable variables when analysing the performance of the ATCo in detecting potential conflicts.