Analysis of Safety Metrics Supporting Air Traffic Management Risk Models ^†

Abstract

This paper provides a study on safety metrics that are used to describe the airspace and identified as important factors influencing and characterizing the safety in airspace considering the traffic of aircraft in different flight phases. Different capabilities, frequently monitored, support the safety warning systems for the airspace, where precursory metrics within the barrier model presented by EUROCONTROL indicate the stages in the evolution of a possible barrier violation, triggering actions by Air Traffic Control and collaborative decision-making when airspace is degrading in terms of safety. The identified metrics have been analyzed, taking into account the overall scenario of aircraft evolution in an air traffic sector related to the intrinsic characteristics of airspace. A case study is presented related to separation minima infringements, and it addresses safety metrics based on the different tactic geometries in en-route ATM.

1. Introduction

To detect, measure, and manage risks in air traffic operations, safety measurements play a key role in ATM risk models. From these measurements, predictive models are created, supporting decision-making and optimizing safety performance. However, issues like data quality, human factors, and dynamic system interdependencies must be taken into account for risk management in air traffic to be successful. ATM domain can more effectively handle the complex safety environment of the current air traffic operations by consistently improving these predictive models.

This paper aims to analyze the adequacy of the integrated risk model (IRM) for a mid-air collision en route [1] to a case study of suborbital flight CONOPS applying a gap assessment to identify further safety metrics to extend the IRM. The identified gaps are reported and the breakdown of key gaps when applying SESAR’s risk model to suborbital missions can help to propose an extension of the existing model to address the unique challenges posed by suborbital flights.

This paper presents a preliminary and qualitative assessment because, currently, there is no risk model for suborbital operations in the literature nor are there any data that assess potential risks and hazards unique to these missions.

2. Safety Definitions

Many definitions were found on the concept of safety. The International Standardization Organization (ISO) defines safety as freedom from unacceptable risk, risk as “…a combination of the probability of occurrence of harm and the severity of the harm…”, and harm as “…physical injury or damage to the health of people either directly or indirectly as a result of damage to property or the environment” (ISO, 1999) [2]. The International Civil Aviation Organization (ICAO) defines safety as: “…the state in which the possibility of harm to persons or of property damage is reduced to, and maintained at or below, an acceptable level through a continuing process of hazard identification and safety risk management” (ICAO, 2013) [3]. The safety II concept [4] and success-based metrics [5] were introduced to define safety as the ability of a system to achieve its objectives under varying conditions and to shift the focus toward measuring how well a system performs under normal, successful operating conditions.

Key Standards, Guidelines, and Processes

The main goal of the standards is to ensure that all systems within an aircraft operate safely under all conditions, minimizing risks to acceptable.

ICAO Annex 19—Safety Management provides an international framework for the implementation of safety management systems (SMS) by aviation organizations, covering aspects like hazard identification, risk management, and safety assurance [6]. It requires both states (through the State Safety Programme) and aviation organizations (through their SMS) to establish and maintain effective safety management practices. The annex emphasizes a proactive, systematic approach to safety, helping the aviation sector continually improve safety performance and reduce risk. SAE ARP4761A and SAE ARP4754A documents provide industry standards and best practices for ensuring the safety, reliability, and performance of civil aircraft and airborne systems [7,8]. RTCA DO-178C is a cornerstone in the certification of avionics software and is recognized by both the FAA and EASA. Similarly, RTCA DO-254 is the standard for certifying airborne electronic hardware, especially in the context of safety-critical applications like flight control systems [9].

3. Safety Performance Metrics in Aviation

Safety performance metrics in aviation are quantifiable measures used to monitor, assess, and improve the overall safety performance of an ATM system. They help organizations to identify trends, evaluate the effectiveness of safety management systems (SMS), and make informed decisions to mitigate risks.

3.1. Safety Performance Indicators

Safety Performance Indicators (SPIs) are specific, measurable parameters used to monitor the safety performance of an organization, system, or process. Many authors provided studies for defining safety metrics [10,11]. They focus on identifying potential risks and tracking performance in specific areas related to safety. SPIs are typically part of continuous monitoring systems to flag areas needing attention and guide proactive safety management. The current safety performance definition considers measurement from the perspective of two main types of indicators. Particularly, SPIs help in assessing leading and lagging indicators of safety, often giving early warning signs of potential problems before they lead to accidents or incidents.

-

Lagging indicators, to measure events (e.g., safety occurrences, such as accidents, incidents, system outages, etc.) that have happened. They also measure whether safety improvement activities have been effective in mitigating identified risks. Lagging indicators measure the outcome of the service’s delivery.

-

Leading indicators, identified principally through the comprehensive analysis of the organizations (providers, regulators, and states). They are designed to help identify whether the providers and regulators are taking action or have processes that are effective in lowering the risk.

3.2. EASA Aviation Safety Report

Both EASA and EUROCONTROL follow ICAO (International Civil Aviation Organization) standards closely, ensuring harmonization in how safety occurrences are defined and reported across the aviation industry in Europe.

Actually, regarding the EASA Annual Safety Review 2024, ref. [12] provides a comprehensive review of the safety performance of the aviation system in 2023. This reflects our unwavering commitment to aviation safety and the industry’s collaborative efforts in upholding the highest standards of safety. Chapter 8 [12] covers accidents and serious incidents related to the provision of Air Traffic Management or air navigation services (ATM/ANS) in EASA Member States. For the period 2019-2023, there were 31 serious incidents and accidents with ATM/ANS contributions. The occurrence category ATM: ATM/CNS was assigned to 27 serious incidents and accidents [12].

3.3. EUROCONTROL Integrated Risk Picture (IRP)

EUROCONTROL is an intergovernmental organization responsible for coordinating and managing air traffic across Europe. Established in 1960, it plays a key role in ensuring the safety and efficiency of Air Traffic Management (ATM) within the European airspace.

EUROCONTROL’s role is to enhance safety performance in European Air Traffic Management (ATM), starting in 1999 with the introduction of Safety Regulatory Requirement ESARR 2. This regulation focuses on reporting, assessing, and classifying safety occurrences in European ATM. Key objectives of ESARR 2 include the following:

• Assessing safety performance and trends over time.
• Identifying ATM risk areas and taking corrective action.
• Investigating ATM contributions to safety incidents and addressing them.
• Improving safety in areas beyond ATM-related accidents.
• Monitoring whether operational or technical changes meet safety requirements.

The EUROCONTROL Integrated Risk Picture (IRP) [13] is the output of a “risk model”, representing the risks of aviation accidents, with particular emphasis on ATM contributions. In order to ensure that the risk model reflects ATM as it develops in the future, the risk model is founded on an “ATM model”, describing the ATM system whose risks are to be modeled (see Figure 1a).

Figure 1. IRP model. (a) Modeling approach. (b) Overall risk model structure [13].

The ATM model represents the operational concept for commercial aviation, i.e., the way in which different actors and systems (particularly within ATM) work together. This is a very simplified description, representing the interdependent nature of modern aviation in a form that is optimized for the development of the risk model. It covers the generic types of operations in the main European countries, rather than the details of all current national variations. The risk model represents the way in which different causal factors (human, procedural, and equipment failures, including the failures of safety nets) combine to result in aviation accidents. The structure of the risk model is shown in Figure 1b. A separate causal model is used for each accident category, built using fault trees. The fault trees are quantified using historical accident and incident experience, with judgmental modifications to represent future ATM changes.

4. Proposal for Risk Assessment Tailored to New Entrants

Recent years have witnessed a rapid technological improvement in the aerospace domain that has led to the entry of new kinds of platforms. These new vehicles will be generally operated above FL550 (Higher Airspace Operations—HAO). For this kind of mission, the current approach adopted by CAAs (Civil Aviation Authority) is to manage by segregating large volumes of airspace for considerable periods of time, clearly causing delays and disruption to other users. The expected increased demand for such missions makes this approach unfeasible in the medium-to-long term. Thus, there is a need to integrate HAO into conventional ATM airspace.

Clearly, HAO can pose risks to ATM. The target is to analyze the adequacy of the integrated risk model for a mid-air collision en route to suborbital flight CONOPS and apply a gap assessment to identify specific safety metrics. EUROCONTROL’s risk model for conventional aviation is based on historical data and statistical analysis of air traffic accidents, near-misses, and operational incidents. To evaluate how the suborbital operations are integrated in ATM in terms of relation to the EUROCONTROL risk model, a gap assessment of the whole IPR model needs to be performed.

4.1. Proposed Approach and Application of the Risk Model to New Entrants

This sub-section describes the proposed approach to perform the gap assessment.

Firstly, the identification of the main differences between suborbital operations versus conventional aircraft operations is performed. These differences can introduce new risks or operational challenges, such as new ATM systems for traffic separation, airspace management, and CNS infrastructure. The suborbital flights differ from traditional aviation missions in several critical ways. The following items represent classes of differences:

-

Flight Profile: Suborbital flights typically involve a very steep trajectory, reaching altitudes above the Kármán line (~100 km), and may not follow standard airways or airspace routes.

-

Airspace Usage: Suborbital vehicles may operate in less congested or entirely segregated airspace, potentially overlapping with space traffic or above typical commercial air traffic levels.

-

Speed and Trajectory: The speed and ballistic nature of suborbital flights are significantly different from the relatively steady, predictable trajectories of commercial aircraft.

-

Return to Earth: Suborbital flights re-enter the atmosphere, which can create different risks such as heat management, aerodynamics, and uncontrolled descent if not managed properly.

The mid-air collision accident category is taken into account for analyzing the separation minima infringements in en-route ATM. Table 1 and Table 2 report the precursors and barriers for the existing mid-air collision model in the case of suborbital flight.

Table 1. Precursors for mid-air collisions.

Precursors for the Existing Mid-Air Collisions	Precursors for Mid-Air Collisions of Suborbital Vehicles with Traditional Aircrafts
Strategic conflict	Diversion from nominal trajectory of the suborbital vehicle.
Tactical conflict	Mismatched speeds and altitudes of the suborbital vehicle.
Loss of separation	Turbulence and wind shear at high altitude or during the transition phase from high altitude to lower altitude.
Air proximity event
Imminent collision

Table 2. Barriers against mid-air collisions.

Barriers Against Existing Mid-Air Collisions	Barriers Against Mid-Air Collisions of Suborbital Vehicles with Traditional Aircrafts
Strategic conflict prevention	Definition of strategic diversion areas (safe buffers) where readdress traffic.
Tactical traffic separation	Fast detection of diversion and tactical separation management in the sectors.
Conflict warnings	Appropriate allocation of responsibility of sectors and diversion sectors to traffic controllers.
Collision avoidance warnings	Appropriate definition of communication protocols and related info to traffic controllers.
	Weather forecasting and monitoring at the boundary of the transition area

4.2. Gaps Analysis Framework

The proposed approach allows us to identify gaps for each module of the IRP model.

The main modules are the ATM model and the risk model. For these two models, the gap assessment delves into the associated sub-modules.

As regards the operational concept and the actors–systems interdependences that are in input to the ATM model, the gaps analysis outlined the following findings:

• The regulatory framework for conventional aviation does not consider the unique characteristics of suborbital flights. Gap: The collaboration between CAAs, ANSPs, and HAO stakeholders to define new regulatory frameworks and procedures to accommodate HAOs into conventional airspace.
• Spaceports currently operate separately from traditional airports. Gap: The coordination between airports and spaceports for integrated takeoff and landing operations.
• ATM system technologies focus on traditional aircraft operations. Gap: Satellite-based navigation integration, real-time tracking, the development of communication protocols for high-speed, high-altitude vehicles, and real-time data sharing.
• Human factor. Gap: A specialized training program on new operational profiles of suborbital flights.
• Safety protocols focus on preventing collisions for traditional air traffic. Gap: A new safety culture considering new separation standards and new safety buffers for recovering operations.

Based on the above-mentioned gaps, the ATM model is affected not only by suborbital operational requirements including type of vehicle and related flight trajectory, but also by the interaction with existing ATC/ATM systems, radar and surveillance coverage, airspace classification and management, and communication/data sharing systems. Fast-time simulations are often used for gap analysis and risk evaluations in the context of emerging air traffic technologies. Suborbital vehicle behaviors and interactions with commercial air traffic may be simulated with the use of these technologies.

With regards to the hazard analysis module and the accident category module for the risk model, the proposed gap assessment reports the following outcomes considering the analysis with respect to mid-air collisions.

Table 3 reports the identified precursors and barriers for suborbital operations.

Table 3. Precursors for mid-air collisions—suborbital operations.

Mission Phases	Precursors and Induced Events Identification	Circumstantial Factors	Orange Barriers	DCB (Demand Capacity Balance) Events
Launch	System malfunctions	MAC Mid-air collision	Temporary segregated airspace	Integration trajectory prediction and conflict resolution
	Communication errors
	Operational hazards
Re-entry	Inadequate pilot experience	MAC Mid-air collision	Data sharing	Performance-based operations
	Cognitive overload
	Lack of situational awareness
	Air–ground miscommunication
	Bad weather conditions
	Overloaded ATC instructions

The key factors affecting the risk model are reported in Table 4.

Table 4. Adequacy assessment—key factors.

Key Factors
Risk Model Framework: Lack of Standardized Procedures: Unlike commercial aviation, suborbital operations do not yet have well-defined, globally agreed-upon procedures, thus making the process of risk assessment more difficult and less robust. Space Traffic Management (STM): As suborbital operations move higher into the atmosphere, there is an increasing overlap with space traffic management, requiring models that integrate both air traffic and space operations. Hazard Identification: New hazards, such as the risk of debris, impact-controlled airspace, or new sources of collision (spacecraft with aircraft), need to be integrated into the risk framework.
Safety and Risk Mitigation: Geographic and Temporal Segregation: Suborbital flights may need exclusive or highly restricted airspace zones at specific times, minimizing the chance of interaction with conventional aircraft. This segregation could be temporary, requiring precise coordination. Safety Zones and Buffering: Establishing safety zones for suborbital trajectories (e.g., no-fly zones) is necessary. These zones would need to be accounted for in the risk model, which may require dynamic spatial modeling of air and space traffic. Real-time Monitoring: To improve risk assessment, there is a need for real-time tracking and monitoring of suborbital vehicles, which may necessitate the use of advanced satellite systems and ground-based tracking stations.
Flight Safety and Emergency Scenarios: Emergency Landings or Failures: In the event of a failure, suborbital flights may need specific emergency protocols, such as identifying areas where a suborbital vehicle can safely descend or land. The risk model would need to account for such contingency operations. Collision Risk with Air Traffic: Even though suborbital flights occur outside the typical commercial flight corridors, there remains a risk of collision with other air traffic, particularly if suborbital flights operate in or near heavily used airspace.
Modeling Tools and Data Inputs: Real-time Flight Data: Accurate modeling for suborbital operations would require high-fidelity data, such as precise trajectories, speed, and altitude data. EUROCONTROL would need to collaborate with suborbital flight operators to integrate this information into the risk model. Environmental Factors: Suborbital flights may be affected by atmospheric conditions at high altitudes, which are not always well modeled in conventional air traffic operations. These factors could influence trajectory predictions and flight safety.

Based on this, the gaps identified for the risk model are reported in Table 5.

Table 5. Adequacy assessment—gaps.

Gaps
Risk assessment to cover space-specific hazards like space debris.
Cooperation between ATM and STM to ensure safe coordination during the launch and re-entry phases.
Risks like system failures during space operations, abort scenarios, and extreme G-forces during re-entry.
Suborbital-specific KPIs, including metrics for vehicle integrity during high-stress phases (launch/re-entry).
Incident reporting systems to handle spaceflight-related anomalies, such as microgravity operations or emergency recovery procedures for atmospheric re-entry.
Involvement of specific human factors challenges, including astronaut crew training, human performance under high G-forces, and the role of ground control in space mission management.
Procedures for risk assessment of debris from rocket launches and possible failure scenarios.
Emergency response planning for ground-level risks posed by potential failures during the ascent or re-entry phases of suborbital flight.

Within the research domain, the gap analysis between the existing risk models and a new concept of risk model, as, for example, the one for suborbital operations, are conducted by tools assessing safety hazards, identifying risks, and comparing how both existing risk model and new integrated risk model handle risk factors in different operational contexts. One of the potential approaches that could be used is the HAZOP (Hazard and Operability Study) approach, which is used to systematically find potential risks while the suborbital vehicle is operating and interacting within ATM. The next step of this study will focus on the implementation of the HAZOP tool for identifying gaps in current ATM risk models with respect to suborbital operations in terms of risk factors, methodologies, and predictions. The process will be structured in the Matlab R2019b/Simulink^® environment [14], as shown in the following Table 6.

Table 6. Process for computing gap analysis.

Step	Process	Output
1. Define the Risk Scenarios.	Identify the primary hazard categories.	List of hazards
2. Define Deviation Parameters.	Identify failure modes.	Differences in identified causes of hazards
3. Identify Causes, Consequences, and safety measures.	Determine the causes, consequences, and safety measures.	Differences in potential consequences
4. Gap Analysis.	Compare the probabilities, impacts, and mitigations between the two models.	Recommendations for risk mitigation

5. Conclusions

Safety metrics have been used to set up the IRM for conventional air traffic.

The application of the EUROCONTROL integrated risk model is not fully adequate in the case of Higher Airspace operations. Modifications or extensions of the existing model are needed to address the unique challenges posed by suborbital flights. The proposed approach identifies the differences related to the suborbital flight to drive the identification of new hazards and risks. Then, starting from these risks, identifying precursors as well, it provides a gap analysis mainly for ATM in order to set up the extended integrated risk model (EIRM). Since, in the literature, no risk model is available for suborbital operations, and there are no data that evaluate possible risks and hazards unique to these missions, a preliminary and qualitative assessment is presented in this paper. Furthermore, the use of the HAZOP tool to identify gaps in current ATM risk models with respect to the suborbital operations in terms of risk variables, methodology, and prediction will be the next phase of this study.