FRAM-Based Analysis of Airport Risk Assessment Process

Abstract

The safety of flight operations and passengers is one of the main criteria for evaluating airport performance. Risk analysis and assessment are used to ensure safety in the airport’s decision-making process. This paper aims to formally analyze these processes and look for weaknesses that can lead to erroneous assessments and, thus, ineffective decisions. Given the specific nature of the issue, which requires the Safety II approach, it was assumed that the analysis would be carried out using a systems approach, considering all factors and relationships affecting the effectiveness of the examined processes. As relatively small-scale deviations from the planned operation of individual system functions are observed in real-world analyses, the Functional Resonance Analysis Method (FRAM) was selected as adequate for such situations. A formal study of the risk assessment process was carried out, focusing on two cases chosen during the initial identification of possible hazards. Applying the FRAM proved an effective way to analyze and search for functional resonance in the airport risk assessment process. It also enabled the identification of mitigating actions, which allows for breaking the chain of variability that leads to an unfavorable course of the process.

1. Introduction

1.1. Airport Safety

Airports play a key role in air transport, enabling the handling of passengers, cargo, and aircraft. The scope of responsibility of the airport operator is very diverse (from tasks related to the construction and maintenance of infrastructure through managing the staff training system to marshaling and handling the aircraft). Each aerodrome constitutes a complex organizational and technical system that guarantees maximum safety for performed flight operations and passengers. In aviation, a hazard is considered a latent potential for harm present in one form or another in a system or its environment. This potential for damage may occur in various forms, for example, as a natural condition (e.g., terrain) or a technical condition (e.g., runway markings). Aerodrome operators have to cope with numerous challenges to ensure operational safety. The aerodrome’s SMS must include various events, such as technical system failures, airport ground traffic, the complexity and strict timing of ground handling operations, or adverse weather conditions.

All of the aspects mentioned above are reflected in the literature. Adverse atmospheric conditions, especially low visibility, cause hazards mainly during the landing phase and in airport traffic during operations such as aircraft taxiing and the movement of maintenance vehicles. The primary way to mitigate these hazards is to use special Low Visibility Procedures (LVPs) and assist traffic participants with technical systems that improve situational awareness and decision support systems [1,2,3,4,5].

The most vulnerable areas are runways and taxiways, mainly due to the much higher speed of aircraft [6]. The most common types of incidents are Runway Incursions and Runway Excursions, which can lead to collisions. Preventive measures in the form of guidance systems, strict adherence to safety protocols, and controller instructions or runway condition monitoring devices have been analyzed in [7,8]. A particularly relevant system in this context is the A-SMGCS (Advanced Surface Movement Guidance and Control System) successfully implemented at many airports [9,10].

To illustrate safety issues that may pose a hazard, several examples are presented below [11].

• Change in airport equipment—the introduction of new radio communication system devices and withdrawal of devices from the currently used radio communication system.
• Risk related to wildlife at and around the airport.
• Change in airport topography/design.
• Construction works at the airport.
• Incorrect taxiing of an aircraft at the airport.
• Carrying out tests of aircraft engines in a place not designated for this purpose.
• The ineffective training process in the aviation organization.
• Lightning strike on an aircraft during ground handling.
• Foreign object debris on the runway.

In each risk analysis, airport specialists must consider human factors. They are social and personal skills such as communication, stress management, and teamwork. These factors decide how people carry out their jobs, so they significantly affect safety. The human factor can be a hazard, contributing factor, or barrier to another hazard [12,13,14].

These issues require following international requirements, conducting proactive risk analyses, and taking necessary countermeasures [15,16]. Such analyses performed in Safety II terms are the focus of our work. Risk analyses are conducted under SMS (safety management system) principles, which have been mandatorily implemented at airports [17,18]. The human factor plays a key role in conducting these analyses, as it is the most significant factor affecting airport safety [19,20].

1.2. Airport Risk Analysis

One of the key tools for ensuring safety is risk analysis. Suppose it is about past events such as an accident or incident in air traffic. In that case, its purpose is to seek among their causes and circumstances favorable, permanent, recurrent phenomena that can be eliminated. If, on the other hand, it is about planned changes and modernization in the airport’s operating system, then it serves to check whether these activities will not decrease safety. Both scopes of activity of analysts performing risk assessments are essential, and both face similar problems, which are addressed in this article.

The methods used in risk analysis can be roughly divided into qualitative and quantitative [21]. Probabilistic methods are fundamental in the latter [22]. The entire system of risk analysis and assessment in air transport is usually based on safety performance indicators [23]. They are, most often, constructed to describe the number and frequency of aviation incidents, the safety of airport management processes, and operational safety [24]. Several review articles have already appeared in the subject literature discussing different areas of airport risk analysis [25].

Recently, methods based on fuzzy logic have become increasingly popular in airport risk analysis [26]. They respond to the problem raised in this work—the subjectivity of assessments and the need to rely on expert knowledge. Similarly, methods based on data analysis [27], Bayesian networks, and Petri nets are developing [28].

A literature review indicates that a risk analysis of airport processes is carried out based on numerous practical experiences and supported by regulations, such as the ICAO Safety Management Manual [29]. However, neither the long-standing practice of conducting safety analyses nor international regulations have fully established a clear-cut approach to specific incidents and practical problems. As a result, conducting these analyses is fraught with a high level of subjectivity, and consequently, the results are not entirely conclusive either. There are many reasons for this, among them the wide variety of events analyzed and the need to determine quantities that are difficult to define objectively, such as the probabilities of events involving humans or the effects that occur if a hazard materializes.

1.3. Safety I Versus Safety II

Even assuming that the very structure of the risk analysis process at the airport is defined precisely, there are minor deviations, which in themselves cannot be considered errors or irregularities. They are expected, as it is impossible to ensure the complete repeatability of a process in which a human plays a central role. However, the consequences of these deviations can cause a significant qualitative difference in the effectiveness of the risk analysis process and the conclusions we draw from the process. The airport risk assessment process thus defined is a classic example of an issue that should be analyzed according to the Safety II approach. It analyzes not so much the errors or malfunctions of the system but rather its proper operation in the context of possible variability, which can lead to different outcomes despite being normal in themselves. Likewise, the subjects of analysis in this approach are minor incidents with little impact on safety rather than accidents with dire consequences. Unlike the traditional Safety I approach, which focuses on risk reduction and preventing adverse events, Safety II focuses on identifying and strengthening mechanisms that enable an organization to operate in dynamic, unpredictable conditions [30,31].

Of the possible approaches to analyzing airport safety as a complex socio-technical system, the most common in the literature are STAMP (System-Theoretic Accident Model and Processes) and FRAM (Functional Resonance Analysis Method). These methods are used to model and analyze such systems to understand better how different functions interact and lead to positive and negative outcomes. This approach is beneficial in Safety II, as it analyzes how a system functions under normal conditions [32,33,34].

The Safety II approach focuses on different approaches to the systemic analysis of a problem. A comparison of earlier linear approaches with the systems approach is presented in [35]. Some methodological considerations regarding the extent of detail in the analysis are presented in [36]. In this type of analysis, proper representation of uncertainty is significant [37]. STAMP and FRAM are generally qualitative, but a clear desire exists to express quantitative aspects [38]. Some specific works using the systems approach are also worth mentioning, which are interesting in the context of the considerations carried out here [39,40,41].

1.4. Concept of Analysis

The literature review indicates the high applicability of systems methods in analyzing complex socio-technical systems, including the airport safety assurance system. It was also noted that most work deals with detailed operational (maintenance) processes implemented at the airport. However, there is a lack of research on the effectiveness of the risk analysis process. This represents a research gap that needs to be filled, as the results of this analysis, according to SMS principles, form the basis for proactive implementation of changes in the airport safety system. Thus, any problems occurring in the risk analysis process translate into inappropriate (or ineffective) actions by the airport operator to maintain an unchanged (or increased) level of airport safety.

This paper aims to analyze the risk assessment process at the airport and evaluate the possible variabilities in this process using a selected airport as an example. This analysis will allow us to identify if this process needs to be changed. To achieve this, we will use the FRAM, mentioned above and discussed in more detail in Section 2. We will use the approach proposed in [42], which will allow us to analyze how the risk assessment process takes place at an airport under normal conditions and what impact this has on ensuring the safety of transportation processes in the area. The FRAM is useful wherever we deal with complex systems in which humans participate. It allows us to see how normal operations can lead to unexpected outcomes. This approach is consistent with the proactive safety culture introduced in aviation.

Section 2 defines the subject of the modeling, the airport risk analysis process. Section 3 briefly describes the FRAM used for the analysis. Section 4 presents the FRAM model of this process, implemented at Warsaw Chopin Airport (ICAO code: EPWA, IATA code: WAW). This airport was chosen as a case study because one of the paper’s authors is professionally engaged in performing risk analysis at this airport. The model was implemented using the FMV package (FRAM Model Visualizer). Subsequently, the functions carried out in the risk assessment process at the airport were discussed, along with an indication of the factors that determine their performance. Then, the possible variability of these functions, resulting from typical deviations in the execution of the process, which are encountered in the operation of this process, was presented. Based on the analysis of the interrelationships between functions, after taking into account the impact of the variability of individual functions on their performance, key risks were identified, and the sequences of functions that form functional resonance were presented. Based on this, mitigating actions were proposed to break such sequences and remove dangerous functional resonance. Section 5 discusses the research results, and the conclusions are listed in Section 6.

2. Risk Assessment Process at the Airport

Under European law, airports with significant air traffic (more than 10,000 commercial air passengers or more than 850 cargo operations per year) must implement a safety management system (SMS). One of the SMS processes is risk assessment. The requirements for this process are primarily defined by [43] and related decisions by the executive director of the European Aviation Safety Agency (EASA) and [44]. When creating airport safety management systems, consideration is also given to guidelines published by the International Civil Aviation Organization (ICAO) and, if they exist, additional national and internal requirements of the organization that manages the airport in question.

Airport managers apply the risk assessment process covered in this article to issues, changes, or activities during which an aviation hazard could occur or where one has already occurred. The process is applied both to airport daily life (e.g., for assessing the risk associated with the presence of FOD (foreign object debris) on the runway), to significant changes at the airport (e.g., for assessing the risk associated with the construction of a new aircraft apron), and to management issues (e.g., for evaluating the risk associated with the turnover of operational services personnel). The listed groups of analyses represent their most common topics but do not represent a closed catalog. The risk assessment aims to ensure safe aircraft operations by identifying hazards and mitigating the associated risks.

Risks are assessed based on an analysis of the issue using available information and data, which is the input to the process. When describing and analyzing a problem, use is made of, among other things, the results of studies of past aviation incidents (especially those that occurred at the airport to which the assessment is being conducted), the results of audits and inspections, the results of safety reviews, reports from safety reporting systems, or previously conducted risk assessments. It is also necessary to have the right resources to carry out the process: competent personnel, the organization’s procedures, risk register, and IT tools. When conducting a risk assessment for a specific airport, we need to know the procedures and possible constraints that exist at the airport. Considering local conditions is very important (risks from materializing the same hazards for two different airports can be quite different).

At its various stages, the risk assessment process involves both SMS personnel and those who support it, e.g., through consultation. Consultants are often specialists in the analyzed risk, i.e., employees of organizational units of the airport manager other than the SMS or employees of different organizations cooperating with the manager. Conducting effective consultations is often a challenge. Their outcome has a significant impact on the quality of risk assessment results.

The first step in conducting a risk assessment is selecting a method or set of methods for use at various process stages. The quality of this selection depends primarily on the competence of the individuals and the features of the security management system procedures. The next step is to identify the risks associated with the subject. Examples of hazards for the construction of a new apron at an airport include (a) the use of cranes that can be an aerial obstruction or (b) the rearrangement of ground traffic. SMS procedures should be used to control hazard identification, including a description and guidelines for applying the methods used. Consulting with specialists in the area is essential to correctly identifying risks.

Subsequently, the risks associated with the listed hazards are assessed. Risk assessment involves estimating the probability and severity of the potential consequences of materializing a given hazard. For the cited example of the hazard “presence of cranes in the airport area”, a possible result is, for example, “collision of an aircraft approaching landing with a crane”, the occurrence of which is characterized by a certain probability and severity. For risk assessment, trained personnel use the method specified in the SMS documentation and IT tools. The product of the risk assessment is a risk index (a letter indicating severity and a number indicating probability). In addition, it provides information on the need for risk mitigation (is it necessary to implement risk mitigation measures to maintain an adequate level of safety?). This information is passed on to management, which accepts (or does not) the indicated risks, i.e., allows the action or change to be implemented at the airport.

If management accepts the risks on condition that they are mitigated, mitigating actions are established at this stage. They can be aimed at reducing the probability or severity of the materialization of risks. If the need to develop risk mitigation actions has not been indicated (risks are accepted without mitigation), then the specified risk index is the final. If risk mitigation actions have been established, the risk must be reassessed. The risk that is the product of this action is called residual risk.

Along with the final risk indexes, conclusions are formulated to help management make decisions that contribute to improving (or at least not weakening) the safety of airport operations. In addition, following the described process, recommendations for improvement of the safety management system can be formulated, which can be used when conducting subsequent risk assessments (the goal is to improve the SMS). It is essential to document the risk assessment conducted reliably. This documentation feeds into the SMS and is one data source for future risk assessments. It is vital to include the results of each evaluation conducted in the airport’s risk register (which is used as input in the risk assessment process).

3. Method of Analysis of the Airport Risk Assessment Process

3.1. Functional Resonance Analysis Method

As mentioned, the Functional Resonance Analysis Method (FRAM) for modeling and analyzing systems was developed to provide a tool to approach a systems study following the Safety II concept [45]. The analysis realized by the FRAM is based on four main principles:

• The incorrect and correct operations of the system are equivalent to each other, which means that different operation effects result from the same internal mechanisms and events.
• Matching human actions to conditions is approximate and never ideally in line with what was planned.
• It is impossible to plan everything, and unforeseen phenomena may occur.
• An emergency arises from functional resonance, which means the superimposition of multiple signals unpredictably.

The FRAM’s principle is to describe a system’s functions in a way that is independent of how they may be connected in a particular situation [46]. Each function is characterized by six components, graphically represented by a hexagon (Figure 1).

The specific values of these components represent the actual linking of functions within a particular case. To build a FRAM model, we must first define the functions of the analyzed system or process. A researcher should analyze available documentation describing it and conduct observations and interviews. The components describing the functions of a system are as follows:

• I (Input)—determines the values that the function transforms;
• O (Output)—determines the values that the function produces;
• P (Preconditions)—defines the preconditions that must be met for the function to be realized;
• T (Time)—determines the time availability of the function;
• C (Control)—determines the signals that control or modify the function;
• R (Resources)—defines the resources that the function needs (or consumes) during its execution.

To set the boundaries of a model, we need functions with only outputs or inputs; these are called background functions. All other functions are called foreground functions. We can also classify each function, based on its type, as an organizational, technological, or human function. Functions that must be delivered before others can begin are called upstream functions. Downstream functions depend on upstream ones. To build the complete model, it is necessary to indicate couplings in the given system (how functions connect). After validating the model, it is possible to analyze the system’s behavior.

The second step, identifying actual or potential variabilities between functions, can be based on observations of actual or simulated work situations and a formal process. The formal process (Hollnagel’s simple solution) is methodically questioning the performance of every function in terms of timing and precision. Possible variabilities of functions are as follows:

• In terms of timing—early, on time, too late, omission;
• In terms of precision—precise, acceptable, imprecise.

Also, the origin of variability needs to be identified for all the outputs. It can be classified as endogenous (internal) or exogenous (external). Subject Matter Experts (SMEs) assign a rank to variability for each function, based on their experience in the field with the knowledge obtained about the specific case, by a simple color-coded scale on three values:

• Red, function’s timing, and precision have a profound effect on how downstream functions are performed.
• Yellow, function’s timing, and precision have a potential impact on how downstream functions are performed.
• Green, function’s timing, and precision have a limited or negligible effect on the system, with no consequences on how downstream functions are performed [47].

The FRAM has been successfully applied to analyzing air traffic issues, particularly airport traffic operations. This work will use it to study a procedure related to airport management, an issue much broader than operational problems. It is worth noting, however, that according to the principles of Safety II and the FRAM, this analysis must be somewhat indirect. This is because it aims to study the correct operation, while conclusions about possible irregularities can be made based on observing the effects of deviations from nominal states. A FRAM analysis is carried out in several stages [44,45].

• Identify the system’s essential functions and characterize each with six basic elements (Figure 1). Using this type of structure allows for describing more complex dependencies because by relating each function to many others simultaneously; it is possible to create a graph structure of a more general nature. Thus, a more systemic approach to incident analysis is possible.
• Determine the type and range of variability of individual functions. In the case of the airport risk analysis process, the range of variability can be determined by defining typical system operating conditions. In doing so, it is essential to consider that the function can map the technological aspect of the system, the organizational aspect, and the human aspect. The type and scope of variability can vary for each type of function and are usually defined descriptively. Functions of a technical nature are typically characterized by low variability and are relatively independent of the environment, but any deviation from the nominal state usually occurs very quickly. On the other hand, human-related functions tend to be highly variable and strongly dependent on the environment, and deviation from the nominal state occurs slowly.
• Determine the so-called functional resonance, which can occur due to overlapping deviations resulting from the variability of all functions simultaneously and the existing relationships between them. As part of the creation of the graph structure, in the first step of the analysis, the relationships between the system’s functions are defined. A change observed in one function will cause a change in the input (precondition, resources, time, control) for another function. This, in turn, can cause a deviation in the results of its function. Thus, any deviation can propagate through the network and amplify or extinguish, depending on the type of ties between functions. If we find that outputs are moving into unacceptable areas, this indicates the possibility of an air accident and thus requires further analysis.
• Identify barriers to variability in individual functions and monitor how much the proposed barriers improve system performance. In general, safety barriers can be described by their physical structure or organization and by how the barrier achieves the purpose for which it was established. The FRAM distinguishes between four types of barriers: (i) physical, which prevent an action from being performed or block undesired effects from occurring; (ii) functional, which specify additional conditions necessary for an action to be performed; (iii) symbolic, corresponding to physically existing elements that impose constraints on the performance of a function; and (iv) intangible, which also impose constraints on the performance of a function but do not exist in a physical sense.

To summarize the importance of applying the FRAM to the study of the airport risk analysis process, it can be said that this technique can allow us to understand better how the system worked (or how it could have worked) when it caused abnormal or ineffective operation. This framing of the problem is particularly beneficial for processes that do not directly lead to hazardous aviation incidents. The FRAM allows us to initially determine the conditions under which the system may operate when an abnormality occurs, and then, after determining the type and extent of deviations in signals and the way functions operate, determine what other possible outcomes of the system’s operation are, both positive and negative.

The Functional Resonance Analysis Method has a wide range of applications. It was first used in aviation and healthcare and later spread to other industries. The FRAM can be used both for retrospective and prospective analysis. In this article, the FRAM is used for design evaluation, one of the forms of prospective approaches. It is performed to find conditions or factors that may counteract or prevent the chosen system from functioning as intended. The aim was to see if combinations of multiple preconditions and resources can weaken the designed system or whether a lack of control or time constraints can impede intended functioning.

3.2. Software

To develop the model presented in this paper, we used the FRAM Model Visualizer (FMV). This tool enables the graphic representation of a FRAM model. The FMV has been developed and written by Rees Hill [48]. Its key features include the following:

• Graphical creation of models; the software allows intuitive placement and combination of functions, providing clarity and a logical model structure.
• Definition and customization of function attributes.
• Dynamic visualization to help identify key interactions or potential weak points.
• Scalability.
• Import and export of data.
• Support for simulation.
• Reporting of model results, including visual diagrams and summaries of function relationships.
• Cross-platform compatibility.

4. Case Study—Warsaw Chopin Airport

4.1. Brief Introduction

This work uses the FRAM to analyze the risk assessment process at Warsaw Chopin Airport. This airport handles international civil air traffic (passenger and cargo), General Aviation, and military flight operations due to the military installation on its premises. In addition, the airport is a hub for the national Polish carrier LOT. In 2023, the airport handled nearly 18.5 million passengers traveling on domestic and international routes through scheduled and charter civil air traffic. The year 2024 ended with a record 21 million passengers served. Chopin Airport is located in the very center of the city (surrounded by buildings), which is its characteristic feature. The topography of the airport is very compact and complicated (intersecting runways, a very dense network of taxiways, and parking stands). The airport currently operates at almost all available capacity. A feature of the airport that is common to many other airports of this size is the multiplicity of external entities operating there. The cited features of the airport significantly impact the airport management system, including the safety management system. Many airports around the world face similar challenges when designing their management processes.

As mentioned, risk assessment is a mandatory component of an airport’s safety management system. Risk assessment consists of hazard identification, risk assessment and mitigation, and risk-related decision-making. Correct risk assessment allows for the prioritization of risks and the proactive taking of appropriate actions to enhance safety. Thus, if this process is not carried out properly, the solutions adopted may not be entirely rational. Likewise, due to the way risk assessment is carried out, mainly its subjectivity and lack of uniform methods, it is possible for risk assessments to vary between airports despite identical conditions under which an aviation event occurred or identical operating conditions. Such a phenomenon is unfavorable from the point of view of transport policy, as it can lead to unreasonable investments, for example, from public funds.

The article presents a prospective analysis in the form of a design evaluation. The aim is to find conditions and factors influencing the risk assessment process. We look for variabilities that may prevent the process from functioning as intended. The goal is to enhance system resilience by understanding and managing these variabilities to identify opportunities for process improvement. The hypothesis that needs to be verified is that Chopin Airport’s risk assessment process can be improved to reduce the negative impact of the human factor that can deteriorate the quality of SMS results and to make better use of the organization’s knowledge and experience. It is possible to make changes to the process that will improve its efficiency by standardizing and stabilizing it.

4.2. Essential System Functions

In selecting the essential functions of the airport risk assessment process, we used the safety management system documentation and consulted Subject Matter Experts (SMEs). We also benefited from the experience of one of the authors, who professionally performs risk analysis and assessment at Warsaw Chopin Airport. A summary of the identified relevant functions of the risk assessment process is presented in

Table 1. Description of essential functions of Warsaw Chopin Airport risk assessment process.

Function	Input (I)	Output (O)	Preconditions (P)	Resources (R)	Time (T)	Control (C)
Describe and analyze the given subject (F1) (human function, background function)		CONOPS (O1)	Aerodrome procedures (O4) Internal expert opinion (O7) External expert opinion (O10) SMS software (O9)	Safety management personnel (O5)
Provide procedures and guidelines for the SMS (F2) (organizational function, background function)		Risk assessment procedure and risk register (O2) Consultation method (O3)	Safety management personnel (O5) Top managers (O6) Conclusions on quality of risk assessment process (O18)	Risk register data (O16)
Provide procedures and guidelines about the aerodrome (F3) (organizational function background function)		Aerodrome procedures (O4)
Provide recruitment and training (F4) (organizational function, background function)		Safety management personnel (O5) Top managers (O6)	Risk assessment procedure and risk register (O2) Consultation method (O3)
Internal consulting (F5) (human function, background function)		Internal expert opinion (O7)	Consultation method (O3)	Safety management personnel (O5)
External consulting (F6) (human function, background function)		External expert opinion (O8)	Consultation method (O3)	Safety management personnel (O5)
Provide software (F7) (technological function, background function)		SMS software (O9)
Choose the assessment method (F8) (human function)	Risk assessment procedure and risk register (O2)	Risk assessment method (O10)		Safety management personnel (O5)
Identify hazards (F9) (human function)	CONOPS (O1)	List of hazards for the given subject (O11)	Risk assessment method (O10) Internal expert opinion (O7) External expert opinion (O8)	SMS software (O9) Safety management personnel (O5)		Risk assessment procedure and risk register (O2)
Risk before mitigation assessment (F10) (human function)	List of hazards for the given subject (O11)	Risk before mitigation index (O12) Need to establish mitigation (O13)	Risk assessment method (O10) Internal expert opinion (O7) External expert opinion (O8)	SMS software (O9) Safety management personnel (O5)		Risk assessment procedure and risk register (O2)
Risk before mitigation acceptance (F11) (human function)	Risk before mitigation index (O12)	Need to establish mitigation (O13)		Top managers (O6)		Risk assessment procedure and risk register (O2)
Establishing mitigation actions (F12) (human function)	Need to establish mitigation (O13)	List of mitigation actions (O14)	Internal expert opinion (O7) External expert opinion (O8) Aerodrome procedures (O4)	SMS software (O9) Safety management personnel (O5)		Risk assessment procedure and risk register (O2)
Final risk assessment (F13) (human function)	List of mitigation actions (O14) Risk before mitigation index (O12)	Final risk index and conclusions (O15) Risk register data (O16)	Risk assessment method (O10) Internal expert opinion (O7) External expert opinion (O8)	SMS software (O9) Safety management personnel (O5)		Risk assessment procedure and risk register (O2)
Final risk acceptance (F14) (human function)	Final risk index and conclusions (O15)	Positive or negative final risk acceptance decision (O17) Conclusions on quality of risk assessment process (O18)		Top managers (O6)		Risk assessment procedure and risk register (O2)
Document risk assessment (F15) (human function)				SMS software (O9) Safety management personnel (O5)		Risk assessment procedure and risk register (O2)

. Each function is characterized by six essential characteristics and by its type. It is noticeable that most of the functions are human functions.

The relationships between the functions shown in Table 1 have been converted to graphical form using the FMV package. The description of functions and their couplings constitute the FRAM model, as shown in Figure 2.

4.3. Variability of Functions

In the regular operation of the process, functions can be carried out with some deviations from the planned reference result. These deviations are called function variations. They are analyzed and identified in this section. Variability does not necessarily mean component failure or human error. They are often minor deviations in terms of time or precision of function execution. These variances can be either favorable (for example, completing a task in a shorter time) or negative (for example, inaccurate execution of a task).

Table 2. Possible variability of functions.

Function	Output (O)	Possible Variability of Function’s Output in Terms of the Following:		Reason for Potential Variability:
		Time	Precision	Endogenous (Internal)	Exogenous (External)
Describe and analyze the given subject (F1)	CONOPS (O1)	early on time too late omission	imprecise acceptable precise	Safety personnel limitations	Data accessibility
Provide procedures and guidelines for the SMS (F2)	Risk assessment procedure and risk register (O2)	-	imprecise acceptable precise	Management resource limitations	-
	Consultation method (O3)	-	imprecise acceptable precise	Management resource limitations	-
Provide procedures and guidelines about the aerodrome (F3)	Aerodrome procedures (O4)	-	imprecise acceptable precise	Management resource limitations	-
Provide recruitment and training (F4)	Safety management personnel (O5)	on time too late omission	imprecise acceptable precise	Decision-making process	Situation in the labor market
	Top managers (O6)	-	imprecise acceptable precise	Decision-making process	Situation in the labor market
Internal consulting (F5)	Internal expert opinion (O7)	early on time too late omission	imprecise acceptable precise	Aerodrome personnel limitations	-
External consulting (F6)	External expert opinion (O8)	early on time too late omission	imprecise acceptable precise	-	External personnel limitations
Provide software (F7)	SMS software (O9)	-	imprecise acceptable precise	Financial resource limitations	-
Choose the assessment method (F8)	Risk assessment method (O10)	on time omission	imprecise acceptable precise	Safety personnel limitations Lack of control	-
Identify hazards (F9)	List of hazards for the given subject (O11)	on time too late	imprecise acceptable precise	Personnel limitations Lack of control	-
Risk before mitigation assessment (F10)	Risk before mitigation index (O12)	on time too late	imprecise acceptable precise	Personnel limitations Lack of control	-
	Need to establish mitigation (O13)	on time too late omission	imprecise acceptable precise	Personnel limitations Lack of control	-
Risk before mitigation acceptance (F11)	Need to establish mitigation (O13)	on time too late omission	imprecise acceptable precise	Personnel limitations Lack of control	-
Establishing mitigation actions (F12)	List of mitigation actions (O14)	on time too late omission	imprecise acceptable precise	Personnel limitations Lack of control	-
Final risk assessment (F13)	Final risk index and conclusions (O15)	on time too late	imprecise acceptable precise	Personnel limitations Lack of control	-
	Risk register data (O16)	on time omission	imprecise acceptable precise	Personnel limitations Lack of control	-
Final risk acceptance (F14)	Positive or negative final risk acceptance decision (O17)	on time too late	imprecise acceptable precise	Personnel limitations
	Conclusions on quality of risk assessment process (O18)	on time omission	imprecise acceptable precise	Personnel limitations Lack of control
Document risk assessment (F15)	-	-	-	-	-

shows the identified variability of functions in the risk assessment process at Chopin Airport.

An analysis was made of the relationship between the inputs and outputs of all functions to look for functions that significantly impact the effectiveness of the airport’s risk assessment process (

Table 3. Dependence of functions on each output.

Output		Function															Number of Relations
		F1	F2	F3	F4	F5	F6	F7	F8	F9	F10	F11	F12	F13	F14	F15	I	P	R	C
O1	CONOPS	O								I							1
O2	Risk assessment procedure and risk register		O		P				I	C	C	C	C	C	C	C	1	1		8
O3	Consultation method		O		P	P	P											3
O4	Aerodrome procedures	P		O									P					2
O5	Safety management personnel	R	P		O	R	R		R	R	R		R	R		R		1	9
O6	Top managers		P		O							R			R			1	2
O7	Internal expert opinion	P				O				P	P		P	P				5
O8	External expert opinion	P					O			P	P		P	P				5
O9	SMS software	P						O		R	R		R	R		R		1	5
O10	Risk assessment method								O	P	P			P				3
O11	List of hazards for the given subject									O	I						1
O12	Risk before mitigation index										O	I		I			2
O13	Need to establish mitigation										O	O	I				1
O14	List of mitigation actions												O	I			1
O15	Final risk index and conclusions													O	I	I	2
O16	Risk register data		R											O					1
O17	Positive or negative final risk acceptance decision														O	I	1
O18	Conclusions on the quality of the risk assessment process		P											O				1

). This analysis aimed to answer the question: is the output of which function the most common input of other functions? As a result, it was determined that <O5 Safety management personnel> and <O2 Risk assessment procedure and risk register> outputs were most frequently used during the process.

4.4. Aggregation of Variability

Variability of functions can resonate (reinforce each other) through couplings and cause unusually high function variability. This phenomenon is called functional resonance, meaning it is the resonance of normal variability of functions. This resonance is dynamic and cannot be identified as a simple combination of links. Of course, the reverse situation is also possible when the function has built-in elements that constitute a safety barrier that does not allow functional resonance to occur.

This section discusses how functional resonance can occur in reality while the risk assessment process is realized at Warsaw Chopin Airport. We decided to analyze the variability of the risk assessment process, focusing on the two chosen cases described below.

• Functional resonance of variability (in terms of precision) of providing procedures and guidelines about the SMS (function F2).
• Functional resonance of variability (in terms of precision) of providing recruitment and training (function F4).

4.4.1. Functional Resonance of Variability of Providing Procedures and Guidelines About the SMS

The first case analyzed the situation when the procedures and guidelines on safety management given to the specialist performing the risk assessment were not adopted to the specifics of Warsaw Chopin Airport. We assume they are very general and have not been updated for 5 years, so the function <F2 Provide procedure and guidelines about SMS> varies in terms of precision.

The F2 function has two outputs: <O2 risk assessment procedure and risk register> and <O3 consultation method>. This means that F2 is the upstream function of twelve other functions. We present the analysis results on the F2 influence on <F8 choose the assessment method> in

Table 4. Possible effects on <F8 choose the assessment method>.

Upstream Function	Output → Input	Downstream Function	Criticality of the Downstream Function	Time	Precision	Effects on Downstream Function
F2 Provide procedures and guidelines about SMS	SMS procedures do not provide guidelines on the choice of assessment method	F8 Choose the assessment method	Function’s timing and precision have a potential effect on how downstream functions are performed	On time	Imprecise	The chosen method is not accurate for the assessed safety case. [increase of variability ]

.

The function <F8 Choose the assessment method> has no controls, making it very probable that the incorrectly selected risk assessment method would be used. The poor quality outcome may be spread through the following downstream functions. We present different effects on the downstream function F9. The variability of F9’s preconditions has been considered (

Table 5. Possible effects on <F10 Identify hazards>.

Upstream Function	Output → Precondition	Downstream Function	Criticality of the Downstream Function	Time	Precision	Effects on Downstream Function
F8 Choose the assessment method	Hazard identification is difficult due to the chosen method, which is inaccurate	F9 Identify hazards	Function’s timing and precision have a profound effect on how downstream functions are performed	Late	Imprecise	The list of hazards for the given subject is incomplete or incorrect. Time was lost during the hazard identification. [high increase of variability]
				On time	Imprecise	The list of hazards for the given subject is incomplete or incorrect. [increase of variability]

).

Both cases analyzed above increase the variability of F9. The variability of the upstream function can influence both time and precision: an inaccurately chosen risk assessment method results in an incomplete and incorrect list of hazards and time loss. Performance variability grows and accumulates as the process continues.

<F9 Identify hazards> is an upstream function for <F10 Risk before mitigation assessment> as an output of F9 is an input of F10. <O10 List of hazards for the given subject> is necessary to perform <F10 Risk before mitigation assessment>. Possible variabilities of the F9 influence on <F10 Risk before mitigation assessment> are presented in

Table 6. Possible effects on <F10 Risk before mitigation assessment>.

Upstream Function	Output → Input	Downstream Function	Criticality of the Downstream Function	Time	Precision	Effects on Downstream Function
F9 Identify hazards	Without a list of hazards, it is not possible to assess risk; the quality of the list influences the quality of risk assessment	F10 Risk before mitigation assessment	Function’s timing and precision have a profound effect on how downstream functions are performed	Insignificant	Imprecise	Risks are wrongly assessed; the assessment does not point to the most probable and severe consequences. [very high increase of variability]

.

Another function of the process is <F11 Risk before mitigation acceptance>. It occurs when managers (Safety, Operational, Technical, Accountable) are introduced to the risk assessment results and asked to decide whether to establish mitigation measures. To perform this function, the output of <F10 Risk before mitigation assessment> is necessary. The output is <O12 Risk before mitigation index>. We present the analysis results on the F10 influence on <F11 Risk before mitigation acceptance> in

Table 7. Possible effects on <F11 Risk before mitigation acceptance>.

Upstream Function	Output → Input	Downstream Function	Criticality of the Downstream Function	Time	Precision	Effects on Downstream Function
F10 Risk before mitigation assessment	The index of risks before mitigation is the basis for deciding on risk acceptance	F11 Risk before mitigation acceptance	Function’s precision has a profound effect on how downstream functions are performed	Insignificant	Imprecise	Relevant risks can remain unnoticed. Acceptance decisions do not provide appropriate risk reactions. [very high increase of variability]

.

For the analysis, we assume that <F11 Risk before mitigation acceptance> was performed imprecisely (certain risks were not noticed, so they were not managed with awareness). This caused the omission of <F12 Establishing mitigation actions>. The results of this analysis are given in

Table 8. Possible effects on <F12 Establishing mitigation actions>.

Upstream Function	Output → Input	Downstream Function	Criticality of the Downstream Function	Time	Precision	Effects on Downstream Function
F11 Risk before mitigation acceptance	The need to establish mitigation triggers the function of establishing mitigation actions	F12 Establishing mitigation actions	Function’s precision has a profound effect on how downstream functions are performed	Omission	Insignificant	Necessary mitigation is not established. [very high increase of variability]

.

The effects described in Table 8 lower the quality of output <O14 List of mitigation actions>, so the variability of <F13 Final risk assessment> increases, and the <O15 Final risk index and conclusions> can be incorrect.

Also, the second output of <F13 Final risk assessment>, which is <O16 Risk register data>, deteriorated in value. It is worth noting that this output constitutes the resource of <F2 Provide procedure and guidelines about SMS>. The functions of the process in question are, therefore, interrelated. The results of this analysis are given in

Table 9. Possible effects on <F13 Final risk assessment>.

Upstream Function	Output → Input	Downstream Function	Criticality of the Downstream Function	Time	Precision	Effects on Downstream Function
F12 Establishing mitigation actions	A list of mitigation actions and previous data are used to assess the risks	F13 Final risk assessment	Function’s precision has a profound effect on how downstream functions are performed	Insignificant	Imprecise	Risk assessment is not reliable. [very high increase of variability]

.

All variabilities described above resonate with each other and increase the probability of an incorrect favorable final risk acceptance decision. This makes the SMS ineffective, i.e., it does not protect the organization from materialization of hazards. The results of this analysis are given in

Table 10. Possible effects on <F14 Final risk acceptance>.

Upstream Function	Output → Input	Downstream Function	Criticality of the Downstream Function	Time	Precision	Effects on Downstream Function
F13 Final risk assessment	The final risk index and conclusions about the assessed safety case are the basis for the final risk acceptance	F14 Final risk acceptance	Function’s precision has a profound effect on how downstream functions are performed	Insignificant	Imprecise	Wrongly positive final risk acceptance decision. [very high increase of variability]

.

The analysis of the variability of individual functions shown in Table 4, Table 5, Table 6, Table 7, Table 8, Table 9 and Table 10 represents the realization of functional resonance. Figure 3 presents this flow of variability.

4.4.2. Functional Resonance of Variability of Providing Recruitment and Training

The possible variability of the <F4 Provide recruitment and training> function was investigated as before. We assume there are problems in obtaining trained personnel (typical of most airports, including the one analyzed in the case study). For this reason, function <F4 Provide recruitment and training> varies in precision.

The F4 function has one output: <O5 Safety management personnel>, which is needed to perform most of the other functions of the overall process. In this case, we analyzed how the variability of F4 affects the final risk acceptance function (F14) via, among other functions, external consulting (F6). Due to the volume of the article, we omit tables analogous to Table 4, Table 5, Table 6, Table 7, Table 8, Table 9 and Table 10, which were used to analyze the subsequent functions included in the functional resonance in this case. We present only the final results of the analysis on the F4 influence on <F14 final risk acceptance> in Figure 4.

4.5. Propose Ways to Manage Variability

A safety barrier to the described variability of the F2 function for providing SMS procedures and guidelines is to make it mandatory for them to be reviewed and evaluated by an external, independent expert at a fixed frequency. This would be an “intangible” type of barrier. Without an up-to-date review of the procedure, the SMS specialist could not choose a risk assessment method while remaining compliant with the procedure. In the case of using an information system to implement the process, the barrier could be created as a physical one, i.e., blocking the possibility of starting the risk assessment process if there was no confirmation in the system that an expert had verified the quality and completeness of the output of the F2 function.

A barrier to the analyzed variability of the F4 function of providing airport safety management personnel would be to make it mandatory for professionals in the field to complete standardized training in communication and brainstorming. This would be a functional barrier. The analysis performed for the above-proposed mitigation measures for F2 and F4 functions indicates that its application hinders or prevents the formation of the described functional resonance.

Of course, in addition to the variability of the F2 and F4 functions described in more detail and the possibility of their propagation up to the formation of functional resonance, the other functions are also characterized by specific variations. They, too, can lead to an incorrect implementation of the airport risk assessment process. In this section, we will present only those mitigation measures whose application comes from the analyses conducted using the FRAM model.

To reduce variability, we propose introducing an expert system to assist the person in choosing a risk assessment method (F8 function), involving the potential selection of an inappropriate method that hinders the identification of risks. Using an expert system would be necessary to implement the F8 function—it would be a functional barrier. We are currently working on such a system, which continues the research presented in this article.

The absence of a hazard list or poor quality results from the function “hazard identification” (F9) if it does not run correctly. This significantly impacts the primary risk assessment function (F10), for which the hazard list is an input. We recommend introducing an additional tool—a checklist verifying that risks have been correctly formulated. This will reduce the effects of the functional resonance, which consists of a sequence: incorrect risk assessment—failure to identify the most critical potential consequences of risks. This tool would constitute a functional barrier.

To reduce the variability (in terms of the quality of implementation) of the F15 (final risk assessment) function, it is possible to introduce a symbolic safety barrier consisting of an obligation for the person who carried out the risk assessment to discuss the results of the evaluation as part of the regular airport management meetings. Such a measure would raise the risk assessor’s awareness of the relevance of their decisions to the process. It could improve communication between those performing the following functions: F13 final risk assessment and F14 final risk acceptance.

5. Discussion

The analysis confirmed that the reasons for the variability of the various functions of the Chopin Airport risk assessment process are related mainly to the human factor. The authors identified opportunities to introduce barriers to reduce the variability of this process, which can ensure its more stable course and improve its efficiency.

It was confirmed that the FRAM could be used to evaluate an aviation organization’s risk assessment process and introduce changes to improve its effectiveness. The functions of this process were identified as follows: describe and analyze the given subject, provide procedure and guidelines about the SMS, provide procedure and guidelines about the aerodrome, provide recruitment and training, internal consulting, external consulting, provide software, choose the assessment method, identify hazards, risk before mitigation assessment, risk before mitigation acceptance, establishing mitigation actions, final risk assessment, final risk acceptance, and document risk assessment. Interactions (couplings) between these functions were plotted on a FRAM map. Functional resonance for two chosen cases was analyzed. The choice of cases was inspired by research about the process in question. It was confirmed that these cases occur in real life.

According to research results, no control mechanism is provided in many functions, which increases variability. Control supervises or regulates a function so that it produces the desired output. Lack or inefficiency of control is observed in many functions of the risk assessment process of Warsaw Chopin Airport. The worst and most dangerous result of variability resonance may be a poor risk acceptance decision. It is inevitable if risk indexes are poorly defined. Wrong decisions in this scope may lead to a situation in which the organization’s resources will be focused on less critical issues, and the hazardous ones will develop unmanaged.

However, our approach has a limitation. The FRAM is a qualitative method. It does not focus on the probability of malfunctioning of the process. The analysis performed using this method provides information about possible function variabilities and their interactions (resonance). The study described in this article does not support the quantification of identified variability resonance.

Findings helped to understand the influence of each function variability on the process results. The research gave leads for improving the risk assessment process. As part of further research, we plan to create an expert system that will organize the process, introducing mechanisms to eliminate the possibility of human error. The system will consider various circumstances when conducting risk assessment at the airport and evaluate sets of features of the analyzed issue. It will allow an easier selection of analysis and risk assessment methods more appropriate to the case under study. At the same time, this system will make a preliminary assessment of the risk index in an automated manner using machine learning methods. This is to be achieved through the use of Bayesian statistical methods. The expert system mentioned will introduce a control mechanism for numerous functions of the airport risk assessment process. The article shows that such a mechanism can help reduce the variability resonance of the process in question.

6. Conclusions

This article provides a formal analysis of the airport risk assessment process. From the results obtained with the case study conducted at Warsaw Chopin Airport, we observed the following:

• The FRAM is an adequate tool for evaluating the risk assessment process in an aviation organization and identifying possible improvements.
• Warsaw Chopin Airport risk assessment process can be improved.
• Human factors are the main reason for the variability of the various functions of the Warsaw Chopin Airport risk assessment process. Variabilities of functions of this process can resonate with each other.
• The most dangerous result of the variability resonance of the Warsaw Chopin Airport risk assessment process may be a poor risk acceptance decision.
• Introducing new or improving existing control mechanisms in the Warsaw Chopin Airport risk assessment process can reduce the variability of this process.