Assessing and Advancing Safety Management in Aviation

Abstract

A safety management system (SMS) is the overall set of procedures, documentation, and knowledge systems as well as the processes using them, which are employed within an organisation to control and improve its safety performance. Safety management systems are often observed as being bureaucratic, distinct from actual operations, and being too much focused on the prevention of deviations from procedures rather than on the effective support of safety in the real operational context. The soft parts of advancing safety in organisations, such as the multitude of interrelations and the informal aspects in an organisation that influence safety, are often only considered to a limited extent. As a way forward, this paper presents two coupled approaches. Firstly, a generic tool for assessing the maturity of safety management of aviation organisations is presented, which accounts for recent insights in effectively incorporating human factors. This assessment tool provides insight into the strong and weak topics of an organisation’s SMS. Secondly, an overview is given of a range of approaches that aim to improve the safety of aviation organisations by strengthening relevant organisational processes and structures, with a focus on human factors. The relations of these approaches with SMS are discussed, and the links with topics of the SMS maturity assessment tool are highlighted.

1. Introduction

A safety management system (SMS) is the overall set of procedures, documentation, and knowledge systems as well as the processes using them, which are employed within an organisation to control and improve its safety performance. The aviation industry has an extensive tradition on processes and procedures towards assuring the safety of air transport operations. The International Civil Aviation Organization (ICAO) is the agency of the United Nations that develops Standards and Recommended Practices (SARPs) for a safe, efficient, secure, economically sustainable, and environmentally responsible civil aviation sector. The ICAO SARPs for SMS [1,2] discern four main parts (

Table 1. SMS parts in ICAO SARPs [1,2].

SMS Part	Element	Description
P1. Safety Policy and Objectives	Management commitment	Definition of a safety policy and safety objectives. The safety policy describes the organisational commitment regarding safety, the provision of resources for implementation of the safety policy, the safety reporting procedures, and the delineation between acceptable and unacceptable behaviour. The safety objectives form the basis for safety performance monitoring in reflection of the organisation’s commitment.
	Safety accountabilities and responsibilities	Designation of an accountable executive for the implementation and maintenance of the SMS, the definition of lines of safety accountability in the organisation, the definition of management levels with the authority to decide about safety risk tolerability, and the documentation and communication of the safety responsibilities, accountabilities, and authorities in the organisation.
	Appointment of key safety personnel	Appointment of a safety manager who is responsible for the implementation and maintenance of the SMS.
	Coordination of emergency response planning	Establishment and maintenance of an emergency response plan for accidents and incidents in aircraft operations and other aviation emergencies, which is well coordinated with the emergency response plans of related organisations.
	SMS documentation	Development and maintenance of an SMS manual that describes the safety policies and objectives, SMS requirements, SMS processes and procedures, and the related accountabilities, responsibilities, and authorities. This documentation also includes SMS operational records.
P2. Safety Risk Management	Hazard identification	Developing and maintaining a process for the identification of hazards associated with an organisation’s aviation products or services, including reactive and proactive methods.
	Safety risk assessment and mitigation	Developing and maintaining a process that ensures analysis, assessment, and control of the safety risks associated with identified hazards.
P3. Safety Assurance	Safety performance monitoring and measurement	Developing and maintaining means to verify the organisation’s safety performance by relating safety performance indicators with safety performance targets and to validate the effectiveness of safety risk control.
	The management of change	Developing and maintaining a process to identify changes that may affect the level of safety risk associated with an organisation’s aviation products or services and to identify and manage the safety risks that may arise from those changes.
	Continuous improvement of the SMS	Monitoring and assessing the effectiveness of an organisation’s SMS processes to enable continuous improvement of the overall performance of the SMS.
P4. Safety promotion	Training and education	Developing and maintaining a safety training programme that ensures that the personnel is competent to perform their SMS duties.
	Safety communication	Developing and maintaining formal means for safety communication regarding the SMS, safety-critical information, explanation of safety actions, and changes to safety procedures.

): P1 safety policy and objectives; P2 safety risk management; P3 safety assurance; and P4 safety promotion.

SMS part P1 describes the safety objectives of an organisation and the principles, processes, and methods of the organisation’s SMS to achieve them, including the accountabilities and responsibilities of the organisation’s personnel. Part P2 describes the safety risk management processes to assure that the safety risks encountered in aviation activities are controlled to achieve an organisation’s safety performance targets. Key processes are hazard identification and safety risk assessment, which lead to requirements for system design and operations that need to be assured by the organisation. Such safety assurance processes and activities to determine whether the SMS is operating according to expectations and requirements are addressed in part P3. It includes the continuous monitoring of its internal processes and operating environment to detect changes or deviations that may introduce new safety risks or the degradation of existing risk controls. Such changes or deviations then need to be addressed by the safety risk management process (P2). Part P4 describes the training, education, and communication in the SMS to achieve a positive safety culture with proper knowledge and awareness of all people in the organisation. There exist interrelations between each of the SMS components.

Regulators, such as the Federal Aviation Administration (FAA) and the European Union Aviation Safety Agency (EASA), provide standards and guidance material for safety management to aviation service providers that build upon the ICAO SARPs [3,4,5,6,7].

The interacting components of an SMS relate with the people and processes at the various levels in an organisation (Figure 1), including staff, middle management, and executives. In such organisational contexts, safety intelligence aims to ensure that managers have the right information when making decisions that can affect safety. Safety mindfulness is about creating flows of information that support the staff’s capability to remain mindful of safety when carrying out their activities and that support managers in maintaining safety oversight. Safety culture refers to the norms, values, and practices shared by all people in the organisation in relation to safety and risk. Typically, a level of safety in an organisation is materialized by the way that operations are conducted (“work as done” rather than “work as imagined”) at the sharp end, which is influenced by decisions and operating conditions created by middle and top management at the blunt end. As such, the level of safety is influenced by the safety culture, by the safety mindfulness and safety intelligence processes, and by the safety management processes.

The effort in maintaining and documenting an SMS can be considerable, as it includes a multitude of components and relates in various ways to the design and conduct of operations in an organisation. Notwithstanding the variety of components considered in SMS standards, including human factors and safety culture, safety management systems are often observed as being bureaucratic, distinct from actual operations, and being too much focused on the prevention of deviations from procedures rather than on the effective support of safety in the real operational context. The soft parts of advancing safety in organisations, such as the multitude of interrelations and the informal aspects in an organisation that influence safety, are often only considered to a limited extent. A resulting reductionist view of safety management may separate human and technical factors and thus undermine holistic views on safety in organisations [8].

A holistic view on human contributions to safety is advocated by safety management from a Safety-II perspective [9]. A key foundation of Safety-II is the recognition that human performance, individually or collectively, is variable, and that such performance variability is indispensable to effectively get the work done in the actual working context of organisations. Resilience engineering [10] supports the Safety-II perspective, indicating the ability of a sociotechnical system to adjust its functioning to sustain required operations notwithstanding changes and disturbances and the “engineering” of the sociotechnical system to achieve such ability. Resilience engineering stresses the key role of performance variability by human operators to adjust for changing demands and conditions in the working context. Based on an understanding of performance variability and disturbances in an operational context and their safety implications, the “engineering” of resilience typically encompasses activities such as training, creating flexibility, providing sufficient resources, establishing a learning culture, etc. Main challenges for organisations towards improving their safety performance often lie in keeping the organisation resilient for changing demands and conditions and advancing the informal, human-related, soft elements. However, Safety-II has not provided a ready-made, pragmatic solution that meshes well with SMS and operational processes.

In order to persuade organisations to change the way they do business safely, two things are needed. The first is a compelling reason to change, and the second is a straightforward way to change that is not too disruptive to their existing business model. These two aspects are, broadly speaking, the “why”, followed by the “what and the how”. The reason to change is often not so easy to see from inside an organisation. However, organisations and, in particular, their leaders always keep an eye on the competition and wish to ensure they are keeping up with their peers and competitors. A safety maturity scheme is therefore one way in which organisations can perceive that they can do better, and if such schemes are backed up by regulators, then there can be an industry-wide drive to “raise the bar” in areas where there are common weak spots. This has to an extent already happened in air traffic organisations (particularly in Europe) in the field of safety culture, which was brought into the existing safety maturity schemes, and now almost all ANSPs in Europe carry out periodic safety culture surveys [11].

The “what and the how” is the more methodological part of this paper, focusing on key human factors approaches relating to safety culture, mindfulness, and safety intelligence, which can be brought inside an organisation’s existing SMS without too much disruption. The following sections “unpack” these two approaches.

As a way forward, therefore, this paper presents two coupled approaches. Firstly, a generic tool for assessing the maturity of safety management of aviation organisations is presented, which accounts for recent insights in effectively incorporating human factors. This assessment tool provides insight into the strong and weak topics of an organisation’s SMS. Secondly, an overview is given of a range of approaches that aim to improve the safety of aviation organisations by strengthening relevant organisational processes and structures, with a focus on human factors. The relationships between these approaches and SMS are discussed, and the links with topics of the SMS maturity assessment tool are highlighted.

2. Assessing Safety Management Systems

2.1. Development of an SMS Maturity Assessment & Refinement Tool (SMART)

This section deals with the objective of assessing the level of maturity of a safety management system within a variety of aviation organisations, including recent insights for effectively incorporating human factors in safety management. There are several existing methods to assess the maturity of a safety management system, which include the following:

• EASA questionnaire. As part of the acceptable means of compliance and guidance material for the implementation and measurement of safety key performance indicators [12], EASA has published a questionnaire for measurement of the effectiveness of safety management. The questionnaire is based on a maturity survey in the ATM Safety Framework [13], which was developed by EUROCONTROL to support ANSPs in assessing the maturity of their SMS. This maturity survey comprises eleven study areas. The study areas are specified in more detail by one to four topics per study area and 26 topics in total. For each of these topics maturity levels are defined on a five-point scale.
• CANSO maturity scheme. The Civil Air Navigation Services Organisation (CANSO) has published a Standard of Excellence in SMS [14]. It includes a definition of SMS maturity along five levels for its thirty-six SMS objectives. The development of the CANSO scheme used the above-mentioned publications of EASA [12] and EUROCONTROL [13], but it has also added some items, and it provides some useful formulations.
• Shell SMS assessment. The SMS HSE MS self-assessment questionnaire of Shell [15] lists safety management topics and related current aviation practices, typical supporting evidence, and interpretation/guidance for aircraft operators. It consists of thirty-two topics distributed over eight groups, which are scored on a four-point scale.

These methods for SMS maturity assessment are focused on particular air transport organisations, namely air navigation service providers (EASA questionnaire, CANSO maturity scheme) or aircraft operators (Shell SMS assessment). All approaches for SMS maturity assessment are mostly based on traditional perspectives on safety and safety management, and they lack insights from recent research in advancing safety management.

To overcome these limitations, a generic air transport SMS Maturity Assessment and Refinement Tool (SMART) was developed. For this development, the EASA questionnaire [12] and the CANSO SMS maturity scheme [14] were used as a basis. Their topics were generalized, reformulated, and combined where appropriate. For instance, specific Air Traffic Management (ATM) systems, procedures, or relationships were removed to allow more general use of the questions. Next, we analysed what questions from the Shell questionnaire [15] could add new aspects, leading to addition of some questions. Furthermore, insights from developments in research in human factors, Safety-II, and resilience engineering were used as a basis for some topics. Finally, the topics in SMART were structured following the SMS components of ICAO Annex 19. Details on the development and coupling with the earlier questionnaires of [12,14,15] can be found in Appendix A of [16].

An overview of the topics for each of the SMS components in SMART is shown in Figure 2, including nine topics for Safety Policy and Objectives, six topics for Safety Risk Management, nine topics for Safety Assurance, and eight topics for Safety Promotion. For each of these 34 topics, five levels of maturity from A to E are defined, ranging from none or a bare minimum approach for addressing the topic (level A) to a most advanced way for addressing the topic, well beyond what is required (level E). A listing of all topics and descriptions of the levels A to E is provided in full detail in Appendix A, and some examples are provided next.

For example, the levels for senior management visibility and involvement (topic 1.8) range from senior management not communicating at all about safety performance to senior managers driving the process for safety excellence, being role models for safety, ensuring all staff have safety results and activity targets in their appraisals, and being personally involved in safety improvement efforts. For safety risk control (topic 2.3), the highest level includes a Safety-II perspective, wherein performance variability is considered as normal and used as a basis to define a range of performance indicators that reflect the work-as-done in the organisation. Levels of safety assurance regarding variations with respect to procedures and standards (topic 3.7) range from complete denial of such variations to systematic processes for organisational learning from variations. Levels for safety lessons (topic 4.5) range from safety lessons being learned only by those who experience them to systematic processes for gathering and effective dissemination of safety lessons.

2.2. SMART Application Schemes

SMART aims to support continuous improvement of SMS and safety in organisations. The basis of the tool use is the multi-level perspective on SMS topics (Appendix A) incorporated in a questionnaire. In the questionnaire, a user is asked for each topic to indicate a level A to E and to provide an explanation that justifies the selection. The importance of providing an explanation needs to be stressed to a user since this is key to trigger thinking and discussion regarding the way that the SMS topics are handled in an organisation and may be improved. In particular, the evaluation of levels of the topics in SMART is a means towards continuous improvement, and achieving maximum scores for the topics is not an end in itself.

The tool has been designed for use in several application schemes or configurations of user groups:

• Single user—single organisation;
• Multiple users—single organisation;
• Multiple users—multiple organisations.

These application schemes are presented next. This paper does not provide results for their use in organisations.

2.2.1. Single User—Single Organisation

The single user–single organisation case is the most straightforward application scheme. The single user can, for instance, be a safety manager or a senior member of staff from a safety department. The single user scores the topics and provides a justification. The results and the justification process help him/her to understand the strong and weak points of the organisation’s SMS. This can support the identification of ways to improve weak points of the SMS.

2.2.2. Multiple Users—Single Organisation

The multiple users–single organisation case uses the tool in a more advanced way, as illustrated in Figure 3. It consists of the following three elements:

• SMART survey. The questionnaire is completed by personnel from whom it is expected to have a reasonable overview over SMS topics, such as safety managers, staff of a safety department, and other managers. The objective is to obtain a multitude of opinions about the SMS topics from different perspectives in the organisation. People are asked to provide their opinions for the scores of the SMS topics as well as explanations of their findings.
• SMART analysis. The results of the survey are collected and analysed by independent analysts with expertise in SMS. This analysis provides statistics of the scores on the various SMS topics, pointing to views on strong and weak points and to differences in opinions on the SMS topics. The analysis of the explanations provided by the participants leads to initial insights in reasons for the scores.
• SMART workshop. The results of the analysis serve as the main input for one or several workshops with participants of the survey, depending on the size and distribution of the survey group. Each workshop is facilitated by the researchers who performed the SMART analysis. The objectives of the workshop are to achieve an improved understanding between the participants of the way that the SMS works in practice in the organisation and to arrive at ways to improve the organisation’s SMS and the ways that it can be effectively applied. Discussion of the differences in the views of the participants is important to arrive at these ends.

Figure 3. Multiple users–single organisation application scheme of SMART.

Figure 3. Multiple users–single organisation application scheme of SMART.

2.2.3. Multiple Users—Multiple Organisations

The multiple users-multiple organisations case extends the application of SMART from a single to multiple organisations. The objective is to improve the safety management systems of different organisations and to improve the interactions between the SMSs of interacting organisations. It is expected that a workshop format involving participants from different organisations is a suitable way to achieve these objectives. A basis of such workshop can be results from single- or multiple-users cases per organisation. The workshop is an opportunity for organisations to learn from the strong points of other organisations and to improve the inter-organisational safety management relations.

3. Advancing Safety Management

This section addresses the objective to provide an overview of a range of methods that aim to improve the safety of aviation organisations by strengthening relevant organisational processes and structures and especially by supporting the ways that people at all levels in the organisation understand, behave, and interact in safety-relevant situations. These methods cover the following research areas: top management, middle management, safety mindfulness, safety dashboards, safety culture, and agile response capability. For each of these areas, the links with safety management systems are discussed. Finally, the connections of the methods with the topics of the SMS Maturity Assessment Research Tool are explained.

3.1. Top Management—Safety Wisdom

Interviews were held with sixteen senior executives from organisations across the European aviation industry (airlines, airports, air traffic management, regulators, research) about their perspectives on safety performance in their organisations [17]. Each executive was interviewed by two interviewers with backgrounds in aviation and organisational safety. A topic guide was used to maintain standardised questions, such as “What kind of safety information do you consult?”, “How does safety information feed into the decision-making process?”, and “How do you monitor the impact of business decisions on safety?”. These interviews have provided a range of views by top managers of air transport organisations about their roles in safety management, such as maintaining safety under pressure, searching for evidence, seeing around the corner, and accountability and responsibility at the top [17]. These insights and practices, known as safety wisdom (to distinguish them from safety intelligence), can inspire and support top managers in improving their contributions and leadership in the safety performance and safety management of their organisation.

The strongest links of top management safety wisdom are with SMS part P1 (safety policy and objectives). This concerns the top management role in defining and keeping up-to-date the authorities, responsibilities, and accountabilities for the management of safety in the organisation. The highest organisational level must recognise its role in the SMS and actively support the development, implementation, maintenance, and promotion of the SMS throughout the organisation (including support departments). There should be continuous monitoring of the effectiveness and efficiency of the various SMS processes, including the search for “weak signals” about safety, and senior management should take effective measures to control the performance of the SMS. Senior management visibility and involvement are important for effective safety management. A strong safety commitment is characterized by senior managers actively participating in and/or supporting safety-related activities, such as training, reward and recognition schemes, safety workshops, safety conferences, and audits. They jointly develop and discuss both safety results and activity-improvement targets with staff and company contractors. They should be fully aware of the high priority areas for improvement identified in the SMS and the status of the follow-up remedial programme. Senior managers drive the process for safety excellence, and they are role models for safety. In relation to SMS part P3, concerning safety assurance, senior management support an integrated risk management system for all relevant key performance areas, such as finance, quality, security, and environment. This systematically addresses all types of risks and their relations, including assessment of costs associated with accidents and incidents and of costs and benefits of risk mitigating measures.

3.2. Middle Management

Interviews were held with 48 middle managers (who are not safety managers) at a range of European ANSPs, aircraft manufacturers, airlines, and airports about their views on safety and their roles and ways of working in the organisation [18,19]. Semi-structured interviews were conducted, and a qualitative content analysis method was used through a data-driven coding frame. It was analysed how middle managers take safety into consideration in their work, including their information management, decision making, and influencing of others. Middle managers have to balance multiple goals in getting the work done, and their leadership role is crucial for the level of safety performance that is achieved in operations. This led to guidance on how best to harness the middle management role in organisational safety.

There are various SMS topics that are relevant to the work by middle managers. It follows from the interviews that for many of these topics, middle managers do not relate to the formalities of a SMS but rather feel responsible for getting the work done while dealing with uncertainty and disturbances that may affect the operations. As such, they serve multiple goals (including productivity, finance, quality, security, environment), of which safety is only one. Doing so, they perform a kind of integrated risk management although without necessarily explicitly assessing all risks. The management of information and knowledge transfer is focused on the overall practices for getting the work done appropriately without special focus on safety. They recognized that it is important to have experts that have a good knowledge of the overall operations and the interactions between its human and technical elements in order to get the work done effectively, efficiently, and safely.

The interviewed middle managers indicated that for understanding the level of safety that they manage, they typically rely more on their own judgements and those of key staff than on using formal safety indicators. They considered safety management practices to be mostly relevant for safety management personnel although some middle managers took safety management training to obtain a broader and/or deeper safety picture. An open-reporting culture was considered important by the interviewed middle managers as part of a management style where one listens to the field. The topic where the strongest connection with the formal SMS seems to exist is for management of change. The interviewed middle managers considered it important to involve all actors impacted by a particular change and to recognize when a change would require a safety risk assessment.

3.3. Safety Mindfulness

The safety mindfulness concept describes flows of information that support mindfulness about safety in an organisation and how such mindfulness impacts decisions and actions to mitigate risks both directly within operations as well as in the management of system improvement [20,21,22,23,24]. The flow of information and the sharing and transformation of knowledge that is grounded in operational constraints represent the core activity. This dynamic information sharing requires nurturing by supportive social relations: both good coordination and leadership across relevant operational units as well as amongst management groups and teams dedicated to improvement. Horizontal escalation supports sharing of information across interdependent operations (e.g., between different teams) such that risk-inducing interdependencies may be recognized and avoided. Vertical escalation supports sharing insights and observances at the sharp end, including “weak signals”, up the hierarchy. This leads to better and more coherent safety intelligence throughout the organisation and “safety as experienced” rather than “safety as imagined”. Metrics were developed in [21] to measure safety mindfulness, ranging from metrics for safety mindfulness principles to metrics that focus on the risk perception and its adaptation of people in the organisation. Questionnaires and workshops are the means to acquire such metrics. Two software applications were designed [22,24]: (1) a social media app that allows operators to share and learn safety-related experiences in their work and (2) an improvement manager app that supports information transfer for change management. The safety mindfulness metrics and apps have not yet been applied in operations.

Given the broad scope of the safety mindfulness concept describing the flows of safety-related information as well as the governance structure for accountability, decision making, and acting in support of safety, the safety mindfulness concept relates with all SMS parts. The safety mindfulness metrics may be used as safety indicators in support of safety assurance (SMS part P3). The safety mindfulness IT applications, such as the social media app that would allow operators to share experiences in their work, may support various SMS topics. They may support the identification of hazards and disturbances for safety risk management (SMS part P2), as the apps can be used to report hazards, disturbances, and changing circumstances by staff. In support of safety assurance (SMS part P3), such apps may be used to report safety occurrences and to suggest corrective actions by staff. Furthermore, they may be used by staff to report variations with respect to procedures and standards in operational conduct and to suggest improvements. In support of management of change, the improvement manager app was designed to provide a structured way to control information flows for change processes. Recording and dissemination of safety lessons learned (SMS part P4) may be supported by the safety mindfulness app.

3.4. Safety Dashboards

The core purpose of safety dashboards is to provide a structured overview of safety-relevant information that effectively supports decision-making processes that may affect safety in an organisation. Best practices for the design and use of safety dashboards at Executive Board level were identified, leading to several prototypes for air navigation service providers [25,26]. Key areas on safety dashboards address (1) operational safety, e.g., graphs of monthly incidents, top contributory factors, and hotspot information; (2) people and culture, e.g., trends of reporting rate for safety occurrences, participation in safety training, and progress on safety culture initiatives; (3) technical systems, e.g., number and duration of technical issues and ratio between planned/unplanned maintenance; and (4) change management, e.g., trend of corrective actions, status of change projects, and top change recommendations coming from external bodies (e.g., regulator). These results support organisations to develop and fine-tune safety dashboards to provide the information that is suitable for their needs. As such, they can lay an effective foundation for the information transfer in the organisation’s safety management system and ensure the key safety intelligence rises to the top of the organisation.

There exist various ways by which safety dashboards can support safety management. In support of safety risk management (SMS-P2), they provide data that support the identification of threats to operations. The research has highlighted a need for a quantitative risk model capable of connecting multiple indicators that provides an overall risk picture, which may be integrated in future safety dashboards [25]. For safety assurance (SMS-P3), dashboards provide high-level data of operational safety, human factors, technical systems, and change management. For integrated risk management, the research has highlighted a need for integrating indicators for different KPAs, thus visualizing capacity, economic, and safety indicators in a single dashboard. In support of safety culture measurement and an improvement (SMS-P4), safety dashboards include indicators for monitoring the progress of safety culture actions and the participation to safety initiatives.

3.5. Safety Culture

The entirety of norms, values, and practices with respect to safety and risk by people in an organisation forms an important foundation for the effectiveness of its safety management and the level of safety that can be achieved in its operational conduct. As such, safety culture is at the core of the human contributions to safety in any air transport organisation. Understanding the weaknesses and strengths in an organisation’s safety culture and effecting means to support weaker aspects are key contributions towards avoiding safety occurrences. An ATM safety culture survey approach was extended to other areas of the air transport system, including airlines, airport organisations, and airframe manufacturers. A questionnaire-based analysis was done in a pan-European safety culture study of pilots [27]. This involved more than 7000 airline pilots who indicated their level of agreement on a 5-point Likert scale with about 55 statements concerning safety culture topics, such as just culture and reporting, communication and learning, risk handling, and commitment to safety. A “safety culture stack” approach was developed for measuring, evaluating and improving safety culture of various organisations in the aviation system [28,29]. First, questionnaire-based analyses were performed for each of the organisations. Next, these measurements set a basis for interaction and collaboration in workshops between these organisations, with the objective of making the total stack system safer. This was applied for a case at Luton Airport, involving key organisations, such as airport, airline, ground handlers, air traffic services, fire services, and de-icing services.

There are two types of relations of safety culture with safety management systems. The first type considers the intrinsic relation of safety culture with safety management. It stipulates that the effectiveness of safety management depends on the safety culture in the organisation. If elements of safety culture, such as safety commitment or willingness to report incidents, are at low levels, it may for instance be harder to achieve effective safety assurance (P3) and safety promotion (P4). Additionally, the safety management system can influence the level of safety culture; in particular, the actions and results of safety management may influence the beliefs of the personnel and the way that the work is done in the organisations. As an example, if there is inadequate just culture in an organisation, staff may be reluctant to report incidents or near misses.

The second type considers the relation with SMS parts. SMS part P4 of ICAO Annex 19 [1] explicitly describes training, education, and communication as part of safety management to achieve a positive safety culture with proper knowledge and awareness of all people in the organisation. A key aspect is a safety culture measurement and improvement programme, where safety culture is assessed regularly, weaknesses are identified, and there is a continuing safety culture improvement. This considers all safety culture dimensions, including the promotion of a just and open culture for reporting and investigation of occurrences, which as noted above, is an important basis for effective safety management. The safety culture stack approach adds relations regarding the interactions with external partners. These concern the improvement of safety-related interfaces with external partners (e.g., identification of newly developing risks), the harmonisation of procedures (e.g., for ground handling services at an airport) to reduce complexity whilst allowing for flexibility, active sharing of safety data and information, and sharing and learning best practices on operational safety and SMS practices with industry stakeholders.

3.6. Agile Response Capability

Development of approaches for an agile response capability (ARC) addresses events focusing on sudden crises. The ARC of actors in the air traffic system refers to their ability to anticipate/detect events, control them, and “bounce back” after they have happened. Doing so involves adapting their organisation and resource use, learning, and self-monitoring as well as the ability to coordinate activities with other actors. New approaches were developed for the development of exercises and preparedness plans for air transport crisis situations [30,31,32]. These structured approaches support single organisations and multiple interacting organisations to augment their intra- and inter-organisational capability of detecting and flexibly responding to dynamic crisis scenarios. They support air transport organisations to be better prepared for crisis situations and to be more resilient and resourceful when crises occur.

There are several relations of the ARC methods with SMS components. The strongest relation exists with the development of emergency/contingency response planning and exercises in SMS part P1. The ARC methodology provides approaches for development of guidance for analysing and aiding the intra- and inter-organisational capability of detecting and flexibly responding to dynamic crisis scenarios. As a core method, ARC supports the development of scenarios for exercises and preparedness planning, including combinations of hazards and disturbances in crisis situations (SMS-P2). It is a key aspect of the ARC approaches to analyse and support the interactions between different organisations for their agile response capability. As such, they improve safety-related interfaces and sharing of safety information with external parties as part of SMS parts P3 and P4. Training in safety and safety management (SMS-P4) is also supported by the ARC approach.

3.7. SMS Impact of the Organisational Safety Approaches

The previous sections provided an overview of approaches to advance safety in organisations. A qualitative evaluation of these approaches was made for the topics of the SMS Maturity Assessment Research Tool as presented earlier in Section 2. This qualitative evaluation considers three levels for the extent to which the approaches addressed an SMS maturity topic: (1) a topic was not addressed; (2) some aspects of a topic were addressed; and (3) the core of a topic was addressed. A graphical description of the results of this qualitative evaluation is shown in Figure 4, where darker colours depict a larger impact.

An overview of the SMS topics that are addressed most effectively by the research streams for advancing safety in organisations is provided in

Table 2. SMS topics that are addressed most effectively by the research streams for advancing safety in organisations.

#	SMS Topic	Research Stream	References
1.1	Authorities, responsibilities, and accountabilities for safety management	Top Management	Section 3.1, [17]
1.6	Emergency/contingency response procedures and plan	Agile Response Capability	Section 3.6, [31]
1.8	Senior management visibility and involvement	Top Management	Section 3.1, [17]
3.1	Integrated risk management and safety-related internal interfaces for key performance areas	Middle Management	Section 3.2, [18]
3.2	Safety-related interfaces with external parties	Safety Culture Stack	Section 3.5, [28,29]
3.3	Reporting and investigation of safety occurrences	Safety Mindfulness	Section 3.3, [20,21,22]
3.4	Monitoring of safety indicators	Safety Dashboard	Section 3.4, [25]
3.6	Auditing and improvement of SMS methods	SMART development	Section 2, [16]
4.1	Safety culture measurement and an improvement programme	Safety Culture	Section 3.5, [27]
4.5	Recording and dissemination of safety lessons learned	Safety Mindfulness	Section 3.3, [20,21,22]
4.6	Sharing of safety information and knowledge with industry stakeholders	Safety Culture Stack	Section 3.5, [28,29]
4.8	Sharing and learning best practices on operational safety and SMS practices	Safety Culture Stack	Section 3.5, [28,29]

. For each of these topics, the research streams and related sections of this paper as well as references to papers and reports are provided. It can be observed that the research streams especially support the SMS parts P1 safety policy and objectives, P3 safety assurance, and P4 safety promotion. The considered research streams have not focussed in detail on topics of P2 safety risk management, such as hazard identification and risk assessment. While this is a key part of safety management, it is already covered by a considerable volume of literature, and the considered research streams emphasized less-researched organisational processes that impact safety.

4. Discussion

Safety management systems are often observed as being bureaucratic, distinct from actual operations, and being too much focused on the prevention of deviations from procedures rather than on the effective support of safety in the real operational context. The soft parts of advancing safety in organisations, such as the multitude of interrelations and the informal aspects in an organisation that influence safety, are often only considered to a limited extent. As a way forward, this paper has presented a generic tool for assessing the maturity of safety management of aviation organisations and coupled this with recent research approaches that aim to improve the safety of aviation organisations by strengthening relevant organisational processes and structures, with focus on human factors.

The development of the SMS Maturity Assessment Research Tool (SMART) was based upon the SMS structure posed by existing standards [1,2], and it built upon existing maturity surveys either focussed on air navigation service providers [12,14] or aircraft operators [15]. The survey questions were generalized for use by general service providers in the aviation domain (airlines, airports, air traffic control centres, and related organisations) and reformulated where appropriate using recent insights in human factors and safety management. The use of existing standards and surveys as the basis of SMART and the integration of recent insights herein implies a gradual change in assessment of the effectiveness of approaches for advancing safety in organisations. Such evolution in thinking about safety management is needed to be able to achieve effective improvements. Given the large safety interests in the aviation domain, revolutionary changes in safety management are very unlikely to attain common ground.

The main objective of the use of SMART is to trigger thinking and discussion about the way that safety is being addressed in an organisation and thereby to support improvement. The evaluation of the levels of the topics in SMART is a means towards this end, and achieving maximum scores is not an end in itself. By performing surveys and explaining their views and in particular by discussing survey results in workshops, people in an organisation can be stimulated to think about the way that safety is handled and to debate how this works out for different people with various roles and responsibilities. It is expected that this will support insight into the effectiveness of an organisation’s existing SMS and supports its advancement. For this, the presented research streams provide effective links that organisations may apply to improve their safety performance.

At present, each aviation organisation must have its own SMS. However, in operations where one organisation must rely heavily on others, there is an argument for at least some kind of alignment of safety, making safety “interoperable”. Furthermore, in such areas of interdependency for safety—such as at an airport—the argument is strong because if one organisation suffers a safety-related event, it can impact others. The application of SMART from a single to multiple organisations enables those different organisations to compare and discuss safety issues as well as to share safety information and improving the coherence of working methods and training by working together to develop common best practices. This may help organisations learn from each other about practical ways for effective safety management.

This approach has been embodied in the Safety Stack approach [28,29], developed and implemented by a number of organisations at London Luton Airport who decided to “partner for safety”. The Safety (Culture) Stack approach uses results of safety culture surveys as a basis for workshop interactions. An additional perspective may be achieved by a multiple organisations application of SMART. These two multi-organisational approaches provide different though partly overlapping perspectives on the way that safety is handled by organisations. A safety culture survey provides a broad overview about the safety attitudes of a large part of the personnel and about their perceptions on the way safety is handled in operations. An SMS maturity survey provides views on the effectiveness of approaches in the organisation’s SMS according to a group of experts on SMS topics (such as safety managers, staff of a safety department, other managers). Attitudes and perceptions throughout the organisations can best be understood in combination with knowledge on safety performance and SMS details to arrive at organisational measures that support advancing the level of safety.

The design of the SMART questionnaire generalized previous questionnaires with extensions based on recent safety management literature, and the types of user groups are based on general insights for the use of questionnaires and workshops. SMART is an agile system, where both the questionnaire and the implementation of use cases can be adapted following feedback from their application and from developing views on effectively achieving safe performance in an organisation. While SMART was developed in the context of the aviation domain, its questions are all formulated in a general, domain agnostic way. As such, it can be a suitable basis to evaluate the maturity of SMSs in other domains (e.g., maritime and rail transport, process industry).

In support of improving human-related, soft aspects impacting safety in organisations, an overview of a range of methods were presented in this paper. The results of the research on top management can support executives in improving their contributions and leadership in the safety performance and safety management of their organisation. The middle management research has provided new insights and guidance that organisations can use to harness the role of middle management in organisational safety. The safety dashboard research supports organisations to develop and fine-tune safety dashboards that lay an effective foundation for the information transfer in the organisation’s safety management system. The safety mindfulness research has provided new methods and techniques that support the flows of information in an organisation to keep everyone mindful of their role in assuring safety and to provide them the knowledge to do so. The safety culture research has provided a broadened safety culture assessment and enhancement approach and a safety culture stack approach, which supports improving the safety culture and safety interfaces of a stack of interrelated air transport organisations. The agile response capability research has provided new approaches for the development of exercises and preparedness plans for air transport crisis situations, which support organisations to advance their intra- and inter-organisational capability of detecting and flexibly responding to dynamic crisis scenarios.

5. Conclusions

Every organisation is unique in the operations it conducts, in the education of its personnel, in the relations it has with other organisations, in its history, in its organisational culture, in its relations with regulators, etc. As a result, every organisation has its own strong and weak elements in assuring the safety of its operations. As such, there cannot be a one-size-fits-all approach for advancing safety in organisations, but this needs to be based on a careful analysis of the organisation at hand, leading to tailored solutions. The SMS maturity assessment approach in combination with the overview of methods for improving human-related, soft aspects in safety management were developed to support such analysis and tailoring.

The SMS Maturity Assessment Research Tool (SMART) provides 34 questions with five maturity levels each along the four main parts of an SMS. Although stemming from the aviation domain, the questions are formulated in a general way, thus supporting potential application in various domains. SMART can be used in various ways by single or multiple users and intra- or inter-organisations. Its main objective is to trigger thinking and discussion to support improvement rather than achieving maximum scores per se.

Improvement of safety-related, soft aspects in organisations is supported by a number of discussed research streams that support top management in safety leadership, harness middle managers in their role in organisational safety, allow fine-tuning of safety dashboards, support safety mindfulness in organisations, broaden safety culture assessment, and support improvement of an organisation’s agile response capability.