A Safety Culture Maturity Matrix for Nuclear Regulatory Bodies

Abstract

The concept of safety culture has attracted a great deal of attention. Since the rise of the concept, progress has been made regarding the definition of safety culture and the development of tools dedicated to safety culture oversight and self-assessment. In addition, these recent advances have been made across different high-hazard industries, and obviously in the nuclear world. Nevertheless, little attention has been paid to tailored methods allowing a regulatory body to assess its own safety culture. The aim of this paper is to present a framework adapted to nuclear regulatory bodies or TSOs (Technical Safety Organisations) in order to guide them in understanding their own safety culture through a “Safety Culture Maturity Matrix”.

1. Introduction

Regulatory bodies have a pivotal role in promoting and overseeing safety culture within nuclear installations. Beneath this commonplace, there is a need for a regulatory body to have a deeper look into its own safety culture and into the way culture shapes the regulatory response to safety issues [1,2,3,4,5,6,7]. In a previous paper we posited that between the compliance-based and the goal-oriented regulatory models, safety culture oversight represents an opportunity to develop a more responsive approach [8]. Here we highlight that developing this kind of oversight strategy implies the regulatory body having a deep understanding of its own safety culture. In this context, a regulatory body safety culture self-assessment model applied within the Belgian Technical Safety Organisation (TSO) is proposed. (As a subsidiary of the FANC (Federal Agency for Nuclear Control), Bel V is aiming to carry out surveillance of Belgian nuclear installations within the frame of the Belgian laws and regulations. Bel V provides technical support concerning nuclear safety assessments (Safety Evaluation Reports), and performs conformity checks of new plants or modifications and inspections in existing installations. The Belgian Regulatory Body is composed of FANC and Bel V.)

At the heart of safety culture is the way in which beliefs and representations related to safety drive licensees’ practices [9,10,11]. Numerous studies have pointed out the link between safety culture and safety operations [12]. From a regulatory body perspective, we stress that a clear view on its own safety culture enables a regulatory body to properly address safety issues and to effectively deal with organisational matters. As an inspector or a safety analyst, are we fully conscious of our concerns or behaviours that could shape the response of a licensee? As an organisation, are we able to consider managerial issues according to safety principles? As a regulatory body, are we aware of the safety model we strive for?

In other words, considering the regulatory body safety culture implies looking at ourselves in a mirror; in addition, we can also notice that implementing a safety culture self-assessment process is considered a requirement within the IAEA GSR part 2 [13]. A recent document from the OECD-NEA [14] identified five principles—and a set of attributes—of the safety culture of an effective regulatory body:

• Leadership for safety is to be demonstrated at all levels in the regulatory body.
• All staff of the regulatory body have individual responsibility and accountability for exhibiting behaviours that set the standard for safety.
• The culture of the regulatory body promotes safety and facilitates co-operation and open communication.
• Implementing a holistic approach to safety is ensured by working in a systematic manner.
• Continuous improvement, learning, and self-assessment are encouraged at all levels in the organisation.

These dimensions and attributes give a valuable framework defining what a regulatory body safety culture is. Also to be mentioned is the IAEA’s [15] set of five methodologies aiming at assessing safety culture within nuclear organisations, i.e., interviews, focus groups, surveys, observations, and document review. Along the lines of these initiatives, the goal of the article is to present a safety culture self-assessment model adapted to regulatory bodies and based on a maturity matrix.

2. Assessing Safety Culture Maturity

The proposed model took benefit from existing typologies initially developed by Westrum [16], extended by Reason [17] and Hudson [18], and from other maturity assessment frameworks proposed by Fleming [19] or Foster and Hoult [20], among others.

In Westrum’s cultural framework [16,21], information is gathered, organised, and acted on differently according to three cultural levels, i.e., “Pathological, Bureaucratic and Generative”. These levels distinguished the organisation’s ability to handle safety-related information and to respond to failures.

At the “Pathological” level of maturity, there is a focus on the personal needs, power, and glory of those in charge. This implies that information is hidden (“messengers are shot”), responsibilities avoided, and failures covered up: the pathological culture is power oriented and related organisations see safety as a problem. Organisations with the “Bureaucratic” culture tend to be fixated on rules, positions, and departmental territories. As a result, information may be ignored, responsibilities are compartmentalised, and failures lead to local repairs: the bureaucratic culture is then rule oriented. At the highest level of safety culture, “Generative” refers to a culture where safety is embedded in every activity of the organisation. This implies that information is actively sought, responsibilities are shared, and failures are resolved with deep inquiry. The focus is set on the mission and novelties are implemented: the generative culture is therefore performance oriented.

Building on these three cultural patterns and two additional levels proposed by Reason [17], i.e., reactive and proactive, Hudson [18,22] extended Westrum’s original typology. These five maturity levels—with calculative instead of bureaucratic—are characterised in the following statements:

• Pathological: Who cares about safety as long as we are not caught;
• Reactive: Safety is important, we do a lot every time we have an accident;
• Calculative: We have systems in place to manage all hazards;
• Proactive: We try to anticipate safety problems before they arise;
• Generative: Safety is how we do business round here.

According to the Fleming’s “Safety Culture Maturity Matrix” [19,23], five similar maturity levels are identified: “Emerging” (Level 1), “Managing” (Level 2), “Involving” (Level 3), “Cooperating” (Level 4), and “Continually improving” (Level 5)”. These five levels are evaluated through 10 elements of safety culture (i.e., Management commitment and visibility; Communication; Productivity versus safety; Learning organisation; Safety resources; Participation; Shared perceptions about safety; Trust; Industrial relations and job satisfaction; Training). From a methodological perspective, an assessment group of approximatively 10 persons uses a 5-point scale in order to assess each of the 10 elements and, on that basis, proposes an overall level of maturity describing an organisation or a department [24].

Another example of a safety maturity model is given by the “DuPont Bradley curve” that places organisations in the following four sequential categories: “Reactive” (safety is focused on compliance and delegated to the safety manager), “Dependent” (emphasis is laid on fear and discipline, rules, and procedures), “Independent” (safety management is internalised and people believe that personal commitment makes the difference), and “Interdependent” (teams feel a sense of ownership for safety and coordination is valued).

As a point of commonality, these models are diagnosis tools that enable us to understand and describe current stages of safety culture development. Applied within the oil and gas industry [19,25], aviation [26], petro-chemical industry [27], healthcare [28], and mining [20], safety culture maturity models are useful as they enable us to identify the different degrees of safety culture that an organisation can move through, ranging from low levels to the highest levels. In that regard, high levels of maturity are close to High-Reliability Organisations (HRO). Actually, HROs’ characteristics can only be achieved through a mature safety culture such as the “Generative” level according to the Westrum typology.

Therefore, maturity levels enable organisations to identify more or less rapidly a current level of safety culture and to define required actions to move towards a more mature level. Another example could be found in the IAEA three-stages model [29]. At Stage I, the organisation sees safety as an external requirement and handles it mainly through a technical perspective. At Stage II, safety is considered as important even in the absence of regulatory pressure. However, behavioural issues are still missing. An organisation at Stage III has adopted the idea of continuous improvement and there is an awareness of the impact of behaviour on safety. However, prior models are not tailored to a regulatory body’s features. Conversely, using a matrix based on four maturity levels, the proposed model has been designed according to the needs of a regulatory body or a TSO.

As further explained below, the presented matrix places safety culture on a continuum from a “Bureaucratic” to a “Holistic” maturity level.

3. A Maturity Model Adapted to Nuclear Regulatory Bodies

As stated, the proposed model is grounded on four maturity levels: namely “Bureaucratic”, “Individual commitment”, “Cooperative”, and “Holistic” (see Figure 1). These levels are defined through a set of items and by a generic “motto”.

• Bureaucratic: “We are driven by rules: our organisation is like clockwork”
• Individual commitment: “We are individually involved”
• Cooperative: “We are interdependent: our job needs teamwork”
• Holistic: “As a regulatory body (RB), we share and promote a holistic view of safety”

At the “Bureaucratic” level, organisations typically focus on processes and procedures to reach their goals. Performance is therefore defined by efficient administrative workflows and procedures are the main coordination instruments. This also implies that people tend to work in isolation and that continuous improvement is mainly ensured through audit systems that are formally required by the management system. The major risk of these organisations is if they stop making sense of the rules and show a lack of accountability. Given that, the “Bureaucratic” maturity level could lead to safety oversight essentially focused on compliance issues.

In contrast, an “Individual commitment” level is defined by the high involvement of people. Procedures and processes are established but individuals are also driven by strong ownership and an attitude of striving for excellence. This implies that safety oversight is grounded on deep technical analysis of safety issues. A large part of the staff and leaders are then convinced of their responsibility for safety. Nevertheless, safety oversight also requires a multidisciplinary approach. Despite the willingness of individuals, the collective side of safety tends to be ignored; this could be particularly true for large organisations or those growing rapidly or with a high turnover.

Next, a “Cooperative” maturity level is defined as one in which teamwork between individuals, groups, and departments is something natural. The organisation pays attention to well-designed transversal processes and leaders discuss safety issues with groups. As a result, safety issues are addressed through a multidisciplinary approach, differing opinions are raised, and learning from each other is valued. Moreover, experiences—internal and from other regulators—are considered as opportunities to assess and improve work processes.

A “Holistic” maturity level is seen as the highest achieved level of safety culture. At this stage, an organisation is focused on processes, promotes individuals’ commitment, and develops strong interfaces. As a specific feature, the entire organisation is focused on safety and is able to continuously adapt tools and structures to potential new needs. In other words, the leadership shows capabilities in terms of proactivity and a long-term approach. Concerning safety oversight, the organisation systematically develops guidelines for conducting safety assessments and allowing a comprehensive view of safety.

4. Assessing Maturity Levels

The maturity level is assessed through safety culture dimensions largely inspired by a recent OECD-NEA document dedicated to the “The Safety Culture of an Effective Nuclear Regulatory Body” [14]. The particular dimensions within the proposed model are as follows:

• Leadership for safety;
• Individual responsibility;
• Safety oversight;
• Open Communication;
• Continuous improvement.

In turn, in order to refine the assessment, these five dimensions are broken down into subdimensions. For instance, regarding the dimension “Leadership for safety”, four subdimensions are considered for the self-assessment:

• Involvement in safety: How do leaders ensure that safety issues are properly considered?
• Responsibilities and work division: How do leaders structure work processes and define responsibilities?

Workload management and support: How do leaders ensure that resources are adequately allocated for performing RB activities?

Supervision and leadership style: How do leaders supervise and exchange with staff?

Taken together, the model contains 13 subdimensions to be assessed (see

Table 1. Dimensions and subdimensions to be assessed.

LEADERSHIP FOR SAFETY	INDIVIDUAL RESPONSIBILITY	SAFETY OVERSIGHT	OPEN COMMUNICATION	CONTINUOUS IMPROVEMENT
Involvement in safety Responsibilities and work division Workload management and support Supervision and leadership style	Raising differing professional opinions Questioning attitude	Problem identification and resolution Communication and relationship with licensees Decision-making	Internal communication Knowledge sharing	Development and training Performance assessment

) by the members of an assessment group through workshops aiming at capturing internal perceptions.

In order to facilitate the evaluation work and discussion within the assessment group, each of the 13 subdimensions of the model is described through four assertions (proposals) reflecting the four maturity levels. Then, for example, the subdimension “Supervision and leadership style” is assessed on the basis of the following statements (see

Table 2. Example of an assessment grid.

Proposals	Score
○ Leaders’ roles are mainly focused on checking work activities (individual performance, work hours, vacation, …)	1
○ Leaders regularly give feedback (positive and negative): there is an open-door policy and managers are in the field	2
○ There is a visible leadership in terms of coaching and mentoring: people are enabled to reach their potential	3
○ Everyone is valued at each level of the organisation: people are respected and esteemed for their contribution	4

). As showed, assessment grids used by the self-assessment group are graded on a 4-point scale corresponding to the four levels of the Safety Culture Maturity Matrix.

For each subdomain, group members select the more suitable assertion, provide facts (statements and arguments), and identify potential improvement actions.

5. A Pilot Implementation within the Belgian TSO

As a main feature, this self-assessment is conducted by means of workshops during which a group of seven staff members—belonging to different departments and with different functions or levels of experience—evaluates the maturity level of the safety culture dimensions and subdimensions. In terms of resources, four meetings of approximatively 3 h duration were needed for the assessment of the 13 subdimensions. A first application of the tool was conducted over a two-month period.

Operationally speaking, each workshop session was dedicated to the assessment of one or two key safety culture dimensions. As a first step, each staff member proposed his/her view regarding good practices, weaknesses, and opportunities for improvement, and selected a rating level (individual phase). Then, supported by a moderator, the group challenged the different positions through questions and entered into a debate mode (collective phase). As described, assertions are grounded on the four maturity levels, but they are also a means to address the cultural dimensions of safety and to open discussions within the group. In other words, these four levels and the corresponding assertions must not be considered as tight statements but as a means to enter into a debate mode and to share views.

As a result, the group found a consensus on the maturity level and proposed potential improvement actions for each subdimension. It should be noted that this consensus can be found by considering the diversity of a context (e.g., a Level 2 for a specific activity or department and a Level 3 for some others). Results for each subdimension can be reported on a radar graph (see Figure 2a).

In the same way, results concerning the five dimensions can be calculated using arithmetic means (see Figure 2b).

The model therefore provides an overall diagnosis of the current safety culture maturity state of a regulatory body. However, behind these quantitative results, a (qualitative) cultural picture regarding “the way we are thinking and doing” also emerged from the group discussions. The aim of this paper is not to discuss the assessment results in detail, but an important finding is related to the fact that the cultural features in place could lead to reaching a “plateau effect” in terms of safety oversight capabilities. For instance, one of the proposed drivers for change is related to the reinforcement of multidisciplinary work and cross-cutting functions.

The assessment results were discussed with the management board and communicated to the entire staff. More precisely, the results and drivers for changes were explained to the staff during a dedicated plenary session. Shortly after the communication of results, a polling system (based on a Likert-type scale) was organised in order to capture the staff’s reactions concerning the diagnosis (Do you agree with the diagnosis elements?), to evaluate the staff’s overall adherence level regarding how to reach the next maturity level (Do you agree with the proposed drivers for change?), and to assess how the staff members perceive their own role regarding the needed changes (How do you evaluate your personal level of involvement regarding these drivers for change?). The results showed that a majority of staff members agreed with the findings (80% of staff agreed or strongly agreed with the findings); 67% found important or very important the proposed drivers for changes; but only 43% of staff considered their involvement as high (against 57% for moderate or low).

These results showed, in our view, a sharp contrast between a common understanding of raised issues and the extent of staff involvement. However, the level of adherence of the staff is crucial information before implementing any changes: a high level of adherence could be considered as a sign of readiness for change; conversely, a low level of adherence requires more effort to be put into staff awareness.

6. Discussion: First Lessons Learned

According to the experience gained, some elements related to the Safety Culture Maturity Matrix could be identified. It should also be noted that after the workshop, an evaluation form related to the process and the model was individually filled out by the group members.

In terms of “usability”, i.e., the extent to which the model can be used to obtain results, it seems that the application of the model does not represent significant difficulties. The logic of the rating scale is quite rapidly understood by participants. Nevertheless, as a reminder, the main outcome of the assessment workshops is to share views and to jointly establish a safety culture maturity picture. This implies nominating a moderator to ensure the group dynamics but also to create synthetic reports and, generally speaking, support the process.

In terms of “relevancy”, i.e., the degree to which the proposed model is related to regulatory bodies’ safety culture, the model proposes locating a regulatory body on a four-level continuum through several subdimensions. As shown, these subdimensions try to cover a large set of regulatory bodies’ safety culture attributes. Nevertheless, as an important feature, the list of subdimensions could be fine-tuned to match specific regulatory issues. For instance, subdimensions related to regulatory capture or financial issues could be added. In addition, the trajectory of a regulatory body through the four levels of maturity could be deeper considered. As an improvement, some metrics could be added in order to determine if safety culture is progressing or regressing from one level to another. (We would like to thank our reviewers for their valuable comments on that issue.)

In terms of “diagnosticity”, i.e., the ability of the tool to capture cultural issues, the tool allows deep-seated cultural assumptions which impact on safety oversight practices to emerge. Cultural dimensions are identified, discussed, and refined within the group. In addition to the selected ratings and maturity levels, it is an overall picture of the cultural issues at play which is drawn out. However, the identified levels reflecting the safety culture maturity of a regulatory body must be considered as an opportunity for defining drivers for change. The assessment model entails a change-oriented process. Nevertheless, regarding the results of the polling, it seems of high importance to ensure a large adherence of staff and, therefore, to assess how ready an organisation is to move from one maturity level to the next.

In terms of “universality”—defined as how applicable a model is to other regulatory bodies or TSOs—the model has been applied within a quite small organisation (~70 engineers). Nevertheless, we posit that the model could be used within larger organisations. In that case, several assessment groups producing potentially different results would allow for comparison between teams or departments. In turn, applied within smaller groups, the assessment model could be useful for the identification of subcultures.

7. Conclusions

Assessment of an organisation’s safety culture can be performed through quantitative and/or qualitative methods. A common approach to assessing safety culture is to apply survey methods such as questionnaires [4,30,31]. Self-completion questionnaires are useful tools for capturing perceptions about safety and for exploring differences between groups or organisational levels [15]. These instruments are also suitable for providing a baseline for further comparison over time [32]. However, results obtained through quantitative methods could be limited to an organisation’s safety climate snapshot [33], i.e., to explicit measures influenced by a set of factors such as organisational circumstances or socially desirable response strategies [34].

Maturity models are also highly flexible and could be tailored to a specific regulatory body’s requirements [35,36]. In contrast with a quantitative approach, the applied tool is participative, solution-focused, and puts the emphasis on underlying causes: through assessment workshops, the process aims to identify implicit and shared assumptions and to explore their influence on safety-related issues.

As a main feature, the model used intends to assess the safety culture maturity level of a regulatory body through group discussions: the four safety culture maturity levels differentiate four types of safety culture. The safety culture could be distinguished along a line from a “Bureaucratic” to a “Holistic” maturity level and, therefore, not only in terms of a “good” or “bad” safety culture.

Nevertheless, it is important to note that the final objective is not to strictly define the most accurate maturity level but to use the model as a way to establish an overall picture of safety culture and to identify drivers for cultural changes. Interactive assessment workshops allow a group to uncover underlying cultural features that drive behaviours and, as well, what needs to happen to move towards a more mature level. Besides this, organising the assessment workshops and communicating the results organisation-wide contribute to enhancing staff safety culture awareness. The process is therefore as important as the results.