Next Article in Journal
Effectiveness Evaluation of Section Speed Control in Czech Motorway Work Zones
Next Article in Special Issue
Measuring Resilience Potentials: A Pilot Program Using the Resilience Assessment Grid
Previous Article in Journal
A Decision Tree Approach to the Risk Evaluation of Urban Water Distribution Network Pipes
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Measuring Resilience Engineering: An Integrative Review and Framework for Bench-Marking Organisational Safety

School of Health Sciences, the University of Newcastle, Callaghan 2308, Australia
Centre for Resources Health and Safety, the University of Newcastle, Callaghan 2308, Australia
SSI Faculty—Université Bretagne-Sud, LAB-STICC, Laboratoire Labsticc, F-56321 Lorient, France
Author to whom correspondence should be addressed.
Safety 2020, 6(3), 37;
Submission received: 25 May 2020 / Revised: 10 August 2020 / Accepted: 12 August 2020 / Published: 14 August 2020
(This article belongs to the Special Issue Methods and Instruments for Evaluating and Measuring Safety)


Interest in resilience engineering for improving organisational safety continues to grow among safety scholars and practitioners, but little attention has focused on a unifying definition, characteristics, and instruments for quantitative measurements. This is a significant gap which can impede efforts at benchmarking and evaluating resilience engineering for organisational safety. This integrative review was undertaken to address this research-practice gap in order to inform a theoretical framework. A five steep integrative literature review process was used to retrieve and critically evaluate peer-reviewed quantitative research articles published or in press from 2003 to November 2019. From the 3884 studies identified, screened, and selected, 17 met the final inclusion criteria. In total, 15 specific instruments were identified, but only four were grounded on a theoretical framework or model—the most common instrument used for included structured surveys. A minimum of three and a maximum of 13 characteristics were measured; however, it is not clear what type of variables they represented. The six most common characteristics included top management commitment, just culture, learning culture, awareness, preparedness, and flexibility. An integrative model of how these can inform a Resilience Climate Questionnaire (RCQ) survey is presented.

1. Introduction

The effective management of organisation safety continues to attract the attention of academics, managers, and policy makers throughout the world. Consistent with contemporary safety practices, most efforts in this regard have focused on unwanted outcomes, injuries and losses arising from adverse events. These are in tandem with the commonly understood view of safety as the absence of unwanted outcomes, freedom from unacceptable harm, or the property of a system that seeks to ensure that harmful events are as low as possible [1]. While there are several strategies for achieving these (such as legislation, behavioural measures, ergonomics, risk management and safety management systems), most of these are based on the assumption that safety can be achieved by people performing work through prescribed norms, following procedures and rules, and reducing human error. This represented a safety I philosophy [2,3]; with safety defined as the absence of negative outcomes and operations deemed to be safe when the number of events that could go wrong were maintained at acceptably low levels. Hence, they focused on identifying and managing deviations from prescribed work. They were based on the assumptions that organisational systems were well designed and correctly maintained; design engineers had the ability to identify and design out all hazards and risks; procedures were comprehensive, complete and correct and people would follow these as they were told and/or trained to do. However, it is impossible to eliminate all hazards and risks at the design stage [4]; so policies, procedures, regulations and automation were incomplete [5]. In such cases operations are successful not because people followed rules or procedures but because they adapted these to match the conditions of work. This represents the safety II philosophy [2,3]; with safety associated more with the presence of positive outcomes, and operations deemed safe when the number of events that could go right were maintained at levels as high as possible. This is consistent with Resilience engineering (RE) as an alternative for managing organisational safety, which is gaining attention in many high-risk industries.
RE rose to prominence following investigations into the Columbia space disaster [6]. However, the specific links between safety and resilience has a longer history dating to the 1980s, when Wildavsky [7] proposed that strategies for coping with uncertain, destructive and collective events could be managed through anticipation and resilience. Anticipation, according to the author, involved predicting and preventing potential dangers arising from threats before damage was done, while resilience assisted organisations to survive unexpected dangers once they manifested by bouncing back [7]. These ideas are reflected in the current notion of RE as a new perspective of organisational safety, although the original ideas of resilience have been extended following insights from research and failures in complex systems, including organizational contributors to risk, and the factors that affect human performance to manage risks proactively [6,8]. Some of the things that make RE different from other contemporary safety strategies involve the focus on successes (in addition to failures), and the recognition that variability in human performance drives improvements in organisational safety. As part of the Safety II philosophy, it focuses on understanding why things go right by understanding everyday activities and treating safety as an emergent property of an organisational system [3,8]. Since its conceptualisation research in RE has escalated, with recent reviews identifying between 128 and 472 studies published from domains such as aviation, building and construction fishing, healthcare, nuclear power plants, petrochemical plants, and railways [9,10,11,12], showing the increased interest among safety scholars and practitioners.
Despite these, there has been a slow adoption of RE across the general industry. Most of these studies are too abstract and general [13], and there is no uniformly accepted definition of RE [9,10,14,15,16], with recent reviews suggesting five to ten different but inter-related explanations [9,10]. Boring [15,16] argued this was because it existed more as a conceptual framework, while Sheridan [14] suggested that it reflected a family of ideas. The lack of common definition also means there are gaps in the understanding of which key factor organizations need to focus on for benchmarking, evaluating or measuring RE quantitatively [9,10]. Some authors have argued for the need to focus on learning, responding, monitoring and anticipating [17,18,19], while others have suggested behaviours [20], cognition [21,22], buffering, flexibility, margins and tolerance [23]; safety culture [24,25]; safety trade-offs alongside production, quality and efficiency targets [23,24,26,27]; the gap between work as prescribed and work as done [27,28]. Some authors have suggested how aspects of these could be measured [29,30,31]. While these provide a rich source of information about selected aspects of RE, most of these studies failed to build on each other’s work, so there is very little shared analytical framework [10,21]. This suggests that RE is ill-defined, and it is unclear which phenomena are to be operationalized for organisational safety [9,13]. These are significant gaps which can hinder efforts in benchmarking RE. There is a clear need to develop a coherent, integrative framework and inform the development of instruments for conducting large-scale comparative studies across multiple high risk settings and sectors [13]. A review of published quantitative studies on RE and safety is an important first step in this process.
The present integrative review was undertaken to address the above by investigating how RE has been conceptualized and measured in published studies of safety, and the instruments used in its measurement. The aim is to inform a theoretical framework for measuring and benchmarking RE for organisational safety. This is the first comprehensive review on the topic, and advances previous work on RE indicators for safety management [12]. In doing so it seeks to address the previously identified need for a coherent integrative framework for RE [13].
The article is organized as follows. Section 2 presents the research paradigm and theoretical framework, which informed the specific method used for this review. This is one area of departure that this review takes in comparison with most published reviews. The specific methods used are discussed in Section 3. Section 4 includes the results of this review, while Section 5 discusses these findings and proposes an integrated framework for informing an instrument for measuring and benchmarking RE for organisational safety. Section 6 presents the strengths, limitations and implications of this review, followed by conclusions in Section 7.

2. Research Paradigm and Theoretical Perspective

Prior to selecting a research methodology, it is important for researchers to elucidate the philosophical and theoretical positions upon which their contributions are based. This will assist in explaining why some methods are appropriate for conducting certain types of research [32], and also connect the basic assumptions inherent in the paradigms and theoretical positioning used [33]. In this regard, the research paradigm and theoretical perspectives play an important role in organisational safety research. Burrell and Morgan [34] proposed that research aimed at investigating organisations can be situated along four paradigms: interpretivism, functionalism, radical structuralism and radical humanism, although the latter two are yet to be fully embraced by safety researchers. Interpretivism is closely associated with qualitative research and most common in RE [10,35]. This paradigm suggests that RE does not have a concrete existence but is something that is constructed by human actors, so cannot be investigated using the methods commonly used in natural and biological sciences [32]. Functionalism, which is closely associated with positivism, is gaining attention in RE studies [10,35]. This paradigm suggests that social phenomena have an objective existence so can be scientifically investigated using methods of natural and biological sciences [32]. This research is aimed at identifying and developing an integrative theoretical framework and informing an objective instrument for investigating RE for organisational safety, so a positivist paradigm was deemed most appropriate.
Apart from research paradigms, researchers also need to embed their inquiries in an appropriate theoretical perspective, which guides data collection and analysis [32]. In this instance, pragmatism was applied, largely because of its recognition as a philosophy of common sense, greater flexibility in terms of methods, and a focus on practical outcomes [36].

3. Method

This research utilized an integrative review, a method commonly used for evaluating strengths of evidence, identifying gaps in research, connecting related areas of published research, generating research question(s), identifying theoretical or conceptual frameworks, and exploring research methods [37]. They also enable researchers to infer generalizations about substantive issues from a set of studies directly bearing on those issues [38], draw together research published from different methodologies [39], and allow for wider perspectives and depth of evidence; including non-experimental research and theoretical literature [40]. Key authorities, such as Whittemore [37] and Soares, Hoga, Peduzzi, Sangaleti, Yonekura and Silva [39] have suggested a five stage approach—comprised of problem identification, literature search, data evaluation, data analysis, and presentation—be utilised. The first is concerned with identifying the variables of interest and the sampling frame to provide a focus and set the boundaries; the second involves ensuring identifying the maximum number of eligible articles; the third involves extracting data and evaluating the quality of the articles; the fourth involves ordering, coding, categorizing, and summarizing the findings; the final stage involves reporting the findings. The specific methods used for this review, adapted from [37], comprised of five stages, which are discussed below.

3.1. Framing the Research Questions

A set of three interrelated research questions were formulated for this review, including:
How has RE been conceptualized and defined in quantitative studies?
Which RE characteristics have been measured in these studies?
What psychometric properties were measured in these studies?

3.2. Searching and Selecting the Relevant Literature

Six electronic databases (CINAHL, Google Scholar, PsycINFO, PubMed, Scopus, and Social Science Journals) were searched on 12 November 2019, using “resilience engineering” as the keyword—i.e., TITLE-ABS-KEY (“resilience engineering,”), in line with a recent systematic review [9] and meta-analysis [11], to identify articles published or in press from January 2003 to November 2019. The search was limited to full-text articles and conference proceedings published in English. “Grey literature” was also searched by reviewing reference lists to identify any articles that may have been missed. The titles and abstracts were screened by two independent research assistants (RA) and included if they: (i) used groups, teams or organisations as a unit of analysis; (ii) were journal articles or conference proceedings; (iii) focused on safety. In making this decision it was observed that previous authors, such as Furniss, Back, Blandford, Hildebrandt and Broberg [21], have suggested that resilience is related to individuals, while others, including Woods [23] and Pillay [10], posited that the collective roles of groups and teams were important. Articles which did not meet the above criteria were excluded. Full-text copies of the remaining articles were retrieved and closely read by two RAs to identify studies eligible for review, including those that (i) described quantitative methods, factors and/or measures, and (ii) referred to and/or discussed instruments used. Disagreements were resolved through discussion with the two authors until consensus was achieved.

3.3. Data Extraction

The key information required for analysis and synthesis was exported into a data extraction sheet. Information regarding research aims, theory/model, research design (methods, participants), factors and variables measured, instruments used and approaches to statistical analysis were extracted. There is no specific published criteria for evaluating psychometric properties for RE studies; however, reliability and validity are two of most common used for evaluating the rigour of in quantitative studies [41].
Reliability is most assessed through Cronbach’s alpha coefficient, with a minimum value of 0.70 suggested to be generally acceptable for new measures, constructs and/or scales [42]. Validity is the degree to which the research phenomenon being investigated is accurately measured, and is generally assessed for content, construct and criterion [42]. According to the author, content validity is generally determined by the subjective opinions of experts, construct validity through exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), and criterion validity by considering convergence, divergence, and predictions.

3.4. Critical Appraisal

The selected articles were critically appraised using an eight-item questionnaire adapted from the Critical Appraisal Skills Programme (CASP) [43] and the quality assessment tool of diverse study designs [44]. The specific questions focused on:
Aim(s): Is the aim(s)/purpose of research clearly stated?
Theory/model: Is an explicit theoretical framework/model used or discussed?
Research design: Is an overall research design mentioned or discussed?
Data collection: Does the data collection include procedures, settings and/or sampling?
Data analysis: Is the data analysis sufficiently rigorous and includes quality aspects?
Bias: Were any biases or limitations of study considered and/or reported?
Results: Are the results clearly stated and discussed?
Value: Can this research be used to advance knowledge and/or practice?
Each question was assessed using a YES, NO, LIMITED response by the first researcher and cross-checked by the second. While this process was used for appraising the quality of published works selected for review, no articles were filtered out at this stage.

3.5. Data Analysis and Synthesis

The key information and findings for each study were gathered and summarized using a narrative approach.

4. Results

Figure 1 presents the findings of our search and selection strategy. The search of six databases and grey literature generated 3884 articles, from which 3731 duplicates were removed, leaving 153 articles for title and abstract screening. A further 78 were screened out at this stage, resulting in 75 articles for full-text review. An additional 58 were deemed not eligible at this stage, resulting in 17 studies for the final review and synthesis, these are summarized in Table 1.

4.1. General Characteristics of Reviewed Studies

The results suggested that propositions for studies in RE did not receive much attention until 2010, two years after the need for the quantification of RE was first pointed out, with the first two studies published in 2013. Since then, there has been at least one study every year, except for in 2015. Eleven (64.7%) articles were published from Iran, while others were from Australia (5.9%), Austria (5.9%), Canada (5.9%), Italy (5.9%), Kuwait (5.9%) and Poland (5.9%). All studies reported the development of the instruments for assessing RE. Organisations in which these were proposed, developed and/or investigated mostly included petrochemical plants (N = 5, 29.4%), process industries (N = 4, 23.5%) and public hospitals (N = 2, 11.8%), with those remaining from aviation (N = 1, 5.9%), aluminium manufacturing (N = 1, 5.9%), Construction (N = 1, 5.9%), gold mining (N = 1, 5.9%), solid waste management (N = 1, 5.9%) and steel manufacturing (N = 1, 5.9%).
The results of the critical appraisal of each article is presented in Table 2. All seventeen articles included details of data collection methods, while sixteen (94.1%) covered in adequate details methods for data analysis and results. Ten (58.8%) included an explanation or mentioned an overall research design, while nine (52.9%) included an aim or purpose of the study and/or article. Only four (23.5%) used or suggested a theoretical framework or model informing the research design or instrument, while only one (5.9%) discussed any biases and/or limitations.

4.2. Conceptualisation and Measurement of RE

The results also suggest that there continues to be a wide diversity in the way RE is conceptualized. For example, it has been suggested to be multidimensional [30,47,50], multi-level [29,30], multi-factorial [29,30,53,57]; associated with both human and organizational performance [30,47,54] and culture [30,53,57]. In addition, it is also concerned with boundaries of operations [30,46], adaptation [30,46], continuity of operations [29,51] and preparedness [48,52,55].
The diversity in the conceptualization of RE also means that there continues to be no unified definition of RE, consistent with previous conclusions [10,14]. Ten inter-related definitions identified in this review are summarized in Table 3. The most common of these suggests it is a paradigm for safety management, consistent with the seminal definition. Seven of the articles suggested it is an ability and/or capability, while one suggested it is a process. Four suggested it was a reactive ability, while four saw it as being proactive, reflecting the two faces of safety advocated by seminal authors of RE, such as Hollnagel, Wears and Braithwaite [2] and Hollnagel [18]. Authors, such as Boring [15,16], have previously argued that the loose definition of RE is intentional to avoid constraining its emergence and adoption. However, the absence of a unified definition around this emerging notion also means it can continue to be a source of confusion, especially outside the circle of researchers and practitioners who promote it, as it is not easy to have a clear sense of what it really denotes [26]. A unified meaning is necessary for setting the boundary and providing a focus RE [10]. In Section 5.4, one is provided, which the authors find useful in both regards, without making it superior to any of the others that have been previously proposed.
The measurement of RE (which includes all the attributes in a set of objective measures, or an aggregated index) generally largely followed a dimensional approach through several variables and/or factors, although the nomenclature used was inconsistently applied. For this review these have been simplified into RE characteristics, which include the main attributes that were measured. Table 4 summarizes the key RE characteristics measured, the instruments used, the theoretical frameworks or models used to develop these, and the statistical measures and results of these. According to Table 4, a minimum of three and a maximum of thirteen characteristics were used.
The instruments varied in scope, with one measuring as few as three characteristics, while others covered up to thirteen. The 11 studies published from Iran, the country with the largest number of articles, did not utilize the same instruments. Three studies from this country [47,49,53] used ten characteristics to measure thirty questions, while two [56,58] measured six characteristics through sixty-one questions. Four instruments [30,47,54,57] were associated with a theoretical framework or model, but one of these [30] has not been empirically tested or validated. One instrument [29] was limited to measuring at the behavioural level, while instruments 4 [46], 5 [47], 8 [50], 12 [54] and 14 [56] were based on the seven key themes identified by Wreathall [27], but also included an additional four characteristics of Fault-Tolerance, Self-organization, Teamwork, and Redundancy.
Instruments 2 [29] and 15 [57] measured characteristics that were different to those suggested by Wreathall in [27] or [19], while instrument 9 [51] was the only one to explicitly measure the four capabilities of a resilient organisation [19]. The instruments developed by teams led by Shirali and Azadeh appear to be central, illustrated in studies 6, 7, 10, 11, 14 and 16 [48,49,52,53,56,59]. Two instruments measured Resilience Safety Culture (RSC) [56,57], while one measured Resilience Safety Climate [55]. This is consistent with the suggestion made in a previous article [30], which suggested that both safety culture and RE could be examined through safety climate surveys. The sampling sizes informing the surveys varied widely, with a minimum of 30 and maximum of 564, and targeted either single levels, such as workers and/or operators, or utilized a cross section across all levels.
The most common characteristics measured included six of seven themes suggested by Wreathall [27]. Specifically:
Behaviours [20]—together with buffering, flexibility, margins and tolerance [23]—were included in two instruments, while cognition [21,22], or the gap between work as imagined and work as performed [27,28], were mentioned in one study but did not feature in any instruments or empirical studies.
From a quantitative measurement point of view, however, there is no clear agreement on which of these acts as independent variables, dependent variables (outcomes) or mediating variables. For example, Azadeh, Salehi, Ashjari and Saberi [47] suggested that all ten characteristics they investigated were outcomes. Pęciłło [51], on the other hand, used all the characteristics as independent variables. The model generated by Chen, McCabe and Hyatt [55] is complex but did include unsafe events as a dependent variable, and measured it through physical injuries and job stress; which represent one aspect of RE outcomes—i.e., failures. The characteristics measured by Shirali, Shekari and Angali [56] simply referred to these as variables. In this regard, the characters measured may best be summarized as key dimensions that represent the construct of RE.

4.3. Psychometric Properties

Fifteen (88.2%) studies reported on psychometric properties of the instruments specific methods for calculating these. Instrument No. 2 [29] had two of four characteristics with a Cronbach’s alpha (⍺) less than 0.70, similar to Instrument No. 3 [45]. However, the latter tool used by Achard, Agnello, Bragatto and Fabbricino [49] on a larger sample (564 vs. 88) showed ⍺ greater than 0.70. Instrument 9 had correlation co-efficient of 0.70 and above, for learning, monitoring, response and anticipation [51]. Instruments 10 and 13 had one of seven characteristics with ⍺ less than 0.70 [52,55], while instrument 12 [54] had four out of ten ⍺ characteristics less than 0.70. The whole ⍺ calculated for instruments 2, 5, 8, 12 and 13 were greater than 0.70 [29,47,50,54,55], similar to the Intraclass Correlation Coefficients (ICC) calculated for instruments No. 6 and 10 [48,52].
With respect to validity assessments, the Content Validity Index/Ratio (CVR and CVI) for instruments 6, 8 and 10 were greater than 0.70 [48,50,52]. Three instruments (4, 11 and 16) utilized a relative weighting of characteristics. The RSC assessed with ⍺ in instrument 14 [56] had one characteristics out of six with a ⍺ less than 0.70, similar to instrument 15 [57], which had one out of thirteen. For the latter instrument, the values obtained for the whole ⍺, the CVR and the CVI, were all greater than 0.70.

5. Discussion

This research was aimed at informing a theoretical framework for measuring and benchmarking RE for organisational safety by reviewing how has it been conceptualized and defined in quantitative studies, instruments used, and psychrometric properties of the studies. The development of a coherent framework for measuring RE is an important first step for conducting benchmarking and evaluation studies within and across industries.
The results suggest there is a wide diversity in the way RE has been conceptualized, so there is no unified understanding of what exactly it is or is not. This resonates with previous findings [9,10,11,12], so is likely to remain a contested area for research and practice. Only a limited number of studies proposed or used a theory or model, and only a few were developed from a collective body of knowledge. Moreover, while additional characteristics have been identified, these add to the ambiguity and confusion about what is or is not, so debates around the best ways of measuring RE, or more specifically, resilience potential [35], will continue. In this regard, quantitative research in RE remains theoretically fragmented. However, what is clear is that it is an organisational construct, which is multidimensional [30,47,50], multi-level [29,30] and multi-factorial [29,30,53,57], so it can be evaluated and investigated at any or all of these. Moreover, it is associated with both human and organizational performance [30,47,54] and culture [30,53,57], and is concerned with boundaries of operations [30,46], adaptation [30,46], continuity of operations [29,51] and preparedness [48,52,55]. Collectively, these make it a relatively complex construct because researchers need to focus not only on a wide range of dimensions, levels, and factors, but also on the interactions within and between these [8,61].
The results also showed that structured surveys were the most common instrument used. Methodologically, such surveys can capture responses from a much wider range of informants and across multiple levels [62,63]. The dimensional approach used in the surveys offer the advantage of focusing on those specific characteristics of interest. However, the survey instruments varied widely in terms of organisational settings and contexts, range of characteristics, and number of questions for measuring each characteristic. In this regard, the development of measurement instruments has been limited to adding new characteristics to a previously used instrument and offering new and presumably refined indicators and rankings. However, only four studies [30,47,54,57] were supported with a theory or model. In the absence of any associated theory, the inclusion of these additional characteristics can be a cause of confusion among those seeking to build on that work. Reliability and validity were the two most common statistical attributes measured.

5.1. A Predominant Dimensional Approach

This review highlights very clearly the predominance of characteristics of resilient organisations proposed by Wreathall and Merritt in [64] and [27]. Early studies suggested that these were akin to leading indicators of organisational health in High Reliability Organisations (HROs) [65]. While there are some common terms and ideas between these, there are also a few philosophical differences. For example, response, learning, flexibility, and culture are common to both, and resilience is one of the core principles associated with HROs [65]. However, the HRO view of resilience is one a system has, so is reactive, while in RE it is a proactive process that a system does [18]. In addition, HRO associates resilience with containment, not anticipation, while RE is associated with both anticipation and response as systems develop mechanisms to create foresight to recognize and defend against paths to failure [5]. In addition, HROs focus on the cognitive capabilities of collective mindfulness of organisational behaviour; learning from failures in order to anticipate, contain and recover from events and mishaps and manage risks through standard routines and protocols [65]. In contrast, RE focuses more on proactive processes and concentrates more on successful outcomes and safety managed through flexibility [5,23,26].
Paradoxically, the key attributes proposed by Hollnagel [17,19] were mobilized in only one instrument, despite the fact that this author is one of the key seminal authors and gurus of RE. This paradox, however, is not one of them. Indeed, the four capabilities described by Hollnagel [61] through its Resilience Analysis Grid: anticipating, monitoring, responding and learning, can be transposed with some of the key themes proposed by Wreathall [27] and widely represented within the instruments analysed. Thus, “flexibility” refers to the ability to “respond,” “Learning Culture” to the ability to “learn,” “situational awareness” to the ability to “perceive/monitor” and “preparedness” to the ability to “anticipate.” Finally, the coupling between the elements of the safety culture model proposed by Reason [66,67] and the four capabilities of Hollnagel [17,19] interact with six of the seven key themes proposed by Wreathall [27] and widely mobilized within the instruments. The psychometric results of these suggest these are robust and will allow for the quantitative assessment of resilience potential in organisations. The six most common characteristics can be used to inform an integrative theoretical framework and the main dimensions for a survey instrument that can be used across the broader industry.

5.2. Going Beyond the Predominant Dimensions

The instruments developed and used since 2013 clearly suggest that, while the key themes suggested by a Wreathall [27] are important, a comprehensive measurement of RE requires going beyond those characteristics. Based on the findings of this review, important characteristics that need to be considered in operationalizing RE instruments include:
  • The available margins of manoeuvre: Resilience, according to Hollnagel [19], assists organisations to adjust their functions before, during, or after any changes or disruptions in order to continue normal operations under both expected and unexpected conditions, so is very closely linked with adaptation. Indeed, resilience involves a return to its adaptive capacity in the face of new forms of variation and challenges [23]. For this reason, the available margins of manoeuvre are necessary for adaptation. In the current sector of French-language ergonomics [68,69,70,71], this notion of margins of manoeuvre was introduced in RE to in relation to resources set aside but which could be used to curtail unexpected demands and perturbations so that the system continued functioning instead of shutting down or reducing operations [72]. The lack or low rate of margins of manoeuvre reduces the adaptive capacity of organisations, making them vulnerable to variations induced by the occurrence of perturbations. The available margins of manoeuvre at the sharp-end level has been suggested to depend on two main factors within the organisation: (i) the degree of prescription/regulation; (ii) the degree of organisational control [73]. One of the instruments reviewed [53] investigated margins of manoeuvre by evaluating the closeness of its operating system in relation to its boundary of safe performance, and tolerance by gauging how well it worked at the borders when subjected to increased pressures and adaptive capacity. However, four instruments also measured tolerance [45,46,49,58]. Collectively, these capture the key facets that can be used to measure the available margins of manoeuvre.
  • Strengthen dimensions in relation to trade-offs between production and safety: The social creation of safety in RE involves the effective management of trade-offs between production and safety (P/S) [24,26,74,75], which involve sacrificial decisions being made at all levels of the hierarchy and will impact on the overall safety level of the organisation. The ability of an organisation to achieve under conditions of high pressures through effective management of production/safety trade-offs is a fundamental in RE. Regarding the instruments analysed in this review, this is taken into account but mainly at the level of the “management commitment” dimension—i.e., at the blunt-end level. However, the number of specific questions aimed at evaluating this remained modest. It would therefore be appropriate to add more questions for evaluating trade-offs between P/S, considering a breakdown of these issues on several dimensions, and questioning the sharp-end level. It might also be relevant to add a specific dimension.
  • Introduce a dimension dealing with managed safety vs. prescribed safety: At an operational level (i.e., sharp-end level), the degree of prescription within an organisation will more or less constrain front-line operators [76]. One of the indicators of RE (or lack of it) has been suggested to be the gap between work as prescribed and work as done [27,28], so monitoring and managing such gaps are important in driving safety achievements. According to Morel, Amalberti and Chauvin [26] and Dekker [28] the greater the degree of prescribing within an organisation, the less it will be possible for front-line operators to have sufficient autonomy and flexibility to adapt and deal with the occurrence of perturbations. On the other hand, the lower the level of prescribing, the greater the autonomy and margins of manoeuvre of front-line operators. From an RE perspective, this highlights two distinct forms of safety: one that is managed by the actors, on the one hand, and that prescribed by the organisation on the other [26,75,77,78]. Managed safety, which involves the adaptation of prescriptions to suit the context of local work situations, depends not only on the autonomy of the actors but also on their level of competence and expertise [26]. For this reason, any new instruments would need to incorporate questions that interrogate not only the level of prescription within organisations but also the autonomy of the actors, as well as their levels of competence and expertise. It should be noted that these three points are intrinsically linked to the issues raised above, which concern the trade-offs between performance and safety, on the one hand, and the margins of manoeuvre on the other.

5.3. Culture of Resilience vs. Climate of Resilience

Early conceptions of RE suggested it was linked to an organisation’s culture [24,25]. This has resulted in new instruments for measuring RE, mostly through safety culture [30,56,57], although one attempt has also been made with safety climate [55]. This is consistent with a recent integrative review, which identified that safety culture was more referenced in organizational system models of safety, such as RE in comparison to safety climate [79]. However, there have been some criticisms of associating RE with safety culture, with suggestions that many high risk and complex organisations were already well advanced in their safety culture, and specifically in the management of anticipated and unanticipated events, so there was no need for RE [80,81]. Similar assertions can also be associated with authors such as Reason [66,67], who argued that resilience, anticipation, senior management commitment, monitoring, feedback and flexibility, in dealing with ill-defined hazardous conditions, were all important in achieving safety culture.
Beyond this, however, the problem of choice in naming instruments resilience safety culture or resilience safety climate is reminiscent of the continued conceptual confusions surrounding these notions [82,83,84]. Safety culture has been suggested to be embedded in the deeper core layer of an organisation’s central assumptions about safety, but expressed in the middle layer (through beliefs, attitudes, motives, norms espoused values); was intangible, and had a distal influence on an organisation’s performance [79,83]. Safety climate, on the other hand, includes the perceptions of the middle and outer layers, was more dynamic, and could be more readily applied to system-based safety models [79,84,85]. In addition, safety climate was more proximal indicator of safety [79].
Choosing between safety culture and safety climate also guides the type of instruments as both do not necessarily use the same methods and therefore differ from this point of view [82]. Most evaluations of safety climate have involved quantitative methods, such as structured questionnaires. The assessment of safety culture, on the other hand, relies more on qualitative methods (e.g., observations and interviews). This is partly due to the fact that the safety climate is visible through attitudes and practices, while the safety culture is reflected in the values and beliefs that underlie those attitudes [83]. According to Flin, Mearns, O‘Connor and Bryden [84], safety climate, associated with the perceptions of safety for a specific location at a given time, was relatively unstable and subject to change, depending on the characteristics of the work environment.
Some authors have argued that, due to the elusive nature of safety culture, it was questionable whether it could be measured through scientifically sound methods [86], with some arguing that safety climate reflects safety culture [84,87]. What is actually being measured, according to Guldenmund [86], is safety climate, which represents the “measurable aspect of safety culture” (p. 29, [86]). This is also alluded to by Pillay, Borys, Else and Tuck [30], who suggested that RE could be measured through safety climate surveys. In addition, a comprehensive evaluation of safety climate research by Zohar [85] concluded that three key aspects which resonate with RE needed to be investigated. These included the relative priorities of competing demands (ways in which safety was prioritized in comparison to other goals, such as productivity or efficiency), gaps between words and deeds (management statements regarding the prioritization of safety, which were compromised under operational demands) and the local adaptation of policies and procedures. The first of these involves trade-offs between safety and efficiency, the second is consistent with the gap between work imagined as done, and the third with managed versus prescribed safety. These provide the basic foundations for measuring RE. In other words, the measurable facet of RE is resilience climate.
In terms of informing an instrument for measuring and benchmarking RE across the general industry, this review has identified that structured questionnaire surveys are the most common, which represents a way forward. However, given the discussions above, the terminology “Resilience Climate Questionnaire” better captures the key characteristics to be measured. Moreover, the choice not to add the word “Safety” to the title is justified because concepts of Resilience and Safety are not synonymous and may even be contradictory [75]. Avoiding any contradiction into the title of the instrument itself will make it clear the aim is not measure either safety climate or safety culture but to focus on resilience.

5.4. A Unified Definition and Integrative Framework

Based on this review, an integrated framework for benchmarking and measuring RE potential across the general industry can be proposed. However, as there is no clear definition of RE, it is important that this be clarified first, in order to set boundaries and provide a focus. The authors propose the following unified definition:
RE is a perspective for organisational safety management which enables organisational members to actively anticipate, respond, monitor and learn; by adapting to operate at the boundary of safe operations by narrowing the gap between work as imagined and work as performed; and manifested in an organisation’s culture, cognition and behaviours
Framing RE as above does highlight several key things. Firstly, RE is a perspective, so represents a philosophical shift. In effect, this shift is proactive and addresses the need for organisations to adapt to changes and threats prior to, during and after disruptions; or in the course of normal work operations, consistent with Safety II [3,18]. Secondly, although an individual can have all the attributes of resilience, it is only when they are inherent across the organisation that these play a role in RE [10]. Thirdly, the collective capabilities enable the organisation to anticipate, respond, monitor and learn, as suggested by Hollnagel [17,19]. Fourthly, it is about narrowing the gap between work as imagined and work as performed [27,28], which leads to adaptive performance at the boundary of safe operations. Finally, it is manifested in the organisation’s culture, cognition and behaviours.
RE can also be operationalized and measured quantitatively as resilience climate. In summarizing the key characteristics discussed previously, it would therefore be relevant to consider the development of a new survey instrument, in the form of a Resilient Climate Questionnaire. This should include, as a minimum, six of the seven most common themes identified by Wreathall [27]. These include top management commitment, just culture, learning culture, reporting culture, flexibility and awareness [30,45,46,47,48,49,50,52,54,55,58,59]. In addition, other characteristics can be incorporated; however, this needs to be supported with adequate theory. Figure 2 presents an integrative framework that captures the collective themes informing a comprehensive Resilience Climate Questionnaire, and act as a new model for advancing research and practice in RE for organisational safety.

6. Strengths, Limitations and Implications

There are several strengths with this review. It utilized a positivist paradigm to investigate objective instruments and measures [10,35] and applied a pragmatic theoretical perspective to focus on practical outcomes [36]. The researchers used a structured approach for searching and selecting articles. The title, abstract and full text article searches and selection were carried out by two independent Ras, and the final set of articles was also subjected to a process of critical appraisal. Collectively, these demonstrate a high degree of rigor in the review process, and confidence in the results and discussions.
There are also some limitations with this review. The exclusion of other bibliographic databases (such as Ei Compedex of Inspec), could potentially lead to overlooking articles that may have been published—for example, from a system engineering perspective. Only one keyword was used in the search and selection criteria, and this could have missed articles that did not use this term. The search was also limited to English language articles—non-English language papers were not searched and this may also have caused us to miss a few articles.
Despite these, this review is one of the first to focus specifically on measurements and instruments in RE, so the findings are significant. It advances previous work on the identification of RE indicators for safety management [12]. It is one of the first to focus specifically on the identification of measurements and tools, so addresses a significant research-practice gap in the field. As RE is multilevel, multi-dimensional and multi-factorial, the integrated model illustrated in Figure 2 can be used to inform a practical survey instrument for measuring, benchmarking, and improving RE as an organisational safety strategy. Findings from this review suggest that six key dimensions provide a good starting point for measuring RE through resilience climate, which is the measurable facet of RE. Consistent with safety climate studies, these six dimensions can be used as independent variables.
This review also identified that a few studies used additional characteristics to measure RE, which have been summarized in Figure 2. While acknowledging that this does capture the complexity of the RE construct, they can be a source of confusion. There is a need to support their inclusions with appropriate theory to identify the type of variables (independent, mediating or dependent) they are expected to represent.
This article also advocates the use of questionnaires as an instrument to assess resilience climate in organisations. Such an instrument is simply a device for collecting information that provides insights into the specific research questions being investigated. For this reason, it is important to understand the advantages and limitations of its usage.
Some of the advantages of questionnaires as used in general research include:
A well-designed one can be used to collect huge quantities of data;
Respondents can be sourced from a wide range of contexts and levels of the organisation;
They are relatively inexpensive to administer;
They can be administered in different ways;
Very little training is required to develop them;
Response rates for some types of instruments, such as group-administered questionnaires, can be higher;
They can be easily and quickly analysed [62,63,88].
The papers informing this review did not clearly discuss the advantages of using questionnaires. However, previous research on measurements of similar organisational constructs, such as safety culture, have suggested that the ease and speed of implementation, reproducibility, and being able to offer comparisons between organisations and groups made them an attractive alternative [89,90,91,92].
Questionnaires also have several limitations, most of which are the due to the poor design of instruments. For example:
It becomes difficult to avoid leading questions;
Questions can contain multiple sets of ideas so can become complicated;
Some questions, such as those on age, gender or the respondent’s specific role, can be irritating if no context is provided for their response;
They may be ambiguously worded [63,88];
The subjective use of scales can induce variations in perceptions among respondents [91];
Prior commitment from management and supervisors are required to enable operators complete these during their work hours [93];
The results generated from questionnaires also provide a superficial description of the organisation [92,94].
In order to address some of these limitations, more than one method can be used to provide greater insights into the key characteristics being. Examples include supplementing questionnaire surveys with qualitative methods, such as audits of workplace practice [30], experts views [46] or semi-structured interviews [53].

7. Conclusions

This review was aimed at reviewing the conceptualization, definition and measurement of RE—identifying any instruments used, and the psychometric measures tested, in order to inform a theoretical framework and measurement instrument that can be used to advance research and practice in the field. A positivist paradigm and the theoretical framework of pragmatism, together with a structured integrative review method, were used to enhance the rigour of the research process. The search across six comprehensive databased generated over 3900 articles, of which 17 were included in the final synthesis. In total, 15 survey instruments were identified, but only four of these were supported with theory. In total, 11 of these were from Iran. A minimum of three and a maximum of 13 dimensions were used in the surveys; however, there was a wide diversity in the number of questions asked for each dimension. There was wide diversity in the conceptualization and definition of RE, which suggests that it is a multidimensional, multifactorial, multi-level construct which exists across an organization, and manifests through culture, cognition and behaviours. A unified definition and an integrative model for informing a Resilience Climate Questionnaire, which is the measurable potential for RE, is proposed and currently being tested at the University of Newcastle, Australia. While questionnaires do offer several advantages, they also have their limitations, so supplementing these with other qualitative methods will help alleviate some of these limitations.

Author Contributions

Conceptualization, M.P. and G.M.; Methodology, M.P.; Results, Critical Analysis and Discussion, M.P. and G.M.; Writing—original draft, M.P.; Writing—reviewing and editing, G.M. and M.P.; Funding, M.P. All authors have read and agreed to the published version of the manuscript


This research was supported through an internal University of Newcastle School of Health Science Strategic Pilot Grant, awarded to the first author in 2018.


The authors wish to thank Emmanuel Boateng and Ishanka Rajapaksha Mudiyanselage, who assisted with the search and selection of articles. The authors are also grateful for the observations, comments, and feedback from the anonymous reviewers whose insights assisted in enhancing the quality of the final review.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Besnard, D.; Hollnagel, E. I want to believe: Some myths about the management of industrial safety. Cogn. Technol. Work 2014, 16, 13–23. [Google Scholar] [CrossRef] [Green Version]
  2. Hollnagel, E.; Wears, R.L.; Braithwaite, J. From Safety-i to Safety-ii: A White Paper; The Resilienet Health Care Net: Sydney, Australia, 2015. [Google Scholar]
  3. Hollnagel, E. Safety -i and Safety-ii: The Past and Future of Safety Management; Ashgate Publishing Ltd.: Farnham, UK, 2014. [Google Scholar]
  4. Hollnagel, E. Risk + barriers = safety? Saf. Sci. 2008, 46, 221–229. [Google Scholar] [CrossRef]
  5. Woods, D.D.; Hollnagel, E. Prologue: Resilience engineering concepts. In Resilience engineering: Concepts and Precepts; Hollnagel, E., Woods, D.D., Leveson, N., Eds.; Ashgate Publishing: Aldershot, UK, 2006; pp. 1–6. [Google Scholar]
  6. Woods, D.D. Creating foresight: Lessons for enhancing resilience from columbia. In Organization at the Limit: Nasa and the Columbia Disaster; Starbuck, W.H., Farjoun, M., Eds.; Blackwell: Malden, MA, USA, 2005; pp. 289–308. [Google Scholar]
  7. Wildavsky, A. Searching for Safety; Transaction Books: Piscataway Township, NJ, USA, 1988. [Google Scholar]
  8. Woods, D.D. Engineering organizational resilience to enhance safety: A progress report on the emerging field of resilience engineering. In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting, Los Angeles, CA, USA, 16–20 October 2006; pp. 2237–2241. [Google Scholar]
  9. Righi, A.W.; Saurin, T.A.; Wachs, P. A systematic literature review of resilience engineering: Research areas and a research agenda proposal. Reliab. Eng. Syst. Saf. 2015, 141, 142–152. [Google Scholar] [CrossRef]
  10. Pillay, M. Resilience engineering: An integrative review of fundamental concepts and directions for future research in safety management. Open J. Saf. Sci. Technol. 2017, 7, 129–160. [Google Scholar] [CrossRef] [Green Version]
  11. Patriarca, R.; Bergström, J.; Di Gravio, G.; Constantino, F. Resilience engineering: Current status of the research and future challenges. Saf. Sci. 2018, 102, 79–100. [Google Scholar] [CrossRef]
  12. Ranasinghe, U.; Jefferies, M.; Pillay, M.; Davis, P. Resilience engineering indicators and safety management: A systematic review. Saf. Health Work 2020, 11, 127–135. [Google Scholar] [CrossRef]
  13. Wiig, S.; Fahlbruch, B. Exploring resilience: An introduction. In Exploring Resilience: A Scientific Journey from Practice to Theory; Marsden, E., Kamate, C., Daniellou, F., Eds.; Springer: Cham, Swizterland, 2019; pp. 1–5. [Google Scholar]
  14. Sheridan, T.B. Risk, human error, and system resilience: Fundamental ideas. Hum. Factors J. Hum. Factors Ergon. Soc. 2008, 50, 418–426. [Google Scholar] [CrossRef]
  15. Boring, R.L. Reconciling resilience with reliability: The complementary nature of resilience engineering and human reliability analysis. In Proceedings of the 53rd Annual Meeting of the Human Factors and Ergonomics Society, San Antonio, TX, USA, 19–23 October 2009; pp. 1589–1593. [Google Scholar]
  16. Boring, R.L. Bridging Resilience Engineering and Human Reliability Analysis; Idaho National Laboratory: Idaho Falls, ID, USA, 2010.
  17. Hollnagel, E. The four cornerstones of resilience engineering. In Resilience Engineering Perspectives, Volume 2: Preparation and Restoration; Nemeth, C.P., Hollnagel, E., Dekker, S.W.A., Eds.; Ashgate: Surrey, UK, 2009; pp. 117–133. [Google Scholar]
  18. Hollnagel, E. Resilience engineering: A new understanding of safety. J. Ergon. Soc. Korea 2016, 35, 185–191. [Google Scholar] [CrossRef]
  19. Hollnagel, E. Prologue: The scope of resilience engineering. In Resilience Engineering in Practice: A Guidebook; Hollnagel, E., Paries, J., Woods, D.D., Wreathall, J., Eds.; Ashgate Publishing Company: Burlington, VT, USA, 2011; pp. xxix–xxxix. [Google Scholar]
  20. Mallak, L.A. Putting organizational resilience to work. Ind. Manag. 1998, 40, 8–13. [Google Scholar]
  21. Furniss, D.; Back, J.; Blandford, A.; Hildebrandt, M.; Broberg, H. A resilience markers framework for small teams. Reliab. Eng. Syst. Saf. 2011, 96, 2–10. [Google Scholar] [CrossRef]
  22. Back, J.; Furniss, D.; Blandford, A. Cognitive resilience: Reflection-in-action and on-action. In Proceedings of the Resilience engineering workshop, Vadstena, Sweden, 25–27 June 2007; Woltjer, R., Johansson, B., Lundberg, J., Eds.; MinesParis Tech: Vadstena, Sweden, 2007. [Google Scholar]
  23. Woods, D.D. Essential characteristics of resilience. In Resilience Enginering: Concepts and Precepts; Hollnagel, E., Woods, D.D., Leveson, N., Eds.; Ashgate Publishing Ltd.: Aldershot, UK, 2006. [Google Scholar]
  24. Flin, R. Erosion of managerial resilience: From vasa to nasa. In Resilience Engineering Concepts and Precepts; Hollnagel, E., Woods, D.D., Leveson, N., Eds.; Ashgate Publishing Ltd.: Aldershot, UK, 2006; pp. 223–233. [Google Scholar]
  25. Woods, D.D. Resilience engineering: Redefining the culture of safety and risk management. Hum. Factors Ergon. Soc. Bull. 2006, 49, 1–3. [Google Scholar]
  26. Morel, G.; Amalberti, R.; Chauvin, C. Articulating the differences between safety and resilience: The decision-making process of professional sea-fishing skippers. Hum. Factors Ergon. Manuf. Serv. Ind. 2008, 50, 1–16. [Google Scholar] [CrossRef] [PubMed]
  27. Wreathall, J. Properties of resilient organizations: An initial view. In Resilience Engineering Concepts and Precepts; Hollnagel, E., Woods, D.D., Leveson, N., Eds.; Ashgate Publishing Ltd.: Aldershot, UK, 2006; pp. 275–285. [Google Scholar]
  28. Dekker, S.W.A. Resilience engineering: Chronicling the emergence of confused consensus. In Resilience Engineering Concepts and Precepts; Hollnagel, E., Woods, D.D., Leveson, N., Eds.; Ashgate Publishing Ltd.: Aldershot, UK, 2006; pp. 77–92. [Google Scholar]
  29. Heese, M.; Kallus, W.; Kolodej, C. Assessing behaviour towards organizational resilience in aviation. In Proceedings of the Fifth Resilience Engineering Symposium Managing Trade-offs, Soesterberg, The Netherlands, 25–27 June 2013; Herrera, I., Schraagen, J.M., van der Vorm, J., Woods, D.D., Eds.; Resilience Engineering Association: Soesterberg, The Netherlands, 2013. [Google Scholar]
  30. Pillay, M.; Borys, D.; Else, D.; Tuck, M. Safety culture and resilience engineering: Theory and application in improving gold mining safety. In Gravity Gold 2010; Dominy, S., Ed.; AusIMM: Ballarat, Australia, 2010; pp. 129–140. [Google Scholar]
  31. Shirali, G.A.; Mohammadfam, I.; Motamedzade, M.; Ebrahimipour, V.; Moghimbeigi, A. Assessing resilience engineering based on safety culture and managerial factors. Process Saf. Prog. 2012, 31, 17–18. [Google Scholar] [CrossRef]
  32. Crotty, M. The Foundation of Social Research: Meaning and Perspective in Social Research; Allen and Unwin: Crows Nest, Australia, 1998. [Google Scholar]
  33. Klockner, K.; Pillay, M. Theorizing and theory building in the safety sciences: A reflective inquiry. Saf. Sci. 2019, 117, 250–256. [Google Scholar] [CrossRef]
  34. Burrell, G.; Morgan, G. Sociological Paradigms and Organisational Analysis: Elements of the Sociology of Corporate Life; Ashgate: Aldershot, UK, 1979. [Google Scholar]
  35. Mendonça, D. Measures of resilient performance. In Resilience Engineering Perspectives: Remaining Sensitive to the Possibility of Failure; Hollnagel, E., Nemeth, C.P., Dekker, S., Eds.; Ashgate Publishing Ltd.: Aldershot, UK, 2008; Volume 1, pp. 29–48. [Google Scholar]
  36. Shields, P.M. Pragmatism as a philosophy of science: A tool for public administration. Res. Public Adm. 1998, 4, 195–225. [Google Scholar]
  37. Whittemore, R. The integrative review: Updated methodology. J. Adv. Nurs. 2005, 52, 546–553. [Google Scholar] [CrossRef]
  38. Cooper, H.M. Scientific guidelines for conducting integrative reviews. Rev. Educ. Res. 1982, 52, 291–302. [Google Scholar] [CrossRef]
  39. Soares, C.B.; Hoga, L.A.K.; Peduzzi, M.; Sangaleti, C.; Yonekura, T.; Silva, D.R. Integrative review: Concepts and methods used in nursing. Rev. Esc. Enferm. USP 2014, 48, 329–339. [Google Scholar] [CrossRef]
  40. Fain, J.A. Systematic and integrative reviews of the literature: Core components of aade’s 2016–2018 strategic plan. Diabetes Educ. 2016, 42, 511–512. [Google Scholar] [CrossRef] [Green Version]
  41. Heale, R.; Twycross, A. Validity and reliability in quantitative studies. Evid. Based Nurs. 2015, 18, 66–67. [Google Scholar] [CrossRef] [Green Version]
  42. Aguinis, H.; Henle, C.A.; Ostroff, C. Measurement in work and organizational psychology. In Handbook of Industrial, Work and Organizational Psychology; Anderson, N.R., Ones, D.S., Sinangil, H.K., Viswesvaran, C., Eds.; Sage Publications: London, UK, 2005; Volume 1, pp. 27–50. [Google Scholar]
  43. Critical Appraisal Skills Programme (CASP) Critical Appraisal Skills Programme. Available online: (accessed on 22 February 2020).
  44. Sirriyeh, R.; Lawton, R.; Gardner, P.; Armitage, G. Reviewing studies with diverse designs: The development and evaluation of a new tool. J. Eval. Clin. Pract. 2011, 18, 746–752. [Google Scholar] [CrossRef]
  45. Shirali, G.A.; Mohammadfam, I.; Ebrahimipour, V. A new method for quantitative assessment of resilience engineering by pca and nt approach: A case study in a process industry. Reliab. Eng. Syst. Saf. 2013, 119, 88–94. [Google Scholar] [CrossRef]
  46. Azadeh, A.; Salehi, V.; Arvan, M.; Dolatkhah, M. Assessment of resilience engineering factors in high-risk environments by fuzzy cognitive maps: A petrochemical plant. Saf. Sci. 2014, 68, 99–107. [Google Scholar] [CrossRef]
  47. Azadeh, A.; Salehi, V.; Ashjari, B.; Saberi, M. Performance evaluation of integrated resilience engineering factors by data envelopment analysis: The case of a petrochemical plant. Process Saf. Environ. Prot. 2014, 92, 231–241. [Google Scholar] [CrossRef]
  48. Azadian, S.; Shirali, G.A.; Saki, A. Designing a questionnaire to assess crisis management based on resilience engineering approach. Jundishapur J. Health Sci. 2014, 6, 245–256. [Google Scholar]
  49. Achard, P.O.; Agnello, P.; Bragatto, P.A.; Fabbricino, M. Assessment of resilience in msw management companies. J. Solid Waste Technol. Manag. 2016, 42, 533–541. [Google Scholar]
  50. Azadeh, A.; Zarrin, M. An intelligent framework for productivity assessment and analysis of human resource from resilience engineering, motivational factors, hse and ergonomics perspectives. Saf. Sci. 2016, 89, 55–71. [Google Scholar] [CrossRef]
  51. Pęciłło, M. The resilience engineering concept in enterprises with and without occupational safety and health management systems. Saf. Sci. 2016, 82, 190–198. [Google Scholar] [CrossRef]
  52. Shirali, G.A.; Azadian, A.; Saki, A. A new framework for assessing hospital crisis management based on resilience engineering approach. Work 2016, 54, 435–444. [Google Scholar] [CrossRef]
  53. Shirali, G.A.; Motamedzade, M.; Mohammadfam, I.; Ebrahimipour, V.; Moghimbeigi, A. Assessment of resilience engineering factors based on system properties in a process industry. Cogn. Technol. Work 2016. [Google Scholar] [CrossRef]
  54. Azadeh, A.; Salmanzadeh-Meydani, N.; Motevali-Haghighi, S. Performance optimization of an aluminum factory in economic crisis by integrated resilience engineering and mathematical programming. Saf. Sci. 2017, 91, 335–350. [Google Scholar] [CrossRef]
  55. Chen, Y.; McCabe, B.; Hyatt, D. A resilience safety climate model predicting safety performance. Saf. Sci. 2018, 109, 434–445. [Google Scholar] [CrossRef]
  56. Shirali, G.A.; Shekari, M.; Angali, K.A. Assessing reliability and validity of an instrument for measuring resilience safety culture in sociotechnical systems. Saf. Health Work 2018, 9, 296–307. [Google Scholar] [CrossRef] [PubMed]
  57. Garg, A.; Alroomi, A.; Tonmoy, F.; Mohamed, S. Quantitative assessment of resilient safety culture model using relative importance index. In Proceedings of the 11th International Conference on Construction in the 21st Century (CITC-11), London, UK, 9–11 September 2019; Springer Nature: London, UK, 2019. [Google Scholar]
  58. Shirali, G.A.; Nematpour, L. Evaluation of resilience engineering using super decisions software. Health Promot. Perspect. 2019, 9, 191–197. [Google Scholar] [CrossRef]
  59. Zarrin, M.; Azadeh, A. Mapping the influences of resilience engineering on health, safety and environment and egronomics management system by using z-number cognitive map. Hum. Factors Ergon. Manuf. Serv. Ind. 2019, 29, 141–153. [Google Scholar] [CrossRef]
  60. Cook, R.; Rasmussen, J. “Going solid”: A model of system dynamics and consequences for patient safety. Qual. Saf. Health Care 2005, 14, 130–134. [Google Scholar] [CrossRef] [Green Version]
  61. Hollnagel, E. Epilogue: Rag -the resilience analysis grid. In Resilience Engineering in Practice: A Guidebook; Hollnagel, E., Paries, J., Woods, D.D., Wreathall, J., Eds.; Ashgate Publishing: Burlington, VT, USA, 2011; pp. 275–296. [Google Scholar]
  62. Creswell, J.W. Research Design: Qualitative, Quantitative and Mixed Methods Approaches, 3rd ed.; Sage Publications Inc.: Thousand Oaks, CA, USA, 2009; p. 260. [Google Scholar]
  63. Gray, D.E. Doing Research in the Real World; Sage Publications: London, UK; Thousand Oaks, CA, USA; New Delhi, India, 2004. [Google Scholar]
  64. Wreathall, J.; Merritt, A.C. Managing human performance in the modern world: Developments in the US nuclear industry. In Innovation and Consolidation in Aviation; Edkins, G., Pfister, P., Eds.; Ashgate: Burlington, VT, USA, 2003. [Google Scholar]
  65. Enya, A.; Pillay, M.; Dempsey, S. A systematic review on high reliability organisational theory as a safety management strategy in construction. Safety 2018, 4, 6. [Google Scholar] [CrossRef] [Green Version]
  66. Reason, J. Achieving a safe culture: Theory and practice. Work Stress 1998, 12, 293–306. [Google Scholar] [CrossRef] [Green Version]
  67. Reason, J. Organizational Accidents Revisited; CRC Press: Boca Raton, FL, USA, 2016. [Google Scholar]
  68. Coutarel, F.; Caroly, S.; Vézina, N.; Daniellou, F. Marge de manœuvre situationnelle et pouvoir d’agir: Des concepts à l’intervention ergonomique. Trav. Hum. 2015, 78, 9–29. [Google Scholar] [CrossRef]
  69. Daniellou, F.; Rabardel, P. Activity-oriented approaches to ergonomics: Some traditions and communities. Theor. Issues Ergon. Sci. 2005, 6, 353–357. [Google Scholar] [CrossRef]
  70. De Keyser, V. Work analysis in french language ergonomics: Origins and current research trends. Ergonomics 1991, 34, 653–669. [Google Scholar] [CrossRef] [PubMed]
  71. Morel, G.; Pillay, M. The occupational risk assessment method: A tool to improve organizational resilience in the context of occupational health and safety management. In Proceedings of the 3rd International Conference on Human Error, Reliability, Resilience, and Performance (Herrp), Washington, DC, USA, 24–28 July 2019; Arezes, P.M., Ed.; Springer: Cham, Swizterland, 2020; Volume 969, pp. 367–376. [Google Scholar]
  72. Woods, D.D.; Branlat, M. How human adaptive systems balance fundamental tradeoffs: Implications for polycentric governance architectures. In Proceedings of the Fourth Resilience Engineering Symposium, Sophia Antipolis, France, 8–10 June 2011; Mines Paris Tech: Sophia Antipolis, France, 2011. [Google Scholar]
  73. Paries, J.; Macchi, L.; Valot, C. Deharvengt, Comparing hros and re in the light of safety management systems. Saf. Sci. 2019, 117, 501–511. [Google Scholar] [CrossRef]
  74. Hollnagel, E. The Etto Principle: Efficiency-Thoroughness Trade-off-why Things that Go Right Sometimes Go Wrong; Ashgate Publishing Company: Burlington, VT, USA, 2009. [Google Scholar]
  75. Morel, G.; Amalberti, R.; Chauvin, C. How good micro/macro ergonomics may improve resilience, but not necessarily safety. Saf. Sci. 2009, 47, 285–294. [Google Scholar] [CrossRef]
  76. Nathanael, D.; Marmaras, N. Work practices and prescription: A key issue for organizational resilience. In Resilience Engineering Perspectives Volume 1: Remaining Sensitive to the Possibility of Failure; Hollnagel, E., Nemeth, C., Dekker, S., Eds.; Ashgate: Aldershot, UK, 2008; pp. 101–118. [Google Scholar]
  77. Cuvelier, L.; Falzon, P. Sécurité réglée et/ou sécurité gérée. Quelles Comb. Possibles 2012, 47, 1–16. [Google Scholar]
  78. Cuvelier, L.; Woods, D.D. Sécurité réglée et/ou sécurité gérée: Quand l’ingénierie de la résilience réinterroge l’ergonomie de l’activité. Trav. Hum. 2019, 82, 41–66. [Google Scholar] [CrossRef]
  79. Tristan, C.; Griffin, M.A.; Flautau, H.; Neal, A. Safety climate and safety culture: Integrating psychological and systems perspectives. J. Occup. Health Psychol. 2017, 22, 341–353. [Google Scholar]
  80. Chevreau, F.-R. Safety culture as a rational myth: Why developing safety culture implies resilience engineering? In Proceedings of the 2nd Symposium on Resilience Engineering, Sophia Antipolis, France, 28–30 October 2006; MinesParis Tech: Sophia-Antipolis, France, 2006; pp. 63–73. [Google Scholar]
  81. Oxstrand, J.; Sylvander, C. Resilience engineering: Fancy talk for safety culture: A nordic perspective on resilience engineering. In Proceedings of the 3rd International Symposium on Resilient Control Systems, Idaho Falls, ID, USA, 10–12 August 2010; IEEE: Idaho Falls, ID, USA, 2010; pp. 135–137. [Google Scholar]
  82. Cox, S.; Flin, R. Safety culture: Philosopher’s stone or man of straw? Work Stress 1998, 12, 189–201. [Google Scholar] [CrossRef]
  83. Guldenmund, F.W. (Mis)understanding about safety culture and its relationship to safety management. Risk Anal. 2010, 30, 1466–1480. [Google Scholar] [CrossRef]
  84. Flin, R.; Mearns, K.; O’Connor, P.; Bryden, R. Measuring safety climate: Identifying the common features. Saf. Sci. 2000, 34, 177–192. [Google Scholar] [CrossRef]
  85. Zohar, D. Thirty years of safety climate research: Reflections and future directions. Accid. Anal. Prev. 2010, 42, 1517–1522. [Google Scholar] [CrossRef]
  86. Guldenmund, F.W. Understanding safety culture through models and metaphors. In Safety Cultures, Safety Models: Taking Stock and Moving Forward; Gilbert, C., Journé, B., Laroche, H., Bieder, C., Eds.; Springer: Cham, Switzerland, 2018; pp. 21–34. [Google Scholar]
  87. Ciavarelli, A.P. Assessing safety climate and culture: From aviation to medicine. In Second Safety Across High-Consequence Industries Conference; Patankar, M.S., Ed.; Saint Louis University: St. Louis, MO, USA, 2005. [Google Scholar]
  88. Wilkinson, D.; Birmingham, P. Using Research Instruments: A Guide for Researchers; Taylor & Francis Group: New York, NY, USA, 2003. [Google Scholar]
  89. Guldenmund, F. Understanding and Exploring Safety Culture; Uitgeverij Box Press: Oisterwijk, The Netherlands, 2010. [Google Scholar]
  90. Weigmann, D.A.; Zhang, H.; von Thaden, T.L.; Sharma, G.; Gibbons, A.M. Safety culture: An integrative review. Int. J. Aviat. Psychol. 2004, 14, 117–134. [Google Scholar] [CrossRef]
  91. Guldenmund, F.W. The nature of safety culture: A review of theory and research. Saf. Sci. 2000, 34, 215–257. [Google Scholar] [CrossRef]
  92. Hopkins, A. Studying organisational cultures and their effects on safety. Saf. Sci. 2006, 44, 875–889. [Google Scholar] [CrossRef] [Green Version]
  93. Allen, S.; Chiarella, M.; Homer, C.S.E. Lessons learnt from measuring safety culture: An australian case study. Midwifery 2010, 26, 497–503. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  94. Antonsen, S. Key issues in understanding and improving safety culture. In Safety Culture, Safety Models: Taking Stock and Moving Forward; Gilbert, C., Journé, B., Laroche, H., Bieder, C., Eds.; Springer: Cham, Switzerland, 2018. [Google Scholar]
Figure 1. Flow diagram for search and selection of relevant articles.
Figure 1. Flow diagram for search and selection of relevant articles.
Safety 06 00037 g001
Figure 2. Integrative Model for Measuring RE through a Resilience Climate Questionnaire.
Figure 2. Integrative Model for Measuring RE through a Resilience Climate Questionnaire.
Safety 06 00037 g002
Table 1. List of included studies.
Table 1. List of included studies.
1Pillay, Borys, Else and Tuck [30]Introduced a theoretical framework for RE, proposed a toolkit for investigating RE through safety cultureGold Mining,
Structured safety climate survey
2Heese, Kallus and Kolodej [29]Developed and validated an Inventory for assessing behaviour and organisation resilience in Aviation (I-BORA)Aviation,
Structured questionnaire surveys
3Shirali, et al. [45]Examined the utility of using surveys for measuring RE11 Units of a large process industry, IranStructured questionnaire survey
4Azadeh, et al. [46]Assessed factors affecting the resilient levels using Fuzzy cognitive maps (FCM)Petrochemical plants, IranExpert Views and Structured questionnaire survey
5Azadeh, et al. [47]Evaluated performance of Integrated RE through questionnaires and data envelopment analysisPetrochemical departments, IranStructured questionnaire survey
6Azadian, et al. [48]Assessed crisis management using a questionnaire design which incorporated RE principlesPublic hospitals, IranStructured questionnaire survey
7Achard, et al. [49]Assessed the potential for Resilience in Municipal Solid Waste Management CompaniesMSWM Companies,
Structured questionnaire survey
8Azadeh and Zarrin [50]Measured human resources productivity considering RE, motivational factors of work, environmental health and safety and ergonomics principlesPetrochemical plant,
Structured questionnaire survey
9Pęciłło [51]Examined the utility of RE concepts in organisations with and without OSH across different sized enterprises.Process industries,
Structured questionnaire survey
10Shirali, et al. [52]Developed a new framework for evaluating crisis management through REHospitals,
Structured questionnaire survey
11Shirali, et al. [53]Assessed RE factors based on system properties Process industry,
Semi-structured interviews with MCQs, analysis of documents
12Azadeh, et al. [54]Used mathematical programming to develop and evaluate Integrated Resilience EngineeringAluminium factory,
Structured questionnaire survey
13Chen, et al. [55]Developed and validated a Safety Climate Resilience Model to predict construction safety performance.Construction industry,
Structured, self-administered questionnaire
14Shirali, et al. [56]Designed a validated instrument to Measure Resilience Safety Culture in Sociotechnical Systems.Petrochemical Plants,
Structured questionnaire survey
15Garg, et al. [57]Used a resilient safety culture (RSC) model to measure the impact of remoteness and mental health.Oil and gas industry, KuwaitStructured questionnaire survey
16Shirali and Nematpour [58]Quantified and ranked RE dimensions through analysis network processSteel industry,
Structured questionnaire survey
17Zarrin and Azadeh [59]Evaluated and analysed impacts of RE on integrated OHS management systemPetrochemical Plants,
Structured questionnaire survey
Table 2. Critical appraisal of selected studies.
Table 2. Critical appraisal of selected studies.
Study No.I.
II. Theory/ModelIII. Research DesignIV. Data CollectionV. Data AnalysisVI. Bias/LimitationsVII. ResultsVIII. Value
Table 3. Conceptualization of RE in quantitative studies.
Table 3. Conceptualization of RE in quantitative studies.
[30]“developing an organisation’s behavioural and cognitive capability such that it can effectively adjust and continue performing optimally near an its safe operating envelop in the presence of everyday threats and environmental stressors at all levels of the organisation” (p. 134)
[29]“ability of a system or an organisation to react and recover from disturbances at an early stage with minimal effect on dynamic stability” (p. 2)
[45]“ability of a system to adapt its functioning before and during disturbances, so that it can continue operations after a major mishap or in the presence of continuous stresses” (p. 89)
[46]“a paradigm for safety management that concentrates on how to help people create foresight, anticipate the different forms of risk in order to cope with complexities under pressure and move towards success” (p. 100)
[47]“a paradigm for safety management that stresses how people, systems and organizations learn, adapt, and create safety in an environment with hazards, tradeoffs, and multiple goals” (pp. 231–232)
[48]“intrinsic ability of a system to adjust its operation before or following changes and disturbances, so it can maintain operations after an accident” (p. 247)
[50]“inherent ability of a system to adapt its function before and during the situations where normal functioning is disrupted, so that it can continue operations after a major disaster or in the presence of constant stresses” (p. 56)
[53]“intrinsic ability of a system to adapt its function before, during, or after major mishaps or changes, so that it can continue the operations required under both expected and unexpected conditions” (p. 20)
[54]“a paradigm for safety management that concentrates on how to help people deal with complexity under stress to achieve success” (p. 336)
[56]“an organizational culture that fosters safe practices for improved safety in an ultra-safe organization striving for cost-effective safety management” (p. 297)
[58]“capability to sustain or rapidly return to a steady state that allows the organization to continue operation during or after a major event or in the presence of continuous stresses” (p. 191)
[59]“ability of an organization to regulate its function before, during, and after perturbations and fluctuations, so that it can continue the operations required under both predicted and unpredicted situations” (p. 142)
Table 4. Summary of instruments, theoretical frameworks, RE characteristics and statistical measures.
Table 4. Summary of instruments, theoretical frameworks, RE characteristics and statistical measures.
Author(s)Sample and SizeInstrumentTheoretical Framework/ModelRE Characteristics/No. of Questions
Psychrometric Properties Results
Pillay, Borys, Else and Tuck [30]Not indicatedToolkit for examining organisational safety culture and REAdapted four-boundary drift to failure model [60](27)
  • Top Management Commitment (TMC)/n = 4
  • Just Culture (JC)/n = 6
  • Learning Culture (LC)/n = 6
  • Awareness (AW)/n = 6
  • Preparedness (PR)/n = 3
  • Flexibility (FL)/n = 2
Heese, Kallus and Kolodej [29]282IBORANot indicated(12)
  • Goal-directed/proactive solutions (GD-PS)/n = 6
  • Flexibility (FL)/n = 2
  • Improvisations (IM)/n = 2
  • Availability of resources (RE)/n = 2
PCA – Kaiser’s and Varimax rotation
Reliability-Cronbach’s alpha (⍺)

GD-PS = 0.787
F = 0.633
I = 0.671
AR =0.708
Shirali, Mohammadfam and Ebrahimipour [45]88
Managers (3%)
Supervisors (20%)
Operators (77%)
Resilience engineering survey questionnaireNot indicated(57)
  • Top Management Commitment (TMC)/n = 9
  • Just Culture (JC)/n = 11
  • Learning Culture (LC)/n = 15
  • Awareness and Opacity (A-O)/n = 11
  • Preparedness (P)/n = 9
  • Flexibility (F)/n = 6
Reliability-Cronbach’s alpha (⍺)

TMC= 0.806
JC = 0.806
LC = 0.777
A-O = 0.660
P = 0.700
F= 0.608
Azadeh, Salehi, Arvan and Dolatkhah [46]60
Experts (n = 30)
Managers, engineers and experienced workers (n = 30)
Questionnaire developed for studyNot indicated(Not disclosed)
  • Top Management Commitment (TMC)
  • Just Culture (JC)
  • Learning Culture (LC)
  • Awareness (A)
  • Preparedness (P)
  • Flexibility (F)
  • Teamwork (T)
  • Redundancy (R)
  • Fault-Tolerant (FT)
Final weights of resilience factors obtained by combination of Fuzzy Cognitive Maps (FCMs) results and data from questionnaireFinal weights
TMC: 10.76
JC: 11.33
LC: 11.83
A: 12.28
P: 12.30
F: 12.20
T: 9.51
R: 7.83
FT: 11.95
Azadeh, Salehi, Ashjari and Saberi [47]115
Managers and supervisors (n = 37)
Engineers, technicians and operators (n = 78)
Questionnaire developed for studyIntegrated RE(30)
  • Top Level Commitment (TLC)/n = 3
  • Just Culture (JC)/n = 3
  • Learning Culture (LC)/n = 3
  • Awareness and Opacity (AO)/n = 3
  • Preparedness (PR)/n = 3
  • Flexibility (FL)/n = 3
  • Self-organisation (SO)/n = 3
  • Teamwork (TW)/n = 3
  • Redundancy (RD)/n = 3
  • Fault-Tolerant (FT)/n = 3
Data Envelopment Analysis (DEA) approach
Reliability of collected data: Cronbach’s alpha (⍺)
To verify the results of DEA approach: Spearman test of relationship (rs)
⍺ whole tool: 0.9
rs = 0.904
Azadian, Shirali and Saki [48]113
Nurses from seven public hospitals
Questionnaire developed for studyNot indicated(26)
  • Top Management Commitment (TMC)/n = 14
  • Learning (LR)/n = 3
  • Awareness (AW)/n = 6
  • Flexibility (FL)/n = 3
Reliability-Cronbach’s alpha, Interclass correlation Coefficient (ICC).
Validity–Content Validity Index/ratio: CVI and CVR

TMC= 0.865/0.865
LR = 0.871/0.843
AW = 0.769/0.769
FL = 0.845/0.845
⍺ whole tool: 0.951
ICC whole tool: 0.95
CVI whole tool: 0.85
CVR whole tool: 0.75
Achard, Agnello, Bragatto and Fabbricino [49]564
Employees of several operative units of the company
Resilience engineering survey questionnaireNot indicated(61)
  • Top Management Commitment (TMC)/n = 9
  • Just Culture (JC)/n = 11
  • Learning Culture (LC)/n = 15
  • Awareness and Opacity (A-O)/n = 11
  • Preparedness (P)/n = 9
  • Flexibility (F)/n = 6
Reliability-Cronbach’s alpha (⍺)

TMC= 0.8
JC = 0.7
LC = 0.7
A-O = 0.7
P = 0.7
F= 0.8
Azadeh and Zarrin [50]165
Operators (n = 90)
Supervisors (n = 53)
Managers (n = 22)
Questionnaire developed for study, including RE (10), WMFs (4) and HSEE (4) factorsNot indicated(30)
  • Top Level Commitment (TLC)/n = 3
  • Learning (LR)/n = 3
  • Flexibility (FL)/n = 3
  • Reporting Culture (RC)/n = 3
  • Awareness (AW)/n = 3
  • Preparedness (PR)/n = 3
  • Self-Organisation (SO)/n = 3
  • Teamwork (TW)/n = 3
  • Redundancy (RD)/n = 3
  • Fault-Tolerant (FT)/n = 3
Reliability-Cronbach’s alpha (⍺)
Validity–Content Validity Ratio (CVR)

TLC= 0.919/0.921
LR = 0.868/0.813
FL = 0.812/0.898
RC = 0.853/0.980
AW = 0.884/0.929
PR= 0.750/0.960
SO = 0.756/0.866
TW = 0.924/0.959
RD = 0.722/0.813
FT = 0.853/0.839
Pęciłło [51]Not indicated
100 Enterprises.
Questionnaire developed for study (Two parts; one deals with resilience)Not indicated(45)
  • Learning (L)/n = 11
  • Monitoring (M)/n = 11
  • Responding (R)/n = 12
  • Anticipation (A)/n = 11
Correlation coefficientsCorrelation coefficients
L = 0.81
M = 0.79
R = 0.80
A = 0.82
Resilience Total = 0.84
Shirali, Azadian and Saki [52]310
Staff from eight hospitals
Managers: 21.94%
Nurses: 78.06%
Questionnaire developed for studyNot indicated(44)
  • Top Level Commitment (TLC)/n = 9
  • Just culture (JC)/n = 6
  • Learning culture (LC)/n = 4
  • Awareness (AW)/n = 8
  • Preparedness (P)/n = 10
  • Flexibility (FL)/n = 5
  • Opacity (O)/n = 2
Reliability-Cronbach’s alpha (⍺), Intraclass Correlation Coefficient (ICC).
Validity–Content Validity Index/ratio: CVI and CVR

TLC = 0.83
JC = 0.85
LC = 0.77
AW = 0.88
P = 0.89
FL = 0.87
O = 0.63
⍺ whole tool: 0.951
ICC whole tool: 0.95
CVI each item: > 0.80
CVR each item: > 0.75
Shirali, Motamedzade, Mohammadfam, Ebrahimipour and Moghimbeigi [53]32
Experienced operators (n = 24)
Managers (n = 8)
Semi-structured interviews with multiple-choice questionsNot indicated(105)
  • Buffering Capacity (BC)/n = 27
  • Flexibility (F)/n = 21
  • Margins (M)/n = 7
  • Tolerance (T)/n = 8
  • Cross-Scale Interactions (CSI)/n = 5
  • Learning (L)/n = 13
  • Attention (A)/n = 9
  • Response (R)/n = 10
  • Anticipation (A)/n = 5
PCA used to prioritize the 9 RE indicators
Numerical Taxonomy (NT) used to validate the results of the PCA
PCA Ranking/Ranking
BC = 1/2
F = 2/1
M = 8/8
T = 6/6
CSI = 9/9
L = 3/3
A= 5/5
R = 4/4
A = 7/7
Azadeh, Salmanzadeh-Meydani and Motevali-Haghighi [54]97
Senior managers (n = 2)
Mid managers (n = 18)
Administrators (n = 37)
Staff (n = 40)
Questionnaire developed for study and based on IRE factorsIntegrated RE(30)
  • Management Commitment (MC)/n = 3
  • Reporting Culture (RC)/n = 3
  • Learning (L)/n = 3
  • Awareness (AW)/n = 4
  • Preparedness (PR)/n = 4
  • Flexibility (F)/n = 3
  • Self-Organisation (SO)/n = 2
  • Teamwork (TW)/n = 3
  • Fault-Tolerant (FT)/n = 2
  • Redundancy (R)/n = 3
Data Envelopment Analysis (DEA)PCA and Numerical Taxonomy (NT) used to validate and verify the resultsReliability of collected data: Cronbach’s alpha
MC= 0.839/5
RC = 0.604/2
L = 0.730/4
AW = 0.776/9
PR = 0.880/10
F= 0.614/3
SO = 0.678/1
TW= 0.833/7
FT = 0.749/8
R = 0.754/6

⍺ whole tool: 0.913
Chen, McCabe and Hyatt [55]431
Questionnaires collected from 68 construction sites
(403 analysed).
Questionnaire developed for studyNot indicated(28)
  • Management Commitment (MC)/n = 6
  • Supervisor Safety Perception (SSP)/n = 6
  • Co-worker Safety Perception (CSP)/n = 4
  • Learning (L)/n = 4
  • Reporting (R)/n = 3
  • Anticipation (A)/n = 2
  • Awareness (AW)/n = 3
Cronbach’s alpha of the scales for each dimension.
MC= 0.87
SSP = 0.86
CSP = 0.72
L = 0.83
R = 0.71
A = 0.68
AW = 0.80
Shirali, Shekari and Angali [56]312
Questionnaires collected from 12 units of plant.
Managers = 61
Operators = 251
Questionnaire developed for studyNot indicated(57)
  • Just Culture (JC)/n = 7
  • Management of Change (MoC)/n = 6
  • Learning Culture (LC)/n = 6
  • Risk Assessment/Management (RAM)/n = 5
  • Preparedness (P)/n = 4
  • Flexibility (F)/n = 6
  • Reporting culture (RC)/n = 4
  • Management Commitment (MC)/n = 6
  • Awareness (AW)/n = 3
  • Safety Management System (SMS)/n = 3
  • Accident investigation (AI)/n = 3
  • Involvement of Staff (IoS)/n = 3
  • Competency (C)/n = 3
Reliability-Cronbach’s alpha (⍺)/test-retest r
Validity–Content Validity Index/ratio: CVI and CVR/Kaiser–Meyer–Olkin (KMO) index

JC = 0.84
MoC = 0.85
LC = 0.86
RAM = 0.89
P = 0.91
F = 0.83
RC = 0.88
MC = 0.77
AW = 0.89
SMS = 0.84
AI = 0.88
IoS = 0.83
C = 0.67
⍺ whole tool: 0.943
CVI: 0.97
CVR: 0.83
KMO: 0.88
r = 0.85; p < 0.001
Garg, Alroomi, Tonmoy and Mohamed [57]139
Engineers, supervisors and managers.
Questionnaire developed for studyResilient safety culture (RSC) model(42)
  • Behavioural Capability (BC)/n = 15
  • Managerial Capability (PC)/n = 18
  • Psychological Capability (MC)/n = 9
Relative importance index method (RII)—used to quantify the relative importance of all the 42 indicators of RSC for remote sitesRII/Ranking of constructs
BC = 0.585/(1)
PC = 0.576/(2)
MC = 0.562/(3)
Shirali and Nematpour [58]489
Resilience engineering survey questionnaireNot indicated(61)
  • Management Commitment (MC)/n = 9
  • Just Culture (JC)/n = 11
  • Learning Culture (LC)/n = 15
  • Awareness and Opacity (A-O)/n = 11
  • Preparedness (P)/n = 9
  • Flexibility (F)/n = 6
Analysis network process (ANP) used to quantify and determine the priorities of RE dimensionsRanking
MC= 1
JC = 2
LC = 3
A-O = 6
P = 5
F= 4
Zarrin and Azadeh [59]71
Questionnaire developed for study, including RE (7), HSEE (4) factorsNot indicated(25)
  • Top Management Commitment (TMC)/n = 4
  • Learning (L)/n = 3
  • Flexibility (F)/n = 4
  • Reporting Culture (RC)/n = 3
  • Awareness (AW)/n = 5
  • Preparedness (P)/n = 4
  • Redundancy (R)/n = 2
Reliability-Cronbach’s alpha (⍺)
For the rest of the study: Z-numbers with Fuzzy Cognitive Map (FCP) approach

TMC = 0.88
L = 0.72
F = 0.77
RC = 0.81
AW = 0.84
P = 0.80
R = 0.86

Share and Cite

MDPI and ACS Style

Pillay, M.; Morel, G. Measuring Resilience Engineering: An Integrative Review and Framework for Bench-Marking Organisational Safety. Safety 2020, 6, 37.

AMA Style

Pillay M, Morel G. Measuring Resilience Engineering: An Integrative Review and Framework for Bench-Marking Organisational Safety. Safety. 2020; 6(3):37.

Chicago/Turabian Style

Pillay, Manikam, and Gaël Morel. 2020. "Measuring Resilience Engineering: An Integrative Review and Framework for Bench-Marking Organisational Safety" Safety 6, no. 3: 37.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop