Identiﬁcation of Contributing Factors to Organizational Resilience in the Emergency Response Organization for Nuclear Power Plants

: Resilience engineering is a new approach to safety, focused on systems for coping with complexity and balancing productivity with safety. Since the early 2000s, several studies have been conducted on the application of resilience to various industries. However, the nuclear industry has yet to harness the full potential of the resilience concept. The International Atomic Energy Agency (IAEA) gave an inkling of the use of this concept in its report on the human and organizational factors related to the Fukushima nuclear power plant (NPP) accident. Although the ability of emergency response organizations (EROs) to reduce the radiation risks to the public in the case of accidents is crucial, no method has been developed so far to evaluate ERO resilience in NPPs. This paper aims to determine the factors that contribute to the resilience of EROs in NPPs. This work commenced by providing a systematic review of the literature on resilience factors as applied in several domains within the last two decades, including general domains, healthcare, transportation, infrastructure, process plants, and business. Based on the review, and the application of additional procedures like resilience analysis grid ﬁltering, ERO applicability assessment, and merger/reclassiﬁcation, the resilience factors are determined. Fifty-two factors contributing to the resilience of EROs in NPPs are proposed. The identiﬁed contributing factors are expected to aid efforts to develop resilience strategies and to measure the resilience of EROs in NPPs.


Introduction
Resilience engineering (RE) is a new approach to safety focused on systems for coping with complexity and balancing productivity with safety. RE aims to provide tools for addressing the proactive management of risk, inherent complexity of systems, and corresponding need(s) for performance variability [1]. Resilience can be defined as the ability of a system to adjust its functioning prior to, during, or following changes and disturbances so that it can sustain the required operations under both expected and unexpected conditions [1]. Resilience is considered a "Safety-II" concept focused on success and how things often go well, in contrast to a "Safety-I" concept focused on failures and analysis of their causes [1,2].
According to E. Hollnagel [1], D. Woods [3], and N. Leveson [4], resilience research has been conducted since the early 2000s. Resilience studies have been reported in the aviation [5], medical [6], space [7], disaster management [8], and nuclear domains. Some studies have been conducted to assess reliability based on the concept of resilience. For instance, A. V. Lee et al. [9] developed a survey tool for organizations in order to identify their strengths and weaknesses and to develop and evaluate the effectiveness of their resilience strategies and investments. A. Azadeh et al. [10] conducted research on a quantitative assessment of RE, based on nine resilience indicators in the process plant field. R. Burch [11] presented a method for the aerospace field for calculating resilience values to represent an expected value of the retained capability for a particular threat scenario studies are specific to the resilience of EROs in NPPs. Thus, evaluation factors and methods for clearly assessing the resilience of NPP EROs have not been developed.
This study aims to identify contributing factors for resilience evaluation as part of the development process of an ERO resilience evaluation method for NPPs. Accordingly, this study conducted a systematic review of the literature on resilience as applied in several industries, aiming to identify all the factors contributing to the resilience of EROs in NPPs. The remainder of this paper is organized as follows. Section 2 provides a review of the literature on resilience application cases. Section 3 introduces the EROs and the four capabilities required for resilience as proposed by Hollnagel. Section 4 describes a classification process for the contributing factors derived from the results of the review of the literature. Section 5 proposes the contributing factors for ERO resilience in NPPs. Section 6 discusses the results and proposes steps for future research. Finally, Section 7 is the concluding statement.

Review of Resilience in Several Domains
The publications used in this study were selected from Google, ScienceDirect, and Google Scholar. In order to properly review the appropriate literature within each website, the search was conducted with the following keywords: "Resilience," "Resilience Analysis Grid," "High Reliability Organization," "Organizational Resilience," and "Resilience Evaluation." The inclusion criteria for papers to be considered in this review were as follows: (1) the paper was published in English or Korean; (2) the paper indicated the resilience contributing factors with corresponding descriptions; and (3) the paper focused on organizational resilience, rather than on individuals.
After reading the abstracts and overall structures of the retrieved studies, irrelevant studies were excluded from the review, so as to give the reader a broad yet focused context of resilience in safety-critical fields, and to ultimately identify and define the factors contributing to organizational resilience (i.e., "contributing factors"). Approximately 166 studies were initially retrieved for this review. However, after considering the purposes of the papers, their definitions of resilience, and the contributing factors to resilience mentioned therein, only 63 papers were fully reviewed for this study.
The fully reviewed papers applied to several fields, but were grouped into larger related fields. A total of seven categories were created, and brief reviews thereof are presented in the following subsections.

Research on Resilience in the General Domain
The "general" domain can be classified as general methodology studies or studies applicable to numerous fields. A total of 13 papers were reviewed for this domain. In this context, E. Hollnagel [1,3] proposed four capabilities of RE required for resilient performance. A total of 38 evaluation factors were presented. Salisu et al. [25] reviewed four methods used to evaluate regulations (the Resilience Scale for Results, Connor-Davidson Resilience Scale, Connor-Davidson Resilience Scale 10, and Brief Resilience Scale). Righi et al. [26] conducted a review of the literature with the aim of presenting a research agenda for RE. Kantur et al. [15] developed a framework for enabling a better understanding of the concepts and relationships in organizational resilience-level analyses. Pillay [27] developed a framework for future research in RE as a safety-management strategy. Nemeth et al. [28] provided scientific and industrial communities with an opportunity to present current RE work diagrams, and to conduct research to identify the progress and contributions in RE research and applications. The study presented inspiring questions regarding the future of RE. Lee et al. [9] developed a survey to enable organizations to understand their strengths and weaknesses, and to develop and evaluate the effectiveness of their resilience strategies and investments. Amir et al. [29] conducted research regarding the understanding of organizational resilience and reinforcement of regulations through a review of growth-based compositions and measurements. Kim [30] studied how the effects of family resilience support the successful adaptation of remarried families. McManus et al. [31] identified the Energies 2022, 15, 7732 4 of 24 key contributors to organizational resilience with case studies from a wide range of industry sectors in New Zealand, and suggested ways to strengthen them. Mendonca et al. [32] proposed experience-based criticism and contributing factors to supplement existing contribution factors, based on the 2011 New York power loss incident. Salanova et al. [33] conceptualized and evaluated the "Health & Resilient Organization" model. Table 1 gives a summary of the identified contributing factors in the general domain, along with the corresponding references.

Research on Resilience in the Process Plant Domain
The process plant domain classification includes NPPs, chemical plants, and industrial domains. In this context, 11 studies were reviewed. Hollnagel et al. [34] demonstrated how RE can replace industrial safety understanding in situations where safety systems pre-equipped in NPPs operate properly, and vice versa. Park et al. [2] developed a Safety-II model for safety in unexpected reactor shutdown situations at NPPs, based on the "Model of Resilience in Situation" developed by Électricité de France. Kim et al. [24] developed a quantitative evaluation model for unexpected scenarios. Kamanja et al. [19] characterized resilience in an NPP EOS, aiming to gain an understanding of the various resilience properties. Labaka et al. [35] developed a comprehensive resilience framework for considering both internal and external organizational regulations and the resilience level of a plant. Gauthereau et al. [36] conducted a study aiming to provide a general view of emergency response plans in the event of an accident at a Swedish NPP. Shiral et al. [37] conducted a quantitative evaluation of RE based on nine resilience factors.
Azadeh et al. [38] conducted research to strengthen the scientific platforms for petrochemical plants and identify the performance of the corresponding human and safety resources by considering a new understanding of the relationship between the performance assessment and resilience level. Rabbani et al. [39] proposed a novel optimization algorithm for evaluating the RE culture and performance in an industry based on graph theory, a matrix approach, and statistical methods for analysis of safety cultures. Shiral et al. [40] conducted a quantitative evaluation of RE in the process plant industry, based on six RE factors, including Top Management Commitment, Just Culture, Learning Culture, and Awareness and Opportunity. Morales et al. [41] identified the factors contributing to organizational resilience when an organization performs particular tasks, and presented a differentiation from the existing RE. Gomes et al. [42] conducted simulations to determine the influence of a system's resilience in real-world disaster situations. Azadeh et al. [43] introduced the main factors contributing to petrochemical plant resilience and conducted research to improve the safety of petrochemical plants by calculating the effects of these contributing factors on the system. Table 2 gives a summary of the identified contributing factors in the process plant domain along with the corresponding references.

Research on Resilience in the Business Domain
Five papers pertaining to the application of the concept of resilience to businesses and companies are reviewed in this subsection. Annarelli et al. [44] presented a methodology for assessing the effectiveness of organizational resilience strategies in the service industry. Azusa et al. [45] identified the psychological factors corresponding to transient trauma or distress from unexpected serious events, aiming to provide quicker recovery. Beek et al. [12] developed the "Analyzing and Developing Adaptability and Performance in Teams to Enhance Resilience" (i.e., ADAPTER) organizational resilience evaluation model. Ljungberg et al. [46] applied the "Resilience Analysis Grid" (RAG) approach to analyze a Swedish Air Traffic Management navigation service company, e.g., with regard to its ability to respond to unexpected incidents and disturbances. Bruggy et al. [47] emphasized the need for organizational resilience, a model for organizational resilience, and methods for improving organizational resilience. A summary of the identified contributing factors in the business domain, along with the corresponding references, is presented in Table 3. Table 3. Summary of identified contributing factors from the business domain.

Reference
Main Topic

No. of Factors Examples
A. Annarelli et al. [44] Presenting a methodology for assessing the effectiveness of organizational resilience strategies in the service industry.

Research on Resilience in the Medical and Healthcare Domain
Seven papers related to the application of resilience to hospital emergency rooms and healthcare, in general, were reviewed. Santomauro et al. [6] reviewed the second victim phenomenon for both symptom studies and organizational responses in healthcare. Goncalves et al. [48] applied a benchmark resilience tool in a Spanish environment in order to identify safety aspects and relationships. Iflaifel et al. [13] identified and analyzed elements for developing and enhancing RHC through conceptualization thereof for research, identification, and analysis methods. Barasa et al. [14] conducted a systematic review, aiming to apply the concept of resilience to the healthcare domain. McEwen et al. [49] developed a team resilience measurement method called "Resilience at Work" to measure individual and organizational regulations, and measured the results and independent contributing factors. Ayed [50] determined the impacts of strategic human resource management in an organizational resilience context. Chuang et al. [51] identified factors for improving the performance of emergency room personnel, based on the four RAG capabilities proposed by Erik Hollnagel. A total of 36 resilience contributing factors were considered, including Event List, Background, and Relevance. A summary of the identified contributing factors in the medical and health care domain along with the corresponding references is presented in Table 4. Table 4. Summary of identified contributing factors from the medical and healthcare domain.

Reference
Main Topic

No. of Factors Examples
C. Santomauro et al. [6] Reviewing the second victim phenomenon for both symptom studies and organizational responses in healthcare.
4 Shame, Self-blame, Loss of Sleep, and Self-doubt.
L. Goncalves et al. [48] Applying a benchmark resilience tool in a Spanish environment to identify safety aspects and relationships. Developing a team resilience measurement method called "Resilience at Work" to measure individual and organizational regulations, and measuring the results and independent contributing factors.
S. Ayed [50] Determining the impacts of strategic human resource management in an organizational resilience context.

Research on Resilience in the Transportation Domain
The transportation domain refers to, e.g., aerospace, railway, and public transportation sectors. In total, 11 papers covering resilience concepts in these sectors were reviewed. Burch et al. [11] proposed an evaluation measure for resilience, aiming to allow a system to maintain its functionality in certain threat scenarios. Owen et al. [52] proposed a key method for resilience to improve readiness for global aviation and shipping transport, and discussed its consequences. A report from the Homeland Defense and Global Security Office [7] (the organization in the USA that defines the Office of the Secretary of Defense's policy perspective and conceptual origins of an actionable taxonomy for missions in space) identified factors contributing to resilience in the aerospace context. Patriarca et al. [53] developed the RAG-State Assessment Tool to combine Safety-I and Safety-II principles. Gabriel [54] tested whether the organizational citizenship behavior in Nigeria, especially in the domestic aviation sector, could lead to resilience. Woltjer et al. [55] successfully applied both resilience and agility to real-world models. Heese et al. [5] developed the "Inventory of Behavioral Resilience in Aviation" for monitoring and evaluating the daily behaviors of groups in the aviation industry, and conducted research to validate it. Azadeh et al. [38] evaluated the resilience performance of a railway transport system. Omer et al. [56] presented metrics to measure the resilience of an intelligent transportation system. Huber et al. [57] suggested contributing factors to represent an organization's ability to cope with interference. Research was also conducted on how this metric could be measured. Saurin et al. [58] introduced a framework for representing the sources of analyses of resilience vulnerabilities, without limiting the identification process to specific systems within socio-technical systems or to specific units of analysis. Table 5 gives a summary of the identified contributing factors in the transportation domain along with the corresponding references. Table 5. Summary of identified contributing factors from the transportation domain.

Reference
Main Topic

No. of Factors Examples
R. Burch et al. [11] Proposing an evaluation measure for resilience, aiming to allow a system to maintain its functionality in certain threat scenarios. J. Gabriel [54] Testing whether the organizational citizenship behavior in Nigeria, especially in the domestic aviation sector, could lead to resilience.
3 Situation Awareness, Keystone Vulnerability, and Adaptive Capacity.
A. Azadeh et al. [38] Evaluating the resilience performance of a railway transport system. 3 Teamwork, Redundancy, and Self-organization.
M. Omer et al. [56] Presenting metrics to measure the resilience of an intelligent transportation system.

5
Leadership, Awareness, Flexibility, Preparedness and Emergency Planning, and Culture.
G. Huber et al. [57] Suggesting contributing factors to represent an organization's ability to cope with interference. 4 Awareness, Efficiency, Commitment, and Adaptability.

Research on Resilience in the Infrastructure Domain
A total of 14 papers applying the concept of resilience to construction, disaster, and socio-technical systems were reviewed. Rehak et al. [59,60] conducted a study to verify the "Critical Infrastructure Elements Resilience Assessment" methodology. They also researched the strengths of organizational resilience, and how such resilience could be measured in critical infrastructure systems. Francis et al. [61] reviewed risk analysis and resilience analysis to provide guidance to infrastructure system engineers. Renschler et al. [62] identified and measured disaster resilience for organizations on a variety of scales, and developed the "Population and Demographics, Environmental/Ecosystem, Organized Government Services, Physical Infrastructure, Lifestyle and Community Competence, Economic Development, and Social-cultural Capital" (i.e., " PEOPLES") resilience framework. Patriarca et al. [63] developed an approach to measure resilience abilities after developing a questionnaire based on the four capabilities of RAG proposed by Erik Hollnagel. Rioli et al. [64] conducted a study on the theory of stress and resilience in the information system field. Lee et al. [65] analyzed existing domestic and international disaster management evaluation systems and developed a model for evaluating disaster management using RE. Yang et al. [66] proposed a method for self-evaluating the cultural heritage disaster management capabilities of cultural heritage administrations, and for managing vulnerable areas based on a regeneration disaster management competency assessment. Rodrigues et al. [67] proposed a systematic approach to evaluating the usefulness of a water sector analysis method by applying the four capabilities of the RAG proposed by Erik Hollnagel, and evaluating the organizational resilience in each water sector. Nam et al. [52] proposed measures for strengthening resilience by minimizing the impacts of climate change on social and economic activities in coastal areas and maximizing the ability to recover. Sungho Hong [8] researched the need for an organization dedicated to national disaster management, and for improvements in resilience capacities and systems to enhance the effectiveness of such an organization. Gwangwu Cho [68] analyzed the impacts on and vulnerability of Korea's coast, owing to rising sea levels and climate change in terms of community planning and adaptation methods, and presented corresponding policy directions. Sung Cho [69] analyzed the impacts of disaster preparedness activities on the disaster management effects of the public officials in charge of disaster management. Stewart et al. [70] analyzed national disaster-related resilience at the community level. A summary of the identified contributing factors in the infrastructure domain, along with the corresponding references, is presented in Table 6.

Research on Resilience in Other Domains
The studies not applicable to the already defined domains, or with only a few examples of resilience-related content, were classified into "Other Domains." Four papers were reviewed. Gonzalo et al. [71] proposed a method for assessing RE in response to a military disaster by identifying the optimal state of resilience. The study also evaluated the current state of organizational resilience in that sector. Johns et al. [72] reviewed the specific resilience contributing factors affecting the overall reliability of any organization. Woods [73] proposed an overall RE approach and also proposed some resilience contributing factors. Wicker et al. [74] investigated the organizational resilience of common sports clubs in the aftermath of natural disasters. Table 7 gives a summary of the identified contributing factors in other domains along with the corresponding references. Investigating the organizational resilience of common sports clubs in the aftermath of natural disasters.

Emergency Response Organizations and Capabilities of Organizational Resilience
This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, and the experimental conclusions that can be drawn.

Emergency Response Organizations
Typically, emergency response organizations (EROs) are set up for nuclear power plant emergencies and may also be mandated by law. For example, the Nuclear Emergency Preparedness Act of Korea has stated that the nuclear utility has an obligation to secure personnel and organizations exclusively in charge of work in preparation for radioactive disasters or mitigating the consequences of such disasters. In the event of a nuclear plant accident, the ERO performs several activities, including initial response to the accident, prevention of accident escalation, accident recovery and decontamination activities, and accident cause investigation. The ERO in Korean plants consists of the Technical Support Center (TSC), Operating Support Center (OSC), and Emergency Operation Facility (EOF). The following describes the roles of the three EROs.

Emergency Operation Facility (EOF)
The role of the EOF is to supervise emergency response activities of power plants, cooperate with external disaster prevention agencies, monitor environmental emission of radioactive materials, measure and evaluate the radiation (ability), and recommend residents' expected exposure dose and protective measures.

Operating Support Center (OSC)
The role of the OSC is to perform emergency maintenance, fire extinguishing and relief activities, and to serve as a waiting place for emergency maintenance support personnel, to establish a cooperation system with the MCR and EOF. The head of the OSC shall operate the OSC in general, keep emergency maintenance consultation with the TSC, and cooperate with related organizations related to fire extinguishing and medical treatment response.

Technical Support Center (TSC)
The TSC performs the functions of the EOF until the EOF is established, in order to prevent confusion among the MCR operators and to provide technical and administrative support necessary for responding to accidents. The purpose of this action is to ensure that MCR operators are free from peripheral tasks that are not directly related to reactor and system operations. The Director of TSC directs the emergency response before the EOF is established, and when the EOF is established, the Director of EOF shall report the results of the emergency situation and make a final decision on important matters. If an EOF is established, the head of TSC proposes to the head of EOF a final decision on the results of the emergency analysis and countermeasures, as well as the overall important matters of critical incident management.

Capabilities of Organizational Resilience
If resilience is defined as described in the Introduction, then the purpose of organizational resilience is to create resilience in an organization. In this context, organizational resilience represents the ability of an organization to adjust its tasks to both unexpected and expected conditions. Hollnagel [75] proposed a RAG concept that requires systems to have four abilities to achieve a high level of resilience. The four capabilities proposed in the RAG are as follows: (1)  These four capabilities are typically associated with NPPs. For example, an NPP will be designed based on other industries or previous experience (learning), and the NPP will have the necessary safety systems to cope with accidents and possible accident scenarios from the NPP at that stage (anticipation). In the plant operation phase, the direct and indirect acquisition of NPP variables will be monitored to cope with current conditions (monitoring), and when events occur, they will respond with NPP stabilization based on monitoring (responding).
All of the aforementioned tasks are currently conducted in NPPs by organizations (including humans). Thus, this study applies the four RAG capabilities as high-level factors to identify NPP organizational resilience factors.

Methodology for Identification and Classification of Contributing Factors to Organizational Resilience in Nuclear Power Plants (NPPs)
A total of 1282 contributing factors were derived from the review of the literature discussed above. However, it is difficult to directly apply all of these contributing factors to the EROs at NPPs. This is because some of the contributing factors in the literature are domain-specific, and not relevant to NPPs. Hence, this study developed a process to identify the specific factors contributing to the resilience of NPP EROs. The process consisted of three steps, as shown in Figure 1.

Step 1: Is It a Contributing Factor That Can Be Classified According to the Resilience Analysis Grid (RAG)?
Based on the review of the literature, this study determined whether the preselected contributing factors were applicable to the RAG concept proposed by Hollnagel [1,3]. As mentioned above, a total of 1282 contributing factors were identified in Step 1. Of these, 650 were rejected, because they were deemed inapplicable to the four RAG capability definitions. For example, Yoojung Kim [30] includes "Belief Systems" as a proposed contributing factor. This is an ambiguous factor, and was deleted because it did not correspond to any of the capabilities of the RAG. Another example is in Rehak et al. [59,60], which suggested "Allocation of Financial Resources." The definition of this contributing factor is "allocation and management of financial capital to help an organization achieve its strategic goals." In the case of individuals, finance can be a significant factor. However, in a case where the nuclear utility is managed by the government, it was judged that issues relating to finance need not be presented. The ultimate goal of Step 1 was to define which contributing factors could be classified as contributing factors within the RAG's four capabilities. Hence, this study compared and analyzed all contributing factors, along with the definition of RAG capabilities.

ganizational Resilience in Nuclear Power Plants (NPPs)
A total of 1282 contributing factors were derived from the review of the literature discussed above. However, it is difficult to directly apply all of these contributing factors to the EROs at NPPs. This is because some of the contributing factors in the literature are domain-specific, and not relevant to NPPs. Hence, this study developed a process to identify the specific factors contributing to the resilience of NPP EROs. The process consisted of three steps, as shown in Figure 1.

Step 1: Is It a Contributing Factor That Can Be Classified according to the Resilience Analysis Grid (RAG)?
Based on the review of the literature, this study determined whether the preselected contributing factors were applicable to the RAG concept proposed by Hollnagel [1,3]. As mentioned above, a total of 1282 contributing factors were identified in Step 1. Of these, 650 were rejected, because they were deemed inapplicable to the four RAG capability definitions. For example, Yoojung Kim [30] includes "Belief Systems" as a proposed contributing factor. This is an ambiguous factor, and was deleted because it did not correspond to any of the capabilities of the RAG. Another example is in Rehak et al. [59,60], which suggested "Allocation of Financial Resources." The definition of this contributing factor is "allocation and management of financial capital to help an organization achieve its strategic goals." In the case of individuals, finance can be a significant factor. However, in a case where the nuclear utility is managed by the government, it was judged that issues relating to finance need not be presented. The ultimate goal of Step 1 was to define which

4.2.
Step 2: Is the Contributing Factor Applicable to Emergency Response Organizations (EROs) in NPPs?
In Step 2, the remaining factors (results from Step 1) were scrutinized in order to verify that the resilience contributing factors applicable to other industries were also applicable to EROs at NPPs. In this phase, some criteria were applied to determine the applicability to the EROs at NPPs. A total of 632 contributing factors were considered in Step 2. Of these, 219 were rejected, because they were deemed inapplicable to the ERO activity.
The criteria were initially reviewed based on the Korean Atomic Energy Act on Protection of Nuclear Facilities, the Act on Countermeasures against Radioactive Disaster Prevention, the Enforcement Decree, and the Enforcement Rule. For example, the context for the declaration and report of a radioactive disaster under Article 23 of the Act is as follows: "in the event of a radioactive disaster, a radioactive disaster must be declared without delay." This content allows other organizations or the public to be informed regarding radioactive disasters, and should be related to the reports on the relevant factors. Accordingly, a contributing factor called "report" was considered to be applicable.
The literature related to nuclear power and safety was also reviewed. Human reliability analysis (HRA) is a major part of the PSAs for NPPs, as it considers the organizational aspects of safety. As such, the PSA literature [76][77][78][79][80], nuclear and nuclear safety literature [81][82][83][84], and safety culture literature [85][86][87] were scrutinized for resilience contributing factors. At this stage, the basic concept was to identify the contributing factors suggested for HRA via performance shaping factors (PSFs). For example, the definition of execution is "The measure of whether the person performs the intended actions correctly." This contributing factor passed Step 1 as a possible contributing factor corresponding to Response in RAG Capability. However, according to HRA, 'execution' is related to the direct manipulation of equipment. In the case of EROs of NPPs, they do not directly operate (manipulate equipment or control) the NPPs. EROs only provide MCR operators with appropriate decision-making results. Accordingly, Execution was rejected in Step 2. The reason why the PSFs were used as the basis for the HRA is that they are used in reliability evaluation methods applied to humans currently working in NPPs. Thus, it was expected that many PSFs would be applicable to EROs. For example, the definition of the "Procedure" PSF in the "Standardized Plant Analysis Risk-Human Reliability Analysis" method (an HRA method) refers to the existence and use of pre-written steps/guidelines for the situation under consideration. It determines whether incorrect or inappropriate information is provided based on the characteristics and sequences of the procedure [76]. Such procedures represent a very important factor for NPPs. In fact, the Korean Nuclear Law stipulates on "matters related to procedures such as operation and repair." In addition, the IAEA has discussed Procedure as an important factor in determining whether expected outcomes are met [88], and the importance of ensuring the appropriate qualification of procedure writers [89].

Step 3: Merge and Classify Identified Factors
In Step 3, a merger and re-classification of the remaining contributing factors was performed based on whether the screening tasks of Steps 1 and 2 are performed. At this stage, redundant contributing factors were eliminated, and the proposed contributing factors were reclassified. A total of 413 contributing factors were identified from Step 2. Finally, a total of 52 contributing factors were identified for the ERO.
This was necessary because the extensive literature review showed that various authors used different words to refer to the same context. For instance, both Park et al. [2] and Kim et al. [24] used a contributing factor called "Teamwork." The definition of Teamwork was "how operators interact with each other to exchange information, coordinate behavior, and maintain social order." In another study, Salisu et al. [25] used a contributing factor called "Cooperation", and the definition represented "cooperation, including information transfer with other organizations." Thus, these factors were judged as the "Communication in Level 2" to have the same meaning, as they both denoted the interactions of operators or different organizations. In this study, the four capabilities proposed by RAG were each denoted as a "Level 1 RAG Capability," and Level 2 Factors were identified as contributing to Level 1. For example, in the process of merging, Cooperation was selected as a Level 2 factor, whereas Teamwork was used as a factor for evaluating Cooperation. There were 35 Level 3 factors identified as sub-contributing factors to the Level 2 factors. The Level 3 contributing factors were necessary because the Level 2 factors had relatively broad scopes. More specific assessment items will likely ease future evaluations of EROs at NPPs.

Contributing Factors for Organizational Resilience in NPPs
A total of 52 factors contributing to the resilience of EROs at NPPs were identified according to the three aforementioned steps. The contributing factors comprised three levels. The structure of the contributing factors is shown in Figure 2. A detailed description of each factor is given in the following subsections.

Responding Capability for the ERO of NPP
A total of 19 contributing factors were considered for the response capability. Five contributing factors were considered as Level 2 factors, and 14 contributing factors were considered as Level 3 factors. The definitions of each contributing factor (Level 2 and Level 3) are shown in Table 8. For each of the contributing factors to responding capability, the relevant literature has also been indicated.

Monitoring Capability for the ERO of NPP
For the monitoring capability, 10 contributing factors were considered. Three contributing factors were considered as Level 2 factors, and seven contributing factors were considered as Level 3 factors. The definitions of each contributing factor (Level 2 and Level 3) are shown in Table 9. For each of the contributing factors to monitoring capability, the relevant literature has also been indicated.
evaluations of EROs at NPPs.

Contributing Factors for Organizational Resilience in NPPs
A total of 52 factors contributing to the resilience of EROs at NPPs were identified according to the three aforementioned steps. The contributing factors comprised three levels. The structure of the contributing factors is shown in Figure 2. A detailed description of each factor is given in the following subsections.    [26,34,55,59,60] Are effective and accurate responses taken (or possible)?  [2,10,14,19,24,30,59] Is there an adequate number of personnel in place?

Learning Capability for the ERO of NPP
For the learning capability, 10 contributing factors were considered. Two contributing factors were considered as Level 2 factors, and eight contributing factors were considered as Level 3 factors. The definitions of the contributing factors (Level 2 and Level 3) are shown in Table 10. For each of the contributing factors to learning capability, the relevant literature has also been indicated.

Anticipating Capability for the ERO of NPP
For the anticipating capability, a total of 13 contributing factors were considered. Three contributing factors were considered as Level 2 factors, and ten contributing factors were considered as Level 3 factors. The definitions of each contributing factor (Level 2 and Level 3) are shown in Table 11. For each of the contributing factors to anticipating capability, the relevant literature has also been indicated.  [2,9,14,19,23,24,26,31,42,44,48,53,56,57,59,60,63] Is there a proper plan for accident management? 4.2.1 Plan Establishment [2,9,14,19,24,53,56,57,59,60,63] Is there a radioactive disaster prevention plan and a manual for on-site measures in preparation for accidents? The results from this study are absolutely essential to any efforts to evaluate the resilience of EROs in NPPs. These factors, as presented in the results, would be the starting point for developing a method for ERO resilience analysis.
For the ERO responding capability, factors such as Speed, Responding Preparedness, Action List, Background, Revision, Duration, Effectiveness, Number of Personnel, Qualification, Role and Responsibility, Stopping Rule, Alternative Strategy, Availability, and Adequacy, should be used for evaluation. For example, the definition of Speed is "Can ERO respond quickly to the situation?" This means how quickly appropriate required action can be performed in an actual emergency situation. In the case of such situations, ERO staff must perform the appropriate actions that eventually prevent the situation from evolving into a severe accident.
Similarly, for the monitoring capability, the factors such as Monitoring List, Decisionmaking Criteria, Monitoring Preparedness, Availability, Adequacy, Support System, and External Support can be used at the lower levels for evaluation. For example, the definition of a Monitoring List is "Does the procedure describe the targets to be monitored?" This checks whether the procedure mentions the appropriate monitoring variables for the ERO in an actual emergency response situation. In actual emergency situations, when EROs are launched, MCR provides the current state or variables of the plant to the ERO without a decision-making authority. The ERO receives necessary information from the MCR to make decisions. This is a limitation of the current strategy. Thus, procedures that enable ERO to identify the required variables, i.e., a monitoring list, can be an important requirement for resilience at this stage.
Factors such as Trainer, Method, Objective, Facility, Evaluation, and Frequency can be used to evaluate the learning capability at the lower levels. For example, the definition of a trainer is "Are the qualities of a trainer/instructor sufficient?" This is a factor that represents the instructor's ability to train in order for the ERO to properly handle emergency situations in real emergency response situations, including those situations that are unprecedented. ERO is a large organization made up of employees in various fields. Thus, instructors who can properly train the varied members of this organization are important contributors to ERO resilience.
In addition, factors such as Communication Equipment, Clarity, Plan Establishment, Scope, Time Horizon, Basis, Level, Cooperative Norm, Risk Factor Recognition, and Trust and Engagement should be used for the lower level evaluation of the Anticipating capability of EROs. For example, the definition of Communication Equipment is "Does the ERO have tools for communication?" This is a factor that identifies whether ERO has an appropriate communication device to handle an emergency situation in an actual emergency response situation. In the case of ERO, communication with other organizations, such as TSC and EOF, is important, starting with the MCR. In addition, inadequate communication in emergency situations will not adequately stabilize the plant and can be the starting point for serious accidents. Accordingly, Communication Equipment is an important contributing factor.

Limitations and Future Work
This study provides readers with a clear idea of the broad applications of the resilience concept in various domains, and based on an extensive review of the literature, a systematic approach is used to derive the factors contributing to the resilience of EROs in NPPs. However, during the practical application of these factors to determine the resilience of EROs in NPPs, some caution must be exercised.
First, when the contributing factors derived from this study are applied to an actual ERO of an NPP, they must be evaluated by considering factors appropriate to the characteristics of each ERO. For example, among the factors contributing to the response, the "execution" may be assessed as contributing to the main control room (MCR), but in contrast, it is difficult to apply "execution" to a technical support center, operations support center, emergency headquarters, etc. The reason for this is that the reactor operation of the NPP is mostly conducted in the MCR; therefore, there is no need to evaluate the top ERO for "execution." As all of the contributing factors currently proposed have been derived for all organizations responsible for NPP emergency responses, each of the necessary factors for a detailed ERO needs to be selected. For this reason, future studies should add procedures for deriving the applicable resilience contributors for each ERO.
In addition, the interactions between the organization and contributing factors must also be considered, but were not considered in the literature review for this work. The interaction between an organization and a factor should be considered, because it is to determine to what extent the level 3 items can affect the level 2 and 1, and also to consider the effects of the items (or factors) between those on the level 1 on the level 2 and 3, so these interactions should be considered in future studies. Two procedures are proposed in this work for considering the interactions between contributing factors. First of all, it is necessary to identify the correlations between each ERO. Next, as mentioned earlier, the factors specifically contributing to each ERO and their various weightings should be reflected appropriately. The two limitations presented above need to be addressed in future research.

Conclusions
This study reviewed several research works related to resilience in a wide variety of domains within the last two decades, including general domains, healthcare, transportation, infrastructure, process plants, business, and other domains, leading to the identification of several organizational resilience factors. This study applied three additional procedures: RAG filtering, ERO applicability assessment, and merger/re-classification. As a result, 52 factors (level 2 and level 3) contributing to the resilience of EROs in NPPs were proposed. The identified contributing factors would be useful in measuring the resilience of EROs in NPPs, e.g., to indicate the robustness of the NPP emergency response system. Moreover, this study can be used as a technical basis for improving the robustness of radiation emergency response strategies. A subsequent study will include expert judgments to test the identified resilience factors, and an analysis of the interactions between the factors. Furthermore, the various weightings of the factors as contributors to the resilience of EROs in NPPs should be determined before practical application. As the reliability of EROs plays a very important role in incident response, the results from this study may indicate a direction for overall NPP organizational safety assessments in the future.