Key Determinant Derivations for Information Technology Disaster Recovery Site Selection by the Multi-Criterion Decision Making Method

: Disaster recovery sites are an important mechanism in continuous IT system operations. Such mechanisms can sustain IT availability and reduce business losses during natural or human-made disasters. Concerning the cost and risk aspects, the IT disaster-recovery site selection problems are multi-criterion decision making (MCDM) problems in nature. For such problems, the decision aspects include the availability of the service, recovery time requirements, service performance, and more. The importance and complexities of IT disaster recovery sites increases with advances in IT and the categories of possible disasters. The modern IT disaster recovery site selection process requires further investigation. However, very few researchers tried to study related issues during past years based on the authors’ extremely limited knowledge. Thus, this paper aims to derive the aspects and criteria for evaluating and selecting a modern IT disaster recovery site. A hybrid MCDM framework consisting of the Decision Making Trial and Evaluation Laboratory (DEMATEL) and the Analytic Network Process (ANP) will be proposed to construct the complex influence relations between aspects as


Introduction
During the past two decades, organizations have become increasingly dependent on information technology (IT) to enhance business operations, facilitate management decision-making, and deploy business strategies [1,2].Therefore, IT system availability has become one of the most critical issues that attracted attention from both IT researchers and practitioners [1].In particular, some organizations (e.g., financial institutes, healthcare organizations, high volume online retailers, government departments, utility companies, etc.) require that an IT system operates continuously; those organizations cannot tolerate any failure [3].Thus, the increased demand for continuous operations of IT systems has created interest in disaster recovery sites (also called remote backup site) [3][4][5][6][7].An IT disaster recovery site is a second location at which back-ups are available to the main data center in case of primary site failure [3,8].IT disaster recovery sites would be able to restore data for the organization to ensure continuous operation, even in the presence of extensive failures that may render an entire site unoperational and for which local replication may be inadequate [9,10].
However, problems arising in disaster recovery site selection are practical, complicated, and usually involve massive subjectivities and uncertainties [11,12].One reason is that disaster recovery sites are often regarded as a "necessary evil", an overhead cost that most organizations would prefer to keep to the minimum instead of being regarded as a revenue-generating function only.Another reason is that disaster recovery site selection decisions are contingent and resource dependent, which may vary based on the available resources, their effects, and their technology.Therefore, the challenges of evaluation and selection of disaster recovery sites has already become the most important issues for public and private firms.
Due to the dominant role of disaster recovery mechanisms on the continuous operation of data centers, various scholars have recently started to investigate related issues.A few researchers explored criteria on the selection of data centers [9,13,14].However, based on the authors' very limited knowledge, very few or no research explored IT disaster recovery site selection in detail.In the real world, the criteria for evaluating a data center site or a disaster recovery site are different.One study on the evaluation and selection of disaster recovery sites in particular can help management make better decisions regarding their disaster recovery systems.Although some consulting firms (e.g., Cisco, RATH, BSI) proposed specific criteria for disaster recovery site selection, they fall short of the derivation procedure and the calculation of weights being associated with that criteria.
To fill the research gap, this study aims to define an analytic framework for IT disaster recovery site selections.Key decision factors for both public and private sectors were derived, and we explored the context of disaster recovery site selection and proposed an evaluation model for IT disaster recovery site(s).Based on knowledge in the related areas of disaster recovery, information technology, decision science, and international standard consultancies, the decision-making framework can be developed.
First off, this study will review literature regarding possible aspects that should be used in evaluating and selecting a disaster recovery site.Possible alternatives (or the disaster recovery sites) will also be proposed according the situation of the target organization being studied.These aspects and criteria will be confirmed by experts through the focus group interviews.Then, a hybrid MCDM model combined with the Decision Making Trial and Evaluation Laboratory (DEMATEL) and the Analytic Network Process (ANP) will be introduced.The DEMATEL will be used to construct the decision-making framework.The ANP will then be introduced for the weights corresponding to each aspect and criterion.The performance scores versus each criterion will then be graded by the experts.The weighted performance score versus each alternative will be aggregated by the simple weighted method.An empirical study on evaluating and selecting a disaster recovery site for a Taiwanese research institute will be used to demonstrate the feasibility of the proposed framework.
The rest of this paper is organized as follows: a literature review on disaster recovery and site section is presented in Section 2. The research methods of the multi-criteria decision-making framework will be presented in Section 3. The empirical study for evaluating and selecting a disaster recovery site for a leading Taiwanese research institute will be presented in Section 4. Advances in management practices and the comparisons between the empirical study results versus past research results will be provided in the Discussion Section.Finally, Section 6 summarizes the results and concludes the paper.

Literature Review
In order to review the latest disaster recovery site selection researches and to construct an analytic framework accordingly, related literature was reviewed and summarized below.This literature review focuses on IT disaster recovery and site selections, with a focus on past studies regarding the evaluation criteria for the selection of IT disaster recovery sites.

Disaster Recovery
A disaster, by definition, is a destructive event that results in victims; further, discrepancies arise between their number and the treatment capacity.An accident is similar to a disaster, but refers to an event without victims [15,16].The impact of disasters can further be classified into the direct and indirect ones.The direct impact of disasters include human injuries, facilities damages, and equipment losses [17,18].Indirect impact includes business operation interruptions, property value reductions, stock market fluctuations, and possible social and environmental effects [18].Disasters may occur as the result of unpredictable conditions or underestimated risks, anything from natural disaster, a technical accident, or a system failure [19].Usually, controlling and preventing disasters is very difficult.Thus, the ability of an organization to recover from a disaster is very critical.
Disaster recovery is defined as the process of recovering an organization or a project affected by some specific damage(s) to its state before the disaster [19].Early disaster recovery literature concerns various types and impacts of natural disasters (i.e., Loma Prieta Earthquake in 1989, Great Midwest Floods in 1993, Northridge Earthquake in 1994, Red River Flood in 1997, etc.) and their different influences on individuals, households, communities, and organizations from environmental and social viewpoints [20][21][22][23][24][25][26].Over the past two decades, an increasing number of publications and empirical studies have started to focus on the organizational capabilities to respond and recover after disasters [27,28].Apparently, the emphasis has been changed from the influences of natural disasters on human injuries directly to the potentially catastrophic impact on the business [19,29].In particular, researchers have started to emphasize the assessment of information infrastructures for disaster recovery after the 11 September 2001 terroristic attacks [30,31].During the past decade, as information infrastructure availability has become the critical issue for both IT researchers and practitioners, the importance of IT disaster recovery can hardly be overestimated [1].

Disaster Recovery and the Modern Business Perspective
Traditionally, the disaster recovery was recognized as a set of procedures to recover and protect IT infrastructure when computer(s) shut down accidentally.According to Jon William Toigo, the author of Disaster Recovery Planning, 15 or 20 years ago a disaster recovery plan might consist of powering down a mainframe and other computers, disassembling components, and drying circuit boards in the parking lot with a hair dryer [32].Thus, the disaster recovery problem was often regarded as the problem belonging to the IT organizations.This recognition assumes that the disaster recovery problem can simply be resolved by constructing good backup system(s) for computers or computer networks.However, this assumption is not realistic as IT systems are gaining greater prominence in the overall structure of most corporations.Except for the traditional IT context, the disaster recover problems are now related to business operations.The seriousness of this problem is supported by a research from Faulkner Information Services, which found that 50% of companies that lose their data due to disasters go out of business within 24 months [33].Further, the number of organizations that rely on computerized systems to perform daily operations and assist in the decision making process has grown at a rapid pace recently and still continues to grow [34].Therefore, the disaster recovery problem should be reviewed further from the modern business perspective.
Disasters are significant outages with a greater critical impact to the modern business [10].Nowadays, there are many more threats except for the natural hazards.Snedaker [35] summarized novel disasters to include human-caused hazards, accidents and technological hazards, electronic data threats, etc. Disasters may strike at any moment in any location [36].Therefore, disaster recovery is becoming increasingly complicated.
As observed by Torabi and Mansouri [37], businesses are increasingly subject to disruptions; it is almost impossible to predict their nature, time and extent.So, the concept of organizational resilience is attracting growing attention among academicians and practitioners [37].The organizational resilience enquires organizations to develop effective plans for both short-term resuming (i.e., business continuity plans) and long-term restoration (i.e., disaster recovery plans) of their disrupted operations following disruptive events [38].However, lack of proactive business continuity and disaster recovery planning may lead to loss of reputation and market share, customer service and business process failure, regulatory liability and increased resuming and restoring times [37,[39][40][41].
Most companies' IT systems are too complex to be recovered by using the traditional approach [32,42].Being prepared for disruptive events requires proactive planning of internal and external resources of the organization so that it can cope with disasters effectively and efficiently [37].Disaster recovery in the modern age is a detailed, step-by-step course of actions for quickly recovering after a natural or manmade disaster; the details may vary depending on the business needs, and can be developed inhouse or purchased as a service [42].In general, the disaster recovery has been re-focused on a broader scope, from the traditional IT recovery and protection to the modern organizational processes which affect business operations.Organizations need a proactive approach equipped with a decision support framework to protect themselves against the outcomes of disruptive events [37].

IT Disaster Recovery
Data centers (DC) have become critical infrastructures for most modern organizations in information society, as eCommerce, cloud-based storage, and a variety of more generic online services are introduced and adopted.Such a trend has become especially significant in the era of cloud-based computation.However, the DCs can easily be affected by a small isolated disaster.Therefore, more and more researchers advocate for IT disaster recovery, the process by which computer systems and the associated infrastructure(s) can be recovered after a service disruption [4].
An IT disaster recovery site is a backup data center that aims to replicate data to a remote location and synchronize those data with the primary site in case of primary site failure [3,43].IT disaster recovery sites would be able to restore data and keep an organization's IT system operating during or after a disaster [3,9,10].Since such IT disaster recovery problems happen frequently in the real world, a majority of previous studies were based on empirical studies by consultants or IT companies.Most real world organizations established disaster recovery sites based on the international standards proposed in the International Standard Organization (ISO), International Electro-technical Commission (IEC), International Telecommunication Union (ITU) and national regulations.
The majority of research on IT disaster recovery focused on process or storage technology [8,44], but very few focused on IT disaster recovery site selection, which usually involves multi-disciplinary knowledge such as massive IT, business process, and decision subjectivities [11,12].For instance, Sembiring and Siregar [3] described risk factors related to information technology, IT organization, and business process, which influence disaster recovery site evaluations and selections from the enterprise architecture perspective.The work by Sembiring and Siregar aims to decide the appropriate disaster recovery DC standard levels (called 1-4 tier) instead of site selection [3].

Site Selection
Site selection problems have been studied widely and can be found everywhere in the real-world governmental, industrial, or firm level management decision problems.For example, Garcia [45] studied the agricultural warehouse site selection failure, in which an agricultural warehouse localization error could drive business to bankruptcy [45].Rikalovic [46] discovered that new industrial park site selections are very critical to the successes of the industrial innovation systems [46].Pereira (2014) found that the power station and the carbon dioxide site selection problems are the most important when it comes to national economy and security [47].
Various groups have worked on site selection problems, and opinions differ as to "where is a suitable site" and "how to select a site".Some scholars advocated that selection of a suitable site requires considerations of various aspects and criteria, especially the facility location.For example, Wang et al. [48] summarized environment and economic factors, including price and distance and constructed a hierarchical decision-making framework for solving the solid waste landfill site-selection problem.Covas et al. [14] argued that considerations for evaluating a new data center location site from an energy efficiency perspective, mainly because data centers need a large amount of power for computing equipment and their infrastructure.Other researchers put more emphasis on the site selection procedure and treat site selection as the first step of a comprehensive decision making process [46,49].A number of studies are based on different decision making methods for evaluating ideal locations.These methods include expanding the classification or scoring methods [50], the Analytic Hierarchy Process (AHP) [51] and the Linear programming (LP) approach [52].

International Standards of Disaster Recovery
According to Herbane et al. [39], many organizations adopted/installed business continuation management due to avoid turning away customers, configuration resilience, or obligation [39].For these reasons, obligation is the most significant factor influencing the disaster recovery site selection.Some organizations (such as utilities, telecommunications, public sector, financial institutions, health care, etc.) have disaster recovery sites or, broadly speaking, a BCM due to legal responsibility; such disaster recovery sites need to be certified by international standards or national regulations [53].Such international standards being related to IT disaster recovery include ISO, IEC and IUT, as summarized in Table 1.Some standards serve as a high-level framework or guideline of business continuation and information security management, such as ISO 22301: 2012, ISO/IEC 27002: 2013, etc.Others focus on Disaster Management, Disaster response policy, and more detailed technology, such as ISO 27031, and ISO 24762.
In some counties, disaster recovery sites are regulated by national government.Such governments defined the disaster recovery site selection guidelines (refer to Table 2).For example, according to the 2003 Information Security Management Act (FISMA) in U.S., critical information assets need to be protected with remote backup and information security plans [54].The U.S. National Institute of Standards and Technology (NIST) enacted regulations of the SP 800-34 in 2006, which requires high and medium level public sectors to implement the Business Impact Analysis (BIA) process and set up disaster recovery sites.The main emphasis on disaster recovery site selection is to assure backup data confidentiality, procedure integrity and availability [55].In contrast to the well-regulated public sector organizations, a majority of private firms did not define BC management or disaster recovery plans [4].However, as a result of the 11 September terrorist attacks, the U.S. Department of Homeland Security is developing a "Voluntary Private Sector Preparedness Accreditation & Certification Program" [4].This voluntary program proposes a guideline for certifications for businesses, not-for-profit corporations, hospitals, stadiums, universities and other entities.This program promotes private sector entities to seriously consider certification of their disaster recovery systems.
The above mentioned site selection studies demonstrated that no specific approach for site selections were available.Most scholars agree that the site selection decision making frameworks vary for different problems and no one site selection plan was proposed to deal with unpredicted phenomena [56,57].To date, no systematic studies have been carried out concerning disaster recovery site selection for data centers.
Based on the very limited knowledge of previous studies, there is a significant research gap between the availability of research work versus the actual needs for the analytic frameworks for IT disaster recovery site selections.Therefore, this paper attempts to investigate disaster recovery site selection criteria, propose an analytic framework, and verify the analytic framework by an empirical study case.Disaster recovery site selection problems include a complex array of evaluation aspects and criteria.Such aspects and criteria always include economic, technical, and environmental and risk management issues, which may result in conflicting objectives [46].This study aims to render a new study by proposing a hybrid MCDM framework, combining aspects from international standards and regulations and summarize the technical and business management related aspects and criteria.The analytic framework can serve as the basis for evaluating and selecting disaster recovery sites for BC by IT managers and researchers.This study is among the very few studies in this field combine international standards, regulations and enterprise architecture.
Furthermore, as disaster recovery related standards and regulations have been developed in the developed countries, such as US, UK and Japan, it is difficult for small and medium enterprises in developing countries to adopt such standards and regulations.The practical cases of this study are from Taiwan and attempt to provide guidelines for disaster recovery sites in developing countries.

Analytical Framework and Methods
In this Section, an MCDM framework for a disaster recovery site selection will be developed.Decision criteria will first be summarized based on literature review results.The literature has been collected based on academia journal papers, international standards, national regulations and others as needed to develop a nation's disaster recovery principle.Possible criteria for evaluating a disaster recovery site are summarized based on the literature review results.Then, the focus-group method is introduced to confirm the criteria being derived.The initial weight versus each criterion will be calculated by the ANP.Based on the criteria and weights being derived, a decision making framework for a disaster recovery site selection can be established.In summary, this decision making framework consists of three major phases: (1) deriving determinants using the focus group method; (2) constructing a network relation map among determinants by using the DEMATEL; (3) calculating the weight versus every determinant by using the ANP based on the NRM being constructed in the phase (2).The flowchart is shown in the following Figure 1.Meanwhile, details of the methods will be introduced in the following subsections and in the Appendices A and B.

Focus Group Method
The focus group method is believed to have originated in the USA and is most usually attributed to the sociologist Robert Merton [58].The term "focus group" is generally assumed to have derived from the focused interview developed by Merton and his colleagues at Columbia University during the 1940s, when investigating audience reactions to radio programs [59].During the past few decades, the focus group method has emerged from a number of different disciplinary locations including anthropology, sociology, psychology, education, and advertising, all of which provide different theoretical perspectives on focus group research today [58].The focus-group is a powerful qualitative research tool which has widely been applied in various academic fields during the past decades.
Using a focus-group method in research is to acquire as much information as possible from a group of experts on a given topic.Such method can be accomplished by prompting the group with pre-specified topics, allowing the discussion to evolve around these open-ended questions, and facilitating interaction among the participants.The focus group process allows participants to interject their own observations and knowledge while also feeding off of the ideas of other participants The focus group further allows a researcher to extract expertise and insights from the participants [60,61].
In the focus group meetings, the participants received feedback about the first round panel results; the meeting was chaired by a consultant pediatrician who had not been previously involved as a participant in the design of the study.The chair person was asked to facilitate discussions and to allow all participants to speak.All participants were encouraged to contribute to the discussion, which was recorded with the consent of the participants.
Focus groups have been used frequently in IT-related studies to address emerging technology-driven phenomena [62].In this research, the focus group method was chosen because the IT disaster recovery site selection problem is an emerging phenomena in which the available data is very limited and unexplored.Focus groups allow us to derive experts' experiences and perspectives, while also exploring their beliefs and attitudes.

DEMATEL Method
The DEMATEL method, an MCDM approach originated from the Geneva Research Centre of the Battelle Memorial Institute [63], has typically served to address the question of "whether solving a problem can help solve another one?"The DEMATEL method can convert complex systems into a clear causal structure which simplifies the interrelationships among consideration factors; as a consequence it assists in locating the core issues and quantifies their causality and influence strength [64,65].In recent years, the DEMATEL method has been popularly applied in diverse fields including decision-making [66,67], technology innovation [68,69], knowledge management [70], operations research [71], marketing and consumer behavior [72], and competence and performance evaluation [73], etc.As the experts of disaster recovery site selection are understandably limited and our research goal is to identify the causation and influence strengths of the consideration factors, we employed the quantitative DEMATEL method in this research.The detailed procedure of the DEMATEL method will be introduced in Appendix A.

The ANP Method
The ANP method, developed by Saaty [74], provides a general framework to deal with decisions without making assumptions about the independence of higher-level elements from lower level elements and about the independence of the elements within a level as in a hierarchy [75].ANP is a flexible MCDM approach that can be used to imitate human thinking process resolving complex decision-making problems with some impact factors through analyzing, estimation and synthesizing processes, and so on.The relative importance of the factors can be confirmed through pair-wise comparing each element in the corresponding level and calibrating them on the numerical scale and sequences about the relative importance of the decision-making factors can be determined by synthesizing the judgment of domain experts.ANP method has been widely applied to site selection decision making, such as: shopping center selection [76], landfill site selection [77] and so on.Compared with traditional MCDM methods, e.g., AHP (Analytic Hierarchy Process)-which usually assume the independence between criteria, ANP is a more effective tool for dealing with dependence in feedback in the real world [74,78].ANP structures a decision problem into a hierarchy as a network.The detailed procedure of the ANP method will be introduced in Appendix B.

Using the Hybrid MCDM Method in Evaluation and Selection of a Disaster Recovery Site for the Computing Facilities of a Taiwanese Research Institute
Taiwan is ranked as the world's 12th economy out of 142 listed in global competitiveness rankings [79].Reflecting on the fact that Taiwanese companies have been active in the information and communication technology sphere for many years, Taiwanese ICT firms now possess first-class hardware design and software development capabilities, flexible production management capabilities, strong global logistics capabilities, and many years of experience in collaborating with leading international brands [80].Taiwan has made significant progress in research and development of data centers.Therefore, the disaster recovery site is the challenge being faced by both public and private sectors.Based on the experiences from other large landscape countries like Japan and the U.S., the Taiwanese disaster recovery system can be viewed as a different strategy-more flexible disaster recovery guidelines for different sectors.The disaster recovery plan in Taiwan is focused on IT solutions, such as Internet Virtual Private Network, disk, and backup type and so on.This section reviews an empirical study based on the selection of a disaster recovery site for the computing facilities of a Taiwanese research institute.
When considering geographic location, Taiwan is an island facing frequent risks from natural disasters [81].In 2005, the World Bank report titled "Natural Disaster Hot Spots-A Global Risk Analysis" indicated that Taiwan might be the most vulnerable area to natural hazards on Earth; 73% of the land and population are exposed to three or more hazards.The five major natural hazards confronting Taiwan include typhoons, earthquakes, landslides, floods, and debris flow [82].Such natural hazards have caused significant economic losses.According to the statistics being provided by Hale and Moberg [20], typhoons have resulted in annual economic losses of about $667 million USD in average [20], with an average of 3.6 typhoons per year.In addition to natural disasters, Taiwan is also one of the most targeted economies for malware [83].Combined, these natural and human made disasters stimulated governmental actions in designing an emergency management mechanism and disaster recovery systems and technologies to reduce the negative impacts of such disasters.
In order to construct disaster recovery sites for the Taiwanese data centers, this study aims to define a decision-making framework based on MCDM methods.All the possible criteria for evaluating DR sites will be derived based on experts' opinions.The experts were selected from the participators of the Taiwanese task force for defining "The Guidelines on Disaster Recovery Mechanism of IT Data Center".The task force was initiated by the National Information and Communication Security Taskforce (NICST) of the Executive Yuan, Taiwan.This project is the first official one for defining the reference guidelines for evaluating the disaster recovery mechanisms in public or private sectors.The experts being selected include the 5 IT managers who are responsible for disaster recovery decisions of the Taiwanese Critical Information Infrastructure (CII), 4 IT consultants who are responsible for the international standard definitions of business continues management, 2 IT researchers who work in data centers, and 1 government officers who were responsible for information security policy.All the experts are with more than 5 years of work experiences in the related fields of business continues management and disaster recovery plan definition.
After confirming the qualification of experts, the direct relation/influence matrices will be derived.The possible aspects and criteria for evaluating disaster recovery sites will first be derived based on comprehensive literature review results.Government and industry standards and existing enterprise IT architectures were considered.Then, experts were invited to provide their opinions.Details regarding every aspect and criteria were derived based on the experts' opinions.The decision will be structured based on the aspects and criteria being derived by using the DEMATEL method.Finally, the key determinants for DR sites will be derived using the ANP method.These detailed procedures will be demonstrated in the following sub-sections.

Aspects and Criteria Derivations by Using the Focus Group Method
At first, the focus-group method being introduced in Section 3.1 was introduced to derive the opinions of 12 Taiwanese experts.The focus group meeting was chaired by a consultant pediatrician who had not been previously involved as a participant in the design of the study.The chair person was asked to facilitate discussions and to allow all participants to speak.All participants were encouraged to contribute to the discussion, which was recorded with the consent of the participants.Based on the results being concluded by the experts in the two iterations of focus groups, five aspects and twenty-two possible criteria for evaluating and selecting a disaster recovery site were derived and demonstrated in the following Table 3. Table 3. Candidate aspects and criteria for evaluating disaster recovery sites.

Aspects Criteria Descriptions
Location and Infrastructure (A) Natural Disaster (a 1 ) [35] An appropriate disaster recovery site should minimize influences by natural disasters.As summarized by Snedaker [35], the natural hazards can be classified into cold weather or warm weather related hazards and geological hazards.
Distance From Primary Site (a 3 ) [84] The distance between the primary and backup recovery sites depends on the risk assessment; the recovery site must be far enough away so that the same catastrophe does not strike both sites [84].
Transportation (a 4 ) [85] The transport network including roads, airports, port, and railways provides essential access to available resources needed for a country's rapid and successful recovery [85,86].The transport system is critical during a natural disaster due to its pivotal role in resourcing recovery.The high cost of resource transportation and lack of transport alternatives were major barriers to post-disaster reconstruction.
Electricity and Cooling (a 5 ) [84,87] The disaster recovery site should have stable power and a cooling system to prevent power outages and system shut down.The major consideration from the aspect of electricity include: monitor the line and filter out spikes, provision of additional power in case of a brownout or partial outage, provision of sufficient temporary power in case of a total outage, and ensure the transition from normal power supply to emergency power supply without loss of service to critical devices [84].Meanwhile, as the amount of heat that newer equipment discharges per square foot of space, cooling equipment are becoming daily important [87].
Detection and Monitoring (a 6 ) [88] The management of a computer security system involves intrusion detection and monitoring of the entire enterprise's computers [88].The disaster recovery site's building should have fast detection, monitoring alarm and operate equipment and design.
IT System Availability (B) Backup Strategies (b 1 ) [84] Proper backups of critical data can survive the organization from a disaster.Effective backups that completely protect critical data require thorough planning.According to Wallace [84], the backup strategies include full system backup, incremental backup, and differential backup.
Backup Servers (b 2 ) [89] Back up in virtual servers, physical servers, and cloud servers.Any backup solution must maintain the transactional integrity of the data so that, when the data is restored, it is left in a transactionally consistent state [89].Recovery Point Objective describes the acceptable amount of data loss measured in time; the Recovery Point Objective is the point in time to which data must be recovered as defined by the organization [92].
Recovery Time Objective (c 2 ) [92] The Recover Time Objective is the duration of time and a service level within which a business process must be restored after a disaster (or disruption) in order to avoid unacceptable consequences associated with a break in business continuity Testing and Exercises (c 3 ) [92] The purpose of testing is to achieve organizational acceptance that the business continuity solution satisfies the organization's recovery requirements.Testing may include: crisis command team call-out testing, technical swing test from primary to secondary work locations, technical swing test from secondary to primary work locations, application test, and the business process test.Three types of exercise can be employed when testing the business continuity plan: simple, medium and complex exercises.

Readiness exercises (D)
Education and Training (d 1 ) [93] Regular training and education programs for disaster recoveries, which include operations, technology, information security, and DRP processes.According to Smith (2012), training, education, and outreach initiatives; and the funding needed to implement them-can play an important but often overlooked role in recovery [93].
Disaster Recovery Work Area (d 2 ) [87] Establish an area where critical IT workers can work during and after the recovery operation [87].The physical precinct in a disaster recovery center, including operation area, equipment handling, and testing area.The project management plan of a disaster recovery site, such as the plan-do-check-act continuous quality improvement model.Elements of project success include: executive support, user involvement, experienced project manager, clearly defined project objectives, clearly defined project requirements, clearly defined scope, shorter schedule, multiple milestones, and clearly defined project management process [35].

Information Security Management Procedure (e 2 ) [35]
A disaster recovery site should be certified by information Security standards and renewed regularly, such as ISMS, BS 7799, etc.As summarized by Snedaker [35], the information security management should be based on an analysis of the most recent, more developed information security statutes, and responsibility for compliance increasingly rests with the BoD or CEO.The development and maintenance of a process-oriented written information security program (WISP) is critical to the ability of a company to meet its legal obligations as it relates to the management of information security.The legal standard dictates minimum requirements your WISP must address, limited to: (1) specific security measures; (2) specific requirements for third-party service provider arrangements; and (3) specific requirements regarding the education of the WISP.

Disaster Recovery
Procedure (e 3 ) [94] Disaster management is the discipline of dealing with and avoiding risks; the availability of a good disaster recovery procedure and emergency response plan in place [94] is essential for successful; disaster recovery.A disaster recovery site should have well-documented process of disaster recovery plan to recover and protect a business IT infrastructure in the event of a disaster.
Top Manager's Supporting (e 4 ) [35] Top manager's supports and commitment for DRP operations, including allocating time and resources required in the disaster recovery.Executive support for any IT project is typically the number one success factor [35].
Resources (e 5 ) [85] In disaster recovery projects, the operational environment is uncertain, complex, and dynamic.The "business as usual" way of managing resources may not be fully applicable.Evidence shows that post-disaster recovery projects are more likely to suffer resource shortages and supply disruption.These resourcing problems contribute to final recovery project failures.

Decision Structuring by Using the DEMATEL
Because the relationships between the twenty-two possible determinants summarized by the focus group methods in Section 4.1 seem to be too complicated to be analyzed, the DEMATEL method being introduced in Section 3.2 is introduced to derive the causal relationships between the criteria.Twelve Taiwanese IT experts have actively participated in disaster recovery site evaluations and selections of public or private organizations.
Based on the experts' opinions, the direct relation/influence matrices for the influence relationships between the five aspects (refer Table 3) and the influence relations between the criteria belonging to each aspect were derived.After that, the direct relation/influence matrices were normalized based on Equation (A1) in the Appendix A. Then, the normalized direct relation/influence matrices are derived by based on Equations (A2) and (A3).Finally, the total relation matrices were derived based on Equation (A4).The detailed calculation procedures are demonstrated in Appendix C. The causal diagram consisting of the influence relationships between the aspects and those influence relationships between each criterion belonging to each aspect are demonstrated in Figure 2.
Because of the complexity of the relationships, the threshold value is defined as 68% (1 sigma).Figure 2 demonstrates the causal diagram mapping in terms of ri + ci (x-axis) and ri − ci (y-axis).The prominence and relation data set of the five aspects and the 22 detail-level criteria, respectively.The direction of the arrows show the direction of influences.In the central influence diagram in Figure 2 demonstrates the influence relations between aspects, "Business continuity (C)" with the highest ri + ci value plays the central role, indicating this aspect's high degree of influence in the overall relationships.At the same time, the ri − ci value represents the causal relationship between aspects."Business continuity (C)" also has the lowest ri − ci value and receives the strongest influence from the others, and thus is the effect.On the other end, "location and infrastructure (A)" has the highest ri − ci value and dispatches strong influence on the others, and thus is the cause.

Weights of the Decision Framework
Based on the influence networks being derived in Section 4.2, the ANP was introduced to derive weights versus each criteria based on opinions being provided by the 12 experts.First, the pairwise comparison results were derived.Then, the limited super matrices were calculated by using the Super Decisions [95].The detailed ANP derivation processes for the weights are demonstrated in the Appendix B. The results are demonstrated in Table 4. Weights corresponding to each aspect and criteria can thus be derived based on the weighted super matrices.The detailed calculation procedures are demonstrated in Appendix D.
From the weights associated with the aspects (Table 4), the top three important aspects can be ranked as ( 1

Discussion
In order to resolve the complex IT disaster recovery site selection problem, this study has established the IT disaster recovery site selection analytic framework by considering aspects such as the disaster recovery site location, IT infrastructure, disaster recovery objectives, etc.The critical criteria have been derived accordingly.An empirical study based on the real case of a Taiwanese government's evaluation and selection of the disaster recovery site have demonstrated the feasibility of the proposed analytic framework.In Section 4, the key determinants for IT disaster recovery site selections have been derived by using the proposed hybrid MCDM framework.In this section, the analytic results will be discussed from both aspects of managerial implications in IT disaster recovery site selections, as well as the managerial implications from sustainable IT operations.

Managerial Implications from the Evaluation and Selection of IT Disaster Recovery Sites
Based on the analytic results being demonstrated in Table 4, we found that "IT System Availability (B)" is the most critical aspect in disaster recovery site selection.The aspect, IT System Availability, is tied closely with overall business continuity management procedures.Business continuity management can be separated into two distinct parts: (1) IT system design and implementation from the IT department and (2) business analysis from IT users' departments (e.g., R&D, manufacturing, etc.).The IT system design and implementation phase identifies the most cost-effective disaster recovery solutions that meet the business's most critical recovery point objective (c1) and recovery time objective (c2) requirements, which are the major job responsibilities of an IT department.Therefore, the mechanisms being used by disaster recovery vary, depending on the kind of IT department.Apparently, IT System Availability is the first and the most critical aspect of any successful disaster recovery process for IT managers.Therefore, the analytic result is consistent with the actual situation.
Therefore, in comparison with the IT System Availability (B), the "Location and Infrastructure (A)" is only ranked the second important aspect for an IT disaster recovery site selection problem.Common sense says that the major concerns of a site selection will be the physical location risk assessment.However, "Location and Infrastructure (A)" was ranked as the second important aspect from the viewpoint of the Taiwanese experts.For small economies like Taiwan, the area is very limited.The distance from the northernmost part to the southernmost point is only about 400 kilometers.The easiest solution is to position a disaster recovery site in a neighboring country with compatible laws and regulations.However, in the case of public sector data, the above solution is not feasible.Furthermore, from the aspect of costs, construction and maintenance of a foreign site are much more expensive than those of a primary building.Therefore, for a small economy, the "IT System Availability" aspect is much more important and reality than the geographic location.
"Disaster Recovery Objectives (C)" is the third most critical aspect in evaluating disaster recovery sites.This result is consistent with the argument by Reich & Benbasat [96] that the establishment of strong alignments between information technology (IT) and organizational objectives has consistently been reported as one of the key concerns of information systems managers.The objective of disaster recovery site selection is not for risk avoidance only; instead, the objective(s) should be aligned with business continuity strategies, such as recovery point, recovery time, etc.As the concept of business continuity has emerged in industries as a systematic process to counter the effects of crises and interruptions, such value preservation is central to the business strategy and business continuity relationships [39].Thus, business continuity is one of the key concerns of information system design [96].
Furthermore, according to the analytic results being derived by DEMATEL (Figure 2), the "Disaster Recovery Objectives (C)" is the main effect aspect to receive influences from the other aspects.Especially, the "Disaster Recovery Objectives (C)" depends highly on "Location and Infrastructure (A)", "IT System Availability (B)" and "Disaster Readiness Exercises (D)".People's common sense is that once the Disaster Recovery Objectives for an IT application have been defined, business continuity managers can decide which disaster recovery technologies are most suitable for such applications.For example, if the recovery time objective for a given IT application is one hour, redundant data backup on external hard drives will be the best alternative.If the recovery time objective is five days, then the tapes, compact disks, or offsite disk storage may be more cost-effective alternatives.Our research results are different.There are two possible explanations for the analytic results.At first, an organization will define disaster recovery objectives after the evaluation of physical risks from location and IT system based on the business continuity management processes has been regulated by international standards, like BS 25999 [97].Thus, location and IT system will influence the Disaster Recovery Objectives.Second, the characteristics of an economy contribute to the analytic results.Disaster recovery site investment is undoubtedly one of the most difficult tasks where IT managers cannot evaluate the cost performance.Due to limitations of performance uncertainty, most IT managers prefer to define disaster recovery objectives under constraints, such as physical location and IT System Availability to control the cost rather than performance.
In the following subsections, the managerial implications for the criteria belonging to each aspect will be discussed further.

Criteria in the IT System Availability Aspect
Based on the analytic results being demonstrated in Table 4, the Backup Strategies ( 1b ) is the most important criteria for a disaster recovery site selection.The major concern here is the data change or new data acquisition frequencies.Data systems that acquire new files regularly (e.g., clinical chemistry or chromatography data systems) or where data are extensively manipulated require higher data backup frequencies than those systems with fewer changes.Various alternatives are available in developing backup strategies.The full backup refers to the regular backup, which is a complete tape copy of the whole system, applications, and user files.Incremental backups refers to a regular, but partial, copy of the system, applications, and user files to standard media; the incremental backups are identified by the backup profile.The differential backup means a regular, but partial, copy of the files that have been changed since the last full backup.The various backup strategies influence both the system architecture and the internet bandwidth, which account for a majority of the cost structure of an IT system in a data center.Thus, IT managers prioritize the Backup Strategies as the most important concern.These backup strategies are usually defined based on customers' applications and the goals, which include the backup operation time, cost, etc. IT systems being defined based on different backup strategies operate differently.Various backup strategies include the full backup, the incremental backup, as well as the differential backup, a mixture of the former two, are the most popular ones [84].For the full backup strategy, all the files should be synchronized to the backup server; thus, more storage space and time are required.For the incremental backup strategy, the files being added or modified should be recorded into a temp file so as to accelerate the backup speed.Such incremental backup strategy can achieve better space performance [98].The backup strategies can be extremely time consuming and are dependent on tape speed, network availability, tape numbers being involved, etc. [99].Some scholars further focus on a mixture of different backup strategies and minimizing the overall cost of the backups [100].
For a disaster recovery site, recovery operations require restorations from the backup image(s) for the full backup scenario or from multiple backup images and tapes for the incremental backup strategy or the mixture strategies.Thus, the availability of the network becomes a major concern risk.The finding is also consistent with the research results being derived by using the DEMATEL (refer Figure 2).Telecommunication Infrastructure ) is another issue to be considered.The major concern here is the allowance for backup time.For critical data, the intervals between data backups and the type of data backups being performed will be higher than the low-priority systems, where the data backup frequency can be lower.Possible options include system restoration within a working day (with little impact on the system performance) or the need for restoration at the earliest possibility.The recovery time will affect both the frequency and nature of the backup as well as the recovery schemes and the linkage with database transaction logging.These issues need to be considered when designing the backup and recovery process [101].Various approaches for system backup architecture are available: (1) hot or online backup, which takes place while the system is still operating and (2) cold or off-line backup which occurs when the system has been stopped and users have been logged out of the system.The cold backup is generally regarded as the safest backup strategy; the hot backup requires the system to be buffered while the backup occurs and the system must be updated after the backup is complete.The above options depend on system usage, data value and cost.For instance, if the data and system needs to operate during a disaster, then a hot backup of the system would be required.Alternatively, if the system was only required to be available 95% of the time, then a cold or off-line backup approach could be devised [101].Thus, Backup Strategies

Criteria in Location and Infrastructure Aspect
The analytic result also demonstrates that electricity and cooling ( 5 a ) and detection and monitoring ( 6 a ) are the most important criteria in location and infrastructure aspect of a disaster recovery site evaluation problem (Table 4).Rising power density and unpredictable disasters are the two major criteria for evaluating and selecting an IT disaster recovery site.For example, the blade servers have tremendously increased power densities and dramatically changed the power and cooling dynamics of the surrounding environments of a data center.Thus, electricity and cooling become the major source of risks and maintenance costs of data centers.Besides, disaster recovery sites being located far away from the primary computing facilities further highlight the need for automated monitoring; it is impractical and unreliable to have people physically present to check conditions such as temperature and humidity of electricity and cooling facilities .The fully automatic remote disaster recovery site requires reliable detection and monitoring systems in place to know what is going on.This paper addresses the importance of electricity and cooling ( 5a ) and detection and monitoring ( 6 a ) issues for disaster recovery site selections.The analytic results being derived by DEMATEL (Figure 2) also confirm the inter-relationships between electricity and cooling ( 5 a ) and detection and monitoring ( 6 a ).
Additionally, there is another point worth attention.The key issue of disaster recovery site selection is to find the ideal distance from the primary site.For instance, in 2002 and 2003, U.S. federal regulators had planned to require financial institutions to move their disaster recovery sites 200 or 300 miles away from primary sites.The Chinese government requires critical business to have disaster recovery site 1000 kilometers from primary sites.In previous studies, Pirkul [102] presents that location of emergency service facilities it is desirable to have primary and backup service facilities within a certain distance from every district [102]; Miyagawa [103] derived the joint distribution of the distance to the first and the second nearest facilities.However, in our research, there being no significant support showing that distance from primary site and disaster recovery site is a key issue.This criterion ranks 16th.
For the U.S. or China, the distance of a few hundred miles is never a problem; however, in small economies like Taiwan with the geographical size of only 400 kilometers, the hundred miles distance from the primary site is quite infeasible.The alternative solution is to position a disaster recovery site in a neighboring country.However, in many cases, such as health care data or public sector data, they could never put those systems and data in another country due to laws and regulations.The international standards, e.g., ISO 22301, BS 25999-2, or any one of the NIST SP 800 or ISO 27000 series standards also have similar regulations.Such standards regulated that a disaster recovery site must be located within a "safe distance" rather than a far away one.
Improving computer or storage system ability to recover from disasters and meet user defined disaster recovery objectives, organizations prefer to use more flexible IT system availability than physical location selection [5,104].For example, other studies discussed how cloud computing platforms are well suited for offering disaster recovery as a service due to their pay-as-you-go pricing model that can lower costs and their use of automated virtual platforms that can minimize the recovery time after a failure [105].Thus, this study demonstrated that the IT System Availability (such as hot site backup) is preferred to geographic location by small economies.

Criteria Belonging to the Disaster Recovery Objectives Aspect
The analytic results demonstrate that the recovery point objective ( 1 c ) and the recovery time objective ( 2 c ) are the most important criteria in the Disaster Recovery Objectives aspect (Table 4).The recovery point objective specifies how much data loss can be tolerated during some specific time period (i.e., minutes, hours, or days) [97].The recovery time objective is defined as the maximum tolerable shutdown time an IT-based business process can tolerate before the organization starts to suffer from unacceptable consequences.The recovery point objective and recovery point objective are determined by the business impact analysis (BIA), which differentiates critical (urgent) and non-critical (non-urgent) organizational functions/activities.A function may also be considered critical if such function is regulated by law.Thus, for firms or organizations belonging to some specific industries or sectors, such as telecommunications, banking, healthcare industries and public sectors, the business continuation should be assured and the recovery point objective ( 1 c ) and the recovery time objective ( 2 c ) should be close to zero, according to the government regulations.Those two criteria are also the major reasons why the disaster recovery sites are essential for such industries and sectors.In general, the recovery time and the recovery point are the most critical key performance indicators (KPI) for business continuity, and thus, sustainable operations.The work by Ning and Luo (2008) d )" rank as the 8th, 14th, and 17th, respectively, among all the criteria being summarized in Table 4.These criteria are related to the emergency response of the organization.Due to the very limited land area of Taiwan (35,980 km 2 ) and the exposition of the majority (70%) of Taiwanese lands and populations to three or more natural hazards, the best ways for organizations to recover from a disaster may be emphasizing more on responding from an emergency instead of preventing the possible disasters.Such emergency response activities are carried out in "Emergency Operation Center ( 3 d )" belonging to the disaster recovery sites.Therefore, according to Figure 2

Managerial Implications from Sustainable IT Operations
The IT disaster recovery site evaluation and selection problems have some intrinsic features that are tightly linked to a number of organizational concerns, including IT technology, risk management, and site selection.Most of the concerns are linked with customer's needs as well as laws or regulations being defined by the third parties or governments.Therefore, the IT disaster recovery site selection decision making problems include a wide range of criteria because the failure of IT disaster recovery may lead to either business losses or over investments for disaster recovery.
According to weights being demonstrated in Table 4, the "Location and Infrastructure (A)" and "Disaster Recovery Objectives (C)" are critical aspects which are as important as the "IT System Availability (B)" in a disaster recovery site selection problem for IT professionals.Such empirical findings provide several important managerial implications from the aspect of IT sustainable operations.Following, the managerial implications for both the changing roles of IT management from the aspect of sustainable operations as well as the disaster recovery objectives will be discussed.

The Changing Roles of IT Management for the Aspect of Sustainable Operations
The "Location and Infrastructure (A)" aspect often includes environment-related risks and disasters as well as the strategies to prevent such risks or disasters.Related issues can be the IT professionals' responsibility.However, the aspect "Disaster Recovery Objectives (C)" is required by business continuity operations, which links to users' needs.This is a critical issue from the viewpoint of organization behavior because IT professionals who respond to IT disaster recovery site selection decision seem to have emphasized both of them.
Albeit very complicated, organizations often assign such roles of decision making to the IT professionals.Such assignments indicate that the IT professionals belonging to the data centers should not only take care of the IT-related issues such as system downtime, but also negotiate the disaster recovery goals, including the recovery point objective ( 1 c ) and the recovery time objective ( 2 c ), with customers.Sometimes, IT professionals are even required to be in charge of the business continuity process so as to assure organizational operations can meet the disaster recovery goal.
Traditional IT professionals are responsible for IT-related issues; customer needs are usually ignored by the IT professionals.Our findings are different.In modern organizations, all the IT technical and logistical activities are carried out by IT professionals, who are responsible for ensuring the availability and functionality of information systems.
However, as IT disaster recovery issues are becoming more important for most of modern organizations, the IT teams are not responsible for the availability and functionality of information systems as usual.Modern IT teams have fiduciary responsibilities for the availability of missioncritical strategies [97].Therefore, IT professionals would place disaster recovery objectives as higher priority missions than functional operations.Therefore, our results are consistent with the argument by Herbane et al. [39].The roles of IT managers have been changing from emphasis on purely functional operations to strategic roles [39].

Recovery Point Objective versus Recovery Time Objective
Further, proactive risk management also requires deeper understanding of the key criteria for IT disaster recovery site selections.The "Disaster Recovery Objectives (C)" is the central role and has the highest degree of influence in the total influence relations being demonstrated in Figure 2. According to the "Disaster Recovery Objectives (C)" aspect, protecting data and maintaining uptime are almost equally important.However, prioritizing the recovery point objective ( 1 c ) and the recovery time objective ( 2 c ) is very critical for IT professionals in the real world because the architecture and cost for the recovery time oriented or the recovery point oriented disaster recovery sites are different.If an organization is recovery time oriented, the disaster recovery site should be application centric.The multiple system approach, e.g., clustering, should be adopted.Such an approach can monitor the system availability and can switch users immediately to a fully redundant hot-standby system whenever the primary system is unavailable [17].
However, if an organization is more recovery point oriented, the disaster recovery site should be data-centric.Feasible solutions include Redundant Array of Independent Disks (RAIDs), disk mirroring, journaling, and/or data vaulting [17].Therefore, based on the above arguments, the cost for a recovery time oriented disaster recovery site are higher than that of a recovery point oriented one.
While prioritizing the recovery point oriented or recovery time oriented approaches in a disaster recovery site problem, our research findings provide another viewpoint.Based on the empirical study results, the recovery time objective ( 2 c ) is more important than the recovery point objective ( 1 c ).
Albeit there can be data losses within a very short recover time objective, the computer system should be recovered at the earliest possibility.So, the nature of the disaster recovery site problem dominates the results.For some situations, organizations focus primarily on recovery time objectives.The telecommunication services (e.g., 3G/4G mobile telecommunications) are very typical examples which maintaining uptime may be more important than protecting data.Such service providers do not need to keep data for analysis and follow-ups.Instead, the most important concern for such firms is to maintain the ability to respond immediately to the current telecommunication service requirements.
Due to government regulations, some public sector organizations are also recovery time oriented.The critical information infrastructure (CII) such as the high speed railway system, the highway toll collection system, etc. are typical examples.Such systems usually adopt active-active configurations, synchronous replications, and network transactions, which can fulfill the zero recovery time and zero recovery point goals concurrently.For such cases, the recovery time objective should be prioritized over the recovery point objective.That is also the major reason why such organizations implement disaster recovery sites.Another possible reason for the implementation of disaster recovery sites may be attributed to the protections of the data integrity.Finally, the most important reason may due to IT managers' requirements to get back online at the earliest possibility, preferably without any noticeable interruption.
According to the analytic research results being derived by the DEMATEL (Figure 2), the recovery point objective is influenced by the recovery time objective.The result is consistent with the inclusion of the recovery point objectives by the recovery time objectives in ISO 22301:2012.Therefore, the research findings provide empirical evidences to clarify the influence relationships of the recovery time objective on the recovery point objective.
For most firms or organizations (e.g., most public sector institutes, research institutes, commercial banks, for the data protection purposes, the recovery point objectives are still more important than the recovery time objectives.For example, commercial banks concern more on data protections than maintaining very short uptime.Bank customers may feel inconvenient if their account information is unavailable for a very short system (e.g., ATM or online banks) down time.However, any loss of the deposit or payment records is not allowable.

Limitations and Future Research Possibilities
In this research, the experts were invited for deriving the aspects and criteria.Albeit some exiting researches have provided valuable insights for disaster recovery site selections, to the best of our knowledge, this research is the first attempt to derive factors for evaluating the disaster recovery site(s) via an MCDM based systematic approach, which consider the influence relationships between criteria.Since the disaster recovery site problem is highly knowledge-intensive, only the experts who are responsible for disaster recovery site evaluations and planning can provide valuable insights.However, the total number of available Taiwanese experts is less than 30.The statistical analysis based approaches including the exploratory and confirmatory factor analysis, which require more than 30 experts to fulfill the minimum sample number requirements, are not suitable.Therefore, the expert system based approach is more feasible and reasonable for this specific problem.
From this aspect, the future disaster recovery researches may include studies of larger economies by using the exploratory or confirmatory factor analyses based on theoretic models, e.g., the resource based theory, the resource dependence theory, etc.The results being derived based on experts' opinions by MCDM methods and the results being derived based on the statistical analyses (e.g., the covariance based structure equation model or the partial least squares structure equation model) can further be compared and studied.Further, other possible studies include the applications of the analytic framework in other economies or industries.Meanwhile, much more research is still required for differentiating the research point versus research time oriented disaster recovery needs.

Concluding Remarks
IT system availability and continuous operations have become increasingly important for most modern organizations.Therefore, the IT disaster recovery site evaluation and selection problems are very critical for such organizations.However, very few scholars tried to resolve the IT disaster recovery site selection problems.This paper attempts to derive the decision criteria.Like the main data center, choosing a disaster recovery site has a lot of criteria for making the best decision for a company and IT.However, a disaster recovery site is more focused on business continuity purposes and to be part of a company's strategy for providing uninterrupted services to its customers and end users.
The main contribution of this study is the definition of a disaster recovery site selection framework from both aspects of international standards and regulations as well as the enterprise IT infrastructure.This framework provides the guidelines for evaluating and selecting of IT disaster recovery site for disaster recovery service providers, organization IT managers and information/data owners, no matter from business or governmental organizations.
A major finding of this research is that the key aspect for disaster recovery site selection is business continuity as well as the location and the infrastructure aspect.The disaster recovery site selection is aligned with business continuity.Therefore, such disaster recovery site selections have played strategic roles, rather than purely IT functional decision.Such decisions cannot be from the IT or the facility viewpoint alone.
Furthermore, this empirical result shows that the recovery time is the most important business continuity management criteria for an organization's evaluation and selection of a disaster recovery site.This finding implies that the "recovery time" is the most important determinant, other than cost or quality.In practice, the priorities for IT recovery should be consistent with the priorities for recovery of business functions and processes.The recovery time for an IT resource should match the recovery time requirements for the business function or processes, which depend on the IT resources.Our findings are not in contradiction with those of empirical studies discussed the disaster recovery was cost-prohibitive.The result of this study may be of interest to organizations attempting to define disaster recovery site strategies and to researchers interested in disaster recovery aspects of IT facilities and technology.
Step 1: Build an initial direct-relation matrix Experts are asked to indicate the direct influence degree between factor i and factor j, as indicated by ij a , using a pair-wise comparison scale designated with five levels.The initial direct-relation matrix A is obtained by deriving the influence relationships between criteria through Equation (A1).
is denoted as the degree to which the ith objective affects the j th objective Step 2: Normalize the direct-relation matrix The normalized direct-relation matrix N is obtained through Equations (A2) and (A3).
Step 3: Build the total relation matrix T The total-relation matrix T is acquired by Equation (A4): ( ) where ε →∞, I is the identity matrix and Step 4: Compute the influence strength of the factors Aggregate the values of the rows and columns in matrix T to obtain a value i r and i c through the Equations (A5) and (A6) respectively.The i r represents the level of direct or indirect impacts on other factor, and i c represents the level to which it is affected by other factor:  − is positive, then the factor is a "cause factor", dispatching influence to the others.If i i r c − is negative, the factor is an "effect factor", receiving influence from others.The higher the value of i i r c − a factor has, the more influence it has on the other factors, and hence this factor is presumed to have a higher priority than the others.In other words, the lower the value of i i r c − a factor has, the greater its received influence from the other factors, and consequently, the lower the priority it is assumed to have.

Appendix B. ANP
The ANP is a coupling of two parts.The first consists of a control hierarchy or network of criteria and subcriteria that control the interactions.The second is a network of influences among the elements and clusters.The network varies from criterion to criterion and a different supermatrix of limiting influence is computed for each control criterion.Finally, each of these supermatrices is weighted by the priority of its control criterion and the results are synthesized through addition for all the control criteria [78].
Analysis of priorities in a system can be thought of in terms of a control hierarchy with dependence among its bottom-level alternatives arranged as a network as shown in Figure B1.Dependence can occur within the components and between them.A control hierarchy at the top may be replaced by a control network with dependence among its components, which are collections of elements whose functions derive from the synergy of their interaction and hence has a higher-order function not found in any single element.The criteria in the control hierarchy that are used for comparing the components are usually the major parent criteria whose subcriteria are used to compare the elements need to be more general than those of the elements because of the greater complexity of the components.
A network connects the components of a decision system.The system is made up of subsystems, with each subsystem made up of components, and each component made up of elements., , , e e e  . The influences of a given set of elements in a component on any element in the decision system are represented by a ratio scale priority vector derived from paired comparisons of the comparative importance of one criterion and another criterion with respect to the interests or preferences of the decision makers.This relative importance value can be determined using a scale of 1-9 to represent equal importance to extreme importance.The influence of elements in the network on other elements in that network can be represented in the following supermatrix: A typical entry Wij in the supermatrix is a principal eigenvector of the influence of the elements in the ith component of the network on an element in the jth component.Some of its entries may be zero corresponding to those elements that have no influence.
After forming the supermatrix, the weighted supermatrix is derived by transforming all columns sum to unity exactly.The weighted supermatrix is raised to limiting powers, such as Equation (B1) to get the global priority vector or called weights.In addition, if the supermatrix has the effect of cyclicity, the limiting supermatrix is not the only one.There are two or more limiting supermatrices in this situation, and the Cesaro sum would need to be calculated to get the priority.The Cesaro sum is formulated as follows.to calculate the average effect of the limiting supermatrix (i.e., the average priority weights) where Wj denotes the jth limiting supermatrix.Otherwise, the supermatrix would be raised to large powers to get the priority weights [107].

Figure 1 .
Figure 1.The flowchart of decision making framework.

Figure 2 .
Figure 2. The causal diagram of total relationship.Note: The threshold is set on the 68% of the total relations.

( 4 b
) plays a central role in Infrastructure Availability (B) aspect.The aspect further influences the Backup Strategies ( 1 b ) significantly.Thus, evaluations of the Backup Strategies and the cost for network services play dominant roles for DR site selections.The Backup System Architecture ( 3 b ( 1 b ) and Backup system architecture ( 3 b ) are the major concerned criteria in the IT System Availability aspect.These two factors are also the most important ones which influence the cost-performance of an overall IT disaster recovery solution.According to the results being derived by the DEMATEL (Figure 2), the Backup Strategies ( 1 b ) is the root cause and dispatches strong influence on Backup System Architecture ( 3 b ) and the Telecommunication Infrastructure ( 4 b ).Our research results provide strong evidence about the influence relationship.
, the "Education and Training ( 1 d )" and "Disaster Recovery Work Area ( 2 d )" criteria influences the operations of an emergency operation center ( 3 d ).

A6) Step 5 :
Produce a causal diagram A causal diagram can be acquired by mapping a data set ( i i r c + , i i r c − ).The value of i i r c + indicates the strength of influence.The higher the value of i i r c + a factor has, the more related it is to the other factors.Similarly, the value of i i r c − indicates the causal relationship between factors.If Figure B2demonstrates a typical network.Those components which no arrow enters are known as source network under each subcriterion of the control hierarchy components such as C1 and C2.Those from which no arrow leaves are known as sink component such as C5.Those components which arrows both enter and exit leave are known as transient components such as C3 and C4.In addition, C3 and C4 form a cycle of two components because they feed back and forth into each other.C2 and C4 have loops that connect them to themselves and are inner dependent.All other connections represent dependence between components which are thus known to be outer dependent.

Table 1 .
International standards for disaster recovery sites.

Table 2 .
National regulations for disaster recovery sites.

Table 3 .
[90].According toBrooks et al.fromIBM[90], backup system architecture and planning plays a critical role in the recovery phase as systems are rebuilt, applications are restored, data is recovered, and systems are put back online into production.Data recovery operations must work synergistically with overall disaster recovery operations.At the completion of the recovery phase, systems will be functioning to the extent determined in the plan.Beyond the critical recovery phases, less critical recovery operations may ensue.
[91]phone companies and long distance carriers offer a wide range of virtual network services; loss of virtual network services, like traditional long distance service, can severely impair a company's ability to conduct business[91].All carriers being present in the vicinity and their support and service models in place.e.g., different source of carriers to avoid unexpected interruption by one carrier.

Table 4 .
Weights versus the aspects and criteria.
is a typical example.Their work mapped the business criticality to disaster recovery readiness by assessing both the recovery point objective ( 1 [106]nd the recovery time objective ( 2 c ) to guarantee business continuity under the maximum tolerable period of disruption[106].The research findings of this work are consistent with the research results by Ning and Luo[106].

Table C1 .
The direct relation/influence matrix dimensions

Table C2 .
The direct relation/influence matrix a A of factors in A dimensions.

Table C3 .
The direct relation/influence matrix bA A of factors in B dimensions.

Table C4 .
The direct relation/influence matrix c A of factors in C dimensions.

Table C5 .
The direct relation/influence matrix d A of factors in D dimensions.

Table C6 .
The direct relation/influence matrix e A of factors in E dimensions.

Table C7 .
The normalized direct relation/influence matrix diemensions N of dimensions.

Table C8 .
The normalized direct relation/influence matrix a N of factors in A dimensions.

Table C9 .
The normalized direct relation/influence matrix b N of factors in B dimensions.

Table C10 .
The normalized direct relation/influence matrix c N of factors in C dimensions.

Table C11 .
The normalized direct relation/influence matrix d N of factors in D dimensions.

Table C12 .
The normalized direct relation/influence matrix e N of factors in E dimensions.

Table C13 .
Total relation matrix dim ensions T of dimensions.

Table C14 .
Total relation matrix a T of factors in dimension A.

Table C15 .
Total relation matrix b T of factors in dimension B.

Table
Total relation matrix c T of factors in dimension C.

Table C17 .
Total relation matrix d T of factors in dimensions D.

Table D6 .
The weighted supermatrix e W for deriving weights of factors in dimension E.

Table D7 .
The limted supermatrix lim a

Table D8 .
The limted supermatrix lim b

Table D9 .
The limted supermatrix lim c

Table D10 .
The limted supermatrix lim d

Table D11 .
The limted supermatrix lim e