Coming to Action: Operationalizing City Resilience

Abstract

There is an urgent need to build city resilience in order to face upcoming foreseen and unforeseen disasters more holistically, economically and collaboratively. Population trends mean that people are moving to urban areas and the traditional approach to crisis management is becoming obsolete as it is no longer able to deal with the new challenges that are emerging such as social dynamics or climate change. In this context, there are numerous studies and strategies that define how to build city resilience and consequently sustainable cities. However, decision-makers have trouble putting the knowledge in the studies and strategies into practice, as they find this information to be too abstract or far from their daily activities. More practical tools are needed to facilitate the operationalization of city resilience and familiarize decision makers with the concept. To that end, this paper presents both a qualitative and quantitative toolkit that enables decision makers to study, understand and train themselves to operationalize city resilience properly. This toolkit is composed of two complementary tools, namely the Resilience Maturity Model (RMM) and a serious game called City Resilience Dynamics (CRD). The paper also discusses the key points that led to a useful, trustworthy and flexible toolkit that decision-makers can use in building city resilience.

1. Introduction

Most of the world’s population now lives in cities, and it is forecasted that 60% of the population will live in urban areas by 2030 [1]. As cities continue to grow, the effects of a wide spectrum of disasters, ranging from acute shocks such as floods, droughts and earthquakes to chronic shocks such as climate change and environmental pollution, are creating new and unforeseen challenges inside cities [2,3,4,5]. For instance, the United Nations reported in 2014 that 4.4 billion people living in cities had been affected by disasters, 1.3 million lives have been lost and USD 2 trillion in economic losses were incurred during the two decades before 2014.

In light of this situation, city stakeholders have become more aware of the need to improve the way cities manage disasters. Originally, a risk management approach was used to tackle crises in cities [6]. However, traditional crisis management mainly responds to expected events that are usually dealt with in an isolated way and without considering the possible cascading effects [7]. This led to the emergence of a new approach to managing disasters, one that focuses on resilience. By adopting a city resilience approach, cities ensure that they will act in a more holistic, economic and collaborative manner and take both expected and unexpected events into account [8]. As a consequence, cities ensure a sustainable development that increases the quality of life with respect to environmental, social and economic considerations without compromising the life of future generations [9,10].

In the last decade, the number of plans and strategies that define the way to adopt city resilience have increased significantly. For instance, the 100 Resilient Cities organization is collecting a total of 100 cities’ resilience programs, including from cities like Kyoto, Glasgow, San Francisco, Montevideo and Paris. In the same vein, the theoretical frameworks that define the concept of city resilience and the process for building it have also been published [2,3,4,11].

However, as the existing strategies and theoretical frameworks have been defined at a strategic level, decision-makers consider the content to be too abstract and far from daily activity [3,8,12,13]. This issue hampers proper operationalization of the strategies and frameworks. In this sense, decision-makers need training tools that bring them closer to the already developed knowledge about building city resilience [14].

The fact that there is still a need to provide decision-makers with guidance in operationalizing resilience and moving from the theoretical concepts of building resilience to their practical application entails the need to make resilience concepts useful and useable beyond their theoretical context. Therefore, the aim of this article is to provide the kind of support that decision-makers need to understand and operationalize city resilience. The research questions (RQ) of this research are the following:

RQ1:

How can the operationalization of city resilience be facilitated to decision makers?

RQ2:

What tools can help decision makers to successfully develop and improve city resilience?

RQ3:

What characteristics do the tools fulfill to be useful for decision makers to improve city resilience?

To answer the three research questions, we present a toolkit developed in the Smart Mature Resilience (SMR) European project (http://smr-project.eu/), funded by the European Union’s Horizon 2020 research program under the call entitled “Crises and disaster resilience—operationalizing resilience concepts”. The SMR project aims to develop a European Resilience Management Guideline that provides support to improve the overall resilience level in European cities, paying special attention to the key role that cities play in any society’s resilience-building process. SMR is composed of seven active city partners that participate in the different parts of the co-creation process of the project. The seven city partners are the following: Glasgow (UK), Donostia/San Sebastian (Spain), Kristiansand (Norway), Rome (Italy), Riga (Latvia), Bristol (UK) and Vejle (Denmark).

This paper presents the combination of a maturity model called the Resilience Maturity Model (RMM) with a serious game called City Resilience Dynamics (CRD). The combination of both tools is presented as a decision-making toolkit for crisis managers to train and learn about how to improve their cities’ resilience level. More in detail, the RMM provides cities with a roadmap for operationalizing the resilience-building process. To that end, it defines a sequence of maturity stages and a set of policies that help cities to assess their current maturity stage and identify the policies that need to be implemented to improve their resilience level. This qualitative tool is complemented with a quantitative tool called CRD to support decision making in the resilience operationalization process. The CRD is a serious game that allows city decision-makers to simulate and quantify the strategy that they should follow to use the resources available efficiently and to obtain the maximum level of resilience in the city.

This paper is structured as follows: First, the state of the art on the concepts of city resilience, sustainability, maturity models and serious games is presented in Section 2. Then, the research methodology followed for developing, testing and validating both tools is described in Section 3. In Section 4, the resulting toolkit is presented. In Section 5, the benefits and the contribution of combining the RMM and CRD tools are explained. Finally, the paper summarizes the main conclusions of our research.

2. State of the Art

The literature provides several definitions for the concept of city resilience [15]. However, in this study, we follow the one given by the SMR project as it provides a holistic perspective on the concept. SMR defines city resilience as “the ability of a city to resist, absorb, adapt to and recover from acute shocks and chronic stresses to keep critical services functioning, and to monitor and learn from on-going processes through the city and cross-regional collaboration, to increase adaptive abilities and strengthen preparedness by anticipating and appropriately responding to future challenges” [16]. Building more resilient cities allows them to withstand and recover from shocks and stresses, such that they are able to adjust plans and procedures before, during and following new or unexpected disturbances and maintain their function as needed throughout the disruption [17]. Building city resilience is a holistic process, where all city stakeholders need to be engaged and act collaboratively [15].

As a consequence of building city resilience, cities ensure a more sustainable development. Sustainability requires the capacity of a system to adapt to changes and continue its functioning over time [18,19] Therefore, a sustainable city needs to develop the attributes that allow to absorb stresses and cope with and adapt to changes [20,21,22]. Several studies have analyzed the relationship among city resilience and sustainability [23,24] but there is still not a clear consensus about which is the relationship among both concepts. However, all the studies ensure that improving the resilience level of a city facilitates sustainable development.

The necessity of acting against the increase in the number of crises and of developing sustainable cities have made building resilience one of the main challenges of the past decade. Several studies and projects that develop conceptual models and frameworks dealing with this challenge can be found in the literature [2,3,4,25,26,27]. These studies aim to define the attributes and actions needed to improve the resilience level of cities [28,29]. While knowledge has been developed with regard to how to build city resilience, debate has been created when the frameworks have been operationalized due to the found difficulties [30]. On the one hand, the information provided by the developed frameworks has been considered too complex to put into practice in everyday activities [30,31,32]. The given definitions seem to be too theoretical and abstract to be easily grasped by all the city stakeholders taking part in the city resilience-building process. Consequently, city stakeholders have had difficulties when operationalizing resilience [3,5]. On the other hand, existing barriers, such as city stakeholder silo-thinking or limited resources, have hampered the proper implementation of the frameworks [7,33]. In fact, in order for the frameworks to be successfully implemented, a holistic perspective is needed, but the transition from thinking in silos to adopting a holistic perspective is complicated, as the resilience-building process brings together different city stakeholders’ cultures, contexts and needs.

In this context, and in order to facilitate the operationalization of the developed resilience frameworks, cities and more precisely decision-makers, demand less abstract resilience policies that can be related more intuitively to everyday activities [5,14]. Actually, there are limited examples of the sequential steps decision-makers should follow to develop city resilience [34]. Furthermore, cities can exhibit a great variation in their resilience level, and there is a lack of guidance on which policies should be implemented as a function of the current situation of the city [31,32,35].

In this context, maturity models have proven to be a useful tool. Maturity models serve to identify the ideal path for the evolution of a process from an initial stage to a more advanced stage, passing through a number of intermediate stages [36,37]. Maturity models consist of a structured collection of elements describing the characteristics of effective processes at different stages, suggesting goals to be met at each stage and transition activities to get from the bottom stage of maturity to the highest level of maturity [38,39].

However, taking into account the fact that city resilience is a complex concept, even with the development of tools such as maturity models, decision-makers are still requesting tools that enable them to train themselves in the resilience-building process. In this sense, the serious game has been defined as an effective tool for teaching, motivating and simulating decisions [40,41,42,43,44]. Serious games are defined as “applications using the characteristics of video or computer games to create engaging and immersive learning experiences for delivering specified learning goals, outcomes and experience” [45]. Indeed, serious games enable users to understand the problem at hand better, to train themselves to take more appropriate decisions and to get to know the system under study [34,35]. Consequently, the learning process and the assimilation of the concepts presented are facilitated [42,46]. In addition, serious games are easily complemented with other activities that might be necessary during the learning process.

A wide range of serious game applications can be found in the literature with different uses and objectives [47]. For example, Patroklos [48] presents a serious game that analyzes and trains decision-makers on the strategic capacity of planning recycling networks. The game makes it possible to test policy implementation and analyze the consequence of the taken decisions.

Regarding critical infrastructures, Balaraman et al. [49] developed a serious game in order to efficiently manage critical infrastructures during crises. The serious game runs into different crisis severity levels and shows for each case the size of the crisis response/management team that would be required.

From a more educational perspective, Chappin et al. [42] developed a serious game called Oil Springs to get population attention and educate about city sustainability. In this case, the serious game simulates a possible future scenario in which players are forced to reconsider their actual strategies to “save” their city. Depending on the taken decisions, the city will evolve in different ways and the player will learn about the effects and consequences of the strategy followed.

More precisely, in the area of city resilience, serious games have an interesting role, as policy implementation can be simulated. Consequently, decision-makers are provided with a tool that brings them closer to the city resilience-building process, as they get to train themselves with scenarios and strategic policies. However, the serious games found in the literature (some examples have been presented in this section) are too specific concerning concrete themes. Given the gaps identified in the resilience operationalization process and taking into account the characteristics of both maturity models and serious games, our research team has developed a toolkit composed by a maturity model; resilience maturity model (RMM) and a serious game; City Resilience Dynamics (CRD). The RMM comprises five well-defined maturity stages that guide cities through the ideal path for building resilience. Cities will start at one stage, and from there move on to a more advanced stage while passing through a number of intermediate stages. The RMM is embraced in the CRD that aims to provide decision-makers with an easy-to-use training tool for the city resilience-building process.

3. Research Methodology

Taking into account the complexity and dynamic nature of the resilience-building process, our research followed a co-creation process involving around 60 multidisciplinary experts in the city resilience-building process and their fragmented knowledge was integrated into the toolkit. This co-creation process ensures that the toolkit is developed specifically for the needs of cities [50].

This research is based on an empirical foundation that collects the knowledge, data and experience from experts in the field. During the development, testing and validation processes the experts were asked to provide the experiences they had during the development of city resilience and the requirements of the toolkit to facilitate the operationalization of city resilience.

For each tool composing the toolkit, a two-phase process for developing, testing and validating the tools was carried out (see Figure 1). In the first phase, the tools were developed through several iterations by combining different research methods. In the second phase, the tools were tested and validated through three pilot testing workshop in three European cities, namely Kristiansand in Norway, Donostia/San Sebastian in Spain and Glasgow in the UK. It is worth mentioning, although during the co-creation process of the toolkit final users tested the tools, no application of the toolkit has been done yet. The aim of the testing was to test the usability of the tool as a resilience building tool.

3.1. Development of the RMM Tool

In order to develop the RMM, we first held four workshops (see

Table 1. Workshops during the Resilience Maturity Model (RMM) development process.

	Place	Topic Addressed	No. of Experts	Field of Expertise
Development workshop 1 [54]	Riga (Latvia)	City’s dependency on critical infrastructures	20	Resilience building researchers, sustainability consultants, standardization consultants and city councils’ departments for: Climate change, social issues, crisis management and critical infrastructures.
Development workshop 2 [55]	Bristol (UK)	Risks associated with climate change	15	Resilience building researchers, sustainability consultants, standardization consultants and city councils’ departments for: Climate change, social issues, crisis management and critical infrastructures.
Development workshop 3 [56]	Rome (Italy)	Challenges associated with social issues	13	Resilience building researchers, sustainability consultants, standardization consultants and city councils’ departments for: Climate change, social issues, crisis management and critical infrastructures.
Development workshop 4 [57]	Vejle (Denmark)	Cities’ experiences, best practices and difficulties concerning resilience operationalization	16	Resilience building researchers, sustainability consultants, standardization consultants and city councils’ departments for: Climate change, social issues, crisis management and critical infrastructures.

) involving from 13 to 20 city representatives and local stakeholders from the seven city partners to gather the requirements of the resilience-building process. A collaborative methodology called group model building was used to gather the fragmented knowledge residing in the experts’ minds and integrate it into the models [51,52,53].

The different challenges cities are facing were discussed in all the workshops. Furthermore, the potential policies that could be implemented to overcome potential challenges were identified. As a result, a list of resilience-building policies, in addition to useful requirements for the development of the RMM, were identified. Then, to get more confidence in the initial results obtained from the workshops, a Delphi study was conducted.

The Delphi process has a set of rounds, and in each round the participants work through a questionnaire, which is returned to the facilitator, who then collects and provides the participants with a summary of the comments made by all the participants so they are aware of the range of opinions and perspectives [58]. In the Delphi that we carried out, 32 multidisciplinary experts took part from all over Europe and from different backgrounds (see

Table 2. Participants during the Delphi process for the RMM.

Field of Expertise	No. of Experts 1st Round	No. of Experts 2nd Round
Critical Infrastructures	9	7
Climate Change	8	8
Social Issues	10	6
Holistic Crisis Management	13	11
Total	40	32

). The variety of stakeholders involved in the development of the RMM guaranteed that different interests and perspectives were taken into account. The Delphi process was structured in two rounds. The first round aimed at validating the description of the five maturity stages included in the RMM and the second round sought to classify the resilience-building policies, considering the maturity stage in which their development should begin in order to guarantee their effectiveness in the resilience-building process. Therefore, the objective of the Delphi process was to validate the description of the five maturity stages defined in the RMM as well as to define an optimal and effective sequence in which the resilience-building policies need to be implemented.

3.2. Testing and Validation of the RMM Tool

In the second phase, the testing and validation of the RMM tool was undertaken through three pilot testing workshops in three European cities (Kristiansand in Norway, Donostia/San Sebastian in Spain and Glasgow in the UK) with 11 to 12 potential users of the RMM (see

Table 3. Participants during the testing and validation workshop of the RMM.

	Place	No. of Experts	Field of Expertise
Testing and validation workshop 1 [54]	Kristiansand (Norway)	20	Critical infrastructure companies (transport, water) and city councils’ departments for: Climate change, social issues, crisis management and critical infrastructures.
Testing and validation workshop 2 [55]	Donostia-San Sebastian (Spain)	15	Critical infrastructure companies (transport, water) and city councils’ departments for: Climate change, social issues, crisis management and critical infrastructures.
Testing and validation workshop 3 [56]	Glasgow (UK)	13	Critical infrastructure companies (transport, water) and city councils’ departments for: Climate change, social issues, crisis management and critical infrastructures.

) [59]. The aim of the pilot workshops was to validate and test the usability of the RMM as a tool to diagnose the current maturity level of each city based on the RMM. Additionally, the RMM was presented as a decision-making tool in order to define the strategy for implementing future policies in order to enhance their resilience level. First, the experts were asked to find real evidence from the already implemented policies in order to assess their current resilience level. Once they diagnosed the current state, they then analyzed the policies whose implementation was pending in order to improve their city’s resilience level and help with the decision-making process that is part of the resilience-building process.

3.3. Development of the CRD Tool

The CRD tool is based on the RMM developed in the SMR project. Taking as a basis the resilience policies defined in the RMM, first, the precedence relationships among the resilience policies were identified through a survey. These precedence relationships determine which policies should have been implemented previously for a policy to be effective in its implementation. In this survey resilience managers from the seven city partners of the SMR project took part (see

Table 4. Participants of the survey about the City Resilience Dynamics (CRD) tool.

Field of Expertise	Place
Manager in a resource centre of the municipality	Vejle, Denmark
Sustainability Project Manager at the City Council	Bristol, UK
Assistant Manager at the City Council	Glasgow, UK
Crisis Manager and Security Officer at the City Council	Kristiansand, Norway
Project Manager in an energy agency	Riga, Latvia
Urban Resilience and Natural Risk assessment at the City Council	Rome, Italy
Technician at the Office of Strategy of the city	Donostia/San Sebastian, Spain

). The participants were asked to assess from 0 to 5 (being 0 for no relationship and 5 for a strong relationship) how the policies were related to each other regarding the precedence dependencies.

Once the precedence relationships among the resilience policies were defined, an initial version of the serious game was designed. System dynamics methodology was used to model the complex system that represents the resilience-building process [47,60,61]. This methodology is based on information feedback control theory and deals with nonlinear dynamics, internal feedback loops and time delays, which influence the whole system [60,62]. The key concept of system dynamics is that all the objects in a system interact through causal relationships [60,62].

This system dynamics model was incorporated into a user-friendly interface. The combination of both result in a serious game that allows all user to interact with the policies defined in the RMM. Furthermore, the results are displayed in the same user-friendly interface to facilitate the interpretation of the results to the user. This user-friendly interface was developed using the software Insight Maker, which is an open source general purpose simulation and modeling tool [63].

This preliminary tool was iteratively improved through two successive workshops with 30 experts carried out in the city of Donostia-San Sebastian in Spain, and the city of Glasgow in the UK. In both workshops, local authorities from seven European city partners of the SMR project participated (see

Table 5. Participants of the workshop to develop the CRD.

	Place	No. of Experts	Field of Expertise
Development workshop 1 [64]	Donostia-San Sebastian (Spain)	30	Resilience building researchers, sustainability consultants, standardization consultants, critical infrastructure companies and city councils’ departments for: Climate change, social issues, crisis management and critical infrastructures.
Development workshop 2 [64]	Glasgow (UK)	30	Resilience building researchers, sustainability consultants, standardization consultants, critical infrastructure companies and city councils’ departments for: Climate change, social issues, crisis management and critical infrastructures.

).

Moreover, during these workshops, a usability test was conducted with all the participants, to examine the ease of use of the user interface. As an overall comment, the participants agreed on the preliminary version of the tool needed to be closer to reality. For example, they suggested the implementation should depend on the level of invested resources, the cost of the policies should vary from one policy to the other one, the tool should allow representing budget cuts from one year to the next year and the relationships among the policies should have different weights. Taking into account the comments gathered from the workshop, the tool was improved.

Finally, a questionnaire was carried out among the partners of the SMR project to gather data for the main parameters of the underlying system dynamics model (see Table 4). The participants were asked to assess, using as a reference the characteristics of their own city, the total cost of implementation of each resilience policy, the total implementation time of each policy and the total depletion time of each policy. These data was used to calibrate the model. As a result, an improved version of the serious game was developed. The Smart Mature Resilience project deliverable [64] explains in great detail the co-creation process to develop the serious game.

3.4. Testing and Validation of the CRD Tool

Similarly to the RMM tool, the validation and testing of the usability of the CRD tool was undertaken through three workshops in the same European cities (Kristiansand in Norway, Donostia-San Sebastian in Spain and Glasgow in the UK) with 10 to 12 potential users of the model (see

Table 6. Participants during the testing and validation workshop of the CRD.

	Place	No. of Experts	Field of Expertise
Testing and validation workshop 1 [64]	Kristiansand (Norway)	11	Critical infrastructure companies (transport, water) and city councils’ departments for: Climate change, social issues, crisis management and critical infrastructures.
Testing and validation workshop 2 [64]	Donostia-San Sebastian (Spain)	12	Critical infrastructure companies (transport, water) and city councils’ departments for: Climate change, social issues, crisis management and critical infrastructures.
Testing and validation workshop 3 [64]	Glasgow (UK)	10	Critical infrastructure companies (transport, water) and city councils’ departments for: Climate change, social issues, crisis management and critical infrastructures.

) [65]. During the pilot workshops, the participants were asked to use the tool as a decision-making tool for their own cities resilience-building process. First, they were asked to calibrate the tool to their cities characteristics. Second, they needed to make a decision on the strategy they would follow in order to efficiently improve their resilience level. After, they simulated their strategy and analyzed the results obtained through the CRD tool and the suitability of the strategy adopted. In the end, the experts were asked to answer a questionnaire to assess the “usability”, the “complexity” and the “suitability” of the tool as a decision-making tool.

Finally, with the aim of not overwhelming the user with the 98 resilience policies defined in the RMM, the final version of the tool has only 45 policies. Nevertheless, the chosen 45 policies have been selected carefully in order to represent the whole RMM.

4. Results

As a result of the carried out co-creation process, a toolkit composed by a maturity model and a serious game was obtained. The developed toolkit aimed to answer the gap found in the literature and provide decision makers with a toolkit that facilitates the operationalization of city resilience. In this section, first the qualitative RMM is presented. Then, the quantitative CRD is explained.

4.1. The Resilience Maturity Model

The RMM is a tool that guides local authorities, policy-makers and critical infrastructure service operators in the resilience-building process. The RMM presents five sequential maturity stages (Starting, Moderate, Advanced, Robust, and verTebrate) that serve as a roadmap for effectively building city resilience. Each of the maturity stages includes a description of the objectives of that stage and a list of policies that should be implemented in order to achieve the stage’s objectives.

The policies in each maturity stage are classified according to four pillars or dimensions that combine effective leadership and governance (L), preparedness (P), infrastructures and resources (I) and cooperation (C) among the stakeholders. Each resilience dimension has been split into several sub-dimensions that group related policies (

Table 7. Resilience dimensions and sub-dimensions.

Resilience Dimensions	Resilience Sub-Dimensions
Leadership and Governance (L)	L1: Municipality, cross-sectorial and multi-governance collaboration
	L2: Legislation
	L3: Learning culture (learning and dissemination)
	L4: Resilience plan development
Preparedness (P)	P1: Diagnosis and Assessment
	P2: Education and Training
Infrastructure and Resources (I)	I1: Reliability of infrastructures
	I2: Resources to build up resilience
Cooperation (C)	C1: Development of partnerships with city stakeholders
	C2: Involvement of resilience networks in cities

).

Thus, the RMM is presented in the form of a matrix consisting of five maturity stages and four dimensions that serve to classify policies and city stakeholders in the different stages and dimensions [66]. Furthermore, the main objectives of each maturity stage as well as the city stakeholders proactively involved in each stage are defined. Figure 2 shows its structure.

The RMM maturity stages describe the following roadmap for the resilience building process: (1) In the starting stage, the cities are working in silos where each department is working on its own in disaster management; (2) in the moderate stage, the awareness level on the importance of building resilience as a whole starts to raise, integrating the efforts of all the city stakeholders; (3) in the advanced stage, the resilience strategy plan is developed where the main actions towards resilience are defined from a holistic point of view; (4) in the robust stage, the city starts to internationalize the resilience building process by involving relevant stakeholders outside the city boundaries and taking an active role in different international networks; finally, (5) in the vertebrate stage, the city becomes a reference for all the European cities in relation to resilience and promotes a culture of continuous improvement based on the lessons learned and with the commitment and involvement of all the stakeholders. These maturity stages are achieved by developing a set of policies classified in the four dimensions mentioned before. Therefore, the RMM tool helps making the resilience development process tangible and practical for cities.

Table 8. The RMM “Infrastructure and Resources” dimension as presented in Hernantes et al. [66].

		Starting	Moderate	Advanced	Robust	Vertebrate
Infrastructure and Resources	Reliability of infrastructures (I1)	(I1S1) Develop cooperation/collaboration agreements with critical providers. (I1S2) Develop plans to monitor critical infrastructures functionality. (I1S3) Develop contingency plans for critical infrastructures.	(I1M1) Identify interdependencies of critical services at local level. (I1M2) Develop periodical preventive maintenance procedures for critical infrastructures. (I1M3) Develop measures to increase critical infrastructure redundancy and reliability. (I1M4) Implement monitoring systems for identifying risk shocks and long-term stresses. (I1M5) Carry out audits for critical infrastructure providers.	(I1A1) Develop flexibility measures.	(I1R1) Identify interdependencies of critical services at international level.	(I1T1) Encourage the continuous improvement of policies, to take advantage of any shock and stress to bounce forward and improve or re-design. (I1T2) Apply big data approaches to analyze the information obtained.
Infrastructure and Resources	Resources to build up resilience (I2)	(I2S1) Assess current initiatives and funding opportunities for the development of resilience. (I2S2) Develop a list of the currently available response physical resources. (I2S3) Deploy a disaster relief fund for emergencies.	(I2M1) Allow for the resilience action plan in the local government budget. (I2M2) Promote resources /tool sharing among critical infrastructure providers within a region during crises.	(I2A1) Promote and provide incentives for initiatives that contribute to build resilience. (I2A2) Implement centralized control of coordination of critical resources and activities during shocks and stresses. (I2A3) Encourage stakeholders to have appropriate insurance coverage. (I2A4) Promote and provide incentives for the development of sustainable urban infrastructures.	(I2R1) Promote and provide incentives to stakeholders for investment in R&D&I projects regarding resilience. (I2R2) Monitor an effective use of resources to ensure the resilience-building process performance.	(I2T1) Assess the impact of innovation in the resilience-building process. (I2T2) Monitor the insurance level of stakeholders.

shows the policies that need to be implemented in each maturity stage in the case of the Infrastructure and Resources dimension.

4.2. The City Resilience Dynamics

The main aim of the CRD tool is to allow decision-makers to test several resilience strategies over time and to help them to understand and learn how the resilience-building process works for a city and how the optimum path can be obtained based on the policies defined in the RMM. Thus, the CRD tool helps crisis managers to diagnose, explore and learn about the resilience path that cities need to follow to improve their resilience levels.

CRD is defined based on the resilience policies of the RMM (see as an example Table 8 Section 4.1). However, even though the RMM defines each of the maturity stages and the policies that need to be implemented to go from one maturity stage to the other, the RMM does not explicitly represent the relationships among the resilience policies. Taking into account the fact that the process for building city resilience is a dynamic process, implementing one policy might affect the implementation of others, and vice versa. Thus, depending on the order in which policies are applied, the efficiency and effectiveness of their implementation will vary, as will the progress in the maturity stage. In this sense, CRD encapsulates the policies developed in the RMM and facilitates the testing of different policy options and an understanding of the temporal order in which the policies should be implemented in order to improve city resilience level.

Two types of resilience policy relationships have been defined and are encapsulated in CRD: Linear and transversal. Linear relationships refer to temporal relationships that exist among the different maturity stages. This means that policies in the lower maturity stages should be developed to implement the policies in the higher maturity stages. For example, within each sub-dimension, policies in the “Starting” stage should be developed in order to be able to implement the policies in the “Moderate” stage, and similarly, policies in the “Moderate” stage should be developed in order to implement the policies in the “Advanced” stage (see red arrows in Figure 3).

Transversal relationships refer to relationships among the policies in different sub-dimensions. As previously explained, policies in the RMM have been grouped into dimensions depending on the topic. They have also been divided into different sub-dimensions with the same dimension depending the specific topic they concern. Hence, although the policies have been divided into different sub-dimensions, these sub-dimensions are interrelated. Therefore, within each maturity stage, the relationships among the sub-dimensions have been defined. These transversal relationships are maintained from one stage to the next one (see green arrows in Figure 3).

Apart from that, as stated in Section 3.3, the CRD tool is composed of two elements: A system dynamics model and a user-friendly interface. The system dynamics model defines the behavior of the CRD based on the resilience policies defined in the RMM, whereas the user-friendly interface interacts with the user in order to obtain the input data and show the results in a visual and user-friendly way.

The system dynamics model is defined by fixed variables, input values introduced by the user and the outputs resulting from the model. The main input of the system dynamics model is the allocated budget in each policy. The fixed variables are the policies’ implementation time and cost, the policies’ depletion time as well as the linear and transversal relationships. As a result, based on the decisions taken and the fixed values, the model calculates the actual and effective implementation level of each policy and the implementation levels both at a dimension level and overall resilience level. These variables are shown in the user-friendly interface through the three different screens that compose the interface: Initial state screen, decision-screen and results-screen.

The initial state screen is where the users enter the tool and get the chance to parameterize the CRD to their cities’ particular characteristics. In this screen, the users need to define the city they want to study by defining certain parameters that represent the city’s context (see Figure 4).

The decision-screen is the one that shows the list of resilience policies of the RMM, classified according to the four resilience dimensions and five maturity stages. In this screen the user decides what resilience policies they should invest in and how much to invest for each policy (see Figure 5).

Finally, the results-screen shows the results of the simulation based on the decision taken in the decision-screen. The percentages of the current resilience policy implementation, labeled “Actual”, and the effective implementation level of the resilience policy, labeled “Effectiveness”, are presented for each resilience dimension. These two percentages aim to represent the consequences of the relations existing between the policies. Therefore, if policies are not implemented in the correct order, the percentages will show low effectiveness and implementation relative to what the user has decided in the decision-screen. In this case, pop-up messages appear in order to warn the user that an ineffective investment has been made. Moreover, at the top right hand of the screen, the resilience dimension’s implementation level results are presented through time evolution graphs (see Figure 6).

Below the graph, the evolution of the total used budget is represented through a time evolution graph where time is represented in years and the budget in Euros. Next to this graph, the speedometers indicate the resilience maturity stage that the user has achieved for each of the resilience dimensions. The speedometers start at 0 and go from the starting stage to vertebrate passing through moderate, advanced and robust (S, M, A, R, T; see Figure 7).

The final version of the tool can be found at the following link: http://crd.smr-project.eu/.

5. Discussion: Combining the RMM and CRD Tools

The previous two sections presented the process of building, validating and testing the RMM and CRD as well as the results obtained. The RMM provides the ideal path decision-makers should follow to build city resilience. Therefore, we considered the RMM to be a qualitative tool defined from a general and holistic level that has the ability to define the current resilience maturity stage of a city and the steps that need to be followed to reach higher maturity stages. That is why it is helpful to combine the qualitative RMM with a more quantitative tool such as CRD. In fact, the aim of the CRD tool to enable decision-makers to test several strategies and help them to understand and learn how the resilience-building process in a city works and how the optimum path can be obtained. Therefore, CRD helps crisis managers to diagnose, explore, assess and learn about the resilience path that cities need to follow to improve their resilience levels as defined in the RMM. The combination of both tools helps cities periodically assess, both qualitatively and quantitatively, their city resilience building process’ effectiveness. It also supports decision makers defining updated sets of requirements in order to improve their performance with regards to the resilience building process. Consequently, a constant monitoring of the resilience building process is facilitated through the toolkit.

Therefore, by combining both tools, decision-makers are provided with the theory to build city resilience and the opportunity to train themselves and quantify the city resilience-building process, contributing this way to the lack of guidance for operationalizing city resilience. The toolkit will support cities in the monitoring and assessment process of developing city resilience as it can be used for benchmarking purposes and as a point of reference. The toolkit has been designed to be useful, trustworthy and flexible. In this sense, as the decision-makers have actively participated in the co-creation process their needs and requirements have been taken into account.

Regarding usefulness, both tools have been designed under the requirements decision-makers have requested in order to build intuitive and easy-to-use tools. For instance, the CRD tool introduces the dynamic aspect of the resilience-building process defined in the RMM, making explicit the precedence relationships that exist among the resilience-building policies. These precedence relationships help the user to make decision about the implementation of the policies by optimizing the resources required and allowing the user to try different policy implementation strategies and see the consequences of each option in order to determine the most convenient one for each case. In fact, the CRD tool informs the user about the effectiveness of the implemented strategy based on the precedence relationships defined through indicators.

In turn, the CRD tool helps to make aware the user about the required resources and time when building resilience. Policies cannot be implemented immediately and, apart from the initial investment, they require some resources to maintain their level over time. Once the policies are implemented, if these policies are not maintained their level would again decrease. For example, if one resilience policy is implemented but is not maintained over time, the policy implementation level would again decrease.

Concerning trustworthiness, the combination of both tools can be powerful, yet if the given results are not close to reality, decision-makers will not trust and therefore not use the toolkit when operationalizing city resilience. That is why the CRD is combined with the theory developed in the RMM, which explicitly determines an ideal path to reach higher resilience levels.

Finally, in order for both tools to be flexible, the decision-maker can particularize the CRD to each city’s characteristics. This way the results showed will be closer to the city under study. In fact, flexibility was one of the aspects decision-makers participating in the co-creation process reported as an important aspect.

6. Conclusions

The concept of building city resilience has emerged as a new way to approach crises and transform crisis management from thinking in silos to taking a holistic perspective to ensure cities’ sustainable development. In the past decade several strategies and theoretical frameworks composed of indicators and metrics have been proposed. However, decision-makers want less theoretical and more practical tools that put the process of building city resilience into practice.

In this context, this paper has presented the process and results of combining the qualitative RMM tool, which defines the path towards higher city resilience levels, with the quantitative CRD tool, which enables decision-makers to train themselves and get closer to the resilience-building process. The RMM defines five sequential maturity stages that cities pass through, starting at the lowest resilience levels and ending with the achievement of resilience excellence. These maturity stages and the policies that compose them are included in the CRD, which quantifies the effort that needs to be made in order to implement each policy, taking into account the policies’ implementation cost, implementation time and depletion time. This way decision-makers are provided with a tool that enables them to train themselves in the process of building city resilience.

We conclude this study with three main ideas that answer the addressed research questions. The first is that the combination of a qualitative tool with a quantitative tool creates and effective toolkit to facilitate the city resilience operationalization as it helps in the monitoring and evaluation of the resilience building process. The second idea is that by combining the RMM and CRD tools, we have developed a trustworthy toolkit that decision-makers can use when working on the city resilience-building process as the theory defined in the RMM is included in the CRD. Finally, the third idea is that using a co-creation method has ensured that the toolkit is useful and flexible for decision-makers, since they participated in the development process for both the RMM and the CRD, and they had the opportunity to state their requirements. Therefore, having answered the three research questions, we conclude this research presents an effective toolkit for decision makers to use it in their daily activities as a support tool in the city resilience building process.