Next Article in Journal
Virtual Worlds for Learning in Metaverse: A Narrative Review
Previous Article in Journal
Utilising eDNA Methods and Interactive Data Dashboards for Managing Sustainable Drinking Water
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

“The Greatest Benefit Is to Think Differently”: Experiences of Developing and Using a Web-Based Tool for Decision-Making under Deep Uncertainty for Adaptation to Sea Level Rise in Municipalities

by
Annika Carlsson Kanyama
1,*,
Jorge Luis Zapico
2,
Chatarina Holmberg
1 and
Per Wikman-Svahn
3
1
Ecoloop AB, 118 60 Stockholm, Sweden
2
Department of Computer Science and Media Technology, Linnaeus University, 351 95 Växjö, Sweden
3
Department of Philosophy and History, KTH Royal Institute of Technology, 114 28 Stockholm, Sweden
*
Author to whom correspondence should be addressed.
Sustainability 2024, 16(5), 2044; https://doi.org/10.3390/su16052044
Submission received: 9 October 2023 / Revised: 25 January 2024 / Accepted: 9 February 2024 / Published: 29 February 2024
(This article belongs to the Section Hazards and Sustainability)

Abstract

:
The need for handling the deep uncertainty surrounding the future climate has led to various novel and robust approaches for decision-making under deep uncertainty (DMDU) when adapting to climate change. Here, an online and self-explanatory web-based tool was developed and tested with civil servants from five municipalities in Sweden challenged by rising sea levels. The municipalities used the tool by themselves and were then interviewed about the usability of the tool, the perceived urgency of climate change adaptation, and the possibilities for municipalities for handling the flexible solutions that are at the core of DMDU. Results were that the civil servants reported that the urgency of climate change adaptation has increased, that the tool changed their perspectives compared to traditional planning methods, and that changes in laws and regulations to accommodate decisions about flexible solutions were recommended.

1. Introduction and Purpose

Human-induced climate change is now causing adverse impacts on humans and ecosystems in every region through weather and climate extremes [1]. As global warming will continue to increase in the near term (2021–2040), changes in weather extremes will be more extensive, and the Intergovernmental Panel on Climate Change (IPCC) assesses that it is “virtually certain” that the sea level will rise [1], (p. 13). However, how much sea levels will rise is extremely uncertain. A global mean sea level rise approaching 2 m in 2100 and more than 15 m in 2300 can no longer be excluded, even though these estimates are considered of low confidence. This possible development may be caused by the rapid melting of the ice sheets in Greenland and Antarctica, a process surrounded by deep uncertainty [1], (p. 19). Deep uncertainty is defined as “when experts or stakeholders do not know or cannot agree on (1) appropriate conceptual models that describe relationships among key driving forces in a system, (2) the probability distributions used to represent uncertainty about key variables and parameters and/or (3) how to weigh and value desirable alternative outcomes” [2], as cited in ref. [3], (p. 2926).
Handling uncertainty, especially the deep uncertainty of sea level rises, is a challenge in climate change adaptation (henceforth “CCA”). A recent IPCC assessment notes that “adaptation gaps exist, and will continue to grow at current rates of implementation” [1], (p. 8) and that key barriers to adaptation include a lack of political commitment, slow uptake of adaptation science, and a low sense of urgency.
Important CCA decisions are taken at the local level, and, using the example of CCA measures for managing sea level rises, the decision of whether to protect or relocate a low-lying settlement may depend on whether the expectation is for the mean sea level to rise by less than 1 m or nearly 2 m by 2100. Though not recommended, one way of handling such uncertainty is to choose a middle scenario, essentially ignoring the uncertainty. More recommendable is to develop “strategies that are well hedged against a variety of different futures and adjustable through time in response to emerging information” [4], (p. 39).
In response to the need for strategies, various approaches for decision-making under deep uncertainty (DMDU) have been developed (see Section 2). The climate change research community has increasingly noted the existence of these methods, and, notably, the most recent IPCC report (AR6) included a cross-chapter box on approaches for managing DMDU [5], (pp. 2578–2580).
The academic literature on DMDU approaches is still quite limited, and more case studies are needed to better understand how DMDU methods can be applied in different contexts and how they can be integrated with existing decision processes. A recent literature review of the applicability of DMDU methods in practice concluded that “more progress is needed for the methods to leave the academic realm and to be widely used in reality” [6], (p. 9). Study [6] suggest four ways to make DMDU methods “more applicable to real cases, beyond academia. The first is to extrapolate DMDU approaches from being mainly quantitative to a more qualitative ‘lighter’ approach…” [6], (p. 9).
The tool developed and evaluated by end-users in this study is based on previous research on a light version of the dynamic adaptive pathways method [7] (as described in Section 2.3 and Section 3 below). The present study contributes to the literature by developing an improved version of this light DAPP method using iterative design with the end-users, which are then using the method by themselves as described in the Section 3 and Section 4. In particular, this study explores the possibilities of digitalisation methods for creating an improved light DAPP method.
New et al. [5] find that there exists a knowledge gap regarding the evidence of the usefulness of methods that can help decision-makers in DMDU. This study contributes to a better understanding of end-users’ experiences of using one particular method. Digitalisation provides new possibilities for working with sustainability, helping in gathering and managing bigger amounts of data, combining and reutilising different data sources, and analysing and visualising data for creating new insights [8,9].
In this paper, we present results from developing a web-based tool (henceforth called “the Tool”) for DMDU at the local level regarding sea level rises. One significant feature of the Tool is its user-friendly interface and design, which allow end users to apply DMDU methods to their specific contexts. This is an important advancement considering the challenges of application and limited uptake of these methods in organisations, as noted by Stanton and Roelich [6]. Therefore, the development of this tool responds directly to the call for more accessible DMDU applications.
In this study, we focused on civil servants’ responses in the municipalities that tested the Tool. The development of the Tool emanated from the results achieved in a research programme implemented from 2016 to 2020 that investigated possibilities and obstacles for using a robust approach when planning for CCA at the municipality level. By the end of the programme (hereafter called “the ROBUST programme”), the conclusion was that a useful idea of a robust CCA method had been developed and tested to a certain extent but that the method was too resource demanding in terms of competence and time for the target group (civil servants in municipalities). The results also showed that in the current organisational setting in Sweden, handling a flexible approach in CCA is difficult. Moreover, using worst-case scenarios in CCA (embracing uncertainties) was considered controversial (see the Section 2).
To overcome some of these obstacles, a new research project was formed, including web and business development competencies. The aim was to create a prototype of an online tool that would be self-explanatory and based on the method used in the ROBUST programme. The Tool was created in the winter/spring of 2022 and tested by civil servants in five municipalities in the summer/autumn of 2022. These civil servants were then interviewed, and their responses were analysed in 2023.
In this paper, we present our findings from the interviews, which were based on three research questions:
  • Has the perceived need for CCA in municipalities increased compared with that around 5 years ago, and if so, why? This question was asked under the assumption that with a higher need, the acceptance of spending time with the Tool would be higher.
  • What are the pros and cons of using the Tool, including how it could be developed? Here, we emphasised the respondents’ views of how easy the Tool was to use, as we assumed that if civil servants could avoid paying for costly consulting hours, the acceptance for using the Tool would be higher.
  • Can municipalities handle flexible CCA solutions? How did the Tool support and/or limit flexible CCA solutions? This question was asked because we wanted a second opinion on this issue compared with previous research.
This study documents the results of the development and application of a self-explanatory and user-friendly web-based application of a DMDU decision-support tool. The tool was developed and tested in an iterative process together with several municipalities, which were then questioned to elicit information on their experiences.

2. Background

2.1. Approaches to Handling Deep Uncertainties in CCA

In recent years, appreciation of the need to manage situations under what has come to be called deep uncertainty has been increasing [10]. Managing climate adaptation under deep uncertainty poses significant challenges, as traditional decision-making methods may not be sufficient for dealing with the uncertainties and complexities involved. Decision-makers must account for a wide range of factors, including scientific uncertainties, societal values, ethical considerations, and stakeholder preferences. Therefore, several approaches have been developed to improve the management of situations of deep uncertainty, including methods such as decision scaling, robust decision-making (RDM), multi-objective RDM (MORDM), dynamic adaptive planning, dynamic adaptive policy pathways (DAPPs), info-gap decision theory, and engineering options analysis (for a review, see ref. [10]).
One of the most significant benefits of DMDU approaches is that they enable decision-makers to account for a wide range of uncertainties, including scientific, societal, and political uncertainties. By explicitly accounting for these uncertainties, decision-makers can develop more robust and adaptive decision-making strategies that are better suited to the complex and uncertain nature of climate change. In addition, DMDU approaches facilitate stakeholder engagement, allowing decision-makers to incorporate the views and preferences of a diverse range of stakeholders into the decision-making process.

2.2. Division of Responsibilities for CCA in Sweden

Sweden consists of 21 counties and 290 municipalities [11], and many national authorities. Traditional town and country planning is mainly a municipal responsibility, and each municipality is requested by law to have an updated comprehensive plan. In the comprehensive plan, the municipality must present the basic characteristics of its intended use of land and water areas; how the built environment is to be used, developed, and preserved; what consideration is to be given to public interests; and what the intention is regarding how national interests and environmental quality standards are to be served. The plan must also indicate how the municipality intends to consider national and regional goals and plans. The municipalities are also the public bodies entitled to develop and decide on legally binding detailed development plans. Only the municipality has the authority to adopt plans and decide whether the plan should be implemented. As a result, municipalities are key actors in the management of CCA in Sweden and have levy taxes. The municipalities decide where to build or not and are responsible for much of the functioning of critical infrastructures such as roads, water and sewage distributions, garbage collection, emergency management, schools, and home-care services. Thus, in the present study, we focused on how civil servants in municipalities experienced the Tool.
The municipalities’ work is overseen by the counties, which have the task of coordinating and guiding CCA, for example. The county administrations are the extended arm of the state in the various regions, and they can intervene in matters that affect national interests, environmental quality standards, inter-municipal interests, and matters relating to health and safety, and the risk of accidents, flooding, or erosion.
At the Swedish national level, a relatively new (2018) National Strategy for Climate Adaptation was established with several guiding principles at all levels [12]. Several of these principles have not been previously part of the mainstream adaptation in Sweden and pose a challenge to Swedish actors at both the national and local levels. This is especially the case for the principles of flexibility, uncertainty management, risk management, and flexible time horizon. Furthermore, at the national level, a highly important stakeholder for CCA is the Swedish Meteorological and Hydrological Institute (SMHI). This is the expert authority for weather and climate forecasts, and hosts a web portal for guiding municipalities in CCA [13]. The municipalities turn to the SMHI for information such as flooding risks and future sea level projections. Another important stakeholder for CCA at the national level is the Swedish National Board of Housing, Building, and Planning, which deals with developments within the housing, building, and other planning-related casework in Sweden. They are also responsible for coordinating the national climate adaptation work for the built environment. Along with the new national adaptation strategy, amendments to the national plan and Building Act 2018 were made that required the municipalities to report their views of the comprehensive plan on climate-related risks that may arise and cause damage to the built environment.

2.3. Findings from the 2016–2020 ROBUST Research Programme

Previous results of relevance from the ROBUST research programme include those from a study of the extent of the use of a robust approach in the existing planning processes for managing sea level rises. The material was collected, among others, from Swedish municipalities in 2016, and the result showed a discrepancy between the current planning paradigms and the core principles of RDM. All municipalities planned for only one level of global sea level rise in 2100, and national authorities provided the regional and local authorities with that number through a top-down approach. A conclusion from that study was that considering several scenarios of sea level rise would probably be more/too time-consuming for the municipalities.
To further investigate the opportunities and constraints in using RDM in practice at the local level, a simplified methodology was developed based on two existing methodologies for handling DMDU: namely Dynamic adaptive policy pathways (DAPP) [7] and Climate Risk-based Decision Analysis (CRIDA) [14]. The simplified method was tested in a multiple experimental case study of spatial planning projects in three municipalities [15]. The method included three workshops with representatives from various disciplines in the municipalities and was presented in detail by Carstens et al. [15]. The method used common tools such as an Excel sheet, a map projected on a screen, and various PowerPoint presentations. All workshops were conducted during 2017 and 2018, led by members of the research team, who also did substantial work between the workshops. The results were adaptation pathways for vulnerable objects and a plan for implementing them [15]. On the basis of the participants’ comments, they felt more confident about dealing with these uncertainties now than before and appreciated the capabilities of the method to facilitate cross-professional interactions. On the downside, the method required substantial economic and human resources, and methodological knowledge. The participants showed a preference for static solutions, as they perceived that the possibilities were low for enforcing future actions in the plans [15]. However, no systematic evaluation of the participants’ opinions was made.
Following the above mentioned two studies, three analyses were performed of the challenges of applying robust decision support methods in the overall Swedish context and their municipalities. The conclusion was that important challenges included current national planning legislation and the lack of resources at the local level. In addition, the communication of uncertainties associated with climate change from relevant national authorities was found to be inadequate, partly because the municipalities felt uncomfortable with them. Local politicians were less concerned with climate change than civil servants, which made it difficult for the latter to argue for resources for CCA.
The challenges for using robust decision support methods in CCA were further explored by [16], who argued that even in a stable country like Sweden, organisational instabilities pose a substantial challenge to the mainstreaming of flexible approaches to climate adaptation. Therefore, the authors concluded that one cannot assume that flexible approaches to climate adaptation will lead to greater robustness, as there is a substantial risk that planned decisions will not be properly implemented and monitored [16]. Finally, a study based on interviews with the civil servants who attended the three workshops when the robust decision support method was used was conducted in 2018 and 2019 [17]. Among the interviewees was a growing realisation that the current working practice for CCA was unsustainable, given the challenges posed by climate change. Many respondents viewed the current national guidelines as blunt instruments that did not match their needs. Identifying local vulnerabilities (working bottom-up), as was carried out in the three workshops, was perceived as useful, but there was a reluctance to adopt a local worst-case scenario when presenting the results to colleagues higher up the decision chain, as this could be viewed as overly negative. The participants pointed out that it would be difficult for politicians to commit to an expensive investment in an event that would likely not occur during their term [17].

3. Materials and Methods

3.1. Tool Development and Testing

On the basis of the existing method of ref. [15], a new web application was created to support users in their process implementation. The aims of the web application were as follows: (a) to facilitate the process of working with larger datasets, (b) to include geographical data layers, and (c) to enable the analysis of complex information arising from the data gathering.
The tool development followed a user-centred participatory design process [18,19] carried out together with the five participating municipalities. The development was performed iteratively by following agile principles [20], starting with an early working prototype that adapted the existing prototype developed during the workshop into a web application, followed by continuous iterations in parallel with the testing performed in the municipalities (See Figure 1).
The tests followed the same process as the existing method but with the support of the created Tool, with three in-person workshops in each municipality. During the workshops, a team of relevant employees from the municipality were present to brainstorm, discuss, and write down the content as follows:
  • The first workshop defined the focus, the success criteria for climate adaptation, and the unwanted events that could endanger them.
  • The second workshop focused on deciding the minimum acceptable time between flooding, identifying vulnerable locations, and proposing possible CCA measures.
  • The third and final workshop focused on discussing and evaluating the different pathways for implementing the proposed CCA measures (which measures and in which order).
The municipalities that participated are all located in the Baltic Sea and work actively with water-related issues and flooding. In the future, sea levels in these municipalities will rise, which they need to consider in their planning. The municipalities were recruited through existing networks established during the ROBUST research programme.
For testing the Tool, the municipalities selected a plan within the Planning and Building Act that they wanted to work with and where they needed to prevent and adapt the area for the risk of future sea level rise. One municipality then chose a comprehensive plan, two municipalities chose in-depth comprehensive plans, and two municipalities detailed the development plans. The plans they chose to work with were all in the early phase of the planning process and were thus suitable for identifying the risks in the area.
People with different expertise were invited to the workshops, with 5 to 20 people participating in each workshop. During these workshops, we worked with pen and paper in groups, and the results were gradually brought into the Tool’s workspace. The municipalities had different tactics for using the tool during the workshops; some had only one person responsible for entering information, whereas others had several. During all workshops, staff from the municipalities managed the tool, albeit with some support from the project crew on some occasions.

3.2. Interviews

Interviews with five civil servants in the five municipalities who tested the tool were conducted in January 2023. The municipalities chose the respondent, and the interviews were semi-structured [21] and conducted in Swedish, with an interview guide that corresponded to the following research questions:
  • Has the perceived need for CCA in the municipalities increased compared with around 5 years ago, and if so, why?
  • What are the Tool’s pros and cons, including how it could be developed?
  • Can municipalities handle flexible CCA solutions?
At the end of each interview, we asked the respondent for additional reflections with which some contributed. The interviews were recorded and transcribed, after which the transcriptions were checked manually against the recordings and corrected when needed. Citations were identified and translated into English. In Table 1, we summarise information about the respondents and the context in which the Tool was tested.

4. The Tool

The web application translated the existing method of working for RDM in a digital format and allows the municipalities to identify, create, and work with vulnerable locations for planning and adaptation strategy. The application also added a map layer showing the extent of sea level rise from 1 to 5 m in half-metre intervals, which was created by the Swedish Civil Contingencies Agency.
The web application was built in Python using Geo-Django 4.0.3 as a framework, PostgreSQL as database, and Leaflet and OpenStreetMap through django-leaflet 0.28.2 for serving the map tiles. The web application was deployed in a Debian 11 (bullseye) server using Apache 2.4.
The web includes the following sections that the municipalities fill in:
  • General information:
    The geographical area affected.
    The aim of the adaptation work.
    The aimed level of protection against sea level rises in metres.
    The current return periods of higher sea levels.
  • Success criteria: Concrete description of what a situation would look like if the aim of the adaptation work was reached. These were qualitatively described in the text.
  • Unwanted events: Events that can happen with higher sea levels that work against the success criteria (see Figure 2). Unwanted events consist of the following:
    A text description.
    Risk tolerance, defined as the acceptable maximum annual exceedance probability of the unwanted event (the risk tolerance was called “MALMÖ” in the Tool).
    A set of vulnerable locations.
  • Vulnerable locations: Buildings, places, or objects vulnerable to unwanted events (See Figure 3 and Figure 4). Vulnerable locations consist of the following:
    A text description.
    Geographical information (point marker or geometrical object).
    Altitude data (metres above sea level).
    Current margin: the amount of sea level rise for which the location is still safe. This is calculated using the altitude, the current return periods of higher sea levels, and the MALMÖ of the corresponding unwanted event.
    Possible adaptation measures.
  • CCA measures: Possible measures connected to one or more vulnerable locations to lower the risk of damage due to higher sea levels. These measures can be either general for all locations or specific for certain locations. The mitigation actions consist of the following:
    A text description.
    A cost estimation.
    An estimation of sea level rise, which they are effective for.
  • Pathways between different CCA measures: Different action lines in time that connect different CCA measures in an adaptation pathway or plan. The visualisation allows for the addition of different combinations of adaptation measures (for either a particular location or the whole area) to compare different possible paths.
The municipalities filled these sections on the website during, and between, the workshops following the workshop instructions. The first step was adding the general information, followed by adding success criteria and unwanted events. From the unwanted events, the vulnerable locations were identified and marked in the geographical interface. Once these sections were filled, the municipalities could start the work with identifying possible CCA measures and pathways. The created data are saved in a relational database, which means that the data are interconnected, allowing for queries such as searching for all vulnerable locations connected to a certain unwanted event or for vulnerable locations that are not covered by a certain CCA measure.
The data can also be exported as follows:
  • Tab-delimited CSV file to be used in a spreadsheet
  • Directly from the database in the municipalities’ own GIS systems.

5. Interview Results

5.1. Perceived Need for CCA in the Municipalities Compared with That 5 Years Ago and the Reasons for the Need

Four of five respondents perceived that the interest in working with CCA in their municipalities had increased during the past 5 years for multiple reasons ranging from local experiences of flooding to new political leadership and the new synthesised CCA knowledge presented by the IPCC [1].
Municipality A is a good example of how extreme weather events can cause an increase in interest and willingness to work with CCA. According to our source from the municipality, the weather changes are now becoming obvious, exemplified by the early melting of snow and torrential rains that lead to inundation that affect many residents:
“But now, you are starting to see that the problems we have are today. We now have a spring flood that affects watercourses in the municipality where there have even been floods in some places.”
(Respondent from municipality A)
However, some climate impacts such as sea level rises are still considered problems of the future by some local politicians in municipality A. Nevertheless, nowadays, detailed plans always consider water-related impacts. One plot located by the sea, now used for industrial development, is already at high risk of avalanches, which will be exacerbated by sea level rises. Therefore, municipality has applied for funds from national authorities to stabilise the ground because of the high costs involved.
Another example of where extreme weather has triggered CCA is in municipality C. Here, the need for CCA has increased significantly during the past few years, primarily owing to the heavy precipitation that caused flooding in the municipality during the summer of 2021. In 2 h, there was 120 mm of downpour, which is very extreme for Sweden and caused “greater pressure from the public with questions” to the politicians:
“The need… has become bigger in recent years after the downpour, which was also the fact that the demand for climate adaptation measures has become actualised in a completely different way than before.”
(Respondent from municipality C)
The increased availability of information about the need for CCA has triggered adaptation work in municipality B, where our respondent noted that especially since 2015, information about CCA has escalated:
“During the 10 years, it has gone at rocket speed, and then maybe it started to become more and more, it feels like, somewhere around the last 5 years.”
(Respondent from municipality B)
In municipality E, the interest in working with CCA has increased recently. Earlier, in 2015, substantial CCA investigations were made when the comprehensive plan was updated. Still, the results were never presented in the same plan for unknown reasons.
“So they did serious investigations, but they then slowed down. They did not really present the results any closer in the comprehensive plan; other than that, they added a reserve for flood protection for rising sea levels. Then it has lain and hibernated….”
(Respondent from municipality E)
Nowadays, there are several large development projects inside the main city of municipality E, located by the sea, and our respondent emphasised the need for long-term planning, also considering the impacts of high or extreme scenarios for future sea level rises.
The only municipality where the respondent had not noticed any higher need for CCA during the past 5 years (municipality D) is one of the most flood-prone municipalities in Sweden. Here, inundation from the sea and water courses prompted substantial CCA measures more than 10 years ago, with the full support of local politicians:
“There is a political consensus that this is something that we should work with, and it has been around for quite some time.”
(Respondent from municipality D)
According to our informant in municipality D, better information and knowledge sharing nowadays has simplified and emphasised the CCA work:
“Actually, it has not changed in general (the need for CCA in the municipality, our comment), but it is clear that the state of knowledge has. It is constantly increasing our collective knowledge of scientific and societal knowledge about climate change.”
(Respondent from municipality D)

5.2. Reflections about the Tool

5.2.1. Praises and Advantages

When allowed to give their spontaneous impressions from using the Tool, all interviewees expressed enthusiasm, but for various reasons. One reason was the opportunity to collaborate with colleagues in a multidisciplinary team where civil servants with various expertise could meet and discuss the vulnerability of objects using a common map. Another was that the Tool represented a new way of planning, starting from a vision of what should be achieved with a bottom-up perspective. The third reason was the advantage of not using the time perspective when addressing CCA.
One example of the opportunity to gather a multidisciplinary group of colleagues that sparked enthusiasm comes from municipality A, where the respondent highlighted the importance of dialogue as the main positive outcome when using the Tool:
“A dialogue between different units in the municipality… we saw the purpose of this to develop a group where we can find a collaboration… We used these workshops and this Tool as a way to sit down and discuss problems around climate adaptation.”
(Respondent from municipality A)
Moreover, in municipality B, especially the opportunity to gather staff with different competences to discuss local CCA was appreciated:
“Maybe just to have a reason and gather so many people in a room and discuss this issue as well and use a place as a little experimental test bed… it was very rewarding.”
(Respondent from municipality B)
In municipality C, the Tool, which was used to facilitate joint discussions with colleagues with varied expertise, received much praise:
“They felt that just sitting together and predicting together that it becomes very valuable as well and that this is what is the great strength of this.”
(Respondent from municipality C who asked their colleagues for their impressions after working with the Tool)
The Tool itself was also appreciated and judged as quite functional despite the fact that the municipalities only used a prototype:
“I think this Tool can really stand on its own two feet so you could use it as a kind of climate adaptation Tool in general.”
(Respondent from municipality B)
Other aspects of the Tool that were appreciated were that it permitted extensive collection of local data that could then be manipulated with different outcomes in terms of, for example, margins to risky sea level rise:
“The biggest benefit of the Tool I would say is that it’s probably the ability to collect this data and twist and turn it into so many different perspectives.”
(Respondent informant from municipality B)
In municipality C, the Tool was also appreciated for ignoring the timeline of sea level rise and instead focusing on various water levels and their impacts on vulnerable objects. This was considered a fruitful approach, as it can handle the uncertainty of sea level rise scenarios that show vastly different outcomes:
“That you actually skip the time perspective, I also think that is a strength.”
(Respondent informant from municipality C)
In municipality D, the thoughts about the Tool were like those in municipality C in that the novel approach (bottom-up) was considered as an advantage in itself, as it challenges conventional planning routines, which mostly use a top-down approach:
“But I think the greatest benefit is to think in a different way…. See what different paths there are and to plan in a way that where you do not lock yourself into something but where there are different paths of action… I think you get a different approach. A different approach broadens the horizons.”
(Respondent from municipality D)
In municipality E, the respondent first knew about the Tool during a presentation by the researchers of the project ROBUST research program (see Section 2.3). The impression was already positive at that point, as the respondent felt that this was a very exciting arrangement and especially appreciated the so-called subway maps. The same respondent pondered over the Tool, which was tried out in the municipality, as follows:
“Actually, it’s not rocket science, but it makes you think…. Once you get it, it’s super… It’s almost rudely simple in a way….
(Respondent from municipality E)
The same respondent also used the Tool after three workshops and developed Excel sheets with additional information about the vulnerable objects in the municipality. The information in these sheets was about which department in the municipality was responsible for taking which CCA measures at which objects or areas at risk of chemical pollution in case of inundation. The same respondent had entered information about 130 vulnerable objects or areas into the Tool.

5.2.2. Critiques and Potential for Improvement

There were also criticisms and suggestions for improvements of the Tool. The main critique was that the Tool did not include the impacts of inundations from heavy precipitation, only those of rising sea levels. For the municipalities, the impacts of torrential rains occur here and now, while those of sea level rises take time to occur, as municipality A expressed:
it (the Tool, our comment) is about rising sea levels and not what is perhaps more acute. Just the problem of flooding in connection with torrential rain or poor drainage and things like that. It’s a bad Tool because it doesn’t work that way.”
(Respondent from municipality A)
Municipality C also criticised the Tool for not including impacts from torrential rains where water reach enclosed areas, as this is an issue already occurring today. Herein, this critique was formulated as a suggestion for improvement:
“We had more difficulty dealing with torrential rain that can reach enclosed areas from completely different directions… So it’s probably a small improvement proposal that could be brought in.”
(Respondent from municipality C)
In municipality B, the critique was quite surprising, as the respondent thought that the whole process was too digitalised and that some parts such as the workshops would benefit from using other more old-fashioned methods such as paper and pens. They thought that such methods would foster higher-quality interactions among the participants rather than everybody looking at a screen. However, the same respondent was positive about using the Tool for summarising the discussions among the participants.
“One should try to find some way that doesn’t use so much computers and monitors… It is almost easier to have paper and pen and glue and a map and write in some tables by hand.”
(Respondent from municipality B)
Another criticism/suggestion for improving/strengthening the Tool mentioned by our respondents was the challenge to recruit colleagues with adequate competence for the workshops. Such recruitment is essential for the success of the exercise, although it has nothing to do with the Tool itself. Our respondents were aware of this, as exemplified by a respondent from municipality C, who emphasised the need to clearly present the positive outcomes from using the Tool when recruiting participants:
“Finding a person with the right competence and being able to provide resources for them regarding the way the municipality is organised… you need to be quite specific so that the line manager understands what the benefit might be?”
(Respondent from municipality C)
The suggestions for improvements were also technical, such as being able to move backwards in the Tool more easily, being able to send selected information by mail, and incorporating topographical information in the Tool. All these improvements are related to user-friendliness.
However, all respondents judged the Tool as user-friendly even at the prototype stage and did not foresee any need for help from consultants to run an ameliorated version of the Tool. This was considered important, as consultants are expensive. Nevertheless, some guidance is necessary, as expressed by the respondent in municipality E:
“I think the Tool is so simple that you could work with it. Perhaps you could get a little introduction film.”
(Respondent from municipality E)
There were also suggestions for activities in connection to the use of the Tool that would be beneficial to the municipalities. One respondent suggested arranging a meeting where the municipalities could share their experiences from using the Tool:
“I think it would be really interesting to also get some results from what has happened in the other municipalities and what they have done and what they have come up with.”
(Respondent from municipality A)
Another suggestion was to include tutorial examples of different ways of using the Tool, with three or more workshops (respondent from municipality D)
It was also pointed out that if the Tool was to be used, it requires qualified support to keep it updated, as new laws are passed and new research is published:
“And then you need support based on the fact that someone needs to update the Tool and keep it with whatever it may be, changes in laws and regulations and so on. But above all, I would say that support. And that’s perhaps the most important thing. And also, of course, not only linked to laws but to the latest research and so on.”
(Respondent from municipality D)
The respondent from municipality E suggested that the Tool should be developed further so that the information about vulnerable objects and places could be sorted on variables such as CCA responsible departments in the municipality and being or not being objects or places where there is a risk for chemical pollution. They called that “cutting the sausage along the other side”, which means that they wanted to quickly (by pushing a button) be able to sort out, for example, all objects or places where a certain department is responsible for measures or that belong to a certain geographical area. The same respondent also wished that the Tool would include a function where the social, economic, and environmental benefits of CCA measures were included:
“That you actually see the environmental, economic, and social, so in some way that you can summarise it with some plus or minus….”
(Respondent from municipality E)
All informants said that they would be willing to pay for using a developed version of the Tool, albeit without mentioning how much. Several respondents seemed to have started liking the Tool and were eager for us to find a solution for where to host it while acknowledging the accompanying challenges:
“I’m really afraid that it (the Tool, our comment) will disappear.”
(Respondent from municipality E)

5.3. Can Municipalities Handle Flexible Measures?

The respondents were somewhat optimistic about the municipalities’ capacity to handle flexible CCA measures, provided that the results from using the Tool were incorporated into the planning documents that municipalities are required to produce and update on a regular schedule.
One example comes from municipality A, where the respondent considered that flexible solutions could be handled despite politicians and civil servants coming and going. The solution was that all CCA suggestions should be documented in, for example, the comprehensive plan, which is a document with legal status; therefore, it will not be neglected.
“Yes, I think we can probably do that (handle flexible measures, our comment). At the same time, it is the case that we people come and go within the municipality. The politicians also come and go, and it is important that you also have a basis in why you made that particular decision.”
(Respondent from municipality A)
In municipality B, the respondent was also positive and appreciated that the Tool enabled the users to identify areas that should be protected from exploitation, as they may be needed for building, for example, walls in the future. They also emphasised that the results from using the Tool should be well documented, as in a detailed plan:
“But what you can do in a detailed plan, as a typical example for this particular area, is to ensure that these critical points are not used for things that make it impossible to use them as a protective wall. And that plan description will always be archived as well.”
(Respondent from municipality B)
In municipality C, however, doubts were raised about the municipal capacity to handle flexible solutions based on the experience of making investigations and plans that were quickly forgotten:
“When we investigate something 5 years later, who will remember that we investigated that and what we came up with? It happens quite often that the municipality investigates the same thing twice two years apart.
(Respondent from municipality C)
In municipality D, there was confidence that flexible solutions could be handled “to a certain extent” in the municipality, giving the example of a wall to protect from sea level rises that can first be built rather low but with a design that would make it possible to raise it later. This, of course, should be documented in a suitable plan.

5.4. Additional Reflections

A couple of informants spontaneously mentioned that communicating future and uncertain climate change impacts to the public and local politicians is a real challenge. In municipality A, the respondent was hesitant to communicate the risk of uncertain sea level rise to those who own property in flood-prone areas because the respondent knew that people who live in the area might become very mad. This is because of the risk of value loss once this consequence becomes known, with the risk of subsequent demands for economic subsidies/reimbursements. Informing those who want to build in flood-prone areas about the risk of inundation is, however, legally requested.
In municipality E, the respondent was aware that the sea level rise scenarios presented by SMHI so far do not consider rapid melting of the big ice sheets due to instability, which was something included only in the IPCC reports from 2021. This makes it difficult to communicate such high-end scenarios, as SMHI is the Swedish authority on weather and climate and their material is supposed to be used by municipalities to inform local politicians and the public. As a result, our respondent in municipality E found it too risky to present these high-end scenarios to local politicians, as the respondent had not yet obtained educational material sanctioned by the national authority:
“We have not dared to raise it to the politicians. Then it is very difficult to do this pedagogically based on the data SMHI has today….”
(Respondent from municipality E)

6. Discussion and Conclusions

In this study, we developed and tested an online tool (the Tool) for DMDU, together with civil servants from five Swedish municipalities located by the sea. We based the development of the Tool and the tests on substantive previous research about how municipalities can or cannot use methods for DMDU when facing uncertain sea level rises and on research about the organisational setting in Sweden. The rationale for developing and testing the Tool is that there is an important knowledge gap regarding the evidence of the usefulness of methods that can help decision-makers in DMDU [5]. Our study provides insight into this, although only one method, which was manifested in the Tool, was tested. In this paper, we answer questions about not only the usability of the Tool itself but also the perceived urgency for CCA and the ability of municipalities to handle decisions about flexible adaptation solutions. Such solutions may be decided upon today given a certain outcome such as sea level rise but will be implemented only in the future.

6.1. Perceived Urgency for CCA

Concerning the question about the perceived urgency for CCA, most respondents confirmed that it has increased during the past 5 years. This is not surprising, as extreme weather events have become more common around the globe, which have led to outcries such as the one from the current UN secretary that climate change is the most important threat to the global economy [22] and the steep increase in the number of climate emergency declarations by jurisdictions and local governments [23]. In fact, it would have been surprising if our respondents had not acknowledged the increased need for CCA, but then again, Sweden is not located in the most vulnerable zones for climate change. This also reflects in the answers in our interviews where a very high urgency to deal with CCA was seldom expressed. Nevertheless, the moderate urgency for CCA still makes us believe that further developing the Tool, which is more time-consuming to use than ordinary planning methods, is worthwhile.

6.2. Learned Lessons about the Tool

Our interview respondents rated the usability of the Tool with high scores, although there was room for improvement. They especially appreciated the fact that civil servants with many different expertise participated in using the Tool, thus pooling together knowledge in a way that was unusual but very appreciated. Avoiding working in “silos” was also stressed by the IPCC [24], which stated that there is a “rapidly narrowing opportunity to enable climate-resilient development” (ibid, p. 26) and that such development includes both mitigation and adaptation measures. Among the conditions that constrain such development are “siloed” responses, which means that teams and experts with different competencies are working in isolation from each other (ibid p. 26). Some who tested the Tool considered it challenging to gather such a group of people with different expertise, but once this had been accomplished, the experience of working together was thus perceived as very positive by all the respondents. Working in non-siloed conditions is obviously advantageous not only to researchers but also to practitioners. Planning methods that facilitate this are needed, and their advantages should be communicated to politicians and department heads so that they agree to allocate staff time for such endeavors. This may be an important point when planning to use CCA methods for handling DMDU.
Our research, which involved the application of our novel tool and the DMDU process, can be viewed as an “agile” approach to DMDU. Agile innovation methods have a long history but are made popular in software development and are increasingly embraced in other industries and applications [25]. Agile methods make initiating a project simple and focus on an iterative process of continuous improvement. Agile methods also facilitate the dynamics of customer satisfaction, collaboration, and flexibility. In a similar vein, the method and tool we developed and tested in this project enables an iterative process, as loosely connected teams participating in a long process can document their progress by, for example, recording points of interest and associated risk assessments, allowing for ongoing review and adjustment.
The iterative process of DMDU approaches is commonly stressed by experienced DMDU experts (e.g., [10]), and our method is in line with the thinking of DMDU methods. Our tool’s continuous documentation and reassessment can be considered as enabling continuous improvement. This ability to revisit and refine analyses and solutions aligns with the agile philosophy of rapidly responding to change by following a fixed plan. When applied appropriately, we believe that agile DMDU approaches such as the method and tool we presented in this paper have the potential to significantly enhance the success rates and productivity of teams planning for climate adaptation.

6.3. Further Work

According to our informants, one drawback of the Tool was that it did not offer any economic evaluation of the CCA measures proposed during the process that were made with, for example, a cost–benefit analysis. In addition, they indicated that this is a serious disadvantage, as such information is crucial when presenting CCA measures and pathways to decision-makers. However, synthesised research concludes that cost–benefit analysis remains limited in its ability to represent all damages from climate change, including non-monetary ones [24]. Thus, adding this function to the Tool may be difficult at present but an interesting track to explore in the future.
Another reflection from the workshops is that discussions about maladaptation were missing and that the Tool itself does not contain any module encouraging this. When CCA measures for sea level rise were discussed, sea walls were often proposed. However, sea walls can be maladaptive in that they reduce impacts in the short term but can result in lock-ins and increased exposure to climate risks in the long term [24], (p. 44). Adding a module that encourages a discussion about the risks of maladaptation to the Tool when discussing sea walls may be a step in the right direction for further development of the Tool.
Concerning the capability of municipalities to handle flexible CCA solutions, research has shown that sustained adaptation actions are strengthened by mainstreaming adaptation into institutional budget and policy planning cycles [24], (p. 75). Our informants also emphasised that using the Tool should be integrated into one of the planning processes required by Swedish law (comprehensive, in-depth comprehensive, and detailed plans) for decisions about CCA measures to be implemented. Stressing the fact that the Tool should always be used as part of such processes is an important conclusion from our study and should also be relevant when using other DMDU methods. Whether or not relevant institutions will remain flexible and strong in accounting for long-term goals, which is a prerequisite for successful adaptation [5], remains to be seen. If municipalities in Sweden, which is a very stable and rich country, cannot handle flexible CCA solutions, the prospects for using DMDU methods seem bleak.

6.4. Conclusions

The results presented in this paper contribute to the ongoing research about CCA by demonstrating a case where DMDU methods can be put into practice at the local level in a way which is compatible with practitioners needs, capabilities, and resources. According to the IPCC, there is a dire need for a transformation of society to curb and adapt to climate change, something that “may require actions that disrupt moral or social boundaries and structures that are perpetuating unsustainable systems and pathways”. It is expected that “extreme events and long-term climatic changes can trigger a realigning of practices, politics, and knowledge” [26], (p. 2668). This harsh message has not yet landed in the parts of Swedish society with which we are familiar, mostly regional and local authorities.
Our case was no doubt limited (only sea-level rise and only five municipalities) and many more are needed before a conclusion about the usability of DMDU methods among practitioners can be drawn. However, we think that by introducing the Tool and thereby DMDU methods, we have expanded the view with which uncertainties can be dealt with, which may be quite useful for the transformation challenge ahead. This study provides useful insights into how a digital tool creates possibilities of working with bigger datasets in a more agile way to help the municipalities in their planning. Expressing this conclusion effectively, one informant indicated that “the greatest benefit is to think differently”.

Author Contributions

Conceptualization, all authors; methodology, all authors; software, J.L.Z.; validation, C.H. and P.W.-S.; formal analysis, A.C.K.; investigation, all authors; resources, C.H.; data curation, J.L.Z.; writing—original draft preparation, A.C.K. and J.L.Z.; writing—review and editing, A.C.K., J.L.Z. and P.W.-S.; visualization, J.L.Z.; supervision, A.C.K.; project administration, C.H.; funding acquisition, C.H. All authors have read and agreed to the published version of the manuscript.

Funding

Funding was granted from Vinnova, Sweden.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was approved by all respondents.

Data Availability Statement

Data are available on request from the corresponding author.

Conflicts of Interest

Authors Annika Carlsson Kanyama and Chatarina Holmberg were employed by Ecoloop AB. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

  1. Intergovernmental Panel on Climate Change (IPCC). Summary for Policymakers. In Climate Change 2023: Synthesis Report; Lee, H., Romero, J., Eds.; Intergovernmental Panel on Climate Change: Geneva, Switzerland, 2023. [Google Scholar]
  2. Lempert, R.; Popper, S.; Bankes, S. Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis; MR-1626-RPC; RAND Corp.: Santa Monica, CA, USA, 2003. [Google Scholar]
  3. Intergovernmental Panel on Climate Change (IPCC). Annex II: Glossary. In Climate Change 2022: Impacts, Adaptation and Vulnerability; Pörtner, H.-O., Roberts, D.C., Tignor, M., Poloczanska, E.S., Mintenbeck, K., Alegría, A., Craig, M., Eds.; Cambridge University Press: Cambridge, UK, 2022; pp. 2897–2930. [Google Scholar]
  4. Intergovernmental Panel on Climate Change (IPCC). IPCC Special Report on the Ocean and Cryosphere in a Changing Climate; Pörtner, H.-O., Roberts, D.C., Masson-Delmotte, V., Zhai, P., Tignor, M., Poloczanska, E., Mintenbeck, K., Alegría, A., Nicolai, M., Okem, A., et al., Eds.; Cambridge University Press: Cambridge, UK; New York, NY, USA, 2019; p. 755. [Google Scholar]
  5. New, M.; Reckien, D.; Viner, D.; Adler, C.; Cheong, S.-M.; Conde, C.; Constable, A.; de Perez, E.C.; Lammel, A.; Mechler, R.; et al. Decision-Making Options for Managing Risk. In Climate Change 2022—Impacts, Adaptation and Vulnerability: Working Group II Contribution to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change; Cambridge University Press: Cambridge, UK, 2023; pp. 2539–2654. [Google Scholar]
  6. Stanton, M.C.B.; Roelich, K. Decision Making under Deep Uncertainties: A Review of the Applicability of Methods in Practice. Technol. Forecast. Soc. Chang. 2021, 171, 120939. [Google Scholar] [CrossRef]
  7. Haasnoot, M.; Kwakkel, J.H.; Walker, W.E.; ter Maat, J. Dynamic adaptive policy pathways: A method for crafting robust decisions for a deeply uncertain world. Glob. Environ. Chang. 2013, 23, 485–498. [Google Scholar] [CrossRef]
  8. Hilty, L.M.; Aebischer, B. ICT for Sustainability: An Emerging Research Field. In ICT Innovations for Sustainability. Advances in Intelligent Systems and Computing; Lorenz, M.H., Aebischer, B., Eds.; Springer: Cham, Switzerland, 2015; pp. 3–36. [Google Scholar]
  9. Zapico, J.L.; Brandt, N.; Turpeinen, M. Environmental Metrics. J. Ind. Ecol. 2010, 14, 703. [Google Scholar] [CrossRef]
  10. Marchau, V.A.; Walker, W.E.; Bloemen, P.J.; Popper, S.W. Decision Making under Deep Uncertainty: From Theory to Practice; Springer Nature: Berlin/Heidelberg, Germany, 2019. [Google Scholar]
  11. SCB (Statistics Sweden). Counties and Municipalities. Available online: https://www.scb.se/en/finding-statistics/regional-statistics/regional-divisions/counties-and-municipalities/ (accessed on 8 May 2023).
  12. The Swedish Government. Regeringens Proposition 2017/18:163. Nationell Strategi för Klimatanpassning. 2018. Available online: https://www.regeringen.se/rattsliga-dokument/proposition/2018/03/prop.-201718163 (accessed on 7 August 2023).
  13. SMHI. Climate Change Adaptation—Information from Swedish Authorities. Swedish Portal for Climate Change Adaptation. Available online: https://www.klimatanpassning.se/en (accessed on 9 May 2023).
  14. UNESCO. Climate Risk-Based Decision Analysis (CRIDA). Available online: https://en.unesco.org/crida (accessed on 8 May 2023).
  15. Carstens, C.; Sonnek, K.M.; Räty, R.; Wikman-Svahn, P.; Carlsson-Kanyama, A.; Metzger, J. Insights from Testing a Modified Dynamic Adaptive Policy Pathways Approach for Spatial Planning at the Municipal Level. Sustainability 2019, 11, 433. [Google Scholar] [CrossRef]
  16. Metzger, J.; Kanyama, A.C.; Wikman-Svahn, P.; Sonnek, K.M.; Carstens, C.; Wester, M.; Wedebrand, C. The Flexibility Gamble: Challenges for Mainstreaming Flexible Approaches to Climate Change Adaptation. J. Environ. Policy Plan. 2021, 23, 543–558. [Google Scholar] [CrossRef]
  17. Wester, M. Robust Municipal Decision Making? A Pilot Study of Applying Robust Decision Making in Three Swedish Municipalities. J. Environ. Plan. Manag. 2022, 65, 745–758. [Google Scholar] [CrossRef]
  18. Gulliksen, J.; Göransson, B.; Boivie, I.; Blomkvist, S.; Persson, J.; Cajander, Å. Key Principles for User-Centred Systems Design. Behav. Inf. Technol. 2003, 22, 397–409. [Google Scholar] [CrossRef]
  19. Spinuzzi, C. The Methodology of Participatory Design. Technol. Commun. 2005, 52, 163–174. [Google Scholar]
  20. Beck, K.; Beedle, M.; Van Bennekum, A.; Cockburn, A.; Cunningham, W.; Fowler, M.; Grenning, J.; Highsmith, J.; Hunt, A.; Jeffries, R. Manifesto for Agile Software Development. 2001. Available online: https://ai-learn.it/wp-content/uploads/2019/03/03_ManifestoofAgileSoftwareDevelopment-1.pdf (accessed on 8 February 2024).
  21. Bryman, A. Social Research Methods; Oxford University Press: Oxford, UK, 2012. [Google Scholar]
  22. UNFCCC. António Guterres: Climate Change Is Biggest Threat to Global Economy. 2023. Available online: https://unfccc.int/news/antonio-guterres-climate-change-is-biggest-threat-to-global-economy (accessed on 13 July 2023).
  23. Climate Emergency Declaration. Climate Emergency Declarations in 2339 Jurisdictions and Local Governments Cover 1 Billion Citizens. 2023. Available online: https://climateemergencydeclaration.org/climate-emergency-declarations-cover-15-million-citizens/ (accessed on 13 July 2023).
  24. IPCC. Climate Change 2023: Synthesis Report. A Report of the Intergovernmental Panel on Climate Change; Contribution of Working Groups I, II and III to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change; IPCC: Geneva, Switzerland, 2023. [Google Scholar]
  25. Rigby, D.K.; Sutherland, J.; Takeuchi, H. Embracing Agile. Harv. Bus. Rev. 2016, 50, 40–48. [Google Scholar]
  26. Intergovernmental Panel on Climate Change (IPCC). Climate Resilient Development Pathways. In Climate Change 2022—Impacts, Adaptation and Vulnerability: Working Group II Contribution to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change; Cambridge University Press: Cambridge, UK, 2023; pp. 2655–2808. [Google Scholar]
Figure 1. Research flow diagram.
Figure 1. Research flow diagram.
Sustainability 16 02044 g001
Figure 2. View presenting an unwanted event and the related vulnerable places (marked in red) (translated dummy account).
Figure 2. View presenting an unwanted event and the related vulnerable places (marked in red) (translated dummy account).
Sustainability 16 02044 g002
Figure 3. View presenting all the created vulnerable places (marked in red for areas and blue markers for single points) and an extra layer showing the sea level rise (translated dummy account).
Figure 3. View presenting all the created vulnerable places (marked in red for areas and blue markers for single points) and an extra layer showing the sea level rise (translated dummy account).
Sustainability 16 02044 g003
Figure 4. View presenting all vulnerable places to allow for sorting and analysis of places that need measures (translated dummy account).
Figure 4. View presenting all vulnerable places to allow for sorting and analysis of places that need measures (translated dummy account).
Sustainability 16 02044 g004
Table 1. Information about the respondents and the type of plan where they tested the Tool.
Table 1. Information about the respondents and the type of plan where they tested the Tool.
Title of the RespondentsMunicipalityType of Plan Where the Tool Was Tested
Sustainability strategistAIn-depth comprehensive plan
PlannerBDetailed plan
Environmental investigatorCIn-depth comprehensive plan
Climate strategistDDetailed plan
StrategistEComprehensive plan
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Carlsson Kanyama, A.; Zapico, J.L.; Holmberg, C.; Wikman-Svahn, P. “The Greatest Benefit Is to Think Differently”: Experiences of Developing and Using a Web-Based Tool for Decision-Making under Deep Uncertainty for Adaptation to Sea Level Rise in Municipalities. Sustainability 2024, 16, 2044. https://doi.org/10.3390/su16052044

AMA Style

Carlsson Kanyama A, Zapico JL, Holmberg C, Wikman-Svahn P. “The Greatest Benefit Is to Think Differently”: Experiences of Developing and Using a Web-Based Tool for Decision-Making under Deep Uncertainty for Adaptation to Sea Level Rise in Municipalities. Sustainability. 2024; 16(5):2044. https://doi.org/10.3390/su16052044

Chicago/Turabian Style

Carlsson Kanyama, Annika, Jorge Luis Zapico, Chatarina Holmberg, and Per Wikman-Svahn. 2024. "“The Greatest Benefit Is to Think Differently”: Experiences of Developing and Using a Web-Based Tool for Decision-Making under Deep Uncertainty for Adaptation to Sea Level Rise in Municipalities" Sustainability 16, no. 5: 2044. https://doi.org/10.3390/su16052044

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop