Multicriteria Spatial Decision Support Systems for Future Urban Energy Retrofitting Scenarios

Abstract

Nowadays, there is an increasing concern about sustainable urban energy development taking into account national priorities of each city. Many cities have started to define future strategies and plans to reduce energy consumption and greenhouse gas emissions. Urban energy scenarios involve the consideration of a wide range of conflicting criteria, both socio-economic and environmental ones. Moreover, decision-makers (DMs) require proper tools that can support their choices in a context of multiple stakeholders and a long-term perspective. In this context, Multicriteria Spatial Decision Support Systems (MC-SDSS) are often used in order to define and analyze urban scenarios since they support the comparison of different solutions, based on a combination of multiple factors. The main problem, in relation to urban energy retrofitting scenarios, is the lack of appropriate knowledge and evaluation criteria. The latter are crucial for delivering and assessing urban energy scenarios through a MC-SDSS tool. The main goal of this paper is to analyze and test two different methods for the definition and ranking of the evaluation criteria. More specifically, the paper presents an on-going research study related to the development of a MC-SDSS tool able to identify and evaluate alternative energy urban scenarios in a long-term period perspective. This study refers to two Smart City and Communities research projects, namely: DIMMER (District Information Modeling and Management for Energy Reduction) and EEB (Zero Energy Buildings in Smart Urban Districts).

1. Introduction

Today, there is a large concern about green building design and the reduction of greenhouse gas emissions in cities. Indeed, the highest amount of energy usage belongs to cities, accounting for 32% of global final energy consumption, and it is expected that this number will increase in the near future due to the growing urban population [1]. A further element of concern is related to the age of the existing buildings stock since most of them are dated back to the 1970s, leading to low energy performances. Consequently, appropriate retrofitting strategies are needed due to the low demolition rate of existing buildings, in order to make successful energy savings targets.

With this in mind, many cities started to define future strategies and plans to reduce energy consumption and greenhouse gas emission [2]. Policies and urban energy scenario development require the consideration of a territorial approach and the analysis of a large stock of buildings and their energy performances [3], rather than the analysis of single building energy efficiency improvement [4].

Cities are dynamic living organisms that are continuously evolving, requiring integrated, collaborative, and inclusive multiple stakeholders and multiple criteria decision processes [5]. Therefore, developing urban energy scenarios for energy transition is a time consuming and delicate decision process that requires a very large number of data and information [6].

Geographic Information Systems (GIS) are recognized as being key players against those tasks mainly due to the extent of required data/information and the presence of multiple stakeholders and criteria with conflicting interests and objectives. GIS can support evaluations and decision processes on the urban scales about the complexity of the related energy strategies scenarios [7], being able to integrate different subsystems and database. Interestingly, GIS can support the decision processes related to the definition of energy urban scenarios identifying critical zones with the use of colored maps [8]. This purpose requires geographical data visualization of the alternative scenarios, producing thematic maps and performing spatial operations [9]; in this sense, a spatial decision support system (SDSS) consist in a system devoted to support the decision processes in spatial problems [10].

In order to properly manage the decision processes related to the definition of urban energy scenarios, multicriteria decision analysis (MCDA) can be applied to consider multiple stakeholder’s point of views, as well as the multiple aspects of the problem in exam. Indeed, MCDA is proved to be a powerful methodology able to consider different aspects of complex situations and to provide priority rankings both in term of alternative scenarios and qualitative/quantitative decision criteria [11].

The synergetic capabilities of SDSS and MCDA are recognized since they can potentially enhance both spatial decision processes, helping to reach a consensus [12]. Accordingly, the Web-based Multicriteria Spatial Decision Support System (MC-SDSS) extends the SDSS tool to include not only the GIS capabilities but also MCDA [13]. One of the main advantages of the MC-SDSS is that they allow the stakeholders expressing their preferences with respect to decision criteria and/or alternative scenarios using GIS-based procedures, which provide feedback, increasing the trust in the results. Moreover, the MC-SDSS are powerful visualization tool through which the maps become ‘visual indices’ offering solutions to the planners to change and optimize the conditions [14].

This paper presents an on-going research related to the development of a MC-SDSS able to provide and evaluate alternative energy urban scenarios in a long-term period perspective. The research study specifically refers to the following two smart urban energy projects: the European DIMMER project (District Information Modeling and Management for Energy Reduction) [15] and the National Smart City and Communities EEB project (Nearly Zero-Energy Buildings in Smart Urban Districts) [16].

The methodological steps involved in this study are: (1) data collection and integration; (2) criteria definition and ranking; (3) scenarios development and evaluation. In particular, the paper focuses on the test of two different methods for the definition and ranking of the decision criteria which are required for delivering and assessing the urban energy scenarios: the Measuring Attractiveness by a Categorical Based Evaluation Technique (MACBETH) [17] and the “Playing Cards” method [18].

The paper is organized as follows: the next section provides an overview of both MCDA and SDSS, including the approaches used to define and rank the evaluation criteria to be used in the decision analysis. Section 3 describes the definition of the decision criteria adopted in the DIMMER and EEB projects, while Section 4 highlights the strengths and weaknesses of the proposed approaches. Finally, Section 5 presents the conclusions and the future development of the research.

2. Methodological Theoretical Framework

MCDA and SDSS are, nowadays, recognized as fundamental tools to define and analyze urban energy retrofitting scenarios since they are able to compare different solutions, based on the combination of multiple factors and criteria [13].

SDSS can be considered as an interactive computer system for assisting the user(s) (i.e., single or group) to efficiently perform decision processes [19]. In this sense, the SDSS is able to visually support the stakeholders during different focus groups and workshops to understand how the criterion trade-offs evolve when one or several decision parameters change [20].

The SDSS acquires, manages and stores the geo-referenced data performing the analysis of spatial problems. Moreover, it provides an interactive environment for performing effective visual activities thanks to the visual interface, which enables a dynamically-interactive session in a real-time exchange of information between the user and the system to support the stakeholders through all decision phases [19]. According to [21] the SDSS should be able to:

• Provide mechanisms for the spatial data input;
• Allow representation of the spatial relations and structures;
• Include the analytical techniques of spatial and geographical analysis; and
• Provide output in a variety of spatial forms, including maps.

The Multicriteria Spatial Decision Support Systems (MC-SDSS) are part of a larger field of SDSS [22]. In the framework of MC-SDSS, two interrelated instruments coexist: the GIS, whose main role is supporting in data storage, management, visualization of maps, and analyzing the decision problems; and the MCDA, which provide a full range of methods for structuring decision problems, and for designing, evaluating, and prioritizing alternative decisions [13].

In the following Section 2.1 and Section 2.2 the main features of both MCDA and SDSS will be highlighted.

2.1. The Multicriteria Decision Analyses (MCDA)

The MCDA are valuable and increasingly widely-used approaches able to help decision-makers (DMs) in making decisions in a structured and intuitive way [11,23]. Despite the diversity of MCDA, the basic ingredients are the following: a finite or infinite set of actions (alternative, solutions, options), a number of decision criteria, and at least one DM or stakeholder. In general terms the MCDA are considered powerful tools able to support decision-making processes where there is a choice to be made between competing alternatives or criteria. Over the years, the MCDA proved to be particularly useful for urban planning, where a complex and inter-connected range of environmental, social, and economic issues must be taken into consideration and where objectives are often competing, making trade-offs unavoidable.

Given the large number of available MCDA approaches, it is necessary to select the most suitable method for the specific decision context [24]. In this paper, we choose to apply two different MCDA methods, respectively: the “Measuring Attractiveness by a Categorical Based Evaluation Technique” (MACBETH) [17] and the “Playing Cards” methods [18]. The choice of applying the two aforementioned methods for the definition and ranking of the decision criteria, which are required for delivering and assessing the urban energy scenarios, is due to a number of reasons.

First, despite the differences existing between the two methods, both of them are considered simple methodologies and easy to be understood, even by those who are not experts in the decision processes [11]. Second, they are able to help DMs in handling values that cannot be easily quantified, involving qualitative judgments Hence, real-life applications show that when one asks the DM what importance he wishes to assign to each decision criterion, he/she expresses his/her preferences spontaneously, without knowing neither the range of the scale nor the procedure used to encode this scale [25]. However, in order to obtain relevant information (output) it is crucial that the output takes into account both the nature and the encoding [26]. This opens a significant debate regarding how to translate qualitative judgements into numerical values, which must reflect the relative importance of the selected evaluation criteria [27]. The MACBETH and the “Playing Cards” methods proved to be relevant in this debate [25]. Third, the technical parameters involved in the two methodologies can be interpreted in an easy way, allowing a simplification of the problem. The results obtained from their application are lists of k-best actions expressed in numerical values to be analysed further by the people involved [17,18]. Finally, the MACBETH method can be supported by the so-called M-MACBETH software, which is compatible with the way of reasoning of the inquired people and with their meaning of useful results. Section 2.1.1 and Section 2.1.2 briefly describe the main characters of the MACBETH and the “Playing Cards” methods. (For a comprehensive analysis of the MACBETH and “Playing Cards” methods please refer to [11,28]).

2.1.1. The MACBETH Method

The MACBETH method is a structured MCDA that was developed in the early 1990s. It is based on the additive value model and requires only qualitative judgments about differences of value to help an individual, or a group, quantify the relative attractiveness of the actions or criteria. Starting from the qualitative judgments requested to the stakeholders, the MACBETH method allows the construction of quantitative values model supporting the interactive learning process about the problem in exam. In this sense, it is able to reduce the “cognitive discomfort” [28] that could arise in the stakeholders when they are asked to express their preferences in a numerical scale. The MACBETH methodology can be divided into three main application phases: model structuring, model evaluating and analyzing the results.

Model Structuring: During the structuring phase, the options to be evaluated and their performances, as well as the values of concern need to be identified and structured in the form of a tree, generally referred to as a “value tree”, offering an organized visual overview of the various concerns at hand.

Model evaluating: After structured the model, MACBETH involves a series of pairwise comparisons, where the stakeholders are asked to specify the difference of attractiveness between all of the alternatives with respect to the criteria. In order to fill in the pairwise comparison matrices, the semantic categories described in

Table 1. Semantic categories of the MACBETH method, Source [17].

Categories	Description
Extremely	Extreme preference of the criterion/option A over the criterion/option B
Very strongly	Very strong preference of the criterion/option A over the criterion/option B
Strongly	Strong preference of the criterion/option A over the criterion/option B
Moderately	Moderate preference of the criterion/option A over the criterion/option B
Weakly	Weak preference of the criterion/option A over the criterion/option B
Very weakly	Very weak preference of the criterion/option A over the criterion/option B
Not at all	No difference in terms of preference

are used.

Analysing the results: Once the model has been structured and filled in, the MACBETH method provides very clear results in the form of ranking allowing identify the attractiveness of the problem’s criteria and alternatives.

The MACBETH method has been applied in a number of case studies related to different fields [29,30]. However, in the emerging field of energy planning, very few examples of application are available in literature. An interesting application is provided by Burton and Hubaceck [31]. The two authors used the MACBETH approach for assessing and comparing eight renewable energy technologies at differing scales, using an official definition of renewable energy provided by the UK government. In particular, the study highlighted the advantages and disadvantages of a number of different renewable energy technologies, concluding that the MACBETH method constitutes a useful support in handling values that cannot be easily quantified. In their study, the authors proved that the decisions reached following the use of the MCDA are likely to be more effective than those realized by using only financial methods; however, this result is reached on the basis of a high conflicting process among stakeholders over the values and weights to be explicated in a MCDA application.

Although this study has been inspired by the work developed by Burton and Hubaceck, in the present paper the MACBETH method is used for defining decision criteria to be adopted in the development of urban energy scenarios rather than comparing different energy technologies on the base of established decision criteria. Furthermore, in this application of the MACBETH method consensus among the stakeholders is reached inside focus groups, while Burton and Hubaceck approached the problem with single interviews.

2.1.2. The “Playing Cards” Method

The “Playing Cards” method is a semi-structured participative procedure suitable to support group discussion. It allows the stakeholders involved to think about and express the way in which they wish to hierarchize the different criteria in a specific context. One of the major advantages of the “Playing Cards” method is the ease of application. This method in fact, consists in associating a ‘‘card’’ with each criterion. Moreover, the stakeholders have a set of “white cards” available, the use of which depends on specific needs. The application of the procedure is very simple: (1) the stakeholders are asked to order the “cards” according to the importance of the criteria (from the last important to the most important one) providing a complete pre-order. If some criteria have the same importance, the stakeholders should build a subset of cards holding them together; (2) according to the fact that the importance of two successive criteria in the ranking can be more or less close, the stakeholders are asked to insert the “white cards” between two successive “cards” (the greater the difference between the mentioned weights of the criteria, the greater the number of white cards) providing a final ranking of the importance of criteria; (3) the final ranking of criteria is converted into weights according to Simos’ algorithm. The fact that the stakeholders involved have to handle the cards in order to rank them allows a rather intuitive understanding of the aim of this procedure [32].

Unfortunately, few applications of this method are available in literature [25,33,34]. Mention has to be made to the study provided by Bottero et al. [33], which proposed an innovative application of the “Playing Cards” method in connection with ELECTRE III [33] in order to compare five urban requalification projects. Although the topic is different and is not related to the energy context, this study constituted an interesting reference highlighting a number of benefits of the “Playing Cards” method, as follows: it is interactive, easy to be understood and accepted by the stakeholders involved. In the application of the “Playing Cards” method, Bottero et al. [33] promoted an individual discussion with the stakeholders. In the present study, on the contrary, the method is applied directly inside a focus group in order to inform the stakeholders and stimulate the discussion. In the urban energy retrofitting context, the present study constitutes one of the first examples.

2.2. Methodological Steps of a MC-SDSS

From a methodological viewpoint, the process needed to define and assess urban scenarios includes the three following phases [19,35]:

• Intelligence phase: the decisional context analysis for structuring and identifying the decision problem should be provided in this phase. Both relevant decision criteria and alternative scenarios should be established, identified and assessed in this phase. The process model includes: (i) data collection and integration; (ii) criteria definition; and (iii) scenario definition;
• Design phase: once the alternative scenarios are defined, it is necessary to carefully choose the most appropriate MCDA method in order to structure the decision model and the evaluation matrix (criteria and alternatives matrix);
• Evaluation and Choice: the selected MCDA method will assess and evaluate the alternative scenarios. During this phase, a sensitivity analysis is suggested in order to examine the consistency of the obtained outcomes and the robustness of the model.

The scenarios definition, therefore, constitutes the final step of a MC-SDSS tool development. In the context of this research project, scenarios analysis is adopted to understand future energy consumption patterns, considering possible refurbishment actions in the built environment. These refurbishment actions consist of: diminishing the energy needs of the building (e.g., window replacement, insulation of the opaque envelope), increasing of the heating system efficiency (e.g., boiler replacement, control improvement), or the use of renewable energy resources on-site (e.g., solar thermal plant or solar photovoltaic (PV) plant installation). Moreover, an attempt was made in this study to introduce different participative scenarios based on the feasibility of retrofit measures, taking into account the influence of socio-economic demographic variables and the capability and desire to invest in renovation measures.

3. Identifying a Coherent Set of Criteria for the MC-SDSS

In this section, a suitable set of criteria to be included in the MC-SDSS for urban energy retrofitting scenarios (from now-on named “Dashboard”) will be presented. This set derives from a collaborative work developed in two different projects: the European project DIMMER and the National Smart City and Communities project Zero Energy Buildings in Smart Urban Districts (EEB). A short explanation of these two projects is provided below, together with the methodologies used for selecting and ranking the final criteria based on focus groups [36]. It is notable to say that the energy consumption reduction and the energy efficiency regulations proposed by the EU legislation were not considered as decision criteria, but as a final objective and target of each scenario. In fact, the recent Second Report on the State of the Energy Union [37] describes the state of the art of energy transition to secure and low carbon sources: adapting and updating existing local energy grids, such as in the DIMMER project, this objective has been enforced. Moreover, the EU has determined a new plan for energy efficiency, setting several policy targets, known as the “20-20-20” in order to reduce greenhouse gas emissions by 20% from 1990 levels [38]. In both mentioned projects, a predictive “what-if” scenarios approach has been taken into account, which are often short/medium term [39]. This approach investigates what will happen on the condition of some specified actions for future development [39]. Therefore, each scenario is defined taking into account the EU targets with the aim at reducing, at a minimum, 20% of total energy consumption.

3.1. The Criteria Set in the DIMMER Project

DIMMER (2013–2016) is a European project coordinated by the Politecnico di Torino, which received funding from the European Union’s Seventh Program for Research, Technological Development and Demonstration under grant agreement no. 609084.

DIMMER consists of a software system destined to energy managers and public authorities to monitor district energy data as well as simulate and implement energy management policies and scenarios at district level. Moreover, thanks to DIMMER, we started to develop the Dashboard, which aims at supporting energy decision processes at a district-scale of intervention.

In order to test and validate the DIMMER innovative system and the Dashboard, both public and private buildings included in urban districts are considered in two different cities: Turin (Italy) and Manchester (The United Kingdom—for more details please refer to the project’s website www.dimmerproject.eu).

During the DIMMER project, we started to define and rank the decision criteria to be used to develop urban energy scenarios. Starting from this, a coherent set of criteria, reflecting the concerns relevant to the decision problem has been identified. The criteria considered in the present application were selected based on the relevant international literature and on the requirements coming from the DIMMER project (

Table 2. Description of the considered criteria (DIMMER project).

Aspect	Criteria	Literature	Description	Unit
Economic	Investment costs	[40]	Investment costs related to refurbishment of buildings (efficiency investment) and new energy sources (infrastructure investment).	Euro
	Payback Period (PBP)	[41]	Performance measure used to evaluate the efficiency of an investment or to compare the efficiency of a number of different investments.	Years
Environmental	Reduction of the CO2 emissions.	[36]	Reduction of the CO2 pollutant emissions.	Percentage
Technical	Reduction of the energy requirement	[34]	Percentage of reduction of the energy requirement due to the buildings’ intervention (coat insulations and windows).	Percentage
	Resilience of the energy system.	[34]	Ability of soak up economy and physical shocks of the energy system.	Ordinal

).

The Ranking of the Criteria Based on MACBETH Approach

The structured method MACBETH [17] was applied with the aim of providing a ranking of evaluation criteria. Three focus groups have been organized with the participation of real stakeholders, including: representatives of the builders’ associations, developers, designers, representatives of administrations’ offices and academic experts (energy and economic evaluations). The M-MACBETH software facilitated the application of the method.

According to the MACBETH method, a series of questions related to the criteria under examination have been posed to the stakeholders involved. They were asked to answer according to the semantic categories described in Table 1 starting from their personal opinion and knowledge and taking into account their area of expertise. The questions were of the type:

• (1a) Looking at the criteria under examination in Table 2, rank them from most preferred to least preferred.
• (1b) According to the rank so far provided, to what extent do you prefer one criterion to another?
• Example: I strongly prefer the criterion “investments’ costs” over the criterion “running costs” and I weakly prefer “running costs” to the criterion “resilience of the energy system”.

In order to obtain an output in form of numerical ranking, we inserted the answers provided by the stakeholders in the M-MACBETH software that is able to translate the semantic categories into numerical judgments according to the MACBETH method [17]. During this phase, some judgmental hesitations and/or disagreement appeared among the stakeholders about which MACBETH category better reflected the difference of attractiveness. In such cases, a discussion within the group has been stimulated in order to reach the consensus.

The overall criteria ranking resulting from the MACBETH applications during the focus groups are presented in

Table 3. Final ranking of the considered criteria.

Criteria Ranking	Scores (%)
Investment costs	30
Payback period (PBP)	27
Reduction of the energy requirement	23
Reduction of the CO2 emissions	18
Resilience of the energy system.	2

.

During the focus groups, the business operators emphasized the need for an economic viability of the investment, while the representatives of the public authorities advocate a reduction of the CO₂ emissions. However, the results highlighted the higher preference given to the economic aspects compared with the environmental ones, as one can notice in Table 3 (Investments costs 30% and PBP 27%). The criterion “resilience of the energy system” was not considered important for the definition of future energy scenarios.

Additional suggestions emerging from the discussion among the stakeholders are: the demand for a more flexible and participative approach for ranking the criteria; and the need to consider additional social criteria and urban planning aspects.

3.2. The Criteria Set in the EEB Project

The EEB is a National project coordinated by ST-Microelectronics [42] and funded by the Ministry of Innovation. The project aims at exploring energy consumption patterns at the urban-scale providing a methodology for evaluating different urban energy retrofitting scenarios based on multi-criteria analysis in the context of sustainable urban planning. In the framework of the EEB project, a focus group involving real stakeholders has been established in order to test the Dashboard. As for the DIMMER project, the criteria considered in the present application have been selected on the basis of both the relevant literature review, as it is presented in

Table 4. Description of the considered criteria (EEB project).

Aspect	Criteria	Literature	Description	Unit
Environmental	Global emissions CO2	[36,43,44]	Measure the equivalent emission of CO2, which is avoided by the examined action.	Tons/year
	Local emissions (NOX + PM10)	[43]	Direct impact on the health of the community and an indirect impact on the social state of the community.	Ordinal scale
Economic	Payback period (PBP)	[41]	Performance measure used to evaluate the efficiency of an investment or to compare the efficiency of a number of different investments.	Years
	Investment cost	[40]	Investment costs related to refurbishment of building (efficiency investment) and/or new heating system (infrastructure investment).	Euro
	Socio-economic feasibility	[45]	The economic capability and willingness of the people.	Number
	Maintenance costs	[46]	Running fixed and variable costs due to maintenance of the heating system (does not take into account fuel costs).	Euro
Technical	Reliability	[36,47]	Efficiency of the technology and the requalification result.	Ordinal scale
	Technical life	[44]	Durability of the whole strategy in relation to the service life of each retrofit measure.	Years
Social	Social acceptability	[47,48]	The perception of the people related to specific impacts due to the refurbishments.	Ordinal scale
	Local Job creation	[49]	Potentiality of creating job and better regularity of the employee.	Man-day/ordinal scale
	Architectural Impact	[49]	The visual and architectural impact of refurbishments in the existing built environment.	ordinal scale

and the specific requirements deriving from the EEB project. Finally, as suggested by the previous focus group organized in the context of the DIMMER project (see Section 3.1), we included social, architectural, and technical criteria in the analysis (Table 4).

Table 4 shows the list of criteria used in EEB project. This is a longer list compared with the one used in the DIMMER project. In this list, a number of relevant additional criteria have considered as follows: (1) “local emissions” is crucial, but is often ignored due to a lack of available data; (2) the “reliability” criterion has been inserted in order to consider the technical feasibility of the possible energy projects at the urban level; and (3) we took into account a number of social aspects. It is proved they are definitely fundamental criteria for people’s acceptance of energy changes [34]. Therefore, they need to be considered in the SDSS, in order to better guide the design of future urban energy transition scenarios.

The Ranking of the Criteria Based on Playing Card Method

To define the importance of the criteria during the EEB project we decided to apply the “Playing Cards” method [18]. As described in Section 2, the “Playing Cards” method is a semi-structured participative procedure, which differs from the MACBETH approach previously applied.

Operatively, the “Playing Cards” method has been here applied during a focus group organized in Turin (Italy) whose purpose was to discuss and rank the most important criteria to be further implemented in the Dashboard. The stakeholders involved in this focus group belong to the following categories: designers, representatives of the public administrations, experts in SDSS development, expert in visualization tool, building administrators, and academic experts (i.e., energy, economic evaluations, and urban planning).

During this focus group, the stakeholders were divided in three heterogeneous groups of work. First, each group received a set of cards: each card represented a decision criterion. Second, the groups were asked to rank the cards according to their preferences. Third, we asked them to think about the fact that the importance of two successive criteria can be more or less close. Therefore, we invited the stakeholders to introduce the white cards between two successive cards according to the logic “the grater the number of white cards, the greater the difference between the mentioned weights of the criteria” [25]. Then, each group provided a rank of criteria. Finally, the three ranks were showed in a plenary session. Differently from the approach used for the Macbeth application, in this case the stakeholders were forced to discuss about the rank in order to obtain a consensual rank of criteria (

Table 5. Final results coming from the “Playing Cards” method.

Rank	Subset of Ex-Equo	Number of Cards	Positions	Non-Normalize Weights	Normalized Weights	Total 2
1	Architectural Impact	1	1	1	1316	132
2	White cards	3	(2,3,4)	-	-	-
3	Local Job creation	1	5	5	6579	658
4	White cards	1	(6)	-	-	-
5	Reliability	1	7	7	9211	921
6	White cards	2	(8,9)	-	-	-
7	Socio/economic feasibility + Local emissions	2	10,11	10,5	13,816	2763
8	White cards	1	(12)	-	-	0
9	Investment costs	1	13	13	17,105	1710
10	Payback Period	1	14	14	18,421	1842
11	Global emissions CO2	1	15	15	19,737	1973
SUM			76 1			100

¹ This sum does not include the positions of the white cards (in brackets). ² The total column reports the normalized weights multiplied for the number of cards of each position.

).

From Table 5 it emerges that some of the criteria included in Table 4 have been removed by the stakeholders during the discussion since they were considered unimportant for the analysis. In particular, the “social acceptability” has not been considered as a crucial aspect due to several reasons: (1) the construction phases are usually very short and, therefore, they do not constitute an inconvenience; and (2) the stakeholders believed that the possible inconveniences occurring during a construction phase are unavoidable and uncontrollable. On the contrary, both the “maintenance costs” and the “technical life” criterion have been considered as relevant criteria, but only in association with the “payback period” criterion.

During this exercise, the stakeholders questioned the calculation methods of the social aspects with particular reference to the “local job creation”. This is probably one of the reasons why the social aspects are partially neglected. Similarly, the “architectural impact” has been considered to not be fundamental from the stakeholders involved since this kind of impact is, nowadays, reduced thanks to international and national norms. On the contrary, the economic and environmental aspects have been considered much more important than the technical and social ones. In particular, the local and global emissions have been generally considered as crucial. The correlation with human health in a specific area will have to take into account the actual concentration of those pollutants in the district environment (air) [43] and to propose a risk methodology for the augmented potential risk created.

4. Discussion

The MACBETH approach proved to be a well-structured method able to organize the evaluation criteria according to a robust and well-accepted methodology. Hence, the use of the M-MACBETH software has facilitated the ranking of the criteria, considering the preference of the stakeholders involved. A positive aspect of this method is the pair-wise comparison approach, which has proved to be quite intuitive and easy to be understood. On the contrary, the negative aspect of this method is the lack of appropriate knowledge of the relationships between the input and the output of the model. In other words, the way data are processed is not clear, resulting in a “black box” for the stakeholders.

On the other side, the “Playing Cards” method showed to be a flexible method able to stimulate the discussion among the stakeholders involved in the focus group. Thanks to these characteristics, the method is useful to support decisions with subjective criteria. Moreover, the stakeholders perceived the “Playing Cards” method as a very intuitive and engaging method, able to support discussion on the criteria involved and useful for ranking them according to their preferences.

Regarding results, both the two MCDA methods highlighted that the most important criteria for the development of energy urban retrofitting scenarios are related to both economic and environmental aspects. On the other hand, the social aspects proved to be difficult to be taken into account.

Table 6. The comparison between two approaches.

	MACBETH Approach	Playing Card (Simos Approach)
Selecting and Weighting Methods	Guided (ordinal scale)	Subjective (subjective scale)
Participation Structure	Participative approach structured through the use of a dedicated software	Semi-structured participative based on free discussion
Approach	Participants are asked to pair-wise compare the importance of criteria through worksheets	Participants are asked to rank the cards according to their personal knowledge and background
Importance ranking	Scales can vary from 0 (equal importance) to 5 (absolutely more important)	Rank importance position by inserting a set of cards “white cards” between colored cards
Stakeholders acceptance	Black box	Intuitive and entertaining

summarizes the main differences between the two approaches, emerged during their application in the DIMMER and EEB projects.

5. Conclusions

This paper has illustrated two different methods for defining and ranking the decision criteria required for assessing urban energy scenarios: (i): the Measuring Attractiveness by a Categorical Based Evaluation Technique (MACBETH) and (ii) the “Playing Cards” method. The two approaches have been applied to two research Smart City projects, DIMMER and EEB, related to the definition of a Spatial Decision Support System (SDSS), the Dashboard.

Although differences exist among the two approaches and outputs, as highlighted in Section 4, both the methods have been useful in order to identify a sensible set of criteria and their ranking. These are required for delivering appropriate, stakeholders-based scenarios and policies for energy demand reduction at urban and district scales. Both the MACBETH and the “Playing Cards” methods are stakeholder-based approaches, i.e., they are based on stakeholders’ preferences. The application of the two approaches from a very earlier phase of a decision process may help to achieve effective outcomes, since the definition and ranking of the criteria strongly influence the final results.

Finally, it is important to underline that the application of both the MACBETH and the “Playing Cards” methods presented in this paper represents a validation step [50] aiming at verifying whether the key issues have been appropriately considered in the decision making process [51] and testing the model by using experimental or real data in order.

In conclusion, this study is only one-step toward the goal of developing future urban energy scenarios through the development of the Dashboard. This will support DMs to deliver retrofitting GIS-based alternative scenarios. In particular, the main advantages of the Dashboard in the field of urban energy planning can be summarized as follows: to allow the participative processes; to give a visualization opportunity for the decision process in specific areas; to consider multiple criteria (e.g., economic, environmental, technical and, particularly, social aspects); to manage and store a very large amount of georeferenced data; to illustrate results requested by users according to different spatial forms (e.g., maps, graphs); to show the distribution of buildings’ geometrical characterization and buildings’ energy consumption. The next steps of this research include both the definition of the urban scenarios and the development of a second focus group to choose the “best” urban energy scenario.