You are currently viewing a new version of our website. To view the old version click .
MDPIMDPI
Search all articles
Sustainability
Download PDF
  • Article
  • Open Access

13 February 2018

A Methodology for the Selection of Multi-Criteria Decision Analysis Methods in Real Estate and Land Management Processes

,
and
Department of Architecture and Design, Sapienza University of Rome, Via Flaminia 359, 00196 Rome, Italy
*
Author to whom correspondence should be addressed.
Sustainability2018, 10(2), 507;https://doi.org/10.3390/su10020507
This article belongs to the Special Issue Real Estate Economics, Management and Investments
Version Notes
Order Reprints

Abstract

Real estate and land management are characterised by a complex, elaborate combination of technical, regulatory and governmental factors. In Europe, Public Administrators must address the complex decision-making problems that need to be resolved, while also acting in consideration of the expectations of the different stakeholders involved in settlement transformation. In complex situations (e.g., with different aspects to be considered and multilevel actors involved), decision-making processes are often used to solve multidisciplinary and multidimensional analyses, which support the choices of those who are making the decision. Multi-Criteria Decision Analysis (MCDA) methods are included among the examination and evaluation techniques considered useful by the European Community. Such analyses and techniques are performed using methods, which aim to reach a synthesis of the various forms of input data needed to define decision-making problems of a similar complexity. Thus, one or more of the conclusions reached allow for informed, well thought-out, strategic decisions. According to the technical literature on MCDA, numerous methods are applicable in different decision-making situations, however, advice for selecting the most appropriate for the specific field of application and problem have not been thoroughly investigated. In land and real estate management, numerous queries regarding evaluations often arise. In brief, the objective of this paper is to outline a procedure with which to select the method best suited to the specific queries of evaluation, which commonly arise while addressing decision-making problems. In particular issues of land and real estate management, representing the so-called “settlement sector”. The procedure will follow a theoretical-methodological approach by formulating a taxonomy of the endogenous and exogenous variables of the multi-criteria analysis methods.

1. Introduction and Aims of the Work

Real estate and land management (including technical, regulatory and governmental aspects) are typically highly complex sectors. Decision-making problems of this type are defined by multidimensional objective profiles and by the multidisciplinary or multi-criteria nature of the factors needing to be evaluated [1]. During the different phases of the construction process, while managing these kinds of decision-making problems, evaluation methodologies have to be designed to interpret the objectives of the land transformation and to translate the premise of the intervention into real actions.
Strategic planning of the decisions executed through decision support systems, or DSS [2], have proven highly effective in the context of settlement transformations. DSS can also be utilised with computerised systems that allow for an interactive use of data and models to support those making decisions while problem solving [1]. In each of the iterative and interactive phases of the construction process, the strategic planning of decisions is geared towards making informed choices based upon methods of evaluation. These methods need to be well suited to the purpose and to the agenda of needs and demands posed by the programs and projects being carried out in the geographical area.
Within DSS, the use of methods of Multi-criteria Decision Analysis (MCDA) can provide support for the multidisciplinary management of the factors, which need to be optimized in fulfillment of the objective being evaluated [3]. The literature regarding MCDA considers different schools of thought [4,5,6,7] and proposes an extensive number of methods with which to resolve decision-making problems in fields of application such as mathematics, management, information technology, psychology, the social sciences and economics. More specifically in the last decade, we find examples in real estate and land management, which demonstrate an increase of interest in the use of formalised analytical decision methods employing structured and comprehensive databases [8]. Several authors act as a reference source for describing the main MCDA methods, which have proven useful in different types of decision-making problems and the approaches taken to resolve them [5,9,10,11,12,13,14,15,16,17]. A helpful synthesis is provided by the contributions of, among others [18,19,20,21], Guitoni, Martel et Vincke (1999) [15], as well as of Roy et Bouyssou (1993) [16], all of which already demonstrate attempts at taking a systematic approach to the methods of MCDA. At the present moment (2018), about 100 different methods are in circulation [22]. Consulting Guitoni, Martel et Vincke [15] and Ishizaka and Nemery [9], it can be noted that the most frequently used and implemented are:
  • ELimination Et Choix Traduisant la REalitè (ELECTRE) [23];
  • Multi-attribute utility theory (MAUT) [24];
  • Analytic Network Process (ANP) [25];
  • Measuring Attractiveness by a Categorical Based Evaluation (MACBETH) [26];
  • Analytic Hierarchy Process (AHP) [27];
  • Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) [28];
  • Preference Ranking Organization Method for Enrichment Evaluations (PROMETHEE) [29].
A summary of the various contributions cited above [9,15,16] should include these points:
-
No method can be considered perfect or applied to every type of problem [9].
-
The range of available procedures offers many different operating opportunities, but also poses the risk of using methods that are not suited to the decision-making problem at hand [16].
-
A conclusive analysis of decision-making procedures has yet to be carried out [9].
Even if the use of MCDA techniques is by now an established practice, there are no specific texts, either from Italy or from the rest of Europe, dealing with the exact procedures to be followed when selecting the MCDA method best suited to the contexts of real estate and land management. In particular when generally dealing with the settlement transformation processes and more specifically the design and planning of public and private projects.
As a rule, the choice of the MCDA method best suited to the objectives of the decision-making problem can significantly affect the efficiency of the procedure. Furthermore, it effects the proper formulation of the decisions that need to be made. For example, the expression of the elements that make up a multi-criteria decision-making problem reflects on the effectiveness of the different methods and on the strength of the results. It follows that the choices in question play a key role in arriving at a solution that provides an informed, suitable response to the needs and demands identified in the project.
In light of these considerations, the present work, that represents an in-depth study of a previous work [30], sets out to define a procedure which, taking into account the relevant factors when addressing decision-making problems, makes it possible to select the MCDA methods best suited to problems of real estate and land management, but which can also be employed in other decision-making contexts. In terms of settlement transformations, these methods can expand the horizons of fact-finding and decision-making situations from a simple consideration of the financial requirements to the full range of relevant criteria (e.g., socio-economic, environmental etc.).
The proposed procedure serves as a useful tool, when added to the regulatory measures of the European Union governing public tenders. The most recent is Directive 2014/24/EU, which has been transposed into Italian laws regarding the Public Contracts Code (Codice dei Contratti Pubblici) via Legislative Decree 50/2016 (plus its subsequent modifications and additions—s.m.a.) as well as its regulations of implementation. These measures contemplate the use of MCDA in public tenders, particularly for the selection of the most economically advantageous bid. MCDA has been considered useful also in the context of planning and design choices for urban regeneration and or transformation as early as 2006. To this end, the European Commission has drawn up a manual with recommendations on systematic approaches to the use of MCDA in different fields of application, including real estate and land management [31].
After having introduced, in the current Section 1, the subject of this study, hereinafter: firstly, after the identification of the framework in which evaluation problems grow up in settlement transformation, it will be proposed a theoretical-methodological approach with a taxonomic catalogue of variables (‘endogenous’ and ‘exogenous’) that characterize MCDA and that must be considered during the selection of the most appropriate method among those most commonly carried out in literature (Section 2); afterwards, a procedure for selecting MCDA methods is built on the basis of proposed taxonomic catalogue (Section 3); finally, the proposed procedure, is applied to a case study in its widest form, taking into account a different points of view of stakeholders, and the results of this application are discussed (Section 4). The conclusions of this study are argued at the end of this work (Section 5).

2. MCDA: Structure, Endogenous Variables, Exogenous Variables

2.1. Framework

Any MCDA is generally structured in two macro-phases. The first one involves the construction and compilation, referring to the evaluation problem in question, of the evaluation matrix, which consists of the different alternatives and their performance, based on the various criteria and sub-criteria (and their weightings), plus their indicators of assessment. The second regards the processing of the data in the evaluation matrix used to evaluate the alternatives, on the basis of the objectives to be reached [30,32]. This operation is similar for all MCDA methods. The second phase involves processing (or aggregating) data via a variety of different procedures, depending on which method is being used, considering that each method comes with its own procedures of application (referred to as endogenous variables later on in this paper).
When selecting the method best suited to meet the objectives posed by the evaluation, it is necessary to take into account the context of the evaluation and that can give rise to many different decision-making problems attached to the phases of the settlement process. As presented by the literature on Life Cycle Management [33], the issues to be resolved during the different phases of the Life Cycle of a settlement process correspond to a variety of objectives [34].
In Italy, the Unique Construction Text (“Testo Unico dell’Edilizia”) Presidential Decree 380/2001 (s.m.a.) and Public Contracts Code (“Codice dei Contratti Pubblici”) Legislative Decree 50/2016 (s.m.a.) are the main laws that control the settlement transformation sector and its respective phases in both private and public works. Using these laws, we can establish categories related to the decision-making problems typical to different types of initiatives. The potential assessment questions and queries can be classified according to the phases of the settlement process and for which of these phases solutions are expected, based on the objectives of evaluation (Table 1).
Table 1. Decision making problems in Life Cycle Management and action to solve them.
Once the decision-making problem has been identified from among the possible assessment queries that present themselves during the settlement transformation process, it is necessary to address the evaluation question promptly, thus resolving the problem.

2.2. The Correlation between the Action to Be Taken and the Variables (Exogenous and Endogenous) of MCDA Methods

Each action can be related to exogenous variables (strictly related to decision problems and the context from which they arise) and endogenous ones (possible properties of MCDA methods). This is described in more detail respectively in Section 2.3 and Section 2.4, where they are defined according to their taxonomic classification (Figure 1).
Figure 1. Action to be taken with exogenous and endogenous variables.
The Exogenous variables have been defined taking into consideration the ‘external’ context of the MCDA. They vary depending on the regulations and other measures in place in the relevant country in which evaluation process occurs.
The endogenous variables have been defined following an analysis of the specific literature regarding MCDA. Understandably, there is not one single definition, which can explain MCDA and many ‘endogenous variables’ can thus be created. Concisely, Roy [10] has suggested that decision-making situations can be categorized on the basis of decision problematics [11]. Different kinds of compensation logic are examined by Vincke [12] and shared by Colson and De Bruyn [13] and further studies [14,35], for example Guitony et al. [15] and Huang et al. [8] who investigated the required input information. In this paper, a set of endogenous variables is defined as that which best represents the most useful occurring in the related literature and according to the survey performed by Ishizaka et Nemery P. [9].
Each variable (exogenous or endogenous) represents the qualifications and properties that exemplify the various forms that the variable can take. Each method retains specific properties, in terms of how successfully it can be implemented. These properties are tied to the qualities of the variables.
In this study, the ensemble of exogenous and endogenous variables has been taken as a starting point on which to structure the selection procedure for the proposed MCDA method (see Section 3) intended for use in the settlement transformation process sector.

2.3. Exogenous Variables

In this section, exogenous variables will be examined: (i) By the number of evaluation elements; (ii) By the typology of the indicators; (iii) The stakeholders involved in the decision process; (iv) The typology of the expected solution; (v) The presence of technical support from a Decision Aid Specialist during the implementation of the procedure. In the settlement transformation process, the number of evaluation elements (criteria, sub-criteria and alternatives) depends on the nature of the different decision-making problems [8,30,36,37,38,39,40].
The numeric configuration of evaluation elements (Table 2) is generally decided by the person responsible for the evaluation process before the selecting the most suitable MCDA method with reference to the specific problem in hand. In the case of settlement transformation processes, which are generally rather complex, the number of criteria and sub-criteria are defined by the process manager (Responsabile del Procedimento), role regulated by the Legislative Decree 50/2016 Article 31. This is the individual who “provides the authority offering the contract with the data and information regarding the main stages of the intervention, necessary for the coordination, guidance and checks of the correct implementation, and oversees the economic management of the intervention” (Legislative Decree 50/2016 Article 31, par. 4, point f). This kind of activity has been present since the definition of administrative acts relating to settlement process initiation. The person responsible for the evaluation process must attempt to construct the set of evaluation elements without allowing the potential repercussions to overly influence the selection of an MCDA method. Although the creation of the set of variables must of course represent the maximum number of requirements with the minimum number of possible elements [35,41,42]. The number of alternatives can be deduced by taking into account what has happened in the past in similar settlement proceedings in the same or in other administrative contexts [43,44]. The typology of the indicators (Table 3) varies in accordance with the decision-making problem to be resolved and the context in which it arises.
Table 2. Numeric configuration of evaluation elements.
Table 3. Typology of the indicators.
The number of stakeholders involved in the decision-making process may vary depending on the different assessment questions that the MCDA needs to answer and from the decision maker’s management related to the activation (or not) of Participatory Processes (P.P.) also deciding the number of stakeholders to be considered (Table 4).
Table 4. Number of stakeholders.
The involvement of stakeholders in the decision-making process aims to keep the decision-maker informed as to the opinions of those who may be interested or influenced by the effects of the decision. The types of stakeholders to be included in the participatory process [45,46] can be classified into the following: (i) standard stakeholders “who have the legitimate responsibility to participate in the process” [47], including all those who could be impacted by the results of the program (e.g., beneficiaries or those who have suffered damage), design engineers and public administration officials, etc. [31]; (ii) interest groups, stakeholders selected from local or professional representatives, leaders of non-governmental organizations (such as stakeholder or environmental protection, consumers and women’s rights), public sector bodies the representatives of financial donors, etc. [31]. In short, interest groups are often political parties, civic organizations, or residents of the impact area [47]. Each interest group, as well as the standard stakeholder, has their own motives when evaluating the potential alternatives and often has different relational systems of preference. The process manager, influenced by his own knowledge of the context from which the decision-making arises, identifies the stakeholders to be included in the process. After attributing indices of importance to each stakeholder, the process manager can select the solution (MCDA method) that is generally best for them, for example the wishes of a simple majority (a solution that is preferred by stakeholders whose added indices of importance is greater than 50%) [30].
The type of solution proposed for the decision-making problem in question depends on the selection criterion sought during the evaluation process (Table 5).
Table 5. Type of solution.
The technical support of a Decision Aid Specialist during the execution of the process is one of the variables that must be considered. As has been noted by the European Commission [31], the management of the multi-criteria evaluation processes can prove anything but simple. Therefore, another factor influencing the selection of the most suitable method is the availability or non-availability of a Decision Aid Specialist who can put into action the MCDA procedure (Table 6).
Table 6. The presence of technical support from a Decision Aid Specialist.
The Decision Aid Specialist deals with the technical implementation of the MCDA by making use of the various software available. In other studies, a wide variety of MCDA software has been described, some of which is commercially available, discussing different kinds of packages that have been developed to facilitate MCDA methods [9,48,49]. They involve several processes to give structure to decision-making problems including the exploration of the situation and a formulation and breaking down of the solution [48] by using mathematical algorithms. In reference to the MCDA methods mentioned above (Section 1), a review of the literature draws attention to some of the software available, which is easy to use and free to access or trial. Examples include MakeItRational [49,50] or ExpertChoice [51] for AHP, SuperDecision [52] for ANP, RightChoice [53] for MAUT, M-MACBETH [54] for MACBETH, SmartPickerPro [55] for PROMETHEE, Electre III–IV software [56] for ELECTRE and Topsis [57] for TOPSIS. The Decision Aid Specialist in his position as “technical” manager needs to have specific knowledge of programming and query languages in order to perform MCDA using this kind of software. Acting as a “facilitator”, the Decision Aid Specialist as a has the task of making the stages of the MCDA clear to non-specialist stakeholders; their presence also increases the level of transparency in the evaluation process allowing for informed decisions to be taken.
Table 7 summarizes the features of the exogenous variables.
Table 7. Features of exogenous variables.

2.4. Endogenous Variables

Each endogenous variable is described in the following: (i) the type of decision-making problem; (ii) the solution approach; (iii) the input level; (iv) the implementation procedure; (v) the output typology or order of alternatives and (vi) the solution to the decision problem [30].
The types of decision-making problem according to Roy’s viewpoint [16] and shared by Ishizaka et Nemery P. [9] and Guitoni and Martel [11], among others, can be grouped into three categories that express the qualification of the variable (Table 8).
Table 8. Types of decision-making problems.
Three clusters distinguish the approach to solving decision-making problems. They are identified from the different qualifications that represent the resolution of the evaluation problem (Table 9).
Table 9. Approaches to resolving problems.
In the case of a full aggregation approach, the scores allow each alternative to be comparable with another. They are expressed while taking into consideration the performance of the alternatives according to the criteria and sub-criteria selected for the analysis.
With regard to the outranking approach the incomparability is defined by observing alternative performance sets that are equally valid but qualified differently because they are based on different sets of criteria. The allocation of a full or partial score to the alternative involves a consideration of the performance set, based on the criteria and sub-criteria selected for the execution of the analysis.
In case of the “goal, aspiration or reference level approach”, the options (alternatives) are evaluated using the aggregate collection (vector sum) of the performance in relation to the different criteria that allow one to define how far (vector) the alternatives fall from the final objective.
The input level describes the “modelling effort” [9] needed to achieve the desired results and is connected to the measurement (Indicators) of the data and the parameters to be drawn and considered in order to solve the decision-making problem (Table 10). It is necessary to emphasize that Strategic Planning Techniques [58] and Participation Techniques are employed in the evaluation and collection of data used in the MCDA. Indeed, several stakeholders are often considered directly or indirectly involved in the decision-making problem, including: (i) institutions (national, regional, local); (ii) contracting stations; (iii) entrepreneurs, economic operators; (iv) property owners; (v) workers and (vi) the population.
Table 10. Input levels.
A score is assigned to each parameter by using the indicators. The input level is a synthetic indicator of these scores, which expresses the level: High (H), Medium (M) or Low (L). The methodology of the calculation is described in [30] Appendix A (Table A1).
The different implementation procedures are defined using logical mathematical operations in order to process the data implemented in the evaluation matrix and to get a summary of results for the classification of the alternatives. Different methods of data processing and aggregation are necessary for the different implementation procedures. The procedures include: (i) preference thresholds, indifference thresholds and veto thresholds [59]; (ii) utility function [9,17]; (iii) pairwise comparisons on a ratio scale [27,60]; (iv) pairwise comparisons on a ratio scale with interdependencies [25]; (v) pairwise comparisons on an interval scale [26,61]; (vi) ideal option and anti-ideal option [9,28,62,63,64] (Table 11) [30].
Table 11. Implementation procedures.
It is possible to obtain output modalities by putting in order alternatives with different qualities. The “granularity order” [9,15] varies according to the type of endogenous variables considered. The output typologies are obtained as a result of the evaluation implementation referring to the number (n) of alternatives evaluated (Table 12) [30]. This depends on different calculation methods, which represent the comparability or incomparability between the alternatives and of the distance (or the type of measurement or procedure by which the alternatives are ordered) of the alternatives from achieving the defined objective.
Table 12. Output typologies.
The types of solution used to resolve the decision-making problem derive from the order (output) of the alternatives and depend on whether the incomparability between two alternatives is admitted or not. The solution is therefore based on different foundations (Table 13) [30].
Table 13. Solutions used for the decision-making problem.
Table 14 summaries the features of the endogenous variables mentioned in this section.
Table 14. Features of the endogenous variables.

2.5. Properties of MCDA Methods Transposed into a Binary Mathematical System

The analyses carried out in Section 2.3 and Section 2.4 above make it possible to construct a matrix that summarizes the properties of the MCDA methods considered [30]. This matrix consists of 7 columns displaying the MCDA methods taken into consideration (Tn) and 38 rows representing the qualifications (Qn) to be deliberated as a set of indicators for each variable (Vn). At the point where each row and column meet, a score, Px(Tn;Vn;Qn), is assigned through a binary mathematical system used to illustrate whether each of the relevant properties of qualification is present (1) or absent (0) (Table 15) [30]. This transposition serves, in the following phase, for the implementation of the procedure used to select the MCDA method best suited to the decision-making problem being addressed. The transposition of the properties of MCDA methods into a binary mathematical system was performed by examining the most popular MCDA [30].
Table 15. Properties of Multi-Criteria Decision Analysis (MCDA) methods transposed in a binary mathematical system.

3. Selecting Methods of Multi-Criteria Decision Analysis: The Proposed Procedure

The selection of the most suitable MCDA method is carried out by comparing the framework of properties that characterize each MCDA method (Table 15) with the qualifications that the method should possess (the expected properties), depending on the decision-making problem to be addressed and taking into consideration both the exogenous and endogenous variables. As already seen in Section 1, the procedure is designed to satisfy the settlement transformation sector as follows:
  • The weighting of variables (optional action): A set of variables (that represents the criteria) and their potential qualifications (Table 15) has been defined (see also Section 3). The variables can be considered of equal importance or weight (equal weight method) or of different importance and weight [65,66,67]. Should it be necessary to consider the varying importance of the variables, a weight can therefore be assigned to each of them [68]. Different weights will directly influence the results of MCDA procedure. Consequently, it is essential to define the rationality and veracity of the criteria weights. Several methods of achieving this are discussed in the reference literature. For example: (i) subjective weighting methods such as direct assignment, Simple Multi-Attribute Rating Technique (SMART), SWING, SIMOS, pairwise comparison, AHP; (ii) Objective weighting methods such as entropy method, TOPSIS and combination weighting methods [66,67,69,70,71]. The most appropriate weighting method can be chosen by taking into consideration: (i) the variance in the degrees of criteria; (ii) the independency of criteria; (iii) the subjective preferences of the decision-makers and stakeholders when communicating their weights [68]. The exact number of criteria (and sub-criteria) may also have some relevance [35,59]. Direct assignment, SMART and SWING are the most used methods for addressing decision-making problems related to the settlement transformation process. The advantages of these being: the fast implementation times and the possibility to collect the views of stakeholders through questionnaires. However, the various weakness must also be considered including the difficulties connected to quantifying the uncertainty of the human input [66] and the subsequent conflict between the thoughts and priorities of the stakeholders and the expression of ranking and values. Appendix B describes how stakeholders may express the index of importance for each variable and their aggregation modalities [30].
  • Determining the framework of expected properties: This involves the identification (presence or absence) of the qualifications needed by the different variables in order to address the decision-making problem in question. Those responsible for the process of settlement transformation must determine the needs and demands involved in the decision-making problem being examined. The choice must be based on the set of exogenous and endogenous variables and composed of both the required and expected properties, EP(Vn;Qn), of the method selected for the decision-making problem. The framework of the expected properties for each exogenous and endogenous variables (for the chosen method) is determined according to the formulas and Table A2 [30] attached in Appendix C.
  • Calculation of the overall index of suitability: This is based on a comparison of the properties of the various MCDA methods (Table 15) with their expected properties. A general index can be obtained for the suitability of each potential method for resolving the evaluation problem. Before an overall index of suitability can be calculated, the suitability, SR(Vn;Qn), must be determined for each qualification of the variables listed on the new table. The suitability is determined by comparing the data of the properties of the MCDA methods, for each qualification of the variable (Table 15) with the data included on the table to be filled identifying the expected properties for each exogenous and endogenous variables (see Appendix C Table A2) [30]. Refer to Appendix D.1, for the possible configurations deriving from the calculation of the overall index of suitability [30]. The suitability results, SR(Vn;Qn), for each variable are then combined for each MCDA method in order to produce an aggregate index of suitability IS(Tn). In order to weigh the variables, the suitability results must be multiplied by the index of importance for the factors expressed by the stakeholders (see Appendix D.2 for the mathematical formula [30]). Should the suitability of 2 or more qualifications have been determined for a single variable, then it holds that if the binary system produces a number of results that are equal to 1, the overall result will be 1 when calculating the overall suitability. In the case of it not being necessary to weigh the variables, the aggregate index of overall suitability or IS(Tn) for each MCDA method is obtained as displayed [30] in Appendix D.3.
  • The Identification of the method best suited to resolving the decision-making problem: Obtaining a ranking of the MCDA methods with respect to the overall suitability indicators acquired. The ranking, POS(Sn), of the overall indexes of suitability for each MCDA method is reached by listing the indexes of aggregate suitability, IS(Tn) or ISW(Tn), in descending order. The most suitable method is the one with the highest index of overall suitability.
As already mentioned in previous sections, the procedure being proposed has been designed by considering the 6 endogenous variables that describe the different methods of MCDA [9]. In addition, we have the 4 exogenous variables derived from the Italian regulatory framework integral to settlement transformation processes. The selection procedure of the most appropriate MCDA method as proposed below may be employed in other territorial contexts, however, the endogenous variables remain unchanged in these possible applications as they are related to the implementation of MCDA “techniques”. Instead, the exogenous variables can be reconsidered by using reduction, integration or substitution depending on the evaluation problem to be answered.

4. Application of the Proposed Procedure to a Case Study

4.1. A Procedural Application: The Evaluation of Design Proposals Responding to the Call for Tenders for a New Office Building at the Chamber of Deputies in Rome

The procedure proposed by this paper was applied to the selection of an MCDA method to be used in a hypothetical international call for tenders. The structure in question is a new office building at the Chamber of Deputies in Rome, for which the design ideas must be evaluated. The new building is to be constructed in an urban void adjacent to Palazzo Montecitorio in the historic center of Rome (Italy). The hypothetical request aims at identifying the best design idea for the solution of a situation unresolved since the first call for tenders for this potential building at the Chamber of Deputies in 1967 [1,72].
The purpose of the request is to choose from amongst multiple design proposals, the best one considering a set of criteria, sub-criteria, indicators and weights for the variables specifically designed by a team of experts (Table 16) [30,72] formed of technical officials from Rome’s Public Administration (4) and professors (6) and researchers (8) involved in studying a new project for the Chamber of Deputies.
Table 16. Evaluation elements to be considered in the call for tenders for the office building at the Chamber of Deputies.

4.2. Weighting of the Variables

In this case study, a direct assignment has been proposed; without any mathematical normalization, so that the weighting operation can be performed easily and quickly. Direct assignment, SMART and SWING, appear to be the most appropriate methods when considering the features of the proposed procedure [35,59,68]. This is because the variables considered are different and independent from each other and limited in number, and the different stakeholders (both standard and interest groups) have the possibility to express their point of view. Direct assignment has been implemented, by attributing to every stakeholder for each variable an index of importance, W(Vn), considering: 0 no importance; 0.25 low importance, 0.5 medium importance, 0.75 high importance and 1 very high importance. This weighting operation has been performed by taking into account the opinions gathered (through interviews) from a representative sample of stakeholders (technical officials from Rome’s Public Administration; professors and researchers involved in studying a new project for the Chamber of Deputies). The results of each category of stakeholders considered have been aggregated (Table 17 column weight) according to the modalities shown in Appendix B considering a simple index of importance between stakeholders.
Table 17. Expected properties for the MCDA method in the call for tenders for the office building at the Chamber of Deputies in Rome.

4.3. Determination of the Framework of Expected Properties

The expected properties of the MCDA method used to evaluate the design proposals have been defined when considering the objectives and the details of the request as well as summarized in Table 17 (columns expected properties to the decision-making problems), according to the modalities [30] shown in Appendix C.

4.4. Calculation of the Overall Index of Suitability

Following the procedure, the comparison between the property framework of MCDA methods (Table 15) and the data included on the table to be filled identifying the expected properties for each exogenous and endogenous variable as indicated [30] in Table A2 in Appendix C, provides the results of consistency for each qualification of the variables (Table 18). We arrive at the synthetic global coherence indicator, the overall suitability index IS(Tn), relative to each of the most commonly used MCDA methods calculated according to the modalities [30] shown in Appendix D.
Table 18. Results of potential MCDA methods suggested for the selection of the best design proposal in the call for tenders for the office building at the Chamber of Deputies.

4.5. Results: Identification of the Method Best Suited to Resolving the Decision-Making Problem

The decreasing order of the global coherence indicators obtained for each MCDA method considered identifies MACBETH as the best performing method according to the objectives of the request. Using this method could facilitate the evaluation problem of choosing the best design proposal, by giving a suitable answer to the established objectives (Table 19).
Table 19. The order of potential MCDA methods to select the best design proposal for the new services building at the Chamber of Deputies in Rome.
MACBETH sits at the top of the order with a consistency index of 0.64; TOPSIS (0.61) and ANP (0.60) also obtain a high consistency; AHP (0.57) and MAUT (0.53) reach a medium consistency and could potentially be used for solving the decision-making problem specific to the case study.
According to the objectives of the international call for tenders mentioned above, the consistency of PROMETHEE (0.43) is low and ELECTRE (0.23) is not recommended at all.

5. Discussion and Conclusions

Applying the proposed procedure to a case study allowed us to identify the TOPSIS method as that which was best adapted to the evaluation of design proposals in the call for tenders to construct a new office building at the Chamber of Deputies in Rome.
During the case study, it was observed that the proposed procedure works better when used in contexts where there is a proper knowledge of MCDA methods. Furthermore, a basic knowledge of how to implement MCDA methods is required by the process manager. The process manager must understand fully the relationship between the most suitable MCDA methods when compared with the suitability and strength of the results for a specific decision-making problem. The implementation of the proposed procedure may prove difficult at times because of a low level of MCDA knowledge, especially in the context of settlement transformation processes. In order to promote a wider use of the procedure proposed in this paper, some upgrades could be useful. These could focus particularly on contexts where there is a low knowledge of MCDA.
Further developments to the proposed procedure could involve: (i) How to identify the relevant stakeholders in relation to the decision-making problem to be solved. (ii) How to attribute indices of importance to them, which truly represent their role in the decision to be taken. (iii) How the process manager can arrive at a final decision (choice of MCDA method) in contexts characterized by multiple stakeholders. (iv) An in-depth analysis of the selection and use of criteria weighing methods. (v) The formulation of guidelines to facilitate creating the framework of expected properties, even by people not particularly qualified in the field of MCDA [30].
As mentioned above, the examination of the decision-making problem in question and the evaluation query (plus the related evaluation objective), represent the main critical aspects when implementing the MCDA. It should also be observed that, during MCDA implementation, other critically important stages could be incorporated. These may include: (i) a definition of the specific objectives, the criteria, the sub-criteria and the indicators; (ii) the ability to obtain the input data needed to structure the problem; (iii) the capacity to implement and control the analysis and to study it in greater depth, in addition to defining the timeframe within which to resolve the evaluation query [73,74].

Author Contributions

The paper must be attributed in equal parts to the authors.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Input Level Calculation

“To define the input level, a score is assigned to each of the 4 parameters considered in relation to the required modeling effort (Table A1) and by creating an average of the scores attributed to the parameters. If the total score is less than 0.33, the input level is low. If the total score is greater than 0.33 and not higher than 0.66, the input level is considered average. If the result is higher than 0.66, the input level is high” [30].
Table A1. Weighing the modelling effort level in relation to the input level parameters.
Table A1. Weighing the modelling effort level in relation to the input level parameters.
Score to Be AssignedParameters of the Input Level Definition and Calculation
Data and Parameter Quantity (1)Definition Time (2)Skills and Level of Knowledge of the Decision Problem (3)Use of Other Integrated Techniques (4)
1HighLongHighNecessary
0.5MediumMediumMediumAdvised
0LowShortLowUnnecessary

Appendix B. Weighting of the Variables by the Options Considered Preferable to the Stakeholders

If the person responsible for the evaluation process opts for a weighting of the variables by the stakeholders grouped into categories, the index of importance (of each variable) for each category must be determined aggregating through mathematical media to define a synthesis result used for the implementation procedure [30].
The different indices of importance for each variable can be aggregated by averaging them, as follows:
-
Simple, if all stakeholders are considered of equal importance;
-
Weighted, if the stakeholders are considered of varying importance [74].
When selecting the most suitable MCDA method, the process manager can select the solution that is preferred by the stakeholders whose added indices of importance are greater than a specified threshold (related to a simple/relative/qualified majority and unanimity). The process manager can set this threshold in relation to the composition of the stakeholders being considered (an increasing threshold as the stakeholders’ points of view rise) [30].

Appendix C. Determination Modality of the Framework of Expected Properties

To define the set of expected properties for each exogenous and endogenous variables must be filled (Table A2) [30] by deciding whether, for a given variable, each qualification is actually required or not ((A1a) and (A1b)):
if EP(Vn;Qn) = request → EP(Vn;Qn) = yes (1)
if EP(Vn;Qn) = no request → EP(Vn;Qn) = no (0)
Table A2. Table to define the expected properties.
Table A2. Table to define the expected properties.
Type of VariablesWeightVariablesQualification of VariablesExpected Properties in Relation to Decision-Making Problem
Exogenous0 ≤ W ≤ 1Number of evaluation elementsLimited number of criteria and sub-criteria and a small number of alternativesRequest = 1;
Not request = 0
Limited number of criteria and sub-criteria and a large number of alternativesRequest = 1;
Not request = 0
Large number of criteria and sub-criteria and a small number of alternativesRequest = 1;
Not request = 0
Large number of criteria and sub-criteria and a large number of alternativesRequest = 1;
Not request = 0
0 ≤ W ≤ 1Typology of indicatorsQuantitativeRequest = 1;
Not request = 0
QualitativeRequest = 1;
Not request = 0
MixedRequest = 1;
Not request = 0
0 ≤ W ≤ 1Stakeholders to be included in the decision processParticipatory process not activatedRequest = 1;
Not request = 0
Participatory process activated with a limited and specialized number of stakeholderRequest = 1;
Not request = 0
Participatory process activated with a significant number of stakeholder preferably organized in categoriesRequest = 1;
Not request = 0
0 ≤ W ≤ 1Expected solutionDefinition of n alternatives valid in relation to objectivesRequest = 1;
Not request = 0
A better overall alternative definition for the purposeRequest = 1;
Not request = 0
The ideal alternative definition closest to the lensRequest = 1;
Not request = 0
0 ≤ W ≤ 1Technical support of a Decision Aid SpecialistYes (advisable)Request = 1;
Not request = 0
No (not necessary)Request = 1;
Not request = 0
Endogenous0 ≤ W ≤ 1Type of decision-making problemsSortingRequest = 1;
Not request = 0
DescriptionRequest = 1;
Not request = 0
Ranking/ChoiceRequest = 1;
Not request = 0
0 ≤ W ≤ 1Solution approachOutranking approachRequest = 1;
Not request = 0
Full aggregation approachRequest = 1;
Not request = 0
Goal, aspiration or reference level approachRequest = 1;
Not request = 0
0 ≤ W ≤ 1Implementation procedurePreference thresholds, indifference thresholds, veto thresholdsRequest = 1;
Not request = 0
Preference thresholds, indifference thresholdsRequest = 1;
Not request = 0
Utility functionRequest = 1;
Not request = 0
Pairwise comparison on rational scale and interdependenciesRequest = 1;
Not request = 0
Pairwise comparison on interval scaleRequest = 1;
Not request = 0
Pairwise comparison on rational scaleRequest = 1;
Not request = 0
Ideal option and anti-ideal optionRequest = 1;
Not request = 0
0 ≤ W ≤ 1Input levelHighRequest = 1;
Not request = 0
MediumRequest = 1;
Not request = 0
LowRequest = 1;
Not request = 0
0 ≤ W ≤ 1Output typologyPartial ordering obtained by expressing pairwise preferences degreesRequest = 1;
Not request = 0
Total ordering obtained by expressing pairwise preferences degreesRequest = 1;
Not request = 0
Full ordering obtained by considering the scoresRequest = 1;
Not request = 0
Full ordering with score closest to the aim assumedRequest = 1;
Not request = 0
0 ≤ W ≤ 1Decision problem solutionn categories of alternatives of equal score but different behaviourRequest = 1;
Not request = 0
Alternative with the higher global scoreRequest = 1;
Not request = 0
Alternative with the closest score to the ideal solutionRequest = 1;
Not request = 0

Appendix D. The Calculation of the Overall Index of Suitability

Appendix D.1. Possible Configurations

The comparison between properties of each method with expected properties can generate 4 possible configurations ((A2a)–(A2d)) [30]:
if P(Tn;Vn;Qn) = 1; EP(Vn;Qn) = 1 → SR(Vn;Qn) = 1
if P(Tn;Vn;Qn) = 1; EP(Vn;Qn) = 0 → SR(Vn;Qn) = 0
if P(Tn;Vn;Qn) = 0; EP(Vn;Qn) = 1 → SR(Vn;Qn) = 0
if P(Tn;Vn;Qn) = 0; EP(Vn;Qn) = 0 → SR(Vn;Qn) = 0

Appendix D.2. Equation to Obtain Weighted Suitability Results (Partial Coherence Results)

To obtain weighted suitability result must be considered the suitability results and the weight of the variables [30]:
SRW = SR ( Vn ; Qn ) ∗ W ( Vn )
where
  • SRW(Vn;Qn): weighted suitability results (partial coherence results);
  • SR(Vn;Qn): suitability results (partial coherence results);
  • W(Vn): weighting judgement expressed in Vn variable (between 0 and 1).

Appendix D.3. Equation to Obtain Index of Overall Suitability (Overall Coherence Index)

To obtain weighted index of overall suitability must be aggregated the partial coherence results.
If the variables are not to be weighted, the aggregate index of overall weighted suitability ISW(Tn) for each MCDA index is obtained via the equation [30]:
IS ( Tn ) = ∑ k = 0 n SR ( Vn ; Qn ) NVn
where
  • IS(Tn): index of overall suitability (overall coherence index);
  • SR(Vn;Qn): suitability results (partial coherence results);
  • NVn: number of variables considered.
If the variables are to be weighted, the aggregate index of overall weighted suitability ISW(Tn) for each MCDA index is obtained via the equation [30]:
ISW ( Tn ) = ∑ k = 0 n SRW ( Vn ; Qn ) NVn
where
  • ISW(Tn): index of overall weighted suitability (overall coherence index);
  • SRW(Vn;Qn): weighted suitability results (partial coherence results);
  • NVn: number of variables considered.

References

  1. Guarini, M.R.; Chiovitti, A.; Battisti, F.; Morano, P. An Integrated Approach for the Assessment of Urban Transformation Proposals in Historic and Consolidated Tissues. In Computational Science and Its Applications—ICCSA 2017; Gervasi, O., Murgante, B., Misra, S., Borruso, G., Torre, C.M., Rocha, A.M.A.C., Taniar, D., Apduhan, B.O., Stankova, E., Cuzzocrea, A., Eds.; Springer: Cham, Switzerland, 2017; pp. 562–574. [Google Scholar]
  2. Marakas, G.M. Decision Support Systems in the 21st Century; Prentice Hall: Upper Saddle River, NJ, USA, 2003; Volume 134. [Google Scholar]
  3. Klapka, J.; Piňos, P. Decision support system for multicriterial R&D and information systems projects selection. Eur. J. Oper. Res. 2002, 140, 434–446. [Google Scholar]
  4. Belton, V.; Stewart, T. Multiple Criteria Decision Analysis—An Integrated Approach; Kluwer Academic Press: Boston, MA, USA, 2002. [Google Scholar]
  5. Figueira, J.; Greco, S.; Ehrgott, M. Multiple Criteria Decision Analysis—State of the Art Survey; Springer: New York, NY, USA, 2005. [Google Scholar]
  6. Nijkamp, P.; Beinat, E. Multi-Criteria Analysis for Land Use Management; Kluwer Academic Publishers: Dordrecht, The Netherlands, 1998. [Google Scholar]
  7. Hartog, J.A.; Hinloopen, E.; Nijkamp, P. A sensitivity analysis of multi-criteria choice-methods: An application on the basis of the optimal site selection for a nuclear power plant. Energy Econ. 1989, 11, 293–300. [Google Scholar] [CrossRef]
  8. Huang Ivy, B.; Keisler, J.; Linkov, I. Multi-criteria decision analysis in environmental sciences: Ten years of application and trends. Sci. Total Environ. 2011, 409, 3578–3594. [Google Scholar] [CrossRef] [PubMed]
  9. Ishizaka, A.; Nemery, P. Multi-Criteria Decision Analysis, Methods and Software; Wiley and Sons Ltd.: Chichester, UK, 2013. [Google Scholar]
  10. Roy, B. Méthodologie Multicritére d’Aide à la Décision; Economica: Paris, France, 1985. [Google Scholar]
  11. Guitoni, A.; Martel, J.M. Tentative guidelines to help choosing an appropriate MCDA method. Eur. J. Oper. Res. 1998, 109, 501–521. [Google Scholar] [CrossRef]
  12. Vincke, P. L’aide Multicritère à la Décision, Édition de l’Université de Bruxelles; Bruxelles: Brussels, Belgium, 1989. [Google Scholar]
  13. Colson, G.; De Bruyn, C. Models and Methods in Multiple Objectives Decision Making, Models and Methods in Multiple Criteria Decision Making; Pergamon Press: Oxford, UK, 1989. [Google Scholar]
  14. Fishburn, P.C. A survey of multiattribute/multicriterion evaluation theories. In Multiple Criterion Problem Solving; Zionts, S., Ed.; Springer: Heidelberg, Germany, 1978; pp. 181–224. [Google Scholar]
  15. Guitouni, A.; Martel, J.M.; Vincke, P.; North, P.B. A Framework to Choose a Discrete Multicriterion Aggregation Procedure; Defence research establishment valcatier (DREV): Ottawa, ON, Canada, 1998; Available online: https://pdfs.semanticscholar.org/27d5/9c846657268bc840c4df8df98e85de66c562.pdf (accessed on 28 July 2017).
  16. Roy, B.; Bouyssou, D. Aide Multicritère à la Décision: Methodes et Cas; Economica: Paris, France, 1993. [Google Scholar]
  17. Keeney, R.L.; Raiffa, H. Decisions with Multiple Objectives: Preferences and Value Trade-Offs; Cambridge University Press: Cambridge, UK, 1993. [Google Scholar]
  18. Cinelli, M.; Stuart, R.; Coles, K.K. Analysis of the potentials of multi criteria decision analysis methods to conduct sustainability assessment. Ecol. Indic. 2014, 46, 138–148. [Google Scholar] [CrossRef]
  19. Al-Shemmeri, T.; Bashar, A.; Pearman, A. Model choice in multi-criteria decision aid. Eur. J. Oper. Res. 1997, 97, 550–560. [Google Scholar] [CrossRef]
  20. Celik, M.; Deha, I.E. Fuzzy axiomatic design extension for managing model selection paradigm in decision science. Expert Syst. Appl. 2009, 36, 6477–6484. [Google Scholar] [CrossRef]
  21. Kurka, T.; Blackwood, D. Selection of MCA methods to support decision making for renewable energy developments. Renew. Sustain. Energy Rev. 2013, 27, 225–233. [Google Scholar] [CrossRef]
  22. Saaty, T.L. The modern science of multicriteria decision making and its practical applications: The AHP/ANP approach. Oper. Res. 2013, 61, 1101–1118. [Google Scholar] [CrossRef]
  23. Roy, B. Classement et choix en presence de points de vue multiples: La méthode ELECTRE. Rev. Fr. Inform. Rech. Opér. 1968, 8, 57–75. [Google Scholar] [CrossRef]
  24. Dyer, J.S. MAUT—Multiattribute utility theory. In Multiple Criteria Decision Analysis: State of the Art Surveys; Springer: New York, NY, USA, 2005; pp. 265–292. [Google Scholar]
  25. Saaty, T.L. Analytic network process. In Encyclopedia of Operations Research and Management Science; Springer: New York, NY, USA, 2001; pp. 28–35. [Google Scholar]
  26. Bana e Costa, C.; Vansnick, J. MACBETH: An interactive path towards the construction of cardinal value functions. Int. Trans. Oper. Res. 1994, 1, 387–500. [Google Scholar] [CrossRef]
  27. Saaty, T. A scaling Method for priorities in hierarchical structures. J. Math. Psychol. 1977, 15, 234–281. [Google Scholar] [CrossRef]
  28. Hwang, C.L.; Yoon, K. Multiple Attribute Decision Making: Methods and Applications; Springer: Heidelberg, Germany, 1981. [Google Scholar]
  29. Brans, J.P.; Vincke, P. Note—A Preference Ranking Organisation Method: The PROMETHEE Method for Multiple Criteria Decision-Making. Manag. Sci. 1985, 31, 647–656. [Google Scholar] [CrossRef]
  30. Guarini, M.R.; Battisti, F.; Chiovitti, A. Public Initiatives of Settlement Transformation: A Theoretical-Methodological Approach to Selecting Tools of Multi-Criteria Decision Analysis. Buildings 2018, 8, 1. [Google Scholar] [CrossRef]
  31. European Commission. Evaluation Methods for the European Union’s External Assistance; Evaluation Tool; European Commission: Brussels, Belgium, 2006; Volume 4, Available online: http://ec.europa.eu/europeaid/sites/devco/files/evaluation-methods-guidance-vol4_en.pdf (accessed on 28 July 2017).
  32. Seixedo, C.; Tereso, A. A Multi-criteria Decision Aid Software Application for selecting MCDA Software Using AHP, Conference Paper In 2nd International Conference on Engineering Optimization, Lisbon, Portugal. 2010. Available online: http://hdl.handle.net/1822/19355 (accessed on 5 October 2017).
  33. Campbell, J.D.; Jardine, A.K.; McGlynn, J. Asset Management Excellence: Optimizing Equipment Life-Cycle Decisions; CRC Press: Boca Raton, FL, USA, 2016. [Google Scholar]
  34. Guarini, M.R.; Battisti, F. Evaluation and Management of Land-Development Processes Based on the Public-Private. Adv. Mater. Res. 2014, 869–870, 154–161. [Google Scholar] [CrossRef]
  35. Bouyssou, D. Some remarks on the notion of compensation in MCDA. Eur. J. Oper. Res. 1986, 26, 150–160. [Google Scholar] [CrossRef]
  36. Chung, E.S.; Lee, K.S. Prioritization of water management for sustainability using hydrologic simulation model and multi-criteria decision making techniques. J. Environ. Manag. 2009, 90, 1502–1511. [Google Scholar] [CrossRef] [PubMed]
  37. Liu, D.F.; Stewarr, T. Object-oriented decision support system modelling for multi-criteria decision making in natural resource management. Comput. Oper. Res. 2004, 31, 985–999. [Google Scholar] [CrossRef]
  38. Qin, X.S.; Huang, G.H.; Chakma, A.; Nie, X.H.; Lin, Q.G. A MCDM-based expert system for climate change impact assessment and adaption planning—A case study for the Georgia Basin, Canada. Expert Syst. Appl. 2008, 34, 2164–2179. [Google Scholar] [CrossRef]
  39. Guarini, M.R.; Locurcio, M.; Battisti, F. GIS-Based Multi-Criteria Decision Analysis for the “Highway in the Sky”, ICCSA 2015; Springer: Cham, Switzerland, 2015; Volume 9157, pp. 146–161. [Google Scholar]
  40. Haimes Yacov, Y. On the Universality and contributions of Multiple Criteria Decision Making: A systems-based Approach. J. Mult. Criteria Decis. Anal. 2011, 18, 91–99. [Google Scholar] [CrossRef]
  41. Bouyssou, D. Building criteria: A prerequisite for MCDA. In Readings Multiple Criteria Decision Aid; Springer: Berlin/Heidelberg, Germany, 1990; pp. 58–80. [Google Scholar]
  42. Roy, B.; Vanderpooten, D. The European school of MCDA: Emergence, basic features and current works. J. Mult. Criteria Decis. Anal. 1996, 5, 22–38. [Google Scholar] [CrossRef]
  43. Del Giudice, V.; de Paola, P.; Torrieri, F. An Integrated Choice Model for the Evaluation of Urban Sustainable Renewal Scenarios. Adv. Mater. Res. 2014, 1030–1032, 2399–2406. [Google Scholar] [CrossRef]
  44. Torrieri, F.; Batà, A. Spatial Multi-Criteria Decision Support System and Strategic Environmental Assessment: A Case Study. Buildings 2017, 7, 96. [Google Scholar] [CrossRef]
  45. Belton, V.; Pictet, J. A framework for group decision using a MCDA model: Sharing, aggregating or comparing individual information? J. Decis. Syst. 1997, 6, 283–303. [Google Scholar] [CrossRef]
  46. Ukeni, A.O.; Anthony, A.; Michael, S.; Sonia, G. Balancing stakeholder views for decision-making in steel structural fire design. In Proceedings of the International Conference on Performance-based and Life-cycle Structural Engineering, Brisbane, Australia, 9–11 December 2015; School of Civil Engineering, The University of Queensland: Brisbane, Australia, 2015; pp. 983–992. [Google Scholar]
  47. Lahdelma, R.; Salminen, P.; Hokkanen, J. Using Multi-criteria Methods in Environmental Planning and Management. Environ. Manag. 2000, 26, 595–605. [Google Scholar] [CrossRef] [PubMed]
  48. Li, Y.; Thomas, M.A. A Multiple Criteria Decision Analysis (MCDA) Software selection Framework. In Proceedings of the 47th Hawaii International Conference on System Science (HICSS), Waikoloa, HI, USA, 6–9 January 2014; pp. 1084–1094. [Google Scholar]
  49. Kaspar, R.; Ossadnik, W. Evaluation of AHP software from a management accounting perspective. J. Model. Manag. 2013, 8, 305–319. [Google Scholar]
  50. MakeItRational AHP Software. Available online: http://makeitrational.com/analytic-hierarchy-process/ahp-software (accessed on 5 October 2017).
  51. Expertchoice. Available online: http://www.expertchoice.com (accessed on 5 October 2017).
  52. Super Decisions CDS. Available online: https://superdecisions.com (accessed on 5 October 2017).
  53. RightChoice. Ventana Systems UK. Available online: http://www.ventanasystems.co.uk/services/software/rightchoice/ (accessed on 5 October 2017).
  54. M-MACBETH Software. Available online: http://www.m-macbeth.com (accessed on 5 October 2017).
  55. Smart Picker Pro: The Desktop Application. Available online: http://www.smart-picker.com/products. (accessed on 5 October 2017).
  56. Electre III-IV Software. Available online: http://www.lamsade.dauphine.fr/spip.php?rubrique64&lang=fr (accessed on 5 October 2017).
  57. Triptych: TOPSIS. Available online: http://www.stat-design.com/Software/TOPSIS.html (accessed on 5 October 2017).
  58. Salet, W.G.; Thornley, A.; Kreukels, A. Metropolitan Governance and Spatial Planning: Comparative Case Studies of European City-Regions; Taylor & Francis: Oxford, UK, 2003. [Google Scholar]
  59. Bouyssou, D.; Perny, P. Ranking methods for valued preference relations: A characterization of a method based on leaving and entering flows. Eur. J. Oper. Res. 1992, 61, 186–194. [Google Scholar] [CrossRef]
  60. Saaty, T. The Analytic Hierarchy Process; Mcgraw Hill: New York, NY, USA, 1980. [Google Scholar]
  61. Bana e Costa, C.; Vansnick, J. On the Mathematical Foundations of MACBETH. In Multiple Criteria Decision Analysis: State of the Art Surveys; Springer: New York, NY, USA, 2005; pp. 409–442. [Google Scholar]
  62. Lai, Y.J.; Hwang, C.L. Fuzzy multiple objective decision making. In Fuzzy Multiple Objective Decision Making; Springer-Verlag: Berlin/Heidelberg, Germany, 1994; Volume 44, pp. 139–262. [Google Scholar]
  63. Hwang, C.L.; Paidy, S.R.; Yoon, K.; Masud, A.S.M. Mathematical programming with multiple objectives: A tutorial. Comput. Oper. Res. 1980, 7, 5–31. [Google Scholar] [CrossRef]
  64. Behzadian, M.; Otaghsara, S.K.; Yazdani, M.; Ignatius, J. A state-of the-art survey of TOPSIS applications. Expert Syst. Appl. 2012, 39, 13051–13069. [Google Scholar] [CrossRef]
  65. Baudry, G.; Macharis, C.; Vallée, T. Range-based Multi-Actor Multi-Criteria Analysis: A combined method of Multi-Actor Multi-Criteria Analysis and Monte Carlo simulation to support participatory decision making under uncertainty. Eur. J. Oper. Res. 2018, 264, 257–269. [Google Scholar] [CrossRef]
  66. Ascough, J.C., II; Maier, H.R.; Ravalico, J.K.; Strudley, M.W. Future research challenges for incorporation of uncertainty in environmental and ecological decision-making. Ecol. Model. 2008, 219, 383–399. [Google Scholar] [CrossRef]
  67. Salo, A.; Hämäläinen, R.P. Multi-criteria Decision Analysis in Group Decision Processes. In Handbook of Group Decision and Negotiation, Advances in Group Decision and Negotiation; Kilgour, D.M., Eden, C., Eds.; Springer: Dordrecht, The Netherlands, 2010; pp. 269–283. [Google Scholar]
  68. Wang, J.-J.; Jing, Y.-Y.; Zhang, C.-F.; Zhao, J.-H. Review on multi-criteria decision analysis aid in sustainable energy decision-making. Renew. Sustain. Energy Rev. 2009, 13, 2263–2278. [Google Scholar] [CrossRef]
  69. Saaty, T.L.; De Paola, P. Rethinking Design and Urban Planning for the Cities of the Future. Buildings 2017, 7, 76. [Google Scholar] [CrossRef]
  70. Ribeiro, F.; Ferreira, P.; Araújo, M. Evaluating future scenarios for the power generation sector using a Multi-Criteria Decision Analysis (MCDA) tool: The Portuguese case. Energy 2013, 52, 126–136. [Google Scholar] [CrossRef]
  71. Dulmin, R.; Mininno, V. Supplier selection using a multi-criteria decision aid method. J. Purch. Supply Manag. 2003, 9, 177–187. [Google Scholar] [CrossRef]
  72. Guarini, M.R.; D’Addabbo, N.; Morano, P.; Tajani, F. Multi-Criteria Analysis in Compound Decision Processes: The AHP and the Architectural Competition for the Chamber of Deputies in Rome (Italy). Buildings 2017, 7, 38. [Google Scholar] [CrossRef]
  73. Nesticò, A.; Sica, F. The sustainability of urban renewal projects: A model for economic multi-criteria analysis. J. Property Invest. Financ. 2017, 35, 397–409. [Google Scholar] [CrossRef]
  74. Guarini, M.R.; Battisti, F. Benchmarking Multi-criteria Evaluation: A Proposed Method for the Definition of Benchmarks in Negotiation Public-Private Partnerships. In Computational Science and Its Applications–ICCSA 2014; Springer: Cham, Switzerland, 2014; Volume 8581, pp. 208–223. [Google Scholar]

Article Metrics

Citations

Article Access Statistics

Journal Statistics
Multiple requests from the same IP address are counted as one view.
Download PDF
Forecasting China’s Coal Power Installed Capacity: A Comparison of MGM, ARIMA, GM-ARIMA, and NMGM Models
Previous
Solar Irradiance Measurements Using Smart Devices: A Cost-Effective Technique for Estimation of Solar Irradiance for Sustainable Energy Systems
Next