Using Analytic Hierarchy Process and Best–Worst Method in Group Evaluation of Urban Park Quality

Abstract

The paper compares two multi-criteria methods, the analytic hierarchy process (AHP) and the best–worst method (BWM), in assessing criteria related to the quality of urban parks. The criteria assessed were accessibility, location, biodiversity preservation, park equipment, water elements, terrain configuration, cultural and historical value, and the presence of small architectural objects. Five decision-makers participated in the research, having expertise in urban greenery, urban forestry, environmental protection, landscape design, and cultural and historical heritage. The results of decision-makers’ evaluations were compared at individual and group levels after the application of three aggregation procedures: CRITIC, ENTROPY, and WGGM (weighted geometric mean method). Similarities in results, i.e., priorities of analyzed criteria after applying the two different decision support methods, indicated high consistency between experts during the cognitive evaluation processes. All applied aggregation schemes performed well and may be considered trustworthy in identifying the group solution. One of the conclusions is that either the AHP or the BWM can be efficiently used in evaluations of criteria for assessing the quality of urban parks if the members of a group are consistent, regardless of whether the consensus process is properly carried out before the decision-making process.

1. Introduction

Urban public parks have an overall positive impact on the environments of cities. Parks and other green spaces play an important social role by offering a healthy environment and multiple recreational opportunities for everyone. There are thousands of published papers and studies related to urban parks and the associated issues related to their planning and maintenance. The qualitative and quantitative aspects are of equal relevance and studies worldwide report useful findings on how to improve the status of these public areas of common interest that are an important part of healthy and fulfilled lives. Quality of life in cities, among other things, depends on having well-maintained urban parks and other green spaces [1], as well as on various cultural and recreational objects [2,3]. In Rinner’s [2] study, a geographic visualization approach and related multi-criteria evaluation tool were presented to assess urban quality of life. The approach and tool are intended to support spatial decision-making by experts. As an illustration, the analytic hierarchy process was used to calculate composite measures of urban quality of life (QoL) for neighborhoods in Toronto and the results were combined with interactive maps to explore various settings of evaluation parameters that define different decision-making strategies. The user interviews were presented to indicate positive feedback on the utility and usability of the tool that was operated by the author.

There are numerous publications related to multi-criteria decision-making applications in urban forests and greening in general, e.g., [4,5,6,7,8]. Neema et al. [9] evaluated the quality of the existing major urban parks in Dhaka City in a multiple-criteria context. The criteria were defined as four factors: environment, safety and security, landscape, and aesthetic value. To quantitatively assess the quality of parks, a simple index was proposed to calculate the value and to rank parks in terms of these factors. A 5-point scale was used to express the quality of each park regarding the mentioned factors and an extremely simple calculation model was created on the summation of responses from several hundred visitors to the parks. Wang and Zhang [10] reported on a service quality evaluation of three urban parks in Zhehou City in China based on public satisfaction (sample of 150 visitors). Six factors were recognized as important when evaluating the service quality of parks: place environment, landscape environment, culture environment, eco-environment, traffic environment, and facilities environment. Factors were divided into 18 impact indices, which were evaluated concerning multiple criteria. However, no clear explanation was given on how the aggregation of visitors’ opinions was performed.

Our research in urban forestry and assessment of urban parks, in particular, includes various multiple-criteria decision-making (MCDM) methods and techniques for deriving group solutions. These methods support making decisions in the environment of multiple and often mutually competing criteria [11]. There are numerous MCDM methods applicable to urban forestry research, such as AHP, SMART, PROMETHEE, ELECTRE, TOPSIS, and BMW [12]. After analyzing the applicability of different methods in group decision-making processes, including consensus-based models and social choice elective models, we have selected and applied two prominent decision-making methodologies for supporting individual and group decision-making, namely the analytic hierarchy process (AHP) [13] and the best–worst method (BWM) [14,15]. Both methods deal with the prioritization of decision elements at a given node of hierarchy created for a particular decision-making problem and use the concept of pair-wise comparisons of the decision elements. Standard AHP requires more pair-wise comparisons of decision elements than BWM, and this may be a reason why the latter could be chosen in certain practical situations, especially in group contexts such as the one presented in this case study application.

Unlike the other MCDM methods, such as PROMETHEE, ELECTRE, and TOPIS, which provide only ordinal information, i.e., the final ranking of decision elements, the selected methods provide both ordinal and cardinal information (i.e., weights of decision elements). Having the final results in the form of cardinal values is valuable in the context of group decision-making, because in such cases the consensual decision can be reached by applying diverse aggregation methods [11], which was one of the main reasons for selecting these two methods in this research. Sustainable urban development requires continuous planning of activities to be undertaken by citizens and responsible city bodies and officials. This paper addresses the importance of developing a comprehensive method for evaluation of the criteria that would be necessary for the monitoring and management of parks. The focus is on criteria that can rank parks by importance, and once preference is determined in a competent group context (e.g., using a participative decision-making model based on the inclusion of citizens, experts, and officials), to help officials in directing development actions, finances, and the implementation of operational procedures (e.g., reconstruction, revitalization, cleaning, risk protection). Our earlier research started with a multi-criteria evaluation of urban parks in the City of Novi Sad in Serbia as presented in [16,17,18]. The importance of criteria for assessments of public urban parks is also discussed along with the role of biodiversity indices in measuring the biodiversity in forest communities, including city parks [19]. These studies are within the scope of this paper and rely on multiplicative preference relations (or judgment matrices that express the decision-makers’ preferences) between criteria elicited independently from experts without prior consensus, but rather by application of scientifically proven aggregation schemes. This study analyzes and explains possible differences in solutions if the two conceptually different multi-criteria methods (here AHP and BWM), based on one-to-one comparisons (multiplicative preference relations) of criteria for evaluating quality of urban parks, are used in a group decision-making context. The weights of criteria were individually derived by five academic experts in two separate sessions: (a) the first session used the matrix-based AHP method; (b) the second session was organized one month later and based on the use of the BWM optimization method. In both cases, criteria weights were used to create rating matrices to enable deriving weights of the experts by two well-known methods described in the next section. In this way, two aspects of group decision-making (GDM) were analyzed. The mathematical description of all methods used in this study is presented in Section 2 (Methods). Section 3 (Results) summarizes the main results of the study, while Section 4 (Discussion) discusses our results as they relate to previous studies and then Section 5 (Conclusions) provides conclusions and an agenda for future research.

2. Materials and Methods

Our approach to this case study consisted of four steps:

• A group of five decision-makers-academic experts in this particular case-individually performed two multi-criteria methods, (a) AHP and (b) BWM, to derive the weights of eight criteria commonly used in the assessment and evaluation of the quality of urban green areas. In the case of AHP, the eigenvector method was used to compute criteria weights and control consistency parameters for each decision-maker [13]. In the case of BWM, linear optimization was used to compute criteria weights by decision-makers and check the value of the error optimization parameter [14];
• Apply the ENTROPY [20] and CRITIC [21] methods to obtain objective weights of decision-makers for both the AHP and BWM methods, based on weights of criteria derived in the previous step;
• Aggregate individual criteria weights by AHP and BWM using a weighted geometric aggregation method for three weighting schemes related to the relative quality of decision-makers: (a) equal weights, (b) ENTROPY weights, and (c) CRITIC weights;
• Analyze and critically discuss the results from previous steps, including identifying the best solution(s) and recommending methods of prioritization (AHP or BWM or combined) and aggregation (equal weighting of DMs, ENTROPY, or CRITIC) for use in further assessments of criteria to evaluate urban public parks.

2.1. Analytic Hierarchy Process (AHP)

The core aim of the AHP is to present the decision-making problem as a hierarchy (typically as a decision tree containing a goal, one or more levels of criteria, and alternatives) and to compare the hierarchical elements in a pair-wise manner. For this purpose, Saaty’s 9-point scale [13] is typically used to express the importance of one element over another, with respect to the next higher level in the hierarchy (

Table 1. Saaty’s importance scale.

Definition	Assigned Value
Equally important	1
Weak importance	3
Strong importance	5
Demonstrated importance	7
Absolute importance	9
Intermediate values	2, 4, 6, 8

).

If n elements at one level of the hierarchy are compared regarding the element in the next higher level, the multiplicative preference relations matrix has the following quadratic form:

$$A=\left[\begin{array}{cccc}{a}_{11}& a_{12}& \dots & a_{1n}\\ a_{21}& a_{22}& \dots & a_{2n}\\ \dots & \dots & \dots & \dots \\ a_{n1}& a_{n2}& \dots & a_{nn}\end{array}\right]$$

(1)

Each matrix element a_ij is a subjective judgment provided by the decision-maker of the relative importance of the two elements, i and j. If the decision-maker is fully consistent, then the transitive rule a_ija_jk = a_ik should apply for i, j, and k in the range of 1 to n [13]. Under perfect consistency, each a_ij is equal to:

$$a_{ij}=w_i/w_j$$

(2)

in which w_i and w_j are the local weights of elements i and j regarding the element in the upper level. Therefore, the weight vector w = (w₁, w₂, …, w_n), which corresponds to the matrix (1), comprises the local weights of all the elements in the given hierarchy level regarding the element in the upper level.

However, the vector w is unknown and the problem is that there is no such unique vector because of the well-known inconsistencies of the decision-maker or the limitations imposed by Saaty’s (or any other) scale. To measure the quality of the w vector computed by any of the existing methods, e.g., [22,23], one can define several metrics and compare the original matrix A and corresponding matrix X:

$$X=\left[\begin{array}{cccc}{w}_1/w_1& w_1/w_2& \dots & w_1/w_n\\ w_2/w_1& w_2/w_2& \dots & w_2/w_n\\ \dots & \dots & \dots & \dots \\ w_n/w_1& w_n/w_2& \dots & w_n/w_n\end{array}\right]$$

(3)

A large number of elements in the hierarchy may not affect the level of congruence between matrices A and X due to scale imperfections or insufficient knowledge of the decision-maker about the decision problem. The differences between corresponding elements in matrices (1) and (3) are usually treated as inconsistencies of the decision-maker. Saaty [13] recommended the Consistency Ratio (CR) as a measure of individual inconsistency, and it is considered a part of the standard AHP.

In addition to CR, the total Euclidean distance is also used in many studies to indicate deviations between original judgments a_ij from matrix (1) and corresponding ratios w_i/w_j from matrix (3) [22,24]. A smaller Euclidean distance corresponds to a better overall agreement of computed weights of decision elements and judgments elicited from the decision-maker. Unlike consistency indicators, which are dependent on the specific prioritization method, the Euclidean distance is a universal error measure. Although it does not have threshold values, this metric may help to compare the level of total (quadratic) agreement between matrices A and X; that is, differences between a_ij and w_i/w_j across all entries of the two matrices.

2.2. Best–Worst Method (BWM)

The BWM was originally developed by [14] as a non-linear model:

$$\begin{gathered} \min {\max }_{\mathrm{j}} \left\{\left|w_B/w_j-a_{Bj}\right|,\left|w_j/w_W-a_{jW}\right|\right\} \\ \mathrm{s}.\mathrm{t}. \\ {{\displaystyle \sum }}_j{w}_j=1,\hspace{1em}\mathrm{for} \mathrm{all} j \\ {w}_j\ge 0\hspace{1em}\mathrm{for} \mathrm{all} j \end{gathered}$$

(4)

Weights and judgments with B and W in the subscript relate to the best and worst criterion, respectively, as they are stated by the decision-maker or analyst in the beginning of the evaluation process.

In (4), ‘for all j’ means ‘for all compared elements’ in the set of decision elements, either criteria, sub-criteria, or alternatives; for instance, if there are n criteria, then ‘for all j’ means j = 1, 2,…, n.

Following ideas introduced by [15] results in a linear model in which instead of minimizing the maximum value among the set of $\left\{\left|w_B/w_j-a_{Bj}\right|,\left|w_j/w_W-a_{jW}\right|\right\}$, minimization is performed over the maximums among the set of $\left\{\left|w_B-a_{Bj}{\mathrm{w}}_{\mathrm{B}}\right|,\left|w_j-a_{jW}{w}_W\right|\right\}$. The problem is now:

$$\begin{gathered} \min {\max }_{\mathrm{j}} \left\{\left|w_B-a_{Bj}{w}_B\right|,\left|w_j-a_{jW}{w}_W\right|\right\} \\ \mathrm{s}.\mathrm{t}. \\ {{\displaystyle \sum }}_j{w}_j=1,\hspace{1em}\mathrm{for} \mathrm{all} j \\ {w}_j\ge 0\hspace{1em}\mathrm{for} \mathrm{all} j \end{gathered}$$

(5)

This model can be transferred to the linear model (6) by introducing the dummy variable $\epsilon$:

$$\begin{gathered} \min \epsilon \\ \mathrm{s}.\mathrm{t}. \\ \left|w_B-a_{Bj}{w}_{\mathrm{B}}\right|\le \epsilon ,\hspace{1em}\mathrm{for} \mathrm{all} j \\ \left|w_j-a_{jW}{w}_W\right|\le \epsilon ,\hspace{1em}\mathrm{for} \mathrm{all} j \\ {{\displaystyle \sum }}_j{w}_j=1,\hspace{1em}\mathrm{for} \mathrm{all} j \\ {w}_j\ge 0\hspace{1em}\mathrm{for} \mathrm{all} j \end{gathered}$$

(6)

which has a unique solution in which the optimal set of weights is $w_j^{*} \mathrm{for} \mathrm{all} j$ and ${\epsilon }^{*}$. Similar to [24] model and the ‘natural measure of consistency’ ${\mu }^{*}$, the optimal value for dummy variable (${\epsilon }^{*}$) in the model (6) can be considered as an indicator of consistency demonstrated by the decision-maker, with lower values of this indicator indicating a higher level of consistency. Various applications of the BWM can be found in the literature [25,26,27,28,29,30].

2.3. Two Methods Used to Objectively Derive Weights of Criteria

2.3.1. Rating Matrix

In our model, the rating matrix is composed in a specific manner. Each column corresponds to one decision-maker, and each row corresponds to one evaluated criterion. For m decision-makers (DM₁, DM₂, …, DM_m) and n criteria (C₁, C₂, …, C_n), the given entry r_i,j in the matrix is the weight of criterion i for decision-maker j. Rating matrix R is given by (7). Values (w₁, w₂, …, w_m) above columns are weights of decision-makers, and the sum of these weights is 1.

$$\begin{gathered} \hspace{1em}\hspace{1em}\hspace{1em}\begin{array}{cccc}D{M}_1& D{M}_2& \dots & D{M}_m\\ w_1& w_2& \dots & w_m\end{array} \\ R=\begin{array}{c}{C}_1\\ C_2\\ \dots \\ C_n\end{array}\hspace{1em}\left[\hspace{1em}\begin{array}{cccc}{r}_{11}& r_{12}& \dots & r_{1m}\\ r_{21}& r_{22}& \dots & r_{2m}\\ \dots & \dots & \dots & \dots \\ r_{n1}& r_{n2}& \dots & r_{nm}\end{array}\hspace{1em}\right] \end{gathered}$$

(7)

Entries in the decision matrix can be interpreted by the ENTROPY principle as information emitted by decision-makers to criteria and modeled by the ENTROPY method described below. In statistically oriented modeling, columns of the decision matrix can be correlated, as is also described later as the CRITIC method [21].

Assuming that criteria rating r_ij (i = 1, …, n; j = 1, …, m) are given as normalized values by columns and that weights of decision-makers are known, the final ratings of the criteria versus all decision-makers can be obtained directly by any known aggregation method.

In this study, ratings in matrix (1) are obtained via two multi-criteria methods, AHP and BWM, which are used separately and at different times by the same five decision-makers.

2.3.2. ENTROPY Method

According to [20], ENTROPY is a measure of the uncertainty of information. For our analysis, the ENTROPY principle is used to determine objective weights of decision-makers w_j (j = 1, 2, …, m), based on their evaluations of criteria represented by their rating r_ij (i = 1, 2, …, n), and obtained separately for the AHP or BWM methods.

For ratings already normalized by columns, which is the result of either AHP or BWM, the ENTROPY method [20] considers that information contained in the rating matrix is emitted from each decision-maker DM_j (j = 1, 2,…, m) as the ENTROPY value e_j given by Equation (8).

$$e_j=-k{{\displaystyle \sum }}_{i=1}^n x_{ij} ln{x}_{ij}, j=1, 2, \dots , m.$$

(8)

By introducing a normalization constant k = 1/lnn, all values e_j are guaranteed to be in the interval of [0,1].

In the second step, the degree of divergence f_j of the average intrinsic information contained in the emitted opinion of each decision-maker DM_j (j = 1, 2, …, m) is calculated as:

$$f_j=1-e_j,\hspace{1em}\hspace{1em}j=1, 2, \dots , m.$$

(9)

The idea is as follows. For a larger divergence of initial rating r_ij of criterion C_i for decision-maker DM_j, value f_j is greater, and the conclusion is that the weight of this decision-maker for deriving a final decision is also greater [31]. If all criteria for a given decision-maker have similar weights, this decision-maker can be considered less influential in deriving the final decision.

Because the value f_j can be considered as the contrast intensity of decision-maker DM_j with other decision-makers, in the final step the aggregation given by Equation (10) is performed to derive relative intensities of emitters in our study weights of decision-makers.

$$w_j=f_j·{\left[{\displaystyle \sum }_{k=1}^m{f}_k\right]}^{-1},\hspace{1em}j=1, 2, \dots , m.$$

(10)

2.3.3. CRITIC Method

For each decision-maker DM_j (j = 1, 2, …, m), the rating values (weights of criteria) r_ij, are mapped into the interval [0,1] by use of normalization Equation (11):

$$h_j\left(i\right)=\frac{r_{ij}-r_j{}^{**}}{r_j{}^{*}-r_j{}^{**}}$$

(11)

Value h_j(i) expresses the degree of closeness of criterion C_i to the ideal value $r_j{}^{*}$. The ideal value $r_j{}^{*}$ and the anti-ideal value $r_j{}^{**}$ are the best and worst rating values of criterion i for the decision-maker DM_j.

After the initial rating, matrix (7) is transformed by Equation (11), a divergence parameter σ_j is computed as a quantity of the intensity contrast that DM_j has concerning the decision-making process for each column of the new matrix. To perform a correlation of columns of the new matrix, a symmetric matrix Q with dimension m·m is created, containing computed correlation coefficients. If ratings of criteria for decision-makers DM_j and DM_k are more different, the correlation coefficient q_jk is lower. The summation represented by Equation (12) indicates the degree of conflict decision-maker DM_j has with respect to other decision-makers, with a greater value of Equation (12), indicating a greater conflict of DM_j with other decision-makers.

$${\mathrm{g}}_j={\displaystyle \sum }_{k=1}^m\left(1-q_{jk}\right),\hspace{1em}j=1, 2, \dots , m.$$

(12)

Considering that the information contained in the decision-making problem relates to the contrast and conflict of decision-makers, the quantity of information emitted by DM_j is determined by applying multiplicative aggregation:

$$t_j={\sigma }_j{\mathrm{g}}_{\mathrm{j}},\hspace{1em}\hspace{1em}j=1, 2, \dots , m.$$

(13)

The greater the value of t_j, the more information that is emitted by the DM_j towards the importance of criteria, and that the decision-maker is more important for deriving the final decision. Objective weights of the decision-makers are finally obtained by applying the normalization process (14):

$$w_j=t_j·{\left[{\displaystyle \sum }_{k=1}^m{t}_k\right]}^{-1}, j=1, 2, \dots , m.$$

(14)

2.4. Aggregating Method

Aggregation is performed here by using the weighted geometric mean method (WGMM), as in Equation (15):

$$Z_i={\displaystyle \prod }_{k=1}^m{Z}_{ik}^{w_k},\hspace{1em}i=1, 2, \dots , n.$$

(15)

Before Equation (15) is applied, the individual weights w_k are additively normalized. In this approach, variable z represents individually derived criteria weights obtained using either the AHP or BWM methods.

2.5. Problem Statement and Assessment Procedure

The first step in assessing the quality of the criteria when evaluating the urban parks was to identify relevant decision-makers who will select and assess the criteria. To enable a comprehensive approach and diverse views of the problem, members of the group (designated as decision-makers DM₁–DM₅) were selected from the following areas of professional interest: DM₁—urban greenery; DM₂—urban forestry; DM₃—environmental protection; DM₄—landscape design and DM₅—cultural and historical heritage. The aim of including the above-listed decision-makers was to provide an objective insight to the analyzed problem, taking into account different perspectives.

The earlier research on criteria for assessing urban parks [16,17,18] was then briefly presented to the experts. Additional information was provided to the experts related to recent ‘greenery’ studies undertaken for the city of Novi Sad, the capital of the Vojvodina Province in Serbia, and we provided insights into pertinent literature published worldwide, e.g., [9,10,16,17,18]. Among many possible criteria, the expert academics agreed by consensus to adopt the following set of eight criteria as the most suitable for a given problem: C₁—accessibility; C₂—location; C₃—biodiversity preservation; C₄—park equipment; C₅—water elements (lakes, fountains, etc.); C₆—terrain configuration; C₇—cultural and historical value; C₈—presence of small architectural objects (pavilions, gazebos, etc.).

A description of the criteria is presented in

Table 2. Criteria description.

Label	Criterion	Description
C1	Accessibility	This refers to two main aspects, the outside and inside accessibility of the park. The former refers to the ease of reaching the park by pedestrian and bike paths as well as by using public transportation and cars. The latter refers to accessibility within the park, especially for people in wheelchairs or with walking disabilities, and takes into account the existence of an appropriate path system with adequate slopes that make the park accessible for all categories of users.
C2	Location	This considers the disposition of a park within the city and takes into account walking distances from different city settlements to a specific park as well as the closeness of a park to city landmarks that city dwellers frequently visit. In these terms, the locations of parks close to city settlements or city attractions are considered the most favorable.
C3	Biodiversity preservation	This takes into account the park’s potential to maintain biodiversity within the urban ecosystem. This criterion encompasses the diversity of plants and some animal species (firstly insects and birds) within the park. When analyzing the plant species composition, the focus is on the presence of autochthonous and endemic species whose survival might be endangered in the city environment.
C4	Park equipment	This refers to the quality of the park equipment for both active (sports) and passive activities (rest) in the park zone. This implies that the equipment is safe, well-maintained, and successfully incorporated into the spatial setting.
C5	Water elements (lakes, fountains, etc.)	This includes the existence of fountains, lakes, etc. These elements have multiple functions; in certain cases, they provide sources of drinking water, but they also have an aesthetic function and are valuable in terms of modifying the microclimate in the park.
C6	Terrain configuration	This refers to the shape of the ground level of the park. Parks with more flat ground are usually more attractive to park visitors.
C7	Cultural and historical value	This refers to the existence of monuments or different park elements (both botanical and architectural) that are associated with past periods, and their value is higher if the remains of the original design of the park still exist.
C8	Presence of small architectural objects	This includes the existence of pavilions, gazebos, and additional park equipment that can be used for different purposes, such as concerts and public gatherings, which are not necessarily seen in every city park and present added value to a park’s functions.

.

We present a graphical representation of the decision framework in Figure 1.

The experts DM₁–DM₅ were asked to evaluate the criteria in two separate sessions by providing multiple-preference relations required by (1) AHP and (2) BWM. The first session was planned as an initial individual evaluation of criteria followed by the application of several aggregation schemes for group prioritization of criteria. While in the first session (AHP), a full-range approach was applied with a complete set of paired comparisons of criteria, the second session (BWM) provided information on how decision-makers prioritize criteria if they make a restricted number of comparisons. In a way, the second session (BWM) simulated possible real-life situations in which changes in the DM’s opinion may occur for many different reasons.

3. Results

3.1. AHP Application

Step 1: Each member of the group provided (8 × 7)/2 = 28 multiple-preference relations of criteria using Saaty’s scale. Criteria weights were computed by the application of the eigenvector method and ordered by importance (

Table 3. Individual weights of criteria derived using AHP and consistency measures of decision-makers.

Decision-Makers	Weights of Criteria and Their Ranks								Consistency Measures
	$w_1$	$w_2$	$w_3$	$w_4$	$w_5$	$w_6$	$w_7$	$w_8$	CR	EDIST
DM1	0.170	0.072	0.313	0.136	0.047	0.023	0.218	0.020	0.081	13.151
Rank	3	5	1	4	6	7	2	8
DM2	0.113	0.137	0.340	0.088	0.034	0.023	0.214	0.052	0.049	9.397
Rank	4	3	1	5	7	8	2	6
DM3	0.232	0.047	0.267	0.075	0.142	0.027	0.114	0.096	0.131	9.019
Rank	2	7	1	6	3	8	4	5
DM4	0.056	0.058	0.301	0.090	0.221	0.054	0.191	0.030	0.102	8.951
Rank	6	5	1	4	2	7	3	8
DM5	0.105	0.189	0.313	0.107	0.191	0.032	0.045	0017	0.084	14.022
Rank	5	3	1	4	2	7	6	8
Average	0.135	0.101	0.307	0.099	0.127	0.032	0.157	0.043	0.089	10.908
Rank	3	5	1	6	4	8	2	7

). The last two columns in Table 3 shows the values for the computed consistency ratio CR and total Euclidean distance EDIST, which is a sum of the squared differences of ratios w_i/w_j and original judgments a_ij (for all i,j = 1, 2, …, 8).

Averages and corresponding ranks in the last two rows of Table 3 show that the highest and lowest position criteria were C3 and C6, respectively. All decision-makers agreed that C3 was the most important criterion; criteria C6 and C8 were recognized as the least important by almost all decision-makers.

The consistency metric CR for all decision-makers was satisfactory. The recommended threshold value CR_max = 0.1 was slightly surpassed by two of the five decision-makers, but can be considered acceptable, as suggested in many AHP applications and discussed in more detail by [32].

Step 2: Criteria weights contained in Table 3 were used to produce objective weights of decision-makers using the ENTROPY and CRITIC methods (

Table 4. Weights of decision-makers determined using ENTROPY and CRITIC methods based on weights of criteria obtained using the AHP method.

Method	Weights of Decision-Makers
	DM1	DM2	DM3	DM4	DM5
ENTROPY	0.229	0.217	0.148	0.193	0.212
Rank	1	2	5	4	3
CRITIC	0.203	0.192	0.202	0.209	0.194
Rank	2	5	3	1	4

). The weights of the decision-makers were different, as would be expected. However, except in the case of DM₃ and the ENTROPY method, in all cases the weights of decision-makers were similar (close to equal values), which may have influenced the final aggregation of criteria weights in the group context.

Step 3: Criteria weights obtained individually by decision-makers (Table 3) were aggregated geometrically via WGMM using three sets of decision-maker weights. The first set contained equal weights for all five DMs and the remaining two contained weights for the ENTROPY and CRITIC methods given in Table 4.

Aggregations indicated almost perfect agreement of both cardinal values (weights) and ordering (ranks) of criteria. The top-ranked criteria were C₃–C₇–C₁, with a total weight of 62%, while the lowest ranked were C₂–C₈–C₆, with a total weight of 17%; the weight of the two remaining criteria, C₄–C₅, positioned between the top and lowest ranked criteria, was about 21%.

Step 4: The AHP application indicated that the group of five DMs unanimously identified criterion C₃ (biodiversity) as the most important in the evaluation of urban green areas, such as parks, grasslands, gardens, and green roofs. Criterion C₇ was ranked second by importance (cultural and historical value), which probably corresponds to the general feeling that urban parks must enable visitors with a broader perception of not only natural characteristics but also human-made or inspired activities and characteristics. Accessibility (C₁) was the third-ranked criterion, which indicates DMs’ preference to validate the quality of infrastructure (streets, roads, traffic means, parking spots, etc.) that is compatible with easy or hard access to green areas and primarily city parks.

3.2. BWM Application

Step 1: Each member of the group provided 2 × 8 − 3 = 13 multiple-preference relations of eight criteria with regard to their preferences, and stated which criterion was the best and which criterion was the worst. Saaty’s scale was used to express individual preferences, as presented in

Table 5. Multiple preference relation vectors for the best criterion.

The Decision-Maker (Best Criterion)	Best-To-Others
	C1	C2	C3	C4	C5	C6	C7	C8
DM1 (C3)	3	6	1	4	7	8	2	9
DM2 (C3)	4	2	1	5	8	9	3	7
DM3 (C3)	3	5	1	5	3	7	3	3
DM4 (C3)	3	3	1	2	2	7	4	8
DM5 (C3)	3	4	1	2	2	6	7	8

and

Table 6. Multiple preference relation vectors for the worst criterion.

Others-To-the-Worst	Decision-Makers (Worst Criterion)
	DM1 (C8)	DM2 (C6)	DM3 (C6)	DM4 (C8)	DM5 (C8)
C1	7	5	5	3	4
C2	4	8	3	4	6
C3	9	9	7	8	9
C4	5	4	3	6	7
C5	3	2	5	7	8
C6	2	1	1	3	4
C7	8	7	7	5	5
C8	1	3	5	1	1

.

All five decision-makers identified criterion C₃ (biodiversity) as the best. Three decision-makers thought that criterion C₈ (presence of small architectural objects) was the worst. The remaining two decision-makers considered criterion C₆ (terrain configuration) as the worst. Through the application of the linear BWM model, weights of criteria were derived for all decision-makers and are summarized in

Table 7. Individually derived weights of criteria using BWM.

Decision-Makers	Weights of Criteria and Their Ranks								Dummy Variable
	$w_1$	$w_2$	$w_3$	$w_4$	$w_5$	$w_6$	$w_7$	$w_8$
DM1	0.138	0.069	0.342	0.103	0.059	0.052	0.207	0.030	0.071
Rank	3	5	1	4	6	7	2	8
DM2	0.102	0.204	0.337	0.082	0.051	0.029	0.136	0.058	0.072
Rank	4	2	1	5	7	8	3	6
DM3	0.129	0.077	0.300	0.077	0.129	0.031	0.129	0.129	0.086
Rank	2–5	6–7	1	6–7	2–5	8	2–5	2–5
DM4	0.112	0.112	0.280	0.168	0.168	0.048	0.084	0.028	0.056
Rank	4–5	4–5	1	2–3	2–3	7	6	8
DM5	0.121	0.091	0.286	0.182	0.182	0.061	0.052	0.026	0.078
Rank	4	5	1	2–3	2–3	6	7	8
Average	0.120	0.111	0.309	0.122	0.118	0.044	0.122	0.054	-
Rank	4	6	1	2–3	5	8	2–3	7	-

. Simple averaging of weights across decision-makers indicated criterion C₃ as the most important and C₈ as the least important.

Step 2: Criteria weights contained in Table 7 were used to compute objective weights of decision-makers using the ENTROPY and CRITIC methods (

Table 8. Weights of decision-makers determined using ENTROPY and CRITIC methods based on weights of criteria obtained using the BWM.

Method	Weights of Decision-Makers
	DM1	DM2	DM3	DM4	DM5
ENTROPY	0.250	0.241	0.149	0.167	0.194
Rank	1	2	5	4	3
CRITIC	0.207	0.205	0.183	0.197	0.207
Rank	1–2	3	5	4	1–2

). The summed weights of the first two decision-makers (DM₁ and DM₂) obtained using the ENTROPY method were the same as the summed weights of the remaining three decision-makers. Conversely, in the case of the CRITIC method, almost all decision-makers had similar weights, ranging from 0.183 to 0.207. Decision-makers DM₁, DM₂, and DM₅ had almost equal weights and can be considered as a sub-group.

Step 3: Criteria weights obtained individually by decision-makers (Table 8) were aggregated geometrically via the WGMM using three sets of weights of decision-makers. The first set contained equal weights for all five DMs and the remaining two contained weights for the ENTROPY and CRITIC methods (Table 7).

Aggregations indicated good agreement of both cardinal values (weights) and ordering (ranks) of criteria. The top-ranked criterion was again C₃, with a total weight range of 31–33%. The next three criteria were C₁, C₄, and C₇, with a total weight of about 37%. The last four criteria weights equaled 30%.

Step 4: The BWM application, one month after the session in which the AHP was applied, repeatedly indicated that the group believed that criterion C₃ (biodiversity) was the most important and that its weight was approximately one-third of the total weight of all eight criteria. Second by importance was criterion C₁ (accessibility), with a weight of approximately 13%, which was very close to the third-ranked C₇ (cultural and historical value), with a weight of approximately 12%.

4. Discussion

This research builds on, and can be compared with, some previously published papers, for example [33]. The article examines urban green spaces from the viewpoint of relevant indicators, in particular the quantity and availability of urban green spaces, changes in green spaces, planning of urban green spaces, financing of urban green spaces, and level of performance. In addition, the paper discusses the issue of implementing a green space policy in cities in Europe and presents successful examples in assessments of the complex and heterogeneous supply of urban green spaces using a multidimensional evaluation approach.

This research was also influenced and shaped by the ideas provided in [7]. The latter study proposes a comprehensive methodology for evaluating the safety of urban parks, which includes a multi-criteria-based ranking of a given set of urban public parks according to their safety level, and uses a modified weighted aggregated sum product technique in order to solve the constructed multi-criteria decision-making problem related to six urban public parks in Vilnius, Lithuania.

Another paper that was a useful reference for designing the structure of the questionnaire in this research was [7]. The authors propose the use of a combined questionnaire–statistical method for the multi-criteria evaluation of urban parks, based on the model of the integrated quality of the park described by its ability to perform its functions during a specified period. The integrated index of park quality is defined through two main factors: park functionality and park protection. Functionality comprises ecological, recreational, ergonomic, and aesthetic functions. Protection includes legal protection and the financial security of the park. A group of experts assigned weights to many sub-factors (such as, for ergonomic function: visitor safety + park accessibility + park convenience); their spreadsheets were statistically evaluated and aggregated with a certain sensitivity analysis if the weights were within an allowable range of variation.

The above referenced papers were useful in terms of both selecting criteria (indicators) and defining the appropriate methodology for interviewing decision-makers. The focus was to include all relevant criteria for defining a decision-making problem and to apply a combination of methods and techniques that has not been used before. Therefore, the paper presents a sequel to papers that have discussed the problem of group decision-making in urban park management.

Two aspects of group decision-making are analyzed in this study. One is related to answering the question: What are the effects of changing information structure and multi-criteria methods that are based on performing a different number of pair-wise comparisons of criteria? The other aspect relates to an assessment of the ‘cognitive performance’ of experts in two distinct sessions with the possibility of changes in their personal opinions, forgetting their earlier judgments, and not using their earlier evaluations. The results of this study show that either the analytic hierarchy process or the best–worst method can efficiently help in evaluations of criteria for assessing urban park quality if the members of a group are consistent, regardless of whether the consensus process is properly carried out before the decision-making process.

Through the weighted geometric aggregation of individual criteria weights in a group context for all combinations of the four named methods, it was possible to evaluate, critically analyze, and formulate conclusions and recommendations. One of the important conclusions is that any of the three aggregation schemes (geometric averaging, ENTROPY, or statistics) perform well and may help in declaring the group solution to be trustworthy.

Five academic experts (‘decision-makers’) evaluated eight criteria as relevant for assessing the quality of urban green areas, with a focus on public city parks. The evaluation process was organized in two separate sessions. During the first session, experts individually applied the AHP method to derive the weights of criteria, and the authors of this research used several types of aggregation to calculate the weights of criteria as the group result. An implicit assumption was that experts will use the full range of Saaty’s 9-point scale and perform all required 8 × 7/2 = 28 pair-wise comparisons of criteria. This case was considered to be a full-information evaluation context. The second session was organized one month after the first one. This time the reduced-information context was established by asking the experts to apply the BWM in which they had to perform fewer pair-wise comparisons of criteria. Instead of 28, only 2 × 8 − 3 = 13 comparisons were made by each expert in this session. The information base during the BWM application for deriving criteria weights was ‘cognitively’ reduced by more than half compared to the AHP method.

An important issue is that the BWM requires the expert to declare in advance which criterion is the best and which criterion is the worst. In the AHP, such declarations are not possible. Instead, experts can implicitly force high importance and low importance of certain criteria through the evaluation process but not in an explicit way as the BWM.

The application of the AHP and BWM in separate and time-distant sessions served as an experiment in which the expertise of evaluators was checked not only about performing a different number of pair-wise comparisons, but also concerning their cognitive characteristics, e.g., possibly forgetting or changing their individual preferences during the one-month pause after the first session. This may happen in all real-life situations as was the case in this study.

Three aggregation schemes were applied for both AHP and BWM: decision-makers had equal importance, weights were obtained using the ENTROPY method, and weights were obtained using the CRITIC method, as summarized in

Table 9. Weights of decision-makers.

MCDM Method	Scheme	Weights of Decision-Makers
		DM1	DM2	DM3	DM4	DM5
AHP	Equal	0.200	0.200	0.200	0.200	0.200
	ENTROPY	0.229	0.217	0.148	0.193	0.212
	CRITIC	0.203	0.192	0.202	0.209	0.194
BWM	Equal	0.200	0.200	0.200	0.200	0.200
	ENTROPY	0.250	0.241	0.149	0.167	0.194
	CRITIC	0.207	0.205	0.183	0.197	0.207

. Differences in their weights were not significant, especially if the values derived by the ENTROPY and CRITIC methods are compared with equal weights associated with all decision-makers.

The final results of the group evaluation of criteria relevant for evaluating the multifunctional value of urban parks are presented in

Table 10. AHP and BW aggregated weights of criteria using geometric, ENTROPY, and CRITIC methods.

MCDM Method	Type of Aggregation	Weights of Criteria **								Criteria Ordering
		$w_1$	$w_2$	$w_3$	$w_4$	$w_5$	$w_6$	$w_7$	$w_8$
AHP	Geometric *	0.133	0.096	0.335	0.107	0.109	0.033	0.149	0.038	(37154286)
	ENTROPY	0.130	0.101	0.339	0.109	0.103	0.033	0.151	0.036	(37145286)
	CRITIC	0.133	0.095	0.335	0.107	0.110	0.033	0.150	0.038	(37154286)
BWM	Geometric *	0.120	0.111	0.309	0.122	0.118	0.044	0.122	0.054	(34715286)
	ENTROPY	0.127	0.110	0.331	0.120	0.101	0.045	0.122	0.045	(31742586)
	CRITIC	0.127	0.109	0.327	0.122	0.109	0.045	0.116	0.046	(31742586)

* Weights of decision-makers are equal (0.20), ** C₁—Accessibility; C₂—Location; C₃—Biodiversity preservation; C₄—Park equipment; C₅—Water elements (lakes, fountains, etc.); C₆—Terrain configuration; C₇—Cultural and historical value; C₈—Presence of small architectural objects (pavilions, gazebos, etc.).

.

In all listed cases, criterion C₃ (biodiversity preservation) was recognized as the most important when evaluating the quality of urban parks. The second most important criterion was C₇ (cultural and historical value) when AHP was used and any of three aggregations were applied. In the case of BWM, the second most important criterion was identified as C₁ (accessibility), except in the case of the ‘equal weights’ scheme (for decision-makers) when the second-ranked criterion was C₄ (park equipment).

The criteria ordering presented in the last column of Table 10 indicates that the three top-ranked criteria were always C₁, C₃, and C₇, except in the case of the BWM with equal weights, in which case criterion C₄ can be added to the group of criteria with dominating weight over the others. This means that in some situations the initial set of eight criteria can be cut by half, reducing the number of required comparisons of decision elements (criteria by importance, and parks vs. criteria) when either method is used, AHP or BWM.

The similarity of the results obtained by the AHP and BWM indicates that in the case of the latter method, the reduction in information may not be critical, especially in cases when a relatively small number of urban parks are evaluated for quality. For instance, in Novi Sad City, the evaluation of five city parks may be of interest, as some recent studies already indicated [14]. A review of the reported research results (presented in the introductory section) obtained in different cities and countries shows that in a prevailing number of cases, assessments were undertaken for only three or four parks.

It is worth mentioning that we identified more than 50 criteria used in different studies of the quality of urban parks [4,7,9,10], and we partly referred to this issue throughout the paper. A more comprehensive review of criteria was avoided, but we are aware that the methodology we have described could help to plan where and how much to invest into maintenance, security, and internal and external infrastructure improvements related to city parks, which means also considering other criteria that might be added to those we used. Furthermore, without losing the generality of our approach, we have also argued that it is important in the assessment of urban parks to elicit judgments not only from academics but even more importantly from experts and other experienced professionals in greenery matters, security provision, culture and heritage initiatives, environmental protection, and diversification of flora and fauna, which may also enlarge the set of criteria. However, it is important to have a balanced approach to the decision-making process and be aware of not having too many criteria that would make the judgment process too tedious for decision-makers. Keeping that in mind, we selected a reasonable number of representative criteria while respecting the limitations of two generally distinct aspects: (1) group MCDM (AHP/BWM)-based methodology with its inherent requirements; (2) assessment of urban park quality by validating the importance of a certain set of criteria.

5. Conclusions

The focus of this study is an analysis of possible differences in solutions of the two conceptually different multi-criteria methods (here, AHP and BWM) based on multiplicative preference relations which assume independence of compared decision elements (criteria in our study) for evaluating the quality of urban parks. The weights of criteria were individually derived by academic experts after they agreed that criteria can be considered sufficiently independent.

There are some limitations of the present study. The critical issues in the quality assessments of urban parks are obviously the selection of criteria for evaluating parks on the one hand, while on the other hand the selection of individuals or sub-groups to be included in the participatory decision-making process. Both issues are related to different aspects involved in deriving the best solutions at a city level and depend on local societal, institutional, and political circumstances; the influence of interests; and readiness to apply scientifically proven methods and methodologies for deriving common implementable solutions [34,35]. Important aspects of the overall strategy and adopted solution methodology could include where and how much to invest into maintenance, security, and internal and external infrastructure improvements, as well as how to elicit the judgment of experts—not only academic, but even more importantly experienced professionals in greenery matters, security provision, culture and heritage initiatives, environmental protection, and diversification of flora and fauna. Notice, however, that any MCDM approach, including the ones presented in this study, must not handle problems with too many criteria because this may lead to a tedious judgment process and possibly significant inconsistencies between decision-makers.

Effective urban park management requires strong involvement from interested parties and the provision of a harmonized decision-making process by applying scientific models and methodologies. In these terms, it is important that all decision-making groups adopt clear and understandable procedures for expressing their options and assessing criteria and attributes related to urban parks, and possible procedures have been demonstrated in this research. A note for future research is that the decision-makers should be carefully selected, especially if they are dealing with certain delicate or controversial topics, such as biodiversity indicators. Although both AHP and BWM are efficient and effective decision support methods, the quality of the final decision relies on the competencies of the selected decision-makers.