Comparison of AHP and a Utility-Based Theory Method for Selected Vertical and Horizontal Forest Structure Indicators in the Sustainability Assessment of Forest Management in the Sierra de Guadarrama National Park, Madrid Region

Abstract

This paper compares two pairwise comparison methods, the analytic hierarchy process (AHP) and a utility theory based method (UTB method), for sustainability assessment in forest management at the local level. Six alternatives were ranked, corresponding to six different types of forest management in the Sierra de Guadarrama National Park in the Madrid Region in Spain. The methods were tested by postgraduate students enrolled in a “Decision Support Systems” course at Universidad Politécnica de Madrid. Three sustainability indicators were considered: structural diversity, timber yield, and amount of biomass. The utility theory based method was the first to be compared, which is implemented in the computer program SILVANET. For each pair of alternatives, the students were asked which one they considered to be more sustainable. In the case of the Analytic Hierarchy Process, the students compared the indicators and the alternatives for each indicator. The Spearman’s correlation coefficient indicated that there was no correlation between the rankings for most of the students. The results revealed that the convergence in opinion in the AHP method was higher than in the utility based method for a low number of participants, and distinguished the differences between the alternatives more accurately. However in the case of the UTB method, the participants considered sustainability as a whole and made a more context-based comparison.

1. Introduction

Since the United Nations Conference on the Environment (UNCED) in Rio de Janeiro in 1992, sustainability and the development of indicators for Sustainable Forest Management (SFM) have received greater attention, and have advanced [1]. The 1993 Helsinki resolution defined the term SFM in the European context as “the stewardship and use of forests and forest lands in a way, and at a rate, that maintains their biodiversity, productivity, regeneration capacity, vitality and their potential to fulfill, now and in the future, relevant ecological, economic and social functions, at local, national, and global levels, and that does not cause damage to other ecosystems” [2].

Several SFM processes have been enacted over the last 25 years to develop criteria and indicators for the conceptualization, monitoring and creation of a solid basis for decision-making at international, national, and local levels [1,3]. During this time, these criteria and indicators have evolved in line with the thinking on SFM and have gradually adapted to the changes in the ecological, economic and social aspects of the forest and to new demands from other processes such as the Convention on Climate Change or the UN Convention to Combat Desertification. However, all these international processes converge to similar principles that are also adopted in the global-United Nations definition of SFM as a “a dynamic and evolving concept [that aims] to maintain and enhance the economic, social, and environmental values of all types of forests, for the benefit of present and future generations”. Key issues like forest resources, afforestation/reforestation, health and vitality, damages, productive functions, biodiversity, protective functions, and economic issues are well covered by indicators in all C&I processes [4].

The Rio conference and the Montreal process marked the incorporation of public participation in forest decision-making. Criteria and indicators were developed to facilitate this public and stakeholder participation and societal consensus in SFM [1,3,5]. This has enabled citizens to influence issues concerning environmental and natural resources and forest management, such as sustainability assessment at the national, regional, and local level. Kates et al. (2001) [6] define sustainability assessment as “an evaluation of global to local integrated nature-society systems in short- and long-term perspectives in order to assist them to determine which actions should or should not be taken in an attempt to make society sustainable”. Sustainability assessment provides scientific support for decision making. Its outcomes improve participation, iteration and transparency in decision-making processes [7]. However, as sustainability is a far-reaching concept, it depends on socio-political meanings and environmental dynamics. Its assessment requires further development to be universally accepted [4,5,8,9]. There have been several initiatives at the country level, such as the one developed by FOREST EUROPE, UNECE, and FAO in 2011, and the one developed by UNECE/FAO in 2016. Both proposed experimental methods based on pan-European criteria and indicators for SFM, and were common for every country involved in the process [10,11].

This paper is framed at the local level in the Sierra de Guadarrama National Park. At this level, forest management plans are the main tool for SFM implementation; however C&I are top-down in their development, so in order to ensure the success of the plans, local conditions (forest types, economy, and traditions) must be taken into account [4], making it essential to obtain the participation of the local community.

One of the features of public participation in forest management is that the process must be adapted to the specific characteristics of the context in which it is to take place [12,13]. One of the greatest problems in currently available methods is their lack of flexibility [14]. Despite this, numerous studies have sought to design more generic processes for public participation that can be applied in various contexts such as negotiating national forest plans [12] or designing regional plans [15,16].

This study defines a process as the different forms of public participation, as described with their respective structures and procedures [17]; and methods are defined as the techniques used to obtain, analyze and synthesize—among other procedures—both qualitative and quantitative data [18].

The methods applied in the process depend on the phase of the process itself, the availability of resources and the prior experiences in public participation in the area. There is therefore no “most suitable” method for all contexts [14,19,20,21,22]. The choice of tools may affect the outcome of the process [23]. This is a key aspect, as in public participation processes the agreements that are reached after applying the corresponding method may derive in beneficial agreements [13] or become a focus of social conflict [24]. The most frequent methods applied in participatory processes in sustainable forest management and assessment are negotiations with stakeholders and voting systems [25,26,27,28,29,30,31], and multicriteria decision-making methods [31,32,33,34,35,36,37,38]. Another method for weighting criteria is the pair wise comparison. It stems from the Analytic Hierarchy Process (AHP), a decision-making framework developed by Thomas Saaty (1980) [39]. Specifically, AHP is the most popular multiple criteria decision support in public participation processes in forest management [19]. In fact, according to Esmail and Geneletti (2017) and Diaz-Balteiro and Romero (2008) [40,41], over 98 implementations of this tool have been recorded in nature conservation and forest management. AHP aims to break down the decision problem into a hierarchy, ordered from the highest-level objective—in this study sustainability—, middle level criteria, SFM indicators and alternatives, and forest sites with different management in the study area at the lowest level. AHP employs pairwise comparison of criteria and alternatives which are translated into weights. The utility theory based method (UTB method), following the developments of Fishburn—and hence utility theory—has been successfully applied to assess management sustainability and to determine stakeholders’ preferred management plan [42,43,44]. It is based on a direct (non-hierarchical) process of pairwise comparison of alternatives that show the same forest sites applied in AHP. The properties of each user’s preferences are analyzed and the alternatives are ordered according to a value function that expresses the number of pairs in which a specific forest site was considered to be more sustainable than the other site in the pair of alternatives. The UTB method has been implemented in the SILVANET computer program [45]. The users are shown pairs of alternatives and their characteristics, and also documentation on indicators and SFM available in the program. The user only has to tell the program which alternative he or she considers to be more sustainable. As output, the first SILVANET module ranks the alternatives, the second shows the user’s sustainability map of the study area, and the third shows the user’s management plan [43,44].

This study was carried out in the Sierra de Guadarrama National Park. The park is managed according to the 2009–2019 Sierra de Guadarrama Management Plan (DECRETO 96/2009). The collaboration of all the stakeholders, including administration, experts, the local community and the general public is essential for the design of this ordination plan and for any future modifications. This is why it is necessary to use the most appropriate tools to design the public participation processes when there is a wide variety of stakeholders, and to consider the objectives of the management plan in these processes.

For the choice of the SFM indicators used in this study we also considered the objectives of the plan: improvement of the forest structure, regulated exploitation of timber, the park’s function to reduce CO₂ by increasing biomass. Finally, these indicators were: structural diversity, timber yield, and amount of biomass [43].

The main aim of this article is to contribute to determining the most suitable method for SFM assessment at the local level through stakeholder participation. Specifically, the objective is to analyze the differences and similarities in the results by comparing two methods based on pairwise comparison: one hierarchical (AHP), and another in which the sustainability of forest sites is directly compared (UTB method) and how their properties affect the results when assessing the sustainability of forest management at the local level in the Sierra de Guadarrama National Park.

2. Material and Methods

2.1. Study Area

The study area is located in the Fuenfría Valley in the Sierra de Guadarrama National Park (Madrid, Spain); see Figure 1. This park covers 34,000 ha in the northwest of the Madrid region and the south of the Castile-León region. The uses of the national park are recreational, conservational and productive.

The study area covers 127 ha (40°45′ N, 4°5′ W) of the national park in the Fuenfría Valley. Average altitude is 1520 m, average annual temperature is 9.4 °C, and average annual rainfall is 1180 mm.

The main species is Scots pine (Pinus sylvestris L.), with five different types of structure, as described by Pascual et al. (2008) [46]:

• “Type 1: Uneven-aged forest (multi-layered canopy) with very high crown cover. Area: 26 ha.
• Type 2: Multi-diameter forest with high crown cover. Area: 24 ha.
• Type 3: Multi-diameter forest with medium crown cover. Area: 29 ha.
• Type 4: Even-aged forest (single-story) with low crown cover. Area: 37 ha.
• Type 5: Zones with scarce tree cover. Area: 11 ha.”

Table 1. Dendrometric characteristics of the forest.

	Forest Attributes
	Maximum	Minimum
Density (tree/ha)	850.0	75.6
Basal area (m2/ha)	40.7	6.6
Mean height (m)	14.7	8.7

shows a summary of the dendrometric characteristics of the forest [46].

2.2. Identifying the Alternatives

The sites or alternatives needed to represent the different management types in the study area. In our case we selected the six different forest sites or alternatives (N = 6) considered in the preference identification module in the SILVANET computer program. A method of similarity to an ideal point [47,48] was used to identify the alternatives. The ideal point (IP) was defined by Martin-Fernández and Martinez-Falero, 2018 [44] as “the theoretical point where the sustainability indicators would have the highest possible value of 100”. The value of the “Mahalanobis distance” [49] between the values of the sustainability variables (which ranged from 0 to 100) at any point, and the ideal point (IP) was obtained.

Table 2. Location of the alternatives and distances between the alternatives and the ideal point (IP).

Alternatives	A	B	C	D	E	F
Distance to IP	1.16%	23.77%	43.46%	60.91%	62.87%	89.11%
UTM-X coordinate	409,016.5	408,716.5	409,096.5	409,116.5	408,316.5	408,616.5
UTM-Y coordinate	4,512,872.5	4,512,272.5	4,512,552.5	4,512,732.5	4,512,312.5	4,512,732.5

shows the distances between all the alternatives finally selected and the IP.

2.3. Utility Theory Based Method

In this method, the participants directly compared the pairs of alternatives and indicated which one was more sustainable according to their preferences.

Pairwise comparison is represented by a binary relation (P) in the set of meaningful locations (Ω) with $\subset \mathsf{\Omega }\subset \mathsf{\Omega }$, which here designated the preference relation. This relation is expressed by a {1, 2, 3} matrix (p_xy = 1 if, and only if, xPy (sustainability in alternative x is greater than in y); p_xy = 2 if, and only if, yPx (sustainability in alternative y is greater than in x); otherwise, p_xy = 3 if, and only if xIy (sustainability in x and in y is indifferent)). Figure 2 shows a participant’s preference matrix.

The individual’s preference type was then determined by analyzing the properties of the preference matrix. The properties fulfilled revealed the type of objective function.

Table 3. Mathematical properties of the preference.

Type of Preference Relation in Alphabetical Order	Mathematical Properties Required
Biorder	Strong Intervality
Interval order	Irreflexivity, Strong intervality
Linear order	Asymetric; Negatively transitive; Complete
Partial order	Irreflexivity; Transitivity
Semiorder	Irreflexivity; Semitransitivity; Strong intervality
Weak order	Irreflexivity; Negatively transitive; Transitivity

shows which properties are fulfilled by each preference type. A detailed description can be seen in [45].

Since this method is based on the utility theory, it is necessary to formulate a value function. The value function used is described in Martinez-Falero et al. [45] as “the number of pairs in which the forest site, x, was considered by an individual to be more sustainable than the other element in the pair of alternatives. The value of u(x) may be from 0 to N − 1, so from 0 to 5”.

If the binary relation (P) was a linear order, then $u\left(x\right)\ge u\left(y\right)⟺x\succcurlyeq y,\forall x,y\in \mathsf{\Omega }$. Therefore u(x) is the objective function for the participant. In another case, “the transitive closure” was applied to improve the linearity of the preference [50].

$$S_0\left(P\right)=\left[IP\cup PI\right]\cap cd\left[IP\cup PI\right];\dots ; S_i\left(P\right)=S_0\left(S_{i-1}\left(P\right)\right) \mathrm{for}i=1,2,\dots$$

(1)

where:

• $IP=\left\{\left(x,y\right):xIz \mathrm{and} zPy \mathrm{for}\mathrm{some} z\in \mathsf{\Omega }\right\}$;
• $c\left(P\right)=\left\{\left(x,y\right):x \mathrm{no}P y\right\} \mathrm{and} d\left(P\right)=\left\{\left(x,y\right):yPx\right\}$

“Transitivity closure” did not modify the individual’s preferences, so the value function could be calculated with the new matrix.

2.4. AHP Method

The analytic hierarchy process (AHP) was developed by Thomas L. Saaty (1977) [51], and is based on humans’ ability to use their own experience and knowledge to produce relative magnitudes through pairwise comparison [52]. According to Saaty (1977) [51]: “The purpose of the method is to enable the decision maker to structure a multi-criteria problem by building a hierarchical model”. In our case the structure had three levels: objective, criteria, and alternatives. The hierarchy structure of the objectives in our study is shown in Figure 3. In this figure, the first level of the structure corresponds to the objective of the AHP model in our study: “Ordering alternatives according to their sustainability”. This objective is broken down into three criteria corresponding to the indicators for structural diversity, timber yield and amount of biomass. Finally, the third level corresponds to the six alternatives, namely the six forest sites described in Section 2.2.

Pairwise comparisons were then performed between the criteria and the alternatives. The participants completed square matrices with the numerical values representing their preferences. The scale for the comparisons in this work was the nine-point scale developed by Saaty [39]. The values ranged from 1, “indifference in the preference” to 9 “much greater preference”.

Any comparison matrix, A, has three properties:

• a_ij > 0, for all i, j
• a_ij = 1, if there is indifference in the preference between elements i and j and a_ii = 1 for each i;
• a_ji = 1/a_ij for each i, j.

To obtain the weights of the criteria and alternatives from the matrices we applied the eigenvector method proposed by Saaty [51].

Finally, weights and preferences are integrated through the following expression for each alternative:

$$X_i = {\displaystyle \sum }_{j=1}^3{W}_j{X}_{ij}$$

(2)

where:

• X_i is the integrated preference of alternative i.
• W_j is the weight obtained for criteria j.
• X_ij is the preference of alternative i for criteria j.

2.5. Transitivity of the Preference

In the case of Fishburn’s method, consistency was measured through the property of semitransitivity. The results for each participant were input in the SILVANET software [45] to verify the semitransitivity of each participant. For semitransitivity to exist, the preference matrix must meet the following conditions [42]:

$$\begin{gathered} 1:xPy e zPt \to xPt ó tPy, \forall x,y,z,t \in \Omega . \mathrm{Also}: \\ 2:xPyPz \to xPt ó zPy, \forall x,y,z,t \in \Omega \end{gathered}$$

(3)

This software incorporates algorithms that assess this property and indicate whether each participant has semitransitivity in their preferences or not.

In the AHP method, the property consistency expresses the degree of coherence in the judgments about the elements in a set.

A positive nxn matrix is consistent if a_ik = a_ija_jk, for all i, j, k, so lmax = n, where lmax is the maximum eigenvalue of the matrix.

As a measure of consistency, Saaty (1980) [39] proposes the consistency ratio CR = CI/RI, where CI = (lmax − n)/(n − 1) is the consistency index, and RI was calculated using DeSchutter’s conjecture based on Saaty’s developments. According to DeSchutter, the relation between RI and n is:

$$RI = \frac{1.98\ast \left(n-2\right)}{n}$$

(4)

where 1.98 is the average consistency index when n goes from n = 3 to n = 15. In this study, as there are six alternatives (n = 6), RI is equal to 1.32.

If CR < 0.1, the preference is consistent and the estimate is accepted.

Consistency in the AHP method was calculated using the Microsoft Excel Professional 2010 program.

2.6. Preference Scoring

In this study the participative process took place without any interaction between the participants [22]. It was therefore necessary to control and monitor the results of the process, and not only to establish the method correctly [20]. 26 students (11 men and 15 women) were involved in this choice. The use of students to test participatory methods is an accepted research method [53]. The students were enrolled in the “Decision Support Systems” course in the master’s program on “Project planning in rural development and sustainable management” (Universidad Politécnica de Madrid) in the spring semester of 2016. 26.92% of the students were environmental or forest engineering graduates and 72.08% business administration or economics graduates. All the students were familiar with the methods applied in this study and with stakeholder participation in forest management, as recommended [19].

The choice problem and the indicators were initially presented to the students in the classroom. The students were shown specific information on each alternative, such as the indicator value, indicator maps and photographs of the forest sites, through the SILVANET computer application [43,45]. They were also provided with a handout summarizing this information.

Next, the students started the participatory process based on Fishburn’s developments. All the pairs of alternatives were compared in the same order as they appear in the SILVANET program. With the previous information available, the students were individually asked to complete the questionnaire shown in

Table 4. UTB method questionnaire.

Alternative 1	Alternative 2	Which Alternative Do You Consider More Sustainable?
		Alternative 1	Alternative 2	Equal
B	A
B	E
C	A
B	C
E	A
E	F
C	D
F	D
D	B
C	E
F	B
A	F
D	A
F	C
D	E

. The photographs of the alternatives can be seen in Figure 3.

The students then addressed the same decision case using AHP. Each student was given a form with the corresponding pairwise comparison tables: one table to compare the three indicators, and three 6 × 6 matrices to compare the alternatives for each indicator. The participants individually assigned their preference values using the AHP’s standard 1 through 9 scale.

The time required to complete the process was three hours in total.

The participants’ preference scores were returned to the researchers. The scores corresponding to the UTB method were input in the SILVANET program [45] to obtain the preference order of the alternatives and characterize each participant’s preference system. The scores corresponding to the AHP method were analyzed with Microsoft Excel Professional 2010 program to obtain the preference order of the alternatives for each student and the consistency of the preference.

After obtaining the results, the students were re-assembled for two purposes:

• In the first meeting, students with an inconsistency of over 0.1 in the AHP method revised their preference matrices to reduce this inconsistency.
• In the second meeting, they were presented with the results and asked their opinion of them and of the methods used.

2.7. Quantifying the Differences in Results between Both Methods

After analyzing each participant’s preferences with each tool, the results were compared. The global results between the two methods were first verified. For each of the six orders of preference, the coincidences in percentage alternatives were calculated between the two methods, along with the distribution of the alternatives according to the method and order of preference.

The analysis was then done by calculating the Spearman’s rank correlation coefficient for each individual.

$$r_i=1-\frac{6{{\displaystyle \sum }}_{j=1}^M{\left(u_{ij}-v_{ij}\right)}^2}{M\left(M^2-1\right)}$$

(5)

where u_ij and v_ij are the ranks of alternative j for participant i, M is the number of alternatives, and r_i takes values from −1 to 1.

For a more detailed analysis, the number of coincidences in the place of any of the alternatives in both methods was calculated for each individual. An analysis of variance was also done to verify whether gender or education had influenced the results.

The Statgraphics Centurion XVII computer program was applied to obtain the statistical results of the study.

3. Results

The size of the final sample was 22 participants, nine men (40.91%) and 13 women (59.09%), as four of the participants did not use the AHP correctly. In terms of academic qualifications, six participants had qualifications in the environmental field (27.27%) and 16 in other fields (72.73%).

With regard to the global results of the study,

Table 5. Relation of alternatives and orders of preference for the UTB and analytic hierarchy process (AHP) methods.

	A		B		C		D		E		F
Order	UTBM	AHP	UTBM	AHP	UTBM	AHP	UTBM	AHP	UTBM	AHP	UTBM	AHP
1	4	8	5	4	6	4	6	1	1	2	0	3
2	8	4	5	9	3	1	3	4	1	2	2	2
3	2	4	5	2	1	7	3	5	4	0	7	4
4	1	2	3	1	6	4	6	8	2	3	4	4
5	6	1	3	3	2	4	4	3	4	8	3	3
6	1	3	1	3	4	2	0	1	10	7	6	6

shows the distribution of the alternatives according to the order of preference for the two methods and for all the participants. The most preferred alternatives are highlighted in bold in Table 5.

In the case of the UTB method, the alternative chosen as the most sustainable was C together with D, followed by B and A in second place and F in third place. C once again appears in fourth place together with D. A appears again in fifth place, while the options considered least sustainable were E followed by F.

In the case of the AHP method, the alternative chosen as most sustainable was A, the second most sustainable was B, the third was C, fourth D, fifth E and the alternative considered by most users as the least sustainable was once again E, followed by F (chosen by 13 people).

In the analysis of the percentage of global coincidences (see

Table 6. Global coincidences for each alternative according to the order of preference.

Alternative Order	A	B	C	D	E	F	Percent. of Coincidences
1	4	4	4	1	1	0	63.6%
2	4	5	1	3	1	2	72.7%
3	2	2	1	3	0	4	54.5%
4	1	1	4	6	2	4	81.8%
5	1	3	2	3	4	3	72.7%
6	1	1	2	0	7	6	77%

), according to the order of preference, 63.6% of the 22 individuals coincide in their preferred alternative (order 1); 72.7% in the alternative in the second order of preference; and 54.5%, 81.8%, 72.7% and 77% in the third, fourth, fifth and sixth order respectively.

A comparison of the results of the two methods for each individual shows that the alternative considered most sustainable coincides in only eight of the 22 participants, 36% of the total (three chose A, three B and one C and D). Five of these eight participants were women and three men. Figure 4a shows the value of the Spearman’s rank correlation coefficient for each individual, and the histogram of the Spearman’s rank correlation coefficient variable (Figure 4b).

The Spearman’s rank correlation coefficients vary between −0.88 and 1, with a mean value of 0.08 and a standard deviation of 0.56. As can be seen in Figure 4, there are six cases in which the correlation coefficient is greater than 0.6 and two cases in which the coefficient is below −0.6, so the preferred alternative in one method is the least preferred in the other. According to the results of the Spearman’s rank correlation coefficient, only five participants showed a statistically significant correlation between the UTB and AHP methods, (p-value less than 0.05 for the null hypothesis, H₀: r = 0). The rankings of the remaining 17 participants were uncorrelated.

The analysis of variance revealed that gender had no influence on the value of the Spearman’s rank correlation coefficient for a p-value of 0.702; and education had no effect for a p-value of 0.86. Residual hypotheses were also tested.

For the 22 individuals, the number of matching alternatives for each individual in any order between the two methods is shown in Figure 5.

With regard to the parameters for each method, the values for the value function of the UTB method were easy to obtain from the ranking of the alternatives. There were six individuals for whom the utility function attained the highest value for alternative C. u(C) = 5, and was the alternative most frequently assessed as the most sustainable for this method. In terms of the alternatives considered the least sustainable, there were three individuals for whom u(F) = 1 and u(E) = 0 at the same time, and three for whom u(E) = 1 and u(F) = 0.

With regard to the transitivity of the preference, in the case of Fishburn the preference of 70% of the participants complied with the property of transitivity or semitransitivity [42]. The chi-square test of homogeneity was used to analyze whether there was any relation between the presence of transitivity and gender and education, and in both cases the variables were independent for a p-value of 0.0002.

In the case of the AHP method, 73% of the participants initially presented inconsistency in some of the preference matrices. After revising the preference matrices, 72% of the participants’ CR < 0.1.

Comparing the results in the evaluation process, in the UTB method, the participants considered which forest management alternative was most likely to maintain biodiversity and productivity in the future, so which one was most sustainable; while in the AHP method they explained that it was easier to focus only on the criteria rather than on the relation between sustainability and criteria. For 70% of the participants, both methods adequately reflected their preferences, although the rankings did not always match. They considered AHP to be time-consuming, and became distracted, which was the main reason they failed to keep track of the evaluations of some alternatives compared to others. Finally, they were asked which method they preferred. 15% preferred the UTB method, 25% AHP, and both methods were preferred equally by 60%.

4. Discussion

This article compares the AHP and utility based methods for measuring the sustainability of forest management at the stand level. One significant advantage of applying these tools is that they can gather both objective and qualitative information, expert knowledge, and subjective preferences [19,42]. The results of the application of the two tools allow us to conclude that there is a clear difference between them. The percentage of coincidence is low in both the alternatives indicated as the most sustainable and in the comparison of the order of all the alternatives.

The process began with a sample size of 26 individuals, but due to errors by four participants in the AHP method and their unavailability to review these errors, the final sample size was 22 individuals. For the case of the AHP method, this reduction in sample size does not affect the reliability of the results. Several studies can be found in the literature that apply the AHP method with a fewer than 25 participants [54,55,56]. In the case of the UTB method, in a previous work [44] the sample size was fifty individuals, and the sampling error was 13%. In that case the participants were visitors to the Sierra of Guadarrama National Park, with significant differences in their backgrounds. In our study the participants’ profile is quite similar, although there is more variability in the preferences compared to AHP results. Further research is needed to identify the appropriate sample size for this method.

Knowledge of the methods and objectives of the participation process makes it easier to obtain convergent results [57]. In the case of the UTB method, the SILVANET program provides comprehensive information on SFM, indicators and alternatives, and the use of the program is self-explanatory. The users do not require any previous knowledge of SFM [44]. However, when working with AHP the user will need to be familiar with the decision scale and the properties of the method, and with the objective, criteria and alternatives to be assessed. In the study case, all the participants were familiar with both the methods and SFM, and were also supported by the researchers during the experiment.

In regard to the global results when applying the UTB method, the preference of the alternatives does not follow any specific pattern. The preferred alternative (order 1) is almost uniformly distributed between alternatives A, B, C, and D. Second-order preferences are A and B, and the least preferred are clearly E and F. However, there is no clearly preferred alternative in the third, fourth and fifth order. The users were able to distinguish two sets of alternatives, one formed by the alternatives in which the value of the indicators is highest (A, B, C) and another with the alternatives whose indicators had the lowest value in terms of the ideal point, IP (E and F). In the case of AHP, there is convergence and a pattern in the combined opinion. The order of preference of the alternatives is the same as the order of proximity of the alternatives to the ideal point. That is, alternative A was the most preferred, and is the closest to the ideal point, followed by B, C, D; alternatives E and F are in fifth and sixth place, and are farthest from the ideal point.

AHP therefore makes it possible to distinguish the differences between the alternatives more accurately. When comparing alternatives to indicators, the participants’ comments suggest that the AHP method is more structured and the concept of indicators is easier to understand than sustainability, indicating that the criterion prevails over the objective. The characteristics of each method therefore influence the order of the alternatives.

This conclusion may help in the choice of method, the number and differences among alternatives in a participatory process in the Guadarrama National Park.

The results of the Spearman’s rank correlation showed that the rankings of 17 of the participants were uncorrelated, demonstrating again that the method influenced the decision outcome [21,58].

Since preferences are expressed in a subjective manner in both methods, it is reasonable for there to be some kind of inconsistency [59]. The results concurred particularly in the case of the UTB method, since the participants could consider sustainability as a whole and make a more context-based comparison. In previous public participatory processes with the UTB method, the participants’ results also concurred [44]. 70% of the participants fulfilled the transitivity requirements for this method. However, inconsistency problems were found in the AHP method. Due to the high number of comparisons—96 per individual—the process was excessively long and produced conflicting judgments when weighting indicators and alternatives [19]. Saaty (2006) points out that most AHP respondents are never consistent [60]. In the case of inconsistency, the AHP recommends inviting the respondents to rethink their opinion of the elements evaluated, as in this study. When choosing between these methods, the decision-makers must consider the problems of inconsistency, the time required in the SFM process, the participants’ previous knowledge of the AHP method and the lack of precision of the UTB method to distinguish the characteristics of each alternative in detail.

5. Conclusions

There is currently no standard public participation method in SFM. Our paper focuses on assessing and comparing individual preferences using two methods, AHP and UTB, to evaluate and order alternatives for forest management at the stand level according to sustainability. AHP is a very useful method when the number of criteria and alternatives is not very high. The structuring of objectives facilitates the evaluation. The main conclusion of this study is that the AHP method is more suitable for use with participants who are experts or closely involved with the objective to be assessed, which avoids problems of inconsistency and distinguishes the differences between the alternatives more accurately. The UTB method is very useful in online public participation processes that are open to all types of stakeholders, but is less precise to identify differences among alternatives compared to AHP.