Next Article in Journal
Four Decades of Land-Cover Change on the Kenai Peninsula, Alaska: Detecting Disturbance-Influenced Vegetation Shifts Using Landsat Legacy Data
Previous Article in Journal
Integrating Ecosystem Services into Land-Use Modeling to Assess the Effects of Future Land-Use Strategies in Northern Ghana
Article

The Sustainable Management of Land and Fisheries Resources Using Multicriteria Techniques: A Meta-Analysis

1
School of Forest Engineering and Natural Resources, Universidad Politécnica de Madrid, 28040 Madrid, Spain
2
School of Agronomic Engineering, Food and Bio-Systems, Universidad Politécnica de Madrid, 28040 Madrid, Spain
3
Basque Centre for Climate Change, 48940 Leioa, Spain
*
Author to whom correspondence should be addressed.
Land 2020, 9(10), 380; https://doi.org/10.3390/land9100380
Received: 6 September 2020 / Revised: 24 September 2020 / Accepted: 6 October 2020 / Published: 8 October 2020

Abstract

In recent years modern societies have attached a multifunctional requirement to the use of renewable resources, making their optimal sustainable management more complex. In the last decades, in many cases, this complexity is addressed by formulating management models with the help of the concepts and methods belonging to the well-known multicriteria decision-making (MCDM) paradigm. The purpose of this paper was to undertake a hermeneutic meta-analysis of the literature provided in primary journals on issues related to the management of these resources with the help of the MCDM paradigm. In this way, the paper aimed to obtain new, basic insights with considerations that might improve the efficiency of future research in the field studied. The meta-analysis was implemented by formulating and testing a battery of hypotheses of how the MCDM methods have been used in the past for the formulation of management models for the type of resource analyzed.
Keywords: agriculture; forestry; meta-analysis; multicriteria decision-making; sustainability agriculture; forestry; meta-analysis; multicriteria decision-making; sustainability

1. Introduction

Land and fisheries resources belong to the renewable resources group. The stock of this type of resource changes according to a fairly predictable biological growth rate. The sustainable management of the resource requires its rate of use to be lower than its corresponding growth rate. Among these resources, the most significant ones are found in the areas of agriculture, fisheries, and forestry. In the past few years, a multifunctional requirement has been associated with them, making their management a complex decision-making problem [1,2,3]. This complexity increases considerably within the current context of sustainability [4]. Obviously, in this new, realistic scenario, the balance between the growth and use rates of the resource is necessary, but it is not a sufficient condition for managing it rationally, for which several functions are typically required by modern societies in the management of this type of resource. Among them, and without being exhaustive, the following ones can be cited: the incorporation into the management process of a diverse ecosystem service other than that of provision [5]; the assimilation of the multifunctionality idea at a landscape level [6]; the integration into the decision-making process of the preferences of several stakeholders [2]; and the presence in the analysis of new information technology systems that optimize the use of provisioning services [7].
In short, the inclusion of the multifunctional requirement implies, in many instances, the consideration in the management process of several criteria that are usually in conflict. For this reason, in general, the use of analytical approaches based on the classic optimization of a single criterion function is somewhat useless for dealing with this type of problem [8]. Due to this type of insufficiency during the past decades, it was proposed that the multifunctional issue underlying the management of these resources could be advantageously dealt with by resorting to concepts and tools belonging to the field of multicriteria decision-making (MCDM). A pioneer work in this direction can be seen in [9].
In accordance with the above orientation and from a historical perspective, the following pioneer works should be cited. For example, the use of multiobjective programming was proposed in the fisheries field [10]; the use of goal programming models for dealing with some forest management problems was depicted in [11]; and other authors have exemplified the use of several multicriteria methods for solving continuous problems in agricultural planning, emphasizing that multiple objectives are the rule rather than the exception in agricultural decision-making [9,12].
Besides the above, other critical reviews following an MCDM orientation in the management of the three biological resources mentioned should be highlighted. Regarding agriculture, some studies can be cited, such as [5,13]; the former critically reviews applications of multicriteria methods to discrete problems, while the latter paper deals with applying multicriteria methods to problems of a continuous nature. In the forestry field, many recent reviews focus on general forest management issues [3,14,15,16,17]. This literature is not so extensive in the fisheries field, but several contributions have stood out, such as [8,18].
Another point worth noting is the successful use of the MCDM methods for solving specific problems related to the sustainable management of the resources studied [19,20,21]. This orientation is especially useful when the purpose of the analysis is to quantify and explain the sustainability of different natural systems by defining and measuring a significant number of criteria and indicators [22,23]. Some relevant and recent examples in this direction include the works in [24,25] for agriculture, those in [26,27] for fisheries, and finally, those in [28,29,30] for forestry. Although there are different methodologies grouped under the umbrella of MCDM techniques, which appear in later sections, one of these techniques should be especially mentioned due to its preponderance in multiple areas, namely the analytic hierarchy process (AHP), which has been the technique most widely used in other fields [20,31,32]. There are several reasons explaining this fact [33], and undoubtedly one of them is the flexibility offered by this methodology in order to hybridize with other tools, whether or not they belong to the toolkit of the MCDM techniques. This point justifies the fact that some of the hypotheses to be formulated are limited to the application of this particular method.
The three biological resources included in this study are mostly analyzed jointly in many bioeconomy references [34,35]. On the other hand, the EU biobased economy strategy considers them to belong to the same kind of economic activity [36]. In addition to this, in the last two decades or so, extensive literature has proposed their optimal management by using different MCDM approaches. This study’s main objective, therefore, is to make a critical assessment of the results provided by a meta-analysis over the last ten years involving the three resources analyzed, and it aims to demonstrate how these methodologies have been used, allowing the reader to gain useful insights into their correct applications in the future. Thus, we have built up an extensive database of works published in primary journals during the past few years (2008–2017) on the management of the three types of biological resources considered with the help of MCDM methods. In a second step, following a hermeneutic meta-analysis orientation, the literature reviewed will be considered as a kind of raw material for testing a set of hypotheses on how MCDM methods were used in the past in the field analyzed. In this way, it will be shown how the corroboration or refutation of the hypothesis formulated might provide valuable new knowledge, the consideration of which could improve several theoretical as well as practical aspects of the future research to be undertaken in the field studied.

2. Materials and Methods

For the bibliographical search, two databases belonging to two different sources were consulted: Web of Science (WOS) and Scopus, as shown in Figure 1. As is well known, both databases are the most important bibliographic ones, although they both have strengths and weaknesses, and there are numerous studies comparing one to the other [37,38]. Here, in order to obtain a reasonably sized collection of articles endorsed by more than one bibliographic source, it was deemed appropriate to select those papers that appear simultaneously in both databases. These papers were then tagged as a result of the intersection in the searches made in WOS and Scopus.
As can be seen, the searches were used separately in both databases, following the procedure explained in Figure 1. The exact searches are detailed in Appendix A. Contrary to what other studies indicated [39], the number of articles selected was somewhat higher in WOS than in Scopus. In the first phase, after eliminating the articles duplicated, those that were repeated in both databases (841 articles in all) were chosen. This coincidence in papers appearing in both databases is much greater than that shown in other reviews, in which it only reaches 4% [40]. Next, and after an exhaustive reading of those articles, 213 were discarded for the reasons given in Figure 1, leaving the 628 articles that supported the conclusions extracted from this work. Some papers (i.e., 213) were not considered in the analysis for several reasons. One is that 119 of them did not use any multicriteria methods. Moreover, we assumed that 51 papers did not belong to the three fields analyzed in this work (i.e., “not belonging to areas” in Figure 1). Reviews that were detected in the searches were also not included in the analysis. Finally, 20 papers could not be considered because we did not have access to these files (i.e., “not founded” in Figure 1).
Regarding the methods, it should be noted that the MCDM techniques’ characteristics are not described in this review. There are two reasons for this decision: On one hand, we wish to expedite its reading, and on the other, there are already many books and scientific papers in which each of these techniques is described in detail. For information purposes, we highlight three of them, i.e., [41,42,43], due to their higher impact. Besides, these books are exclusively dedicated to our study areas. Other articles linked to the application of a specific MCDM technique have also been published in one of the three areas considered; for instance, these papers include the specific works on the employment of the data envelopment analysis (DEA) in agriculture [44], in fisheries [45], and in forestry [46]. Turning to techniques such as goal programming (GP), one study in which specific useful advice on the application of this technique in forest management problems given is in [47]. In relation to agricultural problems, some studies described multiattribute utility theory (MAUT) methods (and compared them with AHP) (e.g., [24]), while others maintained the comparison with AHP but starting from the TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution) method (e.g., [48]). The same techniques were applied in a fishing model [49]. For the two MCDM methods used in continuous problems (i.e., multiobjective programming and compromise programming), we recommend seminal books [50,51,52,53], as well as other works previously referenced [41,43]. In addition, with regard to MCDM techniques applied to discrete problems, there are many books, chapters, and papers focused on AHP (analytic hierarchy process) and ANP (analytic network process) applications for several problems. We recommend books oriented towards agricultural problems [54] or towards forestry/environmental issues [55].
In order to reach any useful conclusions after considering this comprehensive set of articles, it was deemed appropriate to propose diverse hypotheses that have been examined when looking for possible relationships between the three areas and the MCDM techniques found in the literature. Although initially a more significant number was contemplated, 16 hypotheses are finally proposed and are summarized in the Results Section Table 5.
The first of these refer to the application of the techniques in the three fields. Given that we have not seen any comparative studies supporting the opposite, we assume that each MCDM methodology is applied in the same way in each area, that is, we discard the idea that one specific technique is predominantly applied only in one of the areas. Likewise, and following the same reasoning, we figure that the number of criteria used in each paper analyzed is the same, regardless of the area to which that study can be ascribed. However, and based on prior research [20], the number of criteria employed in continuous problems is assumed to be smaller than those used in discrete-type ones. We also assume that the use of techniques usually applied to approach continuous-type problems (e.g., multiobjective programming, compromise programming, goal programming, etc.) is lesser than that of the set of techniques applied only to discrete-type ones [15].
Following on with other aspects of MCDM techniques, our next hypothesis is that there would not be any relationship between the use of a particular technique and the number of countries involved in the analysis. Another hypothesis worth testing concerns the simultaneous use of several MDCM techniques for the same case study. We assume that according to reviews made in other fields, these hybrid approaches are widely used [31], and that they have increased over time [56].
Another hypothesis was also presented in which the different criteria considered were normalized, independently of the field and the multicriteria technique used [57]. In addition, an initial question that is posed when analyzing a study using these techniques is to justify why a certain one was chosen. Our hypothesis here is that this justification is not usually given, regardless what type of method was used or in which area it was applied to. This idea is upheld as true because there is no guide for selecting the appropriate method for each type of problem [6], or, fundamentally, for familiarity and affinity reasons [25].
To go on with specific techniques, given their extensive use in diverse spheres and that the MCDM technique is the one most commonly used according to some reviews [58,59,60], it is considered appropriate to propose various hypotheses associated with the use of AHP (analytic hierarchy process). The first of these is that, on many occasions, this technique is used together with weighted MCDMs [33]. The second hypothesis has to do with a possible positive relationship between the employment of AHP and its application to spatial problems. This hypothesis is based on the considerable number of studies that combine GIS (geographic information system) and AHP [61,62] along with the inclusion of AHP and weighted MCDMs in different GIS packages [63]. The last hypothesis with AHP is that it is mostly used to obtain weights of a group of stakeholders and/or experts and to apply the weights to solve the problem [64]. It should be pointed out that a notable percentage of works that use multicriteria techniques in some of these fields incorporate the stakeholders into the analysis [25].
One aspect of great interest related to MCDM techniques is their use with other methodologies jointly, such as group decision-making (GDM) techniques. Two hypotheses are formulated on these lines. The first one that appears to be plausible, in the spirit of other works [65], is that the hybridization of MCDM and GDM increases throughout time. The second one is that, given the nature of specific forestry problems, it would seem that hybridization could be more frequent in the forestry field [15].
The last hypothesis formulated tackles aspects related to the employment of these methods with certain topics and techniques. In recent years, specific terms have appeared in the literature in abundance (e.g., “ecosystem services”, “climate change”, “bioeconomy”, “sustainability”, etc.) and are frequently featured in many publications that apply MCDM techniques in the three areas. Here, a hypothesis is proposed whereby MCDM methodology applications in these areas are increasing and are only relevant in the latter years of the time series considered [66]. Finally, is tested a predominance in the use of the GIS hybridized with an MCDM technique in the forestry area.
In order to test the proposed hypotheses, different statistical tests and regression analyses were carried out. The statistical tests included Pearson’s chi-squared test, Fisher’s exact test, ANOVA, Welch’s two-sample t-test, and one-sample proportions [67,68]. Pearson’s chi-squared tests were used when both the dependent and independent variables were categorical and when the independent variable had at least two levels. Fisher’s exact tests were used in these same cases, but only when one or more cells of the two-way frequency table had an expected frequency of five or less. One-way ANOVA tests were applied when the dependent variable was an interval and normal, and the independent variable was categorical with more than two levels. Welch’s two-sample t-tests were used when the dependent variable was an interval and normal, and the independent variable had exactly two levels. The one-sample proportions test was used when the independent variable was of a certain value, and the dependent variable was categorical with two levels. The regression analyses included simple linear and logistic regressions. Simple linear regressions were used when both the independent and dependent variables were intervals, while logistic regressions were used when the dependent variable was binary, and the independent variable was an interval.

3. Results

Table 1 shows some general data of the review made. First, the prevalence of works in the fields of agriculture and forestry over those of fisheries can be seen, as well as the large number of countries in which the different case studies analyzed were located. Figure 2 shows the frequency of studies per country considered in the analysis.
The sample of studies covers an ample spectrum of countries across the world (Figure 2). Iran (n = 58 studies), United States (n = 48), Spain (n = 48), China (n = 46), Italy (n = 28), India (n = 25), Finland (n = 23), and Brazil (n = 20) were the countries with the highest number of studies. However, when considering the population size of those countries, a higher concentration of studies was seemingly implemented in Europe. Finland (n = 4.4 studies per million inhabitants), Slovenia (n = 2.0), Austria (n = 1.4), Greece (n = 1.4), Oman (n = 1.2), Cyprus (n = 1.2), Spain (n = 1.1), and Portugal (n = 1.0) were the countries with the highest number of studies per country.
In addition, Figure 3 gives the frequency with which each technique is represented in the articles analyzed. It should be noted that those presenting fewer than ten articles were integrated into the heading “Other MCDM methods”. On the other hand, going deeper into some aspects related to the application of these MCDM techniques, Table 2 shows the number of articles that include, for example, a series of explanations or characteristics that are relative to how the weights assigned to each criterion were calculated.
In addition, in the articles analyzed, the MCDM techniques are very frequently hybridized with others, both those belonging to the operational research and those relative to other disciplines (GIS, statistical methods, etc.). In Table 3 the main results are summarized. Finally, Table 4 gives the number of articles analyzed that address some hot topical aspects. Although the list of topics is fairly heterogeneous in terms of their significance throughout the time considered in this work, it should be noted that the topic “sustainability” appears in around 25% of the papers.
In the next paragraphs, we describe the results obtained after testing the research hypotheses (Table 5). More details on the results of the hypotheses are found in the Supplementary Materials.
H1. “The application of the multicriteria decision-making (MCDM) techniques does not significantly differ in the three fields (agriculture, forestry, and fisheries), given that we have not seen any comparative studies that support the opposite.” This hypothesis was rejected since the application of MCDM techniques significantly differed in the three fields. However, when the studies with DEA were excluded from the analysis, this hypothesis could be accepted since the difference was not significant. First, we tested whether the number of articles differed significantly across the three fields (agriculture, forestry, and fisheries). Since the variable field had three categories, we used Pearson’s chi-squared test. Then, we evaluated whether the application of MCDM techniques significantly differed in the three fields. Since one or more cells of the two-way frequency table had an expected frequency of five or less, we used Fisher’s exact test.
H2. “The number of criteria used in MCDM problems is similar in the three fields (agriculture, forestry, and fisheries).” This hypothesis was accepted because the null hypothesis that they were equal was rejected. Since the variable fields (independent variable) had more than two levels and the criteria used in MCDM problems (dependent variable) was an interval, we used the one-way ANOVA test.
H3. “The number of criteria is lower in continuous problems than in discrete ones.” This hypothesis was accepted since the number of criteria in continuous problems was effectively and significantly lower than in discrete ones. Since the variable continuous/discrete was categorical with two levels and the number of criteria was an interval, we used the Welch two-sample t-test. We used the one-way ANOVA test to include a third level (mixed) apart from continuous and discrete.
H4. “There seems to be a lesser use of MCDM techniques applied to continuous problems than others that can only be applied to discrete ones.” This was accepted because there was a significantly lesser use of techniques that apply to continuous than to discrete problems. Since the variable continuous/discrete was categorical with two levels and the number of MCDM techniques was an interval, we used the Welch two-sample t-test.
H5. “There is no relationship between the use of a particular MCDM technique and the fact that the case studies are from one or several countries.” This hypothesis can be accepted since the relationship is not significant. However, a significant difference between the use of techniques and the number of countries was found. This could be affected by the fact that there are many techniques or groups of techniques, and that in some of them the proportion of the number of countries is quite different. Since the dependent variable (the use of a particular MCDM technique) was categorical with more than two levels, a multinomial logistic regression was used.
H6. “The use of several MCDM techniques has increased over time.” Although this hypothesis was accepted, it is important to take into account that the number of studies using only one MCDM technique increased. This increase was also significant in the fields of forestry and agriculture. The variable indicating the number of studies using MCDM techniques was converted into a categorical variable with two levels (one technique, various techniques). In order to see the difference throughout time, one mean value was obtained for the first year range (2008–2012) and another one for the latter year range (2013–2017). This process was conducted in each category, and in order to evaluate the difference between the two values of each category, the Welch two-sample t-test was used.
H7. “(A) The criteria are usually normalized, independently of the field to be used.” This is accepted due to the normalization process being carried out regardless of the field. “(B) The criteria are usually normalized, regardless of the multicriteria technique to be used.” This hypothesis was rejected since the normalization was done depending on the techniques employed. The significance of the test could be influenced by the large number of techniques or group of techniques. First, we assessed whether the studies, normalized or not, differed significantly across the three fields. Since the variable field had three categories, we used Pearson’s chi-squared test. Then, we evaluated whether the studies, normalized or not, significantly differed in the use of the techniques. In order to evaluate the difference between the two values, the Welch two-sample t-test was used.
H8. “(A) The justification of why the method is chosen is not usually given, whichever method is used.” This hypothesis was rejected. Although 7% of studies accounted for the method used, significant differences were found in those studies that were grouped by techniques or groups of techniques. “(B) The reason for why the method is chosen is not usually given, whichever area it is applied in.” No significant differences were found among the studies grouped by countries. Since one or more cells of the two-way frequency table had an expected frequency of five or less, we used Fisher’s exact test.
H9. “AHP and weighted MCDM always go together.” This hypothesis was rejected since there was no significant relationship between the use of AHP and weighted MCDM. Both variables (use of AHP and weighted MCDM) were binomial. A binomial logistic regression was used to measure the effect of using AHP on the probability of using weighted MCDM. In addition, the chi-squared test was used to evaluate any significant difference between the two variables.
H10. “Given the inclusion of methods such as AHP and weighted MCDM in different GIS packages, it seems logical to point out that there is a positive relationship between the use of AHP and its application to spatial problems.” This hypothesis was accepted due to those studies that used AHP or weighted MCDM being significantly more likely to use GIS than the rest of the techniques. The use of GIS packages was also a binomial variable. A binomial logistic regression was used to measure the effect of using AHP and weighted MCDM on the probability of using GIS packages. In addition, the chi-squared test was used to evaluate any significant difference between the variables.
H11. “AHP is used mostly to obtain weights from a set of stakeholders and/or experts and apply these weights to solving the problem in question.” This hypothesis was accepted because most of the times, the AHP was used for obtaining weights. The variable weights obtained from stakeholders and/or experts were also binomial. A binomial logistic regression was used to measure the effect of using AHP on the probability of using weights obtained from stakeholders and/or experts, and the chi-squared test was used to evaluate any significant differences between the variables.
H12. “The hybridization of MCDM and GDM is increasing over time.” This hypothesis was accepted with some nuances. The number of studies with GDM and MCDM has significantly increased, but the proportion of studies with GDM and MCDM has decreased over time, although this declining is not significant. The variable indicating the hybridization of MCDM and GDM techniques was converted into a categorical variable with two levels (with and without the hybridization of MCDM and GDM techniques). In order to see the difference throughout time, one mean value was obtained for the first years (2008–2012) and another one for the latter years (2013–2017). In order to evaluate the difference between the two values of each category, the Welch two-sample t-test was used.
H13. “Given the nature of forestry problems, the hybridization of GDM and MCDM techniques would seem to be more frequent in forestry.” This hypothesis was accepted because the hybridization of those techniques was significantly more frequent in forestry than in the other fields. We assessed whether the hybridization of those techniques differed significantly in the three fields (agriculture, forestry, and fisheries). Since the variable field had three categories, we used Pearson’s chi-squared test.
H14. “The concepts of ecosystem services, climate change, and sustainability are recent and have only become prominent in recent years.” This hypothesis can be accepted for the cases of ecosystem services and climate change, for which both the number and proportion of papers devoted to these topics have increased significantly. As for sustainability, despite their total number having increased significantly, the proportion has decreased. The variables ecosystem services, climate change, and sustainability were converted into categorical variables with two levels (i.e., with and without ecosystem services, climate change, and sustainability). In order to see the difference throughout time, one mean value for each variable was obtained for the first year range (2008–2012) and another one for the latter year range (2013–2017); to evaluate the difference between the two values of each category, the Welch two-sample t-test was used.
H15. “The topics included as being relevant, such as climate change, multifunctionality, multiple use, sustainability, bioeconomy, corporate social responsibility, certification, protected areas, and life cycle, are only relevant in the last years of the period.” In general, we reject this hypothesis. Only for climate change can we accept that the topic was relevant in the last years of the period. The variables climate change, multifunctionality, multiple use, sustainability, bioeconomy, corporate social responsibility, certification, protected areas, and life cycle were converted into categorical variables with two levels (with and without the above variables, i.e., climate change to life cycle). In order to see the difference throughout time, one mean value for each variable was obtained for the first year range (2008–2012), and another one for the last year range (2013–2017). In order to evaluate the difference between the two values of each category, the Welch two-sample t-test was used.
H16. “There is a higher probability of using the GIS hybridized with some MCDM techniques in the forestry area than in other fields.” This hypothesis is accepted since those studies within the forestry field are more likely (statistically significant) to use GIS techniques than within the fisheries and agriculture fields. First, we assessed whether the use of GIS hybridized differed significantly across the three fields (agriculture, forestry, and fisheries). Since both variables were categorical, we used Pearson’s chi-squared test. Then, a binomial logistic regression was performed to measure the effect of using the forestry field on the probability of using GIS hybridized.

4. Discussion

This is an original study since until now, no other work has analyzed the use of MCDM approaches applied to problems associated with the sustainable management of agriculture, forestry, and fisheries resources. In addition, the remarkable increment of the papers published in these fields using MCDM techniques is a reliable guide to the importance of this type of study. For example, on forestry issues, the number of MCDM papers published in the period analyzed, compared to the previous decade (1998–2007) [15], has increased by more than 62%. In short, this trend, which is also observed in the other fields [20], deserves a study of these characteristics.
The results show an extensive use of MCDM techniques in the three fields considered with a global dissemination. There are case studies in nearly 90 countries, excluding those in which more than three countries are involved. It is also interesting to note a significant increase in the number of works published following this trend. These results are on the lines of different studies, especially concerning applications of AHP, which is the method most employed in the articles reviewed [59,69]. It is also worth noting that over half of the articles selected have been published in journals that do not belong to the three areas chosen.
When the MCDM techniques employed in the articles are analyzed, excluding the cases of fuzzy MCDM and “other techniques” in Table 1, the group of techniques used in discrete problems is predominant (83.3%) compared to those based on distance functions (11.6%). This situation has been assiduously observed in other reviews [2,15,20]. As for the MCDM technique most used (AHP), the results obtained coincide with other recent reviews in the field of sustainability [20,21]. Some authors have even conjectured about its simplicity of use as being a fundamental reason explaining the widespread use of AHP [58].
Another aspect that should be highlighted is the elicitation of preferential weights. Although most works in Table 2 (70%) assign weights to each criterion considered, those incorporating the preferential weights of one or some stakeholders (14%) are much fewer. These figures differ from reviews like that of [70], in which it is stated that nearly all the papers analyzed (86) attach weights to each criterion, and over 40% is done interacting with diverse stakeholders.
A prior step when assigning a weight to each criterion, objective, indicator, etc., would be to normalize them so that all of them can be managed in dimensionless units [8,71]. Some authors pointed to the importance of carrying out this phase correctly [72]. The results presented in this paper show that half of the works analyzed explain how they performed the normalization process, i.e., fewer than the results provided by other reviews [20].
It should be emphasized that it has been demonstrated that the growing number of applications for solving a particular problem hybridize either several MCDM models or one MCDM technique with another GDM one. This type of strategy has been increasingly used and recommended by other authors for its strong potential and for extracting the best out of each technique [65,73]. It is also noted that to solve complex problems, other techniques are very often linked to MCDM methodologies, either those belonging to management science or other disciplines such as statistics. On the other hand, according to what is explicitly affirmed by the authors of the articles, the integration of MCDM techniques into decision supports systems (DSS) is still modest (13% of the papers), despite the potential offered by this combination of methodologies [70,74].
Furthermore, 21% of the works have stated that they made a sensitivity analysis, an aspect that is recommended in numerous works [59,75]. This figure is lower than that reported in other works [20,66,76]. Finally, approximately 15% of the studies combine these MCDM techniques with nondeterministic methods (Markov chains, Monte Carlo simulation, stochastic models, fuzzy methods, etc.). This percentage is similar to the one supplied in a study on the inclusion of biodiversity in forest management [77].
With regard to rejecting the hypothesis on the uniform application of multicriteria decision-making (MCDM) techniques in the three areas, the reason would appear to be inclusion or not of the DEA approach in the analysis. The frequent application of this technique in agriculture undoubtedly explains these results, since almost 73% of the 150 articles using this technique are associated with agriculture [78,79].
Of the two hypotheses related to the number of criteria, it was confirmed that their number was similar in the three fields studied. On checking the results, we verified that the fields which are most represented use a fairly similar number of criteria (a mean of 9.8 in agriculture and 10.9 in forestry) in their work. Comparing these figures with other reviews of MCDM techniques in different fields, they are notably higher than those relative to works focusing on problems associated with sustainability (a mean of 4.2 criteria) [20]. However, some authors, in a review of 86 works related to nature conservation, declared that the number of criteria employed was excessive and unbalanced in the different pillars previously established [76]. The second hypothesis that was corroborated refers to the number of criteria being lower in continuous problems than in discrete ones, as this circumstance was also observed in other reviews [20]. Even when the discrete methods involve the determination of weights in some MCDM methods applied to discrete problems, the number of criteria must not be too large [80].
Concerning the hypotheses related to multicriteria techniques, it was confirmed that the methodologies used in continuous type problems are applied more infrequently than those oriented towards discrete ones. This fact was already verified for works published both in forestry [15] and in sustainability [20]. In detailing some of these techniques, three hypotheses were formulated that were related to the technique most used, according to the data of this study, namely the technique of AHP.
The hypothesis that AHP was linked to a weighted MCDM was rejected. Strangely enough, whereas in this study AHP is much more represented than the weighted MCDM (218 vs. 125), in other reviews this number is similar [20], or the weighted MCDM even greatly exceeds the AHP (143 vs. 34) [81]. However, it was proven that it is mainly used for obtaining weights from different stakeholders, as can be deduced from a recent study [40].
In addition to all of the above, there is a direct relationship between the use of this method and the spatial problems required by a GIS, an aspect already anticipated in the literature [63,82]. At present, the integration of GIS and AHP tools contributes to the wide expansion of its use due to the increase in popularity of GIS. Furthermore, a wide variety of techniques exist to assign weights in case of MCDM with GIS software packages. However, several GIS software packages explicitly incorporate very user-friendly (visually and intuitive) spatial multicriteria and multiobjective decision functionalities (e.g., Idrisi, Ilwis, TerrSet), which appear as a pragmatic method. Otherwise, there is a vast number of freely available plugins and scripts for both the most common commercial GIS software (i.e., ArcGIS) and free options (e.g., Grass, QGis, SAGA), which have popularized GIS-based AHP tools.
The hot topics selected show an uneven distribution in their use in MCDM problems (Table 4). Among them, the idea of sustainability is most utilized, especially in papers focusing on agricultural and forestry issues. This result is perfectly coherent with the current idea of using a bundle of criteria and indicators. Under these assumptions, MCDM fits in very well with addressing this “new sustainability” [82], in contraposition to the idea of “old sustainability” that is so greatly applied in forestry issues [30]. The rest of the topics have similar figures, except “bioeconomy” and “life cycle analysis”. Regarding the former, one explanation could be the relative novelty of this term, which has only appeared in recent years [83]. It is not too bold to suggest that if this type of analysis is carried out in the medium term, the keyword “bioeconomy” will appear in more papers.
From a didactic point of view, attention is called to the fact that very often the authors do not present or justify some of the decisions made when proposing a model (only 7%), although some give somewhat higher figures in their reviews. It has already been demonstrated that, regardless of the area, why the method was elected is not usually defined. Thus, some authors talk about 47% as the percentage of articles analyzed that offered a justification for their election method [76].
It is important to note that when the choice of a particular MCDM method is justified, the reasons supporting the choice are theoretical, and they are not based on a comparison of pros and cons with respect to other possible methods [84]. Moreover, in this review exercise, it was found that only 30% of the cases was the solver used indicated, and in very few works was the size of the model explicit (number of variables, criteria, or constraints). What is more usual is to enlarge the explanation with mathematical formulae (in nearly 60% of the cases), although much fewer authors illustrate their model with the help of a flow chart (23).

5. Conclusions

The number of applications reported in the literature on the use of MCDM methods for a satisfactory management of the natural resources considered is impressive and has a clear rising trend. This line of work started around forty years ago with not too many applications, but in the last decade, the interest in this type of orientation has increased dramatically. From the hermeneutic meta-analysis undertaken, the following conclusions deserve to be highlighted:
  • Without considering the use of DEA, which is extensively deployed in the specific area of agriculture, the application of MCDM methods is somewhat similar in the three areas studied.
  • The number of criteria included in the different applications is also similar in the three areas. However, it is important to note that the number of criteria considered is lower for continuous problems than for discrete ones. This could be explained by the fact that the computational complexity associated with the resolution of the continuous models considerably increases with the number of criteria considered.
  • The criteria involved in decision-making modeling have to be normalized in all cases. This requirement is demanded independently of the field as well as of the MCDM method used.
  • The choice of the particular MCDM method used is made, in most cases, in a somewhat arbitrary way. This type of mechanistic practice does not seem advisable. Thus, in general, the main features of the problem situation, to some extent, suggest the most suitable MCDM method to be used.
  • The combined use of several MCDM methods for dealing with a specific problem was successfully applied. In this sense, the use of AHP for deriving the preferential weights and subsequently attaching them to a multicriteria optimization model is paradigmatic. Despite the wide use of the above case, the hybridization of MCDM methods is of current interest and seems to have many future developments.
  • Modern democratic societies demand a participatory decision-making process for dealing with the management of natural resources. That is why the consideration of the preferential weights of different stakeholders with different perceptions with respect to the criteria considered is becoming of paramount importance.
  • To deal successfully with the above crucial and challenging issue, it would seem useful to hybridize the MCDM methods with those approaches belonging to the GDM field. Although the published works following this orientation are currently very scant, it would appear to be a promising future line of work.
  • The ecosystem services, climate change, and sustainability concepts have been recently incorporated as criteria in the management of the natural resources studied. Although the sustainability topic is widespread, the other orientations will seemingly be of key importance in future works in the field investigated. The same is expected of the term bioeconomy, whose popularity in academic spheres is even more recent.
  • The merger of GIS and MCDM methods is dramatically increasing in forest management, although its use in agriculture and fisheries is fairly negligible. This fact might be explained by the importance of spatial dimension in forest management.
It is logical to think that the above considerations and recommendations, supported by the hermeneutic meta-analysis undertaken, will help future researchers and practitioners to use the arsenal of MCDM methods for dealing with management problems more efficiently in the three types of natural renewable resources analyzed. Finally, this paper clearly shows the substantial extent and depth of MCDM applications in these fields over the past decade. In addition, it seems clear that practitioners interested in the sustainable management of land, forest, and fisheries resources could find this paper useful for identifying gaps and opportunities in order to properly apply these techniques in the future.

Supplementary Materials

A file is available online at https://www.mdpi.com/2073-445X/9/10/380/s1

Author Contributions

Conceptualization, L.D.-B., C.R., and S.G.d.J.,; methodology, L.D.-B., C.I.-M., C.R., and S.G.d.J.; software, S.G.d.J.; validation, L.D.-B., C.R., and S.G.d.J.; formal analysis, L.D.-B., C.I.-M., C.R., and S.G.d.J.; investigation, L.D.-B., C.I.-M., C.R., and S.G.d.J.; writing—original draft preparation, L.D.-B., C.I.-M., C.R., and S.G.d.J.; writing—review and editing, L.D.-B., C.I.-M., C.R., and S.G.d.J.; visualization, L.D.-B., C.I.-M., C.R., and S.G.d.J.; supervision, L.D.-B., C.I.-M., C.R., and S.G.d.J.; project administration, L.D.-B. All authors have read and agreed to the published version of the manuscript.

Funding

The work of Carlos Romero, Carlos Iglesias-Merchan, and Luis Diaz-Balteiro was funded by the Ministry of Economy and Competitiveness of Spain under project AGL2015-68657-R. Additionally, this research was partially financed by the European Union’s H2020 Research and Innovation Programme under the Marie Sklodowska-Curie Actions, grant agreement No. 691149–SuFoRun.

Acknowledgments

Thanks are given to Diana Badder and other native speakers for editing the English. In addition, we appreciate the comments raised by three referees who have helped us in order to increase the clarity and accuracy of our work.

Conflicts of Interest

The authors declare no conflict of interest

Appendix A. Searches in WOS and Scopus

Topics used in both databases:
(“multi-criteria” OR “multicriteria” OR “multiple criteria” OR “MCMD” OR “MCDA”);
(“goal programming”); (“compromise programming”); (“TOPSIS”); (“multiobjective programming” OR “multi objective programming”); (“analytic hierarchy process” OR “AHP” OR “analytical hierarchy process” OR “Analytic Hierarchical Process”); (“ELECTRE” OR “ELimination Et Choix Traduisant la REalité”); (“analytic network process” OR “ANP” OR “analytical network process”); (“data envelopment analysis” OR “DEA”); (“Promethee” OR “Preference Ranking Organization Method for Enrichment Evaluations”); (“MAUT” OR “multi-attribute utility theory” OR “multi attribute utility theory” OR “multiattribute utility theory” OR “MAVT” OR “multi-attribute value theory” OR “multi attribute value theory”); (“SMAA” OR “’stochastic multicriteria acceptability analysis’” OR “’stochastic multi-criteria acceptability analysis’” OR “’stochastic multiobjective acceptability analysis’”); (“VIKOR”); (“fuzzy methods” OR “fuzzy” AND (“multi-criteria” OR “multicriteria” OR “MCDM”)); (“weighted average” OR “WAM” OR “weighted arithmetic mean” OR “weighted geometric mean” OR “geometric mean” AND (“multi-criteria” OR “multicriteria” OR “MCDM”))
  
Searches in WOS
TOPIC: (“XXXX”). Refined by: Databases: (WOS) AND DOCUMENT TYPES: (ARTICLE OR REVIEW) AND RESEARCH AREAS: (AGRICULTURE OR FORESTRY OR FISHERIES)
Timespan: 2008–2017.
  
Searches in Scopus
(TITLE-ABS-KEY (“XXXX”) AND TITLE-ABS-KEY ( “forest” OR “agriculture” OR “ fisheries”)) AND PUBYEAR > 2007 AND PUBYEAR < 2018 AND (LIMIT-TO (DOCTYPE, “ar”) OR LIMIT-TO (DOCTYPE, “re”)
  
Note: “XXXX” indicates the topics shown above.

References

  1. Kavadas, S.; Maina, I.; Damalas, D.; Dokos, I.; Pantazi, M.; Vassilopoulou, V. Multi-Criteria Decision Analysis as a tool to extract fishing footprints and estimate fishing pressure: Application to small scale coastal fisheries and implications for management in the context of the Maritime Spatial Planning Directive. Mediterr. Mar. Sci. 2015, 16, 294–304. [Google Scholar] [CrossRef]
  2. Blagojević, B.; Jonsson, R.; Björheden, R.; Norsdtröm, E.-M.; Lindroos, O. Multi-Criteria Decision Analysis (MCDA) in forest operations—An introductional review. Croat. J. For. Eng. 2019, 40, 191–2015. [Google Scholar]
  3. Memmah, M.M.; Lescourret, M.; Yao, X.; Lavigne, C. Metaheuristics for agricultural land use optimization. A review. Agron. Sustain. Dev. 2015, 35, 975–998. [Google Scholar] [CrossRef]
  4. Gomiero, T.; Pimentel, D.; Paoletti, M.G. Is there a need for a more sustainable agriculture? Crit. Rev. Plant Sci. 2011, 30, 6–23. [Google Scholar] [CrossRef]
  5. Hayashi, K. Multicriteria aid for agricultural decisions using preference relations: Methodology and application. Agric. Syst. 1998, 58, 483–503. [Google Scholar] [CrossRef]
  6. Kaim, A.; Cord, A.F.; Volk, M. A review of multi-criteria optimization techniques for agricultural land use allocation. Environ. Model. Softw. 2018, 105, 79–93. [Google Scholar] [CrossRef]
  7. Santos, A.; Carvalho, A.; Barbosa-Póvoa, A.P.; Marques, A.; Amorim, P. Assessment and optimization of sustainable forest wood supply chains—A systematic literature review. For. Policy Econ. 2019, 105, 112–135. [Google Scholar] [CrossRef]
  8. Andalecio, M.N. Multi-criteria decision models for management of tropical coastal fisheries. A review. Agron. Sustain. Dev. 2010, 30, 557–580. [Google Scholar] [CrossRef]
  9. Romero, C.; Rehman, T. Natural resource management and the use of multiple criteria decision-making techniques: A review. Eur. Rev. Agric. Econ. 1987, 14, 61–89. [Google Scholar] [CrossRef]
  10. Crutchfield, J.A. Economic and political objectives in fishery management. Trans. Am. Fish. Soc. 1973, 102, 481–491. [Google Scholar] [CrossRef]
  11. Field, D.B. Goal programming for forest management. For. Sci. 1973, 19, 125–135. [Google Scholar] [CrossRef]
  12. Romero, C.; Amador, F.; Barco, A. Multiple objectives in agricultural planning: A compromise programming application. Am. J. Agric. Econ. 1987, 69, 78–86. [Google Scholar] [CrossRef]
  13. Hayashi, K. Multicriteria analysis for agricultural resource management: Acritical survey and future perspectives. Eur. J. Oper. Res. 2000, 122, 486–500. [Google Scholar] [CrossRef]
  14. Martins, H.; Borges, J.G. Addressing collaborative planning methods and tools in forest management. For. Ecol. Manag. 2007, 248, 107–118. [Google Scholar] [CrossRef]
  15. Diaz-Balteiro, L.; Romero, C. Making forestry decisions with multiple criteria: A review and an assessment. For. Ecol. Manag. 2008, 255, 3222–3241. [Google Scholar] [CrossRef]
  16. Acosta, M.; Corral, S. Multicriteria decision analysis and participatory decision support systems in forest management. Forests 2017, 8, 116. [Google Scholar] [CrossRef]
  17. Ortiz-Urbina, E.; González-Pachón, J.; Diaz-Balteiro, L. Decision-making in forestry: A review of the hybridisation of multiple criteria and group decision-making methods. Forests 2019, 10, 375. [Google Scholar] [CrossRef]
  18. Mardle, S.; Pascoe, S. A review of applications of multiple-criteria decision-making techniques to fisheries. Mar. Resour. Econ. 1999, 14, 41–63. [Google Scholar] [CrossRef]
  19. Deytieux, V.; Munier-Jolain, N.; Caneill, J. Assessing the sustainability of cropping systems in single- and multi-site studies. A review of methods. Eur. J. Agron. 2016, 72, 107–126. [Google Scholar] [CrossRef]
  20. Diaz-Balteiro, L.; González-Pachón, J.; Romero, C. Measuring systems sustainability with multi-criteria methods: A critical review. Eur. J. Oper. Res. 2017, 258, 607–616. [Google Scholar] [CrossRef]
  21. Stojčić, M.; Zavadskas, E.K.; Pamučar, D.; Stević, Z.; Mardani, A. Application of MCDM methods in sustainability engineering: A literature review. 2008–2018. Symmetry 2019, 11, 350. [Google Scholar] [CrossRef]
  22. Herva, M.; Roca, E. Review of combined approaches and multi-criteria analysis for corporate environmental evaluation. J. Clean. Prod. 2013, 39, 355–371. [Google Scholar] [CrossRef]
  23. Diaz-Balteiro, L.; Belavenutti, P.; Ezquerro, M.; González-Pachón, J.; Nobre Ribeiro, S.; Romero, C. Measuring the sustainability of a natural system by using multi-criteria distance function methods: Some critical issues. J. Environ. Manag. 2018, 214, 197–203. [Google Scholar] [CrossRef] [PubMed]
  24. Sadok, W.; Angevin, F.; Bergez, J.-E.; Bockstaller, C.; Colomb, B.; Guichard, L.; Reau, R.; Doré, T. Ex ante assessment of the sustainability of alternative cropping systems: Implications for using multi-criteria decision-aid methods. A review. Agron. Sustain. Dev. 2008, 328, 163–174. [Google Scholar] [CrossRef]
  25. De Luca, A.I.; Iofrida, N.; Leskinen, P.; Stillitano, T.; Falcone, G.; Strano, A.; Gulisano, G. Life cycle tools combined with multi-criteria and participatory methods for agricultural sustainability: Insights from a systematic and critical review. Sci. Total Environ. 2017, 595, 352–370. [Google Scholar] [CrossRef]
  26. Adrianto, L.; Matsuda, Y.; Yoshiaki, S. Assessing local sustainability of fisheries system: A multi-criteria participatory approach with the case of Yoron Island, Kagoshima prefecture, Japan. Mar. Policy 2005, 29, 9–23. [Google Scholar] [CrossRef]
  27. Rossetto, M.; Bitetto, I.; Spedicato, M.T.; Lembo, G.; Gambino, M.; Accadia, P.; Melià, P. Multi-criteria decision-making for fisheries management: A case study of Mediterranean demersal fisheries. Mar. Policy 2015, 53, 83–93. [Google Scholar] [CrossRef]
  28. Diaz-Balteiro, L.; Alfranca, O.; Bertomeu, M.; Ezquerro, M.; Giménez, J.C.; González-Pachón, J.; Romero, C. Using quantitative techniques to evaluate and explain the sustainability of forest plantations. Can. J. For. Res. 2016, 46, 1157–1166. [Google Scholar] [CrossRef]
  29. Diaz-Balteiro, L.; Alfranca, O.; González-Pachón, J.; Romero, C. Ranking of industrial forest plantations in terms of sustainability: A multicriteria approach. J. Environ. Manag. 2016, 180, 123–132. [Google Scholar] [CrossRef]
  30. Ezquerro, M.; Pardos, M.; Diaz-Balteiro, L. Sustainability in forest management revisited using multi-criteria decision-making techniques. Sustainability 2019, 11, 3645. [Google Scholar] [CrossRef]
  31. Mardani, A.; Jusoh, A.; Nor, K.M.D.; Khalifah, Z.; Zakwan, N.; Valipour, A. Multiple criteria decision-making techniques and their applications—A review of the literature from 2000 to 2014. Econ. Res. Ekon. Istraz. 2015, 28, 516–571. [Google Scholar] [CrossRef]
  32. Broniewicz, E.; Ogrodnik, K. Multi-criteria analysis of transport infrastructure projects. Transp. Res. Part D 2020, 83, 102351. [Google Scholar] [CrossRef]
  33. Ishizaka, A.; Labib, A. Review of the main developments in the analytic hierarchy process. Expert Syst. Appl. 2011, 38, 14336–14345. [Google Scholar] [CrossRef]
  34. Lewandowski, I. Bioeconomy: Shaping the Transition to a Sustainable, Biobased Economy; Springer: Cham, Switzerland, 2018. [Google Scholar]
  35. Viaggi, D. The Bioeconomy: Delivering Sustainable Green Growth; CABI: Wallingford, England, UK, 2018. [Google Scholar]
  36. Kardung, M.; Wesseler, J. EU bio-based economy strategy. In EU Bioeconomy Economics and Policies; Dries, L., Heijman, W., Jongeneel, R., Purnhagen, K., Wesseler, J., Eds.; Palgrave Macmillan: Cham, Switzerland, 2019; Volume II, pp. 277–292. [Google Scholar]
  37. Adriaanse, L.S.; Rensleigh, C. Comparing Web of Science, Scopus and Google Scholar from an environmental sciences perspective. S. Afr. J. Jnl. Libr. Inf. Sci. 2011, 77, 169–178. [Google Scholar] [CrossRef]
  38. Wang, Q.; Waltman, L. Large-scale analysis of the accuracy of the journal classification systems of Web of Science and Scopus. J. Informetr. 2016, 10, 347–364. [Google Scholar] [CrossRef]
  39. Cavacini, A. What is the best database for computer science journal articles? Scientometrics 2015, 102, 2059–2071. [Google Scholar] [CrossRef]
  40. Dos Santos, P.H.; Neves, S.M.; Sant’Anna, D.O.; de Oliveira, C.H.; Carvalho, H.D. The analytic hierarchy process supporting decision making for sustainable development: An overview of applications. J. Clean. Prod. 2019, 212, 119–138. [Google Scholar] [CrossRef]
  41. Romero, C.; Rehman, T. Multiple Criteria Analysis for Agricultural Decisions, 2nd ed.; Elsevier: Amsterdam, The Netherlands, 2003. [Google Scholar]
  42. Kangas, A.; Kurttila, M.; Hujala, T.; Eyvindson, K.; Kangas, J. Decision Support for Forest Management, 2nd ed.; Springer: Berlin, Germany, 2015. [Google Scholar] [CrossRef]
  43. Weintraub, A.; Romero, C.; Bjorndal, T.; Epstein, R. Handbook of Operations Research in Natural Resources; Springer: New York, NY, USA, 2007. [Google Scholar]
  44. Färe, R.; Grosskopf, S.; Pasurka, C.; Martins-Filho, C. On nonparametric estimation: With a focus on agriculture. Annu. Rev. Resour. Econ. 2013, 5, 93–110. [Google Scholar] [CrossRef]
  45. Iliyasu, A.; Mohamed, Z.A.; Ismail, M.M.; Abdullah, A.M.; Kamarudin, S.M.; Mazuki, H. A review of production frontier research in aquaculture (2001–2011). Aquac. Econ. Manag. 2014, 18, 221–247. [Google Scholar] [CrossRef]
  46. Sowlati, T. Efficiency studies in forestry using data envelopment analysis. For. Prod. J. 2005, 55, 49–57. [Google Scholar]
  47. Diaz-Balteiro, L.; González-Pachón, J.; Romero, C. Goal programming in forest management: Customizing models for the decision-maker’s preferences. Scand. J. Forest Res. 2013, 28, 166–173. [Google Scholar] [CrossRef]
  48. Riesgo, L.; Gallego-Ayala, J. Multicriteria Analysis of Olive Farms Sustainability: An Application of TOPSIS Models. In Handbook of Operations Research in Agriculture and the Agri-Food Industry; Plà-Aragonés, L.M., Ed.; Springer: New York, NY, USA, 2015; pp. 327–353. [Google Scholar] [CrossRef]
  49. Gao, L.; Hailu, A. Identifying preferred management options: An integrated agent-based recreational fishing simulation model with an AHP-TOPSIS evaluation method. Ecol. Model. 2013, 249, 75–83. [Google Scholar] [CrossRef]
  50. Cohon, J.L. Multiobjective Programming and Planning; Academic Press: New York, NY, USA, 1978. [Google Scholar]
  51. Steuer, R. Multiple Criteria Optimization: Theory, Computation, and Application; John Wiley and Sons: New York, NY, USA, 1986. [Google Scholar]
  52. Yu, P.L. Multiple-Criteria Decision Making. Concepts, Techniques, and Extensions; Plenum Press: New York, NY, USA, 1985. [Google Scholar]
  53. Zeleny, M. Multiple Criteria Decision Making; McGraw-Hill: New York, NY, USA, 1982. [Google Scholar]
  54. Berbel, J.; Bournaris, T.; Manos, B.; Matsatsinis, N.; Viaggi, D. Multicriteria Analysis in Agriculture. Current Trends and Recent Applications; Springer: Cham, Switzerland, 2018. [Google Scholar]
  55. Schmoldt, D.L.; Kangas, J.; Mendoza, G.A.; Pesonen, M. The Analytic Hierarchy Process in Natural Resource and Environmental Decision Making; Springer Science: Dordrecht, The Netherlands, 2001. [Google Scholar]
  56. Jato-Espino, D.; Castillo-Lopez, E.; Rodriguez-Hernandez, J.; Canteras-Jordana, C. A review of application of multi-criteria decision making methods in construction. Autom. Constr. 2014, 45, 151–162. [Google Scholar] [CrossRef]
  57. Ibáñez-Forés, V.; Bove, M.D.; Pérez-Belis, V. A holistic review of applied methodologies for assessing and selecting the optimal technological alternative from a sustainability perspective. J. Clean. Prod. 2014, 70, 259–281. [Google Scholar] [CrossRef]
  58. Govindan, K.; Rajendran, S.; Sarkis, J.; Murugesan, P. Multi criteria decision making approaches for green supplier evaluation and selection: A literature review. J. Clean. Prod. 2015, 98, 66–83. [Google Scholar] [CrossRef]
  59. Cegan, J.C.; Filion, A.M.; Keisler, J.M.; Linkov, I. Trends and applications of multi-criteria decision analysis in environmental sciences: Literature review. Environ. Syst. Decis. 2017, 37, 123–133. [Google Scholar] [CrossRef]
  60. Frini, A.; Benamor, S. Making decisions in a sustainable development context: A state-of-the-art survey and proposal of a multi-period single synthesizing criterion approach. Comput. Econ. 2018, 52, 341–385. [Google Scholar] [CrossRef]
  61. Huang, I.B.; Keisler, J.; Linkov, I. Multi-criteria decision analysis in environmental sciences: Ten years of applications and trends. Sci. Total Environ. 2011, 409, 3578–3594. [Google Scholar] [CrossRef]
  62. Malczewski, J.; Rinner, C. Multicriteria Decision Analysis in Geographic Information Science; Springer: New York, NY, USA, 2015. [Google Scholar]
  63. Bogdaz, A.; Yavuz, F.; Günay, A.S. AHP and GIS based land suitability analysis for Cihanbeyli (Turkey) County. Environ. Earth Sci. 2016, 75, 813. [Google Scholar] [CrossRef]
  64. Ramanathan, R.; Ganesh, L.S. Group preference aggregation methods employed in AHP: An evaluation and an intrinsic process for deriving members’ weightages. Eur. J. Oper. Res. 1994, 79, 249–265. [Google Scholar] [CrossRef]
  65. Uhde, B.; Hahn, W.A.; Griess, V.C.; Knoke, T. Hybrid MCDA methods to integrate multiple ecosystem services in forest management planning: A critical review. Environ. Manag. 2015, 56, 373–388. [Google Scholar] [CrossRef] [PubMed]
  66. Aznar-Sánchez, J.A.; Belmonte-Ureña, L.J.; López-Serrano, M.J.; Velasco-Muñoz, J.F. Forest ecosystem services: An analysis of worldwide research. Forests 2018, 9, 453. [Google Scholar] [CrossRef]
  67. Agresti, A. An Introduction to Categorical Data Analysis; John Wiley and Sons: Hoboken, NJ, USA, 2017. [Google Scholar]
  68. Howell, D.C. Statistical Methods for Psychology, 5th ed.; Wadsworth, Cengage Learning: Pacific Grove, CA, USA, 2002. [Google Scholar]
  69. Zyoud, S.H.; Fuchs-Hanusch, D. A bibliometric-based survey on AHP and TOPSIS techniques. Expert Syst. Appl. 2017, 78, 158–181. [Google Scholar] [CrossRef]
  70. Nobre, S.; Eriksson, L.-O.; Trubins, R. The use of decision support systems in forest management: Analysis of FORSYS Country Reports. Forests 2016, 7, 72. [Google Scholar] [CrossRef]
  71. Pollesch, N.L.; Dalle, V.H. Normalization in sustainability assessment: Methods and implications. Ecol. Econ. 2016, 130, 195–208. [Google Scholar] [CrossRef]
  72. Romero, C. Handbook of Critical Issues in Goal Programming; Pergamon Press: Oxford, UK, 1991. [Google Scholar]
  73. Kabir, G.; Sadiq, R.; Tesfamariam, S. A review of multi-criteria decision-making methods for infrastructure management. Struct. Infrastruct. Eng. 2014, 10, 1176–1210. [Google Scholar] [CrossRef]
  74. Keenan, P.B.; Jankowski, P. Spatial Decision Support Systems: Three decades on. Decis. Support Syst. 2019, 116, 64–76. [Google Scholar] [CrossRef]
  75. Evans, J.R. Sensitivity analysis in decision theory. Decis. Sci. 1984, 1, 239–247. [Google Scholar] [CrossRef]
  76. Esmail, B.A.; Geneletti, D. Multi-criteria decision analysis for nature conservation: A review of 20 years of applications. Methods Ecol. Evol. 2018, 9, 42–53. [Google Scholar] [CrossRef]
  77. Ezquerro, M.; Pardos, M.; Diaz-Balteiro, L. Operational research techniques used for addressing biodiversity objectives into forest management: An overview. Forests 2016, 7, 229. [Google Scholar] [CrossRef]
  78. Mendes, A.B.; Soares da Silva, E.L.D.G.; Santos, J.M.A. Efficiency Measures in the Agricultural Sector; Springer: Dordrecht, The Netherlands, 2013. [Google Scholar]
  79. Emrouznejad, A.; Yang, G. A survey and analysis of the first 40 years of scholarly literature in DEA: 1978–2016. Socio-Econ. Plan. Sci. 2018, 61, 4–8. [Google Scholar] [CrossRef]
  80. Behzadian, M.; Kazemzadeh, R.B.; Albadvi, A.; Aghdasi, M. PROMETHEE: A comprehensive literature review on methodologies and applications. Eur. J. Oper. Res. 2010, 200, 198–215. [Google Scholar] [CrossRef]
  81. Malczewski, J. Multiple criteria decision analysis and geographic information systems. In Trends in Multiple Criteria Decision Analysis; Ehrgott, M., Rui Figueira, J., Greco, S., Eds.; Springer: New York, NY, USA, 2010; pp. 369–395. [Google Scholar]
  82. Diaz-Balteiro, L.; González-Pachón, J.; Romero, C. Sustainability as a multi-criteria concept: New developments and applications. Sustainability 2020, 12, 7527. [Google Scholar] [CrossRef]
  83. Wesseler, J.; Jongeneel, R.; Purnhagen, K. Bioeconomy Economics and Policies. In EU Bioeconomy Economics and Policies: Volume I; Dries, L., Heijman, W., Jongeneel, R., Purnhagen, K., Wesseler, J., Eds.; Palgrave Macmillan: Cham, Switzerland, 2019; pp. 7–16. [Google Scholar]
  84. Zardari, Z.; Yusop, Z.; Shirazi, S.M.; Roslan, N.A.B. Prioritization of farmlands in a multicriteria irrigation water allocation: PROMETHEE and GAIA applications. Trans. ASABE 2015, 58, 73–82. [Google Scholar] [CrossRef]
Figure 1. Diagram of search process.
Figure 1. Diagram of search process.
Land 09 00380 g001
Figure 2. Number of studies per country in the assessment.
Figure 2. Number of studies per country in the assessment.
Land 09 00380 g002
Figure 3. Frequency of multicriteria decision-making (MCDM) techniques. *TOPSIS: Technique for Order of Preference by Similarity to Ideal Solution; **PROMETHEE: Preference Ranking Organization Method for Enrichment Evaluations.
Figure 3. Frequency of multicriteria decision-making (MCDM) techniques. *TOPSIS: Technique for Order of Preference by Similarity to Ideal Solution; **PROMETHEE: Preference Ranking Organization Method for Enrichment Evaluations.
Land 09 00380 g003
Table 1. Basic results of the survey.
Table 1. Basic results of the survey.
Number of papers analyzed628
Case study
Agriculture 323
Fisheries 59
Forestry 273
Number of countries87
Typology of journals
Agriculture 104
Fisheries 36
Forestry 128
Operations Research18
Multidisciplinary244
Other areas 98
Table 2. Several MCDM characteristics.
Table 2. Several MCDM characteristics.
Justification of the MCDM method chosen 45
Number of criteria 97
Normalization of criteria 314
Interaction with stakeholders 278
Sensitivity analysis 133
Method to assign weights to each criterion:
        Same vector of weights 99
        Implementation of a sensitivity analysis 14
        Requesting information from stakeholders89
        Other238
Software used (specified by the authors) 190
Table 3. Techniques used with MCDM.
Table 3. Techniques used with MCDM.
Statistical techniques 225
Decision support systems (DSS) 80
Nondeterministic 98
GIS 194
Sensitivity analysis 133
Table 4. Issues addressed.
Table 4. Issues addressed.
Ecosystem services 45
Climate change 64
Multifunctionality/Multiple use 59
Sustainability 157
Bioeconomy 9
Life cycle analysis 20
Table 5. Research hypotheses, methods used to contrast the hypotheses and results.
Table 5. Research hypotheses, methods used to contrast the hypotheses and results.
Research HypothesesMethod Used to Contrast the HypothesisHypothesis Result
1. The application of the multicriteria decision-making (MCDM) techniques does not significantly differ in the three fields (agriculture, forestry, and fisheries), given that we have not seen any comparative studies that support the opposite.Pearson’s chi-squared and Fisher’s exact testRejected
2. The number of criteria used in MCDM problems is similar in the three fields (agriculture, forestry, and fisheries).ANOVA testAccepted
3. The number of criteria is lower in continuous problems than in discrete ones.Welch’s two-sample t-test and ANOVA testAccepted
4. There seems to be a lesser use of MCDM techniques that apply to continuous problems than others that can only be applied to discrete ones. Welch’s two-sample t-testAccepted
5. There is no relationship between the use of a particular MCDM technique and the fact that the case studies are from one or from several countries.Multinomial logistic regressionAccepted
6. The use in the same problem of several MCDM techniques simultaneously has increased over time.Welch’s two-sample t-test and temporal analysisAccepted
7. (A) The criteria involved are usually normalized, independently of the field to be used.Chi-squared and Fisher’s exact testAccepted
 (B) The criteria involved are usually normalized, independently of the multicriteria technique to be used.Chi-squared and Fisher’s exact testRejected
8. (A) The justification of why the method is chosen is not usually given, whichever method is used. Fisher’s exact TestRejected
 (B) The justification of why the method is chosen is not usually given, whichever area it is applied in.Fisher’s exact TestAccepted
9. AHP and weighted MCDM always go together.Logistic regression and chi-squared testRejected
10. Given the inclusion of methods such as AHP and weighted MCDM in different GIS packages, it seems logical to point out that there is a positive relationship between the use of AHP and its application to spatial problems.Logistic regression and chi-squared testAccepted
11. AHP is used mostly to obtain weights from a set of stakeholders and/or experts and apply these weights to solve the problem in question. Logistic regression and chi-squared testAccepted
12. The hybridization of MCDM and GDM has increased over time.Welch’s two-sample t-test and temporal analysisAccepted
13. Given the nature of forestry problems the hybridization of GDM and MCDM techniques would seem to be more frequent in forestry.Chi-squared testAccepted
14. The concepts of ecosystem services, climate change, and sustainability are recent and have only become important in recent years.Welch’s two-sample t-test and temporal analysisAccepted
15. The topics that have been included as being relevant are only relevant in the last year range of the period.Welch’s two-sample t-test and temporal analysisRejected
16. There is a higher probability of using GIS hybridized with some MCDM techniques in the forestry area than in other fields.Logistic regression and chi-squared testAccepted
Back to TopTop