An Analytical Approach to Assess the Influence of Expert Panel Answer on Decision Making: The Case of Sustainable Land Use in Ribadavia Banda Norte, Salta (Argentina)

Abstract

Many of the techniques for making decisions, including land use, depend on the weight assigned for each criterion. These criteria can be based on a panel of experts’ opinions, who assess certain decisions’ influence on the final objectives. These opinions should be contrasted to decide if they are used or select the ones used to achieve an internal coherence. In this study, we evaluate the responses provided by an expert panel in the context of future environmental management of an agroforestry territory in the Salta Province (Argentina). The experts belong to different entities in the studied area, such as Universities, Research Centers, Administrative Authorities, Associations, and non-governmental organizations. They evaluated five productive techniques’ influence on 31 criteria related to environmental, social, and economic consequences. The Kendall’s Tau correlation coefficient between each pair of experts’ opinions is proposed to measure the rate of agreement among the expert panel answers. From these coefficients, a concordance matrix is generated. Based on this matrix, dendrograms are created to group the experts. In this case study, the results show two productive techniques with a high discordance rate, while the other three have a higher agreement among the expert panelists. The influence of these results in a multicriteria decision about the productive use of land is evaluated.

1. Introduction

Agriculture plays a vital role in many developing countries. In this sector, the issues of poverty alleviation, food security, and stable income generation for a rapidly growing population must be addressed [1,2]. There is a strong trend towards agricultural intensification in tropical regions, with the advance of its frontier. One example is Argentina, where it has caused the deforestation of approximately half a million hectares of forest in 2002–2011 [3,4]. For this reason, it is essential to analyze the different systems of agricultural production together with the bases of sustainability: productivity, stability, resilience, equity, and autonomy. Sevilla Guzmán [5] indicates three levels of great importance: ecological-productive, socioeconomic, and socio-political. Consequently, when implementing any production technique, policymakers and natural resource managers must have adequate environmental viability information [6]. Therefore, for decision making concerning choosing a productive technique, all possible information is required, such as identifying problems and evaluating, selecting, implementing, and evaluating alternatives.

To assess stakeholders’ and experts’ knowledge, interests, and positions regarding productive techniques, policymakers and managers can use a participatory analysis [7,8,9]. This analysis allows them to interact more effectively with key agents and increase support for a given policy or program [10,11,12]. When this analysis is conducted before a policy or program is implemented, policymakers and managers can detect and act to prevent potential misunderstandings about and/or opposition to the policy or program [13,14]. At the same time, expert knowledge has been valuable to overcome data unavailability problems on sustainability issues [15]. There are several examples of this type of approach. A study based on experts’ opinions was carried out by Van Calker et al. [16], where they analyzed the economic, internal-external social, and ecological-sustainability aspects. Another example is evaluating different production systems, such as rice [17], where they collected experts’ opinions. In both examples, a multi-criteria decision analysis (MCDA) was applied for decision making.

On the other hand, the Delphi technique refers to an iterative process of seeking expert opinion using questionnaires to form a consensus on a specific topic [18], such as production techniques. It has been applied in food safety and policy [19,20], food security [21,22,23] and its governance [24], farm sustainability [25], among others.

Despite the similarity, both methods present slight differences, as expert panels are formed in a structured way to seek clarity on reasons for a divergent opinion, and the survey is not repeated [17]. In Delphi, participants complete the survey questionnaire again, having the chance to confirm or amend their initial answers. The Delphi survey process can be reiterated several times until a consensus is achieved. However, higher numbers of rounds result in greater decline in participation and higher surveys’ costs [20]. Additionally, if the questionnaire needs to be more extensive to extract sufficient information on the topic to study, the participation declines faster.

MCDA uses many methods simultaneously and explicitly considers multiple and conflicting factors, including the value-based measurement models. This method requires the construction comparison of numerical scores to represent how one alternative is preferred with respect to another [26,27,28]. However, in the decision-making models that have been carried out, two shortcomings are identified. The first is that most models use experts’ opinions to obtain the scores but do not evaluate other experts’ opinions, which may cause the scores obtained not to be about scores found in the scientific literature. Secondly, the robustness of the results in many MCA (multi-criteria analysis) is not evaluated, but rather the uncertainty about the scores [29]. Both aspects lead to the need to develop tools to measure the reliability of the opinions expressed by the panel of experts, detect similarities or differences in the results, and establish opinion groups [30,31].

In this study, the aim is to analyze the internal coherence of an expert panel’s responses and discuss their influence on the MCA. We will apply it to the case study of the experts’ opinions about different agroecological techniques related to five agricultural production systems in the Chaco Salteño area.

With this aim, a measure of expert opinion concordance based on the Kendall tau correlation is defined, next employing a dendrogram methodology to cluser experts according to their opinion similitude. These results differentiate between techniques in which the experts present a consensus and others in which there are disagreements. Finally, the impact on a MCDA analysis is evaluated in selecting one or the other of the disagreeing groups or merging/averaging their opinions. Future lines are presented to manage the disagreements eventually detected.

2. Materials and Methods

2.1. The Study Area

This study focuses on Rivadavia Banda Norte’s area, in the Chaco Salteño, province of Salta (Argentina) (see Figure 1). There is an extractive-extensive agricultural production, causing poverty problems of inequality, unsatisfied basic needs, pollution, loss of biodiversity and traditional knowledge-ancestral, and one of the country’s highest deforestation rates.

The Chaco Salteño has a surface of 477,465 km², and its population is 121,578. Based on the statistical yearbook edited by the Province of Salta Government [32], the quality-of-life rate shows that 26.3% of households have unsatisfied basic needs. Similarly, the housing conditions are deplorable, with houses that mostly are made of adobe. They have precarious latrines and a lack of electricity.

Most of the population lacks social services. Because of their low income, they benefit from social assistance (pensions, food subsidies, among others). Telephone communications can only be established from counted places, one, at the posts, another, from cell phone signals or Pluma de Pato and Morillo’s localities. Moreover, families in the region communicate through rural messages transmitted by radio with modulated frequency.

The population comprises families of wichi origin and criollo origin, the latter to differentiate themselves from the natives. These two groups maintain a separation in several fields because of the significant cultural differences and, in some cases, conflicts related to land tenure. The criollo is considered a cattle producer, and the wichi is a collector, sometimes with a goat breeding system, and in many cases artisan. More information can be found from different points of view: socio-political [33], environmental impact [3,34], and livestock farming economy [35].

The study area is considered according to the territorial order of Argentina as a sector of medium conservation value. This classification determines that sustainable use can be carried out, such as tourism and silvopastoral systems. In addition, agriculture activity, with medium and high technology, is located in the study area.

From this situation, different alternatives for use and management are proposed. On the one hand, the maintenance of the traditional forms of criollos and wichis is considered. On the other hand, payment for ecosystem services is considered too, so aid to the population results in the maintenance of original ecosystems. The National Institute of Agricultural Technology (INTA) has a proposal for an integrated agroforestry system. Based on this institution’s technical criteria, this system is exhaustively defined and counted as a technical option in this study. Finally, an advanced agriculture technique highly adapted to the environment and efficient in using inputs and production, such as precision agriculture, is considered. The advantages and disadvantages for the studied area can be summarized in

Table 1. Advantages and disadvantages of techniques considered.

Techniques	Advantages	Disadvantages
Precision agriculture [36,37]	Efficiency in the management and use of agrochemical inputs. Efficiency in the use of natural resources. Reduction in contamination by pesticides. Reduction in soil erosion. High economic income.	High financial investment required. High technological knowledge required. Access to satellite information networks required. Skilled labor required. Deforestation of large areas. Greater intervention of the habitat. Increased greenhouse gas emissions. Low conservation of natural resources.
Integrated agroforestry [38]	Medium economic income (theoretically). Conservation of natural resources.	High financial investment required. High technological know-how required. Skilled labor required. Deforestation. Low conservation of natural cover.
Payment for ecosystem services [39,40]	High conservation of natural resources. Economic income (depends on payment agreements with funders). Requires low economic investment.	High financial investment required. High technological know-how required. Skilled labor required. Deforestation. Low conservation of natural cover.
Traditional agriculture-livestock farming—criollo [41]	Medium income (have some capacity to reinvest or save).	Extraction and depletion of natural resources. No conservation of natural resources.
Traditional forest management—wichi [42]	Conservation of natural resources. No investment required. Conservation of cultural identity.	Low income or economically unprofitable.

.

In summary, the production alternatives considered will be: (1) Precision agriculture (PA), (2) Forests with integrated livestock farming (FL), (3) Payment for ecosystem services (ES), (4) Local traditional agriculture-livestock farming—criollo (TC), and (5) Local traditional forest management—wichi (TW).

2.2. Survey for an Expert Panel

Interventional studies involving animals or humans, and other studies that require ethical approval, must list the authority that provided approval and the corresponding ethical approval code.

The identification of sustainable evaluation criteria was developed among the combination of the intragenerational approach, identification of current quality of life indicators, and the intergenerational approach, identification, and analysis of component aspects of the reference system’s ecological integrity.

Besides, starting from the main problems such as intensive agricultural production, poverty, deforestation, and loss of ancestral knowledge, a total of 31 questions were asked. They are directly related to intragenerational and intergenerational equity. They are intended to meet today’s needs without compromising future generations’ capacity, thus obtaining sustainable development (see Appendix A. Survey questions).

The 31 criteria for a sustainable development are assembled/clustered in five criteria groups based on the influence area: infrastructure (CG1), sustainability (CG2), ecology—natural resources (CG3), economic—productive (CG4), and social (CG5). The categorization corresponds to the structure of the survey developed by [43].

This set of questions aimed to determine how the five techniques or production systems considered previously, named as alternatives, would influence the criteria if applied in the study area.

In the survey application, the expert answers any question about how the technique’s application will influence the study area according to the criteria; otherwise, it will be left blank. If the expert responds, he must assess if the technique has any impact; if not, the score will be 0. If he considers any positive or negative impact, he must rate it from 1 to 5 (from less to more impact).

For these calculations, the scores were transformed on a numerical scale from 1 to 11, as showed in

Table 2. Numerical survey score and corresponding scale for calculations.

	Bad Impact					Non	Good Impact
Scale	1	2	3	4	5	6	7	8	9	10	11
Score	5	4	3	2	1	0	1	2	3	4	5

. In this way, a continuous 11-point Likert scale [44] is obtained. To facilitate the expert’s answer, we divided it into three questions: (1) does the technique impact the criterion(?), (2) has it a positive or negative impact(?), (3) to what degree it impacts(?); 1—little to 5—a lot.

2.3. Selection of Expert Panel

In selecting experts for our study, we considered Scheele’s [45] recommendation to bring together three panellists: stakeholders, experts, and site facilitators or trainers. Besides the group members’ education and profession, their proximity to the production systems was considered. These stakeholders must have extensive knowledge of agriculture and forest policy formulation, so their inclusion in the panels is essential [46].

Our final panel comprises government institutions, expert teachers from academia, independent producers, NGOs (non-governmental organizations), and technicians who work together with people from the influence area. Each expert has been coded with an alphabetical letter for the final analysis (see

Table 3. Code of experts interviewed and institutions to which the professionals who responded to the survey belong.

Code	The Institution They Belong to	Type
A	Catholic University of Salta (UCASAL)—Environmental law—Professor.	University
B	Prograno Association.	Producer Association
C	Association. Social Integration of the Agricultural and Forestry Sector in the Chaco region of Salta (ISFACSA).	NGO
D	Institute of Environmental Law and Sustainability—(IDEAS, UCASAL)—Staff.	University
E	Secretariat of Environment and Sustainable Development of the Nation, Argentina.	Government—Environmental
F	Sustainable use of biodiversity project UNDP ARG15/G53 (Livestock)—Staff.	International organization—Environmental
G	TEPEYAC, a non-profit civil association, is part of the Diocese’s aboriginal pastoral of the New Oran.	NGO (aboriginal)
H	Catholic University of Salta (UCASAL), Law—Professor.	University
I	National Institute of Agricultural Technology (INTA)—Staff.	Research (government)
J	Eco-enterprises.	Producer Association (ecology)
K	Independent Producer.	Producer (independent)
L	National Institute of Agricultural Technology (INTA). Animal health, ecophysiology.	Research (government)
M	Secretariat of Family Agriculture.	Government—Family agriculture
N	Catholic University of Salta (UCASAL)—Professor	University
O	National Institute of Agricultural Technology (INTA).	Research (government)
P	Sustainable use of biodiversity project UNDP ARG15/G53 (Livestock)—Technician.	International organization—Environmental
Q	Fundapaz—Staff.	NGO (aboriginal and family)
S	National Institute of Agricultural Technology (INTA)—Natural Resources and Environment	Research (government)

).

2.4. Concordance Matrix and Dendrograms

Since the answers are qualitative ordinals, non-parametric statistical methods are used [47]. It is not feasible to obtain a global concordance index between judges, such as Kendall’s W, because there are missing answers (do not know/any answer). Therefore, Kendall’s tau correlation coefficient is used following the algorithm of Knight [48]. The following expressions are applied, x and y being the values of the responses of two of the experts:

$$\frac{{{\displaystyle \sum }}_{i<j}\left(sgn\left(x_i-x_j\right)sgn\left(y_i-y_j\right)\right)}{\sqrt{\left(T_0-T_1\right)\left(T_0-T_2\right)}}$$

(1)

$$T_0=\frac{n\left(n-1\right)}{2}$$

(2)

$$T_1={{\displaystyle \sum }}_k{t}_k\frac{\left(t_k-1\right)}{2}$$

(3)

$$T_2={{\displaystyle \sum }}_l{u}_l\frac{\left(u_l-1\right)}{2}$$

(4)

where i and j represent each pair of responses; sgn() is the sign (positive or negative) of a difference between pairs of data; n is the number of data; t_k and u_l are the numbers of ties (pairs of data with equal values) in the variable x and the variable y, respectively. Applying the methodology to each pair of experts (when they had answered at least five common questions), the correlation matrix between the opinions is obtained.

Dendrograms [49] are constructed to group the experts according to their opinions. This method is widely used to group elements according to their characteristics, from social [50] to genetics [51]. In our case, we use the previous correlation matrix as a concordance matrix between experts and the simple methodology of the nearest neighbour’s merging strategy.

The algorithm (single linkage or nearest neighbour clustering) is outlined as follows. Let the similitude between elements i and j be s_ij.

• Find the greatest element s_ij (elements with higher similitude).
• Merge elements i and j into a single new cluster.
• Calculate a new set of distances defined as the minimum distance (maximum similitude) between the new cluster and the remaining elements.
• Repeat steps 1–3 until a single group contains all elements (in step >1, the element could be clusters derived from the previous agglomeration).

The process is also stopped when a non-significant correlation is achieved to avoid grouping experts without agreement (confidence level less than 95%, although it depends on the number of common questions answered by each pair of experts to be compared, approximately 0.3). In Supplementary Materials, we have included the agglomeration process and associated similitudes example for Technique 5.

2.5. Discrete MCDA Method

The primary purpose of the MCDA consists of proposing methods for aggregating criteria (objectives, goals, attributes) in conflict. In this way, compromises among the criteria considered with a clear preferential interpretation are obtained. In the last years, practically all the methods within the MCDA have been applied for solving different problems.

The discrete methods used in multiple criteria problems are characterized by a finite number of alternatives, according to the level of fulfilment of the criteria considered. They can be divided into outranking, hierarchical, and average optimization. Outranking methods build binary relations of the type “option A is at least as good as option B”. Among them, we can find the preference ranking organization method for enrichment evaluation (PROMETHEE), the approach to imprecise assessment and decision environments (NAIADE) and the elimination and choice-expressing reality (ELECTRE). The hierarchical methods set not only the relative importance of criteria but also consider levels of hierarchy. In this type of methods, the analytic hierarchy process (AHP) and the analytic network process (ANP) are the most common. Finally, the average optimization consists of a mathematical optimization involving multiple objective functions. The most well known are: weighted arithmetic mean (WAM), weighted geometric aggregation (WGA), multiattribute utility theory (MAUT), multiattribute value theory (MAVT), heuristic optimization method (HERO), simple multiattribute rating (SMART), and the stochastic multicriteria acceptability analysis (SMAA). An overview of these methods applied to sustainability can be found in [8,52].

For our study, we will perform the discrete MCDA analysis using a classical method, the ELECTRE [53] overcoming method, to obtain a classification and positioning in the alternatives’ order and location, in this case, techniques based on the weighting and opinion of experts. The ELECTRE method was applied following [54,55].

The criteria are designed with subscript j, and the alternatives are designed with i and k. The weights of the criteria (W_(j)) were standardized to sum 1, getting W_j.

To obtain the alternative preference, the Concordance Index Matrix (C_ik) is calculated as:

$$C_{ik}=\left\{\begin{array}{c}\sum W_j, I_j\left(I{m}_{ij}-I{m}_{kj}\right)<0\hfill \\ W_j/2, I{m}_{ij}=I{m}_{kj}\hfill \end{array}\right.$$

(5)

where W_j are the weights of the criteria; I_j is the Criteria Index Vector for criteria j (+1 “more is better” or −1 “more is worst”); and Im_ij is the matrix of each criterion’s values in each alternative.

The Concordance Index Matrix indicates how much alternative i is better than alternative k, adding the weights of the criteria for which that occurs.

The obstacles to these preferences or discordances are considered. First, the ranges ($R$) are calculated as:

$$R_j=Su{p}_{ik} \left|I{m}_{ij}-I{m}_{kj}\right|$$

(6)

Second, the Standardized Decisional Matrix ($Dm$) is calculated as:

$$D{m}_{ij}=I{m}_{ij}·W_j/R_j$$

(7)

Finally, the Discordance Index Matrix ($D$) is computed as:

$$D_{ik}=Su{p}_j\left|I_j,\left(D{m}_{kj}-D{m}_{ij}\right),0\right|/Su{p}_j\left|D{m}_{kj}-D{m}_{ij}\right|$$

(8)

Now, the Matrix of Concordance Dominance ($Mcd$) is computed:

$$Mc{d}_{ik}=\left\{\begin{array}{c}1, C_{ik}x\ge ct\hfill \\ 0, C_{ik}<ct\hfill \end{array}\right.$$

(9)

where $ct$ is the concordance threshold; it is calculated as the average of the non-diagonal elements of the C_ik.

The Matrix of Discordance Dominance ($Mdd$) is obtained as:

$$Md{d}_{ik}=\left\{\begin{array}{c}1, D_{ik}x\ge dt\hfill \\ 0, D_{ik}<dt\hfill \end{array}\right.$$

(10)

where $dt$ is the concordance threshold; it is calculated as the average of the non-diagonal elements of the D_ik.

Finally, we get the Matrix of Aggregated Dominance ($Mad$) for each (i,k):

$$Ma{d}_{ik}=Mc{d}_{ik}\times Md{d}_{ik}$$

(11)

The diagonal elements of the dominance matrices do not intervene and are taken as 0. If, for a given (i,k), the Mcd_ik is 1, that is an indication of dominance, and if the Mdd_ik is 1, there is no discordance of alternative i over alternative k. If Mad_ik = 1, the alternative i is considered better than alternative k. An alternative that is better than some others and worse than none is included in the kernel.

It is convenient to avoid excessive discordances in the Discordance Index Matrix. In this way, the evaluation of the robustness in the sense of sensibility analysis is achieved. For this reason, the values of the D_ik must be changed. With this, it can easily be proved that small changes in data do not change the result.

A sheet of MATHCAD 8 PRO (v. 14.0.0.163) was used, and the values for thresholds of agreement and discordance were 0.5 and 1.0001, respectively. The weighting, carried out by the experts, is on a scale of 0 to 1.

For the MCDA, there are five criteria groups: Infrastructure (CG1), Sustainability (CG2), Ecology—natural resources (CG3), Economic—productive (CG4), and Social (CG5). The weights to assess each criterion’s relative importance are considered balanced in this work, that is, 0.2 for each set (from CG1 to CG5). The alternatives will be the five production systems described in Section 2.1 (PA, FL, ES, TC, TW).

This part of the work is kept methodologically simple (a single MCDA and balanced criteria) to focus on the mainline of the study (in Appendix B, the results with another method, AHP, and three weight variants of the criteria groups are included). It will have to be considered for an eventual operational solution.

The weights matrices (W_(j)) are derived from the scores made by the panel of experts. They are usually obtained [56] by averaging all the experts’ opinions for the joint criteria (CG1 to CG5). In many studies, this is not clarified, or the experts are forced to reach a consensus [57]. This imposition creates the problems that we have indicated for the Delphi method. In our case, if contrasts of opinion between experts are detected with the dendrograms, alternative matrices will be made with the opinions of each disagreeing group of experts.

3. Results

3.1. Survey

The survey was conducted in person, asking questions to each expert and recording them by the interviewer. In this way, the lack of answers to the questions is minimized. The survey result was a set of all the experts’ responses (in Supplementary Materials, the expert responses to Technique 5 is included as an example).

Among the experts, A, H, and L did not respond to one survey technique. Expert K did not respond to two techniques, and expert P did not respond to three techniques. The experts C, O, and S did not respond to four techniques. The experts stated that the lack of response was due to ignorance of the technique’s concept and functioning. In contrast, experts B, D, E, F, G, I, J, M, N, and Q did respond to all surveys. It should be made clear that disagreeing respondents are not part of a single institution or type.

3.2. Dendrograms

Applying Kendall’s tau correlation coefficient to each pair of experts’ responses produces the correlation matrix (or concordance matrix) between opinions. The dendrograms presented below were generated from these matrices (in Supplementary Materials, the correlation matrix for Technique 5 is included as an example).

3.2.1. Precision Agriculture: Technique 1

For the first technique, Precision agriculture, a dendrogram is obtained and showed in Figure 2. In it, we observe a first group composed of two university professors (A, D), two researchers (O, I), one government member (E), one NGO (G), and one international organization worker (F). A second group is composed of two representatives of producer associations (B, J), a researcher (L), an international organization worker (P), a university professor (N), a member of the government (M), and a NGO (Q).

In turn, one NGO, an independent producer, one of the researchers, and a university professor are excluded from the dendrogram due to non-significant correlations or lack of opinion.

The dendrogram shows two groups of experts with no homogeneous characteristics (institution type or position) and no significant correlation. It follows, therefore, that Precision agriculture is perceived very differently by different groups without seeming to affect their professional dedication. The group T1-B = (B, L, P, M, Q, J, N) is perceived with more experience in agricultural production (professional or personal) and as considering difficulties in technique application conditions. The group T1-A = (A, E, O, D, F, I, G) is perceived as considering this technique from its potentialities perspective.

3.2.2. Forest with Integrated Livestock: Technique 2

For the second technique, Forest with integrated livestock, the dendrogram of Figure 3 is obtained, finding two groups of opinions, but which are not identical to Precision agriculture techniques. One group of opinion is larger than the other, including an expert from the government, an international organization, and a producer association.

It also highlights that more experts (6) are out of significant correlation or without opinion in this case.

In this case, the group T2-C = (A, I, N, K, F, B, D, G, L) represents the experts who adhere to the technique, which is the government’s proposal. On the opposite side, the minority group T2-D = (E, J, P) perceives this technique as challenging to implement due to the technical and regulatory requirements.

3.2.3. Payment for Ecosystem Services: Technique 3

For the Payment for Ecosystem Services technique, the dendrogram in Figure 4 is obtained. The experts are grouped to include the majority in a group with a significant correlation. Thus, for this technique, there is a certain consensus among the majority of experts.

3.2.4. Traditional Agriculture-Livestock Farming—Criollo: Technique 4

For the Traditional agriculture-livestock technique, in the dendrogram of Figure 5, the experts are grouped similarly to the previous case. The majority is included in a group with a significant correlation. It also shows a sequenced grouping, one by one, of most experts at low similarities.

3.2.5. Traditional Forest Management—Wichi: Technique 5

Finally, for Forest management’s technique without technology or traditional, we obtain the dendrogram of Figure 6. It is observed that the experts are grouped in two initial sets that merge and end up including the majority in a group with a significant correlation.

3.3. Discrete MCDA Results

The weights assigned by the experts as a whole are shown in

Table 4. Weights assigned for each technique (CG1 until CG5, see Survey section). The experts (T1 till T5) and their subgroups (A, B, C and D) based on the dendrograms.

	CRIT	CG1	CG2	CG3	CG4	CG5
TÉC
T1		6.706	6.239	5.884	6.388	6.407
T1-A		6.848	6.853	7.667	6.767	7.917
T1-B		6.509	5.382	4.029	6.094	5.077
T2		7.560	7.871	7.199	7.730	7.450
T2-C		7.813	8.317	8.400	8.214	8.353
T2-D		7.571	6.929	4.667	7.462	5.500
T3		7.578	7.966	8.992	7.508	9.174
T4		6.245	7.150	4.087	5.407	7.000
T5		5.411	5.419	3.548	4.086	4.375

. Furthermore, the weights assigned in the differentiated subgroups defined through the dendrograms analysis are included too.

The results obtained with ELECTRE are presented in

Table 5. The position occupied by production techniques after been evaluated with ELECTRE for the different variants. With groups of experts (V-AC to V-BD) and without grouping experts (V). A, B, C, and D are the opinion groups involved in each analysis.

	V-AC	V-AD	V-BC	V-BD	V
1st	T2	T3	T2	T3	T3
2nd	T3	T2	T3	T2	T2
3th	T1	T1	T4	T4	T1
4th	T4	T4	T1	T1	T4
5th	T5	T5	T5	T5	T5

. The analysis that we would obtain without considering discrepancies in the experts (the usual way of proceeding) would correspond to V of Table 5. It is appreciated that the T3 Payment for Ecosystem Services technique would be the best valued, followed by T2 (Forests with integrated livestock farming) and T1 (Precision agriculture). In contrast, traditional local techniques are the worst evaluated.

Variations that include opinion group C (which affected technique 2) cause T2 to pass in front of T3 and take it away from first place. This change is consistent, as this group adhered to the government research center’s proposal (Forests with integrated livestock farming).

A change in position is also observed when opinion group B intervenes (affecting technique 1), causing T1 (Precision agriculture) to fall below T4 (Local traditional agriculture-livestock farming—criollo), losing third place. This swap is because this group finds it difficult to implement such an advanced technology as precision agriculture.

Opinion groups A (for technique 1) and D (for technique 2) represent the opinion that is closest to the dominant one, and when they are part of the same variant, they generate an order similar to the total, which does not consider discrepancies.

4. Discussion

In summary, it was evident that techniques 1 and 2 have a high rate of discordance, while techniques 3, 4, and 5 have a higher consensus among the panel of experts.

In addition to non-responses, some experts were left out of a significant correlation in some techniques (B, C, D, H, L, M, and Q); that is to say, some experts disassociate themselves from the general point of view.

In the techniques with group consensus, the weights can be integrated commonly, making averages.

Dissenting and high blank-response-rates experts should be withdrawn for lack of minimum consensus, even at the risk that their knowledge is appropriate in some specific aspect or field.

The proposed methodology identifies both the initial experts who cannot answer specific topics and those who do not agree significantly with the rest of the experts. This identification allows for the withdrawal of uninformed experts (at least from a peer judgment point of view). It maintains a diversified panel, which would be optimal according to Gong and Yang [58].

In contrast to the Delphi technique (e.g., [59], on agricultural pest management), where successive rounds allow a final consensus to be reached, our method becomes more straightforward in the application. It allows a second phase of the analysis of the consensus and its objectification. However, our method does not allow confrontation between experts based on the studies that support each opinion. On the other hand, our method is not sensitive to reducing experts’ answering successive versions of the survey, which can be reduced to 30% of the initial participants [60]. It should also be commented that, in our case, to increase the collaboration of the experts, they were interviewed in person, also minimizing the problems of filling in the survey.

Concerning the environmental, economic, and social areas, [29] report a high variability level. Our survey collects several of these aspects and therefore justifies the variability found, which leads to two opinion groups in techniques 1 and 2 and the detection of experts with problems in agreement with the majority. In higher consensus techniques (3, 4 and 5), the agreement problem is also detected in some experts. Therefore, the method works reasonably well with 18 experts or experts groups, although more robust results can be assumed with more subjects.

Hagerhall [61] makes a procedure to cluster respondents on landscape preference, but he does not define a significant correlation limiting the aggregation process.

The differences between survey respondents are consistent with others referred to in the literature. A study carried out on the uncertainty analysis, and an estimation of the priorities of landscape sensitivity variations was found between the experts and locals and tourists [62], indicating that it is necessary to carry out an individual and collective analysis of the experts. The study of the socio-ecological assessment of flood threats [63] indicates a different opinion between experts and farmers in a basin’s vicinity and those located in a remote part of the basin.

In all cases, the authors are looking for group differences; nevertheless, our objective method checks for expert discrepancies not previously defined or suspected.

Using some opinion groups or others moves the techniques’ position for land use. These changes are not linked to belonging to different institutions or positions. They are related to being immersed in the productive fabric, highlighting the difficulties of implementation or adhering to official criteria.

The proposed analysis allows us to quantify the agreement between the experts and group these experts according to their opinions. This procedure allows facing the discrepancies instead of hiding them behind a mean or a forced consensus. However, this highlights the need for a complementary study to assess the problematic aspects detected.

We aim to study the results with other MDCA to check the results. Moreover, different scenarios can be set, assigning the criteria groups different weights, for example, giving more weight to the sustainability, economic, or ecological aspects. Including these weights could also highlight the underlying perspective of each expert group. It would not be about finding the best option for all the land, but rather assigning percentages of the area to each of the techniques considered according to their assessment, both qualitatively by the judges and with objective criteria.

In future research, already underway, we will try to include the quantification of discrepancies between judges in different decision methodologies, such as those proposed by [64] and/or from a securities portfolio methodology commonly used in the economic field. Then, the discrepancies will be considered as uncertainty in the decision or investment risk, respectively.

5. Conclusions

A method is proposed for analyzing the responses of a panel of experts based on obtaining the matrix concordance between experts with Kendall’s tau correlation and the generation of dendrograms from the matrix to group experts.

The method can locate those techniques that show a high degree of discordance between experts, reflected in the dendrograms that define groups that are kept separate, and those with higher concordance that converge in a single group dendrogram.

The groups of experts detected would not have been easily identified with a priori analysis.

In this case, we have detected two groups of experts who distance themselves from the opinion of the rest in two of the evaluated techniques, precision agriculture and integrated agroforestry, and whose consideration changes the result of the MCDA. Considering different opinion groups reflects changes in assessing the techniques to be implemented in the area, with the consequent repercussion on the future land use.

Experts are also detected who repeatedly distance themselves from the rest of the experts’ opinions without grouping among them. Therefore, we suggest removing these single dissenting experts from the calculation of weights for MCDA.