Sensitivity of Multi-Criteria Analysis Methods in Rural Land Consolidation Project Ranking

Abstract

Decisions around distributing available funds among potential land consolidation projects require a thorough analysis in order to maximize the effects of land consolidation. In order to avoid choosing the wrong land consolidation projects, different methods can be used. Generally, there are two possible groups of methods: one based on a qualitative approach (DELPHI; SWOT) and one based on a quantitative approach (AHP, VIKOR, SAW, TOPSIS, etc.). In this research, the focus was on the sensitivity of the resulting rankings affected by varying the input data in multi-criteria analysis methods, with an emphasis on the variation in the weight and the choice of criteria. This research was motivated by the subjective character of the choice of criteria and their weighting before applying the multi-criteria analysis methods. Four methods were included (AHP, TOPSIS, VIKOR, and SAW) for the multi-criteria analysis, with three ways of defining weights (consistent, modified, or quasi-consistent and freely determined without taking consistency into account), in order to determine the influence of the different methods on the final ranking. The weights were defined only by an acceptable interval of values. The sensitivity of the methods was investigated using the differences in the obtained rankings between each method. A case study is provided on real data, and the results are discussed. The results showed a relatively small variance and possible equal rankings of projects by means of statistical analyses. This finding opens up the possibility of the valuation of projects instead of simple rankings.

1. Introduction

Land consolidation is a complex process distinguished by its importance and effectiveness in the domains of sustainable development and the spatial planning of rural areas, but it is also expensive and takes a long time [1].

According to the research provided by Yin et al. [2], based on 193 cases documented in 92 peer-reviewed articles, rural land consolidation in China has resulted in more positive effects than no/negative effects on rural revitalization, with positive outcomes in 74% of cases. This result implies that, for every four land consolidation projects, one may be unsuccessful. This result also indicates the need to take great caution during the process of land consolidation project selection, because the distribution of one-fourth of the available funds to unsuccessful projects cannot be considered optimal.

The durability of land consolidation processes can be illustrated using Turkey as an example, where land consolidation was initiated in the year 1961 and has not finished yet [3]. This reflects how land property heritage causes parcel fragmentation and consequently necessitates continual land consolidation. Land property fragmentation has many negative effects on agricultural production, including a decrease in yield, an increased duration of land cultivation, and an increase in fuel costs and the time required to access parcels, consequently decreasing agricultural profitability [4].

The importance and benefits of land consolidation projects suggest that a high number of competing land consolidation projects can be expected, or, in other words, there may be more land consolidation projects than can be covered with the available funds. Thus, land consolidation is an inevitable process from the aspect of sustainable development, but it is also limited by the available resources, which has implications for decision making about the optimal distribution of available resources over a set of competing land consolidation projects.

Selecting from multiple possible alternatives in the domain of land consolidation is a complex task that requires a huge amount of information to be properly combined and well understood. In the case of decision making around the choice between complex and resource-demanding projects, it is not recommended to leave this to one person (regardless of their level of expertise), as they may make choices grounded in their own preferences. To find the best or optimal solutions for this decision-making problem, numerous techniques have been developed. In general, these techniques can be divided into two groups: qualitative techniques and quantitative techniques. Qualitative techniques are based on experts’ opinions and an analysis of different parameters, while quantitative techniques are based on mathematical algorithms and computer utilization.

Both qualitative (DELPHI; SWOT matrix—strength–weakness–opportunity–threat) and quantitative (AHP—analytic hierarchy process; ELECTRE—elimination et choix traduisant la realite; TOPSIS—technique for order of preference by similarity to ideal solution; SAW—simple additive weighting; VIKOR—from Serbian “VIseKriterijumska Optimizacija I Kompromisno Resenje”, meaning multi-criteria optimization and compromise solution, with the pronunciation vikor) methods have been used to evaluate land consolidation projects in practice and research. The Delphi method assumes that forecasts or decisions from structured groups are more reliable than those from unstructured groups [5]. The Delphi method is used in land consolidation to solve different problems: to select evaluation indicators and construct a land benefit evaluation index system [6]; to evaluate the influence of a land consolidation project on the cultivated land quality [7]; and to determine weights and quantify land quality [8]. In [9], a SWOT analysis was used to distinguish the strengths and weaknesses of land consolidation and to determine its opportunities as well as its threats. Elsewhere, a SWOT analysis was used as a base strategic planning framework for land consolidation at the national level in China [10], for evaluating a specific Italian approach for consolidating small portions of abandoned land in a functional manner [11], and for an analysis of the initial state of the economic, environmental, and social situation in the Czech Republic’s rural areas [12]. The procedure of land consolidation project ranking is structured as follows: the definition of the ranking criteria; the calculation of quantitative indicators for the selected criteria; data normalization; an evaluation of the weighting coefficients; and the ranking of projects by utilizing a multi-criteria analysis [13]. A multi-criteria analysis is a highly developed domain of science represented by various methods utilized in land consolidation project ranking: the COPRAS method [13]; the best–worst method [14]; the PROMETHEE and ELECTRE methods [15]; the AHP method [16]; and AHP and TOPSIS [17]. The existing methods entail the issue of research and improvement, and including new factors in an AHP analysis could lead to the improvement of existing conventional methods [18].

Some studies have utilized different quantitative methods for land consolidation project ranking [19]. It is also possible to integrate qualitative and quantitative methods in the processes of decision making [6,20,21,22].

A sensitivity analysis is of crucial importance in assessing the reliability of an obtained rank using different MCDM methods. Bearing in mind that the decision maker might not be an expert in the MCDM method utilized, it is logical to assume that the decision making will be left to the mathematical algorithm on some level. In this case, the decision maker could be surprised by the results obtained when applying different MCDM methods, or they could even disagree with them. Such cases could be the consequence of small variations in the set of initial data. Accordingly, in the following sections, research on the sensitivity of different MCDM methods is analyzed and discussed.

One important issue regarding MCDM methods is how sensitive they are to the initial data, i.e., how small variations in the input change the final parameters (in this case, the ranking of alternatives). According to Triantaphyllou and Sánchez [23], the data in multi-criteria decision making are often imprecise and changeable, indicating that performing a sensitivity analysis on the input data is an important step in their many applications. The authors stressed their empirical conclusion that the choice of MCDM method and the number of alternatives have little influence on the sensitivity results, and that the most sensitive decision criterion is the one with the highest weight. This last point seems logical and intuitive, but it also implies that the hierarchy of weights defines the hierarchy of sensitivity.

In their paper, Chen and Kocaoglu [24] provided a comprehensive algorithm for a sensitivity analysis of hierarchical decision models. By utilizing this algorithm, it was possible to identify the allowable range/region of perturbations, the contribution tolerance, the operating point sensitivity coefficient, the total sensitivity coefficient, and the most critical decision element “at a certain level”. This research and algorithm involve complex mathematical explanations, but according to the concluding remarks, there are many possibilities for their improvement. This means the algorithm should be applied with some caution.

Chang et al. [25] stated that the final priorities of the alternatives are highly dependent on the weights attached to the main criteria, meaning that small changes in the relative weights can cause major changes to the final ranking. However, this statement should be taken with caution because it is the result of applying the AHP method in a specific industry.

Vavrek [26] investigated the TOPSIS method by utilizing five objective weighting methods, providing a comprehensive statistical analysis, and testing hypotheses. In addition, the pros and cons of each weighting method were discussed. The conclusion of this research was that “the selection of an adequate weighting method has a significant impact on the overall results of the TOPSIS technique”. This conclusion implies that the process of applying the TOPSIS technique is sensitive and requires a high level of knowledge in order to be utilized properly for each case.

The VIKOR method was developed in 1990 to solve decision problems with conflicting criteria [27]. According to Zhao et al. [28], “it is difficult to precisely determine criteria weights and the ratings of alternatives on each criterion in real-life situations”. This is the reason why the authors developed the intuitionistic fuzzy hybrid VIKOR (IFH-VIKOR) method. Based on the examples given, it was obvious that ranking alternatives are dependent on the decision maker’s opinion, i.e., on the input data.

The sensitivity of the SAW method, meanwhile, is relatively simple because it depends on the variation in the criterion value and the weight [29].

It is also important to stress that, on the basis of using multiple MCDM approaches, it is possible to develop new methods. Bognár et al. [30] developed the partial risk map (PRISM) method based on a multiple AHP-TOPSIS approach. The authors stated (among other statements) that the evaluation process is based on deterministic evaluation scales, and they concluded that the “data sensitivity of the model is critical”. This implies that it is possible to combine two methods and, on this basis, to improve some methods that, alone, suffer from some deficiencies. Opricovic and Tzeng [31] provided a comparative analysis of the VIKOR and TOPSIS methods and identified their similarities and differences. Using a detailed representation of the two methods, the analysis showed that, even though the procedural basis was the same, differences appeared in normalization. The main difference was in the aggregation phase. This example shows that a high level of knowledge and expertise is required in order to distinguish between the two MCDM methods.

According to the literature, the sensitivity of MCDMA methods is an issue of high complexity and, consequently, it has a potentially unpredictable influence on the input data and variations in the results. The application and advanced analysis of MCDMA methods require experts with considerable knowledge and other resources, which are unavailable in some cases. These facts inspired the authors of this paper to analyze the sensitivity of MCDM methods a posteriori, i.e., to simulate the choice of weights by decision makers and to analyze the obtained rankings by utilizing statistical methods.

Quantitative methods based on mathematical algorithms applied to given sets of data, which represent some indicators and defined preferences quantified by weights, are the basis of multi-criteria decision analyses, and they are widely used in the domain of research in land consolidation project ranking and decision making [32,33,34,35].

Bearing in mind that quantitative methods for decision making are not purely quantitative, but rather “hybrid”, and that they consist of the exact mathematical procedure and weighted preferences of the decision maker, the crucial question is how sensitive they are to the preferences of the decision maker. In other words, “does the preference, explicated through the weights, of the decision maker result in significantly different option rankings?”

This article is structured in a manner that reflects the sequence of the research, which was designed in order to investigate the differences in the ranks of the different methods utilized. Firstly, the methods were defined through the choice of alternatives (different land consolidation projects) and criteria, as determined by the data availability. The criteria with available data were then reduced according to their importance and relevance for land consolidation, which was based on experience and consultations with the experts who were involved in the realization of the land consolidation projects. After that, a model that applied four methods was designed in detail, including the formulation of statistical hypotheses. Based on the results of the applied mathematical model and the statistical analysis, the discussion was developed and a conclusion was formulated.

2. Materials and Methods

The materials comprised the official data of cadastral municipalities (belonging to the municipality of Zrenjanin, Vojvodina region, Serbia), taken from the geodetic authority and statistical office of the Republic of Serbia (RZS). The considered cadastral municipalities were located approximately at the following coordinates: latitude, 45°23′58.95″ N; longitude, 20°25′10.45″ E (source—Google Earth). The cadastral municipalities investigated in this research are shown in Figure 1.

Based on the available dataset, four algorithms were chosen (AHP, TOPSIS, VIKOR, and SAW) and weights were randomly assigned; the results were obtained in the form of ranked alternatives for each method and for each set of weight vectors.

The calculation of the weights encompassed forming one hundred weights for each of the following cases: consistent, quasi-consistent, and inconsistent. This design was adopted to provide a large number of different possibilities and a relevant number of results in order to increase the reliability of the statistical analysis. The formulation of the statistical hypothesis was a logical consequence of the main aim of this research: to prove or reject the assumption of rank equality for different alternatives (land consolidation projects). This hypothesis was tested using Student’s t-test and by providing a standard procedure for checking the equality of the averages. The standard deviations were determined based on each alternative rank over one hundred attempts.

The general description of the multi-criteria decision analysis model can be explicated as follows:

$$M=M\left(D,C,A,W\right)$$

(1)

where:

-

$M$ is the multi-criteria decision-making analysis model;

-

$D$ is a vector of the data;

-

$C$ is a criterion;

-

$A$ is the algorithm of a certain method;

-

$W$ is the set of weights.

In a broader sense, the decision maker chose the criteria, algorithm, and weights, and only the data were fixed.

In this research, the four methods (AHP, TOPSIS, VIKOR, and SAW) of multi-criteria decision making (MCDM) were investigated using different sets of weights. Firstly, the set of weights for the AHP method was generated randomly (simulating the preferences of the decision maker), and after that, the resulting vector of weight (resulting eigenvalue vector from the AHP method) was utilized in the TOPSIS, VIKOR, and SAW methods to obtain the land consolidation project ranking. This process was performed a hundred times and the results were analyzed using statistical methods. The three approaches to generating weights in the AHP method were performed and named as follows: consistent, limited consistent (quasi-consistent), and freely formed (inconsistent). The inconsistent method for weight determination was achieved by being “consistent on a certain level of consistency”.

The main goal of this research was to investigate the sensitivity of the different quantitative methods (AHP, TOPSIS, VIKOR, and SAW) to the weights. Bearing in mind that these methods could be classified as hybrid methods, as they consisted of a mathematical algorithm and the subjective preferences explicated by the researcher (quantified by weights), it could be stated that every person has their own preference system. The statistical approach was based on the following assumptions: (1) every optimization method has the same weight (i.e., the resulting rankings are treated as being of the same importance), and (2) the differences between different ranks are the result of random errors [35]. Following this idea, it was possible to assume that every person would define their own matrix of weights in the AHP method. In this case, three possible types of weight-definition models were investigated:

-

Absolutely consistent;

-

Quasi-consistent;

-

Inconsistent.

The absolutely consistent model implies that only the first row in the weight matrix of the AHP model should be defined by the person, while the rest of the matrix is determined as follows:

$$W_{1,j}=Rand\left(x\right); j=1,n; W\in \left\{1, 2, 3, \mathrm{4,5}\right\}$$

(2)

$$W_{i,j}=\frac{w_{1,i}}{w_j}, i=2,n; j=1,n; W\in \left\{1, 2, 3, \mathrm{4,5}\right\}$$

(3)

where $Rand\left(x\right)$ is a random function that returns a positive integer value. This means that only the first row contains integers, while the other rows in matrix $W$ can be positive real numbers.

The quasi-consistent model implies that all the elements in weight matrix $W$ should be positive integers, but they are still determined based on the first row as follows:

$$W_{1,j}=Rand\left(x\right), j=1,n$$

(4)

$$W_{i,j}=\left[\frac{w_{1,i}}{w_j}+0.5\right], i=2,n; j=1,n; W\in \left\{1, 2, 3, 4, 5\right\}$$

(5)

where the brackets $\left[\dots \right]$ denote the integer part of the number.

The inconsistent model means that every element in the weight matrix is a random integer value, explicating the preference without taking into account the relationship defined earlier, and they are defined as follows:

$$W_{i,j}=Rand\left(x\right), i=1,n; j=1,n; W\in \left\{1, 2, 3, 4, 5\right\}$$

(6)

After obtaining the eigenvalue vector from the AHP method, it was treated as the vector of weights in the other utilized methods, and the obtained ranking vectors for each method were registered. This was repeated 100 times and the obtained rankings were analyzed using statistical methods. Firstly, for the set of 100 ranking results for each method (AHP, TOPSIS, VIKOR, and SAW), the average ranking and the root-mean-square errors of the rankings were determined for each cadastral municipality.

After determining the average rankings, the equality of the rankings was determined using Student’s t-test for each of the coupled methods [36]:

$$t=\frac{R_{k,j}-R_{l,j}}{\sqrt{\frac{m_{k,j}^2}{100}+\frac{m_{l,j}^2}{100}}}=10\frac{R_{k,j}-R_{l,j}}{\sqrt{m_{k,j}^2+m_{l,j}^2}}~t_{\mathrm{0.95,99}}=1.9842$$

(7)

where $k=1~4; l=i+1,4; \mathrm{a}\mathrm{n}\mathrm{d} j=1~15$.

In addition, for each method in the three cases (consistent–quasi-consistent, quasi-consistent–inconsistent, and consistent–inconsistent), the equality of the rankings was determined using Student’s t-test for each of the coupled methods [36]:

$$t=\frac{R_{k,j}-R_{l,j}}{\sqrt{\frac{m_{k,j}^2}{4}+\frac{m_{l,j}^2}{4}}}=2\frac{R_{k,j}-R_{l,j}}{\sqrt{m_{k,j}^2+m_{l,j}^2}}~t_{\mathrm{0.95,3}}=3.1824$$

(8)

where $k=1~4; l=1,4; \mathrm{a}\mathrm{n}\mathrm{d} j=1~15$.

The null and alternative hypotheses then read as follows:

H₀: 

The average rankings for two cadastral municipalities obtained using two different methods are equal.

H₁: 

The average rankings for two cadastral municipalities obtained using two methods are not equal.

3. Results

The results obtained using the methodology described above are provided in two sections: the first section shows the average rankings and their root-mean-square errors, and the second section shows the results of the statistical analysis.

The rankings were determined for 15 cadastral municipalities and according to 10 criteria (i.e., the model consisted of 15 alternatives and 10 criteria). The cadastral municipalities were as follows: Banatski Despotovac, Elemir, Ečka, Jankov Most, Klek, Lazarevo, Lukićevo, Mihajlovac, Orlovat, Slovački Aradac, Srpski Aradac, Srpski Elemir, Stajićevo, Taraš, and Tomaševac.

The criteria were as follows:

• (f1) The share of agricultural land in the total land consolidation area;
• (f2) The average surface area of the cadastral parcels for each land consolidation area;
• (f3) The average number of cadastral parcels per participant;
• (f4) The average surface area of each participant’s property;
• (f5) The percentage of farmers owning property larger than 5 ha;
• (f6) The share of state property out of the total agricultural land;
• (f7) The active agricultural population;
• (f8) The cost of the land consolidation project;
• (f9) The state of surveying;
• (f10) The area of state property land leased (%—percentage of total area).

The results of the rankings of the cadastral municipalities, obtained using each method and their corresponding root-mean-square errors (RMSEs), are given in the following tables:

-

Table 1. Average rankings and RMSEs for consistent approach.

Cadastral Municipality Ranking	AHP Ranking	AHP MRSE	TOPSIS Ranking	TOPSIS MRSE	VIKOR Ranking	VIKOR MRSE	SAW Ranking	SAW MRSE
Taraš	1.00	0.00	1.33	0.96	1.61	1.01	1.00	0.00
Orlovat	2.57	0.67	3.86	2.60	2.78	1.23	2.56	0.67
Tomaševac	2.95	1.17	4.41	1.91	3.35	1.45	3.12	1.42
Jankov Most	4.98	1.28	4.51	2.58	3.42	1.69	5.19	1.57
Mihajlovo	5.07	1.55	6.53	2.22	5.76	1.39	5.46	1.37
Lazarevo	6.93	2.04	6.7	2.32	6.25	2.01	6.37	2.34
Elemir	7.23	2.04	6.86	3.17	6.7	2.21	7.46	2.06
Slovački Aradac	7.83	1.52	7.55	3.11	7.69	1.66	7.69	1.81
Banatski Despotovac	8.29	1.41	7.84	3.37	8.67	1.51	8.28	1.75
Srpski Elemeir	8.41	2.12	9.64	3.28	11.13	2.67	8.52	2.28
Stajićevo	11.76	1.17	9.92	2.46	11.88	1.84	11.23	1.33
Srpski Aradac	12.56	1.23	12.07	2.43	12.01	1.65	12.39	1.64
Klek	12.77	1.46	12.19	1.82	12.09	1.6	12.86	1.09
Lukićevo	12.8	0.85	13.15	1.66	12.09	1.41	13.25	0.98
Ečka	14.85	0.46	13.44	2.03	14.57	0.87	14.62	0.86

contains the average rankings and RMSEs for the consistent approach.

-

Table 2. Average rankings and RMSEs for quasi-consistent approach.

Cadastral Municipality Ranking	AHP Ranking	AHP MRSE	TOPSIS Ranking	TOPSIS MRSE	VIKOR Ranking	VIKOR MRSE	SAW Ranking	SAW MRSE
Taraš	1.00	0.00	1.06	0.28	1.34	0.78	1.00	0.00
Orlovat	2.46	0.54	2.70	0.85	2.41	0.83	2.44	0.54
Tomaševac	2.56	0.50	3.00	1.04	2.74	0.82	2.59	0.51
Jankov Most	4.19	0.46	4.06	1.31	3.68	0.95	4.41	0.77
Mihajlovo	5.15	0.99	5.96	2.10	5.01	0.64	5.16	1.08
Lazarevo	7.32	1.39	6.06	1.35	7.03	1.60	6.98	1.65
Elemir	7.35	1.55	7.15	1.46	7.39	1.03	7.09	1.34
Slovački Aradac	8.17	1.64	8.18	1.77	7.44	1.21	7.78	1.49
Banatski Despotovac	8.33	1.37	8.80	1.58	8.43	1.26	8.21	1.29
Srpski Elemeir	8.47	1.14	9.26	2.61	11.18	1.43	9.8	1.04
Stajićevo	11.90	1.13	11.06	1.25	11.46	1.87	10.6	0.84
Srpski Aradac	12.08	1.01	12.26	1.38	11.65	1.27	13.05	0.86
Klek	12.56	0.82	12.78	1.87	12.52	1.57	13.15	1.10
Lukićevo	13.61	0.92	13.33	1.21	13.11	1.33	13.23	0.99
Ečka	14.85	0.41	14.34	0.93	14.61	0.74	14.51	0.95

contains the average rankings and RMSEs for the quasi-consistent approach.

-

Table 3. Average rankings and RMSEs for inconsistent approach.

Cadastral Municipality Ranking	AHP Ranking	AHP MRSE	TOPSIS Ranking	TOPSIS MRSE	VIKOR Ranking	VIKOR MRSE	SAW Ranking	SAW MRSE
Taraš	1.00	0.00	1.00	0.00	1.03	0.22	1.00	0.00
Orlovat	2.05	0.22	2.18	0.41	2.25	0.48	2.25	0.44
Tomaševac	2.98	0.28	2.89	0.47	2.75	0.50	2.75	0.44
Jankov Most	3.97	0.17	4.60	0.83	4.43	0.54	4.00	0.00
Mihajlovo	5.25	0.52	5.11	1.09	4.54	0.52	5.51	0.76
Lazarevo	6.11	0.86	5.35	0.70	6.71	0.76	6.53	1.25
Elemir	7.38	0.98	7.47	0.88	7.11	1.11	6.66	0.99
Slovački Aradac	8.06	0.81	7.93	0.70	8.14	1.49	8.05	1.14
Banatski Despotovac	8.35	1.05	8.66	0.78	8.74	1.12	8.48	0.88
Srpski Elemeir	9.86	0.49	10.66	1.30	10.01	0.82	9.81	0.61
Stajićevo	11.08	0.34	10.97	1.18	10.66	1.21	10.98	0.28
Srpski Aradac	12.11	0.51	12.03	0.98	11.68	0.68	12.22	0.50
Klek	12.82	0.44	12.20	0.75	13.49	0.67	13.44	0.83
Lukićevo	14.38	0.49	14.00	0.25	14.19	0.85	14.01	1.00
Ečka	14.60	0.53	14.95	0.26	14.27	0.76	14.31	0.77

contains the average rankings and RMSEs for the inconsistent approach.

The first and obvious result of the provided multi-criteria analysis was that, in all cases (consistent, quasi-consistent, and inconsistent methods of defining weights), the obtained rankings of the cadastral municipalities were identical, regardless of which method was applied. A deeper analysis revealed that the averaged ranking values based on one hundred cases of formed weights were normally distributed (this conclusion was based on the Shapiro–Wilk test [37]), and that all the data belonged to the same probability distribution function (this conclusion was based on the Kruskal–Wallis test [38]). The provided Shapiro–Wilk tests for each ranking obtained using all the combinations of approaches (consistent, quasi-consistent, and inconsistent) and for all the methods (AHP, TOPSIS, VIKOR, and SAW) resulted in the conclusion that there was no reason to reject the null hypothesis about the normal distribution of ranks. The additional analysis of the obtained rankings for groups of data (grouped by consistency and by the utilized method) with Kruskal–Wallis tests also resulted in the conclusion that there was no reason to reject the null hypothesis about the equality of the probability distribution functions.

4. Discussion

In this research, an approach was introduced that encompasses different ways of defining weights in order to rank cadastral municipalities for making decisions about priorities. The multi-criteria decision-making analysis (MCDMA) includes four models (AHP, TOPSIS, VIKOR, and SAW) and three methods of defining weights for the AHP method. The approaches for weight definition are named consistent, quasi-consistent, and inconsistent. A consistent method of weight definition means that only the initial relationships between favored variables are defined, and that any further relationships between them are strictly calculated and determined according to Formulas (2) and (3). The quasi-consistent approach of weight definition means that the initial relationship between the variables is defined and further weights are calculated, but rounded to the nearest integer number according to Formulas (4) and (5). The inconsistent approach means that all the relationships between the variables are determined randomly according to Formula (6). For each approach and each MCDCA model, one hundred simulated relationships were determined. The weights obtained from the AHP method were utilized in the TOPSIS, VIKOR, and SAW models. This approach resulted in twelve groups of cadastral municipality rankings. Each group was represented by the average ranking of one hundred different generated weights and standard deviations. Further analyses of the differences between rankings were based on the facts that the rankings of cadastral municipalities (land consolidation project priorities) were normally distributed and that the number of ranking variants was limited. This led to the utilization of Student’s t-test for testing the hypotheses about the equality of the rankings inside each group and for each approach (consistent, quasi-consistent, and inconsistent). The meanings of the hypotheses in

Table 4. The hypotheses about the equality of rankings for consistent approach.

Cadastral Municipality Ranking	AHP Ranking	Hypothesis	TOPSIS Ranking	Hypothesis	VIKOR Ranking	Hypothesis	SAW Ranking	Hypothesis
Taraš	1.00		1.33		1.61		1.00
Orlovat	2.57	H1	3.86	H1	2.78	H1	2.56	H1
Tomaševac	2.95	H1	4.41	H0	3.35	H0	3.12	H1
Jankov Most	4.98	H1	4.51	H0	3.42	H0	5.19	H1
Mihajlovo	5.07	H0	6.53	Ha	5.76	Ha	5.46	H0
Lazarevo	6.93	H1	6.7	H0	6.25	H0	6.37	H1
Elemir	7.23	H0	6.86	H0	6.7	H0	7.46	Ha
Slovački Aradac	7.83	H1	7.55	H0	7.69	H0	7.69	H0
Banatski Despotovac	8.29	H1	7.84	H0	8.67	H0	8.28	H1
Srpski Elemeir	8.41	H0	9.64	Ha	11.13	Ha	8.52	H0
Stajićevo	11.76	H1	9.92	H0	11.88	H0	11.23	H1
Srpski Aradac	12.56	H1	12.07	H1	12.01	H1	12.39	H1
Klek	12.77	H0	12.19	H0	12.09	H0	12.86	H1
Lukićevo	12.80	H0	13.15	H1	12.09	H1	13.25	H1
Ečka	14.85	Ha	13.44	H0	14.57	H0	14.62	H1

,

Table 5. The hypotheses about the equality of rankings for quasi-consistent approach.

Cadastral Municipality Ranking	AHP Ranking	Hypothesis	TOPSIS Ranking	Hypothesis	VIKOR Ranking	Hypothesis	SAW Ranking	Hypothesis
Taraš	1.00		1.06		1.34		1.00
Orlovat	2.46	H1	2.70	H1	2.41	H1	2.44	H1
Tomaševac	2.56	H0	3.00	H1	2.74	H0	2.59	H1
Jankov Most	4.19	H1	4.06	H1	3.68	H1	4.41	H1
Mihajlovo	5.15	H1	5.96	H1	5.01	H1	5.16	H1
Lazarevo	7.32	H1	6.06	H0	7.03	H1	6.98	H1
Elemir	7.35	H0	7.15	H1	7.39	H0	7.09	H0
Slovački Aradac	8.17	H1	8.18	H1	7.44	H1	7.78	H1
Banatski Despotovac	8.33	H0	8.80	H1	8.43	H0	8.21	H1
Srpski Elemeir	8.47	H0	9.26	H0	11.18	H0	9.8	H1
Stajićevo	11.90	H1	11.06	H1	11.46	H1	10.6	H1
Srpski Aradac	12.08	H0	12.26	H1	11.65	H0	13.05	H1
Klek	12.56	H1	12.78	H1	12.52	H1	13.15	H0
Lukićevo	13.61	H1	13.33	H1	13.11	H1	13.23	H0
Ečka	14.85	H1	14.34	H1	14.61	H1	14.51	H1

,

Table 6. Average rankings and RMSEs for inconsistent approach.

Cadastral Municipality Ranking	AHP Ranking	Hypothesis	TOPSIS Ranking	Hypothesis	VIKOR Ranking	Hypothesis	SAW Ranking	Hypothesis
Taraš	1.00		1.00		1.03		1.00
Orlovat	2.05	H1	2.18	H1	2.25	H1	2.25	H1
Tomaševac	2.98	H1	2.89	H1	2.75	H1	2.75	H1
Jankov Most	3.97	H1	4.60	H1	4.43	H1	4.00	H1
Mihajlovo	5.25	H1	5.11	H1	4.54	H1	5.51	H1
Lazarevo	6.11	H1	5.35	H0	6.71	H1	6.53	H1
Elemir	7.38	H1	7.47	H1	7.11	H1	6.66	H0
Slovački Aradac	8.06	H1	7.93	H1	8.14	H1	8.05	H1
Banatski Despotovac	8.35	H1	8.66	H1	8.74	H1	8.48	H1
Srpski Elemeir	9.86	H1	10.66	H1	10.01	H1	9.81	H1
Stajićevo	11.08	H1	10.97	H0	10.66	H1	10.98	H1
Srpski Aradac	12.11	H1	12.03	H1	11.68	H1	12.22	H1
Klek	12.82	H1	12.20	H0	13.49	H1	13.44	H1
Lukićevo	14.38	H1	14.00	H1	14.19	H1	14.01	H1
Ečka	14.60	H1	14.95	H1	14.27	H1	14.31	H1

and

Table 7. Average rankings and RMSEs for inconsistent approach.

Cadastral Municipality Ranking	AHP Ranking	Hypothesis	TOPSIS Ranking	Hypothesis	VIKOR Ranking	Hypothesis	SAW Ranking	Hypothesis
Taraš	1.00		1.13		1.11		1.00
Orlovat	2.36	H1	2.91	H1	2.79	H1	2.42	H1
Tomaševac	2.83	H0	3.43	H0	3.32	H0	2.82	H0
Jankov Most	4.38	H1	4.39	H0	4.22	H0	4.53	H1
Mihajlovo	5.16	H0	5.87	H1	5.64	H1	5.38	H0
Lazarevo	6.79	H1	6.04	H0	6.71	H0	6.63	H1
Elemir	7.32	H0	7.16	H0	7.20	H0	7.07	H0
Slovački Aradac	8.02	H1	7.89	H1	7.93	H1	7.84	H1
Banatski Despotovac	8.32	H1	8.43	H0	8.17	H0	8.32	H1
Srpski Elemeir	8.91	H0	9.85	H1	9.32	H0	9.38	H0
Stajićevo	11.58	H1	10.65	H0	10.97	H0	10.94	H1
Srpski Aradac	12.25	H0	12.12	H1	12.09	H0	12.55	H1
Klek	12.72	H0	12.39	H0	12.52	H0	13.15	H0
Lukićevo	13.60	H0	13.49	H1	13.71	H1	13.50	H0
Ečka	14.77	H0	14.24	H0	14.30	H0	14.48	H1

are as follows: If the entry reads “H₀”, this means that the alternative hypothesis was rejected, i.e., that the neighborhood ranks were equal in a statistical sense; if the entry reads “Ha”, this means that the neighborhood ranks were different. The results of the hypothesis testing about the equality of rankings for the consistent approach are given in Table 4, the results for the quasi-consistent approach are given in Table 5, and the results for the inconsistent approach are given in Table 6.

The averaged rankings sorted by the MCDMA model were utilized for all the approaches, and the results for the hypotheses about the equality of rankings, obtained using statistical testing and Formula (8), are given in Table 7.

The data given in Table 7 show that 32 of the 56 hypotheses about the equality between the neighborhood rankings of cadastral municipalities were not rejected. This fact led to the conclusion that, in a statistical sense, some rankings were equal regardless of whether they were obtained with the consistent, quasi-consistent, or inconsistent approach. Furthermore, this result may open the possibility of a new MCDMA approach for the ranking of cadastral municipalities when the prioritization of land consolidation projects is required. Instead of using the term “cadastral municipality ranking” or “land consolidation project ranking”, the more appropriate terminology would be “the valuation of cadastral municipalities” or “the valuation of land consolidation projects” in processes of project prioritization. This result could also lead to greater freedom for stakeholders, who could formulate the weights for each variable in their own way. Furthermore, bearing in mind that no one approach resulted in different ranks for the cadastral municipalities, the idea that “every model is consistent on a certain level of consistency” can be accepted regarding the typically used term in the AHP method, where the model is considered inconsistent if CI > 0.1.

Nonetheless, we must highlight that this research encompassed only a few MCDM methods and was tested only on one set of alternatives, which limits the conclusions about the model’s applicability to other cases as well as about the applied algorithms. Furthermore, the sensitivity of the applied model was limited only to the weights; in future, it should be extended to the other parts of the model given by Formula (1). In particular, problems could appear if the applied model is sensitive to the accuracy of the data that define the criteria. On the other hand, the flexibility of the applied model and the ease of its application could save time when making a decision is urgent. Of course, as can be stated for other MCDM methods, caution is necessary for a proper decision to be made. Further investigations should encompass the uncertainty of the input data because they may not be accurate, and should investigate algorithms based on different approaches. In addition, the basis of land consolidation projects as well as the criteria could be changed if they are of significant importance for a certain set of land consolidation projects.

5. Conclusions

In a hybrid way, this research combined the deterministic and stochastic parts of a model. The deterministic part of the model was defined using real data for each of 15 cadastral municipalities, and the stochastic part was formed as a random variable representing the preferences of virtual stakeholders. In this study, a hundred stakeholders were modeled and classified into three groups named “consistent”, “quasi-consistent”, and “inconsistent”. The main conclusion of this research is that, when using different MCDMA methods, the rankings will follow a normal distribution, which enables a statistical analysis of the obtained rankings. This conclusion is valid for all three approaches, which provides the freedom for every stakeholder to express their own preferences in land consolidation project valuation, i.e., in defining their own matrix of weights regarding the AHP method. In this research, the rankings of cadastral municipalities were the same for each MCDMA method and each approach, but they may differ in other cases, and a series analysis is required in order to utilize the method proposed in this paper for any other case. With that said, despite having limitations, the findings of this research could improve the process of decision making when prioritizing land consolidation projects under conditions of limited budgeting. When the rank of two or more land consolidation projects is statistically equal, it is better to conduct an additional analysis rather than to strictly follow the ranks obtained using MCDM methods.