On the Statistical Discrepancy and Affinity of Priority Vector Heuristics in Pairwise-Comparison-Based Methods

There are numerous priority deriving methods (PDMs) for pairwise-comparison-based (PCB) problems. They are often examined within the Analytic Hierarchy Process (AHP), which applies the Principal Right Eigenvalue Method (PREV) in the process of prioritizing alternatives. It is known that when decision makers (DMs) are consistent with their preferences when making evaluations concerning various decision options, all available PDMs result in the same priority vector (PV). However, when the evaluations of DMs are inconsistent and their preferences concerning alternative solutions to a particular problem are not transitive (cardinally), the outcomes are often different. This research study examines selected PDMs in relation to their ranking credibility, which is assessed by relevant statistical measures. These measures determine the approximation quality of the selected PDMs. The examined estimates refer to the inconsistency of various Pairwise Comparison Matrices (PCMs)—i.e., W = (wij), wij > 0, where i, j = 1,…, n—which are obtained during the pairwise comparison simulation process examined with the application of Wolfram’s Mathematica Software. Thus, theoretical considerations are accompanied by Monte Carlo simulations that apply various scenarios for the PCM perturbation process and are designed for hypothetical three-level AHP frameworks. The examination results show the similarities and discrepancies among the examined PDMs from the perspective of their quality, which enriches the state of knowledge about the examined PCB prioritization methodology and provides further prospective opportunities.


Introduction
The method of creating a ranking based on pairwise comparisons of alternatives was already known in the Middle Ages. The first work on this subject was probably that by Ramon Lull [1], who described election processes based on comparisons of mutual alternatives. Over time, other research on the pairwise comparisons method appeared; e.g., studies on electoral systems, such as the Condorcet method and the Copeland method, and many others on social choice and welfare systems [2]. In time, alternatives began to be compared quantitatively, which was initially connected with the need to compare psychophysical stimuli [3,4]. This path was later developed [5] and used in various forms for different objectives, including economics [6], consumer research, psychometrics, health care and others. Thanks to Saaty and his seminal paper [7] in which he defined the Analytic Hierarchy Process (AHP), comparing alternatives in a pairwise mode began to be considered basically as a multi-criteria decision-making method. The undisputable success of the AHP is probably due to the fact that Saaty proposed a complete solution including a ranking calculation algorithm, an inconsistency index as a method of determining data quality and a hierarchical model allowing decision makers (DMs) to handle multiple criteria [8][9][10][11]. Overtime, numerous studies have presented scientific evidence of the fundamental flows of the AHP; see, e.g., [12][13][14][15].
However, different research studies concerning the pairwise comparison method have resulted in many priority deriving methods (PDMs) which, for brevity, are not discussed various scenarios were simulated in relation to different sources of PCM inconsistency. Fundamentally, PCM inconsistency commonly results from errors caused by the nature of human judgments and errors due to the technical realization of the pairwise comparison procedure; i.e., rounding errors and errors resulting from the forced reciprocity requirement commonly imposed in PCB ranking problems. All the above errors can be simulated, but the nature of human judgments can be represented only as the realization of some random process in accordance with the assumed probability distribution of the perturbation factor; e.g., uniform, gamma, truncated normal and log-normal. As this is only a stochastic process generated by a computer, it constitutes a certain limitation of the presented research.
The research paper is structured as follows: firstly, preliminaries about pairwisecomparison-based problems are presented (Section 2); then, the examination methodology is introduced and exemplified (Section 3) in two subsections: Section 3.1 is devoted to the concept design with preliminary results, and Section 3.2 introduces the target examination scenario and presents further examples of examination results. The results of the complete examination and a discussion are presented in Section 4, leading to our research conclusions (Section 5); finally, we end the paper with final remarks.

Preliminaries about PCB Ranking Problems
The conventional PDM in the AHP is founded on the mathematical structure of consistent PCMs and the related capability of PREV to produce actual or approximate weights.
Oscar Perron proved that if W = (w ij ), w ij > 0, where i, j = 1, . . . , n, then W has a simple positive eigenvalue λ max called the principal eigenvalue of W, and λ max |λ k | for the remaining eigenvalues of W. Moreover, the principal right eigenvector w = [w 1 , . . . , w n ] T , which is a solution of Ww = λ max w, has w i > 0, i = 1, . . . , n. If the relative weights of a set of activities are known, they can be expressed as PCMs.
If we know W(w) but do not know w, we can use Perron's theorem to solve this problem for w. The solution leads to n unique values for lambda, with a bounded vector w for each of the n values. The PCM matrix in the AHP reflects the relative weights of the actions considered (criteria, scenarios, players, alternatives, etc.), so the matrix W(w) has a particular shape. Each subsequent row of this matrix is a constant multiple of the first row. In this case, the matrix W(w) has only one non-zero eigenvalue, and since the sum of the eigenvalues of a positive matrix is equal to the sum of its diagonal elements, the single non-zero eigenvalue in this case is equal to the size of the matrix and can be denoted as lambda(max) = n. The norm of the vector w can be written as w = e T w, where e = [1, 1,..., 1] T and w can be normalized by dividing by its norm. For the sake of clarity, w is further specified in its normalized form.

D[1]:
If the matrix W(w) elements satisfy the prerequisite w ij = 1/w ji for all i, j = 1, . . . , n, then the matrix W(w) is called reciprocal. D [2]: If the following assumptions are true: (a) if for any i = 1, . . . , n, an element w ij is not less than an element w ik , then w ij ≥ w ik for i = 1, . . . , n, and (b) if for any i = 1, . . . , n, an element w ji is not less than an element w ki , then w ji ≥ w ki for i = 1, . . . , n, and the matrix W(w) is called ordinal transitive, D [3]: If the elements of a matrix W(w) satisfy the condition w ik w kj = w ij for all i, j, k = 1, . . . , n, and the matrix is reciprocal, then it is called consistent or cardinal transitive.
Certainly, when the AHP is utilized, a W(w) which would reflect the true weights given by the actual PV is unknown.
Since the human mind is not an accurate measuring device, it does not give accurate results in tasks such as the following: "compare-using a given ratio scale-your preferences for alternative 1 and alternative 2". Thus, W(w) is not known, but only its estimate X(x) containing intuitive judgments that are more or less close to W(w) depending on the DM's individual taste, specific knowledge, experience, ability and even momentary mood or frame of mind. In such cases, the consistency property does not apply, and the conventional notion of the PDM in the AHP is no longer applicable. However, it has been shown that for any PCM, small perturbations in the items entail similar perturbations in the eigenvalues, so Perron's theorem can be used to estimate the true PV. Then, instead of matrix equation (1), the solution of matrix equation (2) gives us w as the right principal eigenvector (PREV) associated with λ max .
In practice, the solution of PREV is obtained by raising the matrix X(x) to a sufficiently large power, then summing the rows of X(x) and normalizing the resulting vector to obtain a vector w, which can be expressed by the following formula: where e = [1, 1, . . . , 1] T . Noticeably, the relation between elements of X(x) and W(w) can be expressed in the form of the following formula: x ij = e ij w ij (4) where e ij is a disturbance coefficient oscillating close to unity; e.g., e ij ∈[0.5, 1.5]. It is important to emphasize that in the statistical approach, e ij reflects the realization of a random variable with a provided probability distribution (PD) that can be modeled and that reflects imperfect human pairwise comparisons. In the literature, the following types of PDs are often considered for different implementation purposes: gamma, log-normal, truncated-normal or uniform [36,47]. However, in addition to these most popular types of PDs, one can also find applications of Cauchy, Laplace, triangular or beta PDs [24] and Fisher-Snedecor PDs, which were recently introduced by Kazibudzki [30]. Usually, the maximal spread for e ij ∈ [0.01, 1.99], which may be perceived as strange as this interval is highly asymmetric. However, it is perfectly reasonable to have asymmetric intervals for perturbation factors as DM judgments are reflected with the application of a particular numerical scale whose numbers are usually not higher than nine (Saaty's scale). So, multiplying five by more than two, for example, is simply pointless, as the result must still be rounded to the nearest value of the scale; e.g., Saaty's scale, whose maximal value equals nine. On the other hand, multiplying 9 by 0.01 gives 0.09, which can be naturally rounded to 0.1(1) and may reflect a reversal phenomenon of preferences, for example. However, the symmetric interval for a perturbation factor with a seminal proposed PD and its application outcome is also presented in this research in Section 4. In general, the discrepancy of the perturbed PCM reflects the results of errors caused by the nature of human judgment and errors due to the technical implementation of the pairwise matching procedure. The latter are mainly due to rounding errors and errors resulting from forced reciprocity requirements. Rounding errors are related to the numerical ratio scale, the values of which must be used by future DMs to express their judgments in a certain way [25,[62][63][64]. In common AHP applications, the Saaty numerical scale, consisting of integers from 1 to 9 and their inverse values, is by far the most popular. However, other scales are also known [25,[65][66][67][68][69][70], such as the geometric scale, for which the linguistic variables of the Saaty scale have different numerical values; the most common, and thus the approach used in this study, is 2 n/2 , where n includes integers from minus 8 to 8, but an arbitrarily defined numerical scale including integers from 1 to n and their inverse values is also possible.
It has been argued that the primary AHP-PDM-i.e., the PREV method-is necessary and sufficient for unambiguous ranking with the ratio scale inherent in inconsistent pairwise comparisons [42,50,54,55,86,87]. However, this approach has also been heavily criticized; see, for example, [13,30,63,80,[88][89][90][91][92]. Therefore, there are optional PDMs that differ from the basic PDM concept. Many of these are based on optimization and finding the vector w as a solution to the minimization problem given by the formula min D(X(x), W(w)) (5) with some accompanying constraints such as positive coefficients and the normalization condition. Since the distance function D measures the interval between the matrices X(x) and W(w), different ways of defining this lead to different PDM estimation results. Chu et al. in [19] describe and compare 18 PDMs, although some authors suggest that only 15 are distinctive. Undoubtedly, several other PDMs have appeared in the literature since Chu et al. [19] published their study; see, for example, [29,[32][33][34][93][94][95]. Obviously, if the PCM is consistent, then all known PDMs match, although they do not guarantee that the resulting PV is error-free; see, e.g., [12,45,70,96]. However, in real situations, as noted earlier, human judgments inevitably lead to inconsistent PDMs, since inconsistency is a natural consequence of the dynamics of the human mind as well as a consequence of query methodology, incorrect inputs of judgment values and scaling procedures (i.e., rounding errors). In this research study, apart from PREV, three optional PDMs are examined. They are defined by the formulae presented in Table 1.

Concept Design with Preliminary Results
The first step in PCB problems using the AHP is to create a hierarchy by breaking the specific problem into its major components. The basic AHP scenario includes a goal (an expression of the overall objective), criteria (factors to be considered in arriving at the final choice) and alternatives (feasible alternatives to achieve the final objective). Thus, the most basic AHP decision model consists of an objective-criteria-alternatives sequence. Therefore, this study adopts a basic three-level hierarchy including three criteria and three alternatives within each criterion.
The intent of this research is to examine the performance of PREV against the background of the performance of other selected PDMs available for PCB problems elaborated within the AHP. In order to achieve this objective, Monte Carlo simulations (MCS) were applied, but not as commonly performed; i.e., dedicated to a single PCM. This research involves an MCS scenario that encompasses the entire goal-criteria-alternatives sequence of the AHP, which is supposed to reflect the hypothetical PCB decisional problem (see the examples presented hereafter (Examples 1A and 1B)).
Firstly, the examination framework is presented in its simplified version as a methodological example. Thus, only technical distortions are used resulting from rounding errors during the application of Saaty's scale and standard requirements of the AHP; i.e., forced reciprocity is demonstrated in the following hypothetical AHP model with three levels (a goal, four criteria and four alternatives). This model assumes that relative ratios of some physical attributes of certain objects are predetermined, and thus H G PV C , H C1 C2 PV A and H C3 C4 PV A are known. On the basis of the provided PV elements, the respective PCMs are reconstructed as shown in Equation (1).

Example 1A:
H G PV C , and its related PCM denoting the weights quotients of H G PV C , reflecting the pairwise comparison results of criteria with respect to the goal: After standard AHP synthesis, the hypothetical total PV (HTPV) is obtained and given as HTPV = [0.25, 0.21, 0.23, 0.31] T . Next, following the simplified version of the MCS scenario in this example, each PCM in the presented framework is perturbed. For simplicity of illustration, only two kinds of prospective PCM distortions are applied: rounding errors (each element of the particular PCM is rounded to Saaty's scale) and reciprocity imposition errors (the PCM is transformed to be reciprocal in the way that only elements above its diagonal are taken into consideration, and elements below its diagonal are replaced by the inverses of their counterparts from above its diagonal). Next, on the basis of each PCM being perturbed in the above way, respective partial PVs (PPV PREV ) are obtained with the application of the selected PDM-i.e., PREV. Finally, the total computed PV (TCPV PREV ) for the exemplary model of the AHP is obtained (see Example 1B).

Example 1B:
PCM with criteria weights designated with respect to the goal and PPV GOAL PREV , computed on the basis of this PCM with the application of PREV: PCM with weights of alternatives designated with respect to criteria C1-C2 and PPV C1C2 PREV , computed on the basis of this PCM with the application of PREV:  Table 2.
For the above illustrative values of HTPV and TCPV PREV , the presented measures are as follows: SRCC = 0.2, PCC = 0.8142, MAD = 0.023325. Noticeably, the comparison of the approximation quality of any PDM available for PCB ranking problems is possible in this way. Thus, it is also possible to examine the selected PDM for the AHP. The MCS designed for this purpose-i.e., processing the algorithm which exactly emulated the above explained steps 10,000 times-provided the scores shown in Tables 3 and 4.     Tables 3 and 4, it should be noticed that from the perspective of rank, the preservation capability of PREV is slightly lower than that of LSDM. It also can be noticed that LSDM performs slightly worse than PREV from the perspective of MPCC and MAAD values. The performance of other examined PDMs is slightly worse in both scenarios and from the perspective of all performance measures taken into consideration during this study.

Research Target Scenario Analysis with Further Results
Hereafter, the examination scenario is exemplified in its target version. Thus, not only are technical distortions resulting from rounding errors during the application of Saaty's scale and standard requirements of the AHP will be used-i.e., forced reciprocity is demonstrated-but also human judgment errors are considered in the hypothetical AHP model with three levels (goal, four criteria and four alternatives) earlier depicted as in Example 1A. Hence, it is still assumed that relative ratios of some physical attributes of certain objects are predetermined; thus, H G PV C , H C1 C2 PV A and H C3 C4 PV A are known, and their respective PCMs are computed as in Equation (1). For the reader's convenience, the model is duplicated herein and renamed as Example 2A for reference hereafter.

Example 2A:
For the assumed H G PV C and its related PCMs representing the weights quotients of H G PV C , reflecting the pairwise comparison results of criteria with respect to the goal, see Equation (6). For the assumed H C1 C2 PV A and its related PCMs denoting the weights quotients of H C1 C2 PV A , reflecting the pairwise comparison results of alternatives with respect to criteria C1-C2, see Equation (7). For the assumed H C3 C4 PV A and its related PCM denoting the weights quotients of H C3 C4 PV A , reflecting the pairwise comparison results of alternatives with respect to criteria C3-C4, see Equation (8).
Next, following the target version of the MCS scenario in this example, each PCM in the presented framework of Example 2A is perturbed. This time, three kinds of prospective PCM distortions are applied: human judgment errors reflected by the applied perturbation factor e ij = 0.5, rounding errors (each element of the particular PCM is rounded to Saaty's scale) and reciprocity imposition errors (the PCM is transformed to be reciprocal in a way that means only elements from above its diagonal are taken into consideration, and elements below its diagonal are replaced by inverses of their counterparts from above its diagonal). On the basis of each PCM being perturbed in the above way, respective partial PVs (PPV PREV ) are obtained with application of the selected PDM; i.e., PREV. Finally, the total computed PV (TCPV PREV ) for the exemplary model of the AHP is obtained; see Examples 2B-2D.

Example 2B:
Pairwise comparison results of criteria with respect to the goal after the implementation of the perturbation factor e ij = 0.5: Pairwise comparison results of alternatives with respect to criteria C1-C2 after the implementation of the perturbation factor e ij = 0.5: Pairwise comparison results of alternatives with respect to criteria C3-C4 after the implementation of the perturbation factor e ij = 0.5: Example 2C: Pairwise comparison results of criteria with respect to the goal after the implementation of the perturbation factor e ij = 0.5 and rounding errors (each element of the particular PCM is rounded to Saaty's scale): Pairwise comparison results of alternatives with respect to criteria C1-C2 after the implementation of the perturbation factor e ij = 0.5 and rounding errors (each element of the particular PCM is rounded to Saaty's scale): Pairwise comparison results of alternatives with respect to criteria C3-C4 after the implementation of the perturbation factor e ij = 0.5 and rounding errors (each element of the particular PCM is rounded to Saaty's scale): Example 2D: Pairwise comparison results of criteria with respect to the goal after the implementation of the perturbation factor e ij = 0.5, rounding errors (each element of the particular PCM is rounded to Saaty's scale) and forced reciprocity errors (the PCM is transformed to be reciprocal in such a way that only elements from above its diagonal are taken into consideration, and elements below its diagonal are replaced by inverses of their counterparts from above its diagonal), with PPV GOAL PREV computed on the basis of the obtained PCM with the application of PREV: Pairwise comparison results of alternatives with respect to criteria C1-C2 after the implementation of the perturbation factor e ij = 0.5, rounding errors (each element of the particular PCM is rounded to Saaty's scale) and forced reciprocity errors (the PCM is transformed to be reciprocal in such a way that only elements from above its diagonal are taken into consideration, and elements below its diagonal are replaced by inverses of their counterparts from above its diagonal), with PPV C1C2 PREV computed on the basis of the obtained PCM with the application of PREV: Pairwise comparison results of alternatives with respect to criteria C3-C4 after the implementation of the perturbation factor e ij = 0.5, rounding errors (each element of the particular PCM is rounded to Saaty's scale) and forced reciprocity errors (the PCM is transformed to be reciprocal in such a way that only elements from above its diagonal are taken into consideration, and elements below its diagonal are replaced by inverses of their counterparts from above its diagonal), with PPV C3C4 PREV computed on the basis of the obtained PCM with the application of PREV:

Results of Complete Examination with Discussion
Noticeably, the comparison of the approximation quality of any PDM available for PCB ranking problems is possible as presented in Section 3. It is thus possible to examine a few selected PDMs for the AHP. Thus, the MCS designed for this purpose-i.e., processing the algorithm emulating steps from Examples 2A-2D 25,000 times (250 distinctive AHP frameworks perturbed 100 times each)-provided the correlation scores presented in Tables A1-A4. However, in Tables 5-8, discrepancies among correlation scores obtained by the examined PDMs are presented in relation to the selected referential values, which in these cases constitute the correlation scores obtained by PREV.      The tables should be read from left to right and row by row from the top to the bottom. In every first three columns on the left side of the tables, simulation parameters taken into consideration are provided; i.e., the applied preference scale, the interval for the perturbation factor, and sets for alternatives and criteria applied during MCS. Then, differences between performance statistics are given for each simulation scenario.
It should be emphasized here that many PDMs have been proposed thus far, and their effectiveness has been evaluated by various means. Various research and different measures of the effectiveness of PDMs lead to different conclusions [97,98]. For example, Choo et al. [99] in their research recommend LLSM as the best PDM with a simple formula for computing PVs, equipped with many desirable properties discovered by Fichtner [100], and the method is very popular as the best alternative for PREV. On the other hand, some support for SNCS also exists-e.g., [36,47,101]-and LSDM is also considered as an efficient PDM; see, e.g., [30]. Basically, it is agreed that many research results, including those of this work, do not provide support for the recommendation by Saaty and Vargas [42] and Saaty and Hu [55] that PREV reputedly is the only PDM which should be used when pairwise comparisons are not entirely consistent. It is also agreed, as stated by Golany and Kress [102], that the selection of a PDM for PCB problems should be dictated by the desired measure of the PDM's effectiveness, as different error measures support mathematically different PDMs. Hence, as suggested by Bajwa et al. [101], the defining question is not which PDM is superior, but which application results are expected and/or what level of effectiveness or performance criteria are more valued. It is also believed that this research supports the conclusion stated by Saaty and Hu [55] that there is a difference between metric topology and order topology, where in the former the central concern is closeness and in the latter both closeness and order preservation features are equally important. It can be agreed after all these years of research that none of the examined PDMs have been found to be universally superior to all others in all aspects. However, to the best of our knowledge, for the first time, a statistical foundation has been created to evaluate scenarios when PDMs coincide and when their discrepancies are statistically significant. Hence, the possibility was created for a DM to assess the risk of accepting an ineffective PDM or rejecting an effective PDM-the standard problem known to every statistician and very important to each DM during the statistical evaluation of decisional options; i.e., statistical alternative hypothesis testing.
For this research, four distinctive PDM have been selected on the basis of different criteria. PREV is studied because it was conceived with the AHP. LLSM is considered because it has a simplified form and is usually promoted as the best alternative for PREV. SNCS is taken into consideration here for its simplicity and good effectiveness as shown in, e.g., [19,47]. LSDM is examined because it combines spectral theory with an optimizationbased approach to PCB problems.
As stated earlier, all the above PDMs have been more or less intensively studied and have shown their effectiveness, efficiency and desired analytical properties [34,103]. They have been evaluated from the perspective of various measures of effectiveness; e.g., Mean Square Error (MSE), Mean Absolute Deviation (MAD), Mean Central Conformity (MCC), Mean Rank Violation (MRV) (see, e.g., [19,36,47,102]), the Coefficient of Multiple Determination (CMD = R 2 ), which is widely applied in regression analysis (see, e.g., [104]), the Garuti Compatibility Index (GCI) and the Saaty Compatibility Index (SCI) (see, e.g., [58,105,106]). However, in this research, focus was given to statistical measures of examined phenomena and their statistical significance; thus, emphasis was placed on the introduced PDM approximation quality measures; i.e., mostly MSRC, but also MPCC and MAAD. Thus, the discrepancies in the correlation scores presented in Tables 5-8 are analyzed from the perspective of the PDM rank preservation capability designated by MSRC and the general correlation significance determined by MPCC.
As can be seen, all PDMs perform steadily under the four MCS scenarios presented in Tables 5-8. It should be noted that PREV does not always outperform the other PDMs in this study; in many cases, it is actually the other way round. The selected PDMs for the examination perform better from the perspective of the approximation quality represented by MSRC. In relation to this phenomenon, it occurs that LSDM dominates over PREV most often in comparison with the other PDMs and from the perspective of all applied MCS scenarios (see the numerical lower subscript of the particular PDM for information on how many times the indexed PDM prevails PREV). This is an important piece of information as the approximation quality of the DM preference intensities of any PDM seems a very crucial issue in PCB problems.
Fortunately, the information provided in Tables 5-8 can be analyzed from the statistical perspective because the significance of the difference between any two correlation coefficients (CC), denoted as CC [1] and CC [2], can be tested using "t" statistics defined by the following formula: where R is the difference between particular CC. These statistics have a distribution of t-student with n-2 degrees of freedom df, where n equals the size of the sample. Thus, the following hypothesizes can be tested: Hence, the following conclusions can be drawn from data provided in Tables 5-8. If the performance of a particular PDM differs from the performance of PREV by less than 0.00160 (t = 0.252972417), then it can be assumed with an 80% confidence level (t = 0.253) that its performance discrepancy in relation to the performance of PREV is negligible. For an 85% confidence level (t = 0.1895), this discrepancy should be smaller than 0.00118 (t = 0.186567049), and for a 90% confidence level (t = 0.126), it should be smaller than 0.00076 (t = 0.120161779).
On the other hand, if the performance of the particular PDM differs from the performance of PREV by more than 0.00815 (t = 1.288619398), then it can be assumed with an 80% confidence level (t = 1.282) that its performance discrepancy in relation to the performance of PREV is significant, and this discrepancy should not be neglected. For an 85% confidence level (t = 1.452) this discrepancy should be greater than 0.00920 (t = 1.454651099), and for a 90% confidence level (t = 1.645), it should be greater than 0.01050 (t = 1.660220885).
To be able to more completely examine this issue, the MAAD scores for four studied scenarios are presented in Table A5, and discrepancies among the MAAD scores of three PDMs in relation to PREV for four studied scenarios are presented in Table 9. Table 9. Performance discrepancies of three arbitrarily selected PDMs in relation to PREV for 25,000 cases of different uniformly drawn and variously perturbed AHP frameworks (%) .
It can be noticed that all examined PDMs perform quite similarly from the perspective of their approximation quality evaluated by their MAAD. Nevertheless, considering MAAD as the performance criterion, LSDM outperforms PREV 21 times, LLSM is better than PREV 22 times and SNCS outperforms PREV 9 times. In conclusion, as PREV is quite frequently criticized in the literature, this outcome should not be surprising.
Taking into account the above examination results, it was decided to analyze one more scenario with MCS. Generally, the algorithm applied for MCS this time is an expanded version of the approach previously applied and presented in earlier examples provided in this research in Section 4. The algorithm can be specified as follows: Step 1. Generate a random PV;-i.e., k = [k 1 ,..., k n ] T of size [n x 1]-for the criteria and the associated original unbiased PCM(k) = K(k).
Step 2. Randomly generate exactly n PVs-i.e., a n = [a n,1 ,..., a n,m ] of size [m x 1]-for the alternatives under each criterion and the associated original unbiased PCMs(a) = A n (a).
Step 3. Compute the joint priority vector w of size [m x 1] by the following procedure: w x = k 1 a 1, x + k 2 a 2, x +...+ k n a n, x Step 4. Randomly select a number e from the given interval [α,β] based on the given PD.
Step 5. Use step 5A and step 5B separately: Step 5A is a case of applying forced reciprocity to the PCM: Replace all elements a ij for i < j of all A n (a) with ea ij and all elements k ij for i < j of K(k) with ek ij .
Step 5B is the case of the acceptance of nonreciprocal PCM: Replace all elements a ij for i = j of all A n (a) with ea ij and all elements k ij for i = j of K(k) with ek ij .
Step 6. Use steps 6A and 6B separately: Step 6A-if Step 5A is satisfied, Round all values of elements a ij for i < j of all A n (a) and all values of elements k ij for i < j of K(k) to the nearest values of the scale under consideration, then replace all elements a ij for i > j of all A n (a) by 1/a ij and all elements k ij for i > j of K(k) by 1/k ij .
Step 6B-after completing step 5B, Round all values of elements a ij for i = j of all A n (a) and all values of elements k ij for i = j of K(k) to the nearest values of the scale under consideration.
Step 7. Given all perturbed A n (a), denoted as A n (a)*, and perturbed K(k), denoted as K(k)*, compute their corresponding PVs a n * and k* using the given PDM; i.e., PREV, SNCS, LLSM, and LSDM.
Step 8. Calculate the TPV w*(PDM) of size [m x 1] by the following procedure: Step 9. Calculate the SRCC for all w*(PDM) and w, as well as any specified quality characteristics of the approximation;-e.g., MAD, PCC, or other relative deviations-e.g., mean relative errors, denoted as or average relative ratios, the value of which is given in [30] , denoted as Step 10. Repeat steps 4 to 9 χ times, with the sample size denoted asχ.
Step 11. Repeat steps 1 to 9 χ times, with the number of AHP models considered denoted asχ.
Step 12. Return the arithmetic averages of all approximation quality functions computed during all executions in Steps 10 and 11.
This time, the MCS scenario used was developed with new assumptions in mind. Therefore, not only the results obtained with a reciprocal PCM (RPCM) but also the results obtained with a non-reciprocal PCM (APCM) were considered. Although the AHP does not allow APCM in its structure, it seems reasonable to analyze its application to PCB problems; see, for example, [37,40,71]. It was also decided to introduce new intervals for perturbation factors in MCS and to apply their new PDs. Obviously, this time, the expected value of e ij was also close to unity; i.e., the value of EV(e) ≈ 1. Although this requirement is relatively easy to fulfill based on an asymmetric interval for e ij (relative to its effect on a particular PCM element), it is quite difficult to realize this assumption with a symmetric interval for e ij . However, this goal was achieved in this seminal study. It is reasonable to apply symmetric intervals to MCS for e ij as well, as they more realistically reflect real human performance in pairwise comparisons without strong outliers. Thus, experiments were conducted with different types of PDs, and it was found that the Fisher-Snedecor PD has a property that may be useful for the intended purpose. Namely, for n 1 = 14 and n 2 = 40 degrees of freedom for 1000 randomly generated numbers based on this PD, their mean is 1.03617, meaning that it is very close to one, and these numbers range from 0.174526 to 5.57826. Under these assumptions, e∈[0.174526, 5.57826] therefore holds, giving a completely symmetric PD for e ij ; i.e., EV(e) ≈ 1. The results for the selected PDMs and their assumed performance quality measures-i.e., mean relative error (MSRC), mean relative error (MARE), and mean relative ratio (MARR)-derived from the MCS scenario described earlier are shown in Table 10.
As can be seen again, PREV is not the dominant PDM in terms of all simulation scenarios in the established framework (it ranks second overall ex aequo with LLSM). Of course, the obvious differences in the quality of the PV approximation depending on the chosen PDMs are evident for non-reciprocal PDMs. LSDM and LLSM outperform the other chosen PDMs, especially with respect to rank correlations, which are crucial for the phenomenon of rank preservation.
The issue of the rank reversal phenomenon (conservation of preference power) was introduced by Bana e Costa and Vansnick [13]. They gave the following definition: for all alternatives A1, A2, A3 and A4 such that A1 dominates A2 and A3 dominates A4 and the degree of dominance of A1 over A2 is greater than the degree of dominance of A3 over A4, not only w1 > w2 and w3 > w4, but also w1/w2 > w3/w4 is true for the obtained PV. Therefore, the following scenario considered by Bana e Costa and Vansnick is revisited. Note: (*) AHP models constructed randomly (uniformly) for a given set of criteria and alternatives. The scenario assumes both factors: a disturbance factor obtained with the F-Snedecor probability for n 1 = 14 and n 2 = 40 degrees of freedom and round-off errors associated with a given scale (geometric or Saaty's). Includes calculation of performance measures for reciprocal PCM (RPCM) or non-reciprocal PCM (APCM).
When the PCM is given as This phenomenon of LSDM is especially interesting because of the perfect rank correlations for PVs derived by LSDM and PREV from randomly generated (uniform probability distribution) transitive and reciprocal inconsistent PCMs (TRPCM) (see Table 11). In order to compare the results obtained using LSDM with those obtained using PREV and to see if they are the same or if there is a possibility to reverse the order between their PV elements, 1000 TRPCMs were generated using MCS. For each randomly generated TRPCM, PVs were determined: PV LSDM and PV PREV were calculated using LSDM and PREV, respectively. In addition, the PCCs between the PV elements and the SRCCs between their priorities were calculated. The numbers of alternatives n considered were chosen as follows: 3, 4, 5, 6, 7, 8, 9, 10 and 12. The number of criteria was set to one. A standard numerical AHP scale was used to express the judgment; i.e., integers 1-9 and their inversions. Table 11 shows the mean correlation coefficients between PV elements and the priority ranks obtained during MCS with respect to the number of alternatives taken into consideration.
Considering the results presented in Table 11, three facts can be noted: firstly, for n = 3, both methods perfectly coincide; secondly, MSRC values for n > 3 equal 1,meaning that there is no rank reversal phenomenon between the LSDM and PREV for 1000 randomly generated TRPCMs; lastly, MPCC values for n > 3 between PVs derived by the LSDM and PREV for 1000 TRPCMs practically coincide with unity-i.e., MPCC ≈ 1-which indicates the almost perfect coincidence of both PDMs.

Conclusions
Discrepancies and similarities among examined PDMs have been examined in this research paper from various perspectives, also including the statistical approach. For this purpose, selected statistical measures of the effectiveness of PDMs (approximation quality) have been applied in this research; i.e., MSRC, MPCC, MARE, MAAD and MARR. Information concerning the statistical significance of discrepancies and similarities among examined PDMs is clearly presented. In this way, to the best of our knowledge, for the first time, a statistical foundation has been created to identify situations in which PDMs coincide and their discrepancies can be considered as negligible, and when their discrepancies are statistically significant and they should not be neglected. Hence, the possibility was created for a DM to assess the risk of accepting an ineffective PDM or rejecting an effective PDMthe standard problem known to every statistician and which is very important to each DM during the statistical evaluation of decisional options; i.e., statistical alternative hypothesis testing. The ranking of the PDMs evaluated in the manuscript based on novel scenarios of Monte Carlo simulations was also presented in this research paper. Furthermore, a certain interesting advantage of LSDM which other evaluated PDMs do not have-i.e., the condition of order preservation satisfaction-is also presented in the article. These research accomplishments will certainly provide fundamental support for any DM during their decisions regarding which PDM to choose in various circumstances.
Given the reality of our physical world, no study is perfect. In order to compare the characteristics of the estimates obtained in the simulation process for the selected PDM, different situations related to various sources of the PCM inconsistency were simulated. Fundamentally, PCM inconsistency commonly results from errors caused by the nature of human judgments and errors due to the technical realization of the pairwise comparison procedure; i.e., rounding errors and errors resulting from the forced reciprocity requirement commonly imposed in PCB ranking problems. All the above errors can be simulated, but the nature of human judgments is represented here as the realization of a stochastic process in accordance with the assumed probability distribution of the perturbation factor; e.g., uniform, gamma, truncated normal and log-normal. As this is only a process generated by a computer, it represents a certain limitation of the presented research. Thus, there is a space for further research in this area with the application of different MCS scenarios, various other measures of PDM effectiveness (performance quality) and a case-based methodology.
Funding: This research received no external funding. The APC was funded by Opole University of Technology, Poland.
Data Availability Statement: Datasets were generated in Wolfram's Mathematica Software 11 concurrently during the study. All statistics obtained during their analysis are an integral part of this research paper.
Acknowledgments: I am grateful for the effort of five anonymous reviewers who significantly improved the first and the second version of this research paper.

Conflicts of Interest:
The author declares no conflict of interest. The APC funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results. Table A1. Performance evaluations of four arbitrarily selected distinctive priority deriving methods for 25,000 cases of various uniformly drawn and uniformly perturbed AHP frameworks (%) .