Consistency Improvement in the Analytic Hierarchy Process

: Consistency checking is one of the reasons for the Analytic Hierarchy Process (AHP) leadership in publications on multiple criteria decision-making (MCDM). Consistency is a measure of the quality of data input in the AHP. The theory of AHP provides indicators for the consistency of data. When an indicator is out of the desired interval, the data must be reviewed. This article presents a method for improving the consistency of reviewing the data input in an AHP application. First, a conventional literature review is presented on the theme. Then, an innovative tool of artificial intelligence is shown to confirm the main result of the conventional review: this topic is still attracting interest from AHP and MCDM researchers. Finally, a simple technique for consistency improvement is presented and illustrated with a practical case of MCDM: supplier selection by a company.


Introduction
The multiple criteria decision-making (MCDM) approach contributes to decisionmaking in situations where multiple alternatives must be evaluated considering multiple criteria [1].The MCDM is a methodology, a collection of methods developed from the 1960s to solve decision problems [2].This article is focused on the Analytic Hierarchy Process (AHP), a leading MCDM method for decades [3][4][5].One main reason for the AHP's leadership in publications on MCDM is its solid mathematical foundation [6].The AHP's fundamentals provide a ground for research and development of this MCDM method.The AHP theory and practice have "seven pillars", which include the following [7]: Ratio scales derived from reciprocal pairwise comparisons.2.
Extending the scale from 1 to 9 to 1-R. 5.
Rank preservation or rank reversal.7.
Group decision-making with an aggregation of individual judgments or priorities.
Another main reason for the great number of AHP publications is the need to solve practical problems with a handy tool.AHP applications include the following [6,8]: Educational decisions: Admitting students and faculty selection.• Financial and marketing decisions: Advertising, credit analysis, downsizing, project management, and resource allocation.• Governmental or social decisions: Affirmative action, energy and fuel regulations, food and drug, and smoking policies.• Human resources and personal decisions: Career choices, entrepreneurial development, performance evaluations, and human tracking.• Sports decisions: Drafts, predictions, and salary cap.
• Supply chain decisions: Information technology, logistics, outsourcing, and supplier and vendor selection.
The pairwise comparison matrix A of a set of n objects is a central element in the AHP.Components of A = [a ij ] represent w i /w j [9], where w is the vector of the weights for the compared objects i = 1, 2, 3 . . .n. Equation (1) presents one way to generate w from A: where w is the right eigenvector of A, and λ max is its maximum eigenvalue.
A consequence of the consistency of A is presented in Equation (3) [10]: In the AHP, pairwise comparisons are usually performed regarding a linear 1-9 scale, which is named the Saaty Scale here but is also named "The Scale" [11] or "Fundamental Scale of Absolute Numbers" [8].With the Saaty Scale, A becomes a positive reciprocal matrix, satisfying conditions a ij > 0 and a ij = 1/a ji , ∀i, j = 1, 2, 3 . . .n.A consequence of this positiveness and reciprocity is that λ max ≥ n.A corollary from consistency is λ max = n [11].
Despite some criticism and the proposal of different scales [12,13], the Saaty Scale prevails in AHP applications [14].After all, the Saaty Scale allows for comparisons concerning weight dispersion and weight uncertainty [15].Nevertheless, the use of the Saaty Scale does not guarantee that A will be a consistent matrix, satisfying Equations ( 2) and (3).In the example below, A, B, and C are all pairwise comparison matrices obtained with the Saaty Scale.However, only A is 100% consistent; B and C are not: To answer Q1 and Q2, this article presents a literature review on consistency measurement and consistency improvement (Section 2), with innovative support from artificial intelligence (AI) in Section 2.2.Then, a simple technique for consistency improvement is presented (Section 3) with a practical case of MCDM: a supplier selection by a manufactur-ing company (Section 4).Finally, Section 5 presents this article's conclusions and proposal for future research.

Literature Review 2.1. Background
Consistency and the Saaty Scale have been major subjects in AHP theory since the presentation of the seminal works [11,16,17].The first document published on the AHP [16] introduced the Saaty Scale, with the former name "The Scale" but starting with zero being defined for "not comparable" when "there is no meaning to compare two objects".The document does not address the consistency measurement, focusing on obtaining the weights with the eigenvector.
The consistency ratio CR is a better measure for the consistency of a comparison matrix since it compares CI with a random index RI obtained with the simulation of positive reciprocal matrices [23][24][25], as presented in Equation ( 5): Table 3 presents values for RI as a function of the matrix order n.In the AHP literature, RI values vary because they were obtained with different numbers of randomly simulated matrices.Originally, RI was obtained with 50 matrices for each n [11].A study performed at the University of Pittsburgh (PITT) with support from the Oak Ridge National Laboratory (ORNL) increased the number of matrices to 500 [26].A statistical experiment conducted at the George Washington University (GWU) with the Software Expert Choice (EC) experimented with incomplete matrices [27], increasing the number of simulated matrices to thousands.Perhaps the most accurate estimation for RI was performed in the University of Ulster (UU), Northern Ireland [28].However, the usual values for RI are presented in the last column of Table 3.The usual values combine the ORNL-PITT values with EC-GWU: for n ≤ 7, the usual values are the EC-CWU values rounded to hundredths; for n > 7, the usual values are the same for ORNL-PITT [8].
Table 4 presents values of CR for matrices A, B, and C (Section 1) for RI presented in Table 3.As λ max A = 3, then CI A = 0, resulting in CR A = 0 for all RI values.This result is expected since A is a 100%-consistent matrix, satisfying Equations ( 2) and (3).
As λ max B ≈ 3.04, then CI B ≈ 0.02, making CR B vary from 0.03 to 0.04.As λ max C ≈ 3.99, then CI C ≈ 0.22, making CR C vary from 0.38 to 0.53.CR B and CR C are expected to be greater than zero, since B and C are not 100%-consistent matrices.However, CR C > CR B , indicating that C is more inconsistent than B. The question is as follows: is the inconsistency of B or C acceptable?To answer this question, the 0.1 threshold was proposed [11].
The 0.1 threshold considers that the normalized values for w i are from 0 to 1; the required order for RI was as small as 10% but not smaller than 1% because inconsistency itself is important, since "without it new knowledge that changes preferences cannot be admitted" [9].Saaty [17] further suggested that for matrices of orders three and four, the thresholds could be 0.5 and 0.8, respectively [29].For larger matrices, even a CR = 0.2 could be tolerated, but no more [30].Other consistency indices were proposed, such as the geometrical consistency index [31].In this article, the usual CI, CR, and its 0.1 threshold are adopted.This adoption is for an alignment with the original AHP theory and its usual practice.
Considering the 0.1 threshold, B is not 100% consistent, but it is an acceptable matrix, and C is an inconsistent unacceptable matrix.Then, the c ij components of C must be revised to improve its consistency, or simply to increase CR C .
One simple way to increase the CR of a comparison matrix is by comparing the differences between its components and the components of a 100%-consistent matrix.As the components with greater differences are more inconsistent with the others, these components are first suggested to be revised.The differences compose the deviations matrix C as in Equation ( 6): ∀i, j = 1, 2, 3 . . .n.
In our case, C is as follows: 64 and CR C ′ ≈ 0.17.C ′ is less inconsistent than C, but the inconsistency of both matrices is unacceptable since CR C and CR C ′ are greater than the 0.1 threshold.
With one more iteration, C ′′ is found: The simple A-B-C example illustrates the concepts and variables of consistency as CI and CR.Section 3 presents a technique for consistency improvement in more complex cases with n > 3. Before it, the next subsection presents how consistency has been measured and analyzed in the more recent AHP literature.

Recent Literature on Consistency Measurement and Improvement
The literature on consistency measurement of pairwise comparison matrix is a major part of the AHP literature.Therefore, it has also been prolific in the literature since the 1970s.This section focuses on the last ten years: documents published from 2013.This is the focus of the new Scopus Database tool, its artificial intelligence (AI) tool.
Most literature reviews are based on two databases: Clarivate's Web of Science or Elsevier's Scopus [32].Despite both databases having similar contents, Scopus was selected for this research because it is free through institutional access [5].Despite expected similar contents between Scopus and Web of Science, a second reason to exclusively search Scopus was the uniformity of search characteristics, such as search strings.Finally, the third reason for choosing Scopus was its new AI tool (https://www.elsevier.com/products/scopus/scopus-ai, accessed on 6 December 2023).Still in a beta phase, this tool allows for focusing on publications from recent years.
The question of "How to measure the consistency for a pairwise comparison matrix?" in the Scopus AI tool resulted in four key insights from the abstracts: 1.
Inconsistency reduction: Various iterative and non-iterative algorithms have been developed to reduce inconsistency in pairwise comparison matrices [33].

3.
New measures: Some studies have introduced new inconsistency measures for incomplete pairwise comparison matrices and interval pairwise comparison matrices [36,37].

4.
Comparative analysis: Comparative analyses have been conducted to evaluate the performance of different inconsistency indices using Monte Carlo simulations [33,37].
Scopus AI concludes that "there are several methods and indices available to measure the consistency of pairwise comparison matrices, and their effectiveness can be evaluated through comparative analyses and simulations" (https://www.scopus.com/search/form.uri?display=basic#scopus-ai, accessed on 29 December 2023).
Figure 1 presents a "conceptual map" generated by Scopus AI.This map groups the keywords into three branches, separating pairwise comparisons from the pairwise comparison matrix.These three points are connected, indeed.For instance, if the CR helps in evaluating the reliability of a pairwise comparison matrix, it affects the accuracy of the decisionmaking process.
The literature review concludes that CR and the 0.1 threshold have been accepted for the consistency measurements and analyses of pairwise comparison matrices.

Consistency Improvement
Sections 1 and 2.1 present the A-B-C example with three 3-n pairwise comparison matrices.Real problems certainly involve more matrices with n > 3. Therefore, consistency improvement becomes more complex.
With n = 2, there is no possibility for inconsistency, since k = i or k = j, always satisfying Equation (3), ∀i, j, k = 1, 2. With n = 3, and, for instance, i = 1, j = 2, and k = 3, Equation ( 3 Iterations with just one change in an inconsistent comparison matrix may not be effective.On the other hand, replacing all comparisons seems to be unfair or illogical.Therefore, we propose to change only the a ij comparisons, which brings significant deviation to a ik a kj , initially computing the expected value γ ij as in Equation ( 7): ∀i, j, k = 1, 2, 3 . . .n and j > i.
The absolute deviation between the value provided in the comparison matrix and the expected value for consistency satisfying Equation ( 3) is ψ ij = |a ij − γ ij |.For inconsistent comparison matrices, we suggest that the a ij with ψ ij between the average ψ plus or less one-third of its standard deviation must be replaced by w i /w j .
It is important to note that our proposed technique for consistency improvement resulted in individual significant changes in the comparison matrix D to D ′ .Therefore, replacing d 14 = 9 and d 23 = 8 by d ′ 14 = d ′ 23 = 1 are big changes that result in a new vector of weights.These must all be validated by the decision-maker.Furthermore, this is a major limitation of our proposal.If the decision maker does not agree with the changes, then he (she or they) must review the comparisons by himself (herself or themselves).However, our proposal is not solely based on mathematics.The comparisons are connected, and the mathematics may capture the connection as presented in the next section, with a case of consistency improvement from the real world.

A Case of Consistency Improvement in Supply Chain Decision-Making
Supplier selection is one of the decision-making problems mostly solved by AHP applications [4].This problem consists of choosing a single alternative (supplier) from a set of alternatives (suppliers).Table 5 presents an example of data for supplier selection considering three criteria (Delivery, Price, and Quality) and four alternatives (Suppliers 1, 2, 3, and 4): In this case, it is clear that Quality is the most important criterion, but it is not clear by how much it is more important than others.Furthermore, it is not clear which one is more important: Delivery or Price.Then, a pairwise comparison matrix is a good tool to figure out the relative importance of the criteria.Table 6 presents a comparison matrix among the criteria.The comparison matrix of the criteria has the same components of matrix B presented in Section 1.This matrix is equal to B T .Then, both matrices have the same λ max ≈ 3.04 and CR ≈ 0.04.Therefore, this matrix is inconsistent but acceptable, since its CR < 0.1.The decision-maker who provided the comparison matrix of the criteria understood the concepts of the Saaty Scale.
The eigenvector for the comparison matrix of the criteria has the same components of w B , but in reverse order: [0.10, 0.26, 0.64].It results in Quality being the most important criterion with 64% of weight, followed by Price and Delivery with 26% and 10%, respectively.
Table 7 presents a comparison matrix among Suppliers 1 to 4 regarding criterion Delivery.According to Table 5, Supplier 3 has the best performance in delivering quickly; Suppliers 2 and 4 deliver regularly, and Supplier 3 delivers slowly.The comparison matrix of suppliers on their deliveries has λ max ≈ 4.064 and CR ≈ 0.024.Therefore, this matrix is inconsistent but acceptable, since its CR < 0.1.The eigenvector for the comparison matrix is [0.08, 0.20, 0.52, 0.20].It results in Supplier 3 being the best in Delivery with 52% of weight, followed by Suppliers 2 and 4 tied at 20%, and Supplier 1 being the worst with 8%.
For Price, there are available data as presented in Table 5. Weights for suppliers on Price are obtained by normalizing their reciprocals, as presented in Table 8.   5, suppliers' performances vary greatly: from Acceptable (Supplier 1) to Excellent (Supplier 2), including Good (Supplier 3) and Very Good (Supplier 4).divergent among all.Then, the decision-maker agreed with the new comparison matrix (Table 11) and its eigenvector.
Complimentary procedures such as Sensitivity Analysis or Robustness Tests are not conducted in this case because they are out of the scope of this work.

Conclusions
Consistency measurement and improvement is still an attractive subject of research in the AHP literature.This is evidenced by the literature review presented in Section 2. After all, consistency checking is an advantage of applying AHP instead of other MCDM methods, which do not include this check.However, when the consistency test fails, the decision process stalls.
This article presents a procedure for the improvement of consistency of pairwise comparison matrices.The simple procedure considers the means and the standard deviations to a consistent matrix.Besides being simple, it is a highly efficient procedure requiring few changes in the pairwise comparison matrix.
The first proposal for future research is the test of the proposed procedure with more cases other than in supply chain management.This proposal is very reliable due to the applicability of the AHP in many fields of decision-making, from computer science and engineering to health and medical applications.Mathematical simulations of inconsistent matrices, for instance, with Monte Carlo experiments or similar algorithms of randomness, could also be interesting.
Finally, some important advances in the AHP not included in this work may be considered in future research, such as the adoption of Fuzzy Sets Theory (FST) or the study of Group Decision-Making.Much older than the AHP literature, FST gained attention earlier this century with the proposal of Fuzzy Hesitant and Fuzzy Intuitionistic Sets.The study on consistency measurements and improvements in hybrid AHP-FST, especially with the new types of fuzzy sets, has not yet been studied.
with a 12 a 23 = 3 × 3 = 9 = a 13 .The inconsistency of B and C is noted with b 12 b 23 = 3 × 3 ̸ = 5 = b 13 and c 12 c 23 = 7 × 3 ̸ = 3 = c 13 .The eigenvalues for A, B, and C are λ max A = 3, λ max B ≈ 3.04, and λ max C ≈ 3.99, respectively.The eigenvectors are w A ≈ [0.69, 0.23, 0.08], w B ≈ [0.64, 0.26, 0.10], and w C ≈ [0.69, 0.19, 0.12].As A is 100% consistent, one question arises: By how much are B and C inconsistent matrices?Since λ max B and w B are closer to λ max A and w A than λ max C and w C , it seems that B is less inconsistent than C. Therefore, Q1 and Q2 are two research questions: Q1: How can we measure the consistency of a pairwise comparison matrix?Q2: How can we improve the consistency of a pairwise comparison matrix?

Figure 1 .
Figure 1.Conceptual map for "How to measure the consistency for a pairwise comparison matrix?".Source: Scopus AI.Scopus AI concludes by highlighting three topics for expert research: • What are the mathematical methods used to measure consistency in pairwise comparison matrices?• How does the CR help in evaluating the reliability of a pairwise comparison matrix?• Can inconsistency in a pairwise comparison matrix affect the accuracy of decisionmaking processes?
) may not be satisfied, as it occurrs with b 13 ̸ = b 12 b 23 and c 13 ̸ = c 12 c 23 .With n ≥ 4, the possibility for inconsistency increases with three combinations of n(n − 1)/2 comparisons.

Table 2 .
Citations of the first published documents on the AHP.

Table 4 .
Consistency ratio values with different random consistency indexes.
As c 12 = 3.34 is the greatest component of C, it is suggested that it should be revised from c 12 = 7 to c ′ 12 = w 1 /w 2 ≈ 0.69/0.19≈ 3.66, resulting in C ′ : ′′ ≈ 3.04 and CR C ′′ ≈ 0.04.Now, C ′′ is an acceptable pairwise comparison matrix with CR C ′′ ≈ 0.096.The changes from C to C ′′ result in w ′′ C ≈ [0.58, 0.27, 0.16], different than the former w c .Of course, this would need approval by the decision-maker or by whoever is in charge of making the comparisons.

Table 5 .
Example of data for a supplier selection problem.

Table 6 .
Pairwise comparison of the criteria for a supplier selection problem.

Table 7 .
Pairwise comparison of suppliers regarding their deliveries.

Table 8 .
Weights for suppliers regarding their prices.

Table 9
presents a comparison matrix for suppliers regarding the Quality criterion.According to Table