Eliciting Correlated Weights for Multi-Criteria Group Decision Making with Generalized Canonical Correlation Analysis

: The proper solution of a multi-criteria group decision making (MCGDM) problem usually involves a series of critical issues that are to be dealt with, among which two are noteworthy, namely how to assign weights to the (possibly distinct) judgment criteria used by the different decision makers (DMs) and how to reach a satisfactory level of agreement between their individual decisions. Here we present a novel methodology to address these issues in an integrated and robust way, referred to as the canonical multi-criteria group decision making (CMCGDM) approach. CMCGDM is based on a generalized version of canonical correlation analysis (GCCA), which is employed for simultaneously computing the criteria weights that are associated with all DMs. Because the elicited weights maximize the linear correlation between all criteria at once, it is expected that the consensus measured between the DMs takes place in a more natural way, not necessitating the creation and combination of separate rankings for the different groups of criteria. CMCGDM also makes use of an extended version of TOPSIS, a multi-criteria technique that considers the symmetry of the distances to the positive and negative ideal solutions. The practical usefulness of the proposed approach is demonstrated through two revisited examples that were taken from the literature as well as other simulated cases. The achieved results reveal that CMCGDM is indeed a promising approach, being more robust to the problem of ranking irregularities than the extended version of TOPSIS when applied without GCCA.


Introduction
Multi-criteria group decision making (MCGDM) is usually described as the process of selecting the best alternative(s) from a given set of viable options that are based on the opinions that are provided by multiple domain experts, frequently referred to as decision makers (DMs), concerning multiple judgment criteria [1,2]. In recent years, a number of MCGDM techniques have been proposed and widely applied in many distinct fields, such as sustainable development [3,4], personnel evaluation [5], social network analysis [6], software selection [7,8], supplier selection [9,10], and economics [11].
Among others, Herrera-Viedma et al. [12] have argued that solving a group decision making (GDM) problem usually involves the carrying out of two complementary processes: a consensus reaching process (generally guided by a moderator), which refers to how to obtain, via one or more stages of negotiation, the maximum degree of agreement between the experts; and, a selection process, which achieves the final solution via the aggregation of the experts' individual preferences over the different alternatives available. Various approaches have been developed so far to help undertaking these processes, especially the latter, in different circumstances; for instance, by addressing dynamic sets of alternatives [13] and criteria [14], changes of preferences [15] and opinions [16,17], as well as differences in the knowledge level between the accessible DMs [18,19].
Specifically concerning the modelling of the DMs' preferences, the assignment of weights to their evaluation criteria turns out to be a crucial task to be accomplished, since the final decision that is delivered by a given MCGDM method usually depends on such weights to a large extent. However, properly calibrating criteria weights in multi-criteria decision making is a hard task to pursue, even when considering the single DM setting [20]. This task becomes even more relevant (and harder to attempt) when different criteria are adopted by different DMs. In this regard, Fan et al. [21] recently pointed out that research concerning this more complex scenario is still relatively scarce in the literature and, thus, developed a method for tackling MCGDM with different evaluation criteria sets.
Based on these considerations, this paper investigates a novel GDM approach that is aimed to address, in an integrated manner, both the elicitation of the DMs' preferences and the consensus of their individual decisions. The approach, which is referred to as canonical multi-criteria group decision making (CMCGDM, for short), can also deal with MCGDM problems having different criteria sets for different DMs and makes use of a generalized version of canonical correlation analysis (CCA) [22] to automatically compute the values of criteria weights.
In a nutshell, the goal of CCA is to maximize the linear correlation between two sets of variables, so as to yield a novel set of canonical variates, which, in turn, may replace the original ones. This procedure involves computing the weights for both sets of original variables that result in the highest possible correlation between the canonical variates [23]. Although the standard CCA only handles two sets of variables, there are extensions that handle more, such as the generalized CCA (GCCA) version that was proposed by Kettenring [24].
CMCGDM also adopts the extended version of TOPSIS (technique for order preference by similarity solution) [25] that was proposed by Shih et al. [26], which is specific for MCGDM. The practical usefulness of CMCGDM is demonstrated by revisiting two examples, one on a human resource selection [26] and the other on a machine acquisition [21]. The results achieved in these examples and other simulated cases reveal that CMCGDM is indeed a promising approach, being more robust to cope with the ranking irregularity problem [27] than the extended TOPSIS for GDM without using GCCA.
In short, the main contributions of this paper are: to show that GCCA is a viable approach to eliciting weights in the MCGDM context; to prove that CMCGDM is more robust for dealing with the problem of classification irregularity than the extended TOPSIS without using GCCA, and that it does it in a straightforward way without the need for changes in the extended TOPSIS procedure; and, to be able to reach the group's consensus by reconciling the different canonical weights that were provided by the GCCA.
The rest of this paper is organized as follows. In Sections 2 and 3, we review the main aspects that are related to standard MCGDM and GCCA. In Section 4, we present the main steps comprising the new CMCGDM methodology and point out some of its relevant properties. In Sections 5 and 6, we evaluate CMCGDM on the examples considered by Shih et al. [26] and Fan et al. [21], whereas, in Section 7, we compare GCCA with other well-known criteria weighting methods on several simulated cases. Finally, Section 8 concludes the paper and brings some remarks on future work.

Multi-Criteria Group Decision Making
Briefly stated, MCGDM refers to the process of making decisions in group when there are multiple (but a finite list of) alternative solutions that are available to the decision problem in hand. In addition, the group of DMs assess the pros and cons of these alternatives by taking multiple (usually conflicting) judgment criteria into account [1,2]. Formally, an MCGDM problem with M alternatives, N criteria, and K DMs can be formulated by defining K decision matrices of the form: where A = {A 1 , A 2 , . . . , A M } denotes the set of feasible alternatives, C k = {c k 1 , c k 2 , . . . , c k n k } represents the n k evaluation criteria that are associated with the k-th DM, x k ij is the performance rating of alternative A i under criterion c k j (j = 1, . . . , n k ), and w k j stands for the weight of this criterion. Notice that 0 ≤ w k j ≤ 1 and ∑ n k j=1 w k j = 1. On the other hand, the K criteria sets may be equal, partially overlap, or be completely disjoint, in which case ∑ k n k = N. Each criterion can be classified as either a benefit ("the higher, the better") or cost ("the lower, the better") criterion.
Several multi-criteria methods have been proposed or extended in order to cope with different variants of the MCGDM problem. In the sequel, we focus on the extended version of TOPSIS that was proposed by Shih et al. [26], which was adopted in the development of CMCGDM. Afterwards, we overview some of the most well-known methods used to compute criteria weights in TOPSIS, formalizing these methods in the context of GDM.

Extended TOPSIS for GDM
TOPSIS [25] is a well-known multi-criteria method, which is based on the notion that the chosen alternative should have the closest distance to a positive ideal solution (PIS) and the farthest distance to a negative ideal solution (NIS). A crucial assumption of this compensatory method is that the decision criteria are either monotonically increasing or decreasing [28]. The variant that was conceived by Shih et al. [26] keeps this assumption, but extends the scope of the method in order to acknowledge the existence of several DMs.
The main steps of the extended TOPSIS for GDM are: Step 1: compose the decision matrix D k for the k-th DM-refer to Equation (1).
Step 2: compute the normalized decision matrix R k = [r k ij ], i = 1, . . . , M, j = 1, . . . , n k , for the k-th DM. For this purpose, the vector normalization scheme is usually employed [29]: Step 3: calculate the positive ideal solution V k+ and the negative ideal solution V k− for the k-th DM, as follows: where J 1 and J 2 are the sets of benefit and cost criteria, respectively.
Step 4: assign a weight vector w k = (w k 1 , w k 2 , . . . , w k n k ) to the criteria set of the k-th DM, such that ∑ n k j=1 w k j = 1.
Step 5: compute the overall separation of a given alternative from the set of positive and negative ideal solutions. Here, two substeps should be performed. The first considers the PIS and NIS that are associated with each DM separately, while the second aggregates the measurements for the whole group.
Substep 5a: compute the distances S k+ i and S k− i of the i-th alternative, i = 1, . . . , M, to the pair of PIS and NIS associated with the k-th DM, k = 1, . . . , K. Here, we have considered the Euclidean distance: where r k ij , v k+ j and v k− j are defined in Equations (2), (3) and (4), respectively. Substep 5b: compute the overall separation measures S + i and S − i for each alternative. For this purpose, one should calculate the geometric mean over the K values of S k+ i (5) and S k− i (6) to yield: and Step 6: compute C * i , the overall relative closeness of the i-th alternative A i , i = 1, . . . , M, to the K positive ideal solutions, which can be expressed as: where 0 ≤ C * i ≤ 1, and S + i and S − i are defined, respectively, in Equations (7) and (8). Hence, the alternatives can be ranked from the best (higher values of closeness) to worst.
Weighting methods, in particular, attach cardinal or ordinal values directly to the criteria, so as to reflect their relative importance. Wang et al. [41] classify the weighting methods into three categories: subjective, objective, and combination methods. The first determine the weights based on the subjective preferences of the DMs, whereas the second make use of mathematical models without any consideration of the DM's preferences. Combination methods are hybrids of the former, including, for instance, multiplication and additive synthesis. In the following, we review some of the most well-known methods of this class because the use of GCCA for computing the criteria weights in the context of CMCGDM can be regarded as an objective method.

Entropy Method
In short, entropy is a measure of uncertainty in information, as formulated in probability theory [31]. Entropy also means that some information cannot be recovered or is lost, like noise in a message. Accordingly, the higher the entropy E of a system, the lower its information content I.
When applied to multi-criteria decision making, this concept can be used to quantitatively measure the capacity of a given criterion to discriminate the quality of different alternatives. High discrimination corresponds to low entropy and, thus, better information content. Conversely, low discrimination corresponds to high entropy and worse information. One particular advantage of using entropy as a criterion weight measure is that it weakens the bad effects from abnormal values (outliers), which makes the result of evaluation more accurate and reasonable.
When this procedure is applied to all criteria at once, it is possible to rank them according to their significance: the higher the information content I of a criterion, the more relevant it is comparatively. It is worthy noticing that these weights can be used for the assessment of alternatives, since they are built based on the dispersion (discrimination) of the alternatives' performance values. Consequently, they are totally objective (unbiased), not depending upon the specific DM doing the analysis.
When considering the GDM context, in order to calculate the criteria weights w k j via the entropy method, the decision matrix D k (1) should be first normalized to yield the probabilities p k ij , as given by and then the following equations should be calculated: where log(·) denotes the logarithm function, and E k j and w k j are, respectively, the entropy and weight values that are associated with the j-th criterion of the k-th DM. As discussed above, the higher the value of E k j , the lower is the value of w k j .

Statistical Variance Method
In statistics, variance is defined as the expectation of the squared deviation of a random variable from its mean. In other words, it measures how far a set of quantitative observations are spread out from their average value. Studying variance allows for one to quantify how much variability is in a probability distribution. If the outcomes of the distribution vary wildly, then it will have a large variance. Otherwise, if the variance is null (it is always non-negative), then the random variable takes a single constant value, which is exactly its expected value.
Based on the above considerations, statistical variance can be used in multi-criteria decision making in order to assess the capability of the judgment criteria to discriminate between the different alternatives available. The larger the variance of a given criterion (viewed as a random variable), the more dispersed are the performance values of the alternatives, which allows the DM to have better judgement on their good/bad characteristics. In this way, the higher the variance, the higher should be the relative weight of a criterion.
In this method, the statistical variance of information is first calculated for the j-th criterion of the k-th DM based on the original score values: where V k j is the statistical variance of the j-th criterion of the k-th DM and x k j is the average value of the original score values x k ij . Subsequently, the weights are obtained via a simple normalization, so that they lie in [0, 1]:

Standard Deviation Method
Because the standard deviation (SD) is defined as the positive square root of the statistical variance, it can also be regarded as a measure of dispersion around the mean of a data set. However, calculating the variance involves squaring deviations, so it does not have the same unit of measurement as the original observations. This negative feature is not shared by SD, whose values are in the same unit of the original scale. On the other hand, both variance and SD can be greatly affected if the mean gives a poor measure of central tendency, which can happen due to the presence of outliers. A single outlier can raise the standard deviation and, thus, distort the picture of spread.
In the context of multi-criteria decision making, the SD measure can be used to assign a small weight to a criterion if it shows similar values across the alternatives; otherwise, the criteria with larger deviations should be assigned the larger weights. If all available alternatives score almost equally with respect to a given criterion, then such a criterion will be regarded as unimportant by most experts and, thus, could be removed from the analysis.
Formally, the SD method determines the weights of the criteria in terms of their SDs, according to the following equations [32]: where x k j and σ k j are, respectively, the mean and SD values that are associated with the j-th criterion of the k-th DM.
Although the computation of variance and SD is similar, their differences are rather significant. By employing squared deviations, variance gives more weight to those criteria whose performance values are more spread around their average. Besides, the sum of square root values used in the denominator of Equation (16) yields a different normalization factor than the one adopted in Equation (14), which, in turn, is based on a sum of squared values. As a result, the rankings of alternatives that are produced by these measures need not be the same. In any case, both statistical measures are sensitive to a wide variation between the measurement scales of the different criteria. Of course, such a drawback is more aggravated in variance, due to the squared deviations. Entropy, in contrast, is robust to criteria scaling.

Criteria Importance through Inter-Criteria Correlation
The CRITIC (criteria importance through intercriteria correlation) method, as proposed by Diakoulaki et al. [33], uses correlation analysis to detect contrasts and dependencies between the criteria. Contrary to the other methods discussed so far, CRITIC was specifically conceived for the multi-criteria decision making domain, performing a detailed investigation of the decision matrix for extracting judicious information that is available in the evaluation criteria as well as their mutual/contrastive relationships. According to this method, objective weights are derived in order to quantify the intrinsic information of each evaluation criterion, while using both its standard deviation and its correlation (as calculated by Pearson correlation) with the other criteria. This way, both contrast and conflict intensity contained in the structure of the decision problem are captured by this method.
Consider the decision matrix of the k-th DM, as given in Equation (1). In order to calculate each weight w k j , the following symbols are used: r k ij is the normalized performance measure of the i-th alternative with respect to the j-th criterion, c k j denotes the quantity of contrastive information contained in the j-th criterion, σ k j stands for the standard deviation of the j-th criterion, and ρ k jj denotes the value of the Pearson correlation coefficient between the j-th and j -th criteria. Based on these notations, the steps of the CRITIC method are given as follows [32]: Step 1: the score values associated with benefit/cost criteria are first normalized using Equations (17) and (18), respectively.
Step 2: the correlation between each pair of criteria is calculated via Equation (19).

DEMATEL
Decision Making Trial and Evaluation Laboratory (DEMATEL) was elaborated as a procedure for solving problems of identifying cause-and-effect relationships [34]. With time, this method has been well adapted for use in multi-criteria decision making. This way, some authors discuss the use of DEMATEL in order to determine the significance of the criteria [42][43][44]. This section describes the approach that was proposed by Kobryń [42], whose formulation was adapted for several DMs.
First, the direct-influence matrix is created for each DM, which is a square matrix whose size is equal to the number of alternatives/criteria. For this purpose, we have adopted a four-degree scale to express the influence of the i-th criterion on the j-th criterion, where: 0-no influence, 1-medium influence, . . ., 4-maximum influence [45]. Besides, in the direct-influence matrix B k of the k-th DM, k = 1, . . . , K, the elements on the main diagonal are null, while non-zero elements b k ij (i = j) reflect the impact of the i-th criterion on the j-th criterion: Matrix (22) is then normalized, as follows: From (23), we calculate the total-influence matrix (T k ), as described by where I is the n k × n k identity matrix. Subsequently, two vectors of indicators are determined based on T k to express a relation between the criteria, covering both direct and indirect influences. They are defined as importance indicator vector (t k+ ) and relation indicator vector (t k− ), whose components are given as follows: From Equations (25) and (26), the weights are determined as proportional to the average value (t k av i ) of the appropriate pair of indicators t k+ i and t k− i , given as follows: Next, the equation below can be used to calculate each normalized weight: It is necessary to correct the weight values calculated from Equation (28), as no criterion can be assigned to zero weight. The key issue is to determine the correction value δ, and the final decision should belong to the decision-maker [42]. The value δ should be as small as possible and as such Kobryń [42] suggests setting δ ≤ min The values w k cor i and w k norm i are given as follows: (30)

DEMATEL-Based ANP (DANP)
The DANP method was proposed by Yang et al. [35]. It consists in forming a matrix S k , which is similar to the Analytic of Network Process (ANP) [46], on the basis of the modified total-influence matrix T k (24), representing the outcome of the application of DEMATEL [47]. The components of S k should be given as: where n k is the number of interrelated criteria adopted by the k-th DM, t k ji denotes the element of the matrix T k depicting the total influence of the j-th criterion on the i-th criterion, and s k ij represents a component of the matrix S k .
Next, the matrix S k lim is determined based on matrix S k , while using the procedure characteristic for ANP, as follows: The matrix S k lim consists of n k identical columns. The elements of the individual columns depict the normalized weights of the criteria.

Generalized Canonical Correlation Analysis
In multivariate statistical analysis, data usually consist of multiple variables measured on a set of observations [48]. In this context, CCA comprises a family of statistical techniques that model the linear relationships between two (or more) sets of variables [22,49]. More specifically, in CCA, the variables of an observation can be partitioned into two or more sets, with each one regarded as a view of the data [22].
Like principal component analysis (PCA) and linear discriminant analysis (LDA) [48,49], CCA can reduce the dimensionality of the original variables, since only a few factor pairs are normally needed to represent the relevant information. Besides, CCA is invariant to any affine transformation of the input variables [50]. Another appealing property is that CCA does not assume a priori the direction of the relationship between the variable sets. This is in contrast to regression methods, which have to designate an independent and a dependent data set [51]. Finally, CCA characterizes relationships between data sets in an interpretable way, a property that is not displayed by other common correlation methods, which simply quantify the similarity between data sets [51].
Formally, given two zero-mean data sets X = (x 1 , x 2 , . . . , x n ) ∈ R d×n and Y = (y 1 , y 2 , . . . , y m ) ∈ R d×m , with x i and y i denoting d-dimensional column vectors, standard CCA finds a canonical coordinate space that maximizes correlations between the projections of the two variable sets onto that space [51]. Associated with the j-th dimension of this new space, there is a pair of projection weight vectors, a j = (a 1j , a 2j , . . . , a nj ) and b j = (b 1j , b 2j , . . . , b mj ), named canonical weights. The resulting projections of variable sets X and Y onto the j-th dimension of the canonical space comprise a pair of d-dimensional vectors, u j = a j , X and v j = b j , Y , which are called canonical variates. Here, ·, · denotes the inner (projection) product operator. CCA maximizes the linear correlations between each pair of canonical variates, as given by where u j denotes the norm of vector u j . More precisely, r = min(n, m) pairs of projection vectors are generated, so that the correlation ρ 1 between u 1 and v 1 is maximum, the correlation ρ 2 between u 2 and v 2 is maximum, subject to the constraint that the canonical variates u 2 and v 2 are orthogonal to u 1 and v 1 , respectively, and so on and so forth, up to the point that the correlation ρ r between u r and v r is maximum, provided that they correlate with neither u 1 , u 2 , . . . , u r−1 nor v 1 , v 2 , . . . , v r−1 , respectively. The symmetry of the correlation matrices guarantees orthogonality.
To accomplish this, CCA solves the following optimization problem [51]: where ρ represents the maximum value of the canonical correlation vector, C XY denotes the sample covariance of the two variable sets, X and Y, and C XX and C YY are their autocovariances. The objective function (34) has infinite solutions if no restriction is imposed on weights a and b. However, the size of the canonical weights can be constrained, such that a T C XX a = 1 and b T C YY b = 1 [49]. This leads to the following Lagrangian [22,51]: which, in turn, can be formulated as the following generalized eigenvalue problem: Several variants of CCA have been proposed along the years, among which those that extend the correlation analysis to encompass more than two sets of variables [22]. These extensions, which are typically referred to as GCCA, aim at generating a series of components (variates) that maximize the association between the multiple variable sets. For instance, when considering the availability of three views on the data, the generalized eigenvalue problem that is defined in (36) can be simply extended, as follows [24,51]: The fact that the sets of variables may differ significantly is an interesting property of this extended formulation. Hence, the number of variables in each set does not need to be the same. We argue that this feature is interesting for the MCGDM context, particularly in those circumstances when the DMs make use of different judgment criteria.

A Canonical Multi-Criteria Group Decision Making Approach
By drawing a parallel between GCCA and MCGDM, one can note that methods pertaining to both areas operate on numerical values arranged in two-dimensional (data/decision) matrices. While, in GCCA, there is a set of data instances (rows) represented by two or more variable sets (columns), in MCGDM one has a set of solution alternatives (rows), each assessed in accordance with two or more sets of criteria (columns). By establishing this correspondence, it is possible to make use of GCCA's functionalities to automatically compute the values of the criteria weights used in MCGDM.
Assume that K decision matrices are available, with D k pertaining to the k-th DM, k = 1, . . . , K. GCCA is directly applied to all K matrices at once, yielding the criteria weights w k that are related to each matrix (refer to Figure 1). These weights are the canonical weights that are associated with the first dimension of the canonical space. To bring about the consensus decision between the DMs, CMCGDM employs the same steps of the extended TOPSIS variant proposed by Shih et al. [26] (see Section 2.1), with some minor modifications. More specifically, while the computation of V k+ and V k− (Equations (3) and (4)) in the third step now takes into account the positive/negative signs of the canonical weights as calculated by GCCA (i.e., before they are normalized), in the fifth step the computation of S k+ i and S k− i (Equations (5) and (6)) makes use of the normalized weights. We claim that the above methodology is useful for capturing the intrinsic relationships between the DMs' beliefs regarding the relative performance of the different alternatives. Usually, the weights of the criteria adopted by all DMs are separately calculated for each DM (sometimes via an unrelated methodology, such as in [52]) and, then, the group consensus is achieved by somehow reconciling the different weights adopted by the different DMs. In the case of CMCGDM, because the canonical weights maximize the correlation between the different decision matrices simultaneously, it is expected that the consensus between the experts can be captured more naturally and effectively, bringing about a more reliable ranking of the available alternatives. In this case, the canonical correlation index can be interpreted as a sort of consensus indicator between the DMs' opinions.
When considering the more complicated setting where different groups of criteria are used by different DMs, our method can work well, even when the sets of criteria are completely disjoint (that is, with no overlap). Moreover, instead of generating separate rankings of alternatives for all criteria subsets, which are then aggregated to compose the final ranking by solving a specific optimization problem, such as in the approach that was proposed by Fan et al. [21], our methodology seems to be more straightforward, not demanding intermediary rankings for different criteria sets.
Another good property of CMCGDM is that it allows for one to interpret each criterion according to the sign of its associated canonical weight, taking the role played by the other criteria as reference. While positively weighted criteria can be considered as benefit ones, those with negative weights can be regarded as cost criteria.
In order to further clarify this important property, consider a fictitious MCGDM problem that involves four DMs and eight alternatives, each of which is evaluated via five judgment criteria, shared by all DMs. Tables 1 and 2 show the criteria weights as elicited by GCCA, either before or after normalization is applied. As one can notice, for the first and second DMs, all criteria, except the last one, should be interpreted as of the cost type, since their weights are negative. For the fourth DM, in contrast, the number of benefit criteria is larger. Therefore, the interpretation is contextualized for each DM. Although the non-normalized weights are small in magnitude, they were induced by GCCA so as to maximize the correlation between the DMs' decision matrices (not shown in this example). After normalization is applied, the magnitude of the criteria weights is rescaled, so that the more relevant ones are more noticeable (they are highlighted in Table 2).
Another good property of CMCGDM is that GCCA is invariant to any affine transformation of the input variables (criteria values), as mentioned before. Besides, like entropy, GCCA is robust against the problem of criteria scaling. By another perspective, the use of GCCA renders our approach flexible enough to allow the dynamic (re)calibration of the criteria weights once the sets of alternatives or criteria change over time. GCCA could be used without modification, even when the number of DMs is allowed to change. However, this important property will not be further explored in this paper and it should be investigated in future work.
We argue that these fine properties are not shared by other popular criteria weighting methods, such as those that are reviewed in Section 2.2. In fact, these methods were not conceived to elicit the weights of different groups of criteria based on the intrinsic relations between the DMs' preferences (decision matrices). Moreover, as we will show in Section 7, the adopted MCGDM method becomes more resilient to the problem of ranking irregularity [27] when using GCCA in place of the aforementioned criteria weighting methods.

Example on Human Resource Selection
To demonstrate the utility of the proposed CMCGDM approach (our implementation uses the Python package Pyrcca [51] for GCCA, which is hosted in http://github.com/gallantlab/pyrcca), we first present in this section the same example considered by Shih et al. [26], which is about the recruitment of an on-line manager by a firm. Subsequently, we compare the performance displayed by the CMCGDM approach with that delivered by the extended TOPSIS version, as originally proposed by Shih et al. [26] (that is, without using GCCA to generate the criteria weights). This is done by considering how resilient each method is to the ranking irregularity problem.

No. Criteria
The According to Wang and Triantaphyllou [27], an intriguing problem that may happen with different decision-making methods is that of generating disparate outcomes (rankings) when submitted to the same decision problem instance. Consequently, it is natural to raise the question of how to properly assess the performance of such methods. Because it is practically impossible to know which is the best alternative solution for a given decision problem, in [53] some tests capturing different ranking irregularities are discussed as a way for assessing the performance of multi-criteria methods in general: Test #1: the best alternative selected should not alter when a non-optimal alternative is added or removed from the problem (assuming that the relative importance of each criterion remains unchanged) [54]. Test #2: the best alternative selected should not change if a non-optimal alternative is replaced by a worse one [27,55]. Test #3: the final ranking of the alternatives should not violate the transitivity property if a non-optimal alternative is added to (or removed from) the problem [27,55]. Table 7 brings the ranking of the alternatives delivered by the extended TOPSIS, as reported in [26], as well as the results of the application of the three ranking irregularity tests that are described above. Table 8 follows the same layout, but it relates to the extended TOPSIS with criteria weights estimated by GCCA (i.e., CMCGDM approach). In both tables, the second and third columns show the new rankings of the alternatives (and their associated scores) that result when an irrelevant alternative is added or removed, respectively, to the alternative set, whereas the last column exhibits the ranking that results from the replacement of a non-optimal alternative by a worse one. Those cases where the transitivity property is violated (Test #3) are indicated by shadowed alternatives.
The extended TOPSIS, using the weights given in Table 5, could pass well Tests #1 and #2, but failed to respect the transitivity requirement, since there is a ranking position change between alternatives A14 and A17 (A4 and A7) when an irrelevant alternative (namely, A1) is duplicated (removed) during the application of Test #1, as one can readily observe from Table 7. On the other hand, the proposed CMCGDM approach could get through all of the three ranking irregularity tests without generating any ranking inconsistencies.

Example on Machine Acquisition
In this section, we conduct the same previous analysis on a second example, taken from [21]. In this example, there are different criteria sets that are associated with different DMs. The decision problem relates to the acquisition of a novel machine by a given manufacturing company. There are seven products (alternatives) under analysis by the manufacture department (DM #1) and the financial department (DM #2). The last alternative was introduced by us in order to turn the decision problem a bit more complex. The criteria concerned by DM #1 include C 1 : positioning accuracy (mm), C 2 : maximum load (kg), C 3 : mean time to failure (h), C 4 : degree of standardization of parts, C 5 : reliable service life (h), and C 6 : delivery time (month). The criteria concerned by DM #2 include C 7 : price ($) and C 8 : down payment ratio (percent), as well as C 5 and C 6 .
Tables 9 and 10 bring the decision matrices that are associated with each DM, respectively, whereas Tables 11 and 12 bring the results of the application of the three ranking irregularity tests described in the previous section. The layouts of these tables are the same of Tables 7 and 8. As before, the extended TOPSIS could not keep up well with the transitivity requisite when a mid-ranked alternative (namely, A2) is duplicated. CMCGDM, in turn, could produce rankings showing no inconsistencies according to all ranking irregularity tests.

Simulated Cases
In the previous sections, the criteria weights that were used in the comparative analysis were set up either by the DMs themselves or via GCCA. In this section, we enlarge the assessment by also considering the objective criteria weighting methods that are discussed in Section 2.2. The idea is to study, via several simulated cases, how the choice of the weighting method affects the behavior of the extended TOPSIS that was proposed by Shih et al. [26] with respect to the ranking irregularity problem.
The simulations were performed by following the guidelines that were provided in [27,53,56]. According to these authors, the simulations comprise a reasonable expedient to conduct controllable and reproducible experiments in order to have a better understanding of the pros and cons of different multi-criteria decision making methods. By this means, these methods can be deeply assessed by considering different samples and amounts of parameters, numbers of criteria and alternatives, as well as distinct ways to assign weights to the criteria and scores to the alternatives. In our investigation, the main parameters considered along the simulations were the following: 4. scores of the alternatives: randomly generated by a uniform distribution in the range [0-100]; 5. criteria weighting approach: GCCA and the methods described in Section 2.2; 6. number of trials: 100 for each parameter configuration, thereby yielding 6300 different decision problem instances; and, 7. performance criteria: the three irregularity tests described in Section 5. Table 13 shows the results that were delivered by extended TOPSIS when configured with the different criteria weighting methods (it is worthy reminding that extended TOPSIS using GCCA refers to the CMCGDM approach). The third and fourth columns of this table indicate the number of problem instances (out of 6300) for which the transitivity requirement was violated when an unimportant alternative was inserted or removed, respectively. The fifth column does the same transitivity test, but it refers to those simulation trials where a non-optimal alternative was replaced by a worse one. Conversely, the sixth and seventh columns express the number of cases where the best alternative solution was altered either by adding/removing an irrelevant alternative or by replacing a non-optimal alternative by a worse one. Finally, the last column indicates the aggregated sum of the values in the third, fourth, and fifth columns. As one can notice, using GCCA to elicit the criteria weights has usually achieved better performance than the other contestant methods, lagging behind the statistical variance procedure and CRITIC when the best alternative solutions were altered after adding irrelevant alternatives.  Figures 2-4 show how the number of failure cases varied for each type of irregularity test when the number of alternatives increased from 17 (minimum) to 30 (maximum) in order to better understand the performance of the different criteria weighting methods across the simulations. Considering Figure 2 (Test #1), one can notice that GCCA's overall performance falls short of CRITIC and VarProc, although, after the 25 mark, it is only surpassed by VarProc. While the number of test failures caused by DANP grew up steadily, the increase of this number for GCCA is less accentuated in the range of 25 to 30 alternatives. Regarding Figure 3 (Test #2), the overall performance of GCCA is much better than all other methods and its stability from the 21 mark is remarkable. In general, all methods (except DANP) show a reasonable resilience to the conditions imposed by this test. Finally, Figure 4 informs how the different methods handled the transitivity requirement (Test #3). The values correspond to the aggregated cases, as shown in the last column of Table 13. All methods displayed a similar monotonically increasing behavior in terms of number of failed trials; however, the performance of GCCA was usually better, regardless of the number of alternatives.

Final Remarks
In this paper, we introduced a novel GDM approach, CMCGDM, which adopts GCCA to automatically elicit the weights of the (possibly distinct) criteria used by the different DMs. By maximizing the correlation between the DMs' decision matrices, we argue that the elicited weights can better reflect the consensus between the DMs' preferences regarding the various alternatives. CMCGDM also makes use of the extended version of TOPSIS conceived by Shih et al. [26], which is specific for MCGDM. We revisited two examples taken from the literature and performed a series of simulations considering popular criteria weighting methods to demonstrate the usefulness of CMCGDM. Overall, the results revealed that CMCGDM is a promising approach, being more resilient to the ranking irregularity problem than the extended TOPSIS without using GCCA.
As future work, we plan to extend CMCGDM to work with other GDM methods available in the literature as well as with uncertain criteria, such as those that involve fuzzy, interval, incomplete, or random values [1,14,57]. We also plan to compare CMCGDM with other methods for eliciting criteria weights, such as CSW-DEA [36,37], nonlinear programming methods [39], and swing-weighting [40]. We shall also investigate the use of GCCA to recalibrate the criteria weights in dynamic settings, particularly in those circumstances where the sets of alternatives or criteria are allowed to change over time [58]. Finally, the use of non-linear versions of CCA (such as those that employ kernels [22]) seems to be a good theme to explore.
Author Contributions: Both authors contributed equally for this work. Both authors have read and agreed to the published version of the manuscript. All authors have read and agreed to the published version of the manuscript.