Information-Theoretic-Entropy Based Weight Aggregation Method in Multiple-Attribute Group Decision-Making

Weight aggregation is the key process to solve a multiple-attribute group decision-making (MAGDM) problem. This paper is trying to propose a possible approach to objectivize subjective information and to aggregate information from attribute values themselves and decision-makers’ judgment. An MAGDM problem without information about decision-makers’ and attributes’ weight is considered. In order to define decision-makers’ subjective preference, their utility function is introduced. The attributes value matrix is converted into a subjective attributes value matrix based on their subjective judgment on attribute values. By utilizing the entropy weighting technique, decision-maker’s subjective weight on attributes and objective weight on attributes are determined individually based on the subjective attributes value matrix and attributes value matrix. Based on the principle of minimum cross-entropy, all decision-makers’ subjective weights are integrated into a single weight vector that is closest to all decision-makers’ judgment without any extra information added. Then, by applying the principle of minimum cross-entropy again, a weight aggregation method is proposed to combine the subjective and objective weight of attributes. Finally, an MAGDM example of project choosing is presented to illustrate the procedure of the proposed method.


Introduction
Making decisions is a part of our daily routine. However, making the right decision has become more and more complicated as problems are growing in magnitude and longitude. More and more attributes of different alternatives must be considered or a group of decision-makers' judgment needs to be collected. Multiple-attribute group decision-making (MAGDM) might be the most common, but complex problem in the decision science field, which has been regarded as one of the most significant activities in industry, service, business, etc.
Group decision-making (GDM) or collaborative decision-making is a situation faced when individuals collectively make a choice from the alternatives before them. The decisions made by groups are often different from those made by individuals. There is much debate as to whether this difference results in decisions that are better or worse, and factors that impact other social group behaviors also affect group decisions [1]. Moreover, consensus-reaching processes play an increasingly important role in the resolution of GDM problems: a solution acceptable to all of the experts participating in a the mentioned divergences in an MAGDM problem. Secondly, in the weight aggregation procedure, it is a fundamental requirement to integrate all information from attributes and decision-makers without additional information. Information-theoretic entropy-based methods, such as the maximum entropy principle, minimum cross-entropy principle and the entropy weighting technique, can meet the requirement correctly. In consideration of this, we will try to utilize the minimum cross-entropy principle and entropy weighting technique to fulfil this demand.
In short, this paper will introduce a utility function to convert the attribute value matrix into the subjective attribute value matrix. Then, by using the entropy weighting technique, we obtain the subjective and objective weight, and by using the principle of minimum cross-entropy, an optimal model is established to aggregate weights in order to minimize the distance measure between subjective and objective weight. This study is completely based on the information entropy method and follows the general research framework of MAGDM, which provides a new way of thinking and a method for solving common MAGDM problems.
The rest of the paper is organized as follows. In Section 2, we briefly describe some preliminaries. Section 3 presents the weight determination method based on the entropy weighting technique and the weight aggregation model based on the principle of minimum cross-entropy. Section 4 provides an illustrative example. Section 5 concludes and summarizes the main conclusions.

Preliminaries
In this section, an MAGDM problem is set up and its general solving steps are discussed. Then, we briefly review the entropy weighting technique and the principle of minimum cross-entropy.

Procedures of Solving MAGDM
A multiple-attribute group decision-making problem can be defined as a quadruple < A, C, D, X >, where: A = {a i | = 1, 2, · · · , m} is the alternative set for every decision-maker and is indexed by i and m 2; C = {c j |j = 1, 2, · · · , n} is the attribute set for each alternative, and attributes are assumed to be additive and independent in this paper for simplicity; D = {d k |k = 1, 2, · · · , l} is the decision-maker set; and X = {x ij |i = 1, 2, · · · , m; j = 1, 2, · · · , n} is the normalized value of the j-th attribute for the i-th alternative, i.e., The matrix X is the objective value of attributes. However, every decision-maker may have their own judgment on these values based on his or her preference. Hence, all decision-makers' judgment has to be integrated in order to solve an MAGDM problem. In this paper, we are going to introduce a utility function to express decision-makers' preference in accordance with the general approach in decision theory. Let u k (x) be the k-th decision-maker's utility function. Therefore, the problem confronted with the k-th decision-maker is: which can be viewed as a multiple attribute decision problem for the k-th decision-maker.
The general process to solving an MAGDM problem can be described as follows. Assume that the weight of attributes determined by the k-th decision-maker's judgment is denoted by β k = (β k 1 , β k 2 , · · · , β k n ). Thus, on the basis of the assumption of additivity and the independence of attributes, the valuation of alternative a i by the k-th decision-maker is: If the decision-makers' importance is defined by a vector w = (w 1 , w 2 , · · · , w l ), then the group decision on different alternatives is: Based on the value of s i , the alternatives can be ranked or the optimal alternative can be determined. In fact, by substituting Equation (3) into Equation (4), we can get that: The above equation demonstrates that the essence of solving an MAGDM problem can be formulated into the following four key procedures, which is depicted in Figure 1, as well.
Step 1 Step 2 Step 3 Step 4 Figure 1. The general steps of solving a multiple-attribute group decision-making (MAGDM) problem.
Step 1: Convert attribute value matrix X into G(X) to eliminate the difference of the unit and direction of attribute value, such as normalization and standardization methods.
Step 2: Determine the decision-makers' weight w and attributes' weight β. Generally speaking, these weights can be subjective, objective or their combination.
Step 3: Aggregate the decision-makers' weight w and attributes' weight β, which is denoted as F(w • β).
Step 4: Integrate the aggregated weight and the value of attributes, which is denoted by F(w • β) • G(X). Then, all alternatives can be ranked by the integration result.
In this paper, we will not investigate all of the procedures in Figure 1. Instead of that, we will focus on the critical procedures of Steps 2 and 3, namely determining and aggregating decision-makers' and attributes' weight.

Entropy Weighting Technique
The entropy weighting technique is a widely-used method to determine the weight of an attribute based on the differences between them without any additional or subjective information. The differences are measured by information-theoretic entropy.
Generally speaking, multiple attribute decision-making has m alternatives, and each alternative has n attributes. Let r ij be a non-negative value of the j-th attribute for the i-th alternative, such that a multiple attribute decision-making problem can be formalized into a matrix R as: In the entropy weighting technique, the entropy-based difference of the j-th attribute between alternatives is viewed as the foundation to determine the weight of attributes. When the difference of two alternatives about the j-th attribute is small, then this attribute does not provide sufficient information to rank or distinguish the two alternatives. Therefore, the less is the difference, the smaller is the weight. Mathematically, the weight of the j-th attribute in Equation (6) can be calculated out as: where E j is an extended and normalized entropy defined as: It is easy to find that 0 ω j 1 and ∑ n j=1 ω j = 1 according to the properties of entropy.

Principle of Minimum Cross-Entropy
Cross-entropy is a distance measure from one probability distribution to another. The principle of minimum cross-entropy (POMCE) was formulated by [31] and is detailed in [32]. Sometimes, it is also referred to as the Kullback-Leibler (K-L) principle. POMCE is also referred to as the principle of minimum discrimination information, the principle of minimum directed divergence, the principle of minimum distance or the principle of minimum relative entropy.
Let Q = {q 1 , q 2 , · · · , q N } be a probability distribution for a random variable X that takes N values. POMCE is to derive a distribution P = {p 1 , p 2 , · · · , p N } of X that takes all of the given information into consideration and makes the distribution as near to Q as possible. Mathematically, POMCE can be formulated as the following model.
where D(P||Q) is the definition of cross-entropy, a r is some known information about random variable X and f ri are functions defined with respect to a r . More information and properties of cross-entropy and POMCE can be found in [33].
In fact, although it is often viewed as a metric of distance, the cross-entropy is not a true metric; for example, it is not symmetric: the cross-entropy from P to Q is generally not the same as that from Q to P. In spite of that, the cross-entropy is still a very important basic divergence measure for probability distribution divergence and is applied in many related fields. In this paper, we will continue to take the cross-entropy as the divergence measure to aggregate different information from attributes and decision-makers.

Weight Aggregation Model Based on POMCE
In an MAGDM problem, determining decision-makers' weight and attributes' weight objectively and aggregating weights plausibly is of great importance. In this paper, we will propose a weight determination and aggregation method based on the entropy weighting technique and the principle of minimum cross-entropy. The framework is presented in Figure 2. First, the attribute value matrix X will be converted into U k by the decision-makers' utility function, which is used to show their subjective judgment on different attributes. Second, by utilizing the entropy weighting method, the subjective attribute weight β k can be derived from U k , respectively. β k are viewed as subjective weight vectors because they are obtained from matrix U k , which is different decision-makers' subjective judgment on attributes. Now, all β k can be formed into a subjective attribute weight matrix β. Then, in order to find a weight vector that can aggregate all decision-makers' judgment on the attribute weight, a model based on POMCE is developed to integrate β k 's into β 0 , which is thought of as the subjective weight vector of attributes integrating all decision-makers' judgment. Meanwhile, objective attribute weight α can be also calculated out by the entropy weighting technique based on the attribute value matrix X. At last, the subjective weight β 0 and the objective weight α are integrated similarly into ω by the established model based on POMCE.
Comparing to the general procedure in Figure 1, the proposed approach has two features. First, instead of using the group utility function, the individual utility function is applied to reflect each decision-maker's judgment on attributes, which may avoid the influences of some unfavorable situations where some decision-makers tend to compromise their own different opinions to keep consistent with the group. Second, by applying POMCE to integrate all decision-makers' judgment on the weight of attributes into a subjective weight vector, any subjective or additional information out of the decision system will be excluded, which may assure the plausibility and objectivity of weights.

Objective Weight of Attributes
By applying the entropy weighting technique onto attribute value matrix X, objective weight of attributes α can be obtained as: α = (α 1 , α 2 , · · · , α n ) (9) where: This weight vector fully depends on the data from the attribute value matrix without considering decision-makers' judgment on the value of attributes. Hence, it can be treated as the objective weight.

Subjective Weight of Attributes
Meanwhile, one has to consider decision-makers' attitudes about attributes to solve an MAGDM problem reasonably. In accordance with the general approach in decision theory, we use the utility function to reflect different decision-makers' preferences upon attributes. In the same way, the k-th decision-maker's subjective weight on attributes can be obtained by using the entropy weighting method for his or her subjective attribute matrix U k as defined in Equation (2).
Then, all decision-makers' weight vector can be formulated into a weight matrix β, such that: where: , j = 1, 2, · · · , n; k = 1, 2, · · · , l The weight vector β k fully depends on the k-th decision-maker's utility function, which reflects a subjective judgment on the importance of attributes. Hence, we viewed β as a subjective weight matrix.

Weight Aggregation
In order to combine all decision-makers' judgment, we introduced a minimum distance method based on POMCE. Let β 0 = (β 0 1 , β 0 2 , · · · , β 0 n ) be the attribute weight that integrates all decision makers' attitudes. For this purpose, β 0 should be as close as possible to every β k . Hence, we assume that the weighted sum of the cross-entropy distance of β 0 to β k should be minimized, such that: where w = (w 1 , w 2 , · · · , w l ) is the decision-makers' power vector and: which is the Kullback-Leibler distance or cross-entropy from β 0 to β k . To solve the above optimization model, define a Lagrange function as: where (λ − 1) is the Lagrange multiplier. The optimality conditions can be obtained by taking partial derivatives of β 0 j (j = 1, 2, · · · , n) and λ, such that: From Equation (16), we can get: By substituting the above equation into Equation (17), Then, it can be achieved that: By now, the proposed method is proven to be feasible. In the above model, the decision-makers' importance is assumed to be known. If it is unknown, the object function can be reformed into: Following the same procedure, it is easy to obtain a similar solution of β 0 . Furthermore, based on the result, the distance from β k to β 0 , namely D k (β k , β 0 ), can be calculated out. The smaller is D k (β k , β 0 ), the more consistent with the common knowledge is the k-th decision-maker's judgment. Hence, his or her judgment should be weighted more. Based on this consideration, we propose a method to determine decision-makers' power as: Therefore, the above model is feasible whether the decision-makers' weight is known or not. Anyway, different decision-makers' weights of attributes can be aggregated based on the minimum cross-entropy model. Furthermore, β 0 can be viewed as the subjective weight of attributes based on all decision-makers' judgment. As we proposed earlier, to solve MAGDM, one should combine the subjective weight β 0 and objective weight α. Assuming that the subjective and objective weight have different importance, let γ 0 denote the importance of subjective weight and 1 − γ be the importance of objective weight. The aggregated weight should be close to the subjective and objective weight. Therefore, we establish the following model to combine the subjective and objective weight. s.t.
where ω = (ω 1 , ω 2 , · · · , ω n ) is the aggregated weight. By using the same method as before, it is easy to obtain that: , j = 1, 2, · · · , n By now, the weight of attributes is determined by the minimum cross-entropy principle with respect to subjective and objective weight. Then, combining with the value of attributes, the MAGDM problem can be solved by other methods, such as the weighted sum method, the technique for order preference by similarity to ideal solution (TOPSIS) method, and so on.

Illustrative Example
In the following, an MAGDM problem of determining what kind of air-conditioning systems should be installed in a library (adapted from [34] and discussed by [35,36]) is used to illustrate the proposed method.
A city is planning to build a municipal library. One of the problems facing the city development commissioner is choosing from five feasible plans (a 1 , a 2 , a 3 , a 4 , a 5 ) the kind of air-condition system to be installed in the library. The alternatives are to be evaluated by three experts d k (k = 1, 2, 3) (whose weight vector is w = (0.3, 0.2, 0.5) under three major factors: economic, functional and operational. Two monetary attributes and six non-monetary attributes are considered. They are: where the * unit is from 0-1 scale, the three attributes c 1 , c 2 and c 4 are cost attributes and the other five attributes are benefit attributes.
In the original example, different decision-maker's judgment are given in Tables 1-3. There is no objective value matrix of attributes. In order to apply our approach, assume that the objective value matrix X is the same as U 1 in Table 1, i.e., assume that u 1 (x) = x.      Now, we apply the procedure in Figure 2 to solve the given MAGDM problem. The procedure is structured in the following phases.
Step 1. Apply the entropy weighting technique to the objective value matrix (in Table 1 Step 2. Again, apply the entropy weighting technique to the subjective value matrices (in Tables 1-3) to obtain the subjective weight matrix. The result is: Step 3. Apply the weight aggregation model as shown in Equations (12) and (13) Step 4. Aggregate the objective weight vector α and the subjective vector β 0 by applying the proposed model as in Equation (21) Step 5. Normalize the objective value matrix in Table 1. For benefit attributes, let: and for cost attributes, let: Then, the normalized decision matrix is obtained as shown in Table 4.  Step 6. By using the additive weighted aggregation (AWA) operator, the group decision values of the alternatives are: Our approach has some different characteristics compared to the approach in [35]. First, the method in [35] modifies every element in the decision matrix of every decision-maker at every step. This means that in reality, every time, the decision-makers have to give new preferences for every element in the decision matrix. However, in our approach, we only need decision-makers to give their preferences once at the beginning. This can reduce a large amount of calculation and is easy to apply in practice. Second, it is well-known that the weighting method is crucial and significant in a multiple attribute group decision-making problem, and different weights probably lead to different results. Hence, more and more attention is paid to objective approaches, which are assumed to be more credible. In our method, all weight vectors are determined by the objective method, even for the determination of the subjective weight vector, which assures the maximum objectivity. However, it is necessary to take decision-makers' subjectivity into consideration in MAGDM problems. In our approach, decision makers' subjective attitudes are expressed by their preference function, which is in accordance with the general approach in decision theory. This makes our approach more plausible.

Conclusions
Weight aggregation is the key process to solving an MAGDM problem. In this paper, by using the entropy weighting technique and the principle of minimum cross-entropy, the method of weight determination and aggregation is discussed. In accordance with the general approach in decision theory, the decision-makers' utility functions are introduced to reflect their preferences upon attributes. By using the entropy weighting technique, the subjective and objective weights of attributes are obtained. Then, based on the principle of minimum cross-entropy, an optimization model is developed to aggregate subjective weights and objective weights. The proposed approach presents a new method to objectivize subjective information and to aggregate information from attribute values themselves and decision-makers' judgment.