# A New Approach to Group Multi-Objective Optimization under Imperfect Information and Its Application to Project Portfolio Optimization

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

- Most interactive methods implicitly assume that group preferences are transitive and comparable relations, although the lack of transitivity is a well-established characteristic of voting systems (e.g., [19]). Even in the case of a single DM, transitivity, and comparability of their preference relation are subject to question, mainly in the presence of veto conditions, and/or when the number of objectives overcomes the cognitive limitations of the human mind.
- Many methods are susceptible to manipulation. According to classical voting theory, under very general conditions, every voting procedure can be manipulated by some voters by declaring insincere preferences (e.g., [19]).
- Popular interactive approaches help to obtain acceptable agreements because each DM learns the preferences from the other DMs and correspondingly fits their own. However, the final accepted solution may significantly differ from those that each DM would have considered as satisfactory if the decision had depended solely of them. Thus, the consensus does not result from the search in the set of possible solutions but from mutual concessions. Group satisfaction is partial because it is only achieved by recognizing that a more satisfactory result is not possible.
- The handle of imprecision, uncertainty, and ill-definition in GDM-MOP is a real concern. GDM-MOP approaches typically assume that the whole group agrees on the resource availability, the resource consumption, and objective values for each point in the decision variable space. However, there could be several sources of imperfect information which affect that assumption. Indeed, each DM may have their own perception (no free of certain imprecision, uncertainty, or arbitrariness) about objective values, available and required resources. Such imperfect knowledge may impact the individual best solutions, on the collective preferences, and the consensus degree. Under imperfect information, the consensus search process is even more difficult and relevant since the diverse perceptions from the DMs and different levels of conservatism should be aggregated and, if possible, agreed.
- In complex problems, some DMs with very different value systems and/or roles with respect to the other group members may consider different sets of objective functions and constraints. Such a case is not addressed by most of the methods to solve GDM-MOPs.

## 2. Background

#### 2.1. An Overview of GDM-MOP Literature

#### 2.2. Toward a Maximum Consensus

- (A)
- There is an important agreeing majority with a particular alternative (or solution);
- (B)
- There is no appreciable disagreeing minority.

_{g}members under the following general premise:

_{sat}(resp. N

_{dis}) denote the number of group members who are satisfied (resp. dissatisfied) by a particular solution z. N

_{sat}and N

_{dis}are integer functions of z. The ideal consensus (if possible) should correspond to a point z* such that N

_{sat}(z*) = n

_{g}and N

_{dis}(z*) = 0.

_{sat}(z), Minimize N

_{dis}(z)

_{sat}(z*) = n

_{g}and N

_{dis}(z*) = 0 as much as the preferences and beliefs of all the individual DMs become close enough through an effective consensus reaching process. Even when the ideal consensus is not possible, good agreements can be identified when N

_{sat}≥ 2/3 n

_{g}(Condition A) and N

_{dis}≈ 0 or N

_{dis}<< 0.5 n

_{g}(Condition B).

_{sat}and N

_{dis}. Therefore, we require a model that permits, for each DM and each z in the decision variable space, knowing to which class of satisfaction/dissatisfaction z is assigned to. Most of the remaining paper is devoted to discussing some preference models and assignment procedures to this end. Since in the objective space, z is described by multiple objective values, its assignment should be considered as a multi-criteria ordinal classification problem.

#### 2.3. Some Fundamental Notions on Interval Mathematics

**E**= [$\underset{\_}{E}+\stackrel{\_}{E}$], where $\underset{\_}{E}$ is the lower limit and $\overline{E}$ is the upper limit. In the rest of this paper, an interval number is denoted with boldface italic letters.

**D**and

**E**two interval numbers and F a real number. Basic arithmetic operations are defined below.

**E**. In [20], P (

**D**≤

**E**) = α is interpreted as the degree of credibility that once two realizations are given from

**E**and

**D**, the realization d will be smaller than or equal to the realization e.

_{α}) on interval numbers is defined in Equation (10).

#### 2.4. Multi-Criteria Ordinal Classification Based on an Interval Outranking Approach

**Definition**

**1.**

- (i)
- xS(β,
**λ**)y ⟺ σ(x,y,**λ**) ≥ β (interval outranking); - (ii)
- xP
_{r}(β,**λ**)y ⟺ σ(x,y,**λ**) ≥ β and σ(y,x,**λ**) < β (interval preference); - (iii)
- xI(β,
**λ**)y ⟺ σ(x,y,**λ**) ≥ β and σ(y,x,**λ**) ≥ β (interval indifference);

- σ is the credibility index of the interval outranking;
**λ**is an interval number representing a majority threshold;**λ**> [0.5, 0.5] and λ^{min}≥ 0.5;- β is a credibility threshold for establishing a credible crisp outranking relation; β > 0.5.

_{1}and C

_{2}be ordered classes (C

_{2}is the most preferred). Set β > 0.5 and

**λ**> [0.5, 0.5] with λ

^{min}≥ 0.5. The boundary between C

_{1}and C

_{2}is characterized by a set of limiting profiles, B = {b

_{j}}, such that:

- (i)
- All b
_{j}of B belongs to C_{2}; - (ii)
- There is no pair (b
_{j}, b_{i}) such that b_{j}P_{r}(β,**λ**)b_{i}.

**Definition**

**2.**

- (i)
- xS(β,
**λ**)B iff there is a w ∈ B such that xS(β,**λ**)w and there is no y ∈ B with yP_{r}(β,**λ**)x; - (ii)
- BP
_{r}(β,**λ**)x iff there is a w ∈ B such that wP_{r}(β,**λ**)x and there is no y ∈ B with xP_{r}(β,**λ**)y.

**Definition**

**3 (“pessimistic” rule).**

- Step 1: Compare x to B;
- Step 2: If xS(β,
**λ**)B_{,}then assign x to class C_{2}; - Step 3: If not(xS(β,
**λ**)B), then assign x to C_{1}.

**Definition**

**4 (“optimistic” rule).**

- Step 1: Compare x to B;
- Step 2: If BP
_{r}(β,**λ**)x, then assign x to class C_{1}; - Step 3: If not(BP
_{r}(β,**λ**)x), then assign x to C_{2}.

## 3. Characterization of GDM-MOPs Under-Study

- There is a group moderator who is in charge to control and guide the consensus reaching process;
- The individual DMs participate in the decision process providing information about their preferences, beliefs, and level of conservatism, and modifying this information during consecutive steps of the CRP;
- Some (even all) objective values may be not the same for different DMs;
- Each DM may handle a different set of objective functions;
- The objective values may be imperfectly known (subject to imprecision or uncertainty);
- The availability of resources may be imperfectly known;
- Resource requirements per activity (project, in case of portfolio optimization) may be imperfectly known;
- Each group member has their own opinion about the availability of resources and requirements per activity (project, in case of portfolio optimization);
- All the DMs consider a common point in the decision variable space.

**Assumption**

**1.**

**AR**

^{j}_{i}be the interval number used to represent the estimated availability of the j-th resource from the point of view of the i-th DM;

**NR**

^{j}

_{i}is the interval number used to denote the aggregation of required resources, which depends on the decision variables vector z.

**AR**

^{j}_{i}> ξ

^{i}

**NR**

^{j}

_{i}(z) according to the order relation given by Equation (10). The i-th DM faces the MOP in Equation (11).

_{i}is the number of the evaluation criterion for the i-th member of a group; F

_{i}is the vector function being considered by the i-th DM; z is the vector of decision variables; in project portfolio optimization, z is a vector composed of binary variables, in which “1” means “the related project belongs to the portfolio”, and “0” otherwise.

_{Fi}is the feasible region from the point of view of the i-th DM. It is defined by the interval-based resource constraints

**AR**

^{j}_{i}> ξ

^{i}

**NR**

^{j}

_{i}(z), and perhaps other non-resource constraints.

^{i}is the credibility threshold for the expression “the available resources are sufficient to satisfy the requirements”. The more conservative the i-th DM, the higher the value of ξ

^{i}.

**Definition**

**5 (α-dominance).**

_{i}be the image of the feasible region of Problem 11. Consider an element (x,y) belonging to O

_{i}× O

_{i}, where x ≠ y, and α a real number in [0.5, 1]. The solution y is α-dominated by x (denoted by xD(α)y) iff min{P(

**f**(x) ≥

_{ji}**f**(y)), j = 1, …, N

_{ji}_{i}} = α and P(

**f**(x) ≥

_{ki}**f**(y)) > 0.5 for some k.

_{ki}**Definition**

**6 (a preferred solution by the i-th DM).**

_{i}is preferred to y∈ O

_{i}by the i-th DM if the following statement is considered as true by them: “x is at least as good as y, but y is not at least as good as x”.

**Remark**

**1.**

_{r}(β,

**λ**)y) and the dominance relation from Definition 5 (xD(α)y) are good arguments in support to “x is preferred to y” for sufficiently high values of β and α.

**Definition**

**7 (necessary conditions to be the best compromise solution for the i-th DM).**

_{i}* ∈ O

_{i}is the best compromise solution to Problem 11 for the i-th DM only if it fulfills two conditions:

- (A)
- There is no y in O
_{i}such that y is preferred to x_{i}* by the i-th DM; - (B)
- There are arguments to justify that the i-th DM considers x
_{i}* as at least as good as many solutions that satisfy Condition A.

**Assumption**

**2 (capacity to identify the best compromise solution from the i-th DM’s point of view).**

_{i}be the image of the feasible region of Problem 11. Each individual DM is able to solve Problem 11, thus identifying their best compromise solution x

_{i}* ∈ O

_{i}.

## 4. Model of Preferences and Judgments of a DM with a Non-Compensatory Aggregation of Preferences

#### 4.1. The Interval Outranking Model

**Assumption**

**3 (modeling arbitrariness and ill-definition of parameters).**

**w**

_{j}= [w

_{j}

^{−}, w

_{j}

^{+}], the veto thresholds as

**v**

_{j}= [v

_{j}

^{−}, v

_{j}

^{+}], the majority threshold as

**λ**= [λ

^{−}, λ

^{+}], and the credibility threshold as

**β**= [β

^{−}, β

^{+}].

_{j}(a′, a) of the assertion “a′ is at least as good as a regarding criterion f

_{j}” is

_{j}(a′, a) = P(f

_{j}(a′) ≥ f

_{j}(a)) j = 1, …, N

_{j}is γ-concordant with the statement “a′ is at least as good as a regarding criterion f

_{j}” (denoted a′S

_{j}a) if and only if a′S

_{j}a with a credibility index of at least γ. The set of criteria f

_{j}such that δ

_{j}(a′, a) ≥ γ (γ = min{δ

_{j}}) is called γ-possible concordance coalition with “a′ is at least as good as a” and is denoted by C(a′S

_{γ}a). γ is the credibility that all criteria in C(a′S

_{γ}a) agree that “a′ outranks a”. All criteria not in C(a′S

_{γ}a) form the γ-possible discordance coalition, D(a′S

_{γ}a).

^{−}(a′, a), c

^{+}(a′, a)], and calculated as follows:

_{γ}a) is determined by γ, the concordance index c depends on such a value. Such a dependence is denoted by c(a′, a, γ).

_{γ}a) provides reasons in favor of “a′ outranks a”, and each possible discordance coalition D(a′S

_{γ}a) provides reasons against it. In the following, we denote as d

_{j}(a′,a) the credibility index of the statement “a′Sa is vetoed by g

_{j}”. d

_{j}(a′,a) is defined as P(f

**(a) ≥ f**

_{j}_{j}(a′) + v

_{j}), where v

_{j}is the interval veto threshold related to f

_{j}.

**Definition**

**8.**

_{j}> 0, j = 1, …, N}. Given $\gamma \in \Omega $ we say that a′ outranks a for the γ-possible concordance coalition with credibility index σ

_{γ}and majority threshold

**λ**> 0.5 with λ

^{−}> 0.5 iff σ

_{γ}= min {γ, P(

**c**(a′, a, γ) ≥

**λ**), (1 − $\underset{fj\in D\left({a}^{\prime}{S}_{\gamma}a\right)}{max}$ d

_{j}(a′,a))}.

**Definition**

**9 (interval outranking credibility index).**

**λ**) = max{σ

_{γ}} (γ ∈ Ω) and majority threshold

**λ**> 0.5 with λ

^{−}> 0.5. If Ω is empty, then σ(a′,a,

**λ**) is zero.

**β**an interval number (0.5 <

**β**< 1 with β

^{−}> 0.5) considered as a credibility threshold to establish crisp preference relations. In the following we will use a′S(

**β,λ**)a, a′I(

**β,λ**)a, and a′P

_{r}(

**β,λ**)a similarly to Definition 1, but using the interval number

**β**instead of the real number β. Such a change gives more flexibility to the DM in setting the credibility threshold.

**Remark**

**2.**

- (a)
- As was proved in [20], if f
_{j}(a) are real numbers for j = 1, …, N, then aS(1,1)a. - (b)
- a′s(
**β,λ**)a ⇒ a′S(ξ,**λ**)a ∀ ξ <**β**.

**Proposition**

**1.**

- i.
- yD(α)x and xS(
**β**,**λ**)z ⇒ yS(**ε**,**λ**)z for some**ε**≥ min(α,**β**) - ii.
- zP
_{r}(**β**,**λ**)y and yD(α)x ⇒ zP_{r}(**ε**,**λ**)x for some**ε**≥ min(α,**β**) - iii.
- yD(α)x and xP
_{r}(**β**,**λ**)z ⇒ yP_{r}(**ε**,**λ**)z for some**ε**≥ min(α,**β**) - iv.
- If α > 0.5, then xD(α)y ⇒ xP
_{r}(**β**,**λ**)y for some**β**≥ α and for all**λ**≤ [1,1] - v.
- xD(α)y and yD(η)z ⇒ xD(
**ε**)z with**ε**= min (α,η)

**β**.

#### 4.2. Finding the Best Compromise solution to Problem 11

**Definition**

**10 ((β**,

**λ) non-strictly outranked solution).**

_{r}(

**β**,

**λ**)x is called a (

**β**,

**λ**) non-strictly outranked solution. The (

**β**,

**λ**) non-strictly outranked frontier of Problem 11 is the set of these solutions.

**Remark**

**3.**

- (i)
- With appropriate values of
**β**and**λ**, a non-strictly outranked solution fulfills Condition A of Definition 7, that is, the first necessary condition to be the best compromise. - (ii)
- Condition B of Definition 7 is proved on the non-strictly outranked frontier, using the outranking strength measure. This measure is described as OS(x) = card {a
_{i}∈ NSF such that xS(**β,λ**)a_{i}}, where NSF denotes the (**β**,**λ**) non-strictly outranked frontier. - (iii)
- More than one solution can fulfill the necessary conditions of Definition 7. The solution selected as the final best compromise should be one with the highest value of the outranking strength.

- The individual DM (perhaps helped by a decision analyst) sets their model parameters according to Assumption 3.
- The (
**β**,**λ**) non-strictly outranked frontier is identified by an optimization algorithm; the set of solutions that fulfill Definition 10 is determined; - The DM selects the best compromise solution according to Remark 3.iii.

#### 4.3. Making Judgments of Satisfaction and Dissatisfaction

**Assumption**

**4 (capacity to set the limiting boundary between classes).**

_{sat}and C

_{dis}denote classes “satisfactory” and “unsatisfactory”, respectively. Consider

**β**> 0.5 (β

^{−}> 0.5) and

**λ**> 0.5 (λ

^{−}> 0.5). Taking x* as reference, the DM is able to set a boundary B= {b

_{1}, … b

_{n}} between C

_{sat}and C

_{dis}fulfilling:

- i.
- f
_{j}(b_{k}) are real numbers, j = 1, … N and k = 1, … n; - ii.
- Each b
_{k}(k = 1, … n) belongs to C_{sat}; - iii.
- For all b
_{k}belonging to B, we have x*P_{r}(**β**,**λ**)b_{k}; - iv.
- There is no pair (b
_{i}, b_{k}) such that b_{i}P_{r}(**β**,**λ**)b_{k}.

**β**. Assumption 4 guarantees the conditions to apply INTERCLASS-nB (see Section 2.4) to making judgments of satisfaction and dissatisfaction.

**β**,

**λ**)B ⇒ not (BP

_{r}(

**β**,

**λ**)x) and BP

_{r}(

**β**,

**λ**)x ⇒ not (xS(

**β**,

**λ**)B). If a solution x outranks the limiting boundary B, x is assigned to the best class by the “pessimistic” procedure (Definition 3); additionally, since xS(

**β**,

**λ**)B ⇒ not (BP

_{r}(

**β**,

**λ**)x), x is also assigned to C

_{sat}by the “optimistic” procedure (Definition 4) contrarily, if the boundary is preferred to x, since BP

_{r}(

**β**,

**λ**)x ⇒ not (xS(

**β**,

**λ**)B), x will be assigned to the worst class by both assignment rules. If not (xS(

**β**,

**λ**)B and not (BP

_{r}(

**β**,

**λ**)x) are both fulfilled, then x is assigned to C

_{dis}by the “pessimistic” rule and to C

_{sat}by the “optimistic” procedure. In such a case, the DM may be doubtful about the class x should be assigned to.

**Assumption**

**5 (compatibility with INTERCLASS-nB).**

**β**,

**λ**)B or not (BP

_{r}(

**β**,

**λ**)x), the nonfulfillment of the constraints vetoes a satisfactory assignment. Based on this assumption, the definitions of what is a satisfactory (non-satisfactory) solution for a DM (who is compatible with the interval outranking model) are straightforward.

**Definition**

**11.**

- (a)
- xS(
**β**,**λ**)B - (b)
- x and its pre-image satisfy the constraints imposed by the DM.

**Definition**

**12.**

- (A)
- BP
_{r}(**β**,**λ**)x - (B)
- x and/or its pre-image do not satisfy the constraints imposed by the DM.

**Definition**

**13.**

- 1.
- not (xS(
**β**,**λ**)B) - 2.
- not (BP
_{r}(**β**,**λ**)x) - 3.
- x and its pre-image satisfy the constraints imposed by the DM.

**λ**) = max {σ(x, b

_{k},

**λ**), k = 1, … card(B)

**Proposition**

**2 (consistency properties of assignments).**

- (a)
- x is assigned to a single element of the set of classes (satisfactory, unsatisfactory, neither satisfactory nor unsatisfactory).
- (b)
- The assignment suggested for x is independent of the assignment of other solutions.
- (c)
- The class to which x is assigned by the i-th DM is independent of the assignment made by any other DM.
- (d)
- Let y be a feasible solution. Given
**λ**, if x and y have the same interval outranking credibility indices with respect to the limiting profiles, then they are assigned to the same element of the set of classes (satisfactory, unsatisfactory, neither satisfactory nor unsatisfactory). - (e)
- If there is b
_{k}∈ B fulfilling x = b_{k}, then x is assigned to the satisfactory class. - (f)
- If there is b
_{k}such that xI(**β**,**λ**)b_{k}and there is no b_{i}∈ B fulfilling b_{i}P_{r}(**β**,**λ**)x, then x is assigned to the satisfactory class. - (g)
- If x = x*, then x is assigned to the satisfactory class.
- (h)
- Let y be a feasible solution such that y D(α)x (α ≥
**β**). If x is assigned to the satisfactory class, then y is assigned to the same class.

**Proof.**

- Proposition 2(a):
- The proof follows from two facts: (i) x has to fulfill one of the three Definitions 11–13, and (ii) the fulfillment of one definition excludes fulfillment of another.
- Proposition 2(b):
- The proof is obvious from Definitions 11–13.
- Proposition 2(c):
- The proof is obvious from Definitions 11–13.
- Proposition 2(d):
- The proof is obvious from Definitions 11–13.
- Proposition 2(e):
- x = b
_{k}, Assumption 4.i, Remark 2.a, Definition 2.i and Assumption 4.iv ⇒ xS(**β**,**λ**)B x is feasible and xS(**β**,**λ**)B ⇒ x is satisfactory for the DM (Definition 11). - Proposition 2(f):
- xI(
**β**,**λ**)b_{k}⇒ xS(**β**,**λ**)b_{k}(Definition 1.iii)xS(**β**,**λ**)b_{k}and there is no b_{i}∈ B such that b_{i}P_{r}(**β**,**λ**)x ⇒ xS(**β**,**λ**)B (Definition 2.i)x is feasible and xS(**β**,**λ**)B ⇒ x is satisfactory for the DM (Definition 11). - Proposition 2(g):
- The proof follows trivially from Assumption 4.iii, Definition 2.i and Definition 11.
- Proposition 2(h):
- x is assigned to C
_{sat}⇒ xS(**β**,**λ**)B from Definition 11 ⇒ ∃ b_{k}∈ B such that xS(**β**,**λ**)b_{k}and there is no b_{i}∈ B with b_{i}P_{r}(**β**,**λ**)x.

- From Proposition 1.i and Remark 2.b, yD(α)x (α ≥ β) and xS(
**β**,**λ**)b_{k}⇒ yS(**β**,**λ**)b_{k}. - There is no b
_{i}∈ B with b_{i}P_{r}(**β**,**λ**)x and y D(α)x (α ≥ β) ⇒ There is no b_{i}∈ B with b_{i}P_{r}(**β**,**λ**)y counter-reciprocal of Proposition 1.ii.

**β**,

**λ**)B (Definition 2.i) ⇒ y is assigned to C

_{sat}(Definition 11). The proof is finished.

_{k}∈ B should be considered as satisfactory by the DM.

## 5. Model for a DM Whose Preferences Are Compatible with a Weighted-Sum Function

#### 5.1. The Preference Model

**U**= Σw

_{j}f

_{j}(j = 1, … N)

**U**is an interval number that is calculated through the arithmetic operations defined in Section 2.3. We refer to a generic DM, and for simplicity, we avoid the use of the subscript “i”.

**Assumption**

**6 (modeling arbitrariness and ill-definition of parameters).**

**Remark**

**4.**

**U**(x) ≥

**U**(y)) > 0.5 or xD(0.5)y may not suffice to guarantee a credible preference favoring x over y. However, there should be a credibility threshold α > 0.5 such that P(

**U**(x) ≥

**U**(y)) ≥ α and/or xD(α)y are good arguments to justify a credible preference.

**Definition**

**14 (α-preference).**

- a.
- P(
**U**(x) ≥**U**(y)) ≥ α - b.
- xD(α)y

#### 5.2. Identifying the Best Compromise Solution to Problem 11 with the Functional Preference Model

**Definition**

**15 (α non-strictly outranked solution).**

**Remark**

**5.**

- With an appropriate value of α, a non-strictly outranked solution from Definition 15 fulfills Condition A of Definition 7, that is, the first necessary condition to be the best compromise solution of Problem 11.
- Condition B of Definition 7 is verified through a value strength measure on the non-strictly outranked frontier. This measure is defined as VS(x) = card {y ∈ NSF such that x is 0.5-preferred to y}, where NSF denotes the α non-strictly outranked frontier.
- Several solutions can fulfill the necessary conditions of Definition 7. The final best compromise should be one of the solutions with the highest measure VS.

- The individual DM (perhaps helped by a decision analyst) sets the interval weights in Equation 18 and the α value.
- An optimization algorithm is used to identify the α non-strictly outranked frontier.
- The set of solutions that fulfill Definition 10 is identified.
- The DM selects the best compromise solution according to Remark 5.III.

#### 5.3. Making Judgments of Satisfaction and Dissatisfaction with the Functional Model

**Assumption**

**7 (capacity to set the limiting boundary between classes).**

_{sat}and C

_{dis}denote classes “satisfactory” and “unsatisfactory”, respectively. Consider a sufficiently high value of the credibility threshold α. Taking x* as reference, the DM is able to set a limiting boundary B= {b

_{1}, … b

_{n}} between C

_{sat}and C

_{dis}fulfilling:

- i.
- For all b
_{k}belonging to B, we have x* is α-preferred to b_{k}; - ii.
- There is no pair (b
_{i}, b_{k}) in B × B such that b_{i}is α-preferred to b_{k}; - iii.
- For all b
_{k}belonging to B, the DM hesitates about its assignment.

**Definition**

**16 (α-preference between a solution and the boundary).**

- (a)
- x is α-preferred to B iff there is b
_{k}∈ B such that x is α-preferred to b_{k}and there is no b_{i}∈ B such that b_{i}is α-preferred to x. - (b)
- The boundary B is α-preferred to x iff there is b
_{k}∈ B such that b_{k}is α-preferred to x and there is no b_{i}∈ B such that x is α-preferred to b_{i}.

**Definition**

**17.**

_{k}∈ B. The credibility index of the statement “x is preferred to b

_{k}” is defined as σ

_{P}(x, b

_{k}) = Maximum χ fulfilling at least one of the following conditions:

- a.
- P(
**U**(x) ≥**U**(b_{k})) = χ - b.
- xD(χ)b
_{k}

_{P}(x, B) of the statement “x is preferred to the boundary B” is defined as:

_{P}(x, B) = Max {σ

_{P}(x, b

_{k})}, k = 1, … card (B)

_{P}(x, B) can be interpreted as the degree of credibility of the statement “x is preferred to the boundary by the i-th DM”.

**Definition**

**18.**

- -
- x is α-preferred to the Boundary B;
- -
- x and its pre-image fulfill the constraints imposed by the DM.

**Definition**

**19.**

- -
- B is α-preferred to x;
- -
- x and/or its pre-image do not fulfill the constraints imposed by the DM.

**Definition**

**20.**

- -
- x is not α-preferred to the Boundary B;
- -
- B is not α-preferred to x.

**Proposition**

**3 (consistency properties of assignments).**

- (a)
- x is assigned to a single element of the set of classes (satisfactory, unsatisfactory, neither satisfactory nor unsatisfactory).
- (b)
- The assignment suggested for x is independent of the assignment of other solutions.
- (c)
- The class to which x is assigned to by the i-th DM is independent of the assignment made by any other DM.
- (d)
- If there is b
_{k}∈ B fulfilling x = b_{k}, then x is neither satisfactory nor unsatisfactory. - (e)
- If x = x*, then x is assigned to the satisfactory class.
- (f)
- Suppose that x is α-preferred to all b
_{k}∈ B. Let y be a feasible solution such that yD(α)x. Then y is assigned to the satisfactory class.

**Proof.**

- Proposition 3(a):
- The proof is obvious from Definitions 16, 18, 19, and 20.
- Proposition 3(b):
- The proof is obvious from Definitions 16, 18, 19, and 20.
- Proposition 3(c):
- The proof is obvious from Definitions 16, 18, 19, and 20.
- Proposition 3(d):
- From the way in which B is built (Assumption 7), there is no b
_{i}∈ B such that b_{k}is α-preferred to b_{i}or b_{i}is α-preferred to b_{k}. Then, b_{k}is not α-preferred to B and B is not α-preferred to b_{k}(Definition 16). From Definition 20, it follows that b_{k}is neither satisfactory nor unsatisfactory. - Proposition 3(e):
- The proof follows from Definitions 16 and 18 and the way in which the limiting boundary is built (Assumption 7).
- Proposition 3(f):
- It is evident that y D(α)x and P(
**U**(x) ≥**U**(b_{k})) ≥ α ⇒ P(**U**(y) ≥**U**(b_{k})) ≥ α. In addition yD(α)x and xD(α)b_{k}⇒ yD(α)b_{k}from transitivity of dominance (Proposition 1.v). Hence, y is α-preferred to the Boundary B (Definitions 14 and 16). From Definition 18, y is assigned to the satisfactory class. The proof is finished.

**Remark**

**6.**

- (a)
- Unlike the proposal in Section 4, the solutions in the limiting boundary do not belong to any class. Objectives of these solutions may be described by interval numbers, what gives more flexibility to the DM and may reduce their cognitive effort.
- (b)
- Proposition 3.d is consistent with Condition iii of Assumption 7. It follows that a solution only slightly different from any b
_{k}∈ B should be considered neither satisfactory nor unsatisfactory by the DM.

## 6. Summary of the Method

_{sat}, N

_{dis}) in Problem 1, but the calculation of these objectives is made by aggregating the independent assignments of all the DMs.

_{sat}(z*) and reduce N

_{dis}(z*) in the points z* where they are really satisfied, and to decrease N

_{sat}(z**) and increase N

_{dis}(z**) in the points z** where they are really dissatisfied. The information required from them is basically of three kinds: (i) parameters of their preference model; (ii) solutions describing the limiting boundary between classes “satisfactory” and “unsatisfactory”; (iii) their constraints and objective values. This information determines if the DMs are satisfied or dissatisfied and counted in N

_{sat}or N

_{dis}. If they report insincere information on any of the three previous aspects, solutions that are really satisfactory for them could become unsatisfactory according to the model, and vice versa. This does not contribute to reaching a final solution really satisfactory for the DMs that provide insincere information. In addition, each DM should be impeded to know the limiting boundaries elicited by other DMs. Under this restriction, the i-th DM cannot evaluate solutions from the point of view of other DMs, being thus unable to determine what insincere information should provide for maliciously influencing other opinions. Therefore, during collective discussions, the i-th DM should defend their real preferences and beliefs and try to convince others.

- Helped by a decision analyst/moderator, the DMs select the model of multi-criteria preferences that they consider as more appropriate. The group is separated into two disjoint subgroups in correspondence to the model of preferences that were chosen by each DM.
- In each subgroup, under the guidance of the moderator, the DMs bring their positions closer. They exchange opinions about the objective functions to consider, the objective values, the related model’s parameter values, levels of conservatism, and constraints.
- Each group member sets their multi-objective optimization problem (Problem 11) and their model’s parameter values. Interval numbers can be used according to Assumptions 1, 3, and 6.
- According to Assumption 2, each group member obtains their best compromise solution by solving Problem 11.
- Each DM sets their limiting boundary in correspondence to Assumption 4 (when the DM is compatible with the outranking model) and Assumption 7 (for DMs compatible with the functional model).
**If there is no solution of good agreement**, further discussions in each subgroup are needed to close divergent beliefs, preference parameters, and constraint settings. We need to update these data.- The DMs should judge whether, given the updated data, they want to modify their limiting boundary. In the case of “yes”, restart the process in Step 5. In the case of “no”, restart the process in Step 6.
**If a good consensus (N**_{sat}, N_{dis})* is found, then: - If the pre-image of (N
_{sat}, N_{dis})* is a single point in the decision variables space, this point corresponds to the best consensus, and the process finishes.**Else:** - Apply some additional criterion to select a single pre-image of (N
_{sat}, N_{dis})*. The process finishes.

**Remark**

**7.**

- i.
- In the case of project portfolio optimization, the computational cost depends mainly on Step 4. The computational complexity of this step is linear with respect to the number of applicant projects (see the description of the I-NOSGA algorithm in Appendix A).
- ii.
- Handling group interactions in Steps 1, 2, and 7 is the main difficulty to extend the proposal to problems with many DMs. In such problems, Steps 3, 4, 5, and 8 should be performed by each DM in an independent and parallel way. Parallel processing in Step 4 is strongly recommended. Steps 9 and 10 are independent of the number of group members. The computational effort in Step 6 is, at most, proportional to the number of DMs (see Appendix C). Therefore, with some modifications in Steps 1, 2, and 7, the proposal can be used in large-scale GDM-MOPs.
- iii.
- In Step 7, in order to accept a solution as a good consensus, the group may agree previously appropriate values of N
_{sat}and N_{dis}to represent what a strong majority and a weak minority mean, respectively. - iv.
- In Step 10, there could be several (even many) pre-images of the best consensus (N
_{sat}, N_{dis})*. To choose a single one, the group and/or its moderator can use different points of view (e.g., impacts of the solutions, resource consumption, who are the satisfied and dissatisfied DMs, number of supported projects in case of portfolio optimization, etc.). Perhaps the most elegant way is a logical approach based on the outranking credibility index of a solution with respect to the limiting boundary (see Equations (17) and (19)).

_{G}

_{1}, …, z

_{GL}the points in the decision variable space, which are all pre-image of the best consensus (N

_{sat}, N

_{dis})*. Let C

_{ag}denote the agreeing coalition (the set of group members who are satisfied with the image of z

_{Gk}). In the paragraph below, we use “i” (respectively “j”) for the DMs who are compatible with the outranking (resp. weighted sum functional) model.

_{Gki}(resp. Z

_{Gkj}) the image of z

_{Gk}in the original objective space O

_{i}(resp. O

_{j}), σ

_{i}(Z

_{Gki}, B

_{i},

**λ**

_{i}) (resp. σ

_{Pj}(Z

_{Gkj}, B

_{j})) can be interpreted as the degree of truth of the predicate “the i-th DM (resp. the j-th DM) considers that Z

_{Gki}(resp. Z

_{Gkj}) outranks the limiting boundary B

_{i}(resp. B

_{j})”. Then, the degree of truth of the predicate “all the DMs belonging to the agreeing coalition consider that Z

_{Gki}(resp. Z

_{Gkj}) outranks the related limiting boundary” can be calculated as the conjunction of all the values σ

_{i}(Z

_{Gki}, B

_{i},

**λ**

_{i}) and σ

_{Pj}(Z

_{Gkj},B

_{j}), where “i” (resp. “j”) is the index of the i-th DM (resp. the j-th DM) within the agreeing coalition. Such a truth value is calculated as:

_{sat}(z

_{Gk}) = min{σ

_{i}(Z

_{Gki}, B

_{i},

**λ**

_{i}), σ

_{Pj}(Z

_{Gkj}, B

_{j})}

i, j ∈ C

_{ag}

**Remark**

**8.**

- To set the limiting boundaries could be a hard cognitive task for DMs; it would be more complex in large scale problems.
- The bi-objective measure of collective satisfaction/dissatisfaction does not contain information about which DMs are strongly (dis)satisfied. This information can be important to discriminate among non-dominated solutions of Problem 1. Perhaps the multi-criteria ordinal classification method should take into account more classes of satisfaction/dissatisfaction, but this would require much more cognitive effort from the DMs in defining more limiting boundaries.
- The role of the moderator is crucial in choosing the final consensus decision among the non-dominated solutions of Problem 1. A model of consensus agreed by the group would be welcome. Such a model should aggregate the information about satisfaction/dissatisfaction, thus helping to make the final choice among non-dominated solutions to Problem 1.

## 7. An Illustrative Example of Project Portfolio Optimization

#### 7.1. Solution When All the DMs Accept the Interval Outranking Model and Its Assumptions

_{sat}and C

_{dis}. Each boundary is composed of three solutions that fulfill the conditions in Assumption 4. The boundaries are described in Table A3 in Appendix B.

_{sat}= 9 and N

_{dis}= 0. In the objective space, such solutions are given in Table 3. The time consumed by MOEA/D to solve Problem 1 was 35 min and 59 s.

_{j}(j = 1, …, 9), upper bound of w

_{j}(j = 1, …, 9), lower bound of v

_{j}(j = 1, …, 9), upper bound of v

_{j}(j = 1, …, 9), lower bound of β, and upper bound of β, α, ξ}.

Algorithm 1. Consensus Round Simulation |

For each p in MP Let p _{i} be the value of Parameter p set by the i-th DM.Let p _{a} denote the average value of p on the set of DMs.Repeat from i = 1 to 10 Calculate d = p _{i} − p_{a}If d > 0, update p _{i} as p_{i} − d/2If d < 0, update p _{i} as p_{i} + |d/2|If d = 0, then p _{i} keeps its valueEnd of Repeat End of For |

_{sat}= 10, N

_{dis}= 0 is achieved, with many pre-images in the decision variable space. In this example, the solutions satisfying the ideal consensus are ordered following the values of μ

_{sat}in Equation (20). The first ranked solutions are shown in Table 4.

_{i}(Z

_{Gki}, B

_{i},

**λ**

_{i}) from Equation 17 are provided by Table A6 (see Appendix B). These values represent measures of the level of satisfaction for each group member.

#### 7.2. Solution When All the DMs Accept the Interval Dum Function Model and Its Assumptions

_{sat}and C

_{dis}. Each boundary is composed of three solutions that fulfill the conditions in Assumption 7. The boundaries are detailed in Table A7 in Appendix B.

_{sat}and N

_{dis}is straightforward through Definitions 18–20. Using MOEA/D (see Appendix C), a single non-dominated solution of Problem 1 was found, with N

_{sat}= 6, N

_{dis}= 0. In the original nine-objective space, this solution is given in Table 6.

_{j}(j = 1, …, 9), upper bound of w

_{j}(j = 1, …, 9), α, ξ}. Since the simulation algorithm is identical, the updated parameters are the same as in Table A4a,c. The interval budget required by projects is updated identically to the way followed in Section 7.1 (see the updated values in Table A5, Appendix B).

_{sat}= 10 and N

_{dis}= 0. To select a small subset of solutions, we use the “min” operator for conjunction as in Equation 20. The first ranked consensus solutions are provided in Table 7.

_{Pi}(ZG

_{ki}, B

_{i}) from Equation 19 are shown in Table A8 (see Appendix B). These represent the level of satisfaction for each DM.

**Remark**

**9.**

## 8. Concluding Remarks

- Since consensus is affected by intense (dis)satisfaction, we require to model also high satisfaction and strong dissatisfaction. This could be addressed by multi-criteria ordinal classification methods, but the model would be more complex due to the increment of classes.
- Development of models for making an appropriate selection of the best consensus among non-dominated solutions of Problem 1. Logic-based models representing predicates like “a strong majority is satisfied with …” and “a very weak minority disagrees with …” may be used to select one of the non-dominated solutions in the space of collective satisfaction/dissatisfaction. This would permit to reduce, perhaps replace, the role of the moderator in choosing the final decision. These models should be able to reflect intense satisfaction and dissatisfaction from the group members.
- To alleviate the DM hard cognitive task in assessing limiting boundaries, this is more relevant as the numbers of objective functions and DMs increase.

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Appendix A. The Extended I-NOSGA Method

_{T}

_{+1}(see Lines 5–11). The algorithm orderly includes complete fronts in Pop

_{T}

_{+1}(Line 6) and complete it with the best solutions from the last front F

_{i}that does not fit entirely; the solutions are taken in order according to the specified SM (see Lines 9 to 11). Finally, a new generation of individuals QT+1 is evolved from Pop

_{T}

_{+1}using the genetic operators chosen for this purpose (see Line 12).

Algorithm A1. Interval Non-Outranked Sorting Genetic Algorithm | |

Input:Pop _{T}, the population of parents,Q _{T}, the children generated in the previous iteration,PM, the binary preference model used to compare pairs of solutions (x,y) SM, the computation model for the strength measure of built solutions Output: Next generation of parents Pop_{T}_{+1} and children Q_{T}_{+1} | |

01: R_{T} = Pop_{T} ∪ Q_{T} | //combine parents and children population |

02: F = sort-by-preferences (R_{T}, |Pop_{T}|, PM) | //create outrank fronts F = (F_{0}, F_{1},…) from R_{T} using PM |

03: Pop_{T+1} = Ø | //initialize new population Pop_{T}_{+1} |

04: i = 0 | |

05: while |Pop_{T+1}| + |F_{i}| ≤ N do | //fill the new population set Pop_{T}_{+1} |

06: Pop_{T+}_{1} = Pop_{T+}_{1} ∪ F_{i} | //include front F_{i} that fits completely in Pop_{T}_{+1} |

07: i = i + 1 | //move to next front in the order set F |

08: end | |

09: FS = strength- assignment(F_{i}, PM, NSF) | //measures the strength of the solutions in F_{i}, NSF = F_{0} |

10: F′_{i} = SORT(F_{i}, FS) | //sort solutions in F_{i} by FS in descending order |

11: Pop_{T+1} = Pop_{T+1} ∪ F′_{i}[1:N-|Pop_{T+1}|] | //complete Pop_{T}_{+1} with best solutions in F’_{I} when |Pop_{T+}_{1}| < N |

12: Q_{T+1} = make-new-pop(Pop_{T+1}) | //construct next generation of children Q_{T}_{+1} using Pop_{T+}_{1} and the chosen operators for selection, crossover and mutation |

13: T = T + 1 | //iterate |

Complexity | |
---|---|

01: R_{T} = Pop_{T} ∪ Q_{T} | O(S) |

02: F = sort-by-preferences (R_{T}, |Pop_{T}|, PM) | O(N^{2}S^{2}) * |

03: Pop_{T+}_{1} = ∅ | O(1) |

04: i = 0 | O(1) |

05: while |Pop_{T}_{+1}|+|F_{i}|≤ N do | O(S) |

06: Pop_{T}_{+1} = Pop_{T}_{+1} ∪ F_{i} | O(S) |

07: i = i + 1 | O(1) |

08: end | |

09: FS = strength-assignment(F_{i}, PM, NSF) | O(S) |

10: F’i = SORT(F_{i}, FS) | O(S^{2}) |

11: Pop_{T}_{+1} = Pop_{T+}_{1} ∪ F’i[1:N-|Pop_{T}_{+1}|] | O(S) |

12: Q_{T}_{+1} = make-new-pop(Pop_{T}_{+1}) | O(NPS) |

13: T = T + 1 | O(1) |

## Appendix B. Updated Budget Requirements

**Table A2.**Parameters values of the model: (a) weight, (b) veto thresholds and (c) credibility and majority thresholds.

(a) | ||||||||||||||||||

DM | O1 | O2 | O3 | O4 | O5 | O6 | O7 | O8 | O9 | |||||||||

1 | 0.066 | 0.142 | 0.108 | 0.16 | 0.064 | 0.134 | 0.096 | 0.162 | 0.05 | 0.083 | 0.092 | 0.162 | 0.078 | 0.128 | 0.053 | 0.081 | 0.138 | 0.211 |

2 | 0.065 | 0.14 | 0.109 | 0.163 | 0.078 | 0.135 | 0.095 | 0.165 | 0.054 | 0.08 | 0.106 | 0.169 | 0.066 | 0.143 | 0.054 | 0.082 | 0.145 | 0.25 |

3 | 0.064 | 0.126 | 0.104 | 0.178 | 0.064 | 0.13 | 0.084 | 0.14 | 0.053 | 0.082 | 0.089 | 0.17 | 0.078 | 0.12 | 0.055 | 0.085 | 0.15 | 0.231 |

4 | 0.064 | 0.141 | 0.096 | 0.163 | 0.065 | 0.128 | 0.094 | 0.168 | 0.053 | 0.092 | 0.092 | 0.175 | 0.08 | 0.138 | 0.055 | 0.09 | 0.145 | 0.222 |

5 | 0.08 | 0.136 | 0.09 | 0.18 | 0.077 | 0.14 | 0.085 | 0.155 | 0.051 | 0.083 | 0.102 | 0.155 | 0.075 | 0.14 | 0.06 | 0.096 | 0.146 | 0.216 |

6 | 0.073 | 0.121 | 0.098 | 0.153 | 0.071 | 0.13 | 0.1 | 0.151 | 0.059 | 0.085 | 0.096 | 0.173 | 0.073 | 0.135 | 0.048 | 0.093 | 0.12 | 0.219 |

7 | 0.077 | 0.123 | 0.096 | 0.158 | 0.077 | 0.138 | 0.097 | 0.159 | 0.053 | 0.09 | 0.092 | 0.16 | 0.078 | 0.123 | 0.048 | 0.083 | 0.142 | 0.245 |

8 | 0.068 | 0.143 | 0.107 | 0.17 | 0.067 | 0.139 | 0.086 | 0.158 | 0.059 | 0.089 | 0.109 | 0.169 | 0.076 | 0.13 | 0.05 | 0.086 | 0.13 | 0.243 |

9 | 0.07 | 0.135 | 0.093 | 0.177 | 0.068 | 0.137 | 0.09 | 0.148 | 0.053 | 0.082 | 0.103 | 0.169 | 0.079 | 0.127 | 0.052 | 0.084 | 0.15 | 0.223 |

10 | 0.076 | 0.13 | 0.09 | 0.154 | 0.069 | 0.136 | 0.087 | 0.147 | 0.052 | 0.095 | 0.099 | 0.176 | 0.08 | 0.128 | 0.053 | 0.091 | 0.132 | 0.234 |

(b) | ||||||||||||||||||

DM | O1 | O2 | O3 | O4 | O5 | O6 | O7 | O8 | O9 | |||||||||

1 | 195,989 | 195,989 | 234,398 | 234,398 | 169,919 | 169,919 | 169,919 | 169,919 | 169,919 | 169,919 | 169,919 | 169,919 | 169,919 | 169,919 | 169,919 | 169,919 | 301,255 | 301,255 |

2 | 256,187 | 256,187 | 245,076 | 245,076 | 298,408 | 298,408 | 298,408 | 298,408 | 298,408 | 298,408 | 298,408 | 298,408 | 298,408 | 298,408 | 298,408 | 298,408 | 329,296 | 329,296 |

3 | 195,544 | 195,544 | 132,601 | 132,601 | 205,307 | 205,307 | 205,307 | 205,307 | 205,307 | 205,307 | 205,307 | 205,307 | 205,307 | 205,307 | 205,307 | 205,307 | 378,402 | 378,402 |

4 | 126,797 | 126,797 | 141,914 | 141,914 | 199,808 | 199,808 | 199,808 | 199,808 | 199,808 | 199,808 | 199,808 | 199,808 | 199,808 | 199,808 | 199,808 | 199,808 | 226,441 | 226,441 |

5 | 222,376 | 222,376 | 173,008 | 173,008 | 207,355 | 207,355 | 207,355 | 207,355 | 207,355 | 207,355 | 207,355 | 207,355 | 207,355 | 207,355 | 207,355 | 207,355 | 254,049 | 254,049 |

6 | 276,395 | 276,395 | 231,282 | 231,282 | 173,649 | 173,649 | 173,649 | 173,649 | 173,649 | 173,649 | 173,649 | 173,649 | 173,649 | 173,649 | 173,649 | 173,649 | 262,777 | 262,777 |

7 | 184,680 | 184,680 | 161,316 | 161,316 | 159,873 | 159,873 | 159,873 | 159,873 | 159,873 | 159,873 | 159,873 | 159,873 | 159,873 | 159,873 | 159,873 | 159,873 | 189,960 | 189,960 |

8 | 129,004 | 129,004 | 194,205 | 194,205 | 224,768 | 224,768 | 224,768 | 224,768 | 224,768 | 224,768 | 224,768 | 224,768 | 224,768 | 224,768 | 224,768 | 224,768 | 284,196 | 284,196 |

9 | 265,641 | 265,641 | 140,995 | 140,995 | 249,798 | 249,798 | 249,798 | 249,798 | 249,798 | 249,798 | 249,798 | 249,798 | 249,798 | 249,798 | 249,798 | 249,798 | 375,703 | 375,703 |

10 | 150,909 | 150,909 | 194,222 | 194,222 | 185,065 | 185,065 | 185,065 | 185,065 | 185,065 | 185,065 | 185,065 | 185,065 | 185,065 | 185,065 | 185,065 | 185,065 | 275,222 | 275,222 |

(c) | ||||||||||||||||||

DM | α | ξ | Λ | Β | ||||||||||||||

1 | 0.75 | 0.75 | 0.51 | 0.67 | 0.66 | 0.77 | ||||||||||||

2 | 0.67 | 0.67 | 0.51 | 0.67 | 0.60 | 0.70 | ||||||||||||

3 | 0.65 | 0.67 | 0.51 | 0.67 | 0.60 | 0.67 | ||||||||||||

4 | 0.66 | 0.67 | 0.51 | 0.67 | 0.60 | 0.67 | ||||||||||||

5 | 0.68 | 0.70 | 0.51 | 0.67 | 0.60 | 0.70 | ||||||||||||

6 | 0.74 | 0.75 | 0.51 | 0.67 | 0.66 | 0.76 | ||||||||||||

7 | 0.75 | 0.75 | 0.51 | 0.67 | 0.66 | 0.77 | ||||||||||||

8 | 0.77 | 0.78 | 0.51 | 0.67 | 0.66 | 0.78 | ||||||||||||

9 | 0.78 | 0.80 | 0.51 | 0.67 | 0.67 | 0.80 | ||||||||||||

10 | 0.73 | 0.75 | 0.51 | 0.67 | 0.65 | 0.75 |

- α:
- credibility threshold for dominance
- ξ:
- credibility threshold for sufficiency of resources
- λ:
- interval majority threshold
- β:
- credibility threshold for the crisp interval outranking.

Frontiers B | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|

O1 | O2 | O3 | O4 | O5 | O6 | O7 | O8 | O9 | ||

DM1 | b_{1} | 1,234,925 | 1,036,475 | 1,417,065 | 968,685 | 1,570,565 | 1,211,330 | 1,925,345 | 1,365,315 | 1,658,795 |

b_{2} | 1,251,926 | 1,036,475 | 1,417,065 | 968,685 | 1,570,565 | 1,211,330 | 1,891,403 | 1,365,315 | 1,658,795 | |

b_{3} | 1,234,925 | 1,036,475 | 1,342,175 | 968,685 | 1,570,565 | 1,234,373 | 1,925,345 | 1,365,315 | 1,658,795 | |

DM2 | b_{1} | 1,259,905 | 975,365 | 1,474,745 | 999,450 | 1,542,150 | 1,279,330 | 1,983,810 | 1,486,095 | 1,633,700 |

b_{2} | 1,296,052 | 975,365 | 1,474,745 | 999,450 | 1,542,150 | 1,230,224 | 1,983,810 | 1,486,095 | 1,633,700 | |

b_{3} | 1,259,905 | 975,365 | 1,475,379 | 999,450 | 1,542,150 | 1,279,330 | 1,983,810 | 1,374,761 | 1,633,700 | |

DM3 | b_{1} | 1,291,140 | 1,024,090 | 1,509,600 | 924,815 | 1,712,030 | 1,255,190 | 2,009,685 | 1,358,450 | 1,661,035 |

b_{2} | 1,291,140 | 1,024,090 | 1,509,600 | 924,815 | 1,712,030 | 1,278,324 | 2,009,685 | 1,276,835 | 1,661,035 | |

b_{3} | 1,291,140 | 9,725,56 | 1,509,600 | 924,815 | 1,712,030 | 1,255,190 | 2,106,229 | 1,358,450 | 1,661,035 | |

DM4 | b_{1} | 1,189,995 | 1,060,215 | 1425755 | 977,190 | 1,667,295 | 1,312,095 | 2,067,265 | 1,370,165 | 1,610,035 |

b_{2} | 1,189,995 | 982,445 | 1441611 | 977,190 | 1,667,295 | 1,312,095 | 2,067,265 | 1,370,165 | 1,610,035 | |

b_{3} | 1,212,503 | 1,060,215 | 1425755 | 977,190 | 1,667,295 | 1,266,805 | 2,067,265 | 1,370,165 | 1,610,035 | |

DM5 | b_{1} | 1,215,945 | 1,010,400 | 1,488,700 | 927,465 | 1,702,140 | 1,234,430 | 1,992,975 | 1,383,445 | 1,715,605 |

b_{2} | 1,230,005 | 1,010,400 | 1,488,700 | 927,465 | 1,702,140 | 1,234,430 | 1,936,667 | 1,383,445 | 1,715,605 | |

b_{3} | 1,215,945 | 1,033,867 | 1,488,700 | 927,465 | 1,702,140 | 1,234,430 | 1,992,975 | 1,268,806 | 1,715,605 | |

DM6 | b_{1} | 1,169,800 | 1,057,620 | 1,502,780 | 962,570 | 1,652,975 | 1,211,310 | 2,012,250 | 1,445,540 | 1,576,135 |

b_{2} | 1,169,800 | 1,057,620 | 1,502,780 | 962,570 | 1,652,975 | 1,211,310 | 1,821,482 | 1,448,256 | 1,576,135 | |

b_{3} | 1,170,982 | 1,057,620 | 1,502,780 | 962,570 | 1,652,975 | 1,211,310 | 2,012,250 | 1,445,540 | 1,420,488 | |

DM7 | b_{1} | 1,211,085 | 1,026,425 | 1,394,220 | 910,105 | 1,588,335 | 1,266,775 | 2,029,310 | 1,414,960 | 1,663,015 |

b_{2} | 1,153,127 | 1,026,425 | 1,394,220 | 921,731 | 1,588,335 | 1,266,775 | 2,029,310 | 1,414,960 | 1,663,015 | |

b_{3} | 1,211,085 | 1,037,823 | 1,394,220 | 910,105 | 1,576,124 | 1,266,775 | 2,029,310 | 1,414,960 | 1,663,015 | |

DM8 | b_{1} | 1,297,695 | 1,009,480 | 1,471,245 | 946,720 | 1,538,740 | 1,231,455 | 1,820,550 | 1,489,320 | 1,596,360 |

b_{2} | 1,297,695 | 1,009,480 | 1,358,643 | 946,720 | 1,538,740 | 1,231,455 | 1,820,550 | 1,489,320 | 1,598,868 | |

b_{3} | 1,297,695 | 1,009,480 | 1,471,245 | 933,477 | 1,541,083 | 1,231,455 | 1,820,550 | 1,489,320 | 1,596,360 | |

DM9 | b_{1} | 1,139,045 | 1,077,665 | 1,377,385 | 925,280 | 1,570,825 | 1,215,730 | 1,982,670 | 1,429,490 | 1,584,005 |

b_{2} | 1,139,045 | 1,077,665 | 1,377,385 | 925,280 | 1,570,825 | 1,154,986 | 1,982,670 | 1,429,490 | 1,606,499 | |

b_{3} | 1,142,960 | 1,077,665 | 1,377,385 | 925,280 | 1,570,825 | 1,215,730 | 1,982,670 | 1,358,901 | 1,584,005 | |

DM10 | b_{2} | 1,262,100 | 1,030,870 | 1,442,840 | 931,350 | 1,629,665 | 1,251,900 | 1,932,615 | 1,450,675 | 1,628,245 |

b_{3} | 1,262,100 | 1,030,870 | 1,442,840 | 931,350 | 1,629,665 | 1,251,900 | 1,959,614 | 1,450,675 | 1,540,127 | |

b_{1} | 1,262,100 | 975,097 | 1,442,840 | 953,011 | 1,629,665 | 1,251,900 | 1,932,615 | 1,450,675 | 1,628,245 |

**Table A4.**The updated (a) weights after the consensus round, (b) veto thresholds after the consensus round, and (c) credibility thresholds after the consensus round.

(a) | ||||||||||||||||||

DM | O1 | O2 | O3 | O4 | O5 | O6 | O7 | O8 | O9 | |||||||||

1 | 0.068 | 0.100 | 0.105 | 0.143 | 0.072 | 0.108 | 0.098 | 0.133 | 0.057 | 0.077 | 0.098 | 0.135 | 0.082 | 0.114 | 0.057 | 0.078 | 0.142 | 0.189 |

2 | 0.068 | 0.099 | 0.105 | 0.144 | 0.079 | 0.115 | 0.097 | 0.133 | 0.059 | 0.079 | 0.104 | 0.142 | 0.076 | 0.108 | 0.058 | 0.078 | 0.145 | 0.192 |

3 | 0.067 | 0.099 | 0.105 | 0.141 | 0.072 | 0.108 | 0.092 | 0.127 | 0.058 | 0.078 | 0.097 | 0.134 | 0.082 | 0.114 | 0.058 | 0.079 | 0.144 | 0.195 |

4 | 0.067 | 0.099 | 0.101 | 0.137 | 0.073 | 0.109 | 0.097 | 0.132 | 0.058 | 0.078 | 0.098 | 0.135 | 0.083 | 0.115 | 0.058 | 0.079 | 0.145 | 0.192 |

5 | 0.065 | 0.107 | 0.098 | 0.134 | 0.079 | 0.115 | 0.092 | 0.128 | 0.057 | 0.077 | 0.103 | 0.140 | 0.081 | 0.112 | 0.061 | 0.081 | 0.146 | 0.193 |

6 | 0.069 | 0.103 | 0.102 | 0.138 | 0.076 | 0.112 | 0.099 | 0.135 | 0.061 | 0.081 | 0.100 | 0.137 | 0.080 | 0.111 | 0.055 | 0.075 | 0.133 | 0.180 |

7 | 0.067 | 0.105 | 0.101 | 0.137 | 0.079 | 0.115 | 0.098 | 0.134 | 0.058 | 0.078 | 0.098 | 0.135 | 0.082 | 0.114 | 0.055 | 0.075 | 0.144 | 0.191 |

8 | 0.069 | 0.101 | 0.106 | 0.143 | 0.074 | 0.110 | 0.093 | 0.128 | 0.061 | 0.081 | 0.102 | 0.144 | 0.081 | 0.113 | 0.056 | 0.076 | 0.138 | 0.185 |

9 | 0.070 | 0.102 | 0.100 | 0.136 | 0.074 | 0.110 | 0.095 | 0.130 | 0.058 | 0.078 | 0.104 | 0.141 | 0.083 | 0.114 | 0.057 | 0.077 | 0.144 | 0.195 |

10 | 0.067 | 0.105 | 0.098 | 0.134 | 0.075 | 0.111 | 0.093 | 0.129 | 0.058 | 0.078 | 0.102 | 0.139 | 0.083 | 0.115 | 0.057 | 0.078 | 0.139 | 0.186 |

(b) | ||||||||||||||||||

DM | O1 | O2 | O3 | O4 | O5 | O6 | O7 | O8 | O9 | |||||||||

1 | 198,171 | 198,171 | 190,206 | 190,206 | 198,904 | 198,904 | 200,051 | 200,051 | 200,166 | 200,166 | 200,178 | 200,178 | 200,179 | 200,179 | 200,179 | 200,179 | 306,014 | 306,014 |

2 | 172,435 | 172,435 | 184,867 | 184,867 | 192,629 | 192,629 | 196,072 | 196,072 | 196,416 | 196,416 | 196,451 | 196,451 | 196,454 | 196,454 | 196,454 | 196,454 | 301,513 | 301,513 |

3 | 197,948 | 197,948 | 168,769 | 168,769 | 216,598 | 216,598 | 217,745 | 217,745 | 217,860 | 217,860 | 217,872 | 217,872 | 217,873 | 217,873 | 217,873 | 217,873 | 276,960 | 276,960 |

4 | 163,575 | 163,575 | 173,425 | 173,425 | 213,848 | 213,848 | 214,996 | 214,996 | 215,111 | 215,111 | 215,122 | 215,122 | 215,123 | 215,123 | 215,123 | 215,123 | 268,607 | 268,607 |

5 | 189,340 | 189,340 | 188,972 | 188,972 | 217,622 | 217,622 | 218,769 | 218,769 | 218,884 | 218,884 | 218,896 | 218,896 | 218,897 | 218,897 | 218,897 | 218,897 | 282,411 | 282,411 |

6 | 162,331 | 162,331 | 191,764 | 191,764 | 200,769 | 200,769 | 201,916 | 201,916 | 202,031 | 202,031 | 202,043 | 202,043 | 202,044 | 202,044 | 202,044 | 202,044 | 286,775 | 286,775 |

7 | 192,516 | 192,516 | 183,126 | 183,126 | 193,881 | 193,881 | 195,028 | 195,028 | 195,143 | 195,143 | 195,155 | 195,155 | 195,156 | 195,156 | 195,156 | 195,156 | 250,367 | 250,367 |

8 | 164,678 | 164,678 | 199,571 | 199,571 | 226,328 | 226,328 | 227,476 | 227,476 | 227,591 | 227,591 | 227,602 | 227,602 | 227,603 | 227,603 | 227,603 | 227,603 | 297,485 | 297,485 |

9 | 167,708 | 167,708 | 172,966 | 172,966 | 216,934 | 216,934 | 220,377 | 220,377 | 220,721 | 220,721 | 220,756 | 220,756 | 220,759 | 220,759 | 220,759 | 220,759 | 278,309 | 278,309 |

10 | 175,631 | 175,631 | 199,579 | 199,579 | 206,477 | 206,477 | 207,624 | 207,624 | 207,739 | 207,739 | 207,751 | 207,751 | 207,752 | 207,752 | 207,752 | 207,752 | 292,998 | 292,998 |

(c) | ||||||||||||||||||

DM | A | ξ | β | |||||||||||||||

1 | 0.70 | 0.72 | 0.62 | 0.70 | ||||||||||||||

2 | 0.69 | 0.70 | 0.62 | 0.67 | ||||||||||||||

3 | 0.68 | 0.70 | 0.62 | 0.67 | ||||||||||||||

4 | 0.69 | 0.70 | 0.62 | 0.67 | ||||||||||||||

5 | 0.70 | 0.71 | 0.62 | 0.67 | ||||||||||||||

6 | 0.71 | 0.72 | 0.62 | 0.70 | ||||||||||||||

7 | 0.70 | 0.72 | 0.62 | 0.70 | ||||||||||||||

8 | 0.69 | 0.70 | 0.62 | 0.70 | ||||||||||||||

9 | 0.69 | 0.69 | 0.62 | 0.70 | ||||||||||||||

10 | 0.71 | 0.72 | 0.63 | 0.69 |

Project | Cost | Project | Cost | Project | Cost | Project | Cost | ||||
---|---|---|---|---|---|---|---|---|---|---|---|

1 | 9260 | 9640 | 26 | 9522 | 9908 | 51 | 8197 | 8533 | 76 | 5472 | 5698 |

2 | 6235 | 6485 | 27 | 9697 | 10,093 | 52 | 5962 | 6208 | 77 | 9657 | 10,053 |

3 | 5772 | 6008 | 28 | 9535 | 9925 | 53 | 7450 | 7750 | 78 | 7450 | 7750 |

4 | 7665 | 7975 | 29 | 8222 | 8558 | 54 | 7327 | 7623 | 79 | 5072 | 5278 |

5 | 9362 | 9748 | 30 | 9012 | 9378 | 55 | 6542 | 6808 | 80 | 8260 | 8600 |

6 | 7410 | 7710 | 31 | 5972 | 6218 | 56 | 9727 | 10,123 | 81 | 6592 | 6858 |

7 | 7675 | 7985 | 32 | 9065 | 9435 | 57 | 5490 | 5710 | 82 | 7422 | 7728 |

8 | 9512 | 9898 | 33 | 5370 | 5590 | 58 | 8780 | 9140 | 83 | 6962 | 7248 |

9 | 7360 | 7660 | 34 | 9085 | 9455 | 59 | 7855 | 8175 | 84 | 8495 | 8845 |

10 | 5602 | 5828 | 35 | 8085 | 8415 | 60 | 6360 | 6620 | 85 | 5790 | 6030 |

11 | 7647 | 7963 | 36 | 5380 | 5600 | 61 | 6217 | 6473 | 86 | 7855 | 8175 |

12 | 4990 | 5190 | 37 | 7677 | 7993 | 62 | 5880 | 6120 | 87 | 8345 | 8685 |

13 | 5747 | 5983 | 38 | 9372 | 9758 | 63 | 5612 | 5838 | 88 | 6002 | 6248 |

14 | 8590 | 8940 | 39 | 7470 | 7770 | 64 | 9565 | 9955 | 89 | 7740 | 8060 |

15 | 7930 | 8250 | 40 | 6922 | 7208 | 65 | 8657 | 9013 | 90 | 9707 | 10,103 |

16 | 8045 | 8375 | 41 | 9012 | 9378 | 66 | 7890 | 8210 | 91 | 6000 | 6240 |

17 | 8410 | 8750 | 42 | 9412 | 9798 | 67 | 6565 | 6835 | 92 | 7392 | 7698 |

18 | 5387 | 5603 | 43 | 5032 | 5238 | 68 | 9767 | 10,163 | 93 | 5592 | 5818 |

19 | 6340 | 6600 | 44 | 7982 | 8308 | 69 | 8165 | 8495 | 94 | 9605 | 9995 |

20 | 7850 | 8170 | 45 | 6052 | 6298 | 70 | 6065 | 6315 | 95 | 6572 | 6838 |

21 | 9360 | 9740 | 46 | 9087 | 9463 | 71 | 8320 | 8660 | 96 | 5012 | 5218 |

22 | 8195 | 8525 | 47 | 7850 | 8170 | 72 | 6380 | 6640 | 97 | 8830 | 9190 |

23 | 5910 | 6150 | 48 | 6787 | 7063 | 73 | 9207 | 9583 | 98 | 5685 | 5915 |

24 | 5787 | 6023 | 49 | 6217 | 6473 | 74 | 9797 | 10,193 | 99 | 5377 | 5593 |

25 | 5237 | 5453 | 50 | 7760 | 8080 | 75 | 6052 | 6298 | 100 | 5695 | 5925 |

Sol. | DM1 | DM2 | DM3 | DM4 | DM5 | DM6 | DM7 | DM8 | DM9 | DM10 |
---|---|---|---|---|---|---|---|---|---|---|

1 | 0.9816 | 0.8796 | 0.8958 | 0.8364 | 0.8345 | 0.8422 | 0.8814 | 1.0000 | 0.9921 | 0.9386 |

2 | 0.8643 | 0.9764 | 0.8479 | 0.8068 | 0.8626 | 0.8161 | 1.0000 | 1.0000 | 1.0000 | 0.8892 |

3 | 0.9030 | 0.8196 | 0.8526 | 0.9027 | 0.8418 | 0.8129 | 0.9427 | 0.8529 | 0.9630 | 0.7984 |

4 | 0.8808 | 0.9054 | 0.7975 | 0.8904 | 0.7910 | 0.9399 | 0.8485 | 1.0000 | 0.9977 | 1.0000 |

5 | 1.0000 | 0.9025 | 0.8559 | 0.7986 | 0.7880 | 0.8374 | 0.8435 | 1.0000 | 1.0000 | 0.9423 |

O1 | O2 | O3 | O4 | O5 | O6 | O7 | O8 | O9 | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

1,172,035 | 1,196,440 | 935,955 | 955,448 | 1,293,390 | 1,320,333 | 921,260 | 940,450 | 1,703,365 | 1,738,853 | 1,059,975 | 1,082,055 | 1,831,405 | 1,869,578 | 1,366,425 | 1,394,890 | 1,420,215 | 1,449,790 |

1,172,035 | 1,196,440 | 936,369 | 955,870 | 1,293,390 | 1,320,333 | 921,260 | 940,450 | 1,703,365 | 1,738,853 | 1,059,975 | 1,082,055 | 1,831,405 | 1,869,578 | 1,366,425 | 1,394,890 | 1,341,051 | 1,368,978 |

1,172,035 | 1,196,440 | 935,955 | 955,448 | 1,293,390 | 1,320,333 | 840,542 | 858,050 | 1,703,365 | 1,738,853 | 1,064,999 | 1,087,184 | 1,831,405 | 1,869,578 | 1,366,425 | 1,394,890 | 1,420,215 | 1,449,790 |

1,210,295 | 1,235,513 | 989,555 | 1,010,173 | 1,322,445 | 1,349,990 | 899,420 | 918,158 | 1,607,385 | 1,640,873 | 1,165,840 | 1,190,123 | 1,915,940 | 1,955,865 | 1,451,480 | 1,481,713 | 1,478,550 | 1,509,343 |

1,210,295 | 1,235,513 | 989,555 | 1,010,173 | 1,322,445 | 1,349,990 | 820,944 | 838,047 | 1,607,385 | 1,640,873 | 1,165,840 | 1,190,123 | 1,915,940 | 1,955,865 | 1,451,480 | 1,481,713 | 1,484,672 | 1,515,592 |

1,210,295 | 1,235,513 | 1,001,082 | 1,021,940 | 1,322,445 | 1,349,990 | 899,420 | 918,158 | 1,607,385 | 1,640,873 | 1,090,067 | 1,112,771 | 1,915,940 | 1,955,865 | 1,451,480 | 1,481,713 | 1,478,550 | 1,509,343 |

1,169,955 | 1,194,315 | 1,012,675 | 1,033,773 | 1,366,235 | 1,394,695 | 871,970 | 890,138 | 1,771,940 | 1,808,860 | 1,191,690 | 1,216,513 | 1,874,150 | 1,913,203 | 1,469,130 | 1,499,725 | 1,433,435 | 1,463,290 |

1,169,955 | 1,194,315 | 1,012,675 | 1,033,773 | 1,366,235 | 1,394,695 | 871,970 | 890,138 | 1,771,940 | 1,808,860 | 1,201,862 | 1,226,897 | 1,874,150 | 1,913,203 | 1,469,130 | 1,499,725 | 1,424,088 | 1,453,749 |

1,169,955 | 1,194,315 | 978,386 | 998,769 | 1,366,235 | 1,394,695 | 882,140 | 900,519 | 1,771,940 | 1,808,860 | 1,191,690 | 1,216,513 | 1,874,150 | 1,913,203 | 1,469,130 | 1,499,725 | 1,433,435 | 1,463,290 |

1,202,270 | 1,227,310 | 1,006,320 | 1,027,280 | 1,393,975 | 1,423,008 | 897,320 | 916,015 | 1,566,755 | 1,618,775 | 1,163,860 | 1,188,108 | 1,914,190 | 1,954,078 | 1,462,680 | 1,493,143 | 1,479,985 | 1,510,815 |

1,202,270 | 1,227,310 | 973,732 | 994,013 | 1,393,975 | 1,423,008 | 897,320 | 916,015 | 1,566,755 | 1,618,775 | 1,175,213 | 1,199,697 | 1,914,190 | 1,954,078 | 1,462,680 | 1,493,143 | 1,479,985 | 1,510,815 |

1,202,270 | 1,227,310 | 1,006,320 | 1,027,280 | 1,393,975 | 1,423,008 | 863,664 | 881,658 | 1,566,755 | 1,618,775 | 1,163,860 | 1,188,108 | 1,914,190 | 1,954,078 | 1,462,680 | 1,493,143 | 1,493,835 | 1,524,954 |

1,210,395 | 1,235,598 | 1,005,315 | 1,026,260 | 1,379,125 | 1,407,855 | 843,045 | 860,610 | 1,668,500 | 1,703,265 | 1,184,000 | 1,208,658 | 1,869,795 | 1,908,760 | 1,465,640 | 1,496,160 | 1,392,025 | 1,421,003 |

1,210,395 | 1,235,598 | 1,013,090 | 1,034,197 | 1,379,125 | 1,407,855 | 827,099 | 844,332 | 1,668,500 | 1,703,265 | 1,184,000 | 1,208,658 | 1,869,795 | 1,908,760 | 1,465,640 | 1,496,160 | 1,392,025 | 1,421,003 |

1,210,395 | 1,235,598 | 1,005,315 | 1,026,260 | 1,379,125 | 1,407,855 | 843,045 | 860,610 | 1,668,500 | 1,703,265 | 1,120,592 | 1,143,929 | 1,869,795 | 1,908,760 | 1,465,640 | 1,496,160 | 1,394,685 | 1,423,718 |

1,134,520 | 1,158,138 | 1,002,865 | 1,023,758 | 1,283,820 | 1,310,563 | 864,205 | 882,210 | 1,667,285 | 1,702,025 | 1,098,580 | 1,121,465 | 1,872,725 | 1,911,748 | 1,356,185 | 1,384,428 | 1,448,755 | 1,478,918 |

1,134,520 | 1,158,138 | 1,002,865 | 1,023,758 | 1,283,820 | 1,310,563 | 828,005 | 845,256 | 1,667,285 | 1,702,025 | 1,102,103 | 1,125,062 | 1,872,725 | 1,911,748 | 1,356,185 | 1,384,428 | 1,448,755 | 1,478,918 |

1,134,520 | 1,158,138 | 958,718 | 978,691 | 1,283,820 | 1,310,563 | 864,205 | 882,210 | 1,667,285 | 1,702,025 | 1,098,580 | 1,121,465 | 1,872,725 | 1,911,748 | 1,356,185 | 1,384,428 | 1,454,714 | 1,485,000 |

1,172,035 | 1,196,440 | 935,955 | 955,448 | 1,293,390 | 1,320,333 | 921,260 | 940,450 | 1,703,365 | 1,738,853 | 1,059,975 | 1,082,055 | 1,831,405 | 1,869,578 | 1,366,425 | 1,394,890 | 1,420,215 | 1,449,790 |

1,172,035 | 1,196,440 | 935,955 | 955,448 | 1,293,390 | 1,320,333 | 910,799 | 929,771 | 1,703,365 | 1,738,853 | 1,059,975 | 1,082,055 | 1,831,405 | 1,869,578 | 1,366,425 | 1,394,890 | 1,424,702 | 1,454,371 |

1,172,035 | 1,196,440 | 939,929 | 959,504 | 1,293,390 | 1,320,333 | 921,260 | 940,450 | 1,703,365 | 1,738,853 | 1,027,471 | 1,048,874 | 1,831,405 | 1,869,578 | 1,366,425 | 1,394,890 | 1,420,215 | 1,449,790 |

1,142,135 | 1,165,920 | 930,485 | 949,865 | 1,331435 | 1,359,165 | 876,760 | 895,023 | 1,619,515 | 1,671,175 | 1,147,440 | 1,171,338 | 1,905,355 | 1,945,068 | 1,465,220 | 1,495,733 | 1,451,000 | 1,481,223 |

1,142,135 | 1,165,920 | 931,023 | 950,415 | 1,331435 | 1,359,165 | 800,414 | 817,086 | 1,619,515 | 1,671,175 | 1,147,440 | 1,171,338 | 1,905,355 | 1,945,068 | 1,465,220 | 1,495,733 | 1,451,000 | 1,481,223 |

1,142,135 | 1,165,920 | 930,485 | 949,865 | 1,331435 | 1,359,165 | 876,760 | 895,023 | 1,619,515 | 1,671,175 | 1,111,037 | 1,134,176 | 1,905,355 | 1,945,068 | 1,465,220 | 1,495,733 | 1,453,084 | 1,483,350 |

1,157,660 | 1,181,773 | 981,860 | 1,002,313 | 1,363505 | 1,391,908 | 890,910 | 909,468 | 1,535,575 | 1,567,575 | 1,182,040 | 1,206,658 | 1,775,935 | 1,812,940 | 1,396,465 | 1,425,550 | 1,434,775 | 1,464,658 |

1,157,660 | 1,181,773 | 982,925 | 1,003,400 | 1,363505 | 1,391,908 | 809,103 | 825,956 | 1,535,575 | 1,567,575 | 1,182,040 | 1,206,658 | 1,775,935 | 1,812,940 | 1,396,465 | 1,425,550 | 1,434,775 | 1,464,658 |

1,157,660 | 1,181,773 | 981,860 | 1,002,313 | 1,363505 | 1,391,908 | 890,910 | 909,468 | 1,535,575 | 1,567,575 | 1,107,575 | 1,130,642 | 1,775,935 | 1,812,940 | 1,396,465 | 1,425,550 | 1,437,577 | 1,467,518 |

1,164,705 | 1,188,958 | 962,750 | 982,798 | 1,275670 | 1,302,250 | 860,665 | 878,600 | 1,605,355 | 1,638,810 | 1,141,455 | 1,165,230 | 1,822,560 | 1,860,530 | 1,452,580 | 1,482,835 | 1,411,810 | 1,441,215 |

1,164,705 | 1,188,958 | 876,601 | 894,854 | 1,275,670 | 1,302,250 | 860,665 | 878,600 | 1,605,355 | 1,638,810 | 1,141,455 | 1,165,230 | 1,822,560 | 1,860,530 | 1,452,580 | 1,482,835 | 1,419,115 | 1,448,672 |

1,164,705 | 1,188,958 | 962,750 | 982,798 | 1,275,670 | 1,302,250 | 826,801 | 844,030 | 1,605,355 | 1,638,810 | 1,142,690 | 1,166,491 | 1,822,560 | 1,860,530 | 1,452,580 | 1,482,835 | 1,411,810 | 1,441,215 |

Sol. | DM1 | DM2 | DM3 | DM4 | DM5 | DM6 | DM7 | DM8 | DM9 | DM10 |
---|---|---|---|---|---|---|---|---|---|---|

1 | 0.7583 | 0.7467 | 0.7087 | 0.7391 | 0.7207 | 0.7674 | 0.7454 | 0.7805 | 0.7758 | 0.7963 |

2 | 0.7566 | 0.7453 | 0.7069 | 0.7374 | 0.7191 | 0.7658 | 0.7437 | 0.7789 | 0.7742 | 0.7947 |

3 | 0.7521 | 0.7417 | 0.7026 | 0.7330 | 0.7155 | 0.7615 | 0.7398 | 0.7746 | 0.7701 | 0.7905 |

4 | 0.7516 | 0.7413 | 0.7019 | 0.7324 | 0.7150 | 0.7613 | 0.7393 | 0.7743 | 0.7696 | 0.7902 |

5 | 0.7514 | 0.7404 | 0.7015 | 0.7320 | 0.7139 | 0.7610 | 0.7383 | 0.7740 | 0.7689 | 0.7895 |

## Appendix C. Description of the Variant of MOEA/D Used in Solving Problem 1

Algorithm A2. Variant of MOEA/D | |

Input:N: number of scalar functions, M: number of objectives, Vector: uniformly distributed set of vectors T = N/10: size of neighborhood of weight vectors. Output:EP: approximatio of Pareto frontier. | |

01: (x,z,FV,B(i)) ← Initializacion() | |

02: For i = 1 to N do | |

03: (x_{k},x_{l}) ← RandomSelection(B(i),T) | |

04: y ← OnePointCrossover(x_{k},x_{l}) | |

05: y′ ← FlipMutation(y) | |

06: y″ ← RepairAndImprovementOperator(y′) | |

07: UpdateSetZ(z,M,y″) | //z: for each j = 1, …, m, if z_{j} < f_{j}(y′) then set zj = f_{j}(y′). |

08: UpdateNeighborhood(B(i),FV,y″) | // for each j ∈ B(i), if g^{te} (y’|V_{j,z}) set x_{j} = y’ and FV_{j} = F(y’) |

09: UpdateEP(EP,y″) | //Remove from EP all the vectors dominated by F(y′), and add F(y′) to EP if no vectors in EP dominate F(y′) |

10: Stopping Criteria: if maxEvaluations is reach, Otherwise, go to step 2. |

_{1}, V

_{2}, …, V

_{N}}, and T. These parameters represent, respectively, the number of scalar functions or sub-problems in which the MOP has been divided, the number of objectives, a uniformly distributed set of size N containing weight vectors with two elements each (the weights were ${V}_{i}=\left(\frac{i}{N},\frac{N-i}{N}\right)$, for each V

_{i}∈ Vector), and the size of the neighborhood of weight vectors. In addition, MOEA/D gives as output an approximation of the Pareto frontier (EP) formed by non-dominated solutions found during the optimization process. For this purpose, the algorithm works in two phases.

_{i}that contain the T closes weight vectors to V

_{i}by Euclidean distance; (3) the initial set of solutions X = {x

_{1}, x

_{2}, …, x

_{N}} where each solution x

_{j}, 1 ≤ j ≤ Number of group members, corresponds to the j-th DM best compromise in the decision variable space, and the remaining ones are randomly generated; (4) the set of fitness values FV = {FV

_{1}, FV

_{2},…, FV

_{N}}, where each FV

_{i}is composed by the M objective values of each solution x

_{i}; and 5) the set z = {z

_{1}, …., z

_{m}} formed by values z

_{j}corresponding to the best objective value among all the solutions built during the initialization process.

_{sat}≥ N

_{dis}. The RIO sorts the projects in the portfolio by the fitness ratio FR, and from the lowest FR to the highest, it eliminates projects one by one until the budget becomes feasible. After that, using a threshold parameter (set to 0.50) it also eliminates a proportion of the remaining projects. Then, from the highest FR to the lowest, the improvement algorithm adds projects into the portfolio while keeping the budget feasible. The fitness ratio FR is a generalized measure of the fitness of a solution based on its objectives; first, for the i-th project, the procedure computes the ratio fc

_{ij}= fitness/cost for each objective j; then, it computes f

_{ci}which is the median among the m values fc

_{ij}previously computed. The value f

_{cij}becomes the value of FR.

_{g}as the size of the population, the number of criteria on the project portfolio optimization, the number of candidate projects, and the number of DMs, respectively. The construction of the set EP was left outside the main loop.

Complexity | |
---|---|

0. (x,z,FV,B(i)) ← Initialization() | |

1. For i = 1 to S do | O(S) |

2. (x_{k}, x_{l}) ← RandomSelection(B(i),T) | O(1) |

3. y ← OnePointCrossover(x_{k}, x_{l}) | O(P) |

4. y′ ← FlipMutation(y) | O(1) |

5. y″ ← RepairAndImprovementOperator (y’) | O(P) |

6. UpdateSetZ(z,M,y″) | O(N^{2}n_{g}) |

7. UpdateNeighborhood(B(i),FV,y″) | O(S) |

8. UpdateEP(EP,y″) | |

9. Stopping criteria: If maxEvaluations is reach, otherwise, go to Step 1. |

_{g}in the decision group. The linear growth in the number of members of a decision group favors this approach in scenarios with large groups of decision-makers.

## References

- Kabak, O.; Ervural, B. Multiple attribute group decision making: A generic conceptual framework and a classification scheme. Knowl. Based Syst.
**2017**, 123, 13–30. [Google Scholar] [CrossRef] - Xu, X.; Du, Z.; Chen, X.; Cai, C. Confidence consensus-based model for large-scale group decision making: A novel approach to managing non-cooperative behaviors. Inf. Sci.
**2019**, 288, 271–293. [Google Scholar] [CrossRef] - Fu, C.; Chang, W.; Xue, M.; Yang, S. Multiple criteria group decision making with belief distributions and distributed preference relations. Eur. J. Oper. Res.
**2019**, 273, 623–633. [Google Scholar] [CrossRef] - Dong, Q.; Zhou, X.; Martínez, L. A hybrid group decision-making framework for achieving agreed solutions based on stable opinions. Inf. Sci.
**2019**, 490, 227–243. [Google Scholar] [CrossRef] - Herrera-Viedma, E.; Palomares, I.; Cabrerizo, F.J.; Dong, Y.; Chiclana, F.; Herrera, F. Revisiting Fuzzy and Linguistic Decision Making: Scenarios and Challenges for Making Wiser Decisions in a Better Way. IEEE Trans. Syst. Man Cybern. Syst.
**2021**, 51, 191–208. [Google Scholar] [CrossRef] - Lu, Y.; Xu, Y.; Herrera-Viedma, E.; Han, Y. Consensus of large-scale group decision making in social network: The minimum cost model based on robust optimization. Inf. Sci.
**2021**, 547, 910–930. [Google Scholar] [CrossRef] - Barykin, S.Y.; Kapustina, I.V.; Sergeev, S.M.; Kalinina, O.V.; Vilken, V.V.; Putikhin, Y.Y.; Volkova, L.V. Developing the physical distribution digital twin model within the trade network. Acad. Strateg. Manag. J.
**2021**, 20, 1–25. Available online: https://www.abacademies.org/articles/developing-the-physical-distribution-digital-twin-model-within-the-trade-network.pdf (accessed on 13 May 2021). - Jecmen, S.; Zhang, H.; Liu, R.; Shah, N.B.; Conitzer, V.; Fang, F. Mitigating manipulation in peer review via randomized reviewer assignments. arXiv
**2020**, arXiv:2006.16437. [Google Scholar] - Cabrerizo, F.J.; Moreno, J.M.; Pérez, I.J.; Herrera-Viedma, E. Analyzing consensus approaches in fuzzy group decisión making: Advantages and drawbacks. Soft Comput.
**2010**, 14, 451–463. [Google Scholar] [CrossRef] [Green Version] - Gao, Y.; Li, D. A consensus model for heterogeneous multi-attribute group decision making with several attribute sets. Expert Syst. Appl.
**2019**. [Google Scholar] [CrossRef] - Labella, A.; Liu, Y.; Rodríguez, R.M.; Martínez, L. Analyzing the performance of classical consensus models in large scale group decision making: A comparative study. Appl. Soft Comput.
**2018**, 67, 677–690. [Google Scholar] [CrossRef] - Cheng, L.; Chen, Y.; Chiang, Y. Identifying conflict patterns to reach a consensus. A novel group decision approach. Eur. J. Oper. Res.
**2016**, 254, 622–631. [Google Scholar] [CrossRef] - Herrera, F.; Herrera-Viedma, E.; Verdegay, J.L. A model of consensus in group decision making under linguistic assessments. Fuzzy Sets Syst.
**1996**, 78, 73–87. [Google Scholar] [CrossRef] - Dong, Y.; Xu, Y.; Li, H.; Feng, B. The OWA-based consensus operator under linguistic representation models using position indexes. Eur. J. Oper. Res.
**2010**, 203, 455–463. [Google Scholar] [CrossRef] - Hwang, C.L.; Lin, M.J. Group Decision Making under Multiple Criteria. In Lecture Notes in Economics and Mathematical Systems; Springer Science and Business Media LLC: Berlin/Heidelberg, Germany, 1987; Volume 281, 411p. [Google Scholar] [CrossRef]
- Jessup, L.; Valacich, J. Introduction to the Special Issue on Group Support Systems. Small Group Res.
**1993**, 24, 427–429. [Google Scholar] [CrossRef] - Leyva, J.C.; Fernandez, E. A new method for group decision support based on ELECTRE-III methodology. Eur. J. Oper. Res.
**2003**, 148, 14–27. [Google Scholar] [CrossRef] - Xiong, J.; Tan, X.; Yang, K.; Chen, Y. Fuzzy group decision making for multiobjective problems: Tradeoffs between consensus and robustness. J. Appl. Math.
**2013**, 2013, 1–9. [Google Scholar] [CrossRef] - Bouyssou, D.; Marchant, T.; Perny, P.; Tsoukias, A.; Vincke, P. Evaluations and Decision Models: A Critical Perspective; Kluwer Academic Publishers: Dordrecht, The Netherlands, 2000; Volume 1. [Google Scholar] [CrossRef]
- Fernández, E.; Figueira, J.; Navarro, J. An interval extension of the outranking approach and its application to multiple-criteria ordinal classification. Omega
**2019**, 84, 189–198. [Google Scholar] [CrossRef] - Fernández, E.; Figueira, J.; Navarro, J. Interval-based extensions of two outranking methods for multi-criteria ordinal classification. Omega
**2020**, 95, 102–125. [Google Scholar] [CrossRef] - Li, J.; Hu, Y. Some rationality conditions of joint efficient mapping in group multi-objective programming. J. Syst. Sci. Complex.
**2007**. [Google Scholar] [CrossRef] - Hu, Y.; Hong, Z.; Zhou, X. Utopian preference mapping and the utopian preference method for group multi-objective optimization. Prog. Nat. Sci.
**2003**, 13. [Google Scholar] [CrossRef] - Lewis, H.S.; Butler, T.W. An interactive framework for multi-person, multi-objective decision. Decis. Sci.
**1993**, 24, 1–22. [Google Scholar] [CrossRef] - Isermann, H. Interactive group decision-making by coalitions. In Plural Rationality and Interactive Decision Processes; Grauer, M., Thompson, M., Wierzbicki, A.P., Eds.; Springer: Berlin/Heidelberg, Germany, 1985; Volume 248. [Google Scholar] [CrossRef]
- Wendell, R.E. Multiple objective mathematical programming concerning multiple decision-makers. Oper. Res.
**1980**, 28, 1029–1257. [Google Scholar] [CrossRef] - Tapia, C.G.; Murtagh, B.A. Interactive group decision-making using fuzzy programming with preference criteria. Fuzzy Sets Syst.
**1992**, 45, 13–23. [Google Scholar] [CrossRef] - Efremov, R.; Insua, D.R.; Lotov, A. A framework for participatory decision support using Pareto frontier visualization, goal identification, and arbitration. Eur. J. Oper. Res.
**2009**, 199, 459–467. [Google Scholar] [CrossRef] - Bechikh, S.; Said, L.B.; Ghédira, K. Negotiating decision-makers’ reference points for group preference-based Evolutionary Multi-objective Optimization. In Proceedings of the IEEE International Conference on Hybrid Intelligent Systems, Melacca, Malaysia, 5–8 December 2011. [Google Scholar] [CrossRef]
- Bechikh, S.; Said, L.B.; Ghédira, K. Group preference-based evolutionary multi-objective optimization with non-equally important decision-makers: Application to the Portfolio Selection Problem. Int. J. Comput. Inf. Syst. Ind. Manag. Appl.
**2013**, 5, 278–288. Available online: http://www.mirlabs.org/ijcisim/regular_papers_2013/Paper104.pdf (accessed on 13 May 2021). - Cinalli, D.; Martí, L.; Sanchez-Pi, N.; Bicharra Garcia, A.C. Integrating collective intelligence into evolutionary multi-objective algorithms: Interactive preferences. In Proceedings of the IEEE 2015 Latin American Congress on Computational Intelligence, Curitiba, Brazil, 13–16 October 2015; pp. 1–6. [Google Scholar] [CrossRef]
- Cinalli, D.; Martí, L.; Sanchez-Pi, N.; Bicharra Garcia, A.C. Collective preferences in Evolutio- nary Multi-objective Optimization: Techniques and potential contributions of collective intelligence. In Proceedings of the 30th ACM/SIGAPP Symposium on Applied Computing, Salamanca, Spain, 13–17 April 2015; pp. 133–138. [Google Scholar] [CrossRef]
- Fernandez, E.; Olmedo, R. An outranking-based general approach to solving group multi-objective optimization problems. Eur. J. Oper. Res.
**2013**, 225, 497–506. [Google Scholar] [CrossRef] - Kadzinski, M.; Tomczyk, M.K. Interactive Evolutionary Multiple Objective Optimization for Group Decision Incorporating Value-based Preference Disaggregation Methods. Group Decis. Negot.
**2017**, 26, 693–728. [Google Scholar] [CrossRef] [Green Version] - Borissova, D.; Mustakerov, I. A two-stage placement algorithm with multi-objective optimization and group decision making. Cybern. Inf. Technol.
**2017**, 17, 87–103. [Google Scholar] [CrossRef] [Green Version] - Dietz, T.; Klamroth, K.; Kraus, K.; Ruzika, S.; Schafer, L.; Schulze, B.; Stiglmayr, M.; Wiecek, M. Introducing multiobjective complex systems. Eur. J. Oper. Res.
**2020**, 280, 581–596. [Google Scholar] [CrossRef] [Green Version] - Guarneri, P.; Wiecek, M. Pareto-based negotiation in distributed multidisciplinary design. Struct. Multidiscip. Optim.
**2016**, 53, 657–671. [Google Scholar] [CrossRef] - Roy, B.; Figueira, J.R.; Almeida-Dias, J. Discriminating thresholds as a tool to cope with imperfect knowledge in multiple criteria decision aiding: Theoretical results and practical issues. Omega
**2014**, 43, 9–20. [Google Scholar] [CrossRef] [Green Version] - Liesio, J.; Mild, P.; Salo, A. Preference programming for robust portfolio modeling and project selection. Eur. J. Oper. Res.
**2007**, 181, 1488–1505. [Google Scholar] [CrossRef] - Liesio, J.; Mild, P.; Salo, A. Robust portfolio modeling with incomplete cost information and project interdependencies. Eur. J. Oper. Res.
**2008**, 190, 679–695. [Google Scholar] [CrossRef] - Fliedner, T.; Liesio, J. Adjustable robustness for multi-attribute project portfolio selection. Eur. J. Oper. Res.
**2016**, 252, 931–946. [Google Scholar] [CrossRef] - Toppila, A.; Salo, A. Binary decision diagrams for generating and storing non-dominated project portfolios with interval-valued project scores. Eur. J. Oper. Res.
**2017**, 260, 244–254. [Google Scholar] [CrossRef] - Balderas, F.; Fernandez, E.; Gomez, C.; Rangel, N.; Cruz-Reyes, L. An interval-based approach for evolutionary multi-objective optimization of project portfolios. Int. J. Inf. Technol. Decis. Mak.
**2019**, 18, 1317–1358. [Google Scholar] [CrossRef] - Eklund, P.; Rusinowska, A.; De Swart, H. Consensus reaching in committees. Eur. J. Oper. Res.
**2007**, 178, 185–193. [Google Scholar] [CrossRef] [Green Version] - Herrera-Viedma, E.; Alonso, S.; Chiclana, F.; Herrera, F. A consensus model for group decision making with incomplete fuzzy preference relations. IEEE Trans. Fuzzy Syst.
**2007**, 15, 863–877. [Google Scholar] [CrossRef] - Fu, C.; Yang, S. The group consensus-based evidential reasoning approach for multiple attributive group decision analysis. Eur. J. Oper. Res.
**2010**, 206, 601–608. [Google Scholar] [CrossRef] - Xu, Z.; Cai, X. Group consensus algorithms based on preference relations. Inf. Sci.
**2011**, 181, 150–162. [Google Scholar] [CrossRef] - Fernandez, E.; Bernal, S.; Navarro, J.; Olmedo, R. An outranking-based fuzzy logic model for collaborative group preferences. TOP
**2010**, 18, 444–464. [Google Scholar] [CrossRef] - Moore, R.E. Methods and applications of interval analysis, Studies in Applied and Numerical Mathematics. Soc. Ind. Appl. Math.
**1979**, 23, 121–123. [Google Scholar] [CrossRef] [Green Version] - Yao, S.; Jiang, Z.; Li, N.; Zhang, H.; Geng, N. A multi-objective dynamic scheduling approach using multiple attribute decision making in semiconductor manufacturing. Int. J. Prod. Econ.
**2011**, 130, 125–133. [Google Scholar] [CrossRef] - Keeney, R.; Raiffa, H. Decision with Multiple Objectives: Preferences and Value Tradeoffs; Wiley: New York, NY, USA, 1976; Volume 7. [Google Scholar] [CrossRef]
- Saaty, T.L. The Analytic Hierarchy Process; McGraw-Hill: New York, NY, USA, 1980. [Google Scholar]
- Roy, B. The outranking approach and the foundations of ELECTRE methods. Theory Decis.
**1991**. [Google Scholar] [CrossRef] - Greco, S.; Matarazzo, B.; Słowiński, R. Rough sets theory for multicriteria decision analysis. Eur. J. Oper. Res.
**2001**, 129, 1–47. [Google Scholar] [CrossRef] - Ishizaka, A.; Pearman, C.; Nemery, P. AHPSort: An AHP-based method for sorting problems. Int. J. Prod. Res.
**2012**, 50, 4767–4784. [Google Scholar] [CrossRef] - Roy, B.; Bouyssou, D. Aide Multicritère à la Décision: Méthodes et Cas; Economica: Paris, France, 1993; Available online: https://basepub.dauphine.fr/handle/123456789/4522 (accessed on 13 May 2021).
- Fernández, E.; Figueira, J.; Navarro, J.; Roy, B. ELECTRE TRI-nB: A new multiple criteria ordinal classification method. Eur. J. Oper. Res.
**2017**, 263, 214–224. [Google Scholar] [CrossRef] - Torres-Jimenez, J.; Izquierdo-Marquez, I. Survey of covering arrays. In Proceedings of the 15th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, Timisoara, Romania, 23–26 September 2013; pp. 20–27. [Google Scholar] [CrossRef]
- Zhang, Q.; Li, H. MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition. IEEE Trans. Evol. Comput.
**2007**, 11, 712–731. [Google Scholar] [CrossRef]

It’s Subject | Allows | Related to | |
---|---|---|---|

Assumption 1 | Interval numbers as model of imprecisions | Modeling imprecision | Steps 1–2 |

Assumption 2 | Capacity to identify the best compromise | Identifying best compromises | Step 4 |

Assumption 3 | Compatibility with an outranking model | Preference modeling | Step 1 |

Assumption 4 | Capacity to set the limiting boundaries related to the outranking model | Identifying limiting boundaries | Step 5 |

Assumption 5 | Compatibility with INTERCLASS-nB | Assigning solutions to classes of satisfaction/dissatisfaction | Step 6 |

Assumption 6 | Compatibility with an interval value function | Preference modeling | Step 1 |

Assumption 7 | Capacity to set the limiting boundaries related to the value function model | Identifying limiting boundaries | Step 5 |

O1 | O2 | O3 | O4 | O5 | O6 | O7 | O8 | O9 | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

1,234,925 | 1,337,795 | 995,015 | 1,077,935 | 1,417,065 | 1,535,135 | 929,930 | 1,007,440 | 1,507,725 | 1,633,405 | 1,211,330 | 1,312,210 | 1,925,345 | 2,085,795 | 1,365,315 | 1,479,045 | 1,658,795 | 1,796,975 |

1,259,905 | 1,364,855 | 975,365 | 1,056,625 | 1,474,745 | 1,597,625 | 959,470 | 1,039,430 | 1,480,450 | 1,603,850 | 1,279,330 | 1,385,880 | 1,904,455 | 2,063,165 | 1,486,095 | 1,609,895 | 1,633,700 | 1,769,780 |

1,239,515 | 1,342,765 | 1,024,090 | 1,109,430 | 1,449,225 | 1,569,975 | 924,815 | 1,001,875 | 1,643,530 | 1,780,530 | 1,255,190 | 1,359,740 | 2,009,685 | 2,177,205 | 1,358,450 | 1,471,650 | 1,661,035 | 1,799,395 |

1,189,995 | 1,289,115 | 1,060,215 | 1,148,545 | 1,425,755 | 1,544,545 | 938,105 | 1,016,275 | 1,600,580 | 1,734,010 | 1,312,095 | 1,421,375 | 1,984,560 | 2,149,970 | 1,370,165 | 1,484,325 | 1,610,035 | 1,744,125 |

1,215,945 | 1,317,235 | 1,010,400 | 1,094,590 | 1,429,165 | 1,548,235 | 927,465 | 1,004,755 | 1,634,045 | 1,770,235 | 1,234,430 | 1,337,260 | 1,992,975 | 2,159,115 | 1,383,445 | 1,498,725 | 1,647,005 | 1,784,205 |

1,169,800 | 1,267,240 | 1,015,325 | 1,099,915 | 1,442,685 | 1,562,875 | 962,570 | 1,042,800 | 1,586,845 | 1,719,105 | 1,211,310 | 1,312,200 | 2,012,250 | 2,179,980 | 1,445,540 | 1,565,980 | 1,576,135 | 1,707,435 |

1,211,085 | 1,311,975 | 1,026,425 | 1,111,955 | 1,394,220 | 1,510,390 | 910,105 | 985,945 | 1,588,335 | 1,720,725 | 1,216,135 | 1,317,415 | 1,948,120 | 2,110,500 | 1,358,375 | 1,471,545 | 1,663,015 | 1,801,545 |

1,245,805 | 1,349,585 | 969,110 | 1,049,850 | 1,471,245 | 1,593,835 | 946,720 | 1,025,610 | 1,538,740 | 1,667,020 | 1,231,455 | 1,334,015 | 1,820,550 | 1,972,310 | 1,392,165 | 1,586,475 | 1,596,360 | 1,729,340 |

1,139,045 | 1,233,925 | 1,034,565 | 1,120,765 | 1,377,385 | 1,492,165 | 925,280 | 1,002,380 | 1,507,990 | 1,633,660 | 1,215,730 | 1,317,000 | 1,903,360 | 2,061,980 | 1,429,490 | 1,548,590 | 1,584,005 | 1,715,955 |

1,262,100 | 1,367,230 | 1,030,870 | 1,116,770 | 1,385,135 | 1,500,545 | 931,350 | 1,008,950 | 1,629,665 | 1,765,505 | 1,201,845 | 1,301,955 | 1,932,615 | 2,093,685 | 1,392,655 | 1,508,695 | 1,628,245 | 1,763,875 |

O1 | O2 | O3 | O4 | O5 | O6 | O7 | O8 | O9 | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

1,199,230 | 1,299,120 | 1,038,945 | 1,125,525 | 1,431,025 | 1,550,265 | 909,215 | 984,995 | 1,546,755 | 1,675,685 | 1,243,915 | 1,347,515 | 1,939,700 | 2,101,360 | 1,355,805 | 1,468,735 | 1,647,945 | 1,785,225 |

1,210,965 | 1,311,855 | 1,048,275 | 1,135,635 | 1,411,335 | 1,528,925 | 908,875 | 984,625 | 1,524,965 | 1,652,065 | 1,243,445 | 1,346,995 | 1,955,170 | 2,118,120 | 1,346,425 | 1,458,585 | 1,633,100 | 1,769,130 |

**Table 4.**Best ranked solutions to Problem 1 with N

_{sat}= 10 and N

_{dis}= 0 (after the consensus round).

O1 | O2 | O3 | O4 | O5 | O6 | O7 | O8 | O9 | μ_{sat} | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

1,283,185 | 1,390,075 | 1,045,960 | 1,133,110 | 1,468,445 | 1,590,815 | 924,670 | 1,001,730 | 1,568,155 | 1,698,875 | 1,253,555 | 1,357,965 | 1,922,775 | 2,083,025 | 1,429,465 | 1,548,565 | 1,646,745 | 1,783,915 | 0.8345 |

1,219,040 | 1,320,580 | 1,041,655 | 1,128,455 | 1,471,845 | 1,594,475 | 922,825 | 999,735 | 1,587,925 | 1,720,275 | 1,291,315 | 1,398,885 | 2,028,640 | 2,197,730 | 1,431,800 | 1,551,090 | 1,640,255 | 1,776,895 | 0.8068 |

1,241,255 | 1,344,665 | 997,255 | 1,080,345 | 1,491,285 | 1,615,545 | 922,435 | 999,305 | 1,513,260 | 1,639,390 | 1,301,545 | 1,409,955 | 1,949,135 | 2,111,575 | 1,426,035 | 1,544,825 | 1,655,110 | 1,792,980 | 0.7984 |

1,222,780 | 1,324,630 | 1,053,970 | 1,141,790 | 1,463,215 | 1,585,125 | 925,105 | 1,002,195 | 1,644,730 | 1,781,850 | 1,262,320 | 1,367,460 | 1,976,335 | 2,141,065 | 1,357,765 | 1,470,895 | 1,642,285 | 1,779,075 | 0.7910 |

1,257,785 | 1,362,565 | 1,043,480 | 1,130,430 | 1,462,865 | 1,584,755 | 913,840 | 989,990 | 1,639,715 | 1,776,395 | 1,245,910 | 1,349,680 | 2,005,805 | 2,172,985 | 1,392,880 | 1,508,940 | 1,641,615 | 1,778,365 | 0.7880 |

O1 | O2 | O3 | O4 | O5 | O6 | O7 | O8 | O9 | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

1,172,035 | 1,269,655 | 935,955 | 1,013,925 | 1,293,390 | 1,401,160 | 921,260 | 998,020 | 1,703,365 | 1,845,315 | 1,059,975 | 1,148,295 | 1,831,405 | 1,984,095 | 1,366,425 | 1,480,285 | 1,420,215 | 1,538,515 |

1,210,295 | 1,311,165 | 989,555 | 1,072,025 | 1,322,445 | 1,432,625 | 899,420 | 974,370 | 1,607,385 | 1,741,335 | 1,165,840 | 1,262,970 | 1,915,940 | 2,075,640 | 1,451,480 | 1,572,410 | 1,478,550 | 1,601,720 |

1,169,955 | 1,267,395 | 1,012,675 | 1,097,065 | 1,366,235 | 1,480,075 | 871,970 | 944,640 | 1,771,940 | 1,919,620 | 1,191,690 | 1,290,980 | 1,874,150 | 2,030,360 | 1,469,130 | 1,591,510 | 1,433,435 | 1,552,855 |

1,202,270 | 1,302,430 | 1,006,320 | 1,090,160 | 1,393,975 | 1,510,105 | 897,320 | 972,100 | 1,566,755 | 1,774,835 | 1,163,860 | 1,260,850 | 1,914,190 | 2,073,740 | 1,462,680 | 1,584,530 | 1,479,985 | 1,603,305 |

1,210,395 | 1,311,205 | 1,005,315 | 1,089,095 | 1,379,125 | 1,494,045 | 843,045 | 913,305 | 1,668,500 | 1,807,560 | 1,184,000 | 1,282,630 | 1,869,795 | 2,025,655 | 1,465,640 | 1,587,720 | 1,392,025 | 1,507,935 |

1,134,520 | 1,228,990 | 1,002,865 | 1,086,435 | 1,283,820 | 1,390,790 | 864,205 | 936,225 | 1,667,285 | 1,806,245 | 1,098,580 | 1,190,120 | 1,872,725 | 2,028,815 | 1,356,185 | 1,469,155 | 1,448,755 | 1,569,405 |

1,172,035 | 1,269,655 | 935,955 | 1,013,925 | 1,293,390 | 1,401,160 | 921,260 | 998,020 | 1,703,365 | 1,845,315 | 1,059,975 | 1,148,295 | 1,831,405 | 1,984,095 | 1,366,425 | 1,480,285 | 1,420,215 | 1,538,515 |

1,142,135 | 1,237,275 | 930,485 | 1,008,005 | 1,331,435 | 1,442,355 | 876,760 | 949,810 | 1,619,515 | 1,826,155 | 1,147,440 | 1,243,030 | 1,905,355 | 2,064,205 | 1,465,220 | 1,587,270 | 1,451,000 | 1,571,890 |

1,157,660 | 1,254,110 | 981,860 | 1,063,670 | 1,363,505 | 1,477,115 | 890,910 | 965,140 | 1,535,575 | 1,663,575 | 1,182,040 | 1,280,510 | 1,775,935 | 1,923,955 | 1,396,465 | 1,512,805 | 1,434,775 | 1,554,305 |

1,164,705 | 1,261,715 | 962,750 | 1,042,940 | 1,275,670 | 1,381,990 | 860,665 | 932,405 | 1,605,355 | 1,739,175 | 1,141,455 | 1,236,555 | 1,822,560 | 1,974,440 | 1,452,580 | 1,573,600 | 1,411,810 | 1,529,430 |

O1 | O2 | O3 | O4 | O5 | O6 | O7 | O8 | O9 | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

1220175 | 1321785 | 1014920 | 1099490 | 1376910 | 1491640 | 893130 | 967550 | 1660130 | 1798520 | 1172815 | 1270545 | 1897530 | 2055690 | 1450070 | 1649190 | 1470300 | 1592790 |

**Table 7.**Best ranked solutions to Problem 1 with N

_{sat}= 10 and N

_{dis}= 0 (after the consensus round).

O1 | O2 | O3 | O4 | O5 | O6 | O7 | O8 | O9 | μ_{sat} | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

1,223,260 | 1,325,180 | 1,026,360 | 1,111,900 | 1,455,145 | 1,576,405 | 905,975 | 981,465 | 1,590,645 | 1,723,195 | 1,201,815 | 1,301,915 | 1,980,240 | 2,145,280 | 1,458,810 | 1,580,330 | 1,603,185 | 1,736,745 | 0.7087 |

1,205,000 | 1,305,400 | 1,023,920 | 1,109,250 | 1,452,195 | 1,573,215 | 918,285 | 994,805 | 1,574,665 | 1,705,875 | 1,218,895 | 1,320,415 | 1,963,310 | 2,126,930 | 1,451,450 | 1,572,360 | 1,605,365 | 1,739,105 | 0.7069 |

1,218,835 | 1,320,385 | 1,000,750 | 1,084,130 | 1,486,750 | 1,610,620 | 902,530 | 977,760 | 1,577,835 | 1,709,335 | 1,223,220 | 1,325,120 | 1,886,760 | 2,044,020 | 1,436,160 | 1,555,800 | 1,630,050 | 1,765,860 | 0.7026 |

1,209,115 | 1,309,855 | 1,006,320 | 1,090,170 | 1,500,955 | 1,626,005 | 906,960 | 982,560 | 1,558,585 | 1,688,475 | 1,253,740 | 1,358,180 | 1,901,730 | 2,060,230 | 1,428,970 | 1,548,010 | 1,597,895 | 1,731,025 | 0.7019 |

1,218,590 | 1,320,110 | 1,020,780 | 1,105,840 | 1,434,715 | 1,554,285 | 929,865 | 1,007,325 | 1,618,785 | 1,753,695 | 1,226,165 | 1,328,295 | 1,895,680 | 2,053,670 | 1,468,330 | 1,590,660 | 1,582,605 | 1,714,455 | 0.7015 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Fernández, E.; Rangel-Valdez, N.; Cruz-Reyes, L.; Gomez-Santillan, C.
A New Approach to Group Multi-Objective Optimization under Imperfect Information and Its Application to Project Portfolio Optimization. *Appl. Sci.* **2021**, *11*, 4575.
https://doi.org/10.3390/app11104575

**AMA Style**

Fernández E, Rangel-Valdez N, Cruz-Reyes L, Gomez-Santillan C.
A New Approach to Group Multi-Objective Optimization under Imperfect Information and Its Application to Project Portfolio Optimization. *Applied Sciences*. 2021; 11(10):4575.
https://doi.org/10.3390/app11104575

**Chicago/Turabian Style**

Fernández, Eduardo, Nelson Rangel-Valdez, Laura Cruz-Reyes, and Claudia Gomez-Santillan.
2021. "A New Approach to Group Multi-Objective Optimization under Imperfect Information and Its Application to Project Portfolio Optimization" *Applied Sciences* 11, no. 10: 4575.
https://doi.org/10.3390/app11104575