Next Article in Journal
Conditional Measurements with Silicon Photomultipliers
Next Article in Special Issue
Variant of Data Particle Geometrical Divide for Imbalanced Data Sets Classification by the Example of Occupancy Detection
Previous Article in Journal
The Structural Characteristics of Collagen in Swim Bladders with 25-Year Sequence Aging: The Impact of Age
Previous Article in Special Issue
Natural Language Description of Videos for Smart Surveillance
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A New Approach to Group Multi-Objective Optimization under Imperfect Information and Its Application to Project Portfolio Optimization

1
Dirección de Investigación y Posgrado, Universidad Autonoma de Coahuila, Saltillo 26200, Mexico
2
División de Estudios de Posgrado e Investigación, Cátedras CONACyT—Tecnológico Nacional de México, Instituto Tecnológico de Ciudad Madero, Los Mangos 89440, Mexico
3
División de Estudios de Posgrado e Investigación, Tecnológico Nacional de México, Instituto Tecnológico de Ciudad Madero, Los Mangos 89440, Mexico
*
Authors to whom correspondence should be addressed.
Appl. Sci. 2021, 11(10), 4575; https://doi.org/10.3390/app11104575
Submission received: 12 April 2021 / Revised: 8 May 2021 / Accepted: 10 May 2021 / Published: 17 May 2021
(This article belongs to the Special Issue Integrated Artificial Intelligence in Data Science)

Abstract

:
This paper addresses group multi-objective optimization under a new perspective. For each point in the feasible decision set, satisfaction or dissatisfaction from each group member is determined by a multi-criteria ordinal classification approach, based on comparing solutions with a limiting boundary between classes “unsatisfactory” and “satisfactory”. The whole group satisfaction can be maximized, finding solutions as close as possible to the ideal consensus. The group moderator is in charge of making the final decision, finding the best compromise between the collective satisfaction and dissatisfaction. Imperfect information on values of objective functions, required and available resources, and decision model parameters are handled by using interval numbers. Two different kinds of multi-criteria decision models are considered: (i) an interval outranking approach and (ii) an interval weighted-sum value function. The proposal is more general than other approaches to group multi-objective optimization since (a) some (even all) objective values may be not the same for different DMs; (b) each group member may consider their own set of objective functions and constraints; (c) objective values may be imprecise or uncertain; (d) imperfect information on resources availability and requirements may be handled; (e) each group member may have their own perception about the availability of resources and the requirement of resources per activity. An important application of the new approach is collective multi-objective project portfolio optimization. This is illustrated by solving a real size group many-objective project portfolio optimization problem using evolutionary computation tools.

1. Introduction

Frequently, real-life decision problems need several or many decision-makers (analysts, experts, stakeholders, participants, voters, etc.). A group decision-making problem (GDM) is described as a decision case in which a group of decision-makers (DMs) recognize the existence of a collective problem and are interested in finding a common action (alternative) that could be accepted as a good agreement.
Since the early Condorcet and Borda’s works, including the seminal Arrow’s impossibility theorem, GDM has attracted the attention of researchers for a very long time to the present date (e.g., [1,2,3,4]). Imprecision and vagueness are important issues in GDM. Since GDM is a process carried out by humans, fuzzy set-based methods, with their ability to model vagueness of human judgments and preferences, are especially suitable to address such problems. For a recent paper that revisits fuzzy and linguistic decision-making, the reader is referred to [5]. Some other modern challenges faced by researchers in GDM are: (i) large-scale problems (LSGDM) involving more than twenty decision-makers (e.g., [6,7]); and (ii) manipulation and group dictatorship (e.g., [8]). These challenges have gained increasing importance as a consequence of the modern digital economy and the relevance of social networks.
The main processes for solving GDM are the consensus reaching process (CRP) and the selection process (SP) [9,10,11]. CRPs have the objective of obtaining the maximum degree of satisfaction from the DMs with the collective decision [9]. The search for consensus is an active process, with repetitive interactions (rounds, consensus steps) among group members, which contains a measure of consensus and a feedback method [10]; the last one suggests whether the group members should change their preferences, beliefs, and judgments to allow a better agreement [11,12]. Usually, the CRP is guided by a moderator [13]. The final solution is found through an SP, considering the preferences, beliefs, and judgments of the group members. Frequently, the SP consists of two phases: (A) aggregation of preferences, beliefs, and judgments from the members; and (B) use of these preferences, beliefs, and judgments, that were aggregated collectively, to find a solution which should correspond, as much as possible, to the aggregated group opinions [9,14]. In this paper, our interest is limited to problems in which the decision alternatives are described by multiple criteria, the so-called multi-person-multi-criteria decision making [5]. Here, the aggregation of the group member’s preferences on conflicting criteria plays a crucial role in identifying an acceptable collective agreement.
Usually, the selection process concerns a relatively small set of options; many methods have been proposed to address such cases (e.g., [3,4,15,16,17]). Alternatively, the group multi-objective decision problem on very large decision sets (characterized by constraints) has received comparatively very little attention. According to [18], we call this problem GDM-MOP, denoting the combination of group decision making and multi-objective optimization.
To the best of our knowledge, the existing GDM-MOP approaches are not free of some of the following criticisms:
  • Most interactive methods implicitly assume that group preferences are transitive and comparable relations, although the lack of transitivity is a well-established characteristic of voting systems (e.g., [19]). Even in the case of a single DM, transitivity, and comparability of their preference relation are subject to question, mainly in the presence of veto conditions, and/or when the number of objectives overcomes the cognitive limitations of the human mind.
  • Many methods are susceptible to manipulation. According to classical voting theory, under very general conditions, every voting procedure can be manipulated by some voters by declaring insincere preferences (e.g., [19]).
  • Popular interactive approaches help to obtain acceptable agreements because each DM learns the preferences from the other DMs and correspondingly fits their own. However, the final accepted solution may significantly differ from those that each DM would have considered as satisfactory if the decision had depended solely of them. Thus, the consensus does not result from the search in the set of possible solutions but from mutual concessions. Group satisfaction is partial because it is only achieved by recognizing that a more satisfactory result is not possible.
  • The handle of imprecision, uncertainty, and ill-definition in GDM-MOP is a real concern. GDM-MOP approaches typically assume that the whole group agrees on the resource availability, the resource consumption, and objective values for each point in the decision variable space. However, there could be several sources of imperfect information which affect that assumption. Indeed, each DM may have their own perception (no free of certain imprecision, uncertainty, or arbitrariness) about objective values, available and required resources. Such imperfect knowledge may impact the individual best solutions, on the collective preferences, and the consensus degree. Under imperfect information, the consensus search process is even more difficult and relevant since the diverse perceptions from the DMs and different levels of conservatism should be aggregated and, if possible, agreed.
  • In complex problems, some DMs with very different value systems and/or roles with respect to the other group members may consider different sets of objective functions and constraints. Such a case is not addressed by most of the methods to solve GDM-MOPs.
This paper presents a proposal that strongly reduces the above criticisms. Since group (dis)satisfaction depends on the number of its (dis)satisfied members, the paper focuses on a consensus measure based on counting the number of DMs who are satisfied (respectively dissatisfied) with their respective objective values on a common point in the decision variable space. For each point in the feasible decision set, satisfaction or dissatisfaction from each group member is determined by a multi-criteria ordinal classification approach, based on comparing solutions with a limiting boundary between classes “unsatisfactory” and “satisfactory”. By counting the group members who are (dis)satisfied with solutions in the decision variable space, group satisfaction and dissatisfaction can be simultaneously optimized. This paper has perhaps the merit to be the first in addressing consensus evaluation as multi-criteria ordinal classification problems solved by the entire set of group members. Consensus is identified by a high level of group satisfaction and a low level of dissatisfaction. The group moderator is in charge of making the final decision, finding the best compromise between collective satisfaction and dissatisfaction.
Interval numbers are used to model imperfect information from each group member. This model is less sophisticated than more general fuzzy approaches, but, to a great extent, without their mathematical complexity. Choosing interval numbers as the model of imprecisions allows using recently proposed interval-based multi-criteria ordinal classification methods. Two different decision models are proposed in this paper; the first one uses the interval outranking approach and INTERCLASS-nB method by Fernandez et al. in [20,21]; the second one is based on building and exploiting an interval weighted-sum value function. On one hand, the outranking model is appropriate for handling non-compensatory preferences, allowing veto effects and incomparability situations. On the other hand, the weighted-sum function is, to a certain extent, a simpler and rougher model, which may be recommended to represent compensatory preferences. With a little loss of generality, we address multi-objective maximization problems under resource constraints (as typically in project portfolio optimization), but the method can be trivially extended to other kinds of problems.
The structure of the paper follows: In Section 2, we outline the background on which our proposal is based. Section 2.1 refers to several relevant precedent papers. The concept of good consensus, some fundamental aspects about interval numbers, and an interval-based multi-criteria classification method are briefly described in Section 2.2, Section 2.3, Section 2.4. The problem is detailed in Section 3. Section 4 and Section 5, combined with the concept of maximum consensus, form the core of the proposal. These sections present the mathematical model of what a satisfied/dissatisfied DM is from two points of view: the outranking model of preferences (Section 4) and the weighted-sum model (Section 5). The method is summarized in Section 6; the whole proposal is illustrated by a real-size many-objective project portfolio optimization problem in Section 7. Lastly, several concluding remarks are discussed in Section 8. Two evolutionary algorithms used in solving optimization problems are described in appendices.

2. Background

2.1. An Overview of GDM-MOP Literature

In the GDM-MOP field, some approaches obtain a representative Pareto sample and then apply a method to aggregate individual preferences in a model of collective preference, which is used to find a final solution (e.g., [22,23]). Other popular approaches propose an integration of group preference handling with interactive procedures of multi-objective optimization (e.g., [24,25,26]). In some methods, the interaction is performed during the optimization process. In other methods, the interaction is performed once a Pareto sample has been generated (e.g., [27]).
Efremov et al. in [28] developed a decision support system for e-democracy inspired on Pareto frontier visualization, goal identification, and arbitration. Collective intelligence methods to aggregate reference points from different DMs have been proposed in ([29,30,31,32]) to approach the region of the Pareto frontier that is more preferred by the DMs. Bechickh et al. (2013) proposed a negotiation support system that includes the DMs’ preferences through reference points; priorities on the set of DMs are characterized by weights; the output of a negotiation round is a single group reference point. Xiong et al. (2013) introduced fuzzy reference points; this approach contributes to improving the robustness of the final solution dealing with imprecise and changing preferences.
Fernandez and Olmedo in [33] search for good consensus solutions by maximizing, (respectively, minimizing), the number of group members who are satisfied (resp. dissatisfied) with the current solution in the decision variable space. To solve the bi-objective optimization problem, they used the NSGA2 evolutionary algorithm. The DMs are declared as (dis)satisfied by comparing the current solution with their best compromise.
The NEMO-GROUP, a set of interactive evolutionary multi-objective optimization (MOO) methods, was developed by Kadzinski and Tomczyk in [34]. In these approaches, an evolutionary algorithm is modified with the introduction of pairwise comparisons of several DMs. Solutions are evaluated using utilitarian and egalitarian additive group value functions; the evolutionary algorithms accept weights assigned to the DMs.
Borissova and Mustakerov in [35] presented a two-step placement algorithm, which combines MOO and GDM. First, MOO is used to identify a set of design alternatives for object placement. In the second step, business intelligence and group decision making are used to evaluate design alternatives.
A particular application of MOO is the distributed engineering design, the complex systems built in this area are getting more popularity (e.g., [36]). These systems are often designed by a group of DMs; the design of each subsystem is in charge of a DM. The design teams only have partial information about the global design, and negotiations are relevant to reach a consensus. Guarneri and Wiecek in [37] developed a mathematical model of the problem considering the aspect of distribution and decomposition. The solution of the model is based on Lagrangian relaxation.
An important case of GDM-MOPs is group multi-objective project portfolio optimization. Project portfolio selection is one of the most important problems faced by top-level managers in large enterprises and public organizations. Often, projects are described by several (or many) conflicting criteria, and the selection of the “best” portfolio should be made by a collective entity. This group may be composed of experts in different/complementary fields or members of the top level management. The group members share an interest in finding the portfolio that could bring the best results for the organization, but they have different value systems, beliefs, preferences on conflicting criteria, and judgments about resource consumption and availability. Individual preferences and beliefs are typically modeled by certain parameters and values in a decision model; they can significantly differ from different DMs. If the whole group is seen as an entity, the dispersion of model parameters and values can be considered as imperfect information in the sense of Roy et al. (2014). According to such a paper, imperfect knowledge comes from arbitrariness, imprecision, ill-determination, and uncertainty in data and model parameters ([38]). In project portfolio optimization, imperfect information concerning weights and criterion scores has been addressed by Liesio et al. in [39,40], Fliedner and Liesio in [41], Toppila and Salo in [42], and Balderas et al. in [43] using interval mathematics, although no one of these papers concern the specific characteristics of GDM.

2.2. Toward a Maximum Consensus

According to Fernández and Olmedo (2013), in GDM, reaching an acceptable agreement is not always possible. This comes from strong contradictions amongst important disjoint subsets of group members; for instance, there could be a clear opposition of a numerically significant minority that does not want to accept a preference coming from a not very strong majority of the group. Reaching a collective agreement is only possible when there is unanimity or an appreciable level of consensus. Among many ways to address and define consensus (e.g., [10,14,44,45,46,47]), Fernandez and Olmedo (2013) consider a good consensus as associated with the fulfillment of two conditions:
(A)
There is an important agreeing majority with a particular alternative (or solution);
(B)
There is no appreciable disagreeing minority.
With Condition (B), Fernandez and Olmedo in [33] and Fernandez et al. in [48] rejected “majority dictatorship”, which neglects the importance of the intensity of disagreement and degrades fairness and equity concerning minorities.
Let us consider a group with ng members under the following general premise:
Fundamental Premise: Each solution z in the decision variable space (with its corresponding objective values and feasibility) can be classified by the i-th DM into one and only one of the following classes: (i) z is a satisfactory solution; (ii) z is an unsatisfactory solution; (iii) the i-th DM is neither satisfied nor dissatisfied with z.
Point (iii) concerns situations in which a DM hesitates about the appropriate classification of a solution and confers to the above premise a very high generality.
Let Nsat (resp. Ndis) denote the number of group members who are satisfied (resp. dissatisfied) by a particular solution z. Nsat and Ndis are integer functions of z. The ideal consensus (if possible) should correspond to a point z* such that Nsat(z*) = ng and Ndis(z*) = 0.
Therefore, the ideal consensus is the ideal solution to the bi-objective optimization problem:
Maximize Nsat(z), Minimize Ndis(z)
Unlike typical multi-objective optimization problems in which its ideal solution is not feasible, Problem 1 may reach the ideal point Nsat(z*) = ng and Ndis(z*) = 0 as much as the preferences and beliefs of all the individual DMs become close enough through an effective consensus reaching process. Even when the ideal consensus is not possible, good agreements can be identified when Nsat ≥ 2/3 ng (Condition A) and Ndis ≈ 0 or Ndis << 0.5 ng (Condition B).
For a given closeness of preferences and beliefs of the individual DMs, the best consensus is a non-dominated solution to Problem 1. If this current consensus level is not high enough, a new CRP step should allow identifying better agreements.
To make operational the process of solving Problem 1, we need a mapping between the decision variable space and the values Nsat and Ndis. Therefore, we require a model that permits, for each DM and each z in the decision variable space, knowing to which class of satisfaction/dissatisfaction z is assigned to. Most of the remaining paper is devoted to discussing some preference models and assignment procedures to this end. Since in the objective space, z is described by multiple objective values, its assignment should be considered as a multi-criteria ordinal classification problem.

2.3. Some Fundamental Notions on Interval Mathematics

Moore in [49] defines an interval number as an extension of the concept of a real number and as a subset of the real line ℝ, describing it as a range, E = [ E _ + E _ ], where E _ is the lower limit and E ¯ is the upper limit. In the rest of this paper, an interval number is denoted with boldface italic letters.
Let D and E two interval numbers and F a real number. Basic arithmetic operations are defined below.
D + E = [ D _ + E _ , D ¯ + E ¯ ]
D E = [ D _ E ¯ , D ¯ E _ ]
D · E = [ m i n { D _ · E _ , D _ · E ¯ , D ¯ · E _ , D ¯ · E ¯ } ,   m a x { D _ · E _ , D _ · E ¯ , D ¯ · E _ , D ¯ · E ¯ } ]
D · F = [ D _ · F , D ¯ · F ]
A possibility measure of an order relation over interval numbers is given by Equation (6) [50].
P ( E D ) = { 1 if   p E D > 1 , p E D if   0 p E D 1 , 0 if   p E D 0
where   p E D = E ¯ D _   ( E ¯ E _ ) + ( D ¯ D _ )
If   e ¯ = e _ = e     and   d ¯ = d _ = d P ( E D ) = { 1   if   e d ,   0   otherwise .
According to Fliedner and Liesio (2016), a real number e within the interval [ E _ , E ¯ ] is said to be a realization of the interval number E. In [20], P (DE) = α is interpreted as the degree of credibility that once two realizations are given from E and D, the realization d will be smaller than or equal to the realization e.
According to Fernández et al. (2019), the possibility function satisfies Equation (8). This equation is combined with the interpretation of the order relation in Equation (9).
P ( E D ) = α P ( D E ) = 1 α   ( negation )
E > D P ( E D ) > 0.5   ( strict   order )
A more reliable strict order (denoted by >α) on interval numbers is defined in Equation (10).
E   > α D     P ( E D )     α > 0.5

2.4. Multi-Criteria Ordinal Classification Based on an Interval Outranking Approach

There are more than one hundred multi-criteria decision-making (MCDM) methods. According to the model of the decision-maker’s preferences, most of the methods can be grouped in one of three basic paradigms:
  • The functional paradigm, based on building functions that model the decision-maker’s preferences (e.g., [51,52]);
  • The relational paradigm, based on building crisp or fuzzy binary relations (e.g., [53]);
  • The symbolic paradigm, which is mainly related to artificial intelligence (e.g., [54]).
Whatever the model of the DM’s preferences, we can distinguish three MCDM fundamental problems: multi-criteria choosing, ranking, and sorting. Each MCDM method addresses one of these general problems. For choosing and ranking, multi-attribute utility theory (MAUT) ([51]) and the analytic hierarchy process (AHP) ([52]) are the most popular approaches. Although Ishizaka et al. in [55] proposed a variant of AHP for multi-criteria sorting (also called multi-criteria ordinal classification) problems, outranking and symbolic methods are more popular for this purpose.
Perhaps the most known multi-criteria ordinal classification method is ELECTRE TRI-B ([56]), which was generalized to ELECTRE TRI-nB in ([57]). Such methods make use of an outranking relation, which means that for a pair of decision actions (x,y), we should check the validity of the assertion “action (alternative) x is at least as good as action (alternative) y”. Two effects (concordance and discordance) are combined to determine the degree of credibility of the outranking (cf. [53]). In ELECTRE TRI-B and nB, the boundary between adjacent classes is described by limiting actions. Unlike ELECTRE TRI-B, its extension allows characterizing the boundary between adjacent classes by several limiting profiles. Recently, ELECTRE TRI-nB was extended to handle imperfect information in terms of interval numbers ([21]). The so-called INTERCLASS-nB shares similar assignment rules and consistency properties with ELECTRE TRI-nB, but using the interval outranking approach proposed in ([20]). This handles imperfect information on criterion scores, weights, veto thresholds, and majority thresholds with interval numbers. Even missing criterion scores can be handled by the approach. According to Fernández et al. (2020), “eliciting model preference parameters generally suffer from arbitrariness and imperfect knowledge (in particular, due to ill-determination). This occurs in a more acute way when there is a group of DMs, so that conflicting views are quite natural and frequent.”
The interval outranking approach and INTERCLASS-nB work with several interval preference relations:
Definition 1.
(i) 
xS(β,λ)y ⟺ σ(x,y,λ) ≥ β (interval outranking);
(ii) 
xPr(β,λ)y ⟺ σ(x,y,λ) ≥ β and σ(y,x,λ) < β (interval preference);
(iii) 
xI(β,λ)y ⟺ σ(x,y,λ) ≥ β and σ(y,x,λ) ≥ β (interval indifference);
where:
  • σ is the credibility index of the interval outranking;
  • λ is an interval number representing a majority threshold; λ > [0.5, 0.5] and λmin ≥ 0.5;
  • β is a credibility threshold for establishing a credible crisp outranking relation; β > 0.5.
The INTERCLASS-nB, adapted to the case of only two ordered classes, is presented below:
Conditions on the limiting profiles: Let C1 and C2 be ordered classes (C2 is the most preferred). Set β > 0.5 and λ > [0.5, 0.5] with λmin ≥ 0.5. The boundary between C1 and C2 is characterized by a set of limiting profiles, B = {bj}, such that:
(i) 
All bj of B belongs to C2;
(ii) 
There is no pair (bj, bi) such that bjPr(β,λ)bi.
The method requires the definition of interval outranking and preference relations between actions and the limiting boundary, as follows:
Definition 2.
(i) 
xS(β,λ)B iff there is a w ∈ B such that xS(β,λ)w and there is no y ∈ B with yPr(β,λ)x;
(ii) 
BPr(β,λ)x iff there is a w ∈ B such that wPr(β,λ)x and there is no y ∈ B with xPr(β,λ)y.
The “pessimistic” and “optimistic” assignment rules are:
Definition 3 (“pessimistic” rule).
Suppose Set B satisfies the conditions on the limiting profiles.
  • Step 1: Compare x to B;
  • Step 2: If xS(β,λ)B, then assign x to class C2;
  • Step 3: If not(xS(β,λ)B), then assign x to C1.
Definition 4 (“optimistic” rule).
Suppose Set B satisfies the conditions on the limiting profiles.
  • Step 1: Compare x to B;
  • Step 2: If BPr(β,λ)x, then assign x to class C1;
  • Step 3: If not(BPr(β,λ)x), then assign x to C2.
It is easy to prove that the “optimistic” rule always suggests an assignment to a class not worse than the one suggested by the “pessimistic” procedure.
For a detailed description of INTERCLASS-nB, the reader is referred to [21].

3. Characterization of GDM-MOPs Under-Study

In this paper, we address group multi-objective optimization problems described by the following characteristics:
  • There is a group moderator who is in charge to control and guide the consensus reaching process;
  • The individual DMs participate in the decision process providing information about their preferences, beliefs, and level of conservatism, and modifying this information during consecutive steps of the CRP;
  • Some (even all) objective values may be not the same for different DMs;
  • Each DM may handle a different set of objective functions;
  • The objective values may be imperfectly known (subject to imprecision or uncertainty);
  • The availability of resources may be imperfectly known;
  • Resource requirements per activity (project, in case of portfolio optimization) may be imperfectly known;
  • Each group member has their own opinion about the availability of resources and requirements per activity (project, in case of portfolio optimization);
  • All the DMs consider a common point in the decision variable space.
Points c, d, e, f, g, and h imply that each group member faces their own MOO problem under imperfect knowledge. The model of such imperfect knowledge is based on the following assumption:
Assumption 1.
Each DM is willing to represent as interval numbers the imperfect information on objective values, availability, and requirement of resources.
In the framework of project portfolio optimization, the above statement has been assumed by Liesio et al. (2007, 2008), Fledner and Liesio (2016), Toppila and Salo (2017), and Balderas et al. (2019). This assumption is a bit more restrictive than it appears. As stated by [41], if we assume that a magnitude is an interval number, one should admit that (i) this magnitude can take any value within this interval, and (ii) the DM does not privilege the credibility of any particular value within the range.
Let ARji be the interval number used to represent the estimated availability of the j-th resource from the point of view of the i-th DM; NRji is the interval number used to denote the aggregation of required resources, which depends on the decision variables vector z.
In order to find a feasibility condition for the i-th DM, a solution z must satisfy ARji > ξi NRji (z) according to the order relation given by Equation (10). The i-th DM faces the MOP in Equation (11).
Maximize z R F i F i = ( f 1 i ( z ) , f 2 i ( z ) ,   , f N i ( z ) )
where Ni is the number of the evaluation criterion for the i-th member of a group; Fi is the vector function being considered by the i-th DM; z is the vector of decision variables; in project portfolio optimization, z is a vector composed of binary variables, in which “1” means “the related project belongs to the portfolio”, and “0” otherwise.
RFi is the feasible region from the point of view of the i-th DM. It is defined by the interval-based resource constraints ARji > ξi NRji (z), and perhaps other non-resource constraints.
In Problem 11, ξi is the credibility threshold for the expression “the available resources are sufficient to satisfy the requirements”. The more conservative the i-th DM, the higher the value of ξi.
Definition 5 (α-dominance).
Let Oi be the image of the feasible region of Problem 11. Consider an element (x,y) belonging to Oi × Oi, where x ≠ y, and α a real number in [0.5, 1]. The solution y is α-dominated by x (denoted by xD(α)y) iff min{P(fji(x) ≥ fji(y)), j = 1, …, Ni} = α and P(fki(x) ≥ fki(y)) > 0.5 for some k.
Identifying satisfactory solutions is a preference judgment. Such a judgment is related to what the DMs consider as their maximum aspiration levels, which come from the best solution to Problem 11. Therefore, the DM should be able to determine such levels.
The best solution to Problem 11 is ill-defined due to the conflicting nature of its objective functions. The following two definitions are useful to characterize a necessary condition to be the best compromise solution.
Definition 6 (a preferred solution by the i-th DM).
A solution x ∈ Oi is preferred to y∈ Oi by the i-th DM if the following statement is considered as true by them: “x is at least as good as y, but y is not at least as good as x”.
Remark 1.
The interval preference relation from Definition 1 (xPr(β,λ)y) and the dominance relation from Definition 5 (xD(α)y) are good arguments in support to “x is preferred to y” for sufficiently high values of β and α.
Definition 7 (necessary conditions to be the best compromise solution for the i-th DM).
xi* ∈ Oi is the best compromise solution to Problem 11 for the i-th DM only if it fulfills two conditions:
(A) 
There is no y in Oi such that y is preferred to xi* by the i-th DM;
(B) 
There are arguments to justify that the i-th DM considers xi* as at least as good as many solutions that satisfy Condition A.
Assumption 2 (capacity to identify the best compromise solution from the i-th DM’s point of view).
Let Oi be the image of the feasible region of Problem 11. Each individual DM is able to solve Problem 11, thus identifying their best compromise solution xi* ∈ Oi.
The way in which the i-th DM can find their best compromise depends on the model of multi-criteria preferences; this should correspond to how the DM aggregates preferences on conflicting criteria. As stated in Section 2, there are three main paradigms: (i) the functional approach; (ii) the relational approach; and (iii) symbolic methods from Artificial Intelligence, mainly those based on rough sets theory. To the best of our knowledge, no symbolic multi-criteria decision method in the frame of interval numbers has been reported. The functional approach bases its prescriptions on a value or utility function, fulfilling axioms of full comparability and transitivity; these value or utility functions are compensatory models in which poor scores of some criteria can be compensated by good scores of other ones. There are a few papers that have used interval-based weighted sum functions as decision tools in project portfolio selection problems (e.g., [39,40,41]). To our knowledge, more complex forms of interval-based value functions have not been proposed to address multi-objective optimization problems; further, interval value functions have not been used in solving multi-criteria classification problems. The relational paradigm is generally focused on building and exploiting outranking relations; this approach can handle non-compensatory effects, veto situations, incomparability, and non-transitive preferences. In the frame of interval numbers, the method in ([20]) allows building credible interval outranking relations, and the INTERCLASS-nB method in ([21]) is able to suggest appropriate assignments in multi-criteria ordinal classification problem under imperfect information.

4. Model of Preferences and Judgments of a DM with a Non-Compensatory Aggregation of Preferences

In this section, we suppose that the i-th DM is compatible with the interval outranking model by Fernández et al. (2019). We refer to a generic DM, and for simplicity, we avoid the use of the subscript “i”. Firstly, let us present an overview of the interval outranking approach from [20].

4.1. The Interval Outranking Model

The method requires the following assumption:
Assumption 3 (modeling arbitrariness and ill-definition of parameters).
The DM is willing to specify an outranking model in terms of weights, veto thresholds, a majority threshold, and a credibility threshold. In order to model imperfect information, those parameters can be specified as interval numbers. Therefore, the weights will be denoted as wj = [wj, wj+], the veto thresholds as vj = [vj, vj+], the majority threshold as λ = [λ, λ+], and the credibility threshold as β = [β, β+].
The credibility index δj(a′, a) of the assertion “a′ is at least as good as a regarding criterion fj” is
δj(a′, a) = P(fj(a′) ≥ fj(a)) j = 1, …, N
where P is the possibility function of Equation (6).
We say that criterion fj is γ-concordant with the statement “a′ is at least as good as a regarding criterion fj” (denoted a′Sja) if and only if a′Sja with a credibility index of at least γ. The set of criteria fj such that δj(a′, a) ≥ γ (γ = min{δj}) is called γ-possible concordance coalition with “a′ is at least as good as a” and is denoted by C(a′Sγa). γ is the credibility that all criteria in C(a′Sγa) agree that “a′ outranks a”. All criteria not in C(a′Sγa) form the γ-possible discordance coalition, D(a′Sγa).
For consistency reasons, the interval weights should satisfy the following constraints:
j = 1 N w j 1 , j = 1 N w j + 1 .
The concordance index of “a′ outranks a” is an interval number denoted by c(a′, a) = [c(a′, a), c+(a′, a)], and calculated as follows:
c ( a ,   a ) = f j C ( a S γ a ) w j
when
f j C ( a S γ a ) w j + f j D ( a S γ a ) w j + 1
otherwise,
c ( a ,   a ) = 1 f j D ( a S γ a ) w j +
similarly,
c + ( a ,   a ) = f j C ( a S γ a ) w j +
when
f j C ( a S γ a ) w j + + f j D ( a S γ a ) w j 1
otherwise,
c + ( a   a ) = 1 f j D ( a S γ a ) w j
Since the set of criteria in C(a′Sγa) is determined by γ, the concordance index c depends on such a value. Such a dependence is denoted by c(a′, a, γ).
Each possible concordance coalition C(a′Sγa) provides reasons in favor of “a′ outranks a”, and each possible discordance coalition D(a′Sγa) provides reasons against it. In the following, we denote as dj(a′,a) the credibility index of the statement “a′Sa is vetoed by gj”. dj(a′,a) is defined as P(fj(a) ≥ fj(a′) + vj), where vj is the interval veto threshold related to fj.
The credibility of a possible outranking given a γ-possible concordance coalition and the corresponding discordance coalition is defined as follows:
Definition 8.
Let Ω be the set {δj > 0, j = 1, …, N}. Given γ Ω we say that a′ outranks a for the γ-possible concordance coalition with credibility index σγ and majority threshold λ > 0.5 with λ > 0.5 iff σγ = min {γ, P(c(a′, a, γ) ≥ λ), (1 − m a x f j D ( a S γ a ) dj(a′,a))}.
For each particular value of γ, this index can be interpreted as a credibility degree of the conjunction of two predicates: (i) “the γ-concordance coalition is strong enough” and (ii) “the γ-discordance coalition does not exert veto”.
The above marginal (single coalition) credibility indices are merged by using the “max” operator in the following definition:
Definition 9 (interval outranking credibility index).
From the point of view of the interval outranking approach and under the above notation, a′ outranks a (“a′ is at least as good as a”) with credibility index σ(a′,a,λ) = max{σγ} (γ ∈ Ω) and majority threshold λ > 0.5 with λ > 0.5. If Ω is empty, then σ(a′,a,λ) is zero.
Let β an interval number (0.5 <β < 1 with β > 0.5) considered as a credibility threshold to establish crisp preference relations. In the following we will use a′S(β,λ)a, a′I(β,λ)a, and a′Pr(β,λ)a similarly to Definition 1, but using the interval number β instead of the real number β. Such a change gives more flexibility to the DM in setting the credibility threshold.
Remark 2.
(a) 
As was proved in [20], if fj(a) are real numbers for j = 1, …, N, then aS(1,1)a.
(b) 
a′s(β,λ)a ⇒ a′S(ξ,λ)a ∀ ξ < β.
Proposition 1.
Some properties of the interval dominance combined with the interval outranking and preference relations:
For all (x,y,z) ∈ O × O × O:
i. 
yD(α)x and xS(β,λ)z ⇒ yS(ε,λ)z for some ε ≥ min(α,β)
ii. 
zPr(β,λ)y and yD(α)x ⇒ zPr(ε,λ)x for some ε ≥ min(α,β)
iii. 
yD(α)x and xPr(β,λ)z ⇒ yPr(ε,λ)z for some ε ≥ min(α,β)
iv. 
If α > 0.5, then xD(α)y ⇒ xPr(β,λ)y for some β ≥ α and for all λ ≤ [1,1]
v. 
xD(α)y and yD(η)z ⇒ xD(ε)z with ε = min (α,η)
The proofs are similar to the ones in [20]. The single difference consists of replacing the real value of β with the middle points of the interval number β.

4.2. Finding the Best Compromise solution to Problem 11

According to Assumption 2, each DM is capable of finding their best solution to Problem 11. Here we discuss a method for this purpose.
Definition 10 ((β,λ) non-strictly outranked solution).
A solution x ∈ O such that there is no y ∈ O fulfilling yPr(β,λ)x is called a (β,λ) non-strictly outranked solution. The (β,λ) non-strictly outranked frontier of Problem 11 is the set of these solutions.
Remark 3.
(i) 
With appropriate values of β and λ, a non-strictly outranked solution fulfills Condition A of Definition 7, that is, the first necessary condition to be the best compromise.
(ii) 
Condition B of Definition 7 is proved on the non-strictly outranked frontier, using the outranking strength measure. This measure is described as OS(x) = card {ai ∈ NSF such that xS(β,λ)ai}, where NSF denotes the (β,λ) non-strictly outranked frontier.
(iii) 
More than one solution can fulfill the necessary conditions of Definition 7. The solution selected as the final best compromise should be one with the highest value of the outranking strength.
Problem 11 can be solved through the following steps:
  • The individual DM (perhaps helped by a decision analyst) sets their model parameters according to Assumption 3.
  • The (β,λ) non-strictly outranked frontier is identified by an optimization algorithm; the set of solutions that fulfill Definition 10 is determined;
  • The DM selects the best compromise solution according to Remark 3.iii.
The optimization algorithm for the step 2 depends on the problem characteristics. Balderas et al. (2019) proposed an evolutionary algorithm called I-NOSGA for solving project portfolio optimization problems with several or many interval-valued objective functions and with an outranking relation defined on the objective space. I-NOSGA separates each population into non-strictly outranked fronts. An extension of I-NOSGA to address the optimization problems in this paper is provided in Appendix A.

4.3. Making Judgments of Satisfaction and Dissatisfaction

Let x* be the best compromise solution obtained by an individual DM by exploiting the interval outranking relation of Section 4.1.
Assumption 4 (capacity to set the limiting boundary between classes).
Let Csat and Cdis denote classes “satisfactory” and “unsatisfactory”, respectively. Consider β > 0.5 (β > 0.5) and λ > 0.5 (λ > 0.5). Taking x* as reference, the DM is able to set a boundary B= {b1, … bn} between Csat and Cdis fulfilling:
i. 
fj(bk) are real numbers, j = 1, … N and k = 1, … n;
ii. 
Each bk (k = 1, … n) belongs to Csat;
iii. 
For all bk belonging to B, we have x*Pr(β,λ)bk;
iv. 
There is no pair (bi, bk) such that biPr(β,λ)bk.
Let us back to Definition 2, which establishes relations between solutions and the boundary. Replace β with the interval number β. Assumption 4 guarantees the conditions to apply INTERCLASS-nB (see Section 2.4) to making judgments of satisfaction and dissatisfaction.
From Definition 2, it follows that xS(β,λ)Bnot (BPr(β,λ)x) and BPr(β,λ)xnot (xS(β,λ)B). If a solution x outranks the limiting boundary B, x is assigned to the best class by the “pessimistic” procedure (Definition 3); additionally, since xS(β,λ)Bnot (BPr(β,λ)x), x is also assigned to Csat by the “optimistic” procedure (Definition 4) contrarily, if the boundary is preferred to x, since BPr(β,λ)xnot (xS(β,λ)B), x will be assigned to the worst class by both assignment rules. If not (xS(β,λ)B and not (BPr(β,λ)x) are both fulfilled, then x is assigned to Cdis by the “pessimistic” rule and to Csat by the “optimistic” procedure. In such a case, the DM may be doubtful about the class x should be assigned to.
Assumption 5 (compatibility with INTERCLASS-nB).
Suppose that x and its pre-image satisfy the constraints imposed by the i-th DM. We assume that the DM is willing to accept the assignment suggested by INTERCLASS-nB.
As a consequence of Assumption 5, even if xS(β,λ)B or not (BPr(β,λ)x), the nonfulfillment of the constraints vetoes a satisfactory assignment. Based on this assumption, the definitions of what is a satisfactory (non-satisfactory) solution for a DM (who is compatible with the interval outranking model) are straightforward.
Definition 11.
Suppose that an individual DM is compatible with the outranking model of Section 4. The DM is said to be satisfied with a solution x iff the following conditions are fulfilled:
(a) 
xS(β,λ)B
(b) 
x and its pre-image satisfy the constraints imposed by the DM.
Definition 12.
Under the same premise of the above definition, the DM is said to be dissatisfied with a solution x iff at least one of the following conditions is fulfilled:
(A) 
BPr(β,λ)x
(B) 
x and/or its pre-image do not satisfy the constraints imposed by the DM.
Definition 13.
The DM is neither satisfied nor dissatisfied with a solution x if the following conditions are all fulfilled:
1. 
not (xS(β,λ)B)
2. 
not (BPr(β,λ)x)
3. 
x and its pre-image satisfy the constraints imposed by the DM.
It is obvious that the above definitions follow from the INTERCLASS-nB assignment rules and Assumption 5.
Suppose that the individual DM is satisfied with a solution x. Let us define the outranking credibility index of x with respect to the boundary B as follows:
σ(x, B, λ) = max {σ(x, bk, λ), k = 1, … card(B)
The value of σ can be interpreted as a measure of how much the DM feels satisfaction with x. The following proposition states several consistency properties of the assignments from Definitions 11–13.
Proposition 2 (consistency properties of assignments).
Under Assumptions 4 and 5, suppose that x and its pre-image satisfy the constraints imposed by the DM. The following propositions are valid:
(a) 
x is assigned to a single element of the set of classes (satisfactory, unsatisfactory, neither satisfactory nor unsatisfactory).
(b) 
The assignment suggested for x is independent of the assignment of other solutions.
(c) 
The class to which x is assigned by the i-th DM is independent of the assignment made by any other DM.
(d) 
Let y be a feasible solution. Given λ, if x and y have the same interval outranking credibility indices with respect to the limiting profiles, then they are assigned to the same element of the set of classes (satisfactory, unsatisfactory, neither satisfactory nor unsatisfactory).
(e) 
If there is bk ∈ B fulfilling x = bk, then x is assigned to the satisfactory class.
(f) 
If there is bk such that xI(β,λ)bk and there is no bi ∈ B fulfilling biPr(β,λ)x, then x is assigned to the satisfactory class.
(g) 
If x = x*, then x is assigned to the satisfactory class.
(h) 
Let y be a feasible solution such that y D(α)x (α ≥ β). If x is assigned to the satisfactory class, then y is assigned to the same class.
Proof. 
Proposition 2(a):
The proof follows from two facts: (i) x has to fulfill one of the three Definitions 11–13, and (ii) the fulfillment of one definition excludes fulfillment of another.
Proposition 2(b):
The proof is obvious from Definitions 11–13.
Proposition 2(c):
The proof is obvious from Definitions 11–13.
Proposition 2(d):
The proof is obvious from Definitions 11–13.
Proposition 2(e):
x = bk, Assumption 4.i, Remark 2.a, Definition 2.i and Assumption 4.iv ⇒ xS(β,λ)B x is feasible and xS(β,λ)Bx is satisfactory for the DM (Definition 11).
Proposition 2(f):
xI(β,λ)bkxS(β,λ)bk (Definition 1.iii)
xS(β,λ)bk and there is no biB such that biPr(β,λ)xxS(β,λ)B (Definition 2.i)
x is feasible and xS(β,λ)Bx is satisfactory for the DM (Definition 11).
Proposition 2(g):
The proof follows trivially from Assumption 4.iii, Definition 2.i and Definition 11.
Proposition 2(h):
x is assigned to CsatxS(β,λ)B from Definition 11 ⇒ bkB such that xS(β,λ)bk and there is no biB with biPr(β,λ)x.
  • From Proposition 1.i and Remark 2.b, yD(α)x (αβ) and xS(β,λ)bkyS(β,λ)bk.
  • There is no biB with biPr(β,λ)x and y D(α)x (αβ) ⇒ There is no biB with biPr(β,λ)y counter-reciprocal of Proposition 1.ii.
 □
Combining (A) and (B), we have yS(β,λ)B (Definition 2.i) ⇒ y is assigned to Csat (Definition 11). The proof is finished.
It should be noticed that the first condition in Assumption 4 (real values in the limiting actions) is required to prove Property 2.e and thus achieving consistency with Condition ii in Assumption 5. From Propositions 2.e and 2.f, it follows that a solution only slightly different from any bkB should be considered as satisfactory by the DM.

5. Model for a DM Whose Preferences Are Compatible with a Weighted-Sum Function

5.1. The Preference Model

In this section, we suppose that the i-th DM’s preferences are compatible with a simple interval weighted-sum value function of the form:
U = Σwjfj (j = 1, … N)
In Equation (18), U is an interval number that is calculated through the arithmetic operations defined in Section 2.3. We refer to a generic DM, and for simplicity, we avoid the use of the subscript “i”.
Assumption 6 (modeling arbitrariness and ill-definition of parameters).
The DM is willing to set a weighted-sum function model in terms of weights and objective function values. With the purpose of modeling imperfect information, those magnitudes will be considered as interval numbers.
This assumption was implicitly made by Liesio et al. (2007, 2008) and Fliedner and Liesio (2016) in the frame of project portfolio optimization.
Remark 4.
Let us consider a pair (x,y) in the objective space of Problem 11. The conditions P(U(x) ≥ U(y)) > 0.5 or xD(0.5)y may not suffice to guarantee a credible preference favoring x over y. However, there should be a credibility threshold α > 0.5 such that P(U(x) ≥ U(y)) ≥ α and/or xD(α)y are good arguments to justify a credible preference.
Definition 14 (α-preference).
Given a pair (x,y) in the objective space of Problem 11 and α a real number greater than 0.5, we say that x is α-preferred to y iff at least one of the following conditions is fulfilled:
a. 
P(U(x) ≥ U(y)) ≥ α
b. 
xD(α)y

5.2. Identifying the Best Compromise Solution to Problem 11 with the Functional Preference Model

We follow here a rather similar way as in Section 4.2 to make operational Assumption 2 in the frame of the functional model of preferences.
Definition 15 (α non-strictly outranked solution).
A solution x ∈ O will be called α non-strictly outranked iff there is no y ∈ O such that y is α-preferred to x. The set of these solutions is called the α non-strictly outranked frontier of Problem 11.
Remark 5.
  • With an appropriate value of α, a non-strictly outranked solution from Definition 15 fulfills Condition A of Definition 7, that is, the first necessary condition to be the best compromise solution of Problem 11.
  • Condition B of Definition 7 is verified through a value strength measure on the non-strictly outranked frontier. This measure is defined as VS(x) = card {y ∈ NSF such that x is 0.5-preferred to y}, where NSF denotes the α non-strictly outranked frontier.
  • Several solutions can fulfill the necessary conditions of Definition 7. The final best compromise should be one of the solutions with the highest measure VS.
Under the interval value function approach, Problem 11 can be solved through the following steps:
  • The individual DM (perhaps helped by a decision analyst) sets the interval weights in Equation 18 and the α value.
  • An optimization algorithm is used to identify the α non-strictly outranked frontier.
  • The set of solutions that fulfill Definition 10 is identified.
  • The DM selects the best compromise solution according to Remark 5.III.
Similarly to Section 4, the appropriate optimization algorithm in the second step depends on the characteristics of the problem. For combinatorial problems like project portfolio optimization, we can exploit the α-preference defined on O × O with only slight changes with respect to the I-NOSGA proposed in [38] (see Appendix A for a detailed description).

5.3. Making Judgments of Satisfaction and Dissatisfaction with the Functional Model

We will present here an idea that is, to a certain extent, similar to the one on which INTERCLASS-nB and the proposal in Section 4.3 are inspired. If the DM is capable of setting a limiting boundary between the two classes, all feasible solutions preferred to the boundary should be assigned to the best class and vice versa.
Formally: Let x* be the best compromise solution obtained by the individual DM following the procedure exposed in the previous section.
Assumption 7 (capacity to set the limiting boundary between classes).
Let Csat and Cdis denote classes “satisfactory” and “unsatisfactory”, respectively. Consider a sufficiently high value of the credibility threshold α. Taking x* as reference, the DM is able to set a limiting boundary B= {b1, … bn} between Csat and Cdis fulfilling:
i. 
For all bk belonging to B, we have x* is α-preferred to bk;
ii. 
There is no pair (bi, bk) in B × B such that bi is α-preferred to bk;
iii. 
For all bk belonging to B, the DM hesitates about its assignment.
Definition 16 (α-preference between a solution and the boundary).
(a) 
x is α-preferred to B iff there is bk ∈ B such that x is α-preferred to bk and there is no bi ∈ B such that bi is α-preferred to x.
(b) 
The boundary B is α-preferred to x iff there is bk ∈ B such that bk is α-preferred to x and there is no bi ∈ B such that x is α-preferred to bi.
Definition 17.
Consider x ∈ O and bk ∈ B. The credibility index of the statement “x is preferred to bk” is defined as σP (x, bk) = Maximum χ fulfilling at least one of the following conditions:
a. 
P(U(x) ≥ U(bk)) = χ
b. 
xD(χ)bk
The credibility index σP (x, B) of the statement “x is preferred to the boundary B” is defined as:
σP (x, B) = Max {σP (x, bk)}, k = 1, … card (B)
σP (x, B) can be interpreted as the degree of credibility of the statement “x is preferred to the boundary by the i-th DM”.
Definition 18.
Suppose that an individual DM is compatible with the weighted-sum function model and Assumptions 6 and 7. The DM is said to be satisfied with a solution x iff the following conditions are fulfilled:
-
x is α-preferred to the Boundary B;
-
x and its pre-image fulfill the constraints imposed by the DM.
Definition 19.
Suppose that an individual DM is compatible with the weighted-sum function model and Assumptions 6 and 7. The DM is said to be dissatisfied with a solution x if at least one of the following conditions is fulfilled:
-
B is α-preferred to x;
-
x and/or its pre-image do not fulfill the constraints imposed by the DM.
Definition 20.
The DM is neither satisfied nor dissatisfied with a feasible solution x iff the following conditions are held:
-
x is not α-preferred to the Boundary B;
-
B is not α-preferred to x.
Proposition 3 (consistency properties of assignments).
Suppose that x is feasible. The following propositions are valid:
(a) 
x is assigned to a single element of the set of classes (satisfactory, unsatisfactory, neither satisfactory nor unsatisfactory).
(b) 
The assignment suggested for x is independent of the assignment of other solutions.
(c) 
The class to which x is assigned to by the i-th DM is independent of the assignment made by any other DM.
(d) 
If there is bk ∈ B fulfilling x = bk, then x is neither satisfactory nor unsatisfactory.
(e) 
If x = x*, then x is assigned to the satisfactory class.
(f) 
Suppose that x is α-preferred to all bk ∈ B. Let y be a feasible solution such that yD(α)x. Then y is assigned to the satisfactory class.
Proof. 
Proposition 3(a):
The proof is obvious from Definitions 16, 18, 19, and 20.
Proposition 3(b):
The proof is obvious from Definitions 16, 18, 19, and 20.
Proposition 3(c):
The proof is obvious from Definitions 16, 18, 19, and 20.
Proposition 3(d):
From the way in which B is built (Assumption 7), there is no bi ∈ B such that bk is α-preferred to bi or bi is α-preferred to bk. Then, bk is not α-preferred to B and B is not α-preferred to bk (Definition 16). From Definition 20, it follows that bk is neither satisfactory nor unsatisfactory.
Proposition 3(e):
The proof follows from Definitions 16 and 18 and the way in which the limiting boundary is built (Assumption 7).
Proposition 3(f):
It is evident that y D(α)x and P(U(x) ≥ U(bk)) ≥ α ⇒ P(U(y) ≥ U(bk)) ≥ α. In addition yD(α)x and xD(α)bk⇒ yD(α)bk from transitivity of dominance (Proposition 1.v). Hence, y is α-preferred to the Boundary B (Definitions 14 and 16). From Definition 18, y is assigned to the satisfactory class. The proof is finished.
 □
Remark 6.
(a) 
Unlike the proposal in Section 4, the solutions in the limiting boundary do not belong to any class. Objectives of these solutions may be described by interval numbers, what gives more flexibility to the DM and may reduce their cognitive effort.
(b) 
Proposition 3.d is consistent with Condition iii of Assumption 7. It follows that a solution only slightly different from any bk ∈ B should be considered neither satisfactory nor unsatisfactory by the DM.

6. Summary of the Method

The models and methods described in Section 4 and Section 5 are alternatives. The DMs may select the model of preferences they feel more compatible with. Each DM sets their own model parameters, solves their own Problem 11, identifies their best compromise, and defines their limiting boundary according to Assumption 4 or Assumption 7. Then, it is possible to evaluate the number of satisfied and dissatisfied DMs, exploring the decision variable space to solve Problem 1. Since the preference models and their parameters differ from two disjoint subsets of the whole group, the consensus reaching process should be carried in different subgroups.
The proposal maintains its validity even if the DMs face different optimization problems (with different objectives and constraints) but with common decision variables ranging in the intersection of the feasible regions of each DM optimization problem. The main reasons for such a general performance come from Propositions 2.c and 3.c and the independent way in which each DM sets their limiting boundary and defines what a satisfactory (unsatisfactory) solution is. The specific characteristics, preferences, and judgments from each DM influence on determining whether particular solutions are satisfactory or not for the respective group member. The collective final decision comes from optimizing (Nsat, Ndis) in Problem 1, but the calculation of these objectives is made by aggregating the independent assignments of all the DMs.
It should be underlined that the proposal is, to a great extent, robust with respect to malicious intents of manipulation from some group members. To analyze this issue, suppose that some DMs are interested in manipulating the final solution. To achieve this, they should try (providing insincere information), to increase Nsat(z*) and reduce Ndis(z*) in the points z* where they are really satisfied, and to decrease Nsat(z**) and increase Ndis(z**) in the points z** where they are really dissatisfied. The information required from them is basically of three kinds: (i) parameters of their preference model; (ii) solutions describing the limiting boundary between classes “satisfactory” and “unsatisfactory”; (iii) their constraints and objective values. This information determines if the DMs are satisfied or dissatisfied and counted in Nsat or Ndis. If they report insincere information on any of the three previous aspects, solutions that are really satisfactory for them could become unsatisfactory according to the model, and vice versa. This does not contribute to reaching a final solution really satisfactory for the DMs that provide insincere information. In addition, each DM should be impeded to know the limiting boundaries elicited by other DMs. Under this restriction, the i-th DM cannot evaluate solutions from the point of view of other DMs, being thus unable to determine what insincere information should provide for maliciously influencing other opinions. Therefore, during collective discussions, the i-th DM should defend their real preferences and beliefs and try to convince others.
The method for finding the maximum consensus can be summarized as steps:
  • Helped by a decision analyst/moderator, the DMs select the model of multi-criteria preferences that they consider as more appropriate. The group is separated into two disjoint subgroups in correspondence to the model of preferences that were chosen by each DM.
  • In each subgroup, under the guidance of the moderator, the DMs bring their positions closer. They exchange opinions about the objective functions to consider, the objective values, the related model’s parameter values, levels of conservatism, and constraints.
  • Each group member sets their multi-objective optimization problem (Problem 11) and their model’s parameter values. Interval numbers can be used according to Assumptions 1, 3, and 6.
  • According to Assumption 2, each group member obtains their best compromise solution by solving Problem 11.
  • Each DM sets their limiting boundary in correspondence to Assumption 4 (when the DM is compatible with the outranking model) and Assumption 7 (for DMs compatible with the functional model).
  • Applying the classification models of Section 4 and Section 5, the moderator finds non-dominated solutions (Nsat, Ndis) of Problem 1.
  • If there is no solution of good agreement, further discussions in each subgroup are needed to close divergent beliefs, preference parameters, and constraint settings. We need to update these data.
  • The DMs should judge whether, given the updated data, they want to modify their limiting boundary. In the case of “yes”, restart the process in Step 5. In the case of “no”, restart the process in Step 6. If a good consensus (Nsat, Ndis)* is found, then:
  • If the pre-image of (Nsat, Ndis)* is a single point in the decision variables space, this point corresponds to the best consensus, and the process finishes. Else:
  • Apply some additional criterion to select a single pre-image of (Nsat, Ndis)*. The process finishes.
The method is based on seven assumptions. Table 1 summarizes the set of assumptions and their role within the approach.
All the DMs should accept Assumptions 1 and 2. DMs who are compatible with the outranking model should agree to Assumptions 3–5, whereas the other DMs should accept Assumptions 6 and 7.
Remark 7.
i. 
In the case of project portfolio optimization, the computational cost depends mainly on Step 4. The computational complexity of this step is linear with respect to the number of applicant projects (see the description of the I-NOSGA algorithm in Appendix A).
ii. 
Handling group interactions in Steps 1, 2, and 7 is the main difficulty to extend the proposal to problems with many DMs. In such problems, Steps 3, 4, 5, and 8 should be performed by each DM in an independent and parallel way. Parallel processing in Step 4 is strongly recommended. Steps 9 and 10 are independent of the number of group members. The computational effort in Step 6 is, at most, proportional to the number of DMs (see Appendix C). Therefore, with some modifications in Steps 1, 2, and 7, the proposal can be used in large-scale GDM-MOPs.
iii. 
In Step 7, in order to accept a solution as a good consensus, the group may agree previously appropriate values of Nsat and Ndis to represent what a strong majority and a weak minority mean, respectively.
iv. 
In Step 10, there could be several (even many) pre-images of the best consensus (Nsat, Ndis)*. To choose a single one, the group and/or its moderator can use different points of view (e.g., impacts of the solutions, resource consumption, who are the satisfied and dissatisfied DMs, number of supported projects in case of portfolio optimization, etc.). Perhaps the most elegant way is a logical approach based on the outranking credibility index of a solution with respect to the limiting boundary (see Equations (17) and (19)).
Let us denote as zG1, …, zGL the points in the decision variable space, which are all pre-image of the best consensus (Nsat, Ndis)*. Let Cag denote the agreeing coalition (the set of group members who are satisfied with the image of zGk). In the paragraph below, we use “i” (respectively “j”) for the DMs who are compatible with the outranking (resp. weighted sum functional) model.
If we denote as ZGki (resp. ZGkj) the image of zGk in the original objective space Oi (resp. Oj), σi (ZGki, Bi, λi) (resp. σPj(ZGkj, Bj)) can be interpreted as the degree of truth of the predicate “the i-th DM (resp. the j-th DM) considers that ZGki (resp. ZGkj) outranks the limiting boundary Bi (resp. Bj)”. Then, the degree of truth of the predicate “all the DMs belonging to the agreeing coalition consider that ZGki (resp. ZGkj) outranks the related limiting boundary” can be calculated as the conjunction of all the values σi(ZGki, Bi, λi) and σPj(ZGkj,Bj), where “i” (resp. “j”) is the index of the i-th DM (resp. the j-th DM) within the agreeing coalition. Such a truth value is calculated as:
μsat(zGk) = min{σi(ZGki, Bi, λi), σPj(ZGkj, Bj)}
i, j ∈ Cag
Remark 8.
Although the proposal exhibits remarkable advantages in comparison with previous approaches to group multi-objective optimization, it can be still criticized from the following points:
  • To set the limiting boundaries could be a hard cognitive task for DMs; it would be more complex in large scale problems.
  • The bi-objective measure of collective satisfaction/dissatisfaction does not contain information about which DMs are strongly (dis)satisfied. This information can be important to discriminate among non-dominated solutions of Problem 1. Perhaps the multi-criteria ordinal classification method should take into account more classes of satisfaction/dissatisfaction, but this would require much more cognitive effort from the DMs in defining more limiting boundaries.
  • The role of the moderator is crucial in choosing the final consensus decision among the non-dominated solutions of Problem 1. A model of consensus agreed by the group would be welcome. Such a model should aggregate the information about satisfaction/dissatisfaction, thus helping to make the final choice among non-dominated solutions to Problem 1.

7. An Illustrative Example of Project Portfolio Optimization

In this section, we will apply both models of multi-criteria preferences to the many-objective project portfolio optimization problem addressed by Balderas et al. (2019). For simplicity of our illustrative purpose and with a little loss of generality, we consider that all the DMs are compatible with the same model (interval outranking or interval weighted-sum function).
The case study involves 100 applicant social projects whose impacts are described by nine objectives. We suppose that all the DMs agree on the same set of objective functions. The description of the project impacts and their budget requirements are shown in: https://www.dropbox.com/s/t80u9kbdcub6jua/Projects%20Description.pdf?dl=0 (accessed on 12 May 2021).
The objectives are the number of beneficiaries identified per social group and the level of impact of this benefit. The group is integrated by 10-equally important DMs, who may interact several rounds until reaching consensus. Consistently with Assumption 1, let us suppose that in the first interaction, the moderator achieved an initial agreement of the DMs on the project impacts and budget requirements, given as interval numbers. These intervals, specified in dollars, represent the variability of the DM’s judgments and beliefs. Derived from the first interactive round, the feasible region, which is defined by the available budget, is set as the interval [240,260] million dollars.

7.1. Solution When All the DMs Accept the Interval Outranking Model and Its Assumptions

Based on Assumption 3, the parameter values of the model (specified by the group members) are shown in Table A2 (See Appendix B). Weights and vetoes are the same as in [38].
Problem 11 was solved by each group member through the procedure explained in Section 4.2 and the evolutionary algorithm in Appendix C with the parameters given in Table A2a–c. The best compromise solution was identified by each DM (Assumption 2), as shown in Table 2. The conditions under which our computer experiments were carried out are the following: (1) the algorithms were implemented using the Java programming language and run on a computer with Intel Core i7-6700HQ 2.6 GHz CPU, 8 GB of RAM, and Windows 10 Pro operating system; and (2) the solutions were obtained from 30 independent runs. The parameter values for the algorithms were tuned through a covering array experimental design [58]. The involved factors were population size, number of evaluations of the objective function vector and mutation probability, with levels of {50, 100, 200}, {10,000, 100,000, 200,000} and {0.01, 0.02, 0.05}, respectively. The results from the fine-tuning process indicated as the best configuration the following values: Population size = 100, number of evaluations = 100,000, and mutation probability = 0.02. The time required by the extended I-NOSGA to solve Problem 11 per group member was of 153 min and 56 s.
Taking their best compromise as an ideal reference, the DMs selects their associated limiting boundary between Csat and Cdis. Each boundary is composed of three solutions that fulfill the conditions in Assumption 4. The boundaries are described in Table A3 in Appendix B.
There is (β,λ)-preference favoring the best compromise in its comparison with the solutions in the limiting boundary (Condition iii in Assumption 4), but there is no α-dominance.
Now, for identifying the best consensus, we should solve Problem 1. We use MOEA/D, whose brief description can be found in Appendix C. This algorithm found two solutions in the decision variable space, both fulfilling Nsat = 9 and Ndis = 0. In the objective space, such solutions are given in Table 3. The time consumed by MOEA/D to solve Problem 1 was 35 min and 59 s.
There is no dissatisfaction with such solutions. All the DMs are satisfied except a single one who is neither satisfied nor dissatisfied. It is a high consensus level that may be accepted as the final collective decision. However, we suppose that the group moderator wishes to carry on a consensus reaching step to close even more the group judgments and beliefs.
Suppose that, under the guidance from the moderator; the group members make closer criterion weights, veto thresholds, credibility thresholds α, β, and ξ, and the required budget per project. We made a computer simulation of this consensus round as follows:
The procedure in Algorithm 1 is applied for Parameter p in the set MP = {lower bound of wj (j = 1, …, 9), upper bound of wj (j = 1, …, 9), lower bound of vj (j = 1, …, 9), upper bound of vj (j = 1, …, 9), lower bound of β, and upper bound of β, α, ξ}.
Algorithm 1. Consensus Round Simulation
For each p in MP
Let pi be the value of Parameter p set by the i-th DM.
Let pa denote the average value of p on the set of DMs.
Repeat from i = 1 to 10
Calculate d = pipa
If d > 0, update pi as pid/2
If d < 0, update pi as pi + |d/2|
If d = 0, then pi keeps its value
End of Repeat
End of For
Applying Algorithm 1, the distance between the previous value of p from the i-th DM and the average value of such a parameter on the set of DMs is reduced by half. The interval budget required by projects is updated from their original values (see https://www.dropbox.com/s/t80u9kbdcub6jua/Projects%20Description.pdf?dl=0 (accessed on 12 May 2021)) in such a way that the new interval number has the same middle point but whose length is reduced by half. The updated weights, vetoes, and credibility thresholds are provided by Table A4a–c, and the updated budget per project is shown in Table A5 (see Appendix B). λ remains the same as in Table A2c.
With the updated values, again, MOEA/D is used in solving Problem 1. Since preferences, attitudes facing imperfect knowledge, and judgments from the DMs are closer than before; the optimization problem is now more relaxed. The ideal consensus Nsat = 10, Ndis = 0 is achieved, with many pre-images in the decision variable space. In this example, the solutions satisfying the ideal consensus are ordered following the values of μsat in Equation (20). The first ranked solutions are shown in Table 4.
The values σi(ZGki, Bi, λi) from Equation 17 are provided by Table A6 (see Appendix B). These values represent measures of the level of satisfaction for each group member.

7.2. Solution When All the DMs Accept the Interval Dum Function Model and Its Assumptions

The criterion weights in this model are in general different from the weights in an outranking model. Based on Assumption 6, for simplicity and without loss of generality, we set in this Section the same values provided by Table A2a. The model of Section 5 requires credibility thresholds α and ξ. These are set as in Table A2c. Specifications about projects and requirements are the same as in https://www.dropbox.com/s/t80u9kbdcub6jua/Projects%20Description.pdf?dl=0 (accessed on 12 May 2021).
Problem 11 was solved by each group member through the procedure explained in Section 5.2 and the evolutionary algorithm in Appendix C with the parameters given in Table A2a,c. According to Assumption 2, the best compromise solution for each DM was determined, as shown in Table 5. Under the same conditions defined in the previous experiments, the extended I-NOSGA algorithm required 37 min and 47 s per group member to solve Problem 11.
With their best compromise as an ideal reference, the DMs identify their associated limiting boundary between Csat and Cdis. Each boundary is composed of three solutions that fulfill the conditions in Assumption 7. The boundaries are detailed in Table A7 in Appendix B.
Once the limiting boundaries have been set, the evaluation of functions Nsat and Ndis is straightforward through Definitions 18–20. Using MOEA/D (see Appendix C), a single non-dominated solution of Problem 1 was found, with Nsat = 6, Ndis = 0. In the original nine-objective space, this solution is given in Table 6.
As in Section 7.1, suppose that, under the guidance from the moderator, the group members make closer criterion weights, credibility thresholds α and ξ, and the required budget per project. Our computer simulation of this consensus round, similarly to Section 7.1, is performed through Algorithm 1. Here, the procedure in Algorithm 1 is applied for Parameter p in the set MP = {lower bound of wj (j = 1, …, 9), upper bound of wj (j = 1, …, 9), α, ξ}. Since the simulation algorithm is identical, the updated parameters are the same as in Table A4a,c. The interval budget required by projects is updated identically to the way followed in Section 7.1 (see the updated values in Table A5, Appendix B).
Solving Problem 1, again, MOEA/D identified many solutions in the decision variable space, which satisfies the best consensus Nsat = 10 and Ndis = 0. To select a small subset of solutions, we use the “min” operator for conjunction as in Equation 20. The first ranked consensus solutions are provided in Table 7.
The values σPi(ZGki, Bi) from Equation 19 are shown in Table A8 (see Appendix B). These represent the level of satisfaction for each DM.
Remark 9.
The example has illustrated the importance of the assumptions on which the proposal is based, the way to build and use both model of multi-criteria preferences, the search for optimum consensus solutions, the handle of imperfect information through interval numbers, and the modeling of conservatism from individual DMs when they face this imperfect information. For simplicity and space consumption reasons, the example assumed that all the DMs face the same set of objective functions, and consider the same resource consumption per project and the same resource availability, although the DMs may have their own conservatism attitude concerning these issues; the ill-definition of proper collective judgments is modeled by interval numbers.

8. Concluding Remarks

The criticisms to GDM-MOP approaches discussed in the introduction have been basically overcome. To our knowledge, this paper has presented one of the most comprehensive approaches to group multi-objective optimization under imperfect information. Its remarkable generality is supported by several important features: (a) some group members may be compatible with an outranking model of preferences, but other ones may feel comfortable with a preference model based on a weighted-sum value function; (b) the use of outranking models allows to handle veto situations, incomparability and intransitive preferences, whereas the value function model is more appropriate for compensatory aggregation of preferences; (c) some (even all) objective values maybe not the same for different group members; (d) group members may consider their own set of objective functions and constraints; (e) objective values may be imprecise or uncertain; (f) imperfect information on available resources and requirements may be handled; and (g) each group member may have their own perception about the availability of resources and the requirement of resources per activity. It should be remarked that, although the problem was formulated in terms of maximization with resource constraints as in project portfolio optimization, the extension of the method to other formulations is straightforward.
Another important feature is related to the definition of what a satisfied group member is. Unlike other group decision making approaches, that definition is independent of the satisfaction stage of other group members. In this paper, the group members’ stage of satisfaction/dissatisfaction is assigned by using multi-criteria ordinal classification approaches. These assignments fulfill several consistency properties whose theoretical proofs have been provided by the paper. The level of group satisfaction is represented by a bi-objective measure containing the number of group members who are satisfied/dissatisfied with each particular solution. Taking as bases (i) a limiting boundary (set by each group member) between classes “satisfactory” and “unsatisfactory”, (ii) an outranking/preference relation, and (iii) an algorithm able to optimize the bi-objective measure of satisfaction/dissatisfaction, the method is capable of finding (within the decision variable space) the highest group satisfaction related to each particular stage of the consensus reaching process.
The whole approach is based on seven reasonable assumptions. The first two assumptions are common to both models of preferences (outranking and weighted-sum function). Assumptions 3–5 support the outranking model, whereas Assumptions 6–7 are basic for the functional model. Assumptions 4 and 7 show only a very slight difference; Assumptions 3 and 6 play the same role in each model of multi-criteria preferences.
In our approach, a good consensus is associated with a high level of satisfaction and a very low level of dissatisfaction. If the bi-objective measure of collective satisfaction/dissatisfaction is not judged as sufficiently close to the desired consensus level, then the group should perform an additional consensus round in which preference parameters, judgments, beliefs, and attitudes facing imperfect knowledge, should be closed. If, after several consensus rounds, the bi-objective measure of collective satisfaction/dissatisfaction does not reach the desired consensus level, the group (or the group moderator) should make a final decision among the different non-dominated solutions in the space of satisfaction/dissatisfaction.
To reach high consensus levels requires collaborative attitudes from the group members; this is frequent in project portfolio optimization since the individual DMs represent the common organizational interest. However, as a consequence of the independence of judgments about satisfaction/dissatisfaction, the method can work even in cases of very conflicting views from the individual DMs, helping to identify the best possible partial consensus. During discussions and exchange of opinions, the group members should report their real preferences and beliefs. There is no way to manipulate the consensus solution maliciously by declaring insincere preferences and beliefs.
The present proposal is capable of modeling priorities on the set of group members; when the moderator has sufficient authority, (s)he can weigh the different non-dominated solutions (group satisfaction, group dissatisfaction) according to who are the satisfied and dissatisfied group members.
The method (with both models of preferences) behaved well in an illustrative group many-objective project portfolio optimization problem. For simplicity, we used the same parameters for both models, what is a rough approximation since the weights have different meanings. The ideal consensus was achieved with a single consensus round. There is no obstacle to the combined use of both preference models.
Another interesting feature of our approach is its potential to handle large scale GDM-MOPs. If parallel processing is used in solving Problem 11, the computational cost of the proposal depends weakly on the number of DMs (see Remark 7.ii in Section 6). With only minor changes, the method could be adapted to handle such optimization problems as an avenue of future research.
Other directions for future research are the following:
  • Since consensus is affected by intense (dis)satisfaction, we require to model also high satisfaction and strong dissatisfaction. This could be addressed by multi-criteria ordinal classification methods, but the model would be more complex due to the increment of classes.
  • Development of models for making an appropriate selection of the best consensus among non-dominated solutions of Problem 1. Logic-based models representing predicates like “a strong majority is satisfied with …” and “a very weak minority disagrees with …” may be used to select one of the non-dominated solutions in the space of collective satisfaction/dissatisfaction. This would permit to reduce, perhaps replace, the role of the moderator in choosing the final decision. These models should be able to reflect intense satisfaction and dissatisfaction from the group members.
  • To alleviate the DM hard cognitive task in assessing limiting boundaries, this is more relevant as the numbers of objective functions and DMs increase.

Author Contributions

Conceptualization, E.F., L.C.-R. and N.R.-V.; methodology, E.F., C.G.-S., N.R.-V. and L.C.-R.; software, N.R.-V. and C.G.-S.; validation, E.F., L.C.-R., N.R.-V. and C.G.-S.; formal analysis, E.F., L.C.-R., C.G.-S.; investigation, L.C.-R., E.F., N.R.-V. and C.G.-S.; writing—original draft preparation, E.F., C.G.-S. and N.R.-V.; writing—review and editing, C.G.-S., L.C.-R., E.F. and N.R.-V. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The instances and other files used here are available at https://www.dropbox.com/s/t80u9kbdcub6jua/Projects%20Description.pdf?dl=0 (accessed on 12 May 2021).

Acknowledgments

Authors thanks to CONACYT for supporting the projects from (a) Cátedras CONACYT Program with Number 3058. (b) CONACYT Project with Number A1-S-11012 from Convocatoria de Investigación Científica Básica 2017–2018 and CONACYT Project with Number 312397 from Programa de Apoyo para Actividades Científicas, Tecnológicas y de Innovación (PAACTI), (c) CONACYT Project with Number 236154 and (d) CONACYT Project with Number 280081.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. The Extended I-NOSGA Method

The I-NOSGA algorithm was proposed by Balderas et al. (2019). Using an outranking relation, this method identifies non-strictly outranked portfolios (modeled using interval mathematics) during the evolutionary search, satisfying a set of constraints that can be described by intervals. In the present paper, we use a variant of the original algorithm.
The pseudo-code of the main loop of I-NOSGA is shown in Algorithm A1. The present work slightly modifies the algorithm to allow a generalization that admits the use of different models of preferences (PM), and strength measures (SM), which are appropriate for the specific characteristics of each decision maker; such elements of the algorithm are used to define the preference relation that supports the incorporation of the DM’s preferences in the evolutionary search. The experiments in this paper explore two different PMs, one based on outranking, and the other based on a weighted-sum value function; each model is associated with its SM.
The Algorithm A1 starts by combining the existing population of parents and children in Line 1; then, using the specified preference model PM, the algorithm builds the non-outranked fronts (see Line 2). The fronts F formed in this way are used to create the new generation of parents PopT+1 (see Lines 5–11). The algorithm orderly includes complete fronts in PopT+1 (Line 6) and complete it with the best solutions from the last front Fi that does not fit entirely; the solutions are taken in order according to the specified SM (see Lines 9 to 11). Finally, a new generation of individuals QT+1 is evolved from PopT+1 using the genetic operators chosen for this purpose (see Line 12).
Algorithm A1. Interval Non-Outranked Sorting Genetic Algorithm
Input:
PopT, the population of parents,
QT, the children generated in the previous iteration,
PM, the binary preference model used to compare pairs of solutions (x,y)
SM, the computation model for the strength measure of built solutions
Output: Next generation of parents PopT+1 and children QT+1
01: RT = PopTQT//combine parents and children population
02: F = sort-by-preferences (RT, |PopT|, PM)//create outrank fronts F = (F0, F1,) from RT using PM
03: PopT+1 = Ø//initialize new population PopT+1
04: i = 0
05: while |PopT+1| + |Fi| ≤ N do//fill the new population set PopT+1
06:  PopT+1 = PopT+1Fi//include front Fi that fits completely in PopT+1
07:  i = i + 1//move to next front in the order set F
08: end
09: FS = strength- assignment(Fi, PM, NSF)//measures the strength of the solutions in Fi, NSF = F0
10: F′i = SORT(Fi, FS)//sort solutions in Fi by FS in descending order
11: PopT+1 = PopT+1F′i[1:N-|PopT+1|]//complete PopT+1 with best solutions in FI when |PopT+1| < N
12: QT+1 = make-new-pop(PopT+1)//construct next generation of children QT+1 using PopT+1 and the chosen operators for selection, crossover and mutation
13: T = T + 1//iterate
Observe that the pseudocode presented in Algorithm A1 reflects the main loop of I-NOSGA, i.e., the actions that are repeated until a user-defined stop criterion is met. In addition, note that the initialization strategies for the parent PopT and children QT populations are left to the user, and the best solutions will be in the front F0 of the last iteration.
Figure A1 represents the individual encoding (chromosome) used by the evolutionary algorithm. This is a binary array x of size n (the number of projects) in which the i-th element (gen) contains 1 if the i-th project belongs to the portfolio and 0 otherwise. Parents are chosen by binary tournament; the solution belonging to the better front is selected for crossover; ties are broken randomly. The implemented strategy used a one-point crossover and flip mutation. The mutation probability for each allele was set to 0.02.
Figure A1. Binary representation of a portfolio in the decision variable space.
Figure A1. Binary representation of a portfolio in the decision variable space.
Applsci 11 04575 g0a1
Table A1 details the worst-case time complexity of the main loop of I-NOSGA. The analysis considers the variables S, N, and P as the size of the population, the number of criteria, and the number of candidate projects, respectively.
Table A1. I-NOSGA time complexity analysis.
Table A1. I-NOSGA time complexity analysis.
Complexity
01: RT = PopTQT O(S)
02: F = sort-by-preferences (RT, |PopT|, PM) O(N2S2) *
03: PopT+1 = ∅O(1)
04: i = 0O(1)
05: while |PopT+1|+|Fi|≤ N doO(S)
06: PopT+1 = PopT+1FiO(S)
07: i = i + 1O(1)
08: end
09: FS = strength-assignment(Fi, PM, NSF)O(S)
10: Fi = SORT(Fi, FS)O(S2)
11: PopT+1 = PopT+1 ∪ F’i[1:N-|PopT+1|]O(S)
12: QT+1 = make-new-pop(PopT+1) O(NPS)
13: T = T + 1O(1)
* Using the fast non-dominated sort proposed by Deb.
Let’s point out that, based on the analysis shown in Table A1, the complexity of I-NOSGA linearly scales w.r.t. to the number of candidate projects P, i.e., its complexity is O(P). This complexity plays a vital role because there might be thousands of candidate projects in some realistic scenarios.

Appendix B. Updated Budget Requirements

Table A2. Parameters values of the model: (a) weight, (b) veto thresholds and (c) credibility and majority thresholds.
Table A2. Parameters values of the model: (a) weight, (b) veto thresholds and (c) credibility and majority thresholds.
(a)
DMO1O2O3O4O5O6O7O8O9
10.0660.1420.1080.160.0640.1340.0960.1620.050.0830.0920.1620.0780.1280.0530.0810.1380.211
20.0650.140.1090.1630.0780.1350.0950.1650.0540.080.1060.1690.0660.1430.0540.0820.1450.25
30.0640.1260.1040.1780.0640.130.0840.140.0530.0820.0890.170.0780.120.0550.0850.150.231
40.0640.1410.0960.1630.0650.1280.0940.1680.0530.0920.0920.1750.080.1380.0550.090.1450.222
50.080.1360.090.180.0770.140.0850.1550.0510.0830.1020.1550.0750.140.060.0960.1460.216
60.0730.1210.0980.1530.0710.130.10.1510.0590.0850.0960.1730.0730.1350.0480.0930.120.219
70.0770.1230.0960.1580.0770.1380.0970.1590.0530.090.0920.160.0780.1230.0480.0830.1420.245
80.0680.1430.1070.170.0670.1390.0860.1580.0590.0890.1090.1690.0760.130.050.0860.130.243
90.070.1350.0930.1770.0680.1370.090.1480.0530.0820.1030.1690.0790.1270.0520.0840.150.223
100.0760.130.090.1540.0690.1360.0870.1470.0520.0950.0990.1760.080.1280.0530.0910.1320.234
(b)
DMO1O2O3O4O5O6O7O8O9
1195,989195,989234,398234,398169,919169,919169,919169,919169,919169,919169,919169,919169,919169,919169,919169,919301,255301,255
2256,187256,187245,076245,076298,408298,408298,408298,408298,408298,408298,408298,408298,408298,408298,408298,408329,296329,296
3195,544195,544132,601132,601205,307205,307205,307205,307205,307205,307205,307205,307205,307205,307205,307205,307378,402378,402
4126,797126,797141,914141,914199,808199,808199,808199,808199,808199,808199,808199,808199,808199,808199,808199,808226,441226,441
5222,376222,376173,008173,008207,355207,355207,355207,355207,355207,355207,355207,355207,355207,355207,355207,355254,049254,049
6276,395276,395231,282231,282173,649173,649173,649173,649173,649173,649173,649173,649173,649173,649173,649173,649262,777262,777
7184,680184,680161,316161,316159,873159,873159,873159,873159,873159,873159,873159,873159,873159,873159,873159,873189,960189,960
8129,004129,004194,205194,205224,768224,768224,768224,768224,768224,768224,768224,768224,768224,768224,768224,768284,196284,196
9265,641265,641140,995140,995249,798249,798249,798249,798249,798249,798249,798249,798249,798249,798249,798249,798375,703375,703
10150,909150,909194,222194,222185,065185,065185,065185,065185,065185,065185,065185,065185,065185,065185,065185,065275,222275,222
(c)
DMαξΛΒ
10.750.750.510.670.660.77
20.670.670.510.670.600.70
30.650.670.510.670.600.67
40.660.670.510.670.600.67
50.680.700.510.670.600.70
60.740.750.510.670.660.76
70.750.750.510.670.660.77
80.770.780.510.670.660.78
90.780.800.510.670.670.80
100.730.750.510.670.650.75
In Table A2c the notation is the following.
α
credibility threshold for dominance
ξ: 
credibility threshold for sufficiency of resources
λ
interval majority threshold
β
credibility threshold for the crisp interval outranking.
Table A3. Limiting boundaries.
Table A3. Limiting boundaries.
Frontiers B
O1O2O3O4O5O6O7O8O9
DM1b11,234,9251,036,4751,417,065968,6851,570,5651,211,3301,925,3451,365,3151,658,795
b21,251,9261,036,4751,417,065968,6851,570,5651,211,3301,891,4031,365,3151,658,795
b31,234,9251,036,4751,342,175968,6851,570,5651,234,3731,925,3451,365,3151,658,795
DM2b11,259,905975,3651,474,745999,4501,542,1501,279,3301,983,8101,486,0951,633,700
b21,296,052975,3651,474,745999,4501,542,1501,230,2241,983,8101,486,0951,633,700
b31,259,905975,3651,475,379999,4501,542,1501,279,3301,983,8101,374,7611,633,700
DM3b11,291,1401,024,0901,509,600924,8151,712,0301,255,1902,009,6851,358,4501,661,035
b21,291,1401,024,0901,509,600924,8151,712,0301,278,3242,009,6851,276,8351,661,035
b31,291,1409,725,561,509,600924,8151,712,0301,255,1902,106,2291,358,4501,661,035
DM4b11,189,9951,060,2151425755977,1901,667,2951,312,0952,067,2651,370,1651,610,035
b21,189,995982,4451441611977,1901,667,2951,312,0952,067,2651,370,1651,610,035
b31,212,5031,060,2151425755977,1901,667,2951,266,8052,067,2651,370,1651,610,035
DM5b11,215,9451,010,4001,488,700927,4651,702,1401,234,4301,992,9751,383,4451,715,605
b21,230,0051,010,4001,488,700927,4651,702,1401,234,4301,936,6671,383,4451,715,605
b31,215,9451,033,8671,488,700927,4651,702,1401,234,4301,992,9751,268,8061,715,605
DM6b11,169,8001,057,6201,502,780962,5701,652,9751,211,3102,012,2501,445,5401,576,135
b21,169,8001,057,6201,502,780962,5701,652,9751,211,3101,821,4821,448,2561,576,135
b31,170,9821,057,6201,502,780962,5701,652,9751,211,3102,012,2501,445,5401,420,488
DM7b11,211,0851,026,4251,394,220910,1051,588,3351,266,7752,029,3101,414,9601,663,015
b21,153,1271,026,4251,394,220921,7311,588,3351,266,7752,029,3101,414,9601,663,015
b31,211,0851,037,8231,394,220910,1051,576,1241,266,7752,029,3101,414,9601,663,015
DM8b11,297,6951,009,4801,471,245946,7201,538,7401,231,4551,820,5501,489,3201,596,360
b21,297,6951,009,4801,358,643946,7201,538,7401,231,4551,820,5501,489,3201,598,868
b31,297,6951,009,4801,471,245933,4771,541,0831,231,4551,820,5501,489,3201,596,360
DM9b11,139,0451,077,6651,377,385925,2801,570,8251,215,7301,982,6701,429,4901,584,005
b21,139,0451,077,6651,377,385925,2801,570,8251,154,9861,982,6701,429,4901,606,499
b31,142,9601,077,6651,377,385925,2801,570,8251,215,7301,982,6701,358,9011,584,005
DM10b21,262,1001,030,8701,442,840931,3501,629,6651,251,9001,932,6151,450,6751,628,245
b31,262,1001,030,8701,442,840931,3501,629,6651,251,9001,959,6141,450,6751,540,127
b11,262,100975,0971,442,840953,0111,629,6651,251,9001,932,6151,450,6751,628,245
Table A4. The updated (a) weights after the consensus round, (b) veto thresholds after the consensus round, and (c) credibility thresholds after the consensus round.
Table A4. The updated (a) weights after the consensus round, (b) veto thresholds after the consensus round, and (c) credibility thresholds after the consensus round.
(a)
DMO1O2O3O4O5O6O7O8O9
10.0680.1000.1050.1430.0720.1080.0980.1330.0570.0770.0980.1350.0820.1140.0570.0780.1420.189
20.0680.0990.1050.1440.0790.1150.0970.1330.0590.0790.1040.1420.0760.1080.0580.0780.1450.192
30.0670.0990.1050.1410.0720.1080.0920.1270.0580.0780.0970.1340.0820.1140.0580.0790.1440.195
40.0670.0990.1010.1370.0730.1090.0970.1320.0580.0780.0980.1350.0830.1150.0580.0790.1450.192
50.0650.1070.0980.1340.0790.1150.0920.1280.0570.0770.1030.1400.0810.1120.0610.0810.1460.193
60.0690.1030.1020.1380.0760.1120.0990.1350.0610.0810.1000.1370.0800.1110.0550.0750.1330.180
70.0670.1050.1010.1370.0790.1150.0980.1340.0580.0780.0980.1350.0820.1140.0550.0750.1440.191
80.0690.1010.1060.1430.0740.1100.0930.1280.0610.0810.1020.1440.0810.1130.0560.0760.1380.185
90.0700.1020.1000.1360.0740.1100.0950.1300.0580.0780.1040.1410.0830.1140.0570.0770.1440.195
100.0670.1050.0980.1340.0750.1110.0930.1290.0580.0780.1020.1390.0830.1150.0570.0780.1390.186
(b)
DMO1O2O3O4O5O6O7O8O9
1198,171198,171190,206190,206198,904198,904200,051200,051200,166200,166200,178200,178200,179200,179200,179200,179306,014306,014
2172,435172,435184,867184,867192,629192,629196,072196,072196,416196,416196,451196,451196,454196,454196,454196,454301,513301,513
3197,948197,948168,769168,769216,598216,598217,745217,745217,860217,860217,872217,872217,873217,873217,873217,873276,960276,960
4163,575163,575173,425173,425213,848213,848214,996214,996215,111215,111215,122215,122215,123215,123215,123215,123268,607268,607
5189,340189,340188,972188,972217,622217,622218,769218,769218,884218,884218,896218,896218,897218,897218,897218,897282,411282,411
6162,331162,331191,764191,764200,769200,769201,916201,916202,031202,031202,043202,043202,044202,044202,044202,044286,775286,775
7192,516192,516183,126183,126193,881193,881195,028195,028195,143195,143195,155195,155195,156195,156195,156195,156250,367250,367
8164,678164,678199,571199,571226,328226,328227,476227,476227,591227,591227,602227,602227,603227,603227,603227,603297,485297,485
9167,708167,708172,966172,966216,934216,934220,377220,377220,721220,721220,756220,756220,759220,759220,759220,759278,309278,309
10175,631175,631199,579199,579206,477206,477207,624207,624207,739207,739207,751207,751207,752207,752207,752207,752292,998292,998
(c)
DMAξβ
10.700.720.620.70
20.690.700.620.67
30.680.700.620.67
40.690.700.620.67
50.700.710.620.67
60.710.720.620.70
70.700.720.620.70
80.690.700.620.70
90.690.690.620.70
100.710.720.630.69
Table A5. Updated budget requirements after the consensus round.
Table A5. Updated budget requirements after the consensus round.
ProjectCostProjectCostProjectCostProjectCost
192609640269522990851819785337654725698
26235648527969710,093525962620877965710,053
357726008289535992553745077507874507750
476657975298222855854732776237950725278
593629748309012937855654268088082608600
674107710315972621856972710,1238165926858
776757985329065943557549057108274227728
895129898335370559058878091408369627248
973607660349085945559785581758484958845
1056025828358085841560636066208557906030
1176477963365380560061621764738678558175
1249905190377677799362588061208783458685
1357475983389372975863561258388860026248
1485908940397470777064956599558977408060
15793082504069227208658657901390970710,103
1680458375419012937866789082109160006240
1784108750429412979867656568359273927698
1853875603435032523868976710,1639355925818
1963406600447982830869816584959496059995
2078508170456052629870606563159565726838
2193609740469087946371832086609650125218
2281958525477850817072638066409788309190
2359106150486787706373920795839856855915
2457876023496217647374979710,1939953775593
25523754535077608080756052629810056955925
Table A6. Values of σi(ZGki, Bi, λi) for the solutions in Table 4.
Table A6. Values of σi(ZGki, Bi, λi) for the solutions in Table 4.
Sol.DM1DM2DM3DM4DM5DM6DM7DM8DM9DM10
10.98160.87960.89580.83640.83450.84220.88141.00000.99210.9386
20.86430.97640.84790.80680.86260.81611.00001.00001.00000.8892
30.90300.81960.85260.90270.84180.81290.94270.85290.96300.7984
40.88080.90540.79750.89040.79100.93990.84851.00000.99771.0000
51.00000.90250.85590.79860.78800.83740.84351.00001.00000.9423
Table A7. Limiting boundaries (three limiting solution for each DM).
Table A7. Limiting boundaries (three limiting solution for each DM).
O1O2O3O4O5O6O7O8O9
1,172,0351,196,440935,955955,4481,293,3901,320,333921,260940,4501,703,3651,738,8531,059,9751,082,0551,831,4051,869,5781,366,4251,394,8901,420,2151,449,790
1,172,0351,196,440936,369955,8701,293,3901,320,333921,260940,4501,703,3651,738,8531,059,9751,082,0551,831,4051,869,5781,366,4251,394,8901,341,0511,368,978
1,172,0351,196,440935,955955,4481,293,3901,320,333840,542858,0501,703,3651,738,8531,064,9991,087,1841,831,4051,869,5781,366,4251,394,8901,420,2151,449,790
1,210,2951,235,513989,5551,010,1731,322,4451,349,990899,420918,1581,607,3851,640,8731,165,8401,190,1231,915,9401,955,8651,451,4801,481,7131,478,5501,509,343
1,210,2951,235,513989,5551,010,1731,322,4451,349,990820,944838,0471,607,3851,640,8731,165,8401,190,1231,915,9401,955,8651,451,4801,481,7131,484,6721,515,592
1,210,2951,235,5131,001,0821,021,9401,322,4451,349,990899,420918,1581,607,3851,640,8731,090,0671,112,7711,915,9401,955,8651,451,4801,481,7131,478,5501,509,343
1,169,9551,194,3151,012,6751,033,7731,366,2351,394,695871,970890,1381,771,9401,808,8601,191,6901,216,5131,874,1501,913,2031,469,1301,499,7251,433,4351,463,290
1,169,9551,194,3151,012,6751,033,7731,366,2351,394,695871,970890,1381,771,9401,808,8601,201,8621,226,8971,874,1501,913,2031,469,1301,499,7251,424,0881,453,749
1,169,9551,194,315978,386998,7691,366,2351,394,695882,140900,5191,771,9401,808,8601,191,6901,216,5131,874,1501,913,2031,469,1301,499,7251,433,4351,463,290
1,202,2701,227,3101,006,3201,027,2801,393,9751,423,008897,320916,0151,566,7551,618,7751,163,8601,188,1081,914,1901,954,0781,462,6801,493,1431,479,9851,510,815
1,202,2701,227,310973,732994,0131,393,9751,423,008897,320916,0151,566,7551,618,7751,175,2131,199,6971,914,1901,954,0781,462,6801,493,1431,479,9851,510,815
1,202,2701,227,3101,006,3201,027,2801,393,9751,423,008863,664881,6581,566,7551,618,7751,163,8601,188,1081,914,1901,954,0781,462,6801,493,1431,493,8351,524,954
1,210,3951,235,5981,005,3151,026,2601,379,1251,407,855843,045860,6101,668,5001,703,2651,184,0001,208,6581,869,7951,908,7601,465,6401,496,1601,392,0251,421,003
1,210,3951,235,5981,013,0901,034,1971,379,1251,407,855827,099844,3321,668,5001,703,2651,184,0001,208,6581,869,7951,908,7601,465,6401,496,1601,392,0251,421,003
1,210,3951,235,5981,005,3151,026,2601,379,1251,407,855843,045860,6101,668,5001,703,2651,120,5921,143,9291,869,7951,908,7601,465,6401,496,1601,394,6851,423,718
1,134,5201,158,1381,002,8651,023,7581,283,8201,310,563864,205882,2101,667,2851,702,0251,098,5801,121,4651,872,7251,911,7481,356,1851,384,4281,448,7551,478,918
1,134,5201,158,1381,002,8651,023,7581,283,8201,310,563828,005845,2561,667,2851,702,0251,102,1031,125,0621,872,7251,911,7481,356,1851,384,4281,448,7551,478,918
1,134,5201,158,138958,718978,6911,283,8201,310,563864,205882,2101,667,2851,702,0251,098,5801,121,4651,872,7251,911,7481,356,1851,384,4281,454,7141,485,000
1,172,0351,196,440935,955955,4481,293,3901,320,333921,260940,4501,703,3651,738,8531,059,9751,082,0551,831,4051,869,5781,366,4251,394,8901,420,2151,449,790
1,172,0351,196,440935,955955,4481,293,3901,320,333910,799929,7711,703,3651,738,8531,059,9751,082,0551,831,4051,869,5781,366,4251,394,8901,424,7021,454,371
1,172,0351,196,440939,929959,5041,293,3901,320,333921,260940,4501,703,3651,738,8531,027,4711,048,8741,831,4051,869,5781,366,4251,394,8901,420,2151,449,790
1,142,1351,165,920930,485949,8651,3314351,359,165876,760895,0231,619,5151,671,1751,147,4401,171,3381,905,3551,945,0681,465,2201,495,7331,451,0001,481,223
1,142,1351,165,920931,023950,4151,3314351,359,165800,414817,0861,619,5151,671,1751,147,4401,171,3381,905,3551,945,0681,465,2201,495,7331,451,0001,481,223
1,142,1351,165,920930,485949,8651,3314351,359,165876,760895,0231,619,5151,671,1751,111,0371,134,1761,905,3551,945,0681,465,2201,495,7331,453,0841,483,350
1,157,6601,181,773981,8601,002,3131,3635051,391,908890,910909,4681,535,5751,567,5751,182,0401,206,6581,775,9351,812,9401,396,4651,425,5501,434,7751,464,658
1,157,6601,181,773982,9251,003,4001,3635051,391,908809,103825,9561,535,5751,567,5751,182,0401,206,6581,775,9351,812,9401,396,4651,425,5501,434,7751,464,658
1,157,6601,181,773981,8601,002,3131,3635051,391,908890,910909,4681,535,5751,567,5751,107,5751,130,6421,775,9351,812,9401,396,4651,425,5501,437,5771,467,518
1,164,7051,188,958962,750982,7981,2756701,302,250860,665878,6001,605,3551,638,8101,141,4551,165,2301,822,5601,860,5301,452,5801,482,8351,411,8101,441,215
1,164,7051,188,958876,601894,8541,275,6701,302,250860,665878,6001,605,3551,638,8101,141,4551,165,2301,822,5601,860,5301,452,5801,482,8351,419,1151,448,672
1,164,7051,188,958962,750982,7981,275,6701,302,250826,801844,0301,605,3551,638,8101,142,6901,166,4911,822,5601,860,5301,452,5801,482,8351,411,8101,441,215
Table A8. Values of σPi(ZGki,Bi) for the solutions in Table 7.
Table A8. Values of σPi(ZGki,Bi) for the solutions in Table 7.
Sol.DM1DM2DM3DM4DM5DM6DM7DM8DM9DM10
10.75830.74670.70870.73910.72070.76740.74540.78050.77580.7963
20.75660.74530.70690.73740.71910.76580.74370.77890.77420.7947
30.75210.74170.70260.73300.71550.76150.73980.77460.77010.7905
40.75160.74130.70190.73240.71500.76130.73930.77430.76960.7902
50.75140.74040.70150.73200.71390.76100.73830.77400.76890.7895

Appendix C. Description of the Variant of MOEA/D Used in Solving Problem 1

Zhang and Li in [59] proposed the decomposition algorithm MOEA/D to solve multi-objective optimization problems. The strategy decomposes a multi-objective problem into several scalar functions, which are subject to simultaneous optimization. Algorithm A2 depicts the pseudocode of a variant of MOEA/D, adapted to solving Problem 1 in the frame of project portfolio optimization. The methods—initialization and repair and improvement operator—make use of interval arithmetic to handle the ill-determination and imprecision that is present in parameters’ values of preference models and constraints. The general idea behind the approach followed in the experiment used in this manuscript is better detailed in the remaining paragraphs of Appendix C.
Algorithm A2. Variant of MOEA/D
Input:
N: number of scalar functions,
M: number of objectives,
Vector: uniformly distributed set of vectors
T = N/10: size of neighborhood of weight vectors.
Output:
EP: approximatio of Pareto frontier.
01: (x,z,FV,B(i)) ← Initializacion()
02: For i = 1 to N do
03:  (xk,xl) ← RandomSelection(B(i),T)
04:  yOnePointCrossover(xk,xl)
05:  y′ ← FlipMutation(y)
06:  y″RepairAndImprovementOperator(y′)
07:  UpdateSetZ(z,M,y″)//z: for each j = 1, …, m, if zj < fj(y′) then set zj = fj(y′).
08:  UpdateNeighborhood(B(i),FV,y″)// for each j B(i), if gte (y’|Vj,z) set xj = y’ and FVj = F(y’)
09:  UpdateEP(EP,y″)//Remove from EP all the vectors dominated by F(y′), and add F(y′) to EP if no vectors in EP dominate F(y′)
10: Stopping Criteria: if maxEvaluations is reach, Otherwise, go to step 2.
The algorithm has as parameter inputs N, M, Vector = {V1, V2, …, VN}, and T. These parameters represent, respectively, the number of scalar functions or sub-problems in which the MOP has been divided, the number of objectives, a uniformly distributed set of size N containing weight vectors with two elements each (the weights were V i = ( i N , N i N ) , for each ViVector), and the size of the neighborhood of weight vectors. In addition, MOEA/D gives as output an approximation of the Pareto frontier (EP) formed by non-dominated solutions found during the optimization process. For this purpose, the algorithm works in two phases.
The first phase is the initialization phase (indicated in Line 0 of the Algorithm A2). Here the algorithm initializes properly the data structures required for the construction of the final set of solutions. These structures are: (1) the set EP, which will become the Pareto frontier approximation, and which initially is empty; (2) the neighborhood sets B(i) of each vector Vi that contain the T closes weight vectors to Vi by Euclidean distance; (3) the initial set of solutions X = {x1, x2, …, xN} where each solution xj, 1 ≤ jNumber of group members, corresponds to the j-th DM best compromise in the decision variable space, and the remaining ones are randomly generated; (4) the set of fitness values FV = {FV1, FV2,…, FVN}, where each FVi is composed by the M objective values of each solution xi; and 5) the set z = {z1, …., zm} formed by values zj corresponding to the best objective value among all the solutions built during the initialization process.
The second phase is an update process based on evolution. In this process, for each solution in the population, two solutions of the population are randomly selected and used to generate a new solution by applying genetic operators. The operators used are the following ones:
One-point crossover: the two randomly chosen parents combine its chromosomes by designating a point in the bit string that represents them. The operator swaps between the two parents the bits in their encoding to the right of that point. The results are a new offspring of two children.
Flip mutation: This mutation process generates a random value for each allele in the bit string encoding the solution; every time that the generated value lies under a given probability (e.g., the probability of 0.02 used in this work), the bit value in the encoding is inverted.
After the application of the genetic operators, the created solution is subject to a repair and improvement phase, which is performed by the repair and improvement operator (RIO). The RIO combines the processes that might repair and improve a portfolio; in detail, it turns an infeasible solution into a quasi-feasible solution that is feasible with respect to the used budget but may not be feasible with respect to in the satisfaction degree of the DMs, i.e., it might not warranty that NsatNdis. The RIO sorts the projects in the portfolio by the fitness ratio FR, and from the lowest FR to the highest, it eliminates projects one by one until the budget becomes feasible. After that, using a threshold parameter (set to 0.50) it also eliminates a proportion of the remaining projects. Then, from the highest FR to the lowest, the improvement algorithm adds projects into the portfolio while keeping the budget feasible. The fitness ratio FR is a generalized measure of the fitness of a solution based on its objectives; first, for the i-th project, the procedure computes the ratio fcij = fitness/cost for each objective j; then, it computes fci which is the median among the m values fcij previously computed. The value fcij becomes the value of FR.
The final step is to check the stopping condition; if it is satisfied, the algorithm finishes and reports the EP as its output; otherwise, the algorithm returns to the second step. The stop criterion is fulfilled when a maximum number of evaluations of the objective function vector (100,000) is reached. During each iteration, the algorithm conveniently updates B, EP, z, and in the end, it returns the non-dominated set found in EP.
Lines 1 to 8 in Algorithm A2 describe the main loop of MOEA/D, and Table A9 details its time complexity based on a worst-case scenario. The analysis considers the variables S, N, P, and ng as the size of the population, the number of criteria on the project portfolio optimization, the number of candidate projects, and the number of DMs, respectively. The construction of the set EP was left outside the main loop.
Table A9. MOEA/D time complexity analysis.
Table A9. MOEA/D time complexity analysis.
Complexity
0. (x,z,FV,B(i)) ← Initialization()
1. For i = 1 to S doO(S)
2. (xk, xl) ← RandomSelection(B(i),T)O(1)
3. y ← OnePointCrossover(xk, xl)O(P)
4. y′ ← FlipMutation(y)O(1)
5. y″ ← RepairAndImprovementOperator (y’) O(P)
6. UpdateSetZ(z,M,y″) O(N2ng)
7. UpdateNeighborhood(B(i),FV,y″) O(S)
8. UpdateEP(EP,y″)
9. Stopping criteria: If maxEvaluations is reach, otherwise, go to Step 1.
Note that, based on the analysis shown in Table A9, the complexity of MOEA/D linearly scales w.r.t. to the number of candidate projects P and the number of DMs ng in the decision group. The linear growth in the number of members of a decision group favors this approach in scenarios with large groups of decision-makers.

References

  1. Kabak, O.; Ervural, B. Multiple attribute group decision making: A generic conceptual framework and a classification scheme. Knowl. Based Syst. 2017, 123, 13–30. [Google Scholar] [CrossRef]
  2. Xu, X.; Du, Z.; Chen, X.; Cai, C. Confidence consensus-based model for large-scale group decision making: A novel approach to managing non-cooperative behaviors. Inf. Sci. 2019, 288, 271–293. [Google Scholar] [CrossRef]
  3. Fu, C.; Chang, W.; Xue, M.; Yang, S. Multiple criteria group decision making with belief distributions and distributed preference relations. Eur. J. Oper. Res. 2019, 273, 623–633. [Google Scholar] [CrossRef]
  4. Dong, Q.; Zhou, X.; Martínez, L. A hybrid group decision-making framework for achieving agreed solutions based on stable opinions. Inf. Sci. 2019, 490, 227–243. [Google Scholar] [CrossRef]
  5. Herrera-Viedma, E.; Palomares, I.; Cabrerizo, F.J.; Dong, Y.; Chiclana, F.; Herrera, F. Revisiting Fuzzy and Linguistic Decision Making: Scenarios and Challenges for Making Wiser Decisions in a Better Way. IEEE Trans. Syst. Man Cybern. Syst. 2021, 51, 191–208. [Google Scholar] [CrossRef]
  6. Lu, Y.; Xu, Y.; Herrera-Viedma, E.; Han, Y. Consensus of large-scale group decision making in social network: The minimum cost model based on robust optimization. Inf. Sci. 2021, 547, 910–930. [Google Scholar] [CrossRef]
  7. Barykin, S.Y.; Kapustina, I.V.; Sergeev, S.M.; Kalinina, O.V.; Vilken, V.V.; Putikhin, Y.Y.; Volkova, L.V. Developing the physical distribution digital twin model within the trade network. Acad. Strateg. Manag. J. 2021, 20, 1–25. Available online: https://www.abacademies.org/articles/developing-the-physical-distribution-digital-twin-model-within-the-trade-network.pdf (accessed on 13 May 2021).
  8. Jecmen, S.; Zhang, H.; Liu, R.; Shah, N.B.; Conitzer, V.; Fang, F. Mitigating manipulation in peer review via randomized reviewer assignments. arXiv 2020, arXiv:2006.16437. [Google Scholar]
  9. Cabrerizo, F.J.; Moreno, J.M.; Pérez, I.J.; Herrera-Viedma, E. Analyzing consensus approaches in fuzzy group decisión making: Advantages and drawbacks. Soft Comput. 2010, 14, 451–463. [Google Scholar] [CrossRef] [Green Version]
  10. Gao, Y.; Li, D. A consensus model for heterogeneous multi-attribute group decision making with several attribute sets. Expert Syst. Appl. 2019. [Google Scholar] [CrossRef]
  11. Labella, A.; Liu, Y.; Rodríguez, R.M.; Martínez, L. Analyzing the performance of classical consensus models in large scale group decision making: A comparative study. Appl. Soft Comput. 2018, 67, 677–690. [Google Scholar] [CrossRef]
  12. Cheng, L.; Chen, Y.; Chiang, Y. Identifying conflict patterns to reach a consensus. A novel group decision approach. Eur. J. Oper. Res. 2016, 254, 622–631. [Google Scholar] [CrossRef]
  13. Herrera, F.; Herrera-Viedma, E.; Verdegay, J.L. A model of consensus in group decision making under linguistic assessments. Fuzzy Sets Syst. 1996, 78, 73–87. [Google Scholar] [CrossRef]
  14. Dong, Y.; Xu, Y.; Li, H.; Feng, B. The OWA-based consensus operator under linguistic representation models using position indexes. Eur. J. Oper. Res. 2010, 203, 455–463. [Google Scholar] [CrossRef]
  15. Hwang, C.L.; Lin, M.J. Group Decision Making under Multiple Criteria. In Lecture Notes in Economics and Mathematical Systems; Springer Science and Business Media LLC: Berlin/Heidelberg, Germany, 1987; Volume 281, 411p. [Google Scholar] [CrossRef]
  16. Jessup, L.; Valacich, J. Introduction to the Special Issue on Group Support Systems. Small Group Res. 1993, 24, 427–429. [Google Scholar] [CrossRef]
  17. Leyva, J.C.; Fernandez, E. A new method for group decision support based on ELECTRE-III methodology. Eur. J. Oper. Res. 2003, 148, 14–27. [Google Scholar] [CrossRef]
  18. Xiong, J.; Tan, X.; Yang, K.; Chen, Y. Fuzzy group decision making for multiobjective problems: Tradeoffs between consensus and robustness. J. Appl. Math. 2013, 2013, 1–9. [Google Scholar] [CrossRef]
  19. Bouyssou, D.; Marchant, T.; Perny, P.; Tsoukias, A.; Vincke, P. Evaluations and Decision Models: A Critical Perspective; Kluwer Academic Publishers: Dordrecht, The Netherlands, 2000; Volume 1. [Google Scholar] [CrossRef]
  20. Fernández, E.; Figueira, J.; Navarro, J. An interval extension of the outranking approach and its application to multiple-criteria ordinal classification. Omega 2019, 84, 189–198. [Google Scholar] [CrossRef]
  21. Fernández, E.; Figueira, J.; Navarro, J. Interval-based extensions of two outranking methods for multi-criteria ordinal classification. Omega 2020, 95, 102–125. [Google Scholar] [CrossRef]
  22. Li, J.; Hu, Y. Some rationality conditions of joint efficient mapping in group multi-objective programming. J. Syst. Sci. Complex. 2007. [Google Scholar] [CrossRef]
  23. Hu, Y.; Hong, Z.; Zhou, X. Utopian preference mapping and the utopian preference method for group multi-objective optimization. Prog. Nat. Sci. 2003, 13. [Google Scholar] [CrossRef]
  24. Lewis, H.S.; Butler, T.W. An interactive framework for multi-person, multi-objective decision. Decis. Sci. 1993, 24, 1–22. [Google Scholar] [CrossRef]
  25. Isermann, H. Interactive group decision-making by coalitions. In Plural Rationality and Interactive Decision Processes; Grauer, M., Thompson, M., Wierzbicki, A.P., Eds.; Springer: Berlin/Heidelberg, Germany, 1985; Volume 248. [Google Scholar] [CrossRef]
  26. Wendell, R.E. Multiple objective mathematical programming concerning multiple decision-makers. Oper. Res. 1980, 28, 1029–1257. [Google Scholar] [CrossRef]
  27. Tapia, C.G.; Murtagh, B.A. Interactive group decision-making using fuzzy programming with preference criteria. Fuzzy Sets Syst. 1992, 45, 13–23. [Google Scholar] [CrossRef]
  28. Efremov, R.; Insua, D.R.; Lotov, A. A framework for participatory decision support using Pareto frontier visualization, goal identification, and arbitration. Eur. J. Oper. Res. 2009, 199, 459–467. [Google Scholar] [CrossRef]
  29. Bechikh, S.; Said, L.B.; Ghédira, K. Negotiating decision-makers’ reference points for group preference-based Evolutionary Multi-objective Optimization. In Proceedings of the IEEE International Conference on Hybrid Intelligent Systems, Melacca, Malaysia, 5–8 December 2011. [Google Scholar] [CrossRef]
  30. Bechikh, S.; Said, L.B.; Ghédira, K. Group preference-based evolutionary multi-objective optimization with non-equally important decision-makers: Application to the Portfolio Selection Problem. Int. J. Comput. Inf. Syst. Ind. Manag. Appl. 2013, 5, 278–288. Available online: http://www.mirlabs.org/ijcisim/regular_papers_2013/Paper104.pdf (accessed on 13 May 2021).
  31. Cinalli, D.; Martí, L.; Sanchez-Pi, N.; Bicharra Garcia, A.C. Integrating collective intelligence into evolutionary multi-objective algorithms: Interactive preferences. In Proceedings of the IEEE 2015 Latin American Congress on Computational Intelligence, Curitiba, Brazil, 13–16 October 2015; pp. 1–6. [Google Scholar] [CrossRef]
  32. Cinalli, D.; Martí, L.; Sanchez-Pi, N.; Bicharra Garcia, A.C. Collective preferences in Evolutio- nary Multi-objective Optimization: Techniques and potential contributions of collective intelligence. In Proceedings of the 30th ACM/SIGAPP Symposium on Applied Computing, Salamanca, Spain, 13–17 April 2015; pp. 133–138. [Google Scholar] [CrossRef]
  33. Fernandez, E.; Olmedo, R. An outranking-based general approach to solving group multi-objective optimization problems. Eur. J. Oper. Res. 2013, 225, 497–506. [Google Scholar] [CrossRef]
  34. Kadzinski, M.; Tomczyk, M.K. Interactive Evolutionary Multiple Objective Optimization for Group Decision Incorporating Value-based Preference Disaggregation Methods. Group Decis. Negot. 2017, 26, 693–728. [Google Scholar] [CrossRef] [Green Version]
  35. Borissova, D.; Mustakerov, I. A two-stage placement algorithm with multi-objective optimization and group decision making. Cybern. Inf. Technol. 2017, 17, 87–103. [Google Scholar] [CrossRef] [Green Version]
  36. Dietz, T.; Klamroth, K.; Kraus, K.; Ruzika, S.; Schafer, L.; Schulze, B.; Stiglmayr, M.; Wiecek, M. Introducing multiobjective complex systems. Eur. J. Oper. Res. 2020, 280, 581–596. [Google Scholar] [CrossRef] [Green Version]
  37. Guarneri, P.; Wiecek, M. Pareto-based negotiation in distributed multidisciplinary design. Struct. Multidiscip. Optim. 2016, 53, 657–671. [Google Scholar] [CrossRef]
  38. Roy, B.; Figueira, J.R.; Almeida-Dias, J. Discriminating thresholds as a tool to cope with imperfect knowledge in multiple criteria decision aiding: Theoretical results and practical issues. Omega 2014, 43, 9–20. [Google Scholar] [CrossRef] [Green Version]
  39. Liesio, J.; Mild, P.; Salo, A. Preference programming for robust portfolio modeling and project selection. Eur. J. Oper. Res. 2007, 181, 1488–1505. [Google Scholar] [CrossRef]
  40. Liesio, J.; Mild, P.; Salo, A. Robust portfolio modeling with incomplete cost information and project interdependencies. Eur. J. Oper. Res. 2008, 190, 679–695. [Google Scholar] [CrossRef]
  41. Fliedner, T.; Liesio, J. Adjustable robustness for multi-attribute project portfolio selection. Eur. J. Oper. Res. 2016, 252, 931–946. [Google Scholar] [CrossRef]
  42. Toppila, A.; Salo, A. Binary decision diagrams for generating and storing non-dominated project portfolios with interval-valued project scores. Eur. J. Oper. Res. 2017, 260, 244–254. [Google Scholar] [CrossRef]
  43. Balderas, F.; Fernandez, E.; Gomez, C.; Rangel, N.; Cruz-Reyes, L. An interval-based approach for evolutionary multi-objective optimization of project portfolios. Int. J. Inf. Technol. Decis. Mak. 2019, 18, 1317–1358. [Google Scholar] [CrossRef]
  44. Eklund, P.; Rusinowska, A.; De Swart, H. Consensus reaching in committees. Eur. J. Oper. Res. 2007, 178, 185–193. [Google Scholar] [CrossRef] [Green Version]
  45. Herrera-Viedma, E.; Alonso, S.; Chiclana, F.; Herrera, F. A consensus model for group decision making with incomplete fuzzy preference relations. IEEE Trans. Fuzzy Syst. 2007, 15, 863–877. [Google Scholar] [CrossRef]
  46. Fu, C.; Yang, S. The group consensus-based evidential reasoning approach for multiple attributive group decision analysis. Eur. J. Oper. Res. 2010, 206, 601–608. [Google Scholar] [CrossRef]
  47. Xu, Z.; Cai, X. Group consensus algorithms based on preference relations. Inf. Sci. 2011, 181, 150–162. [Google Scholar] [CrossRef]
  48. Fernandez, E.; Bernal, S.; Navarro, J.; Olmedo, R. An outranking-based fuzzy logic model for collaborative group preferences. TOP 2010, 18, 444–464. [Google Scholar] [CrossRef]
  49. Moore, R.E. Methods and applications of interval analysis, Studies in Applied and Numerical Mathematics. Soc. Ind. Appl. Math. 1979, 23, 121–123. [Google Scholar] [CrossRef] [Green Version]
  50. Yao, S.; Jiang, Z.; Li, N.; Zhang, H.; Geng, N. A multi-objective dynamic scheduling approach using multiple attribute decision making in semiconductor manufacturing. Int. J. Prod. Econ. 2011, 130, 125–133. [Google Scholar] [CrossRef]
  51. Keeney, R.; Raiffa, H. Decision with Multiple Objectives: Preferences and Value Tradeoffs; Wiley: New York, NY, USA, 1976; Volume 7. [Google Scholar] [CrossRef]
  52. Saaty, T.L. The Analytic Hierarchy Process; McGraw-Hill: New York, NY, USA, 1980. [Google Scholar]
  53. Roy, B. The outranking approach and the foundations of ELECTRE methods. Theory Decis. 1991. [Google Scholar] [CrossRef]
  54. Greco, S.; Matarazzo, B.; Słowiński, R. Rough sets theory for multicriteria decision analysis. Eur. J. Oper. Res. 2001, 129, 1–47. [Google Scholar] [CrossRef]
  55. Ishizaka, A.; Pearman, C.; Nemery, P. AHPSort: An AHP-based method for sorting problems. Int. J. Prod. Res. 2012, 50, 4767–4784. [Google Scholar] [CrossRef]
  56. Roy, B.; Bouyssou, D. Aide Multicritère à la Décision: Méthodes et Cas; Economica: Paris, France, 1993; Available online: https://basepub.dauphine.fr/handle/123456789/4522 (accessed on 13 May 2021).
  57. Fernández, E.; Figueira, J.; Navarro, J.; Roy, B. ELECTRE TRI-nB: A new multiple criteria ordinal classification method. Eur. J. Oper. Res. 2017, 263, 214–224. [Google Scholar] [CrossRef]
  58. Torres-Jimenez, J.; Izquierdo-Marquez, I. Survey of covering arrays. In Proceedings of the 15th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, Timisoara, Romania, 23–26 September 2013; pp. 20–27. [Google Scholar] [CrossRef]
  59. Zhang, Q.; Li, H. MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition. IEEE Trans. Evol. Comput. 2007, 11, 712–731. [Google Scholar] [CrossRef]
Table 1. Summary of assumptions.
Table 1. Summary of assumptions.
It’s SubjectAllowsRelated to
Assumption 1Interval numbers as model of imprecisionsModeling imprecisionSteps 1–2
Assumption 2Capacity to identify the best compromiseIdentifying best compromisesStep 4
Assumption 3Compatibility with an outranking modelPreference modelingStep 1
Assumption 4Capacity to set the limiting boundaries related to the outranking modelIdentifying limiting boundariesStep 5
Assumption 5Compatibility with INTERCLASS-nBAssigning solutions to classes of satisfaction/dissatisfactionStep 6
Assumption 6Compatibility with an interval value functionPreference modelingStep 1
Assumption 7Capacity to set the limiting boundaries related to the value function modelIdentifying limiting boundariesStep 5
Table 2. Best compromise solution for each DM (the i-th row corresponds to the i-th DM).
Table 2. Best compromise solution for each DM (the i-th row corresponds to the i-th DM).
O1O2O3O4O5O6O7O8O9
1,234,9251,337,795995,0151,077,9351,417,0651,535,135929,9301,007,4401,507,7251,633,4051,211,3301,312,2101,925,3452,085,7951,365,3151,479,0451,658,7951,796,975
1,259,9051,364,855975,3651,056,6251,474,7451,597,625959,4701,039,4301,480,4501,603,8501,279,3301,385,8801,904,4552,063,1651,486,0951,609,8951,633,7001,769,780
1,239,5151,342,7651,024,0901,109,4301,449,2251,569,975924,8151,001,8751,643,5301,780,5301,255,1901,359,7402,009,6852,177,2051,358,4501,471,6501,661,0351,799,395
1,189,9951,289,1151,060,2151,148,5451,425,7551,544,545938,1051,016,2751,600,5801,734,0101,312,0951,421,3751,984,5602,149,9701,370,1651,484,3251,610,0351,744,125
1,215,9451,317,2351,010,4001,094,5901,429,1651,548,235927,4651,004,7551,634,0451,770,2351,234,4301,337,2601,992,9752,159,1151,383,4451,498,7251,647,0051,784,205
1,169,8001,267,2401,015,3251,099,9151,442,6851,562,875962,5701,042,8001,586,8451,719,1051,211,3101,312,2002,012,2502,179,9801,445,5401,565,9801,576,1351,707,435
1,211,0851,311,9751,026,4251,111,9551,394,2201,510,390910,105985,9451,588,3351,720,7251,216,1351,317,4151,948,1202,110,5001,358,3751,471,5451,663,0151,801,545
1,245,8051,349,585969,1101,049,8501,471,2451,593,835946,7201,025,6101,538,7401,667,0201,231,4551,334,0151,820,5501,972,3101,392,1651,586,4751,596,3601,729,340
1,139,0451,233,9251,034,5651,120,7651,377,3851,492,165925,2801,002,3801,507,9901,633,6601,215,7301,317,0001,903,3602,061,9801,429,4901,548,5901,584,0051,715,955
1,262,1001,367,2301,030,8701,116,7701,385,1351,500,545931,3501,008,9501,629,6651,765,5051,201,8451,301,9551,932,6152,093,6851,392,6551,508,6951,628,2451,763,875
Table 3. Best consensus solution to Problem 1 with Nsat = 9 and Ndis = 0.
Table 3. Best consensus solution to Problem 1 with Nsat = 9 and Ndis = 0.
O1O2O3O4O5O6O7O8O9
1,199,2301,299,1201,038,9451,125,5251,431,0251,550,265909,215984,9951,546,7551,675,6851,243,9151,347,5151,939,7002,101,3601,355,8051,468,7351,647,9451,785,225
1,210,9651,311,8551,048,2751,135,6351,411,3351,528,925908,875984,6251,524,9651,652,0651,243,4451,346,9951,955,1702,118,1201,346,4251,458,5851,633,1001,769,130
Table 4. Best ranked solutions to Problem 1 with Nsat = 10 and Ndis = 0 (after the consensus round).
Table 4. Best ranked solutions to Problem 1 with Nsat = 10 and Ndis = 0 (after the consensus round).
O1O2O3O4O5O6O7O8O9μsat
1,283,1851,390,0751,045,9601,133,1101,468,4451,590,815924,6701,001,7301,568,1551,698,8751,253,5551,357,9651,922,7752,083,0251,429,4651,548,5651,646,7451,783,9150.8345
1,219,0401,320,5801,041,6551,128,4551,471,8451,594,475922,825999,7351,587,9251,720,2751,291,3151,398,8852,028,6402,197,7301,431,8001,551,0901,640,2551,776,8950.8068
1,241,2551,344,665997,2551,080,3451,491,2851,615,545922,435999,3051,513,2601,639,3901,301,5451,409,9551,949,1352,111,5751,426,0351,544,8251,655,1101,792,9800.7984
1,222,7801,324,6301,053,9701,141,7901,463,2151,585,125925,1051,002,1951,644,7301,781,8501,262,3201,367,4601,976,3352,141,0651,357,7651,470,8951,642,2851,779,0750.7910
1,257,7851,362,5651,043,4801,130,4301,462,8651,584,755913,840989,9901,639,7151,776,3951,245,9101,349,6802,005,8052,172,9851,392,8801,508,9401,641,6151,778,3650.7880
Table 5. Best compromise solution for each DM (the i-th row corresponds to the i-th DM).
Table 5. Best compromise solution for each DM (the i-th row corresponds to the i-th DM).
O1O2O3O4O5O6O7O8O9
1,172,0351,269,655935,9551,013,9251,293,3901,401,160921,260998,0201,703,3651,845,3151,059,9751,148,2951,831,4051,984,0951,366,4251,480,2851,420,2151,538,515
1,210,2951,311,165989,5551,072,0251,322,4451,432,625899,420974,3701,607,3851,741,3351,165,8401,262,9701,915,9402,075,6401,451,4801,572,4101,478,5501,601,720
1,169,9551,267,3951,012,6751,097,0651,366,2351,480,075871,970944,6401,771,9401,919,6201,191,6901,290,9801,874,1502,030,3601,469,1301,591,5101,433,4351,552,855
1,202,2701,302,4301,006,3201,090,1601,393,9751,510,105897,320972,1001,566,7551,774,8351,163,8601,260,8501,914,1902,073,7401,462,6801,584,5301,479,9851,603,305
1,210,3951,311,2051,005,3151,089,0951,379,1251,494,045843,045913,3051,668,5001,807,5601,184,0001,282,6301,869,7952,025,6551,465,6401,587,7201,392,0251,507,935
1,134,5201,228,9901,002,8651,086,4351,283,8201,390,790864,205936,2251,667,2851,806,2451,098,5801,190,1201,872,7252,028,8151,356,1851,469,1551,448,7551,569,405
1,172,0351,269,655935,9551,013,9251,293,3901,401,160921,260998,0201,703,3651,845,3151,059,9751,148,2951,831,4051,984,0951,366,4251,480,2851,420,2151,538,515
1,142,1351,237,275930,4851,008,0051,331,4351,442,355876,760949,8101,619,5151,826,1551,147,4401,243,0301,905,3552,064,2051,465,2201,587,2701,451,0001,571,890
1,157,6601,254,110981,8601,063,6701,363,5051,477,115890,910965,1401,535,5751,663,5751,182,0401,280,5101,775,9351,923,9551,396,4651,512,8051,434,7751,554,305
1,164,7051,261,715962,7501,042,9401,275,6701,381,990860,665932,4051,605,3551,739,1751,141,4551,236,5551,822,5601,974,4401,452,5801,573,6001,411,8101,529,430
Table 6. Best consensus solution to Problem 1 with Nsat = 6 and Ndis = 0.
Table 6. Best consensus solution to Problem 1 with Nsat = 6 and Ndis = 0.
O1O2O3O4O5O6O7O8O9
1220175132178510149201099490137691014916408931309675501660130179852011728151270545189753020556901450070164919014703001592790
Table 7. Best ranked solutions to Problem 1 with Nsat = 10 and Ndis = 0 (after the consensus round).
Table 7. Best ranked solutions to Problem 1 with Nsat = 10 and Ndis = 0 (after the consensus round).
O1O2O3O4O5O6O7O8O9μsat
1,223,2601,325,1801,026,3601,111,9001,455,1451,576,405905,975981,4651,590,6451,723,1951,201,8151,301,9151,980,2402,145,2801,458,8101,580,3301,603,1851,736,7450.7087
1,205,0001,305,4001,023,9201,109,2501,452,1951,573,215918,285994,8051,574,6651,705,8751,218,8951,320,4151,963,3102,126,9301,451,4501,572,3601,605,3651,739,1050.7069
1,218,8351,320,3851,000,7501,084,1301,486,7501,610,620902,530977,7601,577,8351,709,3351,223,2201,325,1201,886,7602,044,0201,436,1601,555,8001,630,0501,765,8600.7026
1,209,1151,309,8551,006,3201,090,1701,500,9551,626,005906,960982,5601,558,5851,688,4751,253,7401,358,1801,901,7302,060,2301,428,9701,548,0101,597,8951,731,0250.7019
1,218,5901,320,1101,020,7801,105,8401,434,7151,554,285929,8651,007,3251,618,7851,753,6951,226,1651,328,2951,895,6802,053,6701,468,3301,590,6601,582,6051,714,4550.7015
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Fernández, E.; Rangel-Valdez, N.; Cruz-Reyes, L.; Gomez-Santillan, C. A New Approach to Group Multi-Objective Optimization under Imperfect Information and Its Application to Project Portfolio Optimization. Appl. Sci. 2021, 11, 4575. https://doi.org/10.3390/app11104575

AMA Style

Fernández E, Rangel-Valdez N, Cruz-Reyes L, Gomez-Santillan C. A New Approach to Group Multi-Objective Optimization under Imperfect Information and Its Application to Project Portfolio Optimization. Applied Sciences. 2021; 11(10):4575. https://doi.org/10.3390/app11104575

Chicago/Turabian Style

Fernández, Eduardo, Nelson Rangel-Valdez, Laura Cruz-Reyes, and Claudia Gomez-Santillan. 2021. "A New Approach to Group Multi-Objective Optimization under Imperfect Information and Its Application to Project Portfolio Optimization" Applied Sciences 11, no. 10: 4575. https://doi.org/10.3390/app11104575

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop