Next Article in Journal
Comprehensive Survey of Multimedia Steganalysis: Techniques, Evaluations, and Trends in Future Research
Next Article in Special Issue
A Modification of the Imperialist Competitive Algorithm with Hybrid Methods for Multi-Objective Optimization Problems
Previous Article in Journal
Anticipated Generalized Backward Doubly Stochastic Differential Equations
Previous Article in Special Issue
A Novel Multi-Population Artificial Bee Colony Algorithm for Energy-Efficient Hybrid Flow Shop Scheduling Problem
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An Ensemble Framework of Evolutionary Algorithm for Constrained Multi-Objective Optimization

1
School of Science, Qiongtai Normal University, Haikou 571127, China
2
School of Computer Science, China University of Geosciences, Wuhan 430074, China
*
Author to whom correspondence should be addressed.
Symmetry 2022, 14(1), 116; https://doi.org/10.3390/sym14010116
Submission received: 7 December 2021 / Revised: 27 December 2021 / Accepted: 6 January 2022 / Published: 9 January 2022
(This article belongs to the Special Issue Meta-Heuristics for Manufacturing Systems Optimization)

Abstract

:
In the real-world, symmetry or asymmetry widely exists in various problems. Some of them can be formulated as constrained multi-objective optimization problems (CMOPs). During the past few years, handling CMOPs by evolutionary algorithms has become more popular. Lots of constrained multi-objective optimization evolutionary algorithms (CMOEAs) have been proposed. Whereas different CMOEAs may be more suitable for different CMOPs, it is difficult to choose the best one for a CMOP at hand. In this paper, we propose an ensemble framework of CMOEAs that aims to achieve better versatility on handling diverse CMOPs. In the proposed framework, the hypervolume indicator is used to evaluate the performance of CMOEAs, and a decreasing mechanism is devised to delete the poorly performed CMOEAs and to gradually determine the most suitable CMOEA. A new CMOEA, namely ECMOEA, is developed based on the framework and three state-of-the-art CMOEAs. Experimental results on five benchmarks with totally 52 instances demonstrate the effectiveness of our approach. In addition, the superiority of ECMOEA is verified through comparisons to seven state-of-the-art CMOEAs. Moreover, the effectiveness of ECMOEA on the real-world problems is also evaluated for eight instances.

1. Introduction

Constrained multi-objective optimization problems (CMOPs) widely exist in real-world applications and scientific research fields, such as the multi-objective caching optimization [1], vehicle routing problems [2], web service location allocation problems [3], 3D indoor redeployment in IoT collection networks [4], asymmetric UAV trajectory planning [5], multi-objective testing resource allocation problems [6], and ring road bus lines and fare design [7]. Generally, CMOPs can be referred to as the optimization problems with constraints and two or more conflicting objectives [8], which can be mathematically formulated as:
Minimize F ( x ) = ( f 1 ( x ) , , f m ( x ) ) T subject to x S g i ( x ) 0 , i = 1 , , p h j ( x ) = 0 , j = p + 1 , , q ,
where m denotes the number of objective functions, x = ( x 1 , , x n ) T represents an n-dimensional decision vector, n denotes the number of decision variables, x S and S R n are the search space. g i ( x ) and h j ( x ) are the i-th inequality and j-th equality constraints, respectively. q is the number of constraints.
The constraint violation ( c v j ( x ) ) of x at the j-th constraint can be formulated as:
c v j ( x ) = max ( 0 , g j ( x ) ) , j = 1 , , p max ( 0 , | h j ( x ) | σ ) , j = p + 1 , , q ,
where σ is a very small positive value to relax the equality constraints into inequality ones. The overall constraint violation (CV) of x is formulated as:
C V ( x ) = j = 1 q c v j ( x ) ,
x is, feasible if C V ( x ) = 0 ; otherwise, it is infeasible.
Due to the population-based feature, which accords to the requirement of a CMOP for a set of tradeoff solutions, and independence of prior knowledge or problem structure, evolutionary algorithms (EAs) are widely used in solving multi-objective optimization problems (MOPs) [9], many-objective optimization problems (MaOPs) [10] and CMOPs [11], for which we termed multi-objective optimization evolutionary algorithms (MOEAs), many-objective optimization evolutionary algorithms (MaOEAs) and constrained multi-objective optimization evolutionary algorithms (CMOEAs). They mainly include the dominance-based [12], decomposition-based [13] and indicator-based [14] ones. During the past two decades, lots of CMOEAs have been proposed [15,16,17], and the constrained multi-objective optimization (CMO) community has made great progress. However, despite the prosperity of CMO research field, these CMOEAs all have their own strengths and weaknesses [18]. For example, CCMO [17], which performs well on C-DTLZ [19] and DC-DTLZ [20] benchmarks, shows poor performance on LIR-CMOP [21] with large infeasible regions. On the contrary, PPS [22], which performs well on LIR-CMOP, fails to well handle C-DTLZ and DC-DTLZ.
According to the No Free Lunch theory, a CMOEA can not be effective for all kinds of CMOPs. That is to say, in face of the proposed CMOP benchmarks, a specific CMOEA can only handle one kind or several kinds of CMOPs. Designing a CMOEA which can adapt to all CMOPs is impossible. Given the fact that a CMOP in real-world situation may have different features and poses different challenges to CMOEA, enhancing the versatility of CMOEA is expected.
To overcome this drawback, in this paper, we propose an ensemble framework of CMOEAs, which aims to generate an adaptively selecting guided CMOEA filter to enhance the versatility of existing CMOEAs. The main contributions can be summarized as follows:
  • An ensemble framework is proposed, into which any CMOEA can be easily embedded. The proposed framework can adaptively delete the poorly performed CMOEAs and gradually select the most suitable CMOEA for the CMOP to be solved.
  • Three state-of-the-art CMOEAs (CCMO [17], ICMA [15] and PPS [22]) have been successfully embedded into the proposed framework and a new CMOEA termed ECMOEA is developed. Experimental results have demonstrated the effectiveness of the proposed framework.
  • Experiments on five benchmarks and eight real-world applications have been conducted. Compared to seven state-of-the-art CMOEAs, ECMOEA achieved better or at least equal performance, which further demonstrated the effectiveness of the proposed framework, as well as ECMOEA.
The remainder of this article is organized as follows. Section 2 introduces the related work and motivations. Section 3 details the proposed framework and the generated ECMOEA, then gives the analysis. Afterwards, the experimental studies are presented in Section 4. Finally, some conclusions are drawn and future work is outlined in Section 5.

2. Related Work and Motivations

This section first presents a brief review and the taxonomy of the state-of-the-art CMOEAs. Then, the motivations are given.

2.1. State-of-the-Art CMOEAs

In order to achieve the constraint satisfaction, a constraint-handling technique (CHT) is necessary for a CMOEA. Based on the CHT adopted, the state-of-the-art CMOEAs can be roughly classified into the following three categories.

2.1.1. CMOEAs Based on Constrained Dominance Principle

The first category adopts constrained dominance principle (CDP) [12] as CHT. Assuming two solutions x and y , x is said to constrained dominate y (denoted as x c y ), if one of the following conditions holds:
  • C V ( x ) < C V ( y ) ;
  • C V ( x ) = C V ( y ) and x y .
where x y means that x dominates y with both of the following conditions are satisfied:
  • i 1 , , m , f i ( x ) f i ( y ) ;
  • i 1 , , m , f i ( x ) < f i ( y ) .
CDP is the most widely used CHT due to its simplicity on logic and implementation. Wang et al. [23] proposed a two phase based framework termed ToP in which a transformed constrained single objective optimization problem is optimized in the first phase and the original CMOP is optimized in the second phase. Yao et al. [20] proposed a two archive framework in which the convergence archive aims to promote convergence and feasibility while the diversity archive aims to promote diversity. Liu et al. [24] proposed a bidirectional coevolution framework in which one population approximates constrained Pareto front (CPF) from the feasible regions and the other from the infeasible regions. Tian et al. [17] proposed a weak coevolutionary framework in which one population solves the original CMOP while the other solves the constraints-ignored helper problem, and the two coevolved populations generate offspring on their own. Latter, Tian et al. [25] proposed an adaptively switched two stage framework in which one stage handles the original CMOP while the other ignores constraints. Wang et al. [18] proposed a two ranking based fitness evaluation mechanism in which CDP and Pareto dominance are both considered.

2.1.2. CMOEAs Based on ε -Constrained

The second category adopts constraint relaxation technique i.e.,  ε -constrained technique [26] as CHT. With the relaxation ε , some infeasible solutions might survive to the next generation. Therefore, it enhances the ability of the algorithm to jump out of local feasible regions. Specifically, for two solutions x and y , x ε -constrained dominates y (denoted as x ε y ), if one of the following conditions holds:
  • C V ( x ) < ε & C V ( y ) < ε and x y ;
  • C V ( x ) = C V ( y ) = ε and x y ;
  • C V ( x ) > ε & C V ( y ) > ε and C V ( x ) < C V ( y ) .
ε -constrained technique is also widely used in CMOP community since its relaxation of constraints can alleviate the shortages of CDP. Fan et al. [21] developed an update strategy of ε for the CMOPs with large infeasible regions. Then, they further did some research works on this topic in PPS [22] and PPS-M2M [27] by introducing new update strategies of ε . Zhang et al. [16] developed the detect-and-escape framework in which a detect strategy is designed to tell whether the search is trapped in local optimal and the escape strategy is designed to escape from the local optimal. Zeng et al. [28] proposed to transform a CMOP to a dynamic MOP and use an ε -constrained technique to handle constraints.

2.1.3. CMOEAs Based on Penalty Function

The third category adopts penalty function [29] as CHT. The penalty function combines the CV and objective function values to evaluate the fitness of a solution. Generally, the fitness of solution x , F ( x ) , based on penalty function is calculated as
F ( x ) = F ( x ) + β C V ( x ) ,
where β is the penalty factor.
Jiao et al. [30] developed a feasible-guiding strategy to preserve some valuable infeasible solutions. In c-DPEA [31], Ming et al.designed a self-adaptive penalty function to preserve competitive infeasible solutions in one of its co-evolutionary populations. In ShiP [32], the positions of infeasible solutions are shifted, and the degree of shift is adaptively controlled by the feasible ratio in the current parent and offspring populations. Then, the shifted infeasible solutions are penalized based on the CV values. However, it is still a challenging task to adjust the penalty factor adaptively by using the population feedback information.

2.1.4. CMOEAs Based on New CHTs

Recently, some new CHTs have been proposed in the CMOP community. Some CMOEAs are also developed based on these new CHTs. Liu et al. [33] developed the indicator-based CMOEA framework, in which indicators are used to help with dealing with CMOPs. In TiGE-2 [34], a tri-goal technique is proposed in which three indicators are used for convergence, diversity and feasibility, respectively. Xiang et al. [35] proposed a new CHT which works for the CPF shape detection mechanism to detect problem type. Yuan et al. [15] proposed a new cost value based indicator of which the convergence to CPF property is proofed.

2.2. Motivations

2.2.1. The Limitation of Using a Single CMOEA

Although there have been many CMOEAs, the versatility of these methods is limited. According to the “No Free Lunch” theory, one single algorithm cannot effectively handle all kinds of problems. Different CMOEAs have different skills, which is to say, a specific CMOEA can only effectively handle one kind or some kinds of CMOPs. For example, the CMOEAs applying constraint relaxation technique (i.e.,  ε -constrained) are effective on CMOPs with complex infeasible regions since some infeasible solutions are expected for the algorithm to cross complex infeasible regions. On the contrary, ignoring constraints is effective on handling CMOPs whose PF is similar to or near CPF, because approaching PF can also approach CPF. To better understand this point, experiments of CCMO and PPS on DC2-DTLZ and LIR-CMOP2 are conducted. The parameter settings are the same to which in Section 4.1.3. It should be noted that CCMO is the one that ignores constraints in one of its coevolved populations while PPS is the one uses constraint relaxation technique. The results are presented in Figure 1. It can be easily seen that CCMO performed well on DC2-DTLZ1 (Figure 1a) but poorly on LIR-CMOP2 (Figure 1b), while PPS performed well on LIR-CMOP2 (Figure 1c) but poorly on DC2-DTLZ1 (Figure 1d).

2.2.2. Evaluating the Performance of a CMOEA by HV

Hypervolume (HV) [36] is a performance indicator used for evaluating the performance of an algorithm [37]. HV calculates the volume or hypervolume of the area surrounded by the obtained solution set and the reference point. Due to the independence of true CPF, HV is widely used in the CMOEA community for the performance evaluation when comparing a proposed approach to existing one, especially for real-world application problems whose true CPF is mostly unknown. Therefore, through the comparison of HV value, we can easily tell which CMOEA is better and which is worse. So in this paper, we propose to use HV as the selection criteria in designing the ensemble framework.

2.2.3. Ensemble of CMOEAs

As mentioned and analyzed above, one single CMOEA is not likely to be omnipotent for all kinds of CMOPs. Given the fact that there have been already many CMOEAs that can suit CMOPs with different features, developing an ensemble framework which can adaptively select the most effective CMOEA for a given CMOP is valuable. To be more specific, if an ensemble framework can integrate different CMOEAS such as CCMO and PPS, and tell which one can solve the current CMOP well, then, the ensemble obtains better versatility. In the next section, our proposed ensemble framework, as well as the generated ECMOEA, will be detailed.

3. Proposed Approach

3.1. The Proposed Ensemble Framework

The proposed ensemble framework of CMOEAs can be expressed as the flowchart shown in Figure 2. After initialization, the three embedded CMOEAs (denoted as CMOEA1, CMOEA2 and CMOEA3) are evolved simultaneously. Before  g < 0.5 G m a x (i.e., the first half of evolutionary process), all the three CMOEAs are evolved. When g = 0.5 G m a x , the worst CMOEA is detected by comparing the HV values of the three CMOEAs’ populations. Then, the CMOEA with worst performance (i.e., HV value) is deleted. The evolution of the rest two CMOEAs (assume they are CMOEA1 and CMOEA2) are continued until g < 0.8 G m a x . When g = 0.8 G m a x , the worse CMOEA is detected through the comparison between their populations’ HV values. Then, the one with smaller HV value is deleted. Afterwards, the last survived CMOEA is evolved to the end of the evolutionary process. Finally, the population of the last CMOEA is output as the final solution set.
Generally, the search process is divided into three stages. In the first stage ( g < 0.5 G m a x ), all the CMOEAs embedded are evolved. In the second stage ( 0.5 G m a x g < 0.8 G m a x ), one CMOEA is deleted and the remained two are continued. In the third stage ( 0.8 G m a x g < G m a x ), the last CMOEA is evolved to the end.

3.2. The Proposed ECMOEA

Based on the proposed ensemble framework as related in the above part, a new CMOEA termed ECMOEA is generated. In this paper, the generated ECMOEA is embedded with ICMA, CCMO and PPS. Specifically, these three state-of-the-art CMOEAs work as the CMOEA1, CMOEA2 and CMOEA3 in the framework. The pseudo code is shown in Algorithm 1. The details are as follows. Similarly, these three CMOEAs are denoted as CMOEA1, CMOEA2 and CMOEA3. Besides, the deleted CMEOAs are assumed to be CMOEA3 and CMOEA2 without losing generality.
Algorithm 1 The Framework of ECMOEA
Input: N, termination condition ( g max )
Output:  P
  1:
P 1 , P 2 , P 3 Initialize the populations and parameters for all CMOEAs embedded;
  2:
Initialize the determined mark t 1 = t 2 = 0 , initialize the generation index g = 0 ;
  3:
while g g max do
  4:
   if  g < 0.5 × g max  then
  5:
      P 1 Evolve by CMOEA1;
  6:
      P 2 Evolve by CMOEA2;
  7:
      P 3 Evolve by CMOEA3;
  8:
   else if  g < 0.8 × g max  then
  9:
     if  t 1 = = 0  then
10:
        Calculate the HV values of all populations P 1 , P 2 , P 3 ;
11:
        Delete the CMOEA with minimum HV value (assume it is CMOEA3);
12:
        Set t 1 = 1 ;
13:
     else
14:
         P 1 Evolve by CMOEA1;
15:
         P 2 Evolve by CMOEA2;
16:
     end if
17:
   else
18:
     if  t 2 = = 0  then
19:
        Calculate the HV values of all populations P 1 , P 2 ;
20:
        Delete the CMOEA with minimum HV value (assume it is CMOEA2);
21:
        Set t 2 = 1 ;
22:
     else
23:
         P 1 Evolve by CMOEA1;
24:
     end if
25:
   end if
26:
    g g + 1 ;
27:
end while
28:
return P 1
First of all, the populations and parameters (include the reference vector set if needed) are initialized (line 1). Furthermore, the determined mark t 1 , t 2 and the generation index g are initialized (line 2). Then, until the termination condition g G m a x met, the following steps are performed.
  • If 0 g < 0.5 G m a x , lines 5–7 are performed, which is the first stage as mentioned above.
  • If 0.5 G m a x g < 0.8 G m a x , lines 9–15 are performed, which is the second stage as mentioned above. The determined mark t 1 is used in this stage to determine whether the worst CMOEA is deleted.
  • If 0.8 G m a x g < G m a x , lines 18–26 are performed, which is the third stage as mentioned above. The determined mark t 2 is used in this stage to determine whether the worse CMOEA is deleted.
With ICMA, CCMO and PPS embedded, an instantiation is presented in Algorithm 2 to better understand the principle and procedure of the proposed framework, as well as the generated ECMOEA. In this example, without losing generality, we assume that ICMA is the most suitable CMOEA for the CMOP, while PPS is the most unsuitable CMOEA. Then the algorithm goes as follows:
First, three populations and all the parameters needed are initialized (lines 1–2). Then, in the first stage (lines 4–7), all the three CMOEAs are co-evolved to evolve P 1 , P 2 and P 3 , respectively. When g = 0.5 × g m a x , the HV values of P 1 , P 2 and P 3 are calculated. Since PPS is the most unsuitable CMOEA, is obtained the minimum HV value. Therefore, at the first decision time, PPS is deleted (lines 9–12).Then, since t 1 is set 1, during  0.5 × g m a x < g < 0.8 × g m a x , ICMA and CCMO are co-evolved (lines 14–15). When g = 0.8 × g m a x , it comes to the second decision time (lines 18–21). Since ICMA is the most suitable CMOEA for the CMOP to be solved, the HV value of P 2 , obtained by CCMO, is less than the HV value of P 1 . Therefore, at this time, CCMO is deleted and in the rest evolutionary process, only ICMA is evolved (lines 22–23). Finally, P 1 , the population obtained by the most suitable CMOEA (i.e., ICMA) is regarded as the final solution set.
Algorithm 2 The instantiation of ECMOEA
Input: N, termination condition ( g max )
Output:  P
  1:
P 1 , P 2 , P 3 Initialize the populations and parameters for ICMA, CCMO and PPS;
  2:
Initialize the determined mark t 1 = t 2 = 0 , initialize the generation index g = 0 ;
  3:
while g g max do
  4:
   if  g < 0.5 × g max  then
  5:
      P 1 Evolve by ICMA;
  6:
      P 2 Evolve by CCMO;
  7:
      P 3 Evolve by PPS;
  8:
   else if  g < 0.8 × g max  then
  9:
     if  t 1 = = 0  then
10:
        Calculate the HV values of all populations P 1 , P 2 , P 3 ;
11:
        PPS is deleted due to its minimum HV value;
12:
        Set t 1 = 1 ;
13:
     else
14:
         P 1 Evolve by ICMA;
15:
         P 2 Evolve by CCMO;
16:
     end if
17:
   else
18:
     if  t 2 = = 0  then
19:
        Calculate the HV values of all populations P 1 , P 2 ;
20:
        CCMO is deleted due to its minimum HV value;
21:
        Set t 2 = 1 ;
22:
     else
23:
         P 1 Evolve by ICMA;
24:
     end if
25:
   end if
26:
    g g + 1 ;
27:
end while
28:
return P 1

3.3. Computational Complexity

As mentioned above in this Section, ECMOEA mainly includes the embedded CMEOAs and the HV calculation. The computational complexity of the embedded CMOEAs are the same to the original CMOEAs. Since the calculation of HV values only performed at g = 0.5 G m a x and g = 0.8 G m a x , the time consumption can be ignored. Therefore, the computational complexity of the proposed ensemble framework is determined by the embedded CMOEAs. In this paper, since ICMA, CCMO and PPS are embedded, whose computational complexities are O ( m N 2 ) , O ( m N 3 ) and O ( m N 2 ) , the generated ECMOEA’s computational complexity is O ( m N 3 ) .

4. Experimental Results and Analysis

In this section, experimental studies are conducted on PlatEMO [38] matlab platform in order to evaluate the effectiveness of the proposed ensemble framework and the generated ECMOEA. Section 4.1 details the experimental settings, then, Section 4.2 demonstrates the effectiveness of the proposed ensemble framework. Section 4.3 presents the comparison results of ECMOEA to other methods and the analysis. Next, Section 4.4 evaluates the effectiveness of ECMOEA on real-world CMOPs. Finally, the effectiveness of ECMOEA on constrained many-objective optimization problems (CMaOPs) are tested in Section 4.5.

4.1. Experimental Settings

4.1.1. CMOEAs and CMOPs in Comparison

In the generated ECMOEA, ICMA [15], CCMO [17] and PPS [22] are embedded as the candidates. Therefore, these three CMOEAs are chosen as comparisons in Section 4.2. Besides, CMOEA-MS, C-TAEA, DCNSGA-III, MOEA/D-DAE, NSGA-II-ToR, TiGE-2 and ToP are chosen as comparisons in Section 4.3, Section 4.4 and Section 4.5.
In order to testify the effectiveness and versatility of the proposed framework and ECMOEA, five CMOP benchmarks are chosen. They are the DTLZ variants (i.e., C-DTLZ [12] and DC-DTLZ [20]), DAS-CMOP [39], FCP [15], LIR-CMOP [21] and MW [40]. The number of objectives m and decision variables n of each CMOP are set as follows. m = 3 for all C-DTLZ and DC-DTLZ, as well as MW4, MW8, MW14, DAS-CMOP7-DAS-CMOP9 and LIR-CMOP13-14, while the rest of the CMOPs are all set m = 2 . n = 7 for C1-DTLZ1 as well as all the DC-DTLZ1 instances, and n = 12 for the rest of C-DTLZ, DC-DTLZ. n = 15 for MW, 30 for FCP, LIR-CMOP and DAS-CMOP.

4.1.2. Performance Indicators

The performance of a CMOEA is evaluated by two performance indicators: inverted generational distance (IGD) [10] and hypervolume (HV) [36]. IGD is calculated by Equation (5) when S is the final solution set, W is the reference vector set sampled on the true PF and d ( w i , x j ) is the Euclidean distance in the objective space between the solution x j in S and the vector w i in W:
I G D ( S , W ) = 1 | W | i = 1 | W | min j = 1 | S | d ( w i , x j ) .
Suppose λ is the Lebesgue measure, v i is the hypervolume shaped by the solution x i and the reference point, then HV is calculated by Equation (6):
H V = λ ( i = 1 | S | v i ) .
A smaller IGD value means better performance of the algorithm. In contrast, the greater the HV value, the better the performance of the algorithm.
According to the approaches in [41], 10,000 uniformly distributed points were sampled on the true CPF for IGD calculation. As for the HV calculation, the objective values were first normalized, and in the normalized objective space, ( 1.1 , 1.1 , , 1.1 ) was adopted as the reference point for the HV calculation. Each algorithm was run independently 30 times on each test case. The mean and standard deviations of each metric (i.e., HV and IGD) were recorded. Then the Wilcoxon rank sum test [42] with a significance level of 0.05 and Friedman test [43] with Bonferroni correction at a significance level 0.05 were adopted to perform statistical analysis on the experimental results. In all the tables showing statistical results, we use “+”, “−”, and “=” to show that the result obtained by another CMOEA is significantly better, significantly worse or statistically similar to that obtained by ECMOEA.

4.1.3. Genetic Operators and Parameter Settings

PPS and MOEA/D-DAE adopt the DE operator since they are embedded with MOEA/D [13]. ToP also adopts DE as genetic operator. For these three methods, the parameter settings are α = 0.95 , τ = 0.1 , c p = 2 , and l = 20 . For ECMOEA and the other CMOEAs, Polynomial Mutation (PM) [12] and Simulated Binary Crossover (SBX) [44] are applied in the GA evolutionary operator with parameters set as follows.
  • SBX: Crossover probability p c = 1 and distribution index η c = 20 .
  • PM: Mutation probability p m = 1 / n and distribution index η m = 20 .
All other parameters for the MOEAs are the same as in their original literatures, which are the default settings in PlatEMO [38]. These parameter settings did not change in our experiments.
The population size N is set to 91 for all problems in the experiments. The maximum function evaluation times is set to 100,000 for C-DTLZ, DC-DTLZ, FCP and MW and 150,000 for DAS-CMOP and LIR-CMOP to ensure the convergence of these CMOP instances.

4.2. Demonstration of the Effectiveness of the Proposed Ensemble Framework

First of all, we designed the experiment to demonstrate the effectiveness of the proposed ensemble framework. Specifically, three state-of-the-art CMOEAS, ICMA, CCMO and PPS are embedded into the proposed framework, and a new CMOEA termed ECMOEA is generated as related above. Then, ECMOEA is compared with ICMA, CCMO and PPS in this part. Thus, the effectiveness of the proposed framework can be verified through comparison to its original implants.
The statistical results of ECMOEA and CCMO, ICMA, PPS on DAS-CMOP, DTLZ, FCP, LIR-CMOP and MW in terms of HV and IGD are reported in Table A1, Table A2, Table A3, Table A4, Table A5, Table A6, Table A7, Table A8, Table A9 and Table A10. Assuming > , < and = mean the former CMOEA performed better than, worse than and equal to the latter one. From these tables we can see that:
  • DAS-CMOP: ICMA > CCMO > PPS, while ICMA > CCMO > ECMOEA > PPS;
  • DTLZ: CCMO > ICMA > PPS, while CCMO > ECMOEA > ICMA > PPS;
  • FCP: ICMA > CCMO = PPS, while ECMOEA = ICMA > CCMO = PPS;
  • LIR-CMOP: ICMA = PPS > CCMO, while ECMOEA > ICMA = PPS > CCMO;
  • MW: ICMA > CCMO > PPS, while ICMA > CCMO = ECMOEA > PPS.
From these results, we can see that the most suitable CMOEA for DAS-CMOP, DTLZ, FCP, LIR-CMOP and MW are ICMA, CCMO, ICMA, ICMA (and PPS) and ICMA, respectively. Under the proposed framework, the generated ECMOEA finally obtained better results than those CMOEAs except for the most suitable one. This is because the evaluations are aparted to those worse CMOEAs, thus, ECMOEA can not concentrate on evolving the most suitable CMOEA. However, the results indicate that the proposed framework can finally select the suitable CMOEA.

4.3. Comparison to Other CMOEAs

Then, in this part, ECMOEA is compared with seven state-of-the-art CMOEAs: CMOEA-MS, C-TAEA, DCNSGA-III, MOEA/D-DAE, NSGA-II-ToR, TiGE-2 and ToP. The statistical results of ECMOEA and CMOEAs in comparison on DAS-CMOP, DTLZ, FCP, LIR-CMOP and MW in terms of HV and IGD are reported in Table A11, Table A12, Table A13, Table A14, Table A15, Table A16, Table A17, Table A18, Table A19 and Table A20, respectively. From the tables, the proposed ECMOEA obtained the best overall performance on DAS-CMOP, FCP, LIR-CMOP and MW benchmarks. As for DTLZ, ECMOEA was slightly worse than MOEA/D-DAE, but was better than other methods. Especially, ECMOEA performed well on FCP and LIE-CMOP benchmarks.
To better illustrate the results, the feasible and non-dominated solution sets obtained by ECMOEA on all DAS-CMOP, DTLZ, FCP, LIR-CMOP and MW instances with the median IGD value among 30 runs are depicted in Figure A1, Figure A2, Figure A3, Figure A4 and Figure A5. From these figures we can see ECMOEA can finally approximate CPF and obtain good distribution in face of the different features of these benchmarks. When dealing with DAS-CMOP with adjustable feasibility, convergence and diversity, the ensemble framework can select ICMA and CCMO to adapt to the requirements. When dealing with DTLZ with regular and irregular CPF, discontinuous feasible regions and inner-outer layer feasible regions, the weak coevoluion based CCMO can be selected to adapt to these features. When dealing with FCP, ICMA can be selected whose indicator-based CHT can preserve some infeasible solutions to better deal with FCP instances. Similarly, when dealing with LIR-CMOP, ICMA and PPS, which is tailored for LIR-CMOP, can be selected. When dealing with MW, which contains different geometric relations between PF and CPF, CCMO can be selected to adapt to the CMOPs whose CPF is close to PF, and ICMA or PPS can be selected when CPF is far from PF.
The KEEL [45] software was used to perform the non-parameter statistical analysis on HV and IGD results. The Friedman test and the Wilcoxon test were conducted. The results of Friedman test are reported in Table 1, where the grey-blodface indicates the best ranking obtained by an algorithm and the blodface means the second ranking of an algorithm. As we can see, ECMOEA obtained the first ranking, while ICMA and CCMO obtained the second and the third, respectively. The results of Wilcoxon test are reported in Table 2 and Table 3, respectively. As can be seen, ECMOEA was significantly better than other all methods. Therefore, ECMOEA outperformed other methods statistically.

4.4. ECMOEA on Real-World CMOPs

In this part, experiments of ECMOEA and CMOEAs in comparison on eight mechanical and chemical applications from RWMOP [46] are performed. The detailed parameter settings and information about these real-world CMOPs are shown in Table 4. The original references can be found in RWMOP literature.
Since the true CPF of these real-world CMOPs are unknown, only HV is used for the comparison of ECMOEA and other methods. The statistical results of HV obtained by ECMOEA and CMOEAs in comparison on these eight instances are reported in Table A21. It is clear that ECMOEA obtained the best overall results on all the eight instances. ECMOEA outperformed most other methods when dealing with various real-world problems with various features. This is because ECMOEA can adapt to different problems with the help of the ensemble framework.
Two bar truss design (TBT) problem is chosen for further studies. The mathematical formulations of objective functions and constraints are as follows:
Minimize : f 1 ( x ) = x 1 16 + x 3 2 + x 2 1 + x 3 2 f 2 ( x ) = 20 16 + x 3 2 x 3 x 1 subject to : g 1 ( x ) = f 1 ( x ) 0.1 , g 2 ( x ) = f 2 ( x ) 10 5 , g 3 ( x ) = 80 1 + x 3 2 x 3 x 2 10 5 0 with bounds : 10 5 x 1 100 , 10 5 x 2 100 , 1 x 3 3 ,
The feasible and non-dominated solution sets obtained by ECMOEA and other methods on TBT with the median IGD value among 30 runs are presented in Figure A6. From the results we can find that ECMOEA finally approximated CPF and obtained good distribution.

4.5. ECMOEA on CMaOPs

In this part, the effectiveness of ECMOEA on CMaOPs is tested through comparisons on C-DTLZ and DC-DTLZ test suites. For m = 5 , 8 , n = 9 , 12 for C1-DTLZ1 and DC1-DTLZ1, n = 14 , 17 for other instances, the population size is N = 126 , 156 and the maximum evaluation is E m a x = 100,000, 140,000, respectively. The experimental settings are the same to which introduced in Section 4.1.
The statistical results on HV and IGD are listed in Table A22 and Table A23, respectively. Generally, ECMOEA obtained slightly better overall performance than ICMA, CCMO and PPS. The reasons can be that, since three CMOEAs (i.e., ICMA, CCMO and PPS) are all co-evolved during the evolutionary process, the function evaluations are divided. Therefore, in dealing with some CMOPs that the embedded CMOEA is good at (e.g., ICMA and CCMO on C-DTLZ), even ECMOEA finally find the most suitable CMOEA, less evaluations are used. On the contrary, ECMOEA can outperform ICMA and CCMO on some DC-DTLZ instances, because ECMOEA finally found that PPS is more suitable for these DC-DTLZ instances as DC-DTLZ instances are of more complex infeasible regions.
However, the other algorithms, especially the constrained many-objective optimization evolutionary algorithms (CMaOEAs) in comparison, obtained better performance than ECMOEA.

5. Conclusions

In this paper, an ensemble framework of CMOEA is proposed, into which any existing CMOEA or new CMOEA can be easily embedded. The framework possesses the ability to tell which CMOEA is the best for the given CMOP, and thus, choosing the most suitable CMOEA as the solver. Based on this ensemble framework and three state-of-the-art CMOEAs, a new CMOEA termed ECMOEA is proposed. The proposed ECMOEA can gradually delete the poorly performed CMOEAs and finally choose the most suitable one.
Experimental results on five benchmarks of totally 52 instances have verified the effectiveness of the proposed framework, and demonstrated the superiority or at least competitiveness of ECMOEA compared to seven state-of-the-art CMOEAs. Furthermore, the effectiveness of ECMOEA on real-world applications have been testified through experiments on eight real-world application CMOPs.
In the future, the following attempts are worth trying:
  • Embedding other CMOEAs or mathematical methods into the framework to further enhance the versatility of ECMOEA;
  • Designing an adaptive strategy instead of the current 0.8 E m a x , 0.5 E m a x strategy can save the evaluations for the most suitable CMOEA;
  • Adopting advanced techniques in Machine learning field such as the k-nearest neighbors algorithm (KNN) [47] to improve the efficiency of the ensemble framework is also worth trying;
  • Embedding effective CMaOEAs in the ensemble framework to enhance the performance on handling CMaOPs is also expected.

Author Contributions

Conceptualization, methodology, validation, data curation, writing—original draft preparation, J.K.; validation, data curation, writing—original draft preparation, F.M.; supervision, writing—review and editing, funding acquisition, W.G. All authors have read and agreed to the published version of the manuscript.

Funding

This work was partly supported by the Hainan Provincial Natural Science Foundation of China under Grant No. 621RC599 and the National Natural Science Foundation of China under Grant No. 62076225.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:
CMOPsConstrained multi-objective optimization problems
CMOEAsConstrained multi-objective optimization evolutionary algorithms
HVHypervolume
ECMOEAEnsemble of constrained multi-objective optimization evolutionary algorithm
CMOConstrained multi-objective optimization
MaOEAsMany-objective optimization evolutionary algorithms
MOPsMulti-objective optimization problems
CHTConstraint-handling technique
CDPConstrained dominance principle
CPFConstrained Pareto front

Appendix A

Table A1. Statistical results of HV obtained by ECMOEA and CMOEAs embedded on DAS-CMOP benchmark problems. Best result in each row is highlighted.
Table A1. Statistical results of HV obtained by ECMOEA and CMOEAs embedded on DAS-CMOP benchmark problems. Best result in each row is highlighted.
ProblemCCMOICMAPPSECMOEA
DASCMOP11.0955 × 10 2 (1.22 × 10 2 ) −2.1186 × 10 1 (4.66 × 10 4 ) ≈1.8005 × 10 1 (4.64 × 10 2 ) −2.1202 × 10 1 (4.62 × 10 4 )
DASCMOP22.5804 × 10 1 (2.26 × 10 3 ) −3.5513 × 10 1 (7.10 × 10 5 ) +3.5448 × 10 1 (1.09 × 10 4 ) −3.5504 × 10 1 (7.11 × 10 5 )
DASCMOP32.1644 × 10 1 (1.65 × 10 2 ) −3.1169 × 10 1 (6.35 × 10 4 ) ≈2.2181 × 10 1 (3.19 × 10 2 ) −3.1191 × 10 1 (4.13 × 10 4 )
DASCMOP42.0215 × 10 1 (3.58 × 10 3 ) +1.7537 × 10 1 (4.37 × 10 2 ) ≈1.3695 × 10 1 (5.84 × 10 2 ) −1.7485 × 10 1 (4.48 × 10 2 )
DASCMOP53.5138 × 10 1 (1.26 × 10 4 ) +3.2192 × 10 1 (6.86 × 10 2 ) −2.9023 × 10 1 (1.09 × 10 1 ) −3.3874 × 10 1 (4.35 × 10 2 )
DASCMOP63.0330 × 10 1 (1.16 × 10 2 ) +2.0688 × 10 1 (1.05 × 10 1 ) ≈1.5702 × 10 1 (1.29 × 10 1 ) −2.4548 × 10 1 (9.70 × 10 2 )
DASCMOP72.8808 × 10 1 (3.28 × 10 4 ) +2.7977 × 10 1 (1.49 × 10 2 ) +1.9470 × 10 1 (1.01 × 10 1 ) −2.7886 × 10 1 (1.84 × 10 2 )
DASCMOP82.0676 × 10 1 (4.33 × 10 4 ) +2.0230 × 10 1 (1.18 × 10 2 ) ≈1.5235 × 10 1 (5.44 × 10 2 ) −1.9989 × 10 1 (1.74 × 10 2 )
DASCMOP91.2726 × 10 1 (1.13 × 10 2 ) −2.0623 × 10 1 (2.37 × 10 4 ) +1.5957 × 10 1 (3.86 × 10 2 ) −2.0613 × 10 1 (2.33 × 10 4 )
× + / / 5/4/03/1/50/9/0
Table A2. Statistical results of IGD obtained by ECMOEA and CMOEAs embedded on DAS-CMOP benchmark problems. Best result in each row is highlighted.
Table A2. Statistical results of IGD obtained by ECMOEA and CMOEAs embedded on DAS-CMOP benchmark problems. Best result in each row is highlighted.
ProblemCCMOICMAPPSECMOEA
DASCMOP17.1644 × 10 1 (3.65 × 10 2 ) −3.5170 × 10 3 (2.94 × 10 4 ) +1.4262 × 10 1 (2.25 × 10 1 ) −3.9235 × 10 3 (5.39 × 10 4 )
DASCMOP22.4445 × 10 1 (2.81 × 10 2 ) −4.6777 × 10 3 (1.14 × 10 4 ) +5.7132 × 10 3 (1.86 × 10 4 ) −4.7686 × 10 3 (1.02 × 10 4 )
DASCMOP33.2465 × 10 1 (3.87 × 10 2 ) −2.0115 × 10 2 (4.85 × 10 4 ) ≈3.0145 × 10 1 (1.01 × 10 1 ) −1.9721 × 10 2 (1.18 × 10 3 )
DASCMOP41.9585 × 10 3 (1.23 × 10 3 ) +8.0629 × 10 2 (1.15 × 10 1 ) ≈2.4094 × 10 1 (2.22 × 10 1 ) −7.8203 × 10 2 (1.17 × 10 1 )
DASCMOP53.0351 × 10 3 (8.18 × 10 5 ) +4.9244 × 10 2 (1.11 × 10 1 ) −1.1761 × 10 1 (2.20 × 10 1 ) −2.2552 × 10 2 (6.96 × 10 2 )
DASCMOP63.3377 × 10 2 (2.57 × 10 2 ) +2.1590 × 10 1 (1.90 × 10 1 ) ≈3.7859 × 10 1 (3.15 × 10 1 ) −1.5724 × 10 1 (2.12 × 10 1 )
DASCMOP73.1554 × 10 2 (6.02 × 10 4 ) +4.7193 × 10 2 (2.93 × 10 2 ) +2.7692 × 10 1 (3.05 × 10 1 ) −5.0821 × 10 2 (3.96 × 10 2 )
DASCMOP84.1461 × 10 2 (7.30 × 10 4 ) +5.0302 × 10 2 (2.08 × 10 2 ) +2.1402 × 10 1 (1.74 × 10 1 ) −5.5965 × 10 2 (3.30 × 10 2 )
DASCMOP93.7207 × 10 1 (6.85 × 10 2 ) −4.2816 × 10 2 (7.71 × 10 4 ) ≈2.3194 × 10 1 (1.65 × 10 1 ) −4.3039 × 10 2 (8.96 × 10 4 )
× + / / 5/4/04/1/40/9/0
Table A3. Statistical results of HV obtained by ECMOEA and CMOEAs embedded on DTLZ benchmark problems. Best result in each row is highlighted.
Table A3. Statistical results of HV obtained by ECMOEA and CMOEAs embedded on DTLZ benchmark problems. Best result in each row is highlighted.
ProblemCCMOICMAPPSECMOEA
C1-DTLZ18.3793 × 10 1 (4.15 × 10 3 ) +8.3640 × 10 1 (9.53 × 10 4 ) ≈8.1831 × 10 1 (4.15 × 10 3 ) −8.3117 × 10 1 (9.67 × 10 3 )
C1-DTLZ35.5756 × 10 1 (1.45 × 10 3 ) ≈3.1578 × 10 1 (2.27 × 10 1 ) −2.8246 × 10 1 (2.54 × 10 1 ) −5.5559 × 10 1 (6.92 × 10 3 )
C2-DTLZ25.1357 × 10 1 (1.66 × 10 3 ) ≈5.0529 × 10 1 (2.24 × 10 3 ) −4.9844 × 10 1 (3.73 × 10 3 ) −5.1346 × 10 1 (1.64 × 10 3 )
C3-DTLZ47.7982 × 10 1 (4.59 × 10 2 ) −7.9037 × 10 1 (9.15 × 10 4 ) ≈7.6121 × 10 1 (2.65 × 10 2 ) −7.9059 × 10 1 (8.34 × 10 4 )
DC1-DTLZ16.3124 × 10 1 (1.32 × 10 3 ) +6.2189 × 10 1 (2.72 × 10 3 ) −5.6239 × 10 1 (7.39 × 10 2 ) −6.3048 × 10 1 (1.77 × 10 3 )
DC1-DTLZ34.7277 × 10 1 (1.12 × 10 3 ) +3.5011 × 10 1 (1.26 × 10 1 ) −2.9593 × 10 1 (1.42 × 10 1 ) −4.6722 × 10 1 (2.48 × 10 2 )
DC2-DTLZ18.4029 × 10 1 (6.69 × 10 4 ) +6.8184 × 10 1 (1.90 × 10 1 ) −7.4682 × 10 1 (1.44 × 10 1 ) −8.1760 × 10 1 (7.45 × 10 2 )
DC2-DTLZ35.5765 × 10 1 (1.30 × 10 3 ) +8.6141 × 10 3 (4.27 × 10 3 ) −1.6481 × 10 1 (2.40 × 10 1 ) −4.4170 × 10 1 (1.82 × 10 1 )
DC3-DTLZ15.3493 × 10 1 (1.36 × 10 3 ) ≈4.4580 × 10 1 (1.66 × 10 1 ) −1.3712 × 10 1 (1.95 × 10 1 ) −5.3397 × 10 1 (2.92 × 10 3 )
DC3-DTLZ33.6618 × 10 1 (1.62 × 10 3 ) +0.0000 × 10 0 (0.00 × 10 0 ) −1.5702 × 10 2 (6.49 × 10 2 ) −1.8319 × 10 1 (1.59 × 10 1 )
× + / / 6/1/30/8/20/10/0
Table A4. Statistical results of IGD obtained by ECMOEA and CMOEAs embedded on DTLZ benchmark problems. Best result in each row is highlighted.
Table A4. Statistical results of IGD obtained by ECMOEA and CMOEAs embedded on DTLZ benchmark problems. Best result in each row is highlighted.
ProblemCCMOICMAPPSECMOEA
C1-DTLZ12.1026 × 10 2 (2.08 × 10 4 ) +2.2331 × 10 2 (3.27 × 10 4 ) ≈2.5579 × 10 2 (6.00 × 10 4 ) −2.2863 × 10 2 (2.55 × 10 3 )
C1-DTLZ35.6293 × 10 2 (7.47 × 10 4 ) ≈2.6582 × 10 1 (2.64 × 10 1 ) −3.5246 × 10 0 (4.00 × 10 0 ) −5.7235 × 10 2 (3.87 × 10 3 )
C2-DTLZ24.5218 × 10 2 (7.21 × 10 4 ) ≈4.8490 × 10 2 (7.42 × 10 4 ) −5.5625 × 10 2 (1.91 × 10 3 ) −4.4994 × 10 2 (4.98 × 10 4 )
C3-DTLZ41.2440 × 10 1 (1.35 × 10 1 ) −1.0368 × 10 1 (1.25 × 10 3 ) −1.6846 × 10 1 (9.08 × 10 2 ) −1.0285 × 10 1 (1.51 × 10 3 )
DC1-DTLZ11.2128 × 10 2 (1.30 × 10 4 ) ≈1.2549 × 10 2 (2.18 × 10 4 ) −4.2156 × 10 2 (5.79 × 10 2 ) −1.2178 × 10 2 (1.84 × 10 4 )
DC1-DTLZ33.5809 × 10 2 (4.23 × 10 4 ) ≈9.8393 × 10 2 (8.14 × 10 2 ) −5.0327 × 10 1 (7.73 × 10 1 ) −4.6968 × 10 2 (6.10 × 10 2 )
DC2-DTLZ12.1196 × 10 2 (1.21 × 10 4 ) +8.3092 × 10 2 (7.43 × 10 2 ) −5.2025 × 10 2 (5.43 × 10 2 ) −2.8151 × 10 2 (2.73 × 10 2 )
DC2-DTLZ35.5824 × 10 2 (6.52 × 10 4 ) +6.0923 × 10 1 (9.23 × 10 2 ) −4.2698 × 10 1 (2.40 × 10 1 ) −1.4449 × 10 1 (1.73 × 10 1 )
DC3-DTLZ17.2739 × 10 3 (9.68 × 10 5 ) ≈3.5131 × 10 2 (6.28 × 10 2 ) −8.8632 × 10 1 (1.39 × 10 0 ) −7.3281 × 10 3 (1.26 × 10 4 )
DC3-DTLZ32.1295 × 10 2 (3.42 × 10 4 ) +1.3236 × 10 0 (6.04 × 10 1 ) −3.4857 × 10 0 (2.53 × 10 0 ) −2.6746 × 10 1 (2.54 × 10 1 )
× + / / 4/1/50/9/10/10/0
Table A5. Statistical results of HV obtained by ECMOEA and CMOEAs embedded on FCP benchmark problems. Best result in each row is highlighted.
Table A5. Statistical results of HV obtained by ECMOEA and CMOEAs embedded on FCP benchmark problems. Best result in each row is highlighted.
ProblemCCMOICMAPPSECMOEA
FCP1NaN (NaN)5.8083 × 10 1 (1.86 × 10 4 ) ≈NaN (NaN)5.8083 × 10 1 (1.37 × 10 4 )
FCP2NaN (NaN)4.3108 × 10 1 (3.92 × 10 4 ) ≈NaN (NaN)4.3120 × 10 1 (1.92 × 10 4 )
FCP3NaN (NaN)3.4654 × 10 1 (6.65 × 10 5 ) ≈NaN (NaN)3.4653 × 10 1 (1.48 × 10 4 )
FCP4NaN (NaN)6.3374 × 10 1 (3.75 × 10 4 ) ≈NaN (NaN)6.3374 × 10 1 (4.57 × 10 4 )
FCP52.4812 × 10 1 (5.29 × 10 2 ) −4.4573 × 10 1 (7.36 × 10 2 ) ≈2.5936 × 10 1 (6.08 × 10 3 ) −4.2380 × 10 1 (9.07 × 10 2 )
× + / / 0/1/00/0/50/1/0
Table A6. Statistical results of IGD obtained by ECMOEA and CMOEAs embedded on FCP benchmark problems. Best result in each row is highlighted.
Table A6. Statistical results of IGD obtained by ECMOEA and CMOEAs embedded on FCP benchmark problems. Best result in each row is highlighted.
ProblemCCMOICMAPPSECMOEA
FCP1NaN (NaN)3.9157 × 10 2 (1.10 × 10 3 ) ≈NaN (NaN)3.9006 × 10 2 (7.10 × 10 4 )
FCP2NaN (NaN)3.9626 × 10 2 (7.77 × 10 3 ) ≈NaN (NaN)3.9383 × 10 2 (1.03 × 10 2 )
FCP3NaN (NaN)4.3004 × 10 2 (8.80 × 10 4 ) ≈NaN (NaN)4.2828 × 10 2 (1.24 × 10 3 )
FCP4NaN (NaN)3.0491 × 10 2 (9.80 × 10 4 ) ≈NaN (NaN)3.0694 × 10 2 (1.50 × 10 3 )
FCP54.6497 × 10 0 (1.07 × 10 1 ) −6.4384 × 10 1 (1.61 × 10 0 ) ≈4.6815 × 10 0 (9.93 × 10 3 ) −1.1119 × 10 0 (2.00 × 10 0 )
× + / / 0/1/00/0/50/1/0
Table A7. Statistical results of HV obtained by ECMOEA and CMOEAs embedded on LIR-CMOP benchmark problems. Best result in each row is highlighted.
Table A7. Statistical results of HV obtained by ECMOEA and CMOEAs embedded on LIR-CMOP benchmark problems. Best result in each row is highlighted.
ProblemCCMOICMAPPSECMOEA
LIRCMOP11.2872 × 10 1 (2.27 × 10 2 ) −2.1268 × 10 1 (1.43 × 10 2 ) −2.2550 × 10 1 (1.34 × 10 2 ) ≈2.2184 × 10 1 (1.41 × 10 2 )
LIRCMOP22.4237 × 10 1 (2.01 × 10 2 ) −3.4296 × 10 1 (6.18 × 10 3 ) −3.4994 × 10 1 (1.70 × 10 2 ) −3.5079 × 10 1 (7.94 × 10 3 )
LIRCMOP31.1113 × 10 1 (1.66 × 10 2 ) −1.8126 × 10 1 (1.57 × 10 2 ) −1.7366 × 10 1 (2.83 × 10 2 ) ≈1.8854 × 10 1 (1.36 × 10 2 )
LIRCMOP42.0371 × 10 1 (2.15 × 10 2 ) −2.8076 × 10 1 (1.46 × 10 2 ) −2.8100 × 10 1 (3.20 × 10 2 ) ≈2.8486 × 10 1 (2.73 × 10 2 )
LIRCMOP51.6274 × 10 1 (2.75 × 10 2 ) −1.8179 × 10 1 (1.08 × 10 1 ) −2.8046 × 10 1 (5.30 × 10 2 ) ≈2.8993 × 10 1 (1.09 × 10 3 )
LIRCMOP61.1719 × 10 1 (1.44 × 10 2 ) −1.3177 × 10 1 (6.56 × 10 2 ) −1.9630 × 10 1 (2.74 × 10 4 ) +1.9391 × 10 1 (6.44 × 10 3 )
LIRCMOP72.4807 × 10 1 (8.05 × 10 3 ) −2.8512 × 10 1 (1.33 × 10 2 ) +2.4584 × 10 1 (1.27 × 10 2 ) −2.7155 × 10 1 (1.91 × 10 2 )
LIRCMOP82.3460 × 10 1 (1.16 × 10 2 ) −2.9214 × 10 1 (6.83 × 10 3 ) ≈2.3764 × 10 1 (2.03 × 10 2 ) −2.8425 × 10 1 (1.70 × 10 2 )
LIRCMOP93.5952 × 10 1 (6.03 × 10 2 ) −4.8875 × 10 1 (3.01 × 10 2 ) ≈4.1392 × 10 1 (6.41 × 10 2 ) −4.7741 × 10 1 (2.89 × 10 2 )
LIRCMOP106.3190 × 10 1 (3.18 × 10 2 ) −6.9252 × 10 1 (2.70 × 10 2 ) +5.7897 × 10 1 (6.37 × 10 2 ) −6.7585 × 10 1 (3.22 × 10 2 )
LIRCMOP116.6408 × 10 1 (1.49 × 10 2 ) −6.7655 × 10 1 (2.98 × 10 2 ) ≈4.7948 × 10 1 (1.44 × 10 1 ) −6.7327 × 10 1 (2.91 × 10 2 )
LIRCMOP125.0812 × 10 1 (4.95 × 10 2 ) −5.8038 × 10 1 (1.99 × 10 2 ) ≈5.1359 × 10 1 (4.81 × 10 2 ) −5.8864 × 10 1 (2.42 × 10 2 )
LIRCMOP135.5146 × 10 1 (1.50 × 10 3 ) ≈5.4305 × 10 1 (1.52 × 10 3 ) −5.0949 × 10 1 (6.83 × 10 3 ) −5.5170 × 10 1 (1.14 × 10 3 )
LIRCMOP145.5160 × 10 1 (1.31 × 10 3 ) ≈5.5067 × 10 1 (1.37 × 10 3 ) ≈5.2900 × 10 1 (4.89 × 10 3 ) −5.5091 × 10 1 (1.62 × 10 3 )
× + / / 0/12/22/7/51/9/4
Table A8. Statistical results of IGD obtained by ECMOEA and CMOEAs embedded on LIR-CMOP benchmark problems. Best result in each row is highlighted.
Table A8. Statistical results of IGD obtained by ECMOEA and CMOEAs embedded on LIR-CMOP benchmark problems. Best result in each row is highlighted.
ProblemCCMOICMAPPSECMOEA
LIRCMOP12.5203 × 10 1 (7.52 × 10 2 ) −6.6140 × 10 2 (4.37 × 10 2 ) −3.0230 × 10 2 (2.54 × 10 2 ) ≈4.0024 × 10 2 (3.57 × 10 2 )
LIRCMOP22.3703 × 10 1 (3.69 × 10 2 ) −3.7748 × 10 2 (1.13 × 10 2 ) −2.3535 × 10 2 (2.66 × 10 2 ) +2.5367 × 10 2 (1.84 × 10 2 )
LIRCMOP32.7298 × 10 1 (5.77 × 10 2 ) −7.0818 × 10 2 (3.72 × 10 2 ) −8.5862 × 10 2 (7.58 × 10 2 ) ≈4.8873 × 10 2 (3.63 × 10 2 )
LIRCMOP42.6501 × 10 1 (4.89 × 10 2 ) −8.2278 × 10 2 (4.35 × 10 2 ) ≈8.1263 × 10 2 (7.43 × 10 2 ) ≈7.1909 × 10 2 (5.97 × 10 2 )
LIRCMOP52.8041 × 10 1 (7.01 × 10 2 ) −3.8995 × 10 1 (4.33 × 10 1 ) −4.8833 × 10 2 (2.19 × 10 1 ) ≈9.0963 × 10 3 (1.35 × 10 3 )
LIRCMOP62.8170 × 10 1 (8.78 × 10 2 ) −3.2398 × 10 1 (3.75 × 10 1 ) −8.5503 × 10 3 (9.04 × 10 4 ) +1.5276 × 10 2 (2.00 × 10 2 )
LIRCMOP71.1862 × 10 1 (2.65 × 10 2 ) −2.7066 × 10 2 (2.98 × 10 2 ) +1.2701 × 10 1 (3.44 × 10 2 ) −5.9428 × 10 2 (4.55 × 10 2 )
LIRCMOP81.8640 × 10 1 (5.22 × 10 2 ) −1.2245 × 10 2 (1.39 × 10 2 ) ≈1.7233 × 10 1 (6.74 × 10 2 ) −3.2441 × 10 2 (4.54 × 10 2 )
LIRCMOP94.9733 × 10 1 (1.40 × 10 1 ) −2.1397 × 10 1 (9.56 × 10 2 ) ≈4.2851 × 10 1 (1.04 × 10 1 ) −2.2752 × 10 1 (8.98 × 10 2 )
LIRCMOP101.3175 × 10 1 (6.23 × 10 2 ) −3.4225 × 10 2 (6.06 × 10 2 ) +2.6110 × 10 1 (1.13 × 10 1 ) −7.4373 × 10 2 (7.04 × 10 2 )
LIRCMOP116.6407 × 10 2 (3.89 × 10 2 ) −3.2105 × 10 2 (4.79 × 10 2 ) ≈3.2091 × 10 1 (1.99 × 10 1 ) −3.8158 × 10 2 (4.76 × 10 2 )
LIRCMOP122.3432 × 10 1 (9.48 × 10 2 ) −9.2714 × 10 2 (4.22 × 10 2 ) ≈2.1935 × 10 1 (9.31 × 10 2 ) −7.4168 × 10 2 (5.49 × 10 2 )
LIRCMOP139.8279 × 10 2 (1.05 × 10 3 ) ≈1.0697 × 10 1 (1.72 × 10 3 ) −1.2826 × 10 1 (3.72 × 10 3 ) −9.8540 × 10 2 (1.15 × 10 3 )
LIRCMOP141.0055 × 10 1 (1.11 × 10 3 ) +1.0661 × 10 1 (1.53 × 10 3 ) −1.1993 × 10 1 (3.18 × 10 3 ) −1.0417 × 10 1 (3.49 × 10 3 )
× + / / 1/12/12/7/52/8/4
Table A9. Statistical results of HV obtained by ECMOEA and CMOEAs embedded on MW benchmark problems. Best result in each row is highlighted.
Table A9. Statistical results of HV obtained by ECMOEA and CMOEAs embedded on MW benchmark problems. Best result in each row is highlighted.
ProblemCCMOICMAPPSECMOEA
MW14.8992 × 10 1 (6.89 × 10 5 ) +4.8826 × 10 1 (3.86 × 10 4 ) −4.8420 × 10 1 (2.69 × 10 3 ) −4.8916 × 10 1 (1.48 × 10 3 )
MW25.5049 × 10 1 (1.05 × 10 2 ) −5.7928 × 10 1 (1.81 × 10 3 ) +3.9366 × 10 1 (8.32 × 10 2 ) −5.7126 × 10 1 (1.07 × 10 2 )
MW35.4384 × 10 1 (6.38 × 10 4 ) −5.4437 × 10 1 (3.89 × 10 4 ) +5.4280 × 10 1 (5.84 × 10 4 ) −5.4418 × 10 1 (2.99 × 10 4 )
MW48.3987 × 10 1 (4.03 × 10 4 ) +8.3380 × 10 1 (9.96 × 10 4 ) +8.0586 × 10 1 (8.67 × 10 3 ) −8.2426 × 10 1 (1.76 × 10 2 )
MW53.2352 × 10 1 (1.23 × 10 3 ) +3.2030 × 10 1 (1.09 × 10 3 ) +2.0772 × 10 1 (1.11 × 10 1 ) −3.1721 × 10 1 (4.36 × 10 3 )
MW62.9419 × 10 1 (3.62 × 10 2 ) −3.2486 × 10 1 (3.09 × 10 3 ) +1.2218 × 10 1 (1.03 × 10 1 ) −3.1025 × 10 1 (1.47 × 10 2 )
MW74.1210 × 10 1 (3.76 × 10 4 ) ≈4.1098 × 10 1 (6.43 × 10 4 ) −4.1124 × 10 1 (4.36 × 10 4 ) −4.1184 × 10 1 (5.86 × 10 4 )
MW85.2782 × 10 1 (1.76 × 10 2 ) ≈5.1042 × 10 1 (1.18 × 10 2 ) −3.5888 × 10 1 (9.51 × 10 2 ) −5.2682 × 10 1 (1.40 × 10 2 )
MW93.8511 × 10 1 (7.28 × 10 2 ) +3.9409 × 10 1 (1.62 × 10 3 ) +3.1711 × 10 1 (1.29 × 10 1 ) −3.8030 × 10 1 (5.76 × 10 2 )
MW104.1229 × 10 1 (1.86 × 10 2 ) ≈4.4776 × 10 1 (6.45 × 10 3 ) +2.2396 × 10 1 (1.10 × 10 1 ) −4.1951 × 10 1 (1.84 × 10 2 )
MW114.4633 × 10 1 (2.31 × 10 3 ) ≈4.4703 × 10 1 (3.13 × 10 4 ) ≈4.4720 × 10 1 (1.86 × 10 4 ) +4.4708 × 10 1 (2.65 × 10 4 )
MW126.0426 × 10 1 (4.06 × 10 4 ) +6.0387 × 10 1 (2.28 × 10 4 ) ≈5.2215 × 10 1 (1.68 × 10 1 ) −6.0390 × 10 1 (3.86 × 10 4 )
MW134.4745 × 10 1 (1.64 × 10 2 ) −4.7034 × 10 1 (5.09 × 10 3 ) +2.8997 × 10 1 (1.20 × 10 1 ) −4.5707 × 10 1 (1.33 × 10 2 )
MW144.7179 × 10 1 (1.69 × 10 3 ) ≈4.6266 × 10 1 (2.77 × 10 3 ) −4.4302 × 10 1 (8.55 × 10 3 ) −4.7087 × 10 1 (2.64 × 10 3 )
× + / / 5/4/58/4/21/13/0
Table A10. Statistical results of IGD obtained by ECMOEA and CMOEAs embedded on MW benchmark problems. Best result in each row is highlighted.
Table A10. Statistical results of IGD obtained by ECMOEA and CMOEAs embedded on MW benchmark problems. Best result in each row is highlighted.
ProblemCCMOICMAPPSECMOEA
MW11.7903 × 10 3 (1.73 × 10 5 ) +1.9771 × 10 3 (4.97 × 10 5 ) −4.3181 × 10 3 (1.07 × 10 3 ) −1.9580 × 10 3 (5.74 × 10 4 )
MW22.2331 × 10 2 (6.86 × 10 3 ) −6.1170 × 10 3 (1.07 × 10 3 ) +1.4619 × 10 1 (7.71 × 10 2 ) −1.0474 × 10 2 (5.85 × 10 3 )
MW35.2787 × 10 3 (2.84 × 10 4 ) +5.6233 × 10 3 (3.72 × 10 4 ) ≈6.6047 × 10 3 (4.63 × 10 4 ) −5.6406 × 10 3 (3.73 × 10 4 )
MW44.2827 × 10 2 (2.81 × 10 4 ) +4.7768 × 10 2 (9.15 × 10 4 ) ≈6.0627 × 10 2 (3.60 × 10 3 ) −5.1730 × 10 2 (8.68 × 10 3 )
MW52.4211 × 10 3 (2.48 × 10 3 ) +6.1436 × 10 3 (1.64 × 10 3 ) +3.5681 × 10 1 (3.64 × 10 1 ) −9.9150 × 10 3 (5.40 × 10 3 )
MW63.7805 × 10 2 (8.56 × 10 2 ) −4.0045 × 10 3 (1.85 × 10 3 ) +5.1408 × 10 1 (3.99 × 10 1 ) −1.4644 × 10 2 (1.06 × 10 2 )
MW74.9387 × 10 3 (3.71 × 10 4 ) +4.9899 × 10 3 (2.32 × 10 4 ) +6.0677 × 10 3 (4.85 × 10 4 ) −5.4523 × 10 3 (5.51 × 10 4 )
MW84.9523 × 10 2 (5.42 × 10 3 ) ≈5.4728 × 10 2 (4.31 × 10 3 ) −1.3686 × 10 1 (6.60 × 10 2 ) −4.9742 × 10 2 (4.20 × 10 3 )
MW92.8468 × 10 2 (1.29 × 10 1 ) −5.9056 × 10 3 (4.58 × 10 4 ) +1.1796 × 10 1 (2.38 × 10 1 ) −1.7184 × 10 2 (5.27 × 10 2 )
MW104.4783 × 10 2 (2.38 × 10 2 ) ≈7.0461 × 10 3 (3.78 × 10 3 ) +4.0289 × 10 1 (2.38 × 10 1 ) −3.5364 × 10 2 (2.39 × 10 2 )
MW116.5537 × 10 3 (4.78 × 10 4 ) +7.0635 × 10 3 (2.52 × 10 4 ) ≈8.1054 × 10 3 (2.89 × 10 4 ) −7.1478 × 10 3 (4.30 × 10 4 )
MW125.3038 × 10 3 (1.70 × 10 4 ) +5.6372 × 10 3 (2.11 × 10 4 ) ≈9.0221 × 10 2 (1.88 × 10 1 ) −5.5178 × 10 3 (2.48 × 10 4 )
MW136.7224 × 10 2 (3.33 × 10 2 ) −2.0817 × 10 2 (8.66 × 10 3 ) +3.8880 × 10 1 (3.31 × 10 1 ) −4.5883 × 10 2 (2.80 × 10 2 )
MW141.0329 × 10 1 (2.11 × 10 3 ) ≈1.0286 × 10 1 (2.82 × 10 3 ) ≈1.4912 × 10 1 (2.67 × 10 2 ) −1.0303 × 10 1 (1.78 × 10 3 )
× + / / 7/4/37/2/50/14/0
Figure A1. Feasible and non-dominated solution sets obtained by ECMOEA on DAS-CMOP instances with the median IGD value among 30 runs.
Figure A1. Feasible and non-dominated solution sets obtained by ECMOEA on DAS-CMOP instances with the median IGD value among 30 runs.
Symmetry 14 00116 g0a1
Figure A2. Feasible and non-dominated solution sets obtained by ECMOEA on DTLZ instances with the median IGD value among 30 runs.
Figure A2. Feasible and non-dominated solution sets obtained by ECMOEA on DTLZ instances with the median IGD value among 30 runs.
Symmetry 14 00116 g0a2
Figure A3. Feasible and non-dominated solution sets obtained by ECMOEA on FCP instances with the median IGD value among 30 runs.
Figure A3. Feasible and non-dominated solution sets obtained by ECMOEA on FCP instances with the median IGD value among 30 runs.
Symmetry 14 00116 g0a3
Figure A4. Feasible and non-dominated solution sets obtained by ECMOEA on LIR-CMOP instances with the median IGD value among 30 runs.
Figure A4. Feasible and non-dominated solution sets obtained by ECMOEA on LIR-CMOP instances with the median IGD value among 30 runs.
Symmetry 14 00116 g0a4
Figure A5. Feasible and non-dominated solution sets obtained by ECMOEA on MW instances with the median IGD value among 30 runs.
Figure A5. Feasible and non-dominated solution sets obtained by ECMOEA on MW instances with the median IGD value among 30 runs.
Symmetry 14 00116 g0a5
Table A11. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on DAS-CMOP benchmark problems. Best result in each row is highlighted.
Table A11. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on DAS-CMOP benchmark problems. Best result in each row is highlighted.
ProblemCMOEA_MSCTAEADCNSGAIIIMOEADDAENSGAIIToRTiGE2ToPECMOEA
DASCMOP17.4593 × 10 3 (1.21 × 10 2 ) −1.6794 × 10 1 (1.88 × 10 3 ) −1.1349 × 10 2 (1.54 × 10 2 ) −2.2463 × 10 2 (2.70 × 10 2 ) −6.6340 × 10 4 (2.09 × 10 3 ) −2.6584 × 10 2 (3.69 × 10 2 ) −7.7065 × 10 3 (1.15 × 10 2 ) −2.1202 × 10 1 (4.62 × 10 4 )
DASCMOP22.5503 × 10 1 (4.28 × 10 3 ) −3.0341 × 10 1 (8.08 × 10 3 ) −2.4932 × 10 1 (4.39 × 10 3 ) −2.8031 × 10 1 (1.98 × 10 2 ) −1.2786 × 10 1 (9.84 × 10 2 ) −2.6909 × 10 1 (1.18 × 10 2 ) −6.6653 × 10 2 (8.09 × 10 2 ) −3.5504 × 10 1 (7.11 × 10 5 )
DASCMOP32.0854 × 10 1 (1.98 × 10 4 ) −2.4766 × 10 1 (1.38 × 10 2 ) −2.2136 × 10 1 (1.43 × 10 2 ) −2.2492 × 10 1 (3.69 × 10 2 ) −1.0781 × 10 2 (1.15 × 10 2 ) −2.1562 × 10 1 (1.61 × 10 2 ) −1.7942 × 10 2 (1.28 × 10 2 ) −3.1191 × 10 1 (4.13 × 10 4 )
DASCMOP41.8476 × 10 1 (3.65 × 10 2 ) ≈1.9554 × 10 1 (4.94 × 10 3 ) ≈1.9591 × 10 1 (3.33 × 10 3 ) ≈2.0212 × 10 1 (4.67 × 10 3 ) +NaN (NaN)1.9157 × 10 1 (7.48 × 10 3 ) ≈NaN (NaN)1.7485 × 10 1 (4.48 × 10 2 )
DASCMOP53.0368 × 10 1 (9.72 × 10 2 ) −3.4768 × 10 1 (7.89 × 10 4 ) +3.4870 × 10 1 (1.12 × 10 3 ) +3.4936 × 10 1 (2.44 × 10 3 ) ≈NaN (NaN)3.3960 × 10 1 (2.54 × 10 3 ) +NaN (NaN)3.3874 × 10 1 (4.35 × 10 2 )
DASCMOP61.3157 × 10 1 (1.02 × 10 1 ) −3.0726 × 10 1 (1.36 × 10 3 ) ≈2.9766 × 10 1 (4.05 × 10 2 ) ≈2.9354 × 10 1 (2.21 × 10 2 ) ≈NaN (NaN)2.9130 × 10 1 (5.21 × 10 2 ) ≈NaN (NaN)2.4548 × 10 1 (9.70 × 10 2 )
DASCMOP72.8840 × 10 1 (2.27 × 10 4 ) +2.8722 × 10 1 (1.73 × 10 3 ) ≈2.8078 × 10 1 (1.50 × 10 3 ) +2.8473 × 10 1 (1.42 × 10 3 ) +NaN (NaN)2.6270 × 10 1 (8.54 × 10 3 ) −NaN (NaN)2.7886 × 10 1 (1.84 × 10 2 )
DASCMOP82.0721 × 10 1 (3.04 × 10 4 ) +2.0349 × 10 1 (1.33 × 10 3 ) +1.9725 × 10 1 (2.07 × 10 2 ) −2.0293 × 10 1 (2.96 × 10 3 ) +NaN (NaN)1.8381 × 10 1 (7.73 × 10 3 ) −NaN (NaN)1.9989 × 10 1 (1.74 × 10 2 )
DASCMOP91.2340 × 10 1 (1.26 × 10 2 ) −1.4192 × 10 1 (1.00 × 10 2 ) −1.1904 × 10 1 (1.45 × 10 2 ) −1.1041 × 10 1 (3.76 × 10 2 ) −5.7373 × 10 2 (8.76 × 10 3 ) −1.3743 × 10 1 (1.06 × 10 2 ) −6.2156 × 10 2 (2.01 × 10 2 ) −2.0613 × 10 1 (2.33 × 10 4 )
× + / / 2/6/12/4/32/5/23/4/20/4/01/6/20/4/0
Table A12. Statistical results of IGD obtained by ECMOEA and CMOEAs in comparison on DAS-CMOP benchmark problems. Best result in each row is highlighted.
Table A12. Statistical results of IGD obtained by ECMOEA and CMOEAs in comparison on DAS-CMOP benchmark problems. Best result in each row is highlighted.
ProblemCMOEA_MSCTAEADCNSGAIIIMOEADDAENSGAIIToRTiGE2ToPECMOEA
DASCMOP17.4102 × 10 1 (3.70 × 10 2 ) −1.8777 × 10 1 (1.25 × 10 2 ) −7.2850 × 10 1 (2.86 × 10 2 ) −7.1755 × 10 1 (4.09 × 10 2 ) −8.1427 × 10 1 (3.00 × 10 2 ) −6.3691 × 10 1 (1.27 × 10 1 ) −7.5260 × 10 1 (8.61 × 10 2 ) −3.9235 × 10 3 (5.39 × 10 4 )
DASCMOP22.6022 × 10 1 (3.66 × 10 2 ) −9.7215 × 10 2 (2.74 × 10 2 ) −2.8495 × 10 1 (3.71 × 10 2 ) −2.1359 × 10 1 (4.29 × 10 2 ) −5.3228 × 10 1 (2.33 × 10 1 ) −2.0882 × 10 1 (6.58 × 10 2 ) −6.6475 × 10 1 (2.15 × 10 1 ) −4.7686 × 10 3 (1.02 × 10 4 )
DASCMOP33.5417 × 10 1 (3.09 × 10 2 ) −1.6825 × 10 1 (5.36 × 10 2 ) −2.7035 × 10 1 (3.10 × 10 2 ) −2.8789 × 10 1 (1.11 × 10 1 ) −7.9033 × 10 1 (6.08 × 10 2 ) −3.2257 × 10 1 (5.22 × 10 2 ) −7.5129 × 10 1 (5.45 × 10 2 ) −1.9721 × 10 2 (1.18 × 10 3 )
DASCMOP44.6356 × 10 2 (9.92 × 10 2 ) ≈1.2134 × 10 2 (2.38 × 10 3 ) ≈5.6580 × 10 3 (1.32 × 10 3 ) ≈3.1917 × 10 3 (4.35 × 10 3 ) ≈NaN (NaN)2.8533 × 10 2 (1.13 × 10 2 ) ≈NaN (NaN)7.8203 × 10 2 (1.17 × 10 1 )
DASCMOP58.1958 × 10 2 (1.62 × 10 1 ) −7.9797 × 10 3 (8.72 × 10 4 ) +6.9278 × 10 3 (3.15 × 10 3 ) +5.2862 × 10 3 (2.27 × 10 3 ) +NaN (NaN)2.7598 × 10 2 (4.94 × 10 3 ) −NaN (NaN)2.2552 × 10 2 (6.96 × 10 2 )
DASCMOP63.6123 × 10 1 (1.87 × 10 1 ) −2.7557 × 10 2 (3.75 × 10 3 ) ≈5.2111 × 10 2 (8.32 × 10 2 ) ≈6.3304 × 10 2 (5.63 × 10 2 ) ≈NaN (NaN)7.2012 × 10 2 (1.23 × 10 1 ) ≈NaN (NaN)1.5724 × 10 1 (2.12 × 10 1 )
DASCMOP73.2090 × 10 2 (8.48 × 10 4 ) ≈3.8548 × 10 2 (3.74 × 10 3 ) +4.1958 × 10 2 (2.55 × 10 3 ) +3.8577 × 10 2 (3.88 × 10 3 ) +NaN (NaN)1.1142 × 10 1 (2.61 × 10 2 ) −NaN (NaN)5.0821 × 10 2 (3.96 × 10 2 )
DASCMOP84.1598 × 10 2 (8.78 × 10 4 ) +5.5200 × 10 2 (4.60 × 10 3 ) +8.5478 × 10 2 (1.23 × 10 1 ) −5.3687 × 10 2 (7.81 × 10 3 ) +NaN (NaN)1.4476 × 10 1 (3.02 × 10 2 ) −NaN (NaN)5.5965 × 10 2 (3.30 × 10 2 )
DASCMOP94.0198 × 10 1 (7.69 × 10 2 ) −2.5623 × 10 1 (5.11 × 10 2 ) −4.0811 × 10 1 (9.38 × 10 2 ) −4.5442 × 10 1 (1.99 × 10 1 ) −6.9550 × 10 1 (9.10 × 10 2 ) −2.7409 × 10 1 (4.91 × 10 2 ) −6.9131 × 10 1 (1.59 × 10 1 ) −4.3039 × 10 2 (8.96 × 10 4 )
× + / / 1/6/23/4/22/5/23/4/20/4/00/7/20/4/0
Table A13. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on DTLZ benchmark problems. Best result in each row is highlighted.
Table A13. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on DTLZ benchmark problems. Best result in each row is highlighted.
ProblemCMOEA_MSCTAEADCNSGAIIIMOEADDAENSGAIIToRTiGE2ToPECMOEA
C1-DTLZ18.3539 × 10 1 (3.15 × 10 3 ) ≈8.3645 × 10 1 (1.66 × 10 3 ) ≈8.3891 × 10 1 (3.76 × 10 3 ) +8.3977 × 10 1 (2.45 × 10 3 ) +NaN (NaN)2.3202 × 10 1 (1.60 × 10 1 ) −NaN (NaN)8.3117 × 10 1 (9.67 × 10 3 )
C1-DTLZ35.5686 × 10 1 (1.66 × 10 3 ) ≈4.4184 × 10 1 (1.75 × 10 1 ) −2.2320 × 10 1 (2.74 × 10 1 ) −5.5639 × 10 1 (1.64 × 10 3 ) ≈0.0000 × 10 0 (0.00 × 10 0 ) −0.0000 × 10 0 (0.00 × 10 0 ) −2.9100 × 10 1 (2.08 × 10 1 ) −5.5559 × 10 1 (6.92 × 10 3 )
C2-DTLZ25.1551 × 10 1 (1.36 × 10 3 ) +5.0680 × 10 1 (2.18 × 10 3 ) −5.0676 × 10 1 (1.55 × 10 3 ) −5.1515 × 10 1 (1.89 × 10 3 ) +3.1487 × 10 1 (2.50 × 10 2 ) −4.3699 × 10 1 (3.49 × 10 2 ) −4.5579 × 10 1 (3.14 × 10 2 ) −5.1346 × 10 1 (1.64 × 10 3 )
C3-DTLZ44.7153 × 10 1 (6.89 × 10 2 ) −7.8531 × 10 1 (9.95 × 10 4 ) −7.5568 × 10 1 (8.98 × 10 2 ) ≈4.6011 × 10 1 (1.81 × 10 1 ) −4.7327 × 10 1 (1.97 × 10 1 ) −7.6502 × 10 1 (3.30 × 10 3 ) −7.5181 × 10 1 (4.15 × 10 3 ) −7.9059 × 10 1 (8.34 × 10 4 )
DC1-DTLZ16.1513 × 10 1 (2.12 × 10 2 ) −6.2714 × 10 1 (1.01 × 10 3 ) −6.2505 × 10 1 (2.15 × 10 3 ) −6.3229 × 10 1 (5.59 × 10 4 ) +0.0000 × 10 0 (0.00 × 10 0 ) −5.2909 × 10 2 (6.00 × 10 2 ) −5.6523 × 10 1 (2.49 × 10 2 ) −6.3048 × 10 1 (1.77 × 10 3 )
DC1-DTLZ34.3719 × 10 1 (5.51 × 10 2 ) ≈4.6142 × 10 1 (2.28 × 10 3 ) −4.6752 × 10 1 (1.63 × 10 3 ) +4.7401 × 10 1 (9.33 × 10 4 ) +0.0000 × 10 0 (0.00 × 10 0 ) −0.0000 × 10 0 (0.00 × 10 0 ) −7.9346 × 10 2 (1.15 × 10 1 ) −4.6722 × 10 1 (2.48 × 10 2 )
DC2-DTLZ18.2582 × 10 1 (6.47 × 10 2 ) +8.3795 × 10 1 (5.24 × 10 4 ) +6.5774 × 10 1 (2.00 × 10 1 ) ≈8.4078 × 10 1 (3.25 × 10 4 ) +NaN (NaN)2.2905 × 10 1 (7.11 × 10 2 ) −NaN (NaN)8.1760 × 10 1 (7.45 × 10 2 )
DC2-DTLZ31.7637 × 10 1 (2.54 × 10 1 ) ≈2.4441 × 10 1 (2.65 × 10 1 ) −7.3796 × 10 3 (2.09 × 10 3 ) −1.6913 × 10 1 (2.45 × 10 1 ) ≈NaN (NaN)0.0000 × 10 0 (0.00 × 10 0 ) ≈NaN (NaN)4.4170 × 10 1 (1.82 × 10 1 )
DC3-DTLZ14.1023 × 10 1 (4.92 × 10 2 ) −5.2363 × 10 1 (3.01 × 10 3 ) −5.2854 × 10 1 (1.72 × 10 3 ) −4.4886 × 10 1 (1.75 × 10 1 ) ≈0.0000 × 10 0 (0.00 × 10 0 ) −2.6655 × 10 3 (1.46 × 10 2 ) −5.3504 × 10 3 (2.05 × 10 2 ) −5.3397 × 10 1 (2.92 × 10 3 )
DC3-DTLZ32.6430 × 10 1 (1.63 × 10 1 ) +3.5048 × 10 1 (1.94 × 10 2 ) +1.2019 × 10 2 (6.58 × 10 2 ) −1.3624 × 10 1 (1.63 × 10 1 ) ≈0.0000 × 10 0 (0.00 × 10 0 ) −0.0000 × 10 0 (0.00 × 10 0 ) −0.0000 × 10 0 (0.00 × 10 0 ) −1.8319 × 10 1 (1.59 × 10 1 )
× + / / 3/3/42/7/12/6/25/1/40/7/00/9/10/7/0
Table A14. Statistical results of IGD obtained by ECMOEA and CMOEAs in comparison on DTLZ benchmark problems. Best result in each row is highlighted.
Table A14. Statistical results of IGD obtained by ECMOEA and CMOEAs in comparison on DTLZ benchmark problems. Best result in each row is highlighted.
ProblemCMOEA_MSCTAEADCNSGAIIIMOEADDAENSGAIIToRTiGE2ToPECMOEA
C1-DTLZ12.1505 × 10 2 (2.44 × 10 4 ) ≈2.3216 × 10 2 (2.43 × 10 4 ) −2.0411 × 10 2 (1.23 × 10 4 ) +2.1039 × 10 2 (1.11 × 10 4 ) +NaN (NaN)3.3618 × 10 1 (9.78 × 10 2 ) −NaN (NaN)2.2863 × 10 2 (2.55 × 10 3 )
C1-DTLZ35.5983 × 10 2 (5.32 × 10 4 ) +2.6155 × 10 1 (5.64 × 10 1 ) −4.4120 × 10 0 (3.95 × 10 0 ) −6.3937 × 10 2 (4.88 × 10 3 ) −1.5109 × 10 + 1 (3.28 × 10 0 ) −6.4159 × 10 0 (3.02 × 10 0 ) −1.1084 × 10 0 (2.40 × 10 0 ) −5.7235 × 10 2 (3.87 × 10 3 )
C2-DTLZ24.4480 × 10 2 (5.37 × 10 4 ) +5.6530 × 10 2 (1.16 × 10 3 ) −4.8055 × 10 2 (5.12 × 10 4 ) −4.4140 × 10 2 (5.20 × 10 4 ) +2.2981 × 10 1 (4.16 × 10 2 ) −1.1979 × 10 1 (3.76 × 10 2 ) −6.7908 × 10 2 (2.07 × 10 2 ) −4.4994 × 10 2 (4.98 × 10 4 )
C3-DTLZ46.8251 × 10 1 (6.92 × 10 2 ) −1.1156 × 10 1 (1.85 × 10 3 ) −1.9617 × 10 1 (2.58 × 10 1 ) −7.3140 × 10 1 (3.70 × 10 1 ) −7.8185 × 10 1 (4.97 × 10 1 ) −1.9485 × 10 1 (1.21 × 10 2 ) −1.4775 × 10 1 (4.83 × 10 3 ) −1.0285 × 10 1 (1.51 × 10 3 )
DC1-DTLZ11.6840 × 10 2 (5.92 × 10 3 ) −1.5336 × 10 2 (2.71 × 10 4 ) −1.3772 × 10 2 (1.88 × 10 4 ) −1.2079 × 10 2 (7.48 × 10 5 ) +1.3100 × 10 + 1 (5.80 × 10 0 ) −6.0613 × 10 1 (2.79 × 10 1 ) −2.6383 × 10 2 (6.41 × 10 3 ) −1.2178 × 10 2 (1.84 × 10 4 )
DC1-DTLZ31.3150 × 10 1 (1.49 × 10 1 ) ≈4.3320 × 10 2 (1.29 × 10 3 ) +4.6115 × 10 2 (8.78 × 10 4 ) +3.6595 × 10 2 (1.13 × 10 3 ) +1.3494 × 10 + 1 (2.79 × 10 0 ) −2.2495 × 10 0 (6.97 × 10 1 ) −1.3916 × 10 0 (1.48 × 10 0 ) −4.6968 × 10 2 (6.10 × 10 2 )
DC2-DTLZ12.6494 × 10 2 (2.59 × 10 2 ) +2.3194 × 10 2 (1.98 × 10 4 ) +9.2959 × 10 2 (7.91 × 10 2 ) ≈2.1173 × 10 2 (1.58 × 10 4 ) +NaN (NaN)3.4202 × 10 1 (5.38 × 10 2 ) −NaN (NaN)2.8151 × 10 2 (2.73 × 10 2 )
DC2-DTLZ34.1012 × 10 1 (2.36 × 10 1 ) ≈3.4190 × 10 1 (2.51 × 10 1 ) −5.8320 × 10 1 (7.08 × 10 2 ) −4.1457 × 10 1 (2.31 × 10 1 ) −NaN (NaN)7.1406 × 10 1 (0.00 × 10 0 ) ≈NaN (NaN)1.4449 × 10 1 (1.73 × 10 1 )
DC3-DTLZ13.4208 × 10 2 (1.05 × 10 2 ) −9.4040 × 10 3 (2.48 × 10 4 ) −1.0045 × 10 2 (3.07 × 10 4 ) −3.9794 × 10 2 (6.57 × 10 2 ) −7.7910 × 10 + 1 (3.62 × 10 + 1 ) −1.6313 × 10 0 (1.09 × 10 0 ) −3.7754 × 10 0 (4.04 × 10 0 ) −7.3281 × 10 3 (1.26 × 10 4 )
DC3-DTLZ32.3006 × 10 1 (3.88 × 10 1 ) +3.2940 × 10 2 (1.63 × 10 2 ) +6.0756 × 10 1 (2.30 × 10 1 ) −6.0916 × 10 1 (6.04 × 10 1 ) ≈5.4843 × 10 + 1 (1.24 × 10 + 1 ) −4.2603 × 10 0 (1.26 × 10 0 ) −8.7643 × 10 0 (3.29 × 10 0 ) −2.6746 × 10 1 (2.54 × 10 1 )
× + / / 4/3/33/7/02/7/15/4/10/7/00/9/10/7/0
Table A15. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on FCP benchmark problems. Best result in each row is highlighted.
Table A15. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on FCP benchmark problems. Best result in each row is highlighted.
ProblemCMOEA_MSCTAEADCNSGAIIIMOEADDAENSGAIIToRTiGE2ToPECMOEA
FCP1NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)5.8083 × 10 1 (1.37 × 10 4 )
FCP2NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)4.3120 × 10 1 (1.92 × 10 4 )
FCP3NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)3.4653 × 10 1 (1.48 × 10 4 )
FCP4NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)6.3374 × 10 1 (4.57 × 10 4 )
FCP51.4401 × 10 1 (1.05 × 10 1 ) −2.2982 × 10 1 (2.37 × 10 2 ) −NaN (NaN)2.4344 × 10 1 (3.14 × 10 2 ) −3.8618 × 10 2 (8.25 × 10 3 ) −2.5879 × 10 1 (1.90 × 10 3 ) −5.3780 × 10 2 (5.24 × 10 5 ) −4.2380 × 10 1 (9.07 × 10 2 )
× + / / 0/1/00/1/00/0/00/1/00/1/00/1/00/1/0
Table A16. Statistical results of IGD obtained by ECMOEA and CMOEAs in comparison on FCP benchmark problems. Best result in each row is highlighted.
Table A16. Statistical results of IGD obtained by ECMOEA and CMOEAs in comparison on FCP benchmark problems. Best result in each row is highlighted.
ProblemCMOEA_MSCTAEADCNSGAIIIMOEADDAENSGAIIToRTiGE2ToPECMOEA
FCP1NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)3.9006 × 10 2 (7.10 × 10 4 )
FCP2NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)3.9383 × 10 2 (1.03 × 10 2 )
FCP3NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)4.2828 × 10 2 (1.24 × 10 3 )
FCP4NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)NaN (NaN)3.0694 × 10 2 (1.50 × 10 3 )
FCP54.4497 × 10 0 (2.03 × 10 1 ) −4.7115 × 10 0 (2.44 × 10 2 ) −NaN (NaN)4.7154 × 10 0 (4.20 × 10 2 ) −4.4003 × 10 0 (5.41 × 10 2 ) −4.7030 × 10 0 (9.14 × 10 3 ) −4.2664 × 10 0 (6.63 × 10 3 ) −1.1119 × 10 0 (2.00 × 10 0 )
× + / / 0/1/00/1/00/0/00/1/00/1/00/1/00/1/0
Table A17. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on LIR-CMOP benchmark problems. Best result in each row is highlighted.
Table A17. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on LIR-CMOP benchmark problems. Best result in each row is highlighted.
ProblemCMOEA_MSCTAEADCNSGAIIIMOEADDAENSGAIIToRTiGE2ToPECMOEA
LIRCMOP11.0115 × 10 1 (1.19 × 10 2 ) −1.1848 × 10 1 (2.50 × 10 2 ) −1.4984 × 10 1 (9.90 × 10 3 ) −1.5432 × 10 1 (1.73 × 10 2 ) −9.5614 × 10 2 (5.70 × 10 3 ) −1.3806 × 10 1 (6.46 × 10 3 ) −1.0697 × 10 1 (9.48 × 10 3 ) −2.2184 × 10 1 (1.41 × 10 2 )
LIRCMOP22.1842 × 10 1 (1.55 × 10 2 ) −2.6877 × 10 1 (3.56 × 10 2 ) −2.7303 × 10 1 (9.13 × 10 3 ) −2.8306 × 10 1 (1.56 × 10 2 ) −2.0501 × 10 1 (8.62 × 10 3 ) −2.6404 × 10 1 (8.71 × 10 3 ) −2.1595 × 10 1 (1.13 × 10 2 ) −3.5079 × 10 1 (7.94 × 10 3 )
LIRCMOP39.5727 × 10 2 (1.27 × 10 2 ) −9.9812 × 10 2 (1.86 × 10 2 ) −1.3522 × 10 1 (1.16 × 10 2 ) −1.4090 × 10 1 (1.76 × 10 2 ) −8.8113 × 10 2 (4.73 × 10 3 ) −1.2159 × 10 1 (1.02 × 10 2 ) −9.1838 × 10 2 (5.94 × 10 3 ) −1.8854 × 10 1 (1.36 × 10 2 )
LIRCMOP41.9142 × 10 1 (1.59 × 10 2 ) −1.8915 × 10 1 (2.48 × 10 2 ) −2.3185 × 10 1 (1.32 × 10 2 ) −2.3742 × 10 1 (1.66 × 10 2 ) −1.7744 × 10 1 (8.34 × 10 3 ) −2.2264 × 10 1 (1.37 × 10 2 ) −1.8521 × 10 1 (8.38 × 10 3 ) −2.8486 × 10 1 (2.73 × 10 2 )
LIRCMOP51.3969 × 10 1 (4.27 × 10 2 ) −2.2548 × 10 2 (5.29 × 10 2 ) −1.0042 × 10 1 (7.35 × 10 2 ) −1.4195 × 10 1 (1.62 × 10 2 ) −0.0000 × 10 0 (0.00 × 10 0 ) −1.4329 × 10 2 (4.39 × 10 2 ) −3.2396 × 10 3 (1.77 × 10 2 ) −2.8993 × 10 1 (1.09 × 10 3 )
LIRCMOP68.7194 × 10 2 (3.59 × 10 2 ) −2.7802 × 10 3 (1.52 × 10 2 ) −6.0182 × 10 2 (5.03 × 10 2 ) −1.0163 × 10 1 (8.32 × 10 3 ) −0.0000 × 10 0 (0.00 × 10 0 ) −1.9501 × 10 2 (3.35 × 10 2 ) −1.8838 × 10 3 (1.03 × 10 2 ) −1.9391 × 10 1 (6.44 × 10 3 )
LIRCMOP72.3623 × 10 1 (4.56 × 10 2 ) −2.3310 × 10 1 (4.53 × 10 2 ) −2.4779 × 10 1 (5.93 × 10 3 ) −2.3852 × 10 1 (7.38 × 10 3 ) −0.0000 × 10 0 (0.00 × 10 0 ) −1.9448 × 10 1 (3.27 × 10 2 ) −2.9611 × 10 2 (7.80 × 10 2 ) −2.7155 × 10 1 (1.91 × 10 2 )
LIRCMOP82.2770 × 10 1 (7.63 × 10 3 ) −1.2629 × 10 1 (1.01 × 10 1 ) −2.2658 × 10 1 (4.24 × 10 3 ) −2.2320 × 10 1 (5.27 × 10 3 ) −0.0000 × 10 0 (0.00 × 10 0 ) −1.7051 × 10 1 (5.00 × 10 2 ) −3.6019 × 10 2 (7.53 × 10 2 ) −2.8425 × 10 1 (1.70 × 10 2 )
LIRCMOP92.9951 × 10 1 (7.05 × 10 2 ) −3.5948 × 10 1 (5.49 × 10 2 ) −2.6570 × 10 1 (7.27 × 10 2 ) −4.8049 × 10 1 (6.35 × 10 2 ) +3.7327 × 10 2 (1.14 × 10 2 ) −2.2646 × 10 1 (9.34 × 10 2 ) −3.0712 × 10 1 (7.43 × 10 2 ) −4.7741 × 10 1 (2.89 × 10 2 )
LIRCMOP104.6143 × 10 1 (8.76 × 10 2 ) −5.1822 × 10 1 (2.12 × 10 2 ) −3.7586 × 10 1 (1.93 × 10 1 ) −6.3167 × 10 1 (1.23 × 10 1 ) ≈7.3311 × 10 2 (2.62 × 10 2 ) −1.2391 × 10 1 (4.61 × 10 2 ) −4.6129 × 10 1 (8.92 × 10 2 ) −6.7585 × 10 1 (3.22 × 10 2 )
LIRCMOP114.9685 × 10 1 (1.09 × 10 1 ) −5.9640 × 10 1 (2.79 × 10 2 ) −5.4404 × 10 1 (9.82 × 10 2 ) −5.9932 × 10 1 (1.33 × 10 1 ) −6.4728 × 10 2 (3.41 × 10 2 ) −1.9026 × 10 1 (1.06 × 10 1 ) −4.0354 × 10 1 (8.82 × 10 2 ) −6.7327 × 10 1 (2.91 × 10 2 )
LIRCMOP124.5963 × 10 1 (4.49 × 10 2 ) −4.9804 × 10 1 (4.56 × 10 2 ) −4.7283 × 10 1 (4.22 × 10 2 ) −5.8668 × 10 1 (2.83 × 10 2 ) ≈7.8454 × 10 2 (1.76 × 10 2 ) −3.5124 × 10 1 (5.79 × 10 2 ) −4.6303 × 10 1 (4.34 × 10 2 ) −5.8864 × 10 1 (2.42 × 10 2 )
LIRCMOP135.5352 × 10 1 (1.32 × 10 3 ) +5.4607 × 10 1 (1.34 × 10 3 ) −4.1686 × 10 4 (7.50 × 10 5 ) −5.5500 × 10 1 (1.30 × 10 3 ) +4.3729 × 10 6 (2.40 × 10 5 ) −1.7644 × 10 5 (7.74 × 10 5 ) −2.0082 × 10 3 (1.08 × 10 2 ) −5.5170 × 10 1 (1.14 × 10 3 )
LIRCMOP145.5298 × 10 1 (1.23 × 10 3 ) +5.4605 × 10 1 (7.72 × 10 4 ) −9.4256 × 10 4 (6.07 × 10 5 ) −5.5319 × 10 1 (1.09 × 10 3 ) +3.3677 × 10 5 (1.70 × 10 4 ) −3.8556 × 10 2 (8.91 × 10 2 ) −1.1349 × 10 2 (3.03 × 10 2 ) −5.5091 × 10 1 (1.62 × 10 3 )
× + / / 2/12/00/14/00/14/03/9/20/14/00/14/00/14/0
Table A18. Statistical results of IGD obtained by ECMOEA and CMOEAs in comparison on LIR-CMOP benchmark problems. Best result in each row is highlighted.
Table A18. Statistical results of IGD obtained by ECMOEA and CMOEAs in comparison on LIR-CMOP benchmark problems. Best result in each row is highlighted.
ProblemCMOEA_MSCTAEADCNSGAIIIMOEADDAENSGAIIToRTiGE2ToPECMOEA
LIRCMOP13.4677 × 10 1 (4.82 × 10 2 ) −3.0267 × 10 1 (9.72 × 10 2 ) −1.8277 × 10 1 (2.38 × 10 2 ) −1.7492 × 10 1 (3.87 × 10 2 ) −3.6403 × 10 1 (9.72 × 10 3 ) −2.1918 × 10 1 (1.78 × 10 2 ) −3.2074 × 10 1 (2.00 × 10 2 ) −4.0024 × 10 2 (3.57 × 10 2 )
LIRCMOP22.8670 × 10 1 (3.19 × 10 2 ) −2.1030 × 10 1 (8.58 × 10 2 ) −1.8254 × 10 1 (2.00 × 10 2 ) −1.6174 × 10 1 (4.41 × 10 2 ) −3.2103 × 10 1 (7.43 × 10 3 ) −1.9561 × 10 1 (1.77 × 10 2 ) −2.8707 × 10 1 (1.13 × 10 2 ) −2.5367 × 10 2 (1.84 × 10 2 )
LIRCMOP33.2784 × 10 1 (4.29 × 10 2 ) −3.2563 × 10 1 (8.99 × 10 2 ) −1.8986 × 10 1 (3.58 × 10 2 ) −1.8257 × 10 1 (4.71 × 10 2 ) −3.5146 × 10 1 (7.84 × 10 3 ) −2.3604 × 10 1 (2.85 × 10 2 ) −3.4335 × 10 1 (1.38 × 10 2 ) −4.8873 × 10 2 (3.63 × 10 2 )
LIRCMOP43.0059 × 10 1 (3.46 × 10 2 ) −3.2288 × 10 1 (7.81 × 10 2 ) −2.1136 × 10 1 (2.83 × 10 2 ) −1.9991 × 10 1 (3.76 × 10 2 ) −3.2496 × 10 1 (6.76 × 10 3 ) −2.1999 × 10 1 (2.44 × 10 2 ) −3.0962 × 10 1 (1.05 × 10 2 ) −7.1909 × 10 2 (5.97 × 10 2 )
LIRCMOP53.7935 × 10 1 (2.29 × 10 1 ) −1.0849 × 10 0 (3.32 × 10 1 ) −6.1546 × 10 1 (4.40 × 10 1 ) −3.3377 × 10 1 (3.33 × 10 2 ) −2.6644 × 10 0 (5.86 × 10 2 ) −1.1355 × 10 0 (2.67 × 10 1 ) −1.1825 × 10 0 (9.90 × 10 2 ) −9.0963 × 10 3 (1.35 × 10 3 )
LIRCMOP65.0145 × 10 1 (3.50 × 10 1 ) −1.3149 × 10 0 (1.72 × 10 1 ) −7.5519 × 10 1 (4.95 × 10 1 ) −3.6595 × 10 1 (6.89 × 10 2 ) −2.8305 × 10 0 (2.96 × 10 1 ) −1.1605 × 10 0 (3.43 × 10 1 ) −1.3224 × 10 0 (1.35 × 10 1 ) −1.5276 × 10 2 (2.00 × 10 2 )
LIRCMOP71.8769 × 10 1 (2.84 × 10 1 ) −1.9967 × 10 1 (2.83 × 10 1 ) −1.1912 × 10 1 (2.03 × 10 2 ) −1.4512 × 10 1 (2.65 × 10 2 ) −2.9434 × 10 0 (6.40 × 10 1 ) −3.3566 × 10 1 (1.45 × 10 1 ) −1.4870 × 10 0 (5.13 × 10 1 ) −5.9428 × 10 2 (4.55 × 10 2 )
LIRCMOP82.1669 × 10 1 (4.42 × 10 2 ) −8.5644 × 10 1 (6.60 × 10 1 ) −2.1996 × 10 1 (3.13 × 10 2 ) −2.2209 × 10 1 (3.22 × 10 2 ) −3.0160 × 10 0 (6.92 × 10 1 ) −5.2013 × 10 1 (3.45 × 10 1 ) −1.4431 × 10 0 (5.04 × 10 1 ) −3.2441 × 10 2 (4.54 × 10 2 )
LIRCMOP96.3197 × 10 1 (1.12 × 10 1 ) −4.8471 × 10 1 (9.91 × 10 2 ) −7.8136 × 10 1 (1.48 × 10 1 ) −2.5027 × 10 1 (1.51 × 10 1 ) ≈1.2696 × 10 0 (5.72 × 10 2 ) −7.7593 × 10 1 (1.94 × 10 1 ) −5.6230 × 10 1 (1.20 × 10 1 ) −2.2752 × 10 1 (8.98 × 10 2 )
LIRCMOP104.3915 × 10 1 (9.85 × 10 2 ) −3.4745 × 10 1 (3.97 × 10 2 ) −4.5050 × 10 1 (2.81 × 10 1 ) −1.1462 × 10 1 (1.90 × 10 1 ) ≈1.0588 × 10 0 (6.48 × 10 2 ) −9.6528 × 10 1 (1.86 × 10 1 ) −4.3900 × 10 1 (8.54 × 10 2 ) −7.4373 × 10 2 (7.04 × 10 2 )
LIRCMOP113.2899 × 10 1 (1.41 × 10 1 ) −2.3137 × 10 1 (5.53 × 10 2 ) −2.7798 × 10 1 (1.59 × 10 1 ) −2.3136 × 10 1 (3.29 × 10 1 ) −1.3025 × 10 0 (2.02 × 10 1 ) −8.4857 × 10 1 (2.98 × 10 1 ) −4.4492 × 10 1 (1.23 × 10 1 ) −3.8158 × 10 2 (4.76 × 10 2 )
LIRCMOP123.4010 × 10 1 (9.10 × 10 2 ) −2.4646 × 10 1 (1.18 × 10 1 ) −4.0424 × 10 1 (1.64 × 10 1 ) −9.9434 × 10 2 (1.24 × 10 1 ) ≈1.3856 × 10 0 (7.24 × 10 2 ) −5.3323 × 10 1 (1.34 × 10 1 ) −3.1937 × 10 1 (8.51 × 10 2 ) −7.4168 × 10 2 (5.49 × 10 2 )
LIRCMOP139.7025 × 10 2 (8.88 × 10 4 ) +1.0935 × 10 1 (2.26 × 10 3 ) −1.3068 × 10 0 (8.78 × 10 4 ) −9.7361 × 10 2 (8.88 × 10 4 ) +1.5044 × 10 0 (3.85 × 10 2 ) −1.3954 × 10 0 (2.04 × 10 2 ) −1.3376 × 10 0 (6.20 × 10 2 ) −9.8540 × 10 2 (1.15 × 10 3 )
LIRCMOP149.9381 × 10 2 (9.46 × 10 4 ) +1.1116 × 10 1 (1.19 × 10 3 ) −1.2631 × 10 0 (8.75 × 10 4 ) −9.9770 × 10 2 (6.42 × 10 4 ) +1.4943 × 10 0 (4.78 × 10 2 ) −1.2359 × 10 0 (3.01 × 10 1 ) −1.2611 × 10 0 (1.25 × 10 1 ) −1.0417 × 10 1 (3.49 × 10 3 )
× + / / 2/12/00/14/00/14/02/9/30/14/00/14/00/14/0
Table A19. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on MW benchmark problems. Best result in each row is highlighted.
Table A19. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on MW benchmark problems. Best result in each row is highlighted.
ProblemCMOEA_MSCTAEADCNSGAIIIMOEADDAENSGAIIToRTiGE2ToPECMOEA
MW14.8676 × 10 1 (5.96 × 10 3 ) ≈4.8748 × 10 1 (3.16 × 10 3 ) −4.8869 × 10 1 (2.21 × 10 4 ) −4.8443 × 10 1 (2.36 × 10 2 ) ≈1.7502 × 10 2 (3.91 × 10 2 ) −4.3587 × 10 1 (4.58 × 10 2 ) −NaN (NaN)4.8916 × 10 1 (1.48 × 10 3 )
MW25.4292 × 10 1 (1.89 × 10 2 ) −5.5433 × 10 1 (1.36 × 10 2 ) −5.5963 × 10 1 (1.33 × 10 2 ) −5.1308 × 10 1 (3.59 × 10 2 ) −4.1775 × 10 1 (7.39 × 10 2 ) −5.2809 × 10 1 (1.34 × 10 2 ) −3.7255 × 10 1 (1.50 × 10 1 ) −5.7126 × 10 1 (1.07 × 10 2 )
MW35.4345 × 10 1 (6.46 × 10 4 ) −5.4417 × 10 1 (6.39 × 10 4 ) ≈5.4419 × 10 1 (5.93 × 10 4 ) ≈5.4329 × 10 1 (5.46 × 10 4 ) −2.6028 × 10 1 (1.19 × 10 1 ) −5.2152 × 10 1 (2.26 × 10 2 ) −2.1977 × 10 1 (1.88 × 10 1 ) −5.4418 × 10 1 (2.99 × 10 4 )
MW48.3732 × 10 1 (8.52 × 10 4 ) +8.3815 × 10 1 (2.25 × 10 4 ) +8.4122 × 10 1 (3.24 × 10 4 ) +8.3998 × 10 1 (4.49 × 10 4 ) +2.9498 × 10 1 (8.83 × 10 2 ) −7.5894 × 10 1 (4.59 × 10 2 ) −NaN (NaN)8.2426 × 10 1 (1.76 × 10 2 )
MW52.8635 × 10 1 (3.85 × 10 2 ) −3.1594 × 10 1 (1.87 × 10 3 ) ≈3.0727 × 10 1 (2.57 × 10 2 ) −3.2158 × 10 1 (4.72 × 10 3 ) +3.4284 × 10 4 (1.14 × 10 3 ) −2.8655 × 10 1 (1.22 × 10 2 ) −3.7421 × 10 2 (2.53 × 10 2 ) −3.1721 × 10 1 (4.36 × 10 3 )
MW62.9857 × 10 1 (1.41 × 10 2 ) −3.1097 × 10 1 (9.79 × 10 3 ) ≈3.0777 × 10 1 (2.03 × 10 2 ) ≈2.6193 × 10 1 (4.41 × 10 2 ) −1.3821 × 10 1 (4.00 × 10 2 ) −2.0971 × 10 1 (8.06 × 10 2 ) −5.4799 × 10 2 (6.31 × 10 2 ) −3.1025 × 10 1 (1.47 × 10 2 )
MW74.0729 × 10 1 (4.03 × 10 3 ) −4.0863 × 10 1 (8.45 × 10 4 ) −4.1127 × 10 1 (4.34 × 10 4 ) −4.1184 × 10 1 (3.67 × 10 4 ) ≈2.6471 × 10 1 (6.16 × 10 2 ) −3.8964 × 10 1 (4.43 × 10 3 ) −3.4344 × 10 1 (7.72 × 10 2 ) −4.1184 × 10 1 (5.86 × 10 4 )
MW85.2026 × 10 1 (1.47 × 10 2 ) ≈5.2489 × 10 1 (1.04 × 10 2 ) ≈5.3028 × 10 1 (1.08 × 10 2 ) ≈4.9523 × 10 1 (5.01 × 10 2 ) −3.0869 × 10 1 (6.99 × 10 2 ) −3.5317 × 10 1 (1.45 × 10 1 ) −8.6399 × 10 2 (1.01 × 10 1 ) −5.2682 × 10 1 (1.40 × 10 2 )
MW92.3871 × 10 1 (1.75 × 10 1 ) −3.9125 × 10 1 (2.03 × 10 3 ) ≈3.8276 × 10 1 (5.89 × 10 3 ) +3.9811 × 10 1 (3.71 × 10 3 ) +1.0307 × 10 1 (5.80 × 10 2 ) −2.9342 × 10 1 (1.36 × 10 1 ) −0.0000 × 10 0 (0.00 × 10 0 ) ≈3.8030 × 10 1 (5.76 × 10 2 )
MW104.0682 × 10 1 (2.53 × 10 2 ) −4.3743 × 10 1 (1.56 × 10 2 ) +4.2341 × 10 1 (1.83 × 10 2 ) ≈3.6348 × 10 1 (7.40 × 10 2 ) −2.4674 × 10 1 (8.80 × 10 2 ) −3.7718 × 10 1 (5.21 × 10 2 ) −NaN (NaN)4.1951 × 10 1 (1.84 × 10 2 )
MW114.2975 × 10 1 (5.16 × 10 2 ) ≈4.4139 × 10 1 (1.35 × 10 3 ) −4.4215 × 10 1 (3.73 × 10 3 ) −4.4494 × 10 1 (5.57 × 10 4 ) −2.5121 × 10 1 (5.09 × 10 3 ) −4.3473 × 10 1 (2.57 × 10 3 ) −2.7045 × 10 1 (2.93 × 10 2 ) −4.4708 × 10 1 (2.65 × 10 4 )
MW126.0336 × 10 1 (5.03 × 10 4 ) −6.0006 × 10 1 (7.21 × 10 4 ) −5.8159 × 10 1 (1.07 × 10 1 ) −4.8591 × 10 1 (2.39 × 10 1 ) −1.3563 × 10 2 (3.42 × 10 2 ) −4.9320 × 10 1 (1.98 × 10 1 ) −2.7375 × 10 4 (6.71 × 10 4 ) −6.0390 × 10 1 (3.86 × 10 4 )
MW134.3660 × 10 1 (1.71 × 10 2 ) −4.5761 × 10 1 (1.49 × 10 2 ) ≈4.4369 × 10 1 (1.22 × 10 2 ) −4.1742 × 10 1 (3.06 × 10 2 ) −2.0166 × 10 1 (4.79 × 10 2 ) −3.1646 × 10 1 (3.37 × 10 2 ) −1.7326 × 10 1 (1.11 × 10 1 ) −4.5707 × 10 1 (1.33 × 10 2 )
MW144.6322 × 10 1 (5.06 × 10 3 ) −4.6632 × 10 1 (2.76 × 10 3 ) −4.6276 × 10 1 (8.80 × 10 3 ) −4.7451 × 10 1 (1.63 × 10 3 ) +3.9445 × 10 2 (7.44 × 10 3 ) −4.4657 × 10 1 (4.74 × 10 3 ) −3.2108 × 10 1 (1.68 × 10 1 ) −4.7087 × 10 1 (2.64 × 10 3 )
× + / / 1/10/32/6/62/8/44/8/20/14/00/14/00/10/1
Table A20. Statistical results of IGD obtained by ECMOEA and CMOEAs in comparison on MW benchmark problems. Best result in each row is highlighted.
Table A20. Statistical results of IGD obtained by ECMOEA and CMOEAs in comparison on MW benchmark problems. Best result in each row is highlighted.
ProblemCMOEA_MSCTAEADCNSGAIIIMOEADDAENSGAIIToRTiGE2ToPECMOEA
MW13.4592 × 10 3 (3.10 × 10 3 ) −2.7882 × 10 3 (1.62 × 10 3 ) −2.5748 × 10 3 (9.80 × 10 5 ) −7.3274 × 10 3 (2.50 × 10 2 ) −6.5822 × 10 1 (1.25 × 10 1 ) −5.2793 × 10 2 (5.54 × 10 2 ) −NaN (NaN)1.9580 × 10 3 (5.74 × 10 4 )
MW22.7738 × 10 2 (1.24 × 10 2 ) −2.0019 × 10 2 (8.16 × 10 3 ) −1.6904 × 10 2 (7.66 × 10 3 ) −4.8670 × 10 2 (2.65 × 10 2 ) −1.4076 × 10 1 (9.48 × 10 2 ) −3.8754 × 10 2 (1.23 × 10 2 ) −1.8214 × 10 1 (1.80 × 10 1 ) −1.0474 × 10 2 (5.85 × 10 3 )
MW35.9683 × 10 3 (3.37 × 10 4 ) −5.5917 × 10 3 (4.27 × 10 4 ) ≈6.9186 × 10 3 (8.62 × 10 4 ) −5.8576 × 10 3 (2.87 × 10 4 ) −3.7038 × 10 1 (2.40 × 10 1 ) −3.5421 × 10 2 (3.43 × 10 2 ) −4.9445 × 10 1 (3.59 × 10 1 ) −5.6406 × 10 3 (3.73 × 10 4 )
MW44.3942 × 10 2 (6.27 × 10 4 ) +4.6598 × 10 2 (3.30 × 10 4 ) +4.1681 × 10 2 (4.31 × 10 4 ) +4.2948 × 10 2 (4.38 × 10 4 ) +4.5506 × 10 1 (1.25 × 10 1 ) −1.1416 × 10 1 (5.25 × 10 2 ) −NaN (NaN)5.1730 × 10 2 (8.68 × 10 3 )
MW57.3763 × 10 2 (8.91 × 10 2 ) −1.4026 × 10 2 (2.86 × 10 3 ) −3.9050 × 10 2 (9.23 × 10 2 ) −4.8949 × 10 3 (6.58 × 10 3 ) +7.7006 × 10 1 (1.43 × 10 1 ) −5.9433 × 10 2 (1.75 × 10 2 ) −3.7858 × 10 1 (2.91 × 10 2 ) −9.9150 × 10 3 (5.40 × 10 3 )
MW62.3006 × 10 2 (1.04 × 10 2 ) −1.2619 × 10 2 (6.48 × 10 3 ) ≈1.6373 × 10 2 (1.52 × 10 2 ) ≈7.5654 × 10 2 (1.16 × 10 1 ) −5.1466 × 10 1 (1.85 × 10 1 ) −3.0101 × 10 1 (3.38 × 10 1 ) −7.7661 × 10 1 (2.64 × 10 1 ) −1.4644 × 10 2 (1.06 × 10 2 )
MW73.2723 × 10 2 (2.33 × 10 2 ) ≈7.5084 × 10 3 (5.13 × 10 4 ) −6.4451 × 10 3 (8.00 × 10 4 ) −5.1806 × 10 3 (2.58 × 10 4 ) ≈3.3723 × 10 1 (1.92 × 10 1 ) −4.1832 × 10 2 (1.97 × 10 2 ) −1.2048 × 10 1 (1.92 × 10 1 ) −5.4523 × 10 3 (5.51 × 10 4 )
MW85.1268 × 10 2 (5.06 × 10 3 ) ≈5.3522 × 10 2 (2.24 × 10 3 ) −4.9775 × 10 2 (2.06 × 10 3 ) ≈6.1235 × 10 2 (2.47 × 10 2 ) −2.7568 × 10 1 (1.39 × 10 1 ) −2.7440 × 10 1 (2.22 × 10 1 ) −8.0288 × 10 1 (2.81 × 10 1 ) −4.9742 × 10 2 (4.20 × 10 3 )
MW92.3976 × 10 1 (2.97 × 10 1 ) −9.2113 × 10 3 (7.71 × 10 4 ) +1.4877 × 10 2 (6.06 × 10 3 ) +5.4641 × 10 3 (1.57 × 10 3 ) +4.6529 × 10 1 (1.70 × 10 1 ) −1.5226 × 10 1 (2.43 × 10 1 ) −7.7575 × 10 1 (0.00 × 10 0 ) ≈1.7184 × 10 2 (5.27 × 10 2 )
MW105.3197 × 10 2 (3.56 × 10 2 ) −1.6739 × 10 2 (1.59 × 10 2 ) +3.1099 × 10 2 (2.09 × 10 2 ) ≈1.2860 × 10 1 (1.47 × 10 1 ) −3.4714 × 10 1 (2.00 × 10 1 ) −9.2950 × 10 2 (1.04 × 10 1 ) −NaN (NaN)3.5364 × 10 2 (2.39 × 10 2 )
MW116.9188 × 10 2 (1.93 × 10 1 ) −1.6156 × 10 2 (2.28 × 10 3 ) −9.3973 × 10 3 (1.65 × 10 3 ) −7.4640 × 10 3 (6.78 × 10 4 ) −7.9454 × 10 1 (3.20 × 10 2 ) −3.2458 × 10 2 (6.61 × 10 3 ) −7.0755 × 10 1 (1.40 × 10 1 ) −7.1478 × 10 3 (4.30 × 10 4 )
MW125.8325 × 10 3 (3.40 × 10 4 ) −8.4772 × 10 3 (5.85 × 10 4 ) −2.9728 × 10 2 (1.22 × 10 1 ) −1.4498 × 10 1 (2.84 × 10 1 ) −8.4008 × 10 1 (1.70 × 10 1 ) −1.3855 × 10 1 (2.57 × 10 1 ) −8.4144 × 10 1 (6.38 × 10 2 ) −5.5178 × 10 3 (2.48 × 10 4 )
MW139.5558 × 10 2 (3.31 × 10 2 ) −4.7509 × 10 2 (3.13 × 10 2 ) ≈8.6314 × 10 2 (3.03 × 10 2 ) −1.1894 × 10 1 (4.56 × 10 2 ) −1.1058 × 10 0 (4.09 × 10 1 ) −5.0790 × 10 1 (1.40 × 10 1 ) −1.0100 × 10 0 (4.70 × 10 1 ) −4.5883 × 10 2 (2.80 × 10 2 )
MW141.3855 × 10 1 (2.53 × 10 2 ) −1.1137 × 10 1 (4.30 × 10 3 ) −1.3632 × 10 1 (2.44 × 10 2 ) −1.0170 × 10 1 (2.48 × 10 3 ) +1.3620 × 10 0 (1.99 × 10 1 ) −1.6612 × 10 1 (9.36 × 10 3 ) −4.7373 × 10 1 (4.85 × 10 1 ) −1.0303 × 10 1 (1.78 × 10 3 )
× + / / 1/11/23/8/32/9/34/9/10/14/00/14/00/10/1
Table A21. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on real-world application problems. Best result in each row is highlighted.
Table A21. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on real-world application problems. Best result in each row is highlighted.
ProblemICMACCMOPPSCMOEA_MSCTAEADCNSGAIIIMOEADDAENSGAIIToRTiGE_2ToPECMOEA
P13.9291 × 10 1 (4.28 × 10 4 ) ≈3.0238 × 10 1 (1.26 × 10 1 ) −3.9196 × 10 1 (1.85 × 10 3 ) −2.7722 × 10 1 (1.55 × 10 1 ) −2.0577 × 10 1 (1.56 × 10 1 ) −3.8811 × 10 1 (1.55 × 10 2 ) −6.8272 × 10 2 (1.39 × 10 1 ) −6.9686 × 10 2 (1.23 × 10 1 ) −1.4383 × 10 1 (1.46 × 10 1 ) −3.9106 × 10 1 (7.43 × 10 3 ) −3.9308 × 10 1 (8.03 × 10 5 )
P29.0144 × 10 1 (2.04 × 10 3 ) ≈8.9854 × 10 1 (7.58 × 10 4 ) −8.5417 × 10 1 (2.86 × 10 2 ) −8.6410 × 10 1 (9.17 × 10 3 ) −8.8007 × 10 1 (1.37 × 10 2 ) −8.9264 × 10 1 (1.47 × 10 3 ) −9.0137 × 10 1 (1.16 × 10 4 ) −7.8863 × 10 1 (8.29 × 10 2 ) −8.0921 × 10 1 (5.56 × 10 2 ) −8.5025 × 10 1 (1.61 × 10 1 ) −9.0260 × 10 1 (1.04 × 10 4 )
P32.6094 × 10 2 (5.59 × 10 5 ) ≈2.5974 × 10 2 (5.76 × 10 5 ) −2.4462 × 10 2 (9.27 × 10 4 ) −2.6161 × 10 2 (4.15 × 10 5 ) +2.6075 × 10 2 (5.32 × 10 5 ) ≈2.4740 × 10 2 (7.61 × 10 4 ) −2.5830 × 10 2 (9.31 × 10 5 ) −2.2711 × 10 2 (5.11 × 10 4 ) −2.0440 × 10 2 (1.80 × 10 4 ) −2.5703 × 10 2 (1.08 × 10 4 ) −2.6106 × 10 2 (5.95 × 10 5 )
P44.0986 × 10 1 (4.85 × 10 5 ) ≈4.0913 × 10 1 (1.69 × 10 4 ) −3.8360 × 10 1 (8.47 × 10 3 ) −3.2252 × 10 1 (1.03 × 10 2 ) −4.0841 × 10 1 (5.60 × 10 4 ) −3.8697 × 10 1 (3.95 × 10 3 ) −3.8660 × 10 1 (4.41 × 10 3 ) −3.7154 × 10 1 (7.10 × 10 3 ) −3.2602 × 10 1 (1.11 × 10 2 ) −4.0900 × 10 1 (1.83 × 10 4 ) −4.0986 × 10 1 (6.41 × 10 5 )
P55.5894 × 10 1 (1.02 × 10 3 ) ≈5.5146 × 10 1 (5.49 × 10 3 ) −5.4242 × 10 1 (9.01 × 10 3 ) −4.3974 × 10 1 (4.62 × 10 2 ) −5.4640 × 10 1 (9.71 × 10 3 ) −5.5898 × 10 1 (1.39 × 10 3 ) ≈5.5285 × 10 1 (4.68 × 10 3 ) −5.4569 × 10 1 (6.12 × 10 3 ) −5.4781 × 10 1 (3.59 × 10 3 ) −5.6002 × 10 1 (3.13 × 10 4 ) +5.5899 × 10 1 (1.49 × 10 3 )
P69.0075 × 10 2 (5.83 × 10 5 ) ≈8.8926 × 10 2 (1.37 × 10 4 ) −8.9313 × 10 2 (4.32 × 10 4 ) −7.8894 × 10 2 (6.20 × 10 3 ) −8.8914 × 10 2 (9.13 × 10 4 ) −8.4795 × 10 2 (8.14 × 10 3 ) −9.0181 × 10 2 (9.24 × 10 5 ) +8.7384 × 10 2 (2.22 × 10 3 ) −8.6743 × 10 2 (5.21 × 10 4 ) −8.9114 × 10 2 (2.68 × 10 4 ) −9.0038 × 10 2 (1.16 × 10 4 )
P77.6371 × 10 1 (1.49 × 10 4 ) ≈7.6217 × 10 1 (2.43 × 10 4 ) −7.6311 × 10 1 (6.53 × 10 4 ) −7.5410 × 10 1 (1.86 × 10 3 ) −7.6217 × 10 1 (6.17 × 10 4 ) −7.6288 × 10 1 (3.55 × 10 6 ) −7.6309 × 10 1 (8.05 × 10 4 ) −7.5543 × 10 1 (3.25 × 10 3 ) −7.5045 × 10 1 (3.68 × 10 3 ) −7.6346 × 10 1 (1.56 × 10 4 ) −7.6376 × 10 1 (8.07 × 10 5 )
P82.5391 × 10 1 (2.79 × 10 2 ) −3.5733 × 10 1 (6.61 × 10 2 ) ≈1.9176 × 10 1 (4.77 × 10 2 ) −2.2423 × 10 1 (3.97 × 10 2 ) −2.4257 × 10 1 (4.09 × 10 2 ) −2.2963 × 10 1 (4.41 × 10 2 ) −2.1218 × 10 1 (5.97 × 10 2 ) −2.2754 × 10 1 (3.59 × 10 2 ) −2.4525 × 10 1 (6.35 × 10 2 ) −2.4840 × 10 1 (4.11 × 10 2 ) −3.9297 × 10 1 (1.84 × 10 2 )
× + / / 0/1/70/7/10/8/01/7/00/7/10/7/11/7/00/8/00/8/01/7/0
Table A22. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on many-objective C-DTLZ and DC-DTLZ benchmarks. Best result in each row is highlighted.
Table A22. Statistical results of HV obtained by ECMOEA and CMOEAs in comparison on many-objective C-DTLZ and DC-DTLZ benchmarks. Best result in each row is highlighted.
ProblemmnICMACCMOPPSCMOEA_MSCTAEADCNSGAIIIMOEADDAENSGAIIToRTiGE_2ToPECMOEA
C1-DTLZ1599.6784 × 10 1 (2.08 × 10 3 ) +9.7253 × 10 1 (2.66 × 10 3 ) +9.5070 × 10 1 (8.32 × 10 3 ) ≈9.5332 × 10 1 (1.16 × 10 2 ) ≈9.7009 × 10 1 (1.52 × 10 3 ) +9.7168 × 10 1 (3.27 × 10 3 ) +9.7222 × 10 1 (8.64 × 10 4 ) +NaN (NaN)1.4802 × 10 1 (6.15 × 10 2 ) −NaN (NaN)9.5140 × 10 1 (9.04 × 10 3 )
8129.9025 × 10 1 (3.08 × 10 3 ) +9.5504 × 10 1 (5.60 × 10 2 ) ≈9.8323 × 10 1 (3.82 × 10 3 ) +9.7307 × 10 1 (1.44 × 10 2 ) ≈9.9375 × 10 1 (4.77 × 10 3 ) +9.8155 × 10 1 (1.56 × 10 2 ) +9.8643 × 10 1 (4.77 × 10 3 ) +3.1324 × 10 1 (1.55 × 10 1 ) −1.6604 × 10 1 (9.54 × 10 2 ) −4.7752 × 10 1 (0.00 × 10 0 ) ≈9.7341 × 10 1 (1.38 × 10 2 )
C1-DTLZ35140.0000 × 10 0 (0.00 × 10 0 ) −2.3085 × 10 1 (1.87 × 10 1 ) +6.3363 × 10 2 (9.44 × 10 2 ) ≈7.1809 × 10 1 (6.42 × 10 2 ) +1.4472 × 10 2 (6.47 × 10 2 ) −4.7781 × 10 2 (1.59 × 10 1 ) +7.6434 × 10 1 (6.19 × 10 3 ) +0.0000 × 10 0 (0.00 × 10 0 ) −0.0000 × 10 0 (0.00 × 10 0 ) −1.5117 × 10 2 (3.68 × 10 2 ) −4.3401 × 10 2 (5.70 × 10 2 )
8170.0000 × 10 0 (0.00 × 10 0 ) −0.0000 × 10 0 (0.00 × 10 0 ) −2.6157 × 10 1 (1.73 × 10 1 ) ≈1.6536 × 10 1 (1.87 × 10 1 ) ≈0.0000 × 10 0 (0.00 × 10 0 ) −1.2875 × 10 2 (5.76 × 10 2 ) −6.1568 × 10 1 (7.87 × 10 2 ) +0.0000 × 10 0 (0.00 × 10 0 ) −0.0000 × 10 0 (0.00 × 10 0 ) −4.4447 × 10 3 (1.99 × 10 2 ) −2.2712 × 10 1 (1.77 × 10 1 )
C2-DTLZ25147.0145 × 10 1 (3.32 × 10 3 ) −7.1600 × 10 1 (4.32 × 10 3 ) ≈6.5196 × 10 1 (1.47 × 10 2 ) −7.3436 × 10 1 (2.72 × 10 3 ) +7.1909 × 10 1 (2.84 × 10 3 ) +7.4802 × 10 1 (4.05 × 10 3 ) +7.2312 × 10 1 (5.63 × 10 3 ) +3.1411 × 10 1 (1.31 × 10 1 ) −6.1283 × 10 1 (2.07 × 10 2 ) −4.7317 × 10 1 (2.03 × 10 2 ) −7.1184 × 10 1 (7.54 × 10 3 )
8178.0223 × 10 1 (6.72 × 10 3 ) +7.8326 × 10 1 (1.47 × 10 2 ) ≈5.7745 × 10 1 (3.20 × 10 2 ) −8.0994 × 10 1 (1.55 × 10 2 ) +7.9557 × 10 1 (9.70 × 10 3 ) +8.2243 × 10 1 (4.05 × 10 2 ) +7.5013 × 10 1 (3.12 × 10 2 ) −2.0918 × 10 1 (1.14 × 10 1 ) −7.7829 × 10 1 (1.51 × 10 2 ) ≈2.8944 × 10 1 (1.06 × 10 1 ) −7.8217 × 10 1 (1.08 × 10 2 )
C3-DTLZ45149.5681 × 10 1 (6.76 × 10 4 ) ≈9.3028 × 10 1 (4.29 × 10 3 ) −9.1443 × 10 1 (8.83 × 10 3 ) −7.9507 × 10 1 (4.39 × 10 2 ) −9.4935 × 10 1 (6.47 × 10 4 ) −9.4910 × 10 1 (2.02 × 10 2 ) −7.9827 × 10 1 (1.38 × 10 1 ) −7.2272 × 10 1 (3.19 × 10 2 ) −9.4388 × 10 1 (3.76 × 10 3 ) −8.4460 × 10 1 (1.45 × 10 2 ) −9.5675 × 10 1 (4.49 × 10 4 )
8179.9591 × 10 1 (1.28 × 10 4 ) ≈4.8232 × 10 2 (5.60 × 10 3 ) −9.6470 × 10 1 (6.04 × 10 3 ) −9.1017 × 10 1 (2.57 × 10 2 ) −9.9436 × 10 1 (1.33 × 10 4 ) −9.8100 × 10 1 (1.60 × 10 2 ) −9.3165 × 10 1 (7.43 × 10 2 ) −1.7348 × 10 1 (5.37 × 10 2 ) −9.9097 × 10 1 (7.99 × 10 4 ) −7.6619 × 10 1 (1.45 × 10 1 ) −9.9588 × 10 1 (1.98 × 10 4 )
DC1-DTLZ1597.5050 × 10 1 (4.92 × 10 3 ) ≈7.5832 × 10 1 (2.67 × 10 3 ) +5.3975 × 10 1 (2.62 × 10 1 ) −7.5888 × 10 1 (1.75 × 10 3 ) +7.6391 × 10 1 (1.25 × 10 3 ) +7.6373 × 10 1 (1.12 × 10 3 ) +7.6419 × 10 1 (1.02 × 10 3 ) +0.0000 × 10 0 (0.00 × 10 0 ) −2.9650 × 10 2 (4.21 × 10 2 ) −8.7014 × 10 2 (2.00 × 10 1 ) −7.4428 × 10 1 (2.74 × 10 2 )
8123.3041 × 10 1 (2.93 × 10 1 ) −1.2436 × 10 2 (5.56 × 10 2 ) −7.7420 × 10 1 (1.83 × 10 2 ) +7.8995 × 10 1 (6.94 × 10 3 ) +7.9040 × 10 1 (1.35 × 10 3 ) +7.9399 × 10 1 (7.36 × 10 4 ) +7.8654 × 10 1 (3.25 × 10 3 ) +0.0000 × 10 0 (0.00 × 10 0 ) −6.3420 × 10 2 (9.55 × 10 2 ) −9.7028 × 10 2 (1.22 × 10 1 ) −7.2102 × 10 1 (1.71 × 10 1 )
DC1-DTLZ35146.3792 × 10 2 (1.60 × 10 1 ) +3.4889 × 10 1 (2.15 × 10 1 ) +2.2863 × 10 2 (3.93 × 10 2 ) −6.3557 × 10 1 (9.68 × 10 2 ) +3.1988 × 10 1 (2.21 × 10 1 ) +6.9099 × 10 1 (8.14 × 10 2 ) +7.4007 × 10 1 (5.51 × 10 3 ) +0.0000 × 10 0 (0.00 × 10 0 ) −0.0000 × 10 0 (0.00 × 10 0 ) −6.5879 × 10 3 (2.03 × 10 2 ) −5.2506 × 10 2 (8.80 × 10 2 )
8170.0000 × 10 0 (0.00 × 10 0 ) −0.0000 × 10 0 (0.00 × 10 0 ) −2.2933 × 10 1 (1.97 × 10 1 ) ≈1.5183 × 10 1 (1.71 × 10 1 ) ≈3.0448 × 10 1 (2.40 × 10 1 ) ≈3.4837 × 10 1 (3.12 × 10 1 ) +6.2247 × 10 1 (9.65 × 10 2 ) +0.0000 × 10 0 (0.00 × 10 0 ) −0.0000 × 10 0 (0.00 × 10 0 ) −0.0000 × 10 0 (0.00 × 10 0 ) −2.0046 × 10 1 (2.00 × 10 1 )
DC2-DTLZ1598.6906 × 10 1 (8.49 × 10 2 ) ≈9.7052 × 10 1 (1.87 × 10 3 ) +9.1509 × 10 1 (8.62 × 10 2 ) −9.6914 × 10 1 (8.49 × 10 4 ) +9.7015 × 10 1 (4.13 × 10 4 ) +9.0679 × 10 1 (1.06 × 10 1 ) ≈9.7385 × 10 1 (2.87 × 10 4 ) +NaN (NaN)2.9180 × 10 1 (9.18 × 10 2 ) −NaN (NaN)9.3158 × 10 1 (8.43 × 10 2 )
8129.7600 × 10 1 (2.67 × 10 2 ) ≈NaN (NaN)9.8031 × 10 1 (1.69 × 10 2 ) +9.9399 × 10 1 (8.18 × 10 4 ) +9.9645 × 10 1 (1.92 × 10 4 ) +9.5857 × 10 1 (4.36 × 10 2 ) ≈9.9064 × 10 1 (2.46 × 10 3 ) +NaN (NaN)3.9341 × 10 1 (2.22 × 10 1 ) −NaN (NaN)9.3892 × 10 1 (5.99 × 10 2 )
DC2-DTLZ35142.7890 × 10 2 (1.55 × 10 2 ) ≈3.3027 × 10 2 (1.05 × 10 2 ) ≈2.1702 × 10 1 (2.84 × 10 1 ) ≈1.5464 × 10 1 (2.68 × 10 1 ) +1.6707 × 10 2 (3.28 × 10 2 ) −2.1928 × 10 2 (7.61 × 10 3 ) ≈7.9433 × 10 1 (1.85 × 10 3 ) +NaN (NaN)NaN (NaN)NaN (NaN)9.0562 × 10 2 (1.61 × 10 1 )
8176.5320 × 10 2 (4.24 × 10 3 ) ≈NaN (NaN)4.1913 × 10 1 (3.03 × 10 1 ) +1.5182 × 10 1 (1.77 × 10 1 ) −6.9932 × 10 2 (2.06 × 10 1 ) −1.0560 × 10 1 (4.01 × 10 2 ) ≈7.7867 × 10 1 (4.39 × 10 2 ) +NaN (NaN)3.2507 × 10 1 (1.15 × 10 1 ) ≈NaN (NaN)1.8597 × 10 1 (2.62 × 10 1 )
DC3-DTLZ1590.0000 × 10 0 (0.00 × 10 0 ) −1.2619 × 10 1 (7.15 × 10 3 ) +0.0000 × 10 0 (0.00 × 10 0 ) −6.7706 × 10 2 (1.03 × 10 2 ) ≈5.2936 × 10 2 (1.23 × 10 2 ) ≈1.0718 × 10 1 (2.55 × 10 2 ) +1.1385 × 10 1 (1.15 × 10 2 ) +0.0000 × 10 0 (0.00 × 10 0 ) −0.0000 × 10 0 (0.00 × 10 0 ) −0.0000 × 10 0 (0.00 × 10 0 ) −3.8474 × 10 2 (4.69 × 10 2 )
DC3-DTLZ35140.0000 × 10 0 (0.00 × 10 0 ) ≈2.6120 × 10 2 (1.17 × 10 1 ) ≈0.0000 × 10 0 (0.00 × 10 0 ) ≈2.8058 × 10 1 (2.68 × 10 1 ) +3.2938 × 10 1 (1.49 × 10 1 ) +0.0000 × 10 0 (0.00 × 10 0 ) ≈3.4169 × 10 1 (1.24 × 10 1 ) +0.0000 × 10 0 (0.00 × 10 0 ) ≈0.0000 × 10 0 (0.00 × 10 0 ) ≈0.0000 × 10 0 (0.00 × 10 0 ) ≈0.0000 × 10 0 (0.00 × 10 0 )
8170.0000 × 10 0 (0.00 × 10 0 ) ≈0.0000 × 10 0 (0.00 × 10 0 ) ≈1.5740 × 10 3 (7.04 × 10 3 ) ≈1.3469 × 10 1 (2.45 × 10 1 ) +1.0194 × 10 1 (1.11 × 10 1 ) +0.0000 × 10 0 (0.00 × 10 0 ) ≈4.3043 × 10 1 (1.55 × 10 1 ) +0.0000 × 10 0 (0.00 × 10 0 ) ≈0.0000 × 10 0 (0.00 × 10 0 ) ≈0.0000 × 10 0 (0.00 × 10 0 ) ≈0.0000 × 10 0 (0.00 × 10 0 )
× + / / 4/6/96/7/64/8/711/3/511/6/210/3/616/3/00/17/20/15/40/16/3
Table A23. Statistical results of IGD obtained by ECMOEA and CMOEAs in comparison on many-objective C-DTLZ and DC-DTLZ benchmarks. Best result in each row is highlighted.
Table A23. Statistical results of IGD obtained by ECMOEA and CMOEAs in comparison on many-objective C-DTLZ and DC-DTLZ benchmarks. Best result in each row is highlighted.
ProblemmnICMACCMOPPSCMOEA_MSCTAEADCNSGAIIIMOEADDAENSGAIIToRTiGE_2ToPECMOEA
C1-DTLZ1596.4089 × 10 2 (7.94 × 10 4 ) +6.2542 × 10 2 (6.23 × 10 4 ) +7.3340 × 10 2 (3.91 × 10 3 ) ≈6.0548 × 10 2 (5.02 × 10 4 ) +7.1597 × 10 2 (8.66 × 10 4 ) ≈6.2533 × 10 2 (3.60 × 10 4 ) +6.5350 × 10 2 (8.19 × 10 4 ) +NaN (NaN)4.4485 × 10 1 (3.86 × 10 2 ) −NaN (NaN)7.2170 × 10 2 (5.93 × 10 3 )
8121.0651 × 10 1 (9.81 × 10 4 ) +1.1013 × 10 1 (9.13 × 10 3 ) +1.2121 × 10 1 (6.44 × 10 3 ) ≈1.0116 × 10 1 (6.41 × 10 4 ) +1.2091 × 10 1 (1.47 × 10 3 ) ≈1.0290 × 10 1 (1.35 × 10 2 ) +1.0984 × 10 1 (2.12 × 10 3 ) +3.8552 × 10 1 (8.02 × 10 2 ) −4.6367 × 10 1 (6.34 × 10 2 ) −3.0665 × 10 1 (0.00 × 10 0 ) ≈1.2156 × 10 1 (1.52 × 10 2 )
C1-DTLZ35148.9975 × 10 0 (4.12 × 10 0 ) −4.8886 × 10 1 (1.45 × 10 1 ) +3.1668 × 10 0 (4.39 × 10 0 ) ≈2.1399 × 10 1 (2.19 × 10 2 ) +1.1081 × 10 + 1 (2.50 × 10 0 ) −8.9902 × 10 0 (4.62 × 10 0 ) −2.5418 × 10 1 (7.89 × 10 3 ) +2.9018 × 10 + 1 (6.39 × 10 0 ) −1.0993 × 10 + 1 (3.20 × 10 0 ) −5.3808 × 10 0 (5.25 × 10 0 ) −9.5493 × 10 1 (3.59 × 10 1 )
8179.5684 × 10 0 (3.93 × 10 0 ) +1.1415 × 10 + 2 (1.62 × 10 + 1 ) −2.4058 × 10 0 (4.09 × 10 0 ) ≈1.9269 × 10 0 (3.41 × 10 0 ) ≈1.0330 × 10 + 1 (3.41 × 10 0 ) +1.0232 × 10 + 1 (3.68 × 10 0 ) +5.4410 × 10 1 (5.02 × 10 2 ) +8.5969 × 10 + 1 (8.38 × 10 0 ) −1.0186 × 10 + 1 (4.22 × 10 0 ) +1.9347 × 10 + 1 (7.00 × 10 0 ) +2.4375 × 10 + 1 (4.93 × 10 + 1 )
C2-DTLZ25141.7265 × 10 1 (2.69 × 10 3 ) −1.6933 × 10 1 (2.69 × 10 3 ) ≈2.0508 × 10 1 (6.94 × 10 3 ) −1.6145 × 10 1 (1.80 × 10 3 ) +1.7245 × 10 1 (1.51 × 10 3 ) −1.6461 × 10 1 (2.00 × 10 3 ) +1.7104 × 10 1 (2.75 × 10 3 ) ≈5.2251 × 10 1 (2.13 × 10 1 ) −2.6320 × 10 1 (2.56 × 10 2 ) −2.2893 × 10 1 (6.94 × 10 3 ) −1.7078 × 10 1 (3.25 × 10 3 )
8172.7097 × 10 1 (4.22 × 10 3 ) +2.8191 × 10 1 (5.51 × 10 3 ) −4.7258 × 10 1 (4.75 × 10 2 ) −2.9316 × 10 1 (2.83 × 10 2 ) ≈2.3786 × 10 1 (1.97 × 10 3 ) +2.6545 × 10 1 (7.38 × 10 2 ) +3.7856 × 10 1 (4.96 × 10 2 ) −8.4991 × 10 1 (2.00 × 10 1 ) −2.9570 × 10 1 (7.90 × 10 3 ) −6.9632 × 10 1 (2.07 × 10 1 ) −2.7614 × 10 1 (4.54 × 10 3 )
C3-DTLZ45143.1782 × 10 1 (3.49 × 10 3 ) ≈3.5761 × 10 1 (8.59 × 10 3 ) −4.9832 × 10 1 (3.51 × 10 2 ) −6.4864 × 10 1 (5.50 × 10 2 ) −3.3794 × 10 1 (2.52 × 10 3 ) −3.2917 × 10 1 (1.03 × 10 1 ) −6.8279 × 10 1 (2.47 × 10 1 ) −7.0873 × 10 1 (3.74 × 10 2 ) −4.2811 × 10 1 (2.07 × 10 2 ) −5.1382 × 10 1 (2.52 × 10 2 ) −3.1989 × 10 1 (3.93 × 10 3 )
8175.2824 × 10 1 (4.79 × 10 3 ) ≈2.1470 × 10 0 (1.20 × 10 2 ) −7.3290 × 10 1 (2.82 × 10 2 ) −7.7702 × 10 1 (3.37 × 10 2 ) −5.4566 × 10 1 (3.06 × 10 3 ) −6.3794 × 10 1 (1.30 × 10 1 ) −7.5830 × 10 1 (1.23 × 10 1 ) −1.6552 × 10 0 (7.46 × 10 2 ) −6.4435 × 10 1 (7.68 × 10 3 ) −9.9799 × 10 1 (1.49 × 10 1 ) −5.2773 × 10 1 (5.37 × 10 3 )
DC1-DTLZ1594.7985 × 10 2 (2.31 × 10 3 ) +5.1281 × 10 2 (1.56 × 10 3 ) +1.6061 × 10 1 (1.28 × 10 1 ) −4.6198 × 10 2 (7.58 × 10 4 ) +5.0032 × 10 2 (7.33 × 10 4 ) +4.6929 × 10 2 (2.69 × 10 4 ) +4.5231 × 10 2 (3.59 × 10 4 ) +1.5567 × 10 + 1 (5.45 × 10 0 ) −9.0009 × 10 1 (4.56 × 10 1 ) −9.9775 × 10 1 (7.05 × 10 1 ) −5.6484 × 10 2 (1.00 × 10 2 )
8122.6733 × 10 1 (1.20 × 10 1 ) −2.1124 × 10 0 (2.08 × 10 0 ) −1.0349 × 10 1 (1.27 × 10 2 ) +8.1678 × 10 2 (6.45 × 10 3 ) +8.2175 × 10 2 (8.41 × 10 4 ) +7.8320 × 10 2 (6.71 × 10 4 ) +8.4085 × 10 2 (2.18 × 10 3 ) +5.9120 × 10 + 1 (1.81 × 10 + 1 ) −7.2285 × 10 1 (4.17 × 10 1 ) −1.8738 × 10 0 (4.33 × 10 0 ) −2.3078 × 10 1 (5.28 × 10 1 )
DC1-DTLZ35141.3150 × 10 0 (7.57 × 10 1 ) −3.4575 × 10 1 (1.17 × 10 1 ) +1.9062 × 10 0 (1.46 × 10 0 ) −2.7626 × 10 1 (5.95 × 10 2 ) +3.4974 × 10 1 (8.21 × 10 2 ) +1.8181 × 10 1 (2.50 × 10 2 ) +2.0528 × 10 1 (5.15 × 10 3 ) +3.3615 × 10 + 1 (7.66 × 10 0 ) −3.0914 × 10 0 (7.39 × 10 1 ) −3.8221 × 10 0 (2.24 × 10 0 ) −7.5318 × 10 1 (3.87 × 10 1 )
8173.7457 × 10 0 (8.67 × 10 1 ) ≈9.8827 × 10 + 1 (1.26 × 10 + 1 ) −1.5747 × 10 0 (1.79 × 10 0 ) ≈8.8332 × 10 1 (2.77 × 10 1 ) ≈6.2607 × 10 1 (1.34 × 10 1 ) +6.1517 × 10 1 (2.07 × 10 1 ) +5.2340 × 10 1 (6.70 × 10 2 ) +7.3010 × 10 + 1 (1.38 × 10 + 1 ) ≈2.8255 × 10 0 (7.22 × 10 1 ) ≈2.1541 × 10 + 1 (4.45 × 10 0 ) ≈3.8898 × 10 + 1 (4.82 × 10 + 1 )
DC2-DTLZ1591.2911 × 10 1 (4.97 × 10 2 ) ≈6.6366 × 10 2 (1.62 × 10 3 ) +9.9274 × 10 2 (3.90 × 10 2 ) −6.1989 × 10 2 (4.94 × 10 4 ) +7.3139 × 10 2 (6.40 × 10 4 ) ≈1.0037 × 10 1 (5.70 × 10 2 ) ≈6.7052 × 10 2 (4.99 × 10 4 ) +NaN (NaN)3.7237 × 10 1 (3.49 × 10 2 ) −NaN (NaN)8.5819 × 10 2 (3.32 × 10 2 )
8121.5083 × 10 1 (3.49 × 10 2 ) ≈NaN (NaN)1.3843 × 10 1 (1.71 × 10 2 ) ≈1.0307 × 10 1 (9.44 × 10 4 ) +1.2258 × 10 1 (1.50 × 10 3 ) +1.4223 × 10 1 (3.66 × 10 2 ) ≈1.1137 × 10 1 (2.26 × 10 3 ) +NaN (NaN)3.7356 × 10 1 (8.44 × 10 2 ) −NaN (NaN)1.5750 × 10 1 (3.73 × 10 2 )
DC2-DTLZ35148.2042 × 10 1 (3.57 × 10 1 ) ≈6.7473 × 10 1 (6.64 × 10 2 ) ≈5.5968 × 10 1 (1.95 × 10 1 ) +5.8358 × 10 1 (1.80 × 10 1 ) +6.7775 × 10 1 (1.63 × 10 1 ) +6.8011 × 10 1 (1.27 × 10 1 ) ≈2.1387 × 10 1 (1.94 × 10 3 ) +NaN (NaN)NaN (NaN)NaN (NaN)6.7852 × 10 1 (1.52 × 10 1 )
8177.5533 × 10 1 (5.41 × 10 3 ) ≈NaN (NaN)6.3943 × 10 1 (2.01 × 10 1 ) +7.3023 × 10 1 (9.11 × 10 2 ) +7.2971 × 10 1 (7.89 × 10 2 ) +7.7951 × 10 1 (8.41 × 10 2 ) +4.7967 × 10 1 (5.84 × 10 2 ) +NaN (NaN)8.2321 × 10 1 (8.89 × 10 2 ) ≈NaN (NaN)8.7926 × 10 1 (2.09 × 10 1 )
DC3-DTLZ1597.1433 × 10 1 (4.20 × 10 1 ) −2.2503 × 10 2 (6.61 × 10 4 ) +4.5723 × 10 0 (4.44 × 10 0 ) −4.5767 × 10 2 (4.23 × 10 3 ) ≈5.4737 × 10 2 (1.53 × 10 2 ) ≈3.9678 × 10 2 (6.06 × 10 2 ) +2.9923 × 10 2 (1.44 × 10 2 ) +6.6757 × 10 + 1 (2.77 × 10 + 1 ) −1.7174 × 10 0 (1.06 × 10 0 ) −1.1618 × 10 + 1 (6.91 × 10 0 ) −1.8316 × 10 1 (1.86 × 10 1 )
DC3-DTLZ35146.2764 × 10 0 (2.12 × 10 0 ) −7.2240 × 10 1 (3.15 × 10 1 ) +8.5656 × 10 0 (3.97 × 10 0 ) −3.8016 × 10 1 (2.41 × 10 1 ) +2.4185 × 10 1 (1.51 × 10 1 ) +7.1637 × 10 1 (3.47 × 10 1 ) +3.7372 × 10 1 (3.70 × 10 1 ) +5.3403 × 10 + 1 (1.09 × 10 + 1 ) −4.6013 × 10 0 (1.31 × 10 0 ) −1.3763 × 10 + 1 (7.37 × 10 0 ) −1.4824 × 10 0 (7.75 × 10 1 )
8171.0528 × 10 + 1 (2.47 × 10 0 ) +5.2001 × 10 + 1 (8.26 × 10 0 ) −1.0751 × 10 + 1 (5.11 × 10 0 ) +6.7646 × 10 1 (2.39 × 10 1 ) +6.9828 × 10 1 (2.78 × 10 1 ) +1.5111 × 10 0 (7.60 × 10 1 ) +7.4731 × 10 1 (1.83 × 10 1 ) +5.8024 × 10 + 1 (1.15 × 10 + 1 ) −4.6590 × 10 0 (1.41 × 10 0 ) +2.9455 × 10 + 1 (5.71 × 10 0 ) +4.6116 × 10 + 1 (9.64 × 10 0 )
× + / / 6/6/78/9/24/9/613/2/411/4/413/3/315/3/10/18/12/15/22/15/2
Figure A6. Feasible and non-dominated solution sets obtained by ECMOEA and CMOEAs in comparison on Two Bar Truss Design problem with the median IGD value among 30 runs.
Figure A6. Feasible and non-dominated solution sets obtained by ECMOEA and CMOEAs in comparison on Two Bar Truss Design problem with the median IGD value among 30 runs.
Symmetry 14 00116 g0a6

References

  1. Bani-Bakr, A.; Hindia, M.N.; Dimyati, K.; Hanafi, E.; Tengku Mohmed Noor Izam, T.F. Multi-Objective Caching Optimization for Wireless Backhauled Fog Radio Access Network. Symmetry 2021, 13, 708. [Google Scholar] [CrossRef]
  2. Wang, J.; Zhou, Y.; Wang, Y.; Zhang, J.; Chen, C.L.P.; Zheng, Z. Multiobjective Vehicle Routing Problems With Simultaneous Delivery and Pickup and Time Windows: Formulation, Instances, and Algorithms. IEEE Trans. Cybern. 2016, 46, 582–594. [Google Scholar] [CrossRef] [PubMed]
  3. Tan, B.; Ma, H.; Mei, Y.; Zhang, M. Evolutionary Multi-Objective Optimization for Web Service Location Allocation Problem. IEEE Trans. Serv. Comput. 2018, 14, 458–471. [Google Scholar] [CrossRef] [Green Version]
  4. Mnasri, S.; Nasri, N.; Van Den Bossche, A.; Val, T. 3D indoor redeployment in IoT collection networks: A real prototyping using a hybrid PI-NSGA-III-VF. In Proceedings of the 2018 14th International Wireless Communications Mobile Computing Conference (IWCMC), Limassol, Cyprus, 25–29 June 2018; pp. 780–785. [Google Scholar] [CrossRef]
  5. Zheng, A.; Li, B.; Zheng, M.; Zhong, H. Multi-Objective UAV Trajectory Planning in Uncertain Environment. Symmetry 2021, 13, 2160. [Google Scholar] [CrossRef]
  6. Su, Z.; Zhang, G.; Yue, F.; Zhan, D.; Li, M.; Li, B.; Yao, X. Enhanced Constraint Handling for Reliability-Constrained Multi-objective Testing Resource Allocation. IEEE Trans. Evol. Comput. 2021, 25, 537–551. [Google Scholar] [CrossRef]
  7. Li, X.; Zhu, X.; Li, B. Multi-Objective Optimization of Differentiated Urban Ring Road Bus Lines and Fares Based on Travelers Interactive Reinforcement Learning. Symmetry 2021, 13, 2301. [Google Scholar] [CrossRef]
  8. Ishibuchi, H.; Murata, T. A multi-objective genetic local search algorithm and its application to flowshop scheduling. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 1998, 28, 392–403. [Google Scholar] [CrossRef]
  9. Wu, H.C. Using Shapley Values and Genetic Algorithms to Solve Multiobjective Optimization Problems. Symmetry 2021, 13, 2021. [Google Scholar] [CrossRef]
  10. Li, B.; Li, J.; Ke, T.; Xin, Y. Many-Objective Evolutionary Algorithms: A survey. ACM Comput. Surv. 2015, 48, 1–35. [Google Scholar] [CrossRef] [Green Version]
  11. Tian, Y.; Si, L.; Zhang, X.; Cheng, R.; He, C.; Tan, K.; Jin, Y. Evolutionary Large-Scale Multi-Objective Optimization: A Survey. ACM Comput. Surv. 2022, 54, 1–34. [Google Scholar] [CrossRef]
  12. Deb, K.; Pratap, A.; Agarwal, S.; Meyarivan, T. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 2002, 6, 182–197. [Google Scholar] [CrossRef] [Green Version]
  13. Zhang, Q.; Li, H. MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition. IEEE Trans. Evol. Comput. 2007, 11, 712–731. [Google Scholar] [CrossRef]
  14. Zitzler, E.; Künzli, S. Indicator-Based Selection in Multiobjective Search. In International Conference on Parallel Problem Solving from Nature; Springer: Berlin/Heidelberg, Germany, 2004. [Google Scholar]
  15. Yuan, J.; Liu, H.L.; Ong, Y.S.; He, Z. Indicator-based Evolutionary Algorithm for Solving Constrained Multi-objective Optimization Problems. IEEE Trans. Evol. Comput. 2021. [Google Scholar] [CrossRef]
  16. Zhu, Q.; Zhang, Q.; Lin, Q. A Constrained Multiobjective Evolutionary Algorithm With Detect-and-Escape Strategy. IEEE Trans. Evol. Comput. 2020, 24, 938–947. [Google Scholar] [CrossRef]
  17. Tian, Y.; Zhang, T.; Xiao, J.; Zhang, X.; Jin, Y. A Coevolutionary Framework for Constrained Multi-Objective Optimization Problems. IEEE Trans. Evol. Comput. 2020, 25, 102–116. [Google Scholar] [CrossRef]
  18. Ma, Z.; Wang, Y.; Song, W. A New Fitness Function With Two Rankings for Evolutionary Constrained Multiobjective Optimization. IEEE Trans. Syst. Man Cybern. Syst. 2019, 51, 5005–5016. [Google Scholar] [CrossRef]
  19. Jain, H.; Deb, K. An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point Based Nondominated Sorting Approach, Part II: Handling Constraints and Extending to an Adaptive Approach. IEEE Trans. Evol. Comput. 2014, 18, 602–622. [Google Scholar] [CrossRef]
  20. Li, K.; Chen, R.; Fu, G.; Yao, X. Two-Archive Evolutionary Algorithm for Constrained Multiobjective Optimization. IEEE Trans. Evol. Comput. 2019, 23, 303–315. [Google Scholar] [CrossRef] [Green Version]
  21. Fan, Z.; Li, W.; Cai, X.; Huang, H.; Fang, Y.; Yugen, Y.; Mo, J.; Wei, C.; Goodman, E. An Improved Epsilon Constraint-handling Method in MOEA/D for CMOPs with Large Infeasible Regions. Soft Comput. 2019, 23, 12491–12510. [Google Scholar] [CrossRef] [Green Version]
  22. Fan, Z.; Li, W.; Cai, X.; Li, H.; Wei, C.; Zhang, Q.; Deb, K.; Goodman, E. Push and pull search for solving constrained multi-objective optimization problems. Swarm Evol. Comput. 2019, 44, 665–679. [Google Scholar] [CrossRef] [Green Version]
  23. Liu, Z.; Wang, Y. Handling Constrained Multiobjective Optimization Problems With Constraints in Both the Decision and Objective Spaces. IEEE Trans. Evol. Comput. 2019, 23, 870–884. [Google Scholar] [CrossRef]
  24. Liu, Z.Z.; Wang, B.C.; Tang, K. Handling Constrained Multiobjective Optimization Problems via Bidirectional Coevolution. IEEE Trans. Cybern. 2021, 1–14. [Google Scholar] [CrossRef]
  25. Tian, Y.; Zhang, Y.; Su, Y.; Zhang, X.; Tan, K.; Jin, Y. Balancing Objective Optimization and Constraint Satisfaction in Constrained Evolutionary Multi-Objective Optimization. IEEE Trans. Cybern. 2020. [Google Scholar] [CrossRef]
  26. Takahama, T.; Sakai, S. Constrained Optimization by the ε Constrained Differential Evolution with Gradient-Based Mutation and Feasible Elites. In Proceedings of the 2006 IEEE International Conference on Evolutionary Computation, Vancouver, BC, Canada, 16–21 July 2006; pp. 1–8. [Google Scholar] [CrossRef]
  27. Fan, Z.; Wang, Z.; Li, W.; Yuan, Y.; You, Y.; Yang, Z.; Sun, F.; Ruan, J. Push and pull search embedded in an M2M framework for solving constrained multi-objective optimization problems. Swarm Evol. Comput. 2020, 54, 100651. [Google Scholar] [CrossRef] [Green Version]
  28. Jiao, R.; Zeng, S.; Li, C.; Yang, S.; Ong, Y.S. Handling Constrained Many-Objective Optimization Problems via Problem Transformation. IEEE Trans. Cybern. 2020, 51, 4834–4847. [Google Scholar] [CrossRef] [PubMed]
  29. Tessema, B.; Yen, G.G. A Self Adaptive Penalty Function Based Algorithm for Constrained Optimization. In Proceedings of the 2006 IEEE International Conference on Evolutionary Computation, Vancouver, BC, Canada, 16–21 July 2006; pp. 246–253. [Google Scholar] [CrossRef]
  30. Jiao, L.; Luo, J.; Shang, R.; Liu, F. A modified objective function method with feasible-guiding strategy to solve constrained multi-objective optimization problems. Appl. Soft Comput. 2014, 14, 363–380. [Google Scholar] [CrossRef]
  31. Ming, M.; Trivedi, A.; Wang, R.; Srinivasan, D.; Zhang, T. A Dual-Population based Evolutionary Algorithm for Constrained Multi-Objective Optimization. IEEE Trans. Evol. Comput. 2021, 25, 739–753. [Google Scholar] [CrossRef]
  32. Ma, Z.; Wang, Y. Shift-Based Penalty for Evolutionary Constrained Multiobjective Optimization and Its Application. IEEE Trans. Cybern. 2021, 1–13. [Google Scholar] [CrossRef]
  33. Liu, Z.; Wang, Y.; Wang, B. Indicator-Based Constrained Multiobjective Evolutionary Algorithms. IEEE Trans. Syst. Man Cybern. Syst. 2019, 51, 5414–5426. [Google Scholar] [CrossRef]
  34. Zhou, Y.; Zhu, M.; Wang, J.; Zhang, Z.; Xiang, Y.; Zhang, J. Tri-Goal Evolution Framework for Constrained Many-Objective Optimization. IEEE Trans. Syst. Man Cybern. Syst. 2020, 50, 3086–3099. [Google Scholar] [CrossRef]
  35. Xiang, Y.; Yang, X.; Huang, H.; Wang, J. Balancing Constraints and Objectives by Considering Problem Types in Constrained Multiobjective Optimization. IEEE Trans. Cybern. 2021, 1–14. [Google Scholar] [CrossRef] [PubMed]
  36. Bader, J.; Zitzler, E. HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization. Evol. Comput. 2011, 19, 45–76. [Google Scholar] [CrossRef] [PubMed]
  37. Ishibuchi, H.; Imada, R.; Masuyama, N.; Nojima, Y. Comparison of Hypervolume, IGD and IGD+ from the Viewpoint of Optimal Distributions of Solutions. In Evolutionary Multi-Criterion Optimization; Springer: Cham, Switzerland, 2019; pp. 332–345. [Google Scholar]
  38. Tian, Y.; Cheng, R.; Zhang, X.; Jin, Y. PlatEMO: A MATLAB Platform for Evolutionary Multi-Objective Optimization. IEEE Comput. Intell. Mag. 2017, 12, 73–87. [Google Scholar] [CrossRef] [Green Version]
  39. Fan, Z.; Li, W.; Cai, X.; Li, H.; Wei, C.; Zhang, Q.; Deb, K.; Goodman, E. Difficulty Adjustable and Scalable Constrained Multiobjective Test Problem Toolkit. Evol. Comput. 2020, 28, 339–378. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. Ma, Z.; Wang, Y. Evolutionary Constrained Multiobjective Optimization: Test Suite Construction and Performance Comparisons. IEEE Trans. Evol. Comput. 2019, 23, 972–986. [Google Scholar] [CrossRef]
  41. Tian, Y.; Cheng, R.; Zhang, X.; Su, Y.; Jin, Y. A Strengthened Dominance Relation Considering Convergence and Diversity for Evolutionary Many-Objective Optimization. IEEE Trans. Evol. Comput. 2019, 23, 331–345. [Google Scholar] [CrossRef] [Green Version]
  42. Caliński, T. Review of Principles and Procedures of Statistics: A Biometrical Approach, by R. G. D. Steel & J. H. Torrie. Biometrics 1981, 37, 859–860. [Google Scholar] [CrossRef]
  43. Derrac, J.; García, S.; Molina, D.; Herrera, F. A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol. Comput. 2011, 1, 3–18. [Google Scholar] [CrossRef]
  44. Agrawal, R.; Deb, K.; Agrawal, R. Simulated Binary Crossover for Continuous Search Space. Complex Syst. 2000, 9, 115–148. [Google Scholar]
  45. Alcalá-Fdez, J.; Sánchez, L.; García, S.; del Jesus, M.J.; Ventura, S.; Garrell, J.M.; Otero, J.; Romero, C.; Bacardit, J.; Rivas, V.M.; et al. KEEL: A software tool to assess evolutionary algorithms for data mining problems. Soft Comput. 2009, 13, 307–318. [Google Scholar] [CrossRef]
  46. Kumar, A.; Wu, G.; Ali, M.; Mallipeddi, R.; Suganthan, P.; Das, S. A test-suite of non-convex constrained optimization problems from the real-world and some baseline results. Swarm Evol. Comput. 2020, 56, 100693. [Google Scholar] [CrossRef]
  47. Hassanat, A.B.A. Two-point-based binary search trees for accelerating big data classification using KNN. PLoS ONE 2018, 13, e0207772. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. Representative runs of two CMOEAs on two instances from two benchmarks. (a) CCMO on DC2-DTLZ1. (b) PPS on DC2-DTLZ1. (c) CCMO on LIR-CMOP2. (d) PPS on LIR-CMOP2.
Figure 1. Representative runs of two CMOEAs on two instances from two benchmarks. (a) CCMO on DC2-DTLZ1. (b) PPS on DC2-DTLZ1. (c) CCMO on LIR-CMOP2. (d) PPS on LIR-CMOP2.
Symmetry 14 00116 g001aSymmetry 14 00116 g001b
Figure 2. The flowchart of the proposed ensemble framework.
Figure 2. The flowchart of the proposed ensemble framework.
Symmetry 14 00116 g002
Table 1. Average rankings of HV and IGD by Friedman test for ECMOEA and CMOEAs in comparison.
Table 1. Average rankings of HV and IGD by Friedman test for ECMOEA and CMOEAs in comparison.
AlgorithmHV RankingIGD Ranking
ICMA3.56733.3846
CCMO3.85583.6731
PPS6.25966.3269
CMOEA-MS6.35586.1346
C-TAEA5.00965.2404
DCNSGA-III5.72125.8654
MOEA/D-DAE4.53854.8077
NSGA-II-ToR10.240410.2500
TiGE-27.8757.9423
ToP9.57699.4135
ECMOEA3.00002.9615
Table 2. Summary of the Wilcoxon test on HV indicator. Hereinafter, • = the method in the row improves the method of the column, ◦ = the method in the column improves the method of the row. Upper diagonal of level significance α = 0.9 , Lower diagonal level of significance α = 0.95 .
Table 2. Summary of the Wilcoxon test on HV indicator. Hereinafter, • = the method in the row improves the method of the column, ◦ = the method in the column improves the method of the row. Upper diagonal of level significance α = 0.9 , Lower diagonal level of significance α = 0.95 .
(1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11)
ICMA (1)-
CCMO (2) -
PPS (3)-
CMOEA-MS (4) -
CTAEA (5)-
DCNSGAIII (6) -
MOEADDAE (7) -
NSGAIIToR (8)-
TiGE2 (9)-
ToP (10) -
CMPort1 (11) -
Table 3. Summary of the Wilcoxon test on IGD indicator.
Table 3. Summary of the Wilcoxon test on IGD indicator.
(1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11)
ICMA (1)-
CCMO (2) -
PPS (3)-
CMOEA-MS (4) -
CTAEA (5) -
DCNSGAIII (6) -
MOEADDAE (7) -
NSGAIIToR (8)-
TiGE2 (9)-
ToP (10)-
CMPort1 (11) -
Table 4. Parameter settings for ECMOEA and CMOEAs in comparison on real-world application problems, where m is the number of objective functions, n is the number of decision variables, N is the population size and E max is the maximum number of function evaluations.
Table 4. Parameter settings for ECMOEA and CMOEAs in comparison on real-world application problems, where m is the number of objective functions, n is the number of decision variables, N is the population size and E max is the maximum number of function evaluations.
ProblemmnN E max
P1Vibrating Platform Design259160,000
P2Two Bar Truss Design239160,000
P3Car Side Impact Design379180,000
P4Four Bar Plane Truss249160,000
P5Simply Supported I-beam Design249160,000
P6Gear Box Design379180,000
P7Cantilever Beam Design229160,000
P8Bulk Carrier Design369180,000
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Ku, J.; Ming, F.; Gong, W. An Ensemble Framework of Evolutionary Algorithm for Constrained Multi-Objective Optimization. Symmetry 2022, 14, 116. https://doi.org/10.3390/sym14010116

AMA Style

Ku J, Ming F, Gong W. An Ensemble Framework of Evolutionary Algorithm for Constrained Multi-Objective Optimization. Symmetry. 2022; 14(1):116. https://doi.org/10.3390/sym14010116

Chicago/Turabian Style

Ku, Junhua, Fei Ming, and Wenyin Gong. 2022. "An Ensemble Framework of Evolutionary Algorithm for Constrained Multi-Objective Optimization" Symmetry 14, no. 1: 116. https://doi.org/10.3390/sym14010116

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop