Improved Lebesgue Indicator-Based Evolutionary Algorithm: Reducing Hypervolume Computations

Abstract

One of the major limitations of evolutionary algorithms based on the Lebesgue measure for multi-objective optimization is the computational cost required to approximate the Pareto front of a problem. Nonetheless, the Pareto compliance property of the Lebesgue measure makes it one of the most investigated indicators in the design of indicator-based evolutionary algorithms (IBEAs). The main deficiency of IBEAs that use the Lebesgue measure is their computational cost which increases with the number of objectives of the problem. On this matter, the investigation presented in this paper introduces an evolutionary algorithm based on the Lebesgue measure to deal with box-constrained continuous multi-objective optimization problems. The proposed algorithm implicitly uses the regularity property of continuous multi-objective optimization problems that has suggested effectiveness when solving continuous problems with rough Pareto sets. On the other hand, the survival selection mechanism considers the local property of the Lebesgue measure, thus reducing the computational time in our algorithmic approach. The emerging indicator-based evolutionary algorithm is examined and compared versus three state-of-the-art multi-objective evolutionary algorithms based on the Lebesgue measure. In addition, we validate its performance on a set of artificial test problems with various characteristics, including multimodality, separability, and various Pareto front forms, incorporating concavity, convexity, and discontinuity. For a more exhaustive study, the proposed algorithm is evaluated in three real-world applications having four, five, and seven objective functions whose properties are unknown. We show the high competitiveness of our proposed approach, which, in many cases, improved the state-of-the-art indicator-based evolutionary algorithms on the multi-objective problems adopted in our investigation.

1. Introduction

In several engineering and sciences applications, some problems require the simultaneous optimization of a number of objective functions. In the specialized literature, such problems are referred to as multi-objective optimization problems (MOPs). The optimization of a multi-objective problem involves determining the best compensation alternatives considered in a set of conflicting objective functions. Therefore, instead of an optimal solution, as in single-objective optimization, a set of solutions manifesting the best trade-offs among objectives is reached. The population on which evolutionary algorithms are based makes these algorithms a practical tool to solve these types of problems. For this reason, evolutionary multi-objective algorithms (EMOAs) have become a flexible and popular instrument to deal with MOPs. In the specialized literature, a variety of investigations concerning the development of evolutionary approaches for multi-objective optimization can be found. See the extensive review of such approaches presented in [1,2]. According to their conceptual foundations, EMOAs are categorized into three main groups: Pareto-based, decomposition-based, and indicator-based approaches. These approaches incorporate different search strategies that define by themself the performance of a particular EMOA. Distinctly, EMOAs based on indicators—the topic investigated in this work—explicitly optimize a quality indicator (e.g., $R2$ [3], Lebesgue measure [4], $ϵ$ indicator [5], $IGD$ [6], among others) to approximate the Pareto front of a MOP. In this manner, since its origin in the early 2000s, the indicator-based evolutionary algorithm (IBEA) [7] traced a new research line investigated to date.

IBEAs adopting indicators that use reference sets (e.g., $R2$, $ϵ$ indicator, $IGD$, etc.) are a design challenge since the optimal solutions are unknown. Consequently, reference sets cannot be adequately pre-established. Despite this, some researchers have studied diverse techniques to predict the reference set for these IBEAs [8,9]. In the evolutionary multi-objective optimization (EMOO) literature, the Lebesgue measure, also referred to as hypervolume indicator or S-metric, was introduced by Zitzler and Thiele [4] to evaluate the performance of EMOAs. This quality indicator possesses an attractive property—it is Pareto compliant [5]—that has called the attention of several researchers working on IBEAs. In particular, IBEAs adopting the Lebesgue measure benefit from not requiring reference sets because they exclusively employ reference vectors that are much simpler to state. Therefore, these IBEAs have been a practical approach to solving real-life applications where the characteristics of the problems are not known. Although IBEAs based on the Lebesgue measure are highly docile solving MOPs, their application is restricted by the computational cost of the Lebesgue measure, which grows with the number of objective functions. As pointed out in [10], this indicator cannot be calculated in polynomial time concerning the number of objectives except that $P=NP$. In addition, the complex characteristics of multi-objective problems (for example, multimodality, bias, non-separability, etc.) faced by an IBEA, further increase the computational cost in the search process for such algorithms. In other words, IBEAs use many more iterations (computational efforts) to approximate the real Pareto front of a problem. As a consequence, extensive investigations concerning the design of IBEAs using the Lebesgue measure as a quality indicator have been studied in the last few years [11,12,13,14,15]. To date, the development of EMOAs based on the hypervolume indicator is recognized as an actual area of investigation within the EMOO community, and this is precisely the topic of the investigation presented in this work.

This paper introduces an improved Lebesgue indicator-based evolutionary algorithm for multi-objective optimization. The introduced approach can be seen as an improvement of the Lebesgue indicator-based evolutionary algorithm (LIBEA) [15]. Analogous to LIBEA, the proposed algorithm addresses the notion of IBEA [7] in the sense of optimizing a quality indicator. Nevertheless, it is directed at maximizing the Lebesgue measure of non-dominated solutions obtained through the search. In contrast to several Lebesgue indicator-based EMOAs, the introduced algorithm implicitly applies the regularity property of continuous MOPs advised to approximate continuous MOPs with complicated characteristics [16,17,18]. Additionally, in order to reduce the computational time, the local property of the Lebesgue measure is considered in the survival mechanism of the proposed algorithm [19]. We hypothesize that an algorithm considering the Lebesgue measure, the regularity property of continuous MOPs, and the local property of the Lebesgue measure can solve problems with difficult features more efficiently than traditional EMOAs based on the Lebesgue measure.

The proposed IBEA is tested by solving a set of artificial test problems known to be challenging in the EMOO literature. As discussed by some researchers [20], algorithms able to solve test problems having different difficulties can be candidates to deal properly with real-life problems. Consequently, we present an analysis of the proposed algorithm solving three real-life applications where the fitness landscapes and Pareto fronts are unknown. A comparison is carried out to analyze the performance of the suggested IBEA versus three state-of-the-art IBEAs based on the Lebesgue measure. We show that the algorithmic proposal outperforms the state-of-the-art IBEAs in most test problems, including the three real-life problems considered in our study. The obtained results are statistically validated over a number of experiments performed as part of our experimental research.

The rest of the manuscript is organized as follows. Section 2 introduces the fundamental concepts to understand the content of this work. Section 3 introduces an overview of the related work to this investigation. Section 4 introduces the proposed algorithm and details its components. Section 5 presents an experimental study of performance on a set of test problems with complicated features. Section 6 introduces three real-life applications from practice in which the suggested algorithm is tested and analyzed with other IBEAs. Lastly, Section 7 presents our outcomes and describes some paths for future investigation.

2. General Background

This section provides the foundations of multi-objective optimization, introduces the indicator-based multi-objective evolutionary algorithms, and presents some concepts related to performance quality indicators.

2.1. Multi-Objective Optimization

Using standard notation and terminology, a multi-objective optimization problem (MOP) can be defined as follows.

Definition 1

(Multi-objective optimization problem). Without loss of generality, assuming minimization in all the objective functions, a multi-objective optimization problem can be defined as:

$$\begin{array}{cccc}\hfill & \underset{\mathbf{x}\in \mathcal{X}}{\mathrm{minimize}:}\hfill & & \mathbf{F}\left(\mathbf{x}\right)={(f_1\left(\mathbf{x}\right),\dots ,f_m\left(\mathbf{x}\right))}^T\hfill \\ \hfill & \mathit{subject}\mathit{to}\hfill & & g_i\left(\mathbf{x}\right)\le 0,\forall i\in \{1,\dots ,p\},\hfill \\ \hfill & & & h_j\left(\mathbf{x}\right)=0,\forall j\in \{1,\dots ,q\}.\hfill \end{array}$$

(1)

where $\mathbf{x}\in \mathcal{X}\subset {\mathbb{R}}^n$ is a solution to the problem, $\mathcal{X}$ is the solution space, and $f_i:\mathcal{X}\to \mathbb{R}$, for all $i\in \{1,\dots ,m\}$, are m objective functions. The constraint functions $g_i,h_j:\mathcal{X}\to \mathbb{R}$ restrict $\mathbf{x}$ to a feasible region ${\mathcal{X}}^{\prime }\subseteq \mathcal{X}$.

In multi-objective optimization, a set of trade-off solutions are normally aimed for, because the minimization of one objective function could lead to the deterioration of the others. To describe the concept of optimality in which we are interested in, the following definitions are presented.

Definition 2

(Pareto dominance). Let $\mathbf{x},\mathbf{y}\in {\mathcal{X}}^{\prime }$. We say that $\mathbf{x}$ weakly dominates $\mathbf{y}$ ($\mathbf{x}⪯\mathbf{y}$) if $f_i\left(\mathbf{x}\right)\le f_i\left(\mathbf{y}\right)$ for all $i\in \{1,\dots ,m\}$. If, in addition $\mathbf{y}⋠\mathbf{x}$, we say that $\mathbf{x}$ strictly dominates $\mathbf{y}$ ($\mathbf{x}\prec \mathbf{y}$). If $f_i\left(\mathbf{x}\right)<f_i\left(\mathbf{x}\right)$ for all $i\in \{1,\dots ,m\}$, we say that $\mathbf{x}$ strongly dominates $\mathbf{y}$ ($\mathbf{x}\prec \prec \mathbf{y}$).

Definition 3

(Pareto optimality). Let ${\mathbf{x}}^{\star }\in {\mathcal{X}}^{\prime }$. We say that ${\mathbf{x}}^{\star }$ is a Pareto optimal solution if there is no other solution $\mathbf{y}\in \mathcal{X}$ such that $\mathbf{y}⪯{\mathbf{x}}^{\star }$.

Definition 4

(Pareto optimal set). The Pareto optimal set $\mathcal{PS}$ of a multi-objective problem is defined by $\mathcal{PS}=\{\mathbf{x}\in {\mathcal{X}}^{\prime }|\mathbf{x}$ is Pareto optimal solution}.

Definition 5

(Pareto optimal front). The Pareto optimal front$\mathcal{PF}$ of a multi-objective problem is stated by the image of the Pareto optimal set, that is, $\mathcal{PF}=\left\{\mathbf{F}\right(\mathbf{x}\left)\right|\mathbf{x}\in \mathcal{PS}\}$.

An interesting property observed in continuous multi-objective problems that has been considered when designing multi-objective algorithms is presented below.

Property 1

(Regularity property of continuous MOPs). From the Karush–Kuhn–Tucker conditions, it can be induced that under certain assumptions, the $\mathcal{PS}$ ($\mathcal{PF}$) of a continuous MOP with m objectives defines an ($m-1$)-dimensional piecewise continuous manifold in the decision space (objective space) [21,22].

The regularity property of continuous MOPs defined above was firstly employed by Hillermeier [22] in the well-known continuation methods for multi-objective optimization. As identified by some authors [18], multi-objective solvers should take into account this property explicitly or implicitly.

A critical condition of a multi-objective optimization problem is the conflict among its objectives. If there is no conflict among the objectives, then the problem could be solved by the optimization of each objective function independently. Although several authors have given distinct definitions for the relation between pairs of objectives, the following definition will be used in this paper.

Definition 6

(Conflict relation). Let $\mathcal{S}\subset {\mathcal{X}}^{\prime }$. According to Carlsson and Fullér [23], two objective functions $f_i$ and $f_j$ can be related in the following three ways (assuming minimization):

• $f_i$ is in conflict with $f_j$ on $\mathcal{S}$ if $f_i\left(\mathbf{x}\right)\le f_i\left(\mathbf{y}\right)\Rightarrow f_j\left(\mathbf{x}\right)\ge f_j\left(\mathbf{y}\right)\forall \mathbf{x},\mathbf{y}\in \mathcal{S}$;
• $f_i$ supports $f_j$ on $\mathcal{S}$ if $f_i\left(\mathbf{x}\right)\ge f_i\left(\mathbf{y}\right)\Rightarrow f_j\left(\mathbf{x}\right)\ge f_j\left(\mathbf{y}\right)\forall \mathbf{x},\mathbf{y}\in \mathcal{S}$;
• $f_i$ and $f_j$ are independent on $\mathcal{S}$, otherwise.

2.2. Indicator-Based Evolutionary Algorithms for Multi-Objective Optimization

Quality indicators have been introduced to compare the outcomes of multi-objective algorithms in a quantitative manner. They map a Pareto front approximation to a scalar number that quantifies the performance of a multi-objective approach.

Definition 7

(Quality indicator). An n-ary quality indicator $\mathcal{I}$ is a function $\mathcal{I}:{\mathcal{F}}^n\to \mathbb{R}$, which assigns each vector $({\mathcal{A}}_1,\dots ,{\mathcal{A}}_n)$ of n approximation sets (which can be singletons) a real value $I({\mathcal{A}}_1,\dots ,{\mathcal{A}}_n)$.

Currently, we can find a large number of quality indicators for multi-objective optimization. A comprehensive compilation of them can be found in [5,24,25,26]. Quality indicators can assess convergence and diversity of solutions along the Pareto front of a given MOP. However, some indicators require certain knowledge of the problem which, in many cases, is not available. For example, quality indicators based on reference sets (e.g., R2, $\epsilon$ indicator, IGD, etc.) require a discretization of the entire Pareto front.

Although quality indicators were initially employed for comparison purposes of multi-objective solvers, their use has been extended to guide the optimization process in EMOAs. In this way, with its emergence in the early 2000s, the indicator-based evolutionary algorithm (IBEA) [7] posed the possibility to optimize a quality indicator to approximate the Pareto front of a MOP.

Let us consider the $(\mu +\lambda )$-selection scheme of an EMOA and the combined population $Q_t$ of $\mu$ parents and $\lambda$ offspring. In order to choose the best $\mu$ solutions for the next population (i.e., the updated set of parents), the fitness assignment to each individual is necessary. Traditionally, EMOAs employ the Pareto ranking and a diversity indicator to update the parents set. The selection mechanism in IBEAs consists of finding the solution that contributes the least to the indicator under consideration. Making allowance for the fitness value $\phi \left({\mathbf{q}}^i\right)$ of an individual ${\mathbf{q}}^i\in Q_t$, Algorithm 1 shows the survival selection mechanism in IBEAs.

Algorithm 1: IBEAs survival selection mechanism.

In the sense of evolutionary algorithms, the higher the fitness value $\phi \left({\mathbf{q}}^i\right)$, the better the individual ${\mathbf{q}}^i$. Note, however, that according to their conceptual foundations, quality indicators can be either maximized or minimized. Therefore, the adequate fitness values assignment depends directly on the concerned quality indicator. Note that besides the computational time required to estimate the worst solution in IBEAs could be considerably costly (line 2 in Algorithm 1). To date, in the EMOO literature, we can find several evolutionary algorithms based on various quality indicators. A comprehensive review of these types of algorithms can be found in [27].

2.3. Performance Quality Indicators

As pointed out, performance indicators are employed in IBEAs and are also employed to compare performance between algorithms. In the follows, we present some relevant performance indicators in this study.

2.3.1. Hypervolume Performance Indicator

The hypervolume performance indicator, as well known as Lebesgue measure or S metric, has been employed to guide the search in evolutionary algorithms for multi-objective optimization. The follow definitions are relevant in this work [28,29].

Definition 8

(Hypervolume indicator). Let $S\subset {\mathbb{R}}^m$ and $\mathbf{r}\in {\mathbb{R}}^m$ be a point set and a reference point, respectively. The hypervolume indicator of S is the measure of the region weakly dominated by S and bounded by vector $\mathbf{r}$. Formally:

$$\mathcal{H}(S,\mathbf{r})=\mathcal{L}\left(\{\mathbf{q}\in {\mathbb{R}}^m|\exists \mathbf{p}\in S:\mathbf{p}⪯\mathbf{q}\mathit{and}\mathbf{q}⪯\mathbf{r}\}\right)$$

(2)

where $\mathcal{L}(·)$ refers to the Lebesgue measure.

Definition 9

(Hypervolume Contribution). The exclusive hypervolume contribution of a solution $\mathbf{q}\in {\mathbb{R}}^m$ to a set $S\in {\mathbb{R}}^m$ respect to the reference vector $\mathbf{r}$, is defined as:

$${\mathcal{H}}_c(S,\mathbf{q},\mathbf{r})=\mathcal{H}(S\cup \left\{\mathbf{q}\right\},\mathbf{r})-\mathcal{H}(S\backslash \left\{\mathbf{q}\right\},\mathbf{r})$$

(3)

The hypervolume contribution of a point is sometimes referred to as Lebesgue contribution, incremental, or exclusive hypervolume contribution. In this regard, some contributions to the state of the art on this topic can be found in [30,31,32].

In the specialized literature, we can find several issues addressed by investigators in relation to the hypervolume indicator, see the comprehensive review on this topic presented in [33]. However, one of the most important challenges in this research area is the exact computation of the hypervolume indicator on a point set S. In this regard, some researchers have designed algorithms that are efficient in a few dimensions, see the works reported in [31,34,35]. The computational complexity of the hypervolume computation is exponential to the number of points in S [19]. An interesting property observed in the two-dimensional objective space that has been exploited for a fast hypervolume computation is presented below.

Property 2

(Locality property of the hypervolume indicator). Given three consecutive points on the Pareto front, moving the middle point will only affect the hypervolume contribution that is solely dedicated to this point, but the joint hypervolume contribution of the other points remains fixed [19].

Nonetheless, the challenges presented in high-dimensional objective spaces have motivated a vast research in the design of algorithms for the efficient hypervolume computation. As a flavor of approaches devoted to the exact hypervolume calculation generalized in the number of dimension,

Table 1. Algorithms for the exact hypervolume computation on an m-dimensional set of n points.

Algorithm	Dimension	Computational Complexity
HSO [36]	$m\ge 2$	$O\left(n^{m-1}\right)$
LebMeasure [37]	$m\ge 2$	$O\left(n^m\right)$
FPL [38]	$m\ge 2$	$O(n^{m-2}logn)$
HOY [39]	$m\ge 2$	$O(n^{d/2}logn)$
WFG [40]	$m\ge 2$	$\mathrm{\Omega }(n^{m/2}logn)$
QHV [41]	$m\ge 2$	$O\left(2^{m(n-1)}\right)$

presents some algorithms known by the EMOO community and their complexities for an m-dimensional set of n points.

2.3.2. Normalized Hypervolume Indicator

The $\mathcal{H}$ indicator (stated in Definition 8) can quantify convergence and distribution of solutions on the $\mathcal{PF}$ of a given problem. The normalized hypervolume can be defined as follows.

Definition 10

(Normalized hypervolume indicator). Let $S\subset {\mathbb{R}}^m,\mathbf{u}\in {\mathbb{R}}^m$ and $\mathbf{r}\in {\mathbb{R}}^m$ be a point set, an ideal point, and a reference point, respectively, such that $\mathbf{u}⪯\mathbf{s}⪯\mathbf{r}$ (for all $\mathbf{s}\in S$). The normalized hypervolume indicator of S is the measure of the region weakly dominated by S and bounded by vector $\mathbf{u}$ and $\mathbf{r}$. Mathematical it can be stated as:

$${\displaystyle {\mathcal{H}}_n(S,\mathbf{u},\mathbf{r})=\frac{\mathcal{H}(S,\mathbf{r})}{{\mathrm{\Pi }}_{i=1}^m|r_i-u_i|}}$$

(4)

where $\mathcal{H}(S,\mathbf{r})$ denotes the hypervolume indicator of S with reference vector $\mathbf{r}$.

The ${\mathcal{H}}_n$ indicator value is in the range $[0,1]$. In this way, a large ${\mathcal{H}}_n$ value indicates that the set of solutions S has a suitable approximation and spread on the real $\mathcal{PF}$.

2.3.3. $IG{D}^{+}$ Indicator

The inverted generational distance plus ($IG{D}^{+}$) [42] is an extension of the $IGD$ indicator [6]. This quality indicator is weakly Pareto compliant and it can quantify how far a given approximation set is from the real Pareto front. Formally, the $IG{D}^{+}$ indicator is stated as follows.

Definition 11

(Inverted Generational Distances plus). Let $F\in {\mathbb{R}}^m$ and $S\in {\mathbb{R}}^m$ be a discretization of the real Pareto front of a given MOP and a set of objective vectors given by an algorithm, respectively. The $IG{D}^{+}$ quality indicator is stated as:

$$IG{D}^{+}(F,S)={\left(\frac{1}{\left|F\right|}\sum _{\mathbf{r}\in F}\underset{\mathbf{s}\in S}{min}{d}^{+}(\mathbf{r},\mathbf{s})\right)}^{1/p}$$

(5)

where $p=2$ and $d^{+}$ is defined by,

$$d^{+}(\mathbf{r},\mathbf{s})=\sqrt{\sum _{i=1}^m{(max\{s_i-r_i,0\})}^2}$$

(6)

where m is the number of objective functions of a given MOP.

A value of zero of the $IG{D}^{+}$ indicator notices that all the objective vectors obtained by an algorithm are on the true $\mathcal{PF}$.

3. Previous Related Work

The hypervolume indicator ($\mathcal{H}$), as well known as Lebesgue or S metric, is a quality indicator widely used to assess the performance of evolutionary multi-objective algorithms [26]. Its peculiar property—it is strictly Pareto compliant [5]—has motivated several investigators working on the design of IBEAs. It has been proved that given a finite search space and a reference point, maximizing the hypervolume indicator is equivalent to finding the Pareto optimal set of a given problem [37]. For this reason, several IBEAs have incorporated this indicator in their survival selection mechanism (see the comprehensive survey of approaches presented in [43]). Lebesgue indicator-based EMOAs need to compute the hypervolume contribution (${\mathcal{H}}_c$) of non-dominated objective vectors to estimate the worst solution in the current population. As pointed out before, the main disadvantage of the hypervolume indicator is its computation cost which increases exponentially with the number of objectives of the problem. Traditionally, EMOAs based on the $\mathcal{H}$ indicator need to compute the ${\mathcal{H}}_c$ of each individual in the population per iteration. Examples of these algorithms are SIBEA [44], SMS-EMOA [11], MO-CMA-ES [45], HMOPSO [46], FV-MOEA [14], LIBEA [15], among others. These approaches become impractical when dealing with many objective functions (more than three), employing large populations, or requiring a significant number of generations. Consequently, some authors have focused their investigation on reducing the computational complexity of methods to compute either the $\mathcal{H}$ or ${\mathcal{H}}_c$ [33,43]. Other alternatives studied by some researchers are the approximation methods to estimate the $\mathcal{H}$ or ${\mathcal{H}}_c$ [33,43]. In this regard, some authors have incorporated into their IBEAs, approximation methods to calculate ${\mathcal{H}}_c$. A pioneering study adopting this idea is the HypE algorithm introduced in [47]. Another example of these types of approaches is the R2HCA-EMOA [48], which works similar to SMS-EMOA, but it uses the R2-based hypervolume contribution approximation method [49]. Experimental results presented by the authors show that it outperforms the HypE algorithm in terms of $\mathcal{H}$. Although the approximation methods have decreased the computational cost of IBEAs based on the Lebesgue measure, the performance quality in these algorithms is compromised. This is, in effect, the price to compensate for efficiency in these types of IBEAs.

In this paper, we are interested in designing IBEAs based on the exact hypervolume computation. In this regard, Menchaca-Mendéz and Coello [50] presented an improved version of SMS-EMOA called iSMS-EMOA. iSMS-EMOA generates an offspring per iteration. After that, the nearest individual to the offspring (measured by the Euclidean distance in the objective space) and another randomly selected individual compete with the offspring to survive (comparing their ${\mathcal{H}}_c$). Therefore, iSMS-EMOA only needs to compute three hypervolume contributions per iteration, unlike SMS-EMOA that calculates n contributions, where n is the population size. The core idea of iSMS-EMOA is to move a solution within its neighborhood to improve its ${\mathcal{H}}_c$. This idea is based on the locality property stated in [19] (see Property 2). iSMS-EMOA significantly improves the efficiency of SMS-EMOA, and it achieves comparable performance to SMS-EMOA. In [51], the authors studied the behavior of iSMS-EMOA using the approximation method to calculate ${\mathcal{H}}_c$ proposed by Bringmann and Friedrich [52]. The experimental results show that this version of iSMS-EMOA outperforms HypE. In [53], the authors studied the behavior of iSMS-EMOA if it does not use the randomly selected individual in the competition always. Rostami and Neri [54] proposed the algorithm CMA-PAES-HAGA, which incorporates a fast hypervolume-driven selection mechanism for many-objective optimization called HAGA to CMA-ES. HAGA divides the objective space into grids. Then, it separates the population into subpopulations (each grid contains one subpopulation). When a new individual is created, it only competes with the individuals in its grid. Experimental results show that CMA-PAES-HAGA is able to solve problems with more than three objective functions. Recently, Zapotecas-Martínez et al. [15] introduced a novel Lebesgue-based IBEA (LIBEA) adopting the regularity property of continuous MOPs (see Property 1). The introduced LIBEA employs different neighborhoods for the mating selection mechanism. In this way, if a solution is close to the $\mathcal{PS}$ of a problem, it is possible to create new solutions close to the $\mathcal{PS}$ recombining with neighboring solutions. The authors show the effectiveness of LIBEA when solving continuous MOPs with roughed Pareto optimal sets.

In contrast to the related work, we introduce an improved multi-objective solver considering the Lebesgue measure, the regularity property of continuous MOPs, and the local property of the Lebesgue measure. We investigate a new framework to solve problems with difficult features and unknown fitness landscapes more efficiently than traditional IBEAs based on the Lebesgue measure. In the next section, we describe the components of the proposed algorithm thoroughly.

4. Improved Evolutionary Multi-Objective Algorithm Based on the Lebesgue Indicator

The proposed algorithm presented in this paper is an improvement of the Lebesgue indicator-based evolutionary algorithm (LIBEA) [15] for multi-objective optimization. In analogy to LIBEA, the suggested algorithm addresses the notion of IBEA [7] regarding the optimization of a quality indicator. Nevertheless, it is directed at maximizing the Lebesgue indicator of non-dominated solutions obtained through the search. The differences are clearly observed between our algorithmic proposal and IBEAs adopting the Lebesgue measure. This section introduces details of the new algorithm and its components to be compared against state-of-the-art IBEAs.

4.1. Framework of the Improved Lebesgue Indicator-Based Evolutionary Algorithm

Analogous to its predecessor (LIBEA [15]), the proposed algorithm implicitly adopts the regular property of continuous MOPs to approximate solutions towards the Pareto front of a given problem. The framework of the improved LIBEA (namely here LIBEA-II) is presented in Algorithm 2. Initially, a set $P_t=\{{\mathbf{x}}^1,\dots ,{\mathbf{x}}^N\}$$(t=0)$ of N candidate solutions is generated randomly (Algorithm 2, line 2). A matrix $\mathbf{D}$ allocating the distances between pairs of objective vectors is calculated and used in the parent selection mechanism of LIBEA-II (Algorithm 2, line 3). At each iteration, for each candidate solution ${\mathbf{x}}^i\in P$, a parent solution $\mathbf{y}$ is selected according to the mating selection mechanism (Algorithm 2, line 5). Thus, the recombination procedure is performed by employing the current solution ${\mathbf{x}}^i$ and the parent solution $\mathbf{y}$ (Algorithm 2, line 6). Section 4.3 illustrates different recombination models that could be adopted into LIBEA-II. Finally, in line 8 of Algorithm 2, a the new population $P_{t+1}$ is updated employing the current population $P_t$ and the offspring solution ${\mathbf{y}}^{\prime }$ according to the survival selection mechanism described in Section 4.4. In the following, the rest of the components of LIBEA-II are described.

Algorithm 2: LIBEA-II Framework.

4.2. Mating Selection Mechanism

The regularity property of continuous MOPs, establish that, under certain conditions, the $\mathcal{PS}$ ($\mathcal{PF}$) of a continuous MOP with m objectives defines an ($m-1$)-dimensional piecewise continuous manifold in the decision space (objective space) [21,22]. Although this property was firstly introduced by Hillermeier [22] to solve multi-objective problems, its use has been adopted by several EMOAs based on different natures (see for example, the approaches reported in [15,16,17,55,56]). LIBEA-II adopts the regularity property of continuous MOPs in an implicit form by promoting the recombination between neighboring solutions. In this way, if a solution ${\mathbf{x}}^i$ and its neighbors are close to the $\mathcal{PS}$ ($\mathcal{PF}$), the new offspring solution should also be close to the $\mathcal{PS}$ ($\mathcal{PF}$). In other words, the local manifold approximated by solution ${\mathbf{x}}^i$ and its neighbors should generate a new solution also close to the $\mathcal{PS}$ ($\mathcal{PF}$). In the following, the mating selection mechanism of LIBEA-II is described.

Let $C_i\subset P_t$ be the solutions set of the closest solutions to ${\mathbf{x}}^i$ (in the space of the objective functions). LIBEA-II uses a probability $\delta$ to select the solutions set ($\beta$) to be taken into account in the recombination procedure. In the proposed approach, the solutions set $\beta$ is stated by either the neighboring solutions to ${\mathbf{x}}^i$ or the solutions in $P_t$ according to a probability $\delta$. More precisely:

$$\begin{array}{cc}\hfill \beta & =\left\{\begin{array}{cc}{C}_i,\hfill & rand(0,1)<\delta ,\hfill \\ P_t,\hfill & otherwise.\hfill \end{array}\right.\hfill \end{array}$$

(7)

In this way, the parameter $\delta$ denotes the probability of picking a neighboring solution to be recombined with solution ${\mathbf{x}}^i$. Otherwise, with probability $1-\delta$, any other solution taken from the whole population $P_t$ can be chosen for recombination.

Once the solutions set $\beta$ is stated, a parent solution $\mathbf{y}\ne {\mathbf{x}}^i$ is chosen randomly from $\beta$. It is worth noticing that LIBEA-II keeps a distance matrix $\mathbf{D}$ updated during the search process (we refer to Section 4.4 for more details). Therefore, the solutions set $C_i$ can be computed by employing the partial sorting algorithm [57] with a computational complexity of $O(N+TlogN)$, such that T denotes the number of desirable solutions in $C_i$ and N represents the number of solutions in $P_t$.

4.3. Recombination Mechanism

LIBEA-II can be seen as a framework that allows incorporating any recombination mechanism available in the evolutionary computation research area. Nonetheless, it is worth mentioning that for certain recombination operators coming from some meta-heuristics (e.g., PSO [58], DE [59], etc.), consider more than one solution. In such cases, the mating selection mechanism should produce more than one solution for the concerned recombination operator. That is, instead of picking one solution $\mathbf{y}$ from $\beta$, various solutions ${\mathbf{y}}^1,{\mathbf{y}}^2,\dots \in \beta$, such that ${\mathbf{y}}^1\ne {\mathbf{y}}^2\ne \dots \ne {\mathbf{x}}^i$ have to be selected. In order to exemplify the mating selection and recombination procedures in the LIBEA-II framework, two popular operators coming from the evolutionary computing field are illustrated below.

Operators from Genetic Algorithms.

Genetic algorithms employ crossover and mutation operators to create offspring solutions throughout the search process. In LIBEA-II, an offspring solution ${\mathbf{y}}^{\prime }$ can be created employing such operators according to following equation

$${\mathbf{y}}^{\prime }=MUT\left(CX({\mathbf{x}}^i,\mathbf{y})\right)$$

(8)

where $\mathbf{y}$ is a solution randomly chosen from $\beta$, $CX$, and $MUT$ are the crossover and mutation operators, respectively. Therefore, LIBEA-II could adopt operators for combinatorial, integer, or mixed optimization.

Operators from Differential Evolution.

A recombination method employed for solving MOPs with difficult $\mathcal{PS}$ [17] exhibiting good results, is the differential evolution (DE) operator [58]. In LIBEA-II, an offspring solution ${\mathbf{y}}^{\prime }$ can be created employing DE operator according to the following equation:

$${\mathbf{y}}^{\prime }=CX({\mathbf{x}}^i,\mathbf{u})$$

(9)

such that $CX$ is the DE crossover, $\mathbf{u}={\mathbf{y}}^1+F({\mathbf{y}}^2-{\mathbf{y}}^3)$ is the perturbed vector, where ${\mathbf{y}}^1,{\mathbf{y}}^2$ and ${\mathbf{y}}^3$ are solutions chosen randomly from $\beta$ with, ${\mathbf{y}}^1\ne {\mathbf{y}}^2\ne {\mathbf{y}}^3\ne {\mathbf{x}}^i$, and F denotes the differential factor, respectively. After performing the DE crossover, a mutation mechanism can be applied to improve the search capabilities, as has been employed by a few researchers [17,18].

4.4. Survival Selection Mechanism

The survival selection mechanism in LIBEA-II (line 8 in Algorithm 2) chooses the best solutions from $Q=P_t\cup \left\{\mathbf{y}\right\}$ considering either the Pareto dominance relation or the Lebesgue measure. Since the number of solutions in Q is $N+1$, it is necessary to remove one solution from Q to make way for the next iteration.

Let $d(\mathbf{q},Q)$ and $P^{\star }$ be the number of solutions from Q that dominate solution $\mathbf{q}\in Q$, and the set of non-dominated solutions in Q, respectively. That is,

$$\begin{array}{ccc}\hfill d(\mathbf{q},Q)& =& \left|\right\{\mathbf{p}\in Q\left|\mathbf{F}\right(\mathbf{p})\prec \mathbf{F}(\mathbf{q}),\mathbf{F}(\mathbf{p})\ne \mathbf{F}(\mathbf{q}\left)\right\}|\hfill \end{array}$$

(10)

$$\begin{array}{ccc}\hfill P^{\star }& =& \{\mathbf{q}\in Q|\nexists \mathbf{p}\in Q:\mathbf{F}\left(\mathbf{p}\right)\prec \mathbf{F}\left(\mathbf{q}\right)\}\hfill \end{array}$$

(11)

Traditionally, IBEAs based on the Lebesgue indicator employ Pareto ranking [60] followed by computing the exclusive Lebesgue contribution (see Definition 2) of each solution allocated in the last rank (see for example the algorithmic proposals introduced in [11,15,61,62]). Therefore, if the last rank contains a large number of solutions, the computational complexity to estimate the solution to be removed becomes extremely high. In evolutionary many-objective optimization, it can be observed that a population constituted by non-dominated solutions can be preserved for several iterations of an algorithm. Therefore, a high computational time is required to decide which solution should be removed from the population. LIBEA-II considers the following two scenarios in the survival selection mechanism.

• If $P^{\star }\ne Q$, there are solutions in Q dominated by some solution in $P^{\star }$. In such a case, we shall remove the solution with the largest $d(\mathbf{q},Q)$ value avoiding the Lebesgue measure computation and reducing the computational cost of LIBEA-II;
• If $P^{\star }=Q$, all solutions in Q are non-dominated, and all of them are equally acceptable in terms of the Pareto dominance relation. In such a case, we shall remove the solution ${\mathbf{x}}^j\in S\subseteq Q$ that maximizes the contribution to the Lebesgue measure. In other words, the solution to be removed is the one such that:

  $${\mathbf{x}}^j=\mathtt{arg}{\mathtt{min}}_{\mathbf{s}\in S}{\mathcal{H}}_c(S,\mathbf{s},\mathbf{r})$$

  (12)
  Therefore, a total of $\left|S\right|$ Lebesgue measures are required to identify the worst solution (i.e., the solution that contributes the least to the Lebesgue measure) in the population.

Note that in the case of $S=Q$, the worst solution is found after $N+1$ Lebesgue measures. This is, in fact, computationally expensive and impractical in many-objective optimization problems. LIBEA-II saves Lebesgue measures by reducing the number of candidate solutions in the set S.

A problem observed in IBEAs based on the Lebesgue indicator is the overestimation of the reference vector, which could divert the search. Although the correct estimation of the reference vector for certain types of $\mathcal{PF}$ has been discussed [63], there is no strategy to correctly define this vector for all $\mathcal{PF}$ forms. In such a case, a reference vector close to the nadir vector (the vector opposite the ideal vector) should properly measure the coverage and distribution of solutions along the $\mathcal{PF}$, including the extreme portions of it. Therefore, the solutions that provide information on the nadir vector should be considered in the survival selection mechanism. In other words, the solutions whose objectives vectors are the farthest to the ideal vector should be considered.

The following criteria to define the set S (line 5 in Algorithm 3) are based on the locality property of the hypervolume indicator studied in [19] (the first three criteria) and the problem to estimate the reference vector for computing the Lebesgue indicator discussed in [63] (the last criterion). The set of candidate solutions S is composed by:

• The offspring solution $\mathbf{y}$;
• The solution $\mathbf{q}\in Q$ such that the objective vector $\mathbf{F}\left(\mathbf{q}\right)$ is the closest to the objective vector $\mathbf{F}\left(\mathbf{y}\right)$;
• A percentage (${\rho }_c$) of solutions $\mathbf{q}\in Q$ which objective vectors are the closest;
• A percentage (${\rho }_n$) of solutions $\mathbf{q}\in Q$ which objective vectors are the farthest to the ideal vector $\mathbf{z}$, where the ideal vector $\mathbf{z}={(z_1,\dots ,z_m)}^T$ is estimated by $z_j={min}_{\mathbf{x}\in Q}{f}_j\left(\mathbf{x}\right)$ for all $j\in \{1,\dots ,m\}$.

In the case that the offspring solution $\mathbf{y}$ is accepted, it shall replace solution ${\mathbf{x}}^j\in Q$ (${\mathbf{x}}^j\leftarrow \mathbf{y}$) and the distance matrix $\mathbf{D}$ has to be updated calculating the Euclidean distances between the objective vector $\mathbf{F}\left({\mathbf{x}}^j\right)$ and each objective vector $\mathbf{F}\left({\mathbf{x}}^i\right)$, that is:

$$D_{ij}\leftarrow \left|\right|\mathbf{F}\left({\mathbf{x}}^i\right)-\mathbf{F}\left({\mathbf{x}}^j\right){\left|\right|}_2,\forall {\mathbf{x}}^i\in P_{t+1}$$

In order to deal with different scales (in the objective space), LIBEA-II considers objective vectors normalized in the hypercube bounded by the ideal ($\mathbf{z}$) and the nadir ($\mathbf{n}$) vectors. In such cases, the ideal and nadir vectors are defined by the smallest and the largest values of each objective function found in the set of solutions $Q\cup \left\{\mathbf{y}\right\}$. Therefore, the Lebesgue measure is computed employing the normalized objective vectors and considering the reference vector $\mathbf{r}={(1.1,\dots ,1.1)}^T$. In Algorithm 3, we show the complete survival selection mechanism of LIBEA-II.

Algorithm 3:$Environmental Selection\left(Q\right)$.

5. Experimental Study

This section presents the experimental setup and the analysis of results. Firstly, the IBEAs and the benchmark problems adopted for comparison are introduced. Then, the experimental settings are given. Finally, the results on the benchmark problems are analyzed.

5.1. IBEAs Considered for Comparison of Performance

The performance of the proposed LIBEA-II is compared with respect to state-of-the-art IBEAs based on the hypervolume quality indicator. In the first instance, we adopt the S-metric selection EMOA (SMS-EMOA) [11] for performance comparison. SMS-EMO employs a survival mechanism based on Pareto ranking joined by the exclusive Lebesgue contribution of each solution located in the last rank. This evolutionary algorithm has shown its effectiveness and has become popular among state-of-the-art IBEAs. Another algorithm contemplated in this investigation is the improved version of the SMS-EMOA (iSMS-EMOA) [50], which adopts ideas coming from the local property of the hypervolume indicator. Finally, the Lebesgue indicator-based algorithm (LIBEA) [15] for multi-objective optimization is selected. As noticed before, LIBEA adopts the regularity property of continuous MOPs in the mating selection mechanism. Since the proposed LIBEA-II also employs this regularity property, its predecessor, LIBEA, is an obvious competing algorithm.

5.2. Adopted Test Problems

The study presented in this investigation considers the continuous box-constrained MOPs with difficult Pareto sets introduced in [64]. These problems are part of the CEC’2009 competition related to the performance of evolutionary multi-objective algorithms. The adopted test problems have been formulated to assess the performance of EMOAs solving continuous MOPs that exhibit the property of complicated $\mathcal{PS}$ topologies. Since this property has been seen in real-life problems [17], this test suite is a challenge to evaluate the performance of our algorithmic proposal. The adopted test problems (as well known as UFs) offer diverse characteristics regarding separability, multi-modality, and include different $\mathcal{PF}$ shapes, incorporating discontinuities, concavity, convexity, etc. More precisely, we consider the two-objective problems UF1–UF7 and the three-objective problems UF8–UF10.

5.3. Experimental Settings

As pointed out, the results achieved by our proposed algorithm (i.e., LIBEA-II) are analyzed versus those obtained by SMS-EMOA, iSMS-EMOA, and LIBEA on the test problems with roughed $\mathcal{PF}$ (UF1–UF10). As discussed by some authors, MOPs exhibiting complicated $\mathcal{PF}$ shapes shall test specific components of EMOAs, such as the parent selection mechanism and the recombination operators [15,17]. In this work, we use the DE operator whose effectivity has been proved in MOPs with this singular characteristic (see the studies reported in [17]). Therefore, all the IBEAs adopted for performance comparison employ DE operator as their main recombination procedure, such as described in Section 4.3. In order to improve the search capabilities after performing the DE operator, the Polynomial-based mutation [65] is implemented. However, note that for other test problems such as ZDT [66] and DTLZ [67], the performance of IBEAs could be improved by using recombination operators from genetic algorithms, for example, Simulated Binary Crossover (SBX) and Polynomial-based mutation (PBM) [65]. The parameters used by the IBEAs are presented in

Table 2. Parameter settings for SMS-EMOA, iSMS-EMOA, LIBEA, and LIBEA-II.

Parameter	SMS-EMOA	iSMS-EMOA	LIBEA	LIBEA-II
N	100	100	100	100
$G_{max}$	2000	2000	2000	2000
F	0.5	0.5	0.5	0.5
$CR$	1	1	1	1
$P_m$	$1/n$	$1/n$	$1/n$	$1/n$
${\eta }_m$	20	20	20	20
T	—	—	20	20
$\delta$	—	—	0.9	0.9
${\rho }_c$	—	—	—	0.1
${\rho }_n$	—	—	—	0.1

, where N denotes the population size. $G_{max}$ represents the maximum number of generations, in our study $G_{max}=2000$. Therefore, the search process was limited to performing 200,000 fitness function evaluations. In the case of the DE operator, F and $CR$ denote the differential amplitude factor and the crossover rate, which were set as guested in [17] to solve MOPs with complicated $\mathcal{PS}$. $P_m$ and ${\eta }_m$ are the mutation probability and the mutation index used by PBM, respectively. For LIBEA and LIBEA-II, T and $\delta$ are the neighborhood size and the probability of picking solutions from a determined neighborhood (see Section 4.2), respectively. Note that the smaller the $\delta$ value, the effect of the regular property of continuous MOPs is diluted. For LIBEA-II, ${\rho }_c$ and ${\rho }_n$ are parameters that define the percentage of solutions to be considered in the survival selection mechanism (see Section 4.4). It is worth emphasizing that the smaller these parameters values (${\rho }_c$ and ${\rho }_n$), the more efficient LIBEA-II is.

For each IBEA, 30 executions were independently performed on each MOP. The IBEAs were assessed employing the ${\mathcal{H}}_n$ and $IG{D}^{+}$ quality indicators presented in Section 2.3. For each test problem, a statistical analysis was performed over the final approximation produced by the IBEAs in all the experiments using the concerned quality indicator. Since the properties of the UF test functions are known, the ${\mathcal{H}}_n$ quality indicator was calculated by employing the reference vector $\mathbf{r}={(1.1,\dots ,1.1)}^T$ and the ideal vector $\mathbf{u}={(0,\dots ,0)}^T$. Therefore, a reliable measure of approximation and distribution of solutions obtained by the algorithms along the Pareto front is reported. The $IG{D}^{+}$ indicator was calculated by employing the reference sets provided by the authors of the UF test functions.

5.4. Analysis of Results on the UF Test Problems

The non-dominated solutions found by LIBEA-II, and those from SMS-EMOA, iSMS-EMOA, and LIBEA, to each UF test function, were subjected to the ${\mathcal{H}}_n$ and $IG{D}^{+}$.

Table 3. Average ${\mathcal{H}}_n$ values for the non-dominated solutions found by the IBEAs to each UF test problem.

MOP	SMS-EMOA	iSMS-EMOA	LIBEA	LIBEA-II
UF1	0.6674 ± 0.010	0.6654 ± 0.010	0.6654 ± 0.009	0.6669 ± 0.008
UF2	0.7048 ± 0.001	0.7039 ± 0.002	0.7051 ± 0.001	0.7054 ± 0.002
UF3	0.6057 ± 0.028	0.6050 ± 0.029	0.6117 ± 0.023	0.6108 ± 0.025
UF4	0.2779 ± 0.007	0.2792 ± 0.006	0.2781 ± 0.006	0.2761 ± 0.006
UF5	0.0087 ± 0.015	0.0143 ± 0.034	0.0154 ± 0.024	0.0175 ± 0.021
UF6	0.3137 ± 0.054	0.3001 ± 0.047	0.3329 ± 0.037	0.3319 ± 0.043
UF7	0.5595 ± 0.004	0.5511 ± 0.039	0.5608 ± 0.004	0.5489 ± 0.039
UF8	0.3539 ± 0.041	0.3563 ± 0.038	0.3641 ± 0.054	0.3667 ± 0.052
UF9	0.7582 ± 0.012	0.7464 ± 0.011	0.7550 ± 0.015	0.7366 ± 0.025
UF10	0.0000 ± 0.000	0.0000 ± 0.000	0.0000 ± 0.000	0.0000 ± 0.000

and

Table 4. Average $IG{D}^{+}$ values for the non-dominated solutions found by the IBEAs to each UF test problem.

MOP	SMS-EMOA	iSMS-EMOA	LIBEA	LIBEA-II
UF1	0.0341 ± 0.006	0.0354 ± 0.005	0.0350 ± 0.005	0.0340 ± 0.005
UF2	0.0142 ± 0.001	0.0148 ± 0.001	0.0141 ± 0.001	0.0140 ± 0.001
UF3	0.0709 ± 0.025	0.0696 ± 0.021	0.0666 ± 0.018	0.0665 ± 0.019
UF4	0.1232 ± 0.006	0.1223 ± 0.005	0.1230 ± 0.005	0.1252 ± 0.005
UF5	0.7429 ± 0.101	0.7470 ± 0.133	0.7232 ± 0.109	0.7106 ± 0.098
UF6	0.2163 ± 0.048	0.2221 ± 0.061	0.2005 ± 0.036	0.2028 ± 0.045
UF7	0.0149 ± 0.002	0.0229 ± 0.035	0.0142 ± 0.002	0.0241 ± 0.036
UF8	0.1542 ± 0.029	0.1528 ± 0.027	0.1468 ± 0.038	0.1454 ± 0.037
UF9	0.0518 ± 0.007	0.0563 ± 0.008	0.0517 ± 0.007	0.0642 ± 0.017
UF10	1.3045 ± 0.222	1.2999 ± 0.227	1.2335 ± 0.203	1.3563 ± 0.207

show the average ${\mathcal{H}}_n$ and $IG{D}^{+}$ values, respectively, over 30 repetitions for each UF problem. These tables have five columns: the first identifies the UF test function, and the remaining four correspond to each of the four algorithms under comparison. The best average ${\mathcal{H}}_n$ and $IG{D}^{+}$ values for each UF problem are in boldface. Moreover, in order to distinguish if there is a statistically significant difference among the average ${\mathcal{H}}_n$ and $IG{D}^{+}$ values for each test function, the Mann–Whitney–Wilcoxon [68] non-parametric statistical test, employing a p-value of $0.05$, and Bonferroni correction [69] were applied on them. In this manner, an algorithm can be considered the best regarding the test function and quality indicator if it statistically surpasses the other three. If this is the case, the value presented in the Table is underlined.

In Table 3, we can see the average results for the ${\mathcal{H}}_n$ indicator. As we can see, the performance of the four algorithms was very similar: SMS-EMOA obtained the best average results for two test problems, solutions from iSMS-EMOA were the best for one test problem, LIBEA found solutions that were the best to three test problems, and the solutions found by LIBEA-II were the best for three test problems. Actually, these results were expected since all four algorithms are based on the $\mathcal{H}$ indicator. However, something remarkable is that LIBEA-II was able to find statistically better solutions than those from the other three algorithms for test instance UF8.

${\mathcal{H}}_n$ quality indicator assesses, to some extent, the closeness and spreading of the non-dominated solutions obtained by an EMOA. Nevertheless, quality indicators based on reference sets could provide more information regarding how distant a set of solutions is from the real $\mathcal{PF}$. To this end, the $IG{D}^{+}$ indicator was selected to further evaluate the performance of the IBEAs. Table 4 presents the obtained results of the proposed LIBEA-II and those reached by the adopted IBEAs in terms of the $IG{D}^{+}$ indicator. It can be observed that the results achieved by LIBEA-II exceeded those obtained by SMS-EMOA, iSMS-EMOA, and LIBEA in five out of the ten test problems in terms of ${\mathcal{H}}_n$ indicator. LIBEA obtained the best average results in four test problems, while iSMS-EMOA was the best in only one. More importantly, LIBEA-II obtained results that are statistically better than the results from the other three algorithms in problem UF8.

Additionally to the quality indicators, Figure 1 shows the average convergence of the four algorithms under comparison. This figure contains ten plots, one for each UF test function, that show the convergence of the $IG{D}^{+}$ indicator for each algorithm. We can see that the convergence of the $IG{D}^{+}$ indicator is very similar for all four algorithms in each test problem, except for test problem UF7.

After these results, regarding the ${\mathcal{H}}_n$ and $IG{D}^{+}$ quality indicators, we can say that LIBEA-II performs slightly better on the UF test problems than SMS-EMOA, iSMS-EMOA, and LIBEA, since, despite solutions from LIBEA-II, have comparable hypervolumes, they are closer to the $\mathcal{PF}$ in more benchmark functions. Moreover, for the ${\rho }_c$ and ${\rho }_n$ parameters adopted in this study, LIBEA-II reduces up to approximately $80\%$ the hypervolume indicator calculations, as shown in Figure 2. This means that LIBEA-II is more efficient than the other three algorithms since, with fewer computational resources, it can find solutions with as high quality as those found by SMS-EMOA, iSMS-EMOA, and LIBEA.

In order to illustrate the Pareto approximations obtained by the algorithms, Figure 3 presents the non-dominated solutions found by the four algorithms under consideration to the test problems UF1, UF3, and UF7. It is clear that no algorithm could find a proper approximation set to any of the three test problems. However, solutions from SMS-EMOA and LIBEA-II show the best approximations to the Pareto front.

6. Three Real-World Applications from Practice

After testing LIBEA-II on the UF benchmark test problems, it is now tested on three real-world applications. This section introduces, firstly, the three real-world applications under consideration. Secondly, the experimental setup is presented. Thirdly, the results are analyzed. Finally, the correlation between pairs of objectives are analyzed.

6.1. Description of the Real-World Applications

The three real-world applications considered in this study are introduced next.

6.1.1. RWA1: Liquid-Rocket Single Element Injector Design

The design of a liquid-rocket single element injector aims at improving its performance and enlarging its life [70]. Vaidyanathan et al. [71] states that, in order to optimally design such an injector, four objectives should be considered: the maximum injector face temperature ($T{F}_{max}$), the wall temperature at a distance of three inches from the injector face ($T{W}_4$), the maximum oxidizer post tip temperature ($T{T}_{max}$), and the centerline axial location where the combustion is 99% complete ($X_{cc}$). Specifically, this multi-objective optimization problem can be written as:

$$\begin{array}{cc}\hfill \mathrm{minimize}:& f_1\left(\mathbf{x}\right)=X_{cc}\hfill \\ \hfill \mathrm{minimize}:& f_2\left(\mathbf{x}\right)=T{F}_{max}\hfill \\ \hfill \mathrm{minimize}:& f_3\left(\mathbf{x}\right)=T{W}_4\hfill \\ \hfill \mathrm{minimize}:& f_4\left(\mathbf{x}\right)=T{T}_{max}\hfill \end{array}$$

(13)

where $\mathbf{x}={(\alpha ,\Delta HA,\Delta OA,OPTT)}^T$, such that $0^{\circ }\le \alpha \le {20}^{\circ }$ is the hydrogen flow angle, $0\%\le \Delta HA\le 25\%$ is the hydrogen area increment with respect to the baseline cross-section area ($0.0186{\mathrm{in}}^2$), $-40\%\le \Delta OA\le 0\%$ is the oxygen area decrement with respect to the baseline cross-section area ($0.0423{\mathrm{in}}^2$) of the tube carrying oxygen, and $X^{″}\le OPTT\le 2X^{″}$ is the oxidizer post tip thickness, where $X^{″}$ denotes the tip thickness with a baseline value $0.01\mathrm{in}$. The mathematical definition of this problem can be seen in [71].

6.1.2. RWA2: Ultra-Wideband Antenna Design

In order to design an ultra-wideband antenna with two stopbands within the WiMAX and WLAN bands, besides achieving the expected impedance features, gain uniformity and high fidelity are also required [72]. Such antenna comprises a planar rectangular patch and a pair of notches at the two lower corners. Two U-shaped thin slots are carved in the monopole patch for the two stopbands. In order to design this antenna, ten parameters have to be considered and five objective functions, which are the voltage standing wave ratio (VSWR) over the passband ($f_1$), the VSWR among the WiMAX ($f_2$) and WLAN ($f_3$) bands, respectively, the fidelity factors in the E-plane and H-plane ($f_4$), and the relatively uniform peak gains over the passband ($f_5$) [73]. Hence, the multi-objective optimization problem is stated as:

$$\begin{array}{cc}\hfill \mathrm{minimize}:& f_1\left(\mathbf{x}\right)\hfill \\ \hfill \mathrm{maximize}:& f_2\left(\mathbf{x}\right)\hfill \\ \hfill \mathrm{maximize}:& f_3\left(\mathbf{x}\right)\hfill \\ \hfill \mathrm{maximize}:& f_4\left(\mathbf{x}\right)\hfill \\ \hfill \mathrm{minimize}:& f_5\left(\mathbf{x}\right)\hfill \end{array}$$

(14)

where $\mathbf{x}={(a_1,a_2,b_1,b_2,d_1,d_2,l_1,l_2,w_1,w_2)}^T$, such that $5\le a_1\le 7$, $10\le a_2\le 12$, $5\le b_1\le 6$, $6\le b_2\le 7$, $3\le d_1\le 4$, $11.5\le d_2\le 12.5$, $17.5\le l_1\le 22.5$, $2\le l_2\le 3$, $17.5\le w_1\le 22.5$, and $2\le w_2\le 3$. The mathematical formulation of this problem is presented in [73].

6.1.3. RWA3: Development of Oil and Water Repellent Fabric

In the textile industry, one aim is to produce fabrics with added high value. Particularly, the hydrophobicity effect, that is, water, oil, and stain repellence, is one of the most widely used textile surface modification [74]. Hydrophobicity depends on several process parameters, such as the concentration of oil and water repellent (O-CPC) finish, the concentration of the crosslinking agent (K-FEL), and the curing temperature (C-Temp) [75]. The hydrophobicity effect can be measured by means of the following seven responses [76]: the contact angle of a water ($WCA$) and oil ($OCA$) droplet touching a surface; the air permeability ($AP$), which is the comfort property of a woven fabric used to measure the flow of air through it; the crease recovery angle ($CRA$), which measures the textiles ability to recover from creasing; the $stiffness$, which is a comfort property of cotton fabric; the $tearstrength$ of the finished fabric, which depends on the chemical finishing treatment applied to the fabric; and the $tensilestrength$, which describes the behavior of the fabric under axial stretching load. These seven responses can be considered as objective functions as follows:

$$\begin{array}{cc}\hfill \mathrm{maximize}:& f_1\left(\mathbf{x}\right)=WCA\hfill \\ \hfill \mathrm{maximize}:& f_2\left(\mathbf{x}\right)=OCA\hfill \\ \hfill \mathrm{maximize}:& f_3\left(\mathbf{x}\right)=AP\hfill \\ \hfill \mathrm{maximize}:& f_4\left(\mathbf{x}\right)=CRA\hfill \\ \hfill \mathrm{minimize}:& f_5\left(\mathbf{x}\right)=Stiffness\hfill \\ \hfill \mathrm{maximize}:& f_6\left(\mathbf{x}\right)=Tearstrength\hfill \\ \hfill \mathrm{maximize}:& f_7\left(\mathbf{x}\right)=Tesilestrength\hfill \\ \hfill \end{array}$$

(15)

where $\mathbf{x}={(\mathrm{O}-\mathrm{CPC},\mathrm{K}-\mathrm{FEL},\mathrm{C}-\mathrm{Temp})}^T$, such that 10 g/L $\le \mathrm{O}-\mathrm{CPC}\le$ 50 g/L, 10 g/L $\le \mathrm{K}-\mathrm{FEL}\le 50$ g/L, and 150 ${}^{\circ }$C $\le \mathrm{C}-\mathrm{Temp}\le$ 170 ${}^{\circ }$C. The mathematical description of the problem is presented in [76].

6.2. Experimental Setup

In order to analyze the results achieved by LIBEA-II versus those achieved by SMS-EMOA, iSMS-EMOA, and LIBEA, the following experimental setup was carried out. Since the characteristics of the real-world applications(RWA) described above are not known, the reference $\mathcal{PF}$ had to be constructed to compute the quality indicator $IG{D}^{+}$.

• The non-dominated solutions obtained by all four IBEAs from the 30 executions were recorded;
• The maximin fitness function [77] was applied to choose 5000 from these non-dominated solutions and they were considered as the reference set for the $ID{G}^{+}$ quality indicator.

Regarding the ${\mathcal{H}}_n$ quality indicator, for each RWA problem, the ideal point $\mathbf{u}={(u_1,\dots ,u_m)}^T$ was calculated by finding the minimum value for each objective function in the reference $\mathcal{PF}$. On the other hand, the reference vector $\mathbf{r}={(r_1,\dots ,r_m)}^T$ was stated by finding the maximum value for each objective function in the reference $\mathcal{PF}$ and scaling its magnitude (with respect to the ideal point) by 10% for each dimension. More precisely, $r_j=1.1\times |f_j^{max}-u_j|$, such that $f_j^{max}$ is the maximum value of each objective function in the reference $\mathcal{PF}$, for all $j\in \{1,\dots ,m\}$. Hence, the ${\mathcal{H}}_n$ indicator will consider, in a more appropriate scope, the boundaries of the $\mathcal{PF}$ approximation found by each IBEA. Due to the computational cost of the original SMS-EMOA and LIBEA when dealing with more than four objectives, the exact calculation of the ${\mathcal{H}}_c$ was replaced by the HypE indicator [13] employing $1000\times m$ samples for the ${\mathcal{H}}_c$ approximation, where m denotes the number of objectives in the problem. It is worth noticing that the computational cost of LIBEA-II and the other IBEAs depends directly on the population size and on the number of objectives. Our experimental study adopts $N=100$ solutions to solve the three real-world applications. With this number of solutions, LIBEA-II could deal with problems with up to seven objectives in approximately five days. However, LIBEA-II spent less than 24 h performing a single run for problems having four and five objective functions. The experimental study presented in this work was carried out on a desktop PC with a 32-core 2.6 GHz processor and 64GB of RAM.

6.3. Analysis of Results

The results achieved by LIBEA-II were examined versus those obtained by SMS-EMOA, iSMS-EMOA, and LIBEA.

Table 5. Average ${\mathcal{H}}_n$ values for the non-dominated solutions found by the IBEAs to each real-world application.

MOP	SMS-EMOA	iSMS-EMOA	LIBEA	LIBEA-II
RWA1	0.5495 ± 0.001	0.5550 ± 0.001	0.5487 ± 0.001	0.5557 ± 0.001
RWA2	0.6883 ± 0.005	0.6947 ± 0.007	0.6906 ± 0.005	0.6969 ± 0.005
RWA3	0.1982 ± 0.007	0.1951 ± 0.012	0.1963 ± 0.009	0.2052 ± 0.007

and

Table 6. Average $IG{D}^{+}$ values for the non-dominated solutions found by the IBEAs to each real-world application.

MOP	SMS-EMOA	iSMS-EMOA	LIBEA	LIBEA-II
RWA1	0.0157 ± 0.001	0.0126 ± 0.000	0.0160 ± 0.001	0.0126 ± 0.000
RWA2	4318.4205 ± 952.777	4163.9243 ± 1770.699	3939.6710 ± 539.990	3829.6227 ± 1063.089
RWA3	3.3550 ± 0.871	3.9980 ± 1.691	3.5676 ± 1.213	3.0701 ± 0.797

show the results achieved by the algorithms in the three real-world applications described above, for the ${\mathcal{H}}_n$ and $IG{D}^{+}$ quality indicators, respectively. The structure of these tables is similar to that of Table 3 and Table 4. That is, the best average ${\mathcal{H}}_n$ and $IG{D}^{+}$ values for each real-world application are in boldface, while the best algorithm regarding the concerned real-world application and quality indicator is underlined.

Regarding the ${\mathcal{H}}_n$ indicator, we can see from Table 5 that LIBEA-II found solutions that cover a larger hypervolume for the three real-world problems and, remarkably, for problem RWA3, the difference is statistically significant. Concerning the $IG{D}^{+}$ indicator, Table 6 shows that solutions from LIBEA-II obtained, on average, the smallest values for all three real-world problems. In this case, there is a tie between LIBEA-II and iSMS-EMOA for problem RWA1.

Figure 4 shows the average convergence for the $IG{D}^{+}$ indicator. This Figure contains three plots, one for each real-world application. It is evident that the convergence of LIBEA-II is faster than those of the other three algorithms for the two real-world applications RWA1 and RWA3. The convergence for problem RWA2 is similar for all three algorithms.

After these results, it is clear that the performance of LIBEA-II on the three real-world applications is higher than that of SMS-EMOA, iSMS-EMOA, and LIBEA.

6.4. Analysis of the Conflict Relation between Pairs of Objectives

In addition to studying the performance of the four IBEAs on real-world applications, it is also of interest to know the conflict relation (see Definition 5) between the objective functions for each RWA. To this end, Figure 5 presents the parallel coordinates plots for the three real-world applications considered in this study.

Parallel coordinates are a handy tool for identifying conflict, support, or independence between pairs of objectives. Even though they do not specify information regarding the characteristics of the approximation sets, they are generally utilized for realizing the correlation, whether positive, negative, or neutral, between pairs of objectives [78]. As mentioned earlier, the reference $\mathcal{PF}$ was obtained, for each RWA, by recording all non-dominated solutions found by all four algorithms. For each of these solutions, the values for each objective function were plotted. In Figure 5, the conflict between pairs of objectives is illustrated. This Figure shows three boxes, one for each RWA. For each problem, the objective values are normalized in the vertical axis in the range $[0,1]$ for a straightforward interpretation, while the objective functions are on the horizontal axis. Lines are plotted from one objective function $f_i$ to the next adjacent $f_{i+1}$ to reflect the correlation between the pair of objectives $(f_i,f_{i+1})$. If a line depicts a significant slope (whether negative or positive) from one objective to the next, it can be interpreted that those objectives are in conflict, and the longer the slope, the greater the conflict. On the contrary, if a line is horizontal, i.e., it has no slope at all, the objectives support each other.

In the case of RWA1, we can see that most of the lines between objectives $(f_1,f_2)$ are nearly horizontal or with a slight slope, which indicates that those objectives support each other. For the pair of objectives $(f_2,f_3)$, a considerable number of lines have a more significant slope, whether positive or negative, from which we can infer that those objective functions are in conflict. For the last pair $(f_3,f_4)$, we see that some lines have a slope while others are almost horizontal. Hence, there is no clear conclusion for this pair of objectives.

Following the same analysis as in RWA1, in the case of RWA2, we see that objectives $(f_1,f_2)$ and $(f_4,f_5)$ are clearly in conflict since nearly all the lines present a significant slope and only a few are horizontal or with a slight slope. For the pair of objectives $(f_2,f_3)$ and $(f_3,f_4)$, on the contrary, most of the lines have a slight slope or are horizontal, while the rest of the lines present a significant slope. For these cases, nothing can be said from these plots.

Finally, for problem RWA3, it is evident that there is conflict for the pairs of objectives $(f_1,f_2)$, $(f_2,f_3)$, $(f_3,f_4)$, and $(f_5,f_6)$, since the vast majority of the lines presents a significantly large slope, whether positive or negative. For the other two pairs of objectives, $(f_4,f_5)$ and $(f_6,f_7)$, there is no clear indication whether the objectives are in conflict, support each other, or are independent.

In order to complete the conflict relation analysis between objectives, a numerical analysis of the reference $\mathcal{PF}$s is presented next. Figure 6 contains three matrices, one for each RWA problem. Each matrix shows: in the upper triangular matrix, the Pearson correlation coefficients [79] between pairs of objectives; in the lower triangular matrix, the projection of the objective function values of the non-dominated solutions between pairs of objectives; and in the diagonal, the densities of each objective function.

For problem RWA1, we can see that the pairs of objectives $(f_1,f_2)$ and $(f_3,f_4)$ are positively correlated. Remarkably, the correlation for $(f_1,f_2)$ is approximately $1.0$, which means that optimizing one of them, whether $f_1$ or $f_2$, will lead to the optimization of the other and vice versa. These results are in accordance with the analysis of the parallel coordinates plots. The remaining four pairs of objectives, i.e., $(f_1,f_3)$, $(f_1,f_4)$, $(f_2,f_3)$, and $(f_2,f_4)$, present a negative correlation. This means that there is a conflict between the objectives in each pair. That is, the optimization of one objective function deteriorates the other and vice versa.

In the case of problem RWA2, we see a positive correlation for pairs of objectives $(f_1,f_5)$, $(f_2,f_3)$, $(f_2,f_4)$, and $(f_3,f_4)$, which means that, to some extent, the objectives in each pair support each other. Particularly for the pair $(f_1,f_5)$, we can observe that the correlation is nearly $1.0$. The other six pairs of objective functions, that is $(f_1,f_2)$, $(f_1,f_3)$, $(f_1,f_4)$, $(f_2,f_5)$, $(f_3,f_5)$, and $(f_4,f_5)$, present a negative correlation. From these last pairs of objectives, the conflict that exists in $(f_1,f_2)$ and $(f_4,f_5)$ is in agreement with the observed in the parallel coordinates plots.

Finally, for the RWA3 problem, we can confirm what was noticed from the parallel coordinates plots, that is, the pairs of objective functions $(f_1,f_2)$, $(f_2,f_3)$, $(f_3,f_4)$, and $(f_5,f_6)$ presents a negative correlations, which means the objectives in each pair are in conflict with each other. Other pairs of objectives that present a negative correlation are $(f_1,f_6)$, $(f_1,f_7)$, $(f_2,f_4)$, $(f_2,f_5)$, $(f_3,f_6)$, $(f_3,f_7)$, $(f_4,f_6)$, $(f_4,f_7)$, and $(f_5,f_7)$. The remaining eight pairs of objectives show a positive correlation, however, this does not mean that they can be removed from the problem since they show conflict with other objective functions.

7. Conclusions

This paper introduced an improved Lebesgue indicator-based evolutionary algorithm (LIBEA-II) for solving multi-objective optimization problems. The hypothesis put forward in this paper about the efficiency of IBEAs considering the Lebesgue measure, the regularity property of continuous MOPs, and the local property was held. In terms of results, the proposed LIBEA-II and the other three IBEAs, namely SMS-EMOA, iSMS-EMOA, and LIBEA, were tested on the well-known UF benchmark set. These test functions have properties that have been seen in real-life optimization problems in terms of separability, multi-modality, and different $\mathcal{PF}$ shapes, including convexity, concavity, discontinuities, etc. The non-dominated solutions achieved by LIBEA-II and by the other three algorithms, for each test function, were applied the two quality indicators: normalized hypervolume (${\mathcal{H}}_n$) and inverted generational distance plus ($IG{D}^{+}$). Results from the ${\mathcal{H}}_n$ indicator showed that the performance of the four algorithms is similar. Given that all four algorithms are based on the ${\mathcal{H}}_n$ quality indicator, this result was rather expected. Regarding the $IG{D}^{+}$ indicator, the performance of LIBEA-II was slightly better than the other three algorithms since it obtained the best average results for five out of the ten test functions, and the difference was statistically significant for one of them. In general, LIBEA-II is an efficient algorithm since it can find solutions with the same quality as those found by the other three algorithms but using only 20% of the computing resources.

LIBEA-II was also tested on three real-world applications, precisely: the liquid-rocket single element injector design (RWA1), which has four objective functions and four variables, the ultra-wideband antenna design (RWA2), which considers five objective functions and ten variables, and the development of oil and water repellent fabric (RWA3), which optimizes seven objective functions with three variables. In this case, LIBEA-II was also compared with the same IBEAs used for the UF test instances. Remarkably, LIBEA-II was able to obtain non-dominated solutions with higher quality than those found by the other three IBEAs, since the average value from both quality indicators, i.e., ${\mathcal{H}}_n$ and $IG{D}^{+}$, was the best. The superiority of LIBEA-II was demonstrated in real-world applications since it obtained higher-quality non-dominated solutions and saved up to approximately 80% of the hypervolume calculations.

As part of our future research, we are interested in extending the applicability of LIBEA-II to deal with constrained MOPs. This line of research has been slightly explored, and it is the course of our outcoming investigations. On the other hand, we would like to test the performance of the proposed LIBEA-II in other real-life applications in order to identify insights that allow us to understand the main weaknesses of IBEAs based on the Lebesgue measure. On the other hand, the hybridization of these types of approaches with mathematical programming is certainly a good path that deserves to be investigated. These are, in fact, part of our future program of investigations.