Next Article in Journal
Modeling Sound Propagation Using the Corrective Smoothed Particle Method with an Acoustic Boundary Treatment Technique
Previous Article in Journal
Safeguarding against Inactivation Temperatures during Plasma Treatment of Skin: Multiphysics Model and Phase Field Method
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Weakly Pareto Compliant Quality Indicator

by
Emanuele Dilettoso
,
Santi Agatino Rizzo
* and
Nunzio Salerno
Department of Electrical, Electronics and Computer Engineering, University of Catania, Catania 95125, Italy
*
Author to whom correspondence should be addressed.
Math. Comput. Appl. 2017, 22(1), 25; https://doi.org/10.3390/mca22010025
Submission received: 15 December 2016 / Revised: 22 February 2017 / Accepted: 27 February 2017 / Published: 6 March 2017

Abstract

:
In multi- and many-objective optimization problems, the optimization target is to obtain a set of non-dominated solutions close to the Pareto-optimal front, well-distributed, maximally extended and fully filled. Comparing solution sets is crucial in evaluating the performance of different optimization algorithms. The use of performance indicators is common in comparing those sets and, subsequently, optimization algorithms. Therefore, an effective performance indicator must encompass these features as a whole and, above all, it must be Pareto dominance compliant. Unfortunately, some of the known indicators often fail to properly reflect the quality of a solution set or cost a lot to compute. This paper demonstrates that the Degree of Approximation (DOA) quality indicator is a weakly Pareto compliant unary indicator that gives a good estimation of the match between the approximated front and the Pareto-optimal front.

1. Introduction

The optimized design of industrial applications is often problematic because of the simultaneous occurrence of many conflicting targets [1,2,3]. In real-world optimization problems, the decision maker needs to have a wide range of solutions to choose from [4]. Some optimization methods solve a single function obtained by aggregating different objective functions [5,6,7]. The choice of weights is the major weakness to this approach [5]. Other multi- and many-objective optimization algorithms (MOOAs) search for a non-dominated solution set [8,9,10], i.e., a set of multiple alternative solutions. This set is the Approximation Set in the decision space and the Approximated Pareto Front (APF) in the objective functions space. The main goal of such algorithms is to provide an APF matching the Pareto-optimal front (POF). The problem is to assess how well the approximated front fits the optimal one [11]. The notion of optimization algorithms performance involves evaluating the quality of the solution and the required computational effort [12]. This proves troublesome in the case of multi- and many-objective optimization problems (MOOPs): a good approach would be to use a quality indicator (QI), i.e., a function of the APF that simplifies the quantitative performance comparison of different optimization algorithms. The simplest comparison method would be to check whether one APF is better than another with respect to the Pareto dominance relations [11]. Thus, a QI must be able to account for Pareto dominance to properly compare two different algorithms. This is known as completeness with respect to Pareto dominance relations, and is the most desired property of a QI. More specifically, assuming there are two MOOAs providing, respectively, two non-dominated set A and B, a (weakly) Pareto compliant QI must comply with the following relation: if A dominates B then QI(A) must be better than QI(B). On the other hand, if there exists a scenario where A dominates B, but the QI(A) is worse than QI(B), then the QI is not Pareto compliant, therefore it could provide misleading results when it is used to compare MOOAs.
Moreover, when APFs are incomparable with respect to Pareto dominance relations, more information is needed to compare the APFs provided by different MOOAs. Generally, a good MOOA should [13,14,15]:
  • minimize the APF distance from the POF;
  • obtain a good (usually uniform) distribution of the solutions found;
  • maximize the APF extension i.e., for each objective the non-dominated solutions should cover a wide range of values (best case: the global optimum of each objective function must be found);
  • maximize the APF density, i.e., high cardinality for the approximation set is desirable.
Each goal represents a desired feature of the APF: in the following we refer to them as closeness, distribution, extension and cardinality, respectively. It is worth noticing that, given a scenario where:
  • A is closer to the POF than B;
  • the solutions in A are better distributed than the ones in B;
  • A is more extended than B;
  • the size of A is greater than the size of B,
it is very probable that A dominates B, or, at least B does not dominate A.
Although it is improbable that the two APFs will be incomparable with respect to Pareto dominance relations, when this event occurs (an example is shown in Figure 1) QI(A) should be better than QI(B). In other words, the QI should reveal that A outperforms B with respect to all features and, consequently, A is preferable to B even though the two APFs are incomparable.
A unary QI (UQI) estimates a non-dominated solution set quality by means of a real number [16]; then it is useful to estimate the effectiveness of a MOOA. Several known UQIs have no or limited completeness as regards Pareto dominance relations and are unable to take into account all the features listed previously. Few UQIs overcome these limitations despite needing much computational effort.
This paper demonstrates the ≻-completeness of the UQI, called Degree of Approximation (DOA) [17]. Moreover, it proves its ability to take into account all four features.
To our knowledge, Hypervolume [1] is the only ⊳-complete UQI (then Hypervolume is a Pareto compliant UQI), and for this reason it is considered the best UQI for comparing optimization algorithms. Nevertheless, the relation A⊲B differs from A≺B since the former accounts for the case in which A contains some solutions of B but the probability of this specific event is very low, and it can be considered null when the objective functions’ space belongs to the set of real numbers. Therefore, DOA can be used to evaluate the performance of optimization algorithms instead of the Hypervolume since it is proven that DOA is ≻-complete (then DOA is a weakly Pareto compliant UQI). Note that the calculation of Hypervolume is difficult as the number of objective functions increases, while the calculation of DOA is usually very simple and fast even in the case of many-objective optimization. DOA calculation needs the knowledge of the POF, while the Hypervolume indicator needs to know the reference point. However, it is not a limitation when they are applied to compare the performance of different MOOAs. In fact, the best way to perform the comparisons is using properly designed benchmark problems for which the POF is known [18]: Schaffer’s benchmark [19], Fonseca’s benchmark [20], Poloni’s benchmark [21], Kursawe’s benchmark [22], ZDT1-6 [13], and DTLZ1-7 [23].
In the Evolutionary Multi-Objective community, UQIs are also adopted as the guidance mechanism during the optimization process, e.g., SMS-EMOEA [24] exploits the Hypervolume indicator during optimization. The use of a given UQI in indicator-based optimization algorithms enables one to implicitly scalarize the multi-objective problem. In this case, the compliance with the dominance relation of the indicator is one reason for the slow convergence rates of elitist multi- and many-objective evolutionary algorithms (MOEAs) [25]. On the other hand, DOA is devised to evaluate the performance of the final set of non-dominated solutions and it is not devised to be included as guidance mechanism of a MOOA. Hence, in the DOA’s application field, Pareto compliance is a suitable and necessary feature.
The paper is organized as follows. Section 2 recalls the definitions and terminology typically used in multi-objective optimization related to the Pareto dominance concept. Section 3 outlines the characteristics of a QI and presents a review of the most common UQIs. Section 4 describes DOA in detail, while Section 5 and Section 6 mathematically demonstrate its ≻≻-completeness and ≻-completeness, respectively. Section 7 proves DOA compatibility with respect to the not better dominance relation. Finally, Section 8 validates DOA with some examples to highlight its accounting for closeness, distribution, extension and cardinality. Conclusions and future works are drawn in Section 9 and minor details of the proof in Section 6 have been reported in the appendices.

2. Definitions and Terminology

2.1. Multi- and Many-Objective Optimization Problem

Solving a MOOP means finding the optimal and feasible parameter configurations. A feasible solution (configuration) is called decision vector (x = x1,x2,...,xm) and is a point in the decision space(X). An objective vector(y = y1,y2,...,ym), that is a point in the objective space (Y), is linked to each decision vector by means of evaluating function f. So, a MOOP, with m decision variables (parameters to be set), n targets (objective functions to be optimized), and c constraints ( equality and c-ℓ are inequality constraints), can be mathematically represented as follows.
Maximize or minimize:
y = f ( x ) = ( f 1 ( x ) ,   f 2 ( x ) , ,   f n ( x ) )
subject to:
g i ( x ) = 0 i = 1 , 2 , , g i ( x ) 0 i = + 1 , + 2 , , c
where:
x = ( x 1 , x 2 , , x m )   X y = ( y 1 , y 2 , , y n )   Y y i = f i ( x ) i = 1 , 2 , , n
Without loss of generality, in the following it is assumed that each objective function has to be minimized.

2.2. Pareto Dominance

Usually, in real-world MOOPs, there is no single parameter configuration that simultaneously optimizes all objective functions, i.e., a point does not exist in the decision space that is a global optimum. Thus, solving a real-world MOOP means offering the designer a set of alternative optimal solutions in the Pareto dominance sense.
Pareto dominance—A decision vector x1dominates another decision vector x2 iff:
f i ( x 1 ) f i ( x 2 ) i = 1 , 2 , , n and i : f i ( x 1 ) < f i ( x 2 )
This relation is denoted as x1x2. When one or more of these relations is not satisfied, x1 does not dominate x2; this condition is denoted as x1x2. It is worth noticing that, for a single objective function, the standard relation less than is generally used to define the corresponding minimization problem, while the symbol ≺ represents a natural extension of < in the case of MOOPs [26].
Pareto optimality—A decision vector x’ is said to be Pareto-optimal iff:
x X : x x
The set that groups this kind of solution is known as a Pareto-optimal set, and all the solutions of this set are alternative, with no one solution being dominated by the other solutions.
In addition to dominance, other types of relation between the solutions can be defined:
strictly dominance: a decision vector x1 strictly dominates another decision vector x2 (denoted as x1 ≺≺ x2) iff:
f i ( x 1 ) < f i ( x 2 ) i = 1 , 2 , , n
weakly dominance: a decision vector x1 weakly dominates the decision vector x2 (denoted as x1x2) iff:
f i ( x 1 ) f i ( x 2 ) i = 1 , 2 , , n
Finally, when x1 is better than x2 with respect to a subset of objective functions but x2 is better than x1 with respect to another subset, the two solutions are said incomparable, denoted as x1||x2 (or x2||x1):
x 1 x 2       x 2 x 1 x 1 | | x 2 ( o r x 2 | | x 1 )
Table 1 resumes the dominance relations. It is worth to note that a relation may imply other relations:
x1 ≺≺ x2x1x2x1x2
x1x2x1x2x1 ⊀⊀ x2.
By relating the solutions of one APF A to those of another APF B it is possible to extend the dominance relations between two solutions to two APFs. Table 2 shows the relations between two APFs.

3. Quality Indicator

3.1. Definitions

A quality indicator QI is a function q: S→ ℝ, where S is the objective functions space, that assigns a real value to a set of APFs belonging to S related to a MOOP. When the function q has just one argument (i.e., one APF), the quality indicator is called unary, when it has two arguments (i.e., two APFs) it is called binary, and so on.
The aim of a QI is to compare APFs and so QIs are mainly used to indicate if a MOOA works any better than others. Some QIs can also be applied as the acceptance criterion to the selection operator of the stochastic search algorithms [27], but DOA is not devised for such a scope.

3.2. Comparison Methods

This paper focuses on the use of QIs for evaluating the performances of different optimization algorithms. To do this the QI results must be interpreted by means of an interpretation function E: ℝq→ Bool, where q depends on the size of the QI set. Figure 2 shows some examples of interpretation functions (A and B are two APFs).
Finally, the combination of a quality indicator, I, and an interpretation function, E, is called a comparison method [11], and is referred to as CI,E: CI,E(A,B) = E(I(A),I(B)).

3.3. Compatibility and Completeness

Usually, one or a set of QIs can be useful to compare different optimization algorithms to figure out which works better on a particular class of problems.
Non–dominated solutions are preferred to the dominated ones from the designer’s point of view. Then, when a comparison method shows that APF A is preferable to APF B, A must be better than B. In a similar way, when A is better than B, a comparison method must indicate that A is preferable to B. Such features are known as ⊳-compatibility and ⊳-completeness [11].
Let ► be an arbitrary dominance relation among those defined in Table 2 (≻≻ or ≻ or ⊳). A comparison method CI,E is said ►-compatible if for each possible pair of APFs A and B:
C I , E ( A , B )  is true A B
A comparison method CI,E is said ►-complete if for each possible pair of APFs A and B:
A B C I , E ( A , B )  is true .
It has been demonstrated [11] that a comparison method based on a UQI (or on a finite combination of UQIs) that is both ⊳-compatible and ⊳-complete cannot exist. Moreover, Pareto dominance is sufficient but not necessary to consider an APF preferable to another: there are pairs of APFs with considerable quality difference which are considered, by Pareto dominance relations, as not comparable [16]. Hence, if a comparison method based on UQI were ⊳-compatible, the indicator could not provide any preference in the case of two incomparable APFs. Therefore, it would be better if the UQI were only compatible with ⋫ [28] and it should take into account all the features (closeness, distribution, extension, cardinality) that are desirable for an APF.
Finally, while a comparison method ⊳-complete is necessary (i.e., when APF A is better than APF B the comparison method must highlight it), when a comparison method shows that A is preferable to B, one of the following two cases must hold:
  • A is better than B (A⊲B);
  • A and B are incomparable and A outperforms B with respect to closeness, distribution, extension and cardinality.

3.4. Closeness, Distribution, Extension, and Cardinality

The main target of an optimization algorithm to solve a MOOP is to find an APF as similar as possible to the POF. Hence, as said before, the APF must be:
  • close to the POF; Figure 3 represents the extreme cases: an APF exhibiting good closeness only, and an APF with all good features but not close to the POF;
  • well distributed (usually uniform); Figure 4 shows an APF exhibiting a uniform distribution only and an APF with all good features but not uniformly distributed;
  • very extended (in the best case the global optimum of each objective function belongs to the APF); Figure 5 shows an APF with only a good extension and one with all good features but not extended;
  • of high cardinality; Figure 6 shows an APF with good cardinality only and an APF with all good features but poor cardinality.
Figure 7 shows an APF with all the desired features. A good QI must take into account all these features to give a correct measure of APF quality.
Table 3 points out if a specific feature partially Mca 22 00025 i001 or totally ⚫ affects the value of some UQIs. A heuristic approach has been applied to determine whether a feature (closeness, distribution, extension, cardinality) affects the QI value. In particular, an APF B obtained by improving a given feature of another APF A is expected to have an indicator value better than that of A when the indicator is sensitive to this feature. For example, if an APF is gradually moved towards the POF and the indicator increasingly improves, then the indicator is influenced by the closeness feature. An indicator is partially affected by a feature when it sometimes improves and other times does not change.
The Inverted Generational Distance indicator (IGD) [29] and a similar indicator using a weighted-scalar distance function called Average Distance from Reference Set indicator (ADRS) [30], the Completeness indicator [31,32], the Potential Function indicator [27] and the Hypervolume indicator [1] account for all the features but they present some drawbacks.
The IGD and ADRS indicators have the same complexity of DOA, but they are ≻≻-complete only [11]. The Δp indicator [39] can be considered as the combination of slight variations of the Generational Distance and the IGD indicator [25]. Therefore, it presents the advantage and limitations of these indicators, that is it accounts for all the features but it is ≻≻-complete only.
The Completeness and Potential Function indicators are as ≻-complete as the DOA indicator. Nevertheless, the Completeness indicator cannot be directly computed, but can be estimated by drawing samples from the feasible set and computing completeness for these samples. The confidence interval for the true value can be evaluated with any reliability value, given sufficiently large samples [26]. For the Potential Function indicator similar considerations hold. Hence, the drawback of both indicators is the high computational cost.
To our knowledge, Hypervolume is the only ⊳-complete UQI, and so is considered the best UQI for comparing optimization algorithms. Nevertheless, the relation A⊲B differs from A≺B since the former accounts for the case in which B contains some solutions of A but the probability of this specific event is very low, and it can be considered null when the objective functions’ space belongs to the set of real numbers.
Moreover, Hypervolume running time grows exponentially with the number of objective functions [41,42,43,44]. The most obvious method for calculating Hypervolume is the inclusion-exclusion algorithm, with complexity O(n2m), where n is the number of objectives and m is the number of APF points. The fastest methods for calculating Hypervolume (e.g., LebMeasure [45], Hypervolume by Slicing Objectives algorithm [46]) lead to a O(m2n3)complexity. The DOA indicator has a lower computational cost, presenting a O(nMm) complexity, where M is the number of POF points.

4. The Weakly Pareto Compliant Quality Indicator

The comparison method based on DOA and its associated interpretation function is ≻-complete. In detail, for a given POF and an APF A, DOA is computed as follows.
First, given a solution i belonging to the POF, Di,A, is determined from the sub-set of A containing the solutions dominated by i (Figure 8). Hence, if the number of solutions belonging to Di,A is not null, for each approximated solution aDi,A the Euclidean distance dfi,a between a and i is computed as:
d f i , a = k = 1 n [ f k , a f k , i ] 2 ,
with:
n 
number of objective functions,
fk,a
value of k-th objective function of the approximated solution a,
fk,i
value of k-th objective function of optimal solution i.
Euclidean distance di,A (Figure 9) between i and the nearest approximated solution belonging to Di,A is computed in the objective function space as:
d i , A = { min ( d f i , a ) a D i , A i f | D i , A | > 0 i f | D i , A | = 0 ,
where |Di,A| is the number of solutions belonging to Di,A.
Another quantity ri,A (similarly to di,A) is computed for i considering the solutions of A not dominated by i (i.e., A\Di,A):
r i , A = { min ( r f i , a ) a A \ D i , A i f | A \ D i , A | > 0 i f | A \ D i , A | = 0 ,
where rfi,a is a reduced distance (Figure 10) between i and a non-dominated solution a of A (i.e., ∀a ∊ A: i || a), i.e., computed only for objectives k with fk,afk,i:
r f i , a = k = 1 n [ max ( 0 , f k , a f k , i ) ] 2 .
Note that, rfi,a is equal to dfi,a when a Di,A. Moreover, defining na (na < n) as the number of functions for which the fk,afk,i ≥ 0 (fk,afk,i, k = 1,..,na and fk,a < fk,i, k = na + 1,..,n) expression (16) can be rewritten as:
r f i , a = k = 1 n a [ f a ( k ) f i ( k ) ] 2 + k = n a + 1 n 0 .
Finally, defining
si,A = min(di,A,ri,A)
the DOA indicator for the APF A is computed as:
D O A ( A ) = 1 | P O F | i = 1 | P O F | s i , A .
DOA and IGD present some similarities. More specifically, if di,A, Equation (14), is computed considering all solutions in A regardless of the dominance relation with i, and only this quantity is considered then the IGD indicator is obtained, that is:
I G D ( A ) = 1 | P O F | i = 1 | P O F | d i , A .
The IGD indicator is not ≻-complete while DOA is ≻-complete, as will be proved in the following. Therefore, the use of ri,A is the enabling key for DOA ≻-completeness.
While the Hypervolume indicator needs to know the reference point, the DOA calculation needs the knowledge of the POF, like the IGD indicator. This is not a drawback in MOOA benchmarking which is usually carried out for problems with known POF. More specifically, the Pareto optimal solutions and the analytical expression of the objective functions are usually known. Therefore, the POF is obtained by points uniformly discretizing the decision parameter intervals where the Pareto optimal solutions are located. The value of DOA does not depend on the number of points in the POF if |POF|>>|APF|.
Considering two APFs A and B, the proposed quality indicator needs interpreting [11] to affirm either that A is preferable to B or B is preferable to A or A and B are equivalent: the proposed obvious interpretation function is illustrated in Figure 11. Moreover, DOA changes with arbitrary scaling of the objective functions, since the DOA indicator is a distance-based metric, while the relationship between DOA(A) and DOA(B) does not change.
In the following, it is demonstrated that A≺B implies A is preferable to B, that is DOA(A) < DOA(B), in order to affirm that DOA is a ≻-complete quality indicator.
For the sake of clarity, the ≻≻-completeness of DOA is demonstrated before proving its ≻-completeness.

5. ≻≻-Completeness

DOA is a ≻≻-complete quality indicator if DOA(A)<DOA(B) for any pair of APF A and B, with A≺≺B.
In the hypothesis that A≺≺B, each solution of B is strictly dominated by, at least, one solution of A. To demonstrate that DOA(A)<DOA(B) is sufficient to prove that si,A is always lesser than si,B for each point i ∊ POF. In other words,
  • if
    si,A < si,B ∀ i ∊ POF,
  • then
    D O A ( A ) = 1 | P O F | i = 1 | P O F | s i , A < 1 | P O F | i = 1 | P O F | s i , B = D O A ( B ) .
Considering a point i ∊POF, in the following, b indicates the solution belonging to B which provides si,B and a a solution of A that strictly dominates b (a≺≺b); only four scenarios are possible (see Figure 12):
  • A1. i≺≺b Λ ia
  • B1. i≺≺b Λ i||a
  • C1. ib Λ i||a
  • D1. i||b Λ i||a
Note that the other scenarios ib Λ ia and i||b Λ ia are not possible because a≺≺b: in fact, from either ib Λ a≺≺b and i||b Λ a≺≺b follows ia.
Moreover, it is worth mentioning the following:
Remark 1.
ia implies that si,A ≤ dfi,a, in detail:
  • si,A = dfi,a iff si,A = di,A Λ di,A = dfi,a;
  • si,A < dfi,a either if si,A = di,A Λ di,A = dfi,a* < dfi,a (where a*∊A and a*≠a) or if si,A = ri,A (this implies that ri,A < di,A ≤ dfi,a).
Remark 2.
i||a implies that si,A ≤ rfi,a, in detail:
  • si,A = rfi,a iff si,A = ri,A Λ ri,A = rfi,a;
  • si,A < rfi,a either if si,A = ri,A Λ ri,A = rfi,a* < rfi,a (where a*A and a*≠a) or if si,A = di,A (this implies that di,A < ri,A ≤ rfi,a).
Finally, the inequality si,A < si,B will be proved for the four scenarios A1–D1: this inequality naturally implies the ≻≻-completeness of the DOA indicator.

5.1. A1. i≺≺b Λ ia

In this case, i strictly dominates b then si,B = di,B = dfi,b, because b is the solution which provides si,B. Moreover, ia implies that si,Adfi,a (see Remark 1).
So, in order to demonstrate that si,A < si,B it is sufficient to demonstrate that dfi,a < dfi,b.
Recalling that
ia ⇒
fk,ifk,a, ∀ k = 1,…, n
a≺≺b
fk,a < fk,b, ∀ k = 1,…, n
the following inequalities hold:
0 f k , a f k , i < f k , b f k , i k = 1 , . . , n d f i , a = k = 1 n [ f k , a f k , i ] 2 < k = 1 n [ f k , b f k , i ] 2 = d f i , b s i , A d f i , a < d f i , b = s i , B .

5.2. B1. i≺≺b Λ i||a

In this case, i strictly dominates b then si,B = di,B = dfi,b, because b is the solution which provides si,B. Moreover, i||a implies that si,Arfi,a (see Remark 2).
So, in order to demonstrate that si,A < si,B it is sufficient to demonstrate that rfi,a < dfi,b. Proof is given in the next section because scenario C encompasses scenario B.

5.3. C1. ib Λ i||a

In this case, i dominates b then si,B = di,B = dfi,b, because b is the solution which provides si,B. Moreover, i || a implies that si,Arfi,a (see Remark 2).
So, in order to demonstrate that si,A < si,B it is sufficient to demonstrate that rfi,a < dfi,b. Ordering the n objective functions of solution a in such a way that the first na (with na < n) are greater than those of i and recalling that:
i||a  ⇒
fk,i < fk,a, ∀ k = 1,…, na
fk,ifk,a, ∀ k = na + 1,…, n
ib ⇒
fk,ifk,b, ∀ k = 1,…, n
a≺≺b
fk,a < fk,b, ∀ k = 1,…, n
then the following inequalities hold:
0 < f k , a f k , i < f k , b f k , i k = 1 , . . , n a 0 f k , b f k , i          k = n a + 1 , . . , n r f i , a = k = 1 n a [ f k , a f k , i ] 2 + k = n a + 1 n 0 < k = 1 n a [ f k , b f k , i ] 2 + k = n a + 1 n [ f k , b f k , i ] 2 = d f i , b s i , A r f i , a < d f i , b = s i , B .

5.4. D1. i||b Λ i||a

In this case, i and b are incomparable then si,B = ri,B = rfi,b, because b is the solution which provides si,B. Moreover, i||a implies that si,Arfi,a (see Remark 2).
So, in order to demonstrate that si,A < si,B it is sufficient to demonstrate that rfi,a < rfi,b.
Ordering the n objective functions of solution a in such a way that the first na are greater than those of i, ordering the n objective functions of solution b in such a way that the first nb are greater than those of i (with nanb < n, since a≻≻b Λ i||a implies nanb, while i||b implies nb < n) and recalling that:
i||a ⇒
fk,i < fk,a, ∀ k = 1,…, na
fk,ifk,a, ∀ k = na + 1,…, n
i||b ⇒
fk,i < fk,b, ∀ k = 1,…, nb
fk,ifk,b, ∀ k = nb + 1,…, n
a≺≺b  ⇒
fk,a < fk,b, ∀ k = 1,..., n
then the following inequalities hold:
0 < f k , a f k , i < f k , b f k , i k = 1 , . . , n a 0 < f k , b f k , i k = n a + 1 , . . , n b 0 f k , b f k , i k = n b + 1 , . . , n r f i , a = k = 1 n a [ f k , a f k , i ] 2 + k = n a + 1 n b 0 + k = n b + 1 n 0 < k = 1 n a [ f k , b f k , i ] 2 + k = n a + 1 n b [ f k , b f k , i ] 2 + k = n b + 1 n 0 = r f i , b s i , A r f i , a < r f i , b = s i , B .

6. ≻-Completeness

In this section is proved that DOA is a ≻-complete quality indicator. Consider any pair of APF A and B, with A≺B, the ≻-completeness of DOAis demonstrated by proving that si,A is never greater than si,B (for each point i∊POF) and always exists a point i*∊ POF for which si*,A is lesser than si*,B:
  • if
    si,A ≤ si,Bi ∊ POF Λ ∃ i*∊POF: si*,A < si*,B
  • then
    DOA ( A ) = 1 | P O F | ( s i * , A + i = 2 | P O F | s i , A ) < 1 | P O F | ( s i * , B + i = 2 | P O F | s i , B ) = DOA ( B ) .
In the following, b indicates the solution belonging to B which provides si,B for a point i ∊ POF and a a solution of A that dominates b (ab). Moreover, the ≻-completeness of DOA is proved in the worst and most general case, i.e., when ∀ b∊B ∄ a∊A: a≺≺b (i.e., ab Λ a⊀⊀b, limit case); only five scenarios are possible (see Figure 13):
  • A2. i≺≺b Λ ia
  • B2. i≺≺b Λ i||a
  • C2. ib Λ ia
  • D2. ib Λ i||a
  • E2. i||b Λ i||a
Note that the scenario i||b Λ ia is not possible because ab: in fact, from i||b Λ ab follows ia. Moreover, scenario A2 does not include i = a, differently from section 5, because in this case i≺≺b while ab Λ a⊀⊀b. Analogously, differently from section 5, when ib the scenario ia is possible. Finally, the scenario ib is not considered because A≺B, in fact i = b would lead to the absurd ab=i.
In order to demonstrate the ≻-completeness of DOA, it is proved that the inequality si,A < si,B is verified ∀i for the three scenarios A2, B2 and C2. While for the remaining two scenarios D2 and E2 we will prove that the following two sufficient conditions hold:
α.
si,Asi,B
β.
i* ∊ POF: si*,A < si*,B.
For the sake of simplicity, the proof of β will be given in Appendix A.

6.1. A2. i≺≺b Λ ia

In this case, i strictly dominates b then si,B = di,B = dfi,b, because b is the solution which provides si,B. Moreover, ia implies that si,Adfi,a (see Remark 1).
So, in order to demonstrate that si,A < si,B it is sufficient to demonstrate that dfi,a < dfi,b.
Recalling that:
ia ⇒
fk,ifk,a, ∀ k = 1,…, n
ab ⇒
fk,afk,b, ∀ k = 1,…, n Λ ∃ j: fj,a < fj,b
the following inequalities hold:
0 f k , a f k , i f k , b f k , i k = 1 , . . , n 0 f k , a f k , i < f k , b f k , i    k = j d f i , a = k = 1 k j n [ f k , a f k , i ] 2 + [ f j , a f j , i ] 2 < k = 1 k j n [ f k , b f k , i ] 2 + [ f j , b f j , i ] 2 = d f i , b s i , A d f i , a < d f i , b = s i , B .

6.2. B2. i≺≺b Λ i||a

In this case, i strictly dominates b then si,B = di,B = dfi,b, because b is the solution which provides si,B. Moreover, i||a implies that si,Arfi,a (see Remark 2).
So, in order to demonstrate that si,A < si,B it is sufficient to demonstrate that rfi,a < dfi,b.
Ordering the n objectives f of solution a in such a way that the first na (with na < n) are greater than those of i and recalling that:
i||a ⇒
fk,i < fk,a, ∀ k = 1,…, na
fk,ifk,a, ∀ k = na + 1,…, n
i≺≺b  ⇒
fk,i < fk,b, ∀ k = 1,..., n
ab ⇒
fk,afk,b, ∀ k = 1,…, n
then the following inequalities hold:
0 < f k , a f k , i f k , b f k , i k = 1 , . . , n a 0 < f k , b f k , i    k = n a + 1 , . . , n r f i , a = k = 1 n a [ f k , a f k , i ] 2 + k = n a + 1 n 0 < k = 1 n a [ f k , b f k , i ] 2 + k = n a + 1 n [ f k , b f k , i ] 2 = d f i , b s i , A r f i , a < d f i , b = s i , B .

6.3. C2. ib Λ ia

In this case, i dominates b then si,B = di,B = dfi,b, because b is the solution which provides si,B. Moreover, ia implies that si,Adfi,a (see Remark 1).
So, in order to demonstrate that si,A < si,B it is sufficient to demonstrate that dfi,a < dfi,b.
Recalling that:
ia ⇒
fk,ifk,a, ∀ k = 1,…, n
ab ⇒
fk,afk,b, ∀ k = 1,…, n Λ ∃ j: fj,a < fj,b
then the following inequalities hold:
0 f k , a f k , i f k , b f k , i k = 1 , . . , n 0 f k , a f k , i < f k , b f k , i k = j d f i , a = k = 1 k j n [ f k , a f k , i ] 2 + [ f j , a f j , i ] 2 < k = 1 k j n [ f k , b f k , i ] 2 + [ f j , b f j , i ] 2 = d f i , b s i , A d f i , a < d f i , b = s i , B .

6.4. D2. ib Λ i||a

In this case, i dominates b then si,B = di,B = dfi,b, because b is the solution which provides si,B. Moreover, i||a implies that si,Arfi,a (see Remark 2).
So, in order to demonstrate that si,Asi,B it is sufficient to demonstrate that rfi,adfi,b.
Ordering the n objectives f of solution a in such a way that the first na (with na < n) are greater than those of i and recalling that:
i||a ⇒
fk,i < fk,a, ∀ k = 1,…, na
fk,ifk,a, ∀ k = na + 1,…, n
ib ⇒
fk,ifk,b, ∀ k = 1,…, n Λ ∃ h: fh,i < fh,b
ab ⇒
fk,afk,b, ∀ k = 1,…, n Λ ∃ j: fj,a < fj,b
then the following inequalities hold
0 < f k , a f k , i f k , b f k , i k = 1 , . . , n a 0 f k , b f k , i    k = n a + 1 , . . , n r f i , a = k = 1 n a [ f k , a f k , i ] 2 + k = n a + 1 n 0 k = 1 n a [ f k , b f k , i ] 2 + k = n a + 1 n [ f k , b f k , i ] 2 = d f i , b s i , A r f i , a d f i , b = s i , B .
Note that rfi,a is strictly lesser than dfi,b when 1 ≤ jna, since
k = 1 n a [ f k , a f k , i ] 2 < k = 1 n a [ f k , b f k , i ] 2 .
Otherwise si,Asi,B. In particular, rfi,a = dfi,b iff fk,a = fk,bk = 1,…,na Λ fk,i = fk,bk = na + 1,…,n. In this case, obviously, fh,i < fh,a = fh,b with 1 ≤ hna and fj,a < fj,i = fj,b with j > na. As said before, the proof that ∃ i*∊POF: si*,A < si*,B has been reported in Appendix A.

6.5. E2. i||b Λ i||a

In this case, i and b are incomparable then si,B = ri,B = rfi,b, because b is the solution which provides si,B. Moreover, i||a implies that si,Arfi,a (see Remark 2).
So, in order to demonstrate that si,Asi,B, it is sufficient to demonstrate that rfi,arfi,b. Ordering the n objectives f of solution a in such a way that the first na are greater than those of i, ordering the n objectives f of solution b in such a way that the first nb are greater than those of i (with nanb < n, since ab Λ i||a implies nanb, while i||b implies nb < n) and recalling that:
i||a ⇒
fk,i < fk,a, ∀ k = 1,…, na
fk,ifk,a, ∀ k = na + 1,…, n
i||b ⇒
fk,i < fk,b, ∀ k = 1,…, nb
fk,ifk,b, ∀ k = nb + 1,..,n
ab ⇒
fk,afk,b, ∀ k = 1,…, n Λ ∃ j: fj,a < fj,b
then the following inequalities hold:
0 < f k , a f k , i f k , b f k , i k = 1 , . . , n a 0 < f k , b f k , i k = n a + 1 , . . , n b 0 f k , b f k , i k = n b + 1 , . . , n r f i , a = k = 1 n a [ f k , a f k , i ] 2 + k = n a + 1 n b 0 + k = n b + 1 n 0 k = 1 n a [ f k , b f k , i ] 2 + k = n a + 1 n b [ f k , b f k , i ] 2 + k = n b + 1 n 0 = r f i , b s i , A r f i , a r f i , b = s i , B .
It is worth noting that:
  • rfi,a < rfi,b if nbna;
  • rfi,arfi,b if nb = na.
In the last case, rfi,a = rfi,b iff fk,a = fk,b ∀ k = 1,..,na. In this case, obviously, fj,a < fj,bfj,i whit j > na. As said before, the proof that ∃ i*∊POF: si*,A < si*,B has been reported in Appendix A.

7. ⋫-Compatibility

In this section it is proved that DOA is a ⋫-compatible quality indicator. The following remarks are necessary.
Remark 3.
A solution a’ belonging to A does not influence DOA(A) if, ∀ iPOF:
  • si,A < dfi,a’ when a’ is dominated by i;
  • si,A < rfi,a’ when a’ is not dominated by i.
In fact, in this case si,Adfi,a’ and si,Arfi,a’, by which follows that DOA(A) does not change its value if a’ is moved off from A. These means that an APF B = A\{a’} has the same quality indicator value, i.e., DOA(A) = DOA(B). Using DOA seems B equivalent to A while A, having one more solution, is better than B [11]. Then the proposed method is not a ⊳-complete quality indicator. On the other hand, DOA together with its interpretation function is a ≻-complete comparison method, as it is demonstrated before.
Remark 4.
The relation A⊲B is equivalent to assume that B = C ⋃ D, where C ⊊ A, D ⋂ A = Ø and ∀ bDaA: a≺b. Note that, when C = Ø then A≺B. Hence, the difference between case A≺B and A⊲B is that in the latter B could contains some solutions of A. Moreover, obviously, only the last case includes B ⊊ A (i.e., D = Ø). For these reasons when A⊲B, DOA(A) is never greater than DOA(B).
Remark 5.
The relation A||B involves that DOA(A) can be greater or less than or equal to DOA(B).
For two generic APF A and B, the ≻-completeness of DOA, together with Remarks 3–5, imply that if DOA(A) < DOA(B) then A≺B or A⊲B or A||B, i.e., it is sure that B⋪A. Then DOA is a ⋫-compatible unary quality indicator.

8. DOA Validation

To demonstrate that DOA takes into account all features (closeness, distribution, extension, and cardinality) three typical POFs have been considered:
  • convex and connected;
  • non-convex and connected;
  • convex and disconnected.
In particular, the DOA unary quality indicator has been computed for different examples of APFs. Both POFs and APFs are drawn in Figure 14, Figure 15 and Figure 16. Table 4 shows the values of DOA for the APFs considered.
Let us examine the strategy adopted to choose the examples of APF for each POF: the APF B obtained by improving a given feature of APF A is expected to have a better indicator value than that of A if the indicator is sensitive to this feature. For example, if an APF is gradually moved towards the POF and the indicator increasingly improves, then the indicator is influenced by the closeness feature. The validation was performed according to these considerations. Hence, from an APF (APF1, indicated in the figures by symbol◊) with only poor features, the second APF (APF2, indicated by +) is created by improving APF1 closeness that is by converging APF1 on the POF. Therefore, APF2 has better closeness than APF1 but the same distribution, extension and cardinality. The third APF (APF3, indicated by ○) is obtained by improving the distribution of APF2 by uniformly distributing the solutions of APF2. APF3 has better distribution than APF2 but the same closeness, extension and cardinality. The fourth APF (APF4, indicated by □) is created by improving the extension of APF3, preserving the other features. Finally, the fifth APF (APF5, indicated by ●) is created by adding new points to APF4, and hence improving the cardinality of APF5 with respect to the fourth APF. It is worth noting that it does not matter in which order the different features are added to the initial APF1, and the order used in Table 4 simply follows the features’ description given in the Introduction.
Whatever the characteristics of the POF, such a strategy highlights that the value of DOA decreases when one feature improves.
The results in Table 4 highlight the ≻-completeness of DOA too. In fact, the first APF is dominated by the others and it has a DOA value worse than those of the other APFs.

9. Conclusions and Future Work

Evaluating the performance of MOOAs is very difficult because their comparison involves comparing APFs. QIs are used to measure the goodness of the APF provided by different optimization algorithms to highlight which works better.
Therefore, a QI must be able to account for Pareto dominance to properly compare two different algorithms. Moreover, when APFs are incomparable, further data (closeness, distribution, extension and cardinality) must be taken into account to compare the APFs provided by different MOOAs. Few UQIs are ≻-complete and able to account for the aforementioned features, yet they require great computational effort.
This paper has described the DOA unary quality indicator, which could be very useful in assessing the performance of an MOOA by estimating the match between the approximation front found by the MOOA and the optimal one. It has been proven that it is ≻-complete, ⋫-compatible, and requires little computational cost. Moreover, a numerical validation was carried out to demonstrate that it accounts for closeness, distribution, extension, and cardinality. An implementation of the DOA indicator is available online: http://wwwelfin.diees.unict.it/esg/DOA.html.
The major drawback of the proposed indicator is that it is not ⊳-complete. Even though the dominance relation A ⊲ B differs from A ≺ B since the former accounts for the case in which B contains some solutions of A, the probability of this specific event is next to nil. Nevertheless, a future development of this work is to properly modify DOA to make it ⊳-complete, maintaining low computational complexity. In this perspective, the underlying reasons leading from a not ≻-complete indicator, IGD, to the similar one proposed in this paper, DOA, having ≻-completeness can be the enabling keys.
Moreover, an integral formulation of DOA can be developed to deal with continuous POFs that can be expressed by an analytical function.
Finally, innovative studies have proposed interesting strategies to use UQIs requiring the knowledge of the POF as the guidance mechanism for indicator-based MOEAs. Therefore, the use of DOA for indicator-based evolutionary algorithms is another attractive development of the work.

Author Contributions

The authors contributed equally to this work.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A.

Hypothesis. 
A≺B.
Thesis. 
i* ∊ POF: si*,A = di*,A < si*,B.
Proof by Reductio ad absurdum.
In this proof some remarks must be taken into account:
  • dfi*,a* = di*,Adfi*,a*dfi*,a’a’∊Di*,A
  • dfi*,a* < ri*,Adfi*,a* < rfi*,a″a″∊A\Di*,A
  • when iabdfi,a < dfi,b (see Section C2)
  • when i||aibabrfi,adfi,b (see Section D2)
  • when i||ai||babrfi,arfi,b (see Section E2)
  • i*∊ POF and ∃ a*∊A: si*,A = di*,A = dfi*,a* < ri*,A (proved in Appendix B).
Assume the opposite of the thesis:
si*,B ≤ si*,A;
from(A1) and Remark (VI), it follows that:
b’Di*,B: si*,B = dfi*,b’ (≤ dfi*,a* = si*,A)
or
b″B\Di*,B: si*,B = rfi*,b″ (≤ dfi*,a* = si*,A).
Considering inequality (A2), three scenarios are to be analyzed:
(a)
a*b’
(b)
a’b’, where a’Di*,A
(c)
a″b’, where a″∊A\Di*,A.
In Scenario (a), from Remark (III), follows:
s i * , A = d i * , A = d f i * , a * < d f i * , b = s i * , B s i * , B > s i * , A ,
which is in contradiction with (A1).
In Scenario (b), considering Remarks (I) and (III), follows:
s i * , A = d i * , A = d f i * , a * d f i * , a < d f i * , b = s i * , B s i * , B > s i * , A ,
which is in contradiction with (A1).
In Scenario (c), by considering Remarks (II) and (IV), follows:
s i * , A = d i * , A = d f i * , a * < r f i * , a d f i * , b = s i * , B s i * , B > s i * , A ,
which is in contradiction with (A1).
Considering (A3), si*,B = rfi*,b″ implies i*||b″, i.e., ∄ aDi*,A: ab″. Hence only one scenario has to be analyzed: a″b″, where a″∊A\Di*,A.
By considering Remarks (II) and (V), follows:
s i * , A = d i * , A = d f i * , a * < r f i * , a r f i * , b = s i * , B s i * , B > s i * , A ,
which is in contradiction with (A1). □

Appendix B.

Hypothesis. 
A is a generic APF.
Thesis. 
i ∊ POF and ∃ a∊A: si,A = di,A = dfi,a < ri,A.
Proof. 
The proof is given by induction, starting from |A| = 1 and adding to A other solutions recursively. First, only two objective functions (n = 2) are considered, then the same procedure is used for the general case (n > 2).

B.1. Proof by Induction with n = 2

Hypothesis. 
|A| = 1.
Proof. 
Being A = {a} and recalling that POF≼A, follows that there exist a solution i∊POF for which ia, then aDi,A and |A\Di,A| = 0. This implies that:
d i , A = d f i , a < r i , A = s i , A = d i , A = d f i , a < r i , A
Hypothesis. 
|A| = 2.
Proof. 
Being A = {a1,a2}, if ∃ i∊POF: ia1ia2, then a1 and a2 belong to Di,A and |A\Di,A| = 0:
d i , A = d f i , a = min ( d f i , a 1 , d f i , a 2 ) < r i , A = s i , A = d i , A = d f i , a < r i , A ,
where a is equal to the solution which provides si,A between a1 and a2.
On the other hand, when there does not exist such a solution i, recalling that POF≼A, then there exist two solutions i1, i2 belonging to the POF for which i1a1 and i2a2. Obviously, i1||a2 and i2||a1. Hence, without loss of generality it is assumed that:
f 1 , i 2 f 1 , a 2 < f 1 , i 1 f 1 , a 1 f 2 , i 1 f 2 , a 1 < f 2 , i 2 f 2 , a 2 .
The thesis is proved if si1,A = di1,A = dfi1,a1 < rfi1,a2 = ri1,Asi2,A = di2,A = dfi2,a2 < rfi2,a1 = ri2,A. The proof is by reductio ad absurdum. Supposing that:
si1,A = ri1,A = rfi1,a2 < dfi1,a1 = di1,Asi2,A = ri2,A = rfi2,a1 < dfi2,a2 = di2,A,
in which:
r f i 1 , a 2 = f 2 , a 2 f 2 , i 1 d f i 1 , a 1 = [ f 1 , a 1 f 1 , i 1 ] 2 + [ f 2 , a 1 f 2 , i 1 ] 2 r f i 2 , a 1 = f 1 , a 1 f 1 , i 2 d f i 2 , a 2 = [ f 1 , a 2 f 1 , i 2 ] 2 + [ f 2 , a 2 f 2 , i 2 ] 2 .
Note that:
[ f 1 , a 1 f 1 , i 1 ] 2 + [ f 2 , a 1 f 2 , i 1 ] 2 < f 1 , a 1 f 1 , i 1 + f 2 , a 1 f 2 , i 1 [ f 1 , a 2 f 1 , i 2 ] 2 + [ f 2 , a 2 f 2 , i 2 ] 2 < f 1 , a 2 f 1 , i 2 + f 2 , a 2 f 2 , i 2 ,
because the quantities in the brackets are not negative. From (A11) and (A13) follows:
f 2 , a 2 f 2 , i 1 = r f i 1 , a 2 < d f i 1 , a 1 < f 1 , a 1 f 1 , i 1 + f 2 , a 1 f 2 , i 1 f 1 , a 1 f 1 , i 2 = r f i 2 , a 1 < d f i 2 , a 2 < f 1 , a 2 f 1 , i 2 + f 2 , a 2 f 2 , i 2 0 < f 2 , a 2 + f 2 , i 1 + f 1 , a 1 f 1 , i 1 + f 2 , a 1 f 2 , i 1 0 < f 1 , a 1 + f 1 , i 2 + f 1 , a 2 f 1 , i 2 + f 2 , a 2 f 2 , i 2 ,
then
0 < [ f 1 , a 1 + f 1 , i 2 + f 1 , a 1 f 1 , i 1 + f 1 , a 2 f 1 , i 2 ] +    + [ f 2 , a 2 + f 2 , i 1 + f 2 , a 1 f 2 , i 1 + f 2 , a 2 f 2 , i 2 ] = = f 1 , a 2 f 1 , i 1 + f 2 , a 1 f 2 , i 2 , ,
which leads to an absurdity because, according to(A10), it is:
f 1 , a 2 f 1 , i 1 < 0 f 2 , a 1 f 2 , i 2 < 0 .
Adding to A other solutions recursively, the previous procedure can be applied to demonstrate that the thesis is always true whatever |A| is.

B.2. Proof by Induction with n = 3

Hypothesis. 
|A| = 1.
Proof. 
The proof is the same provided for n = 2 in (A8).
Hypothesis. 
|A| = 2
Proof. 
Being A = {a1,a2}, if ∃ i∊POF: ia1 Λ ia2, then a1 and a2 belong to Di,A and |A\Di,A| = 0 ⇒ The proof is the same provided in (A9).
On the other hand, when does not exist such solution i, recalling that POF≼A then there exist two solutions i1, i2 belonging to the POF for which i1a1 and i2a2. Obviously, i1||a2 and i2||a1. Hence, without loss of generality it is assumed that:
f 1 , i 2 f 1 , a 2 < f 1 , i 1 f 1 , a 1 f 2 , i 1 f 2 , a 1 < f 2 , i 2 f 2 , a 2 f 3 , i 1 f 3 , a 1 < f 3 , i 2 f 3 , a 2 .
Likewise, for the case of two objective functions the thesis it will be proven that si1,A = di1,A = dfi1,a1 < rfi1,a2 = ri1,Asi2,A = di2,A = dfi2,a2 < rfi2,a1 = ri2,A.
The proof is by reductio ad absurdum.
Assuming that:
si1,A = ri1,A = rfi1,a2 < dfi1,a1 = di1,Asi2,A = ri2,A = rfi2,a1 < dfi2,a2 = di2,A,
which implies:
r f i 1 , a 2 = [ f 2 , a 2 f 2 , i 1 ] 2 + [ f 3 , a 2 f 3 , i 1 ] 2 < [ f 1 , a 1 f 1 , i 1 ] 2 + [ f 2 , a 1 f 2 , i 1 ] 2 + [ f 3 , a 1 f 3 , i 1 ] 2 = d f i 1 , a 1 [ f 2 , a 2 f 2 , i 1 ] 2 + [ f 3 , a 2 f 3 , i 1 ] 2 < [ f 1 , a 1 f 1 , i 1 ] 2 + [ f 2 , a 1 f 2 , i 1 ] 2 + [ f 3 , a 1 f 3 , i 1 ] 2
and
r f i 2 , a 1 = f 1 , a 1 f 1 , i 2 < [ f 1 , a 2 f 1 , i 2 ] 2 + [ f 2 , a 2 f 2 , i 2 ] 2 + [ f 3 , a 2 f 3 , i 2 ] 2 = d f i 2 , a 2 [ f 1 , a 1 f 1 , i 2 ] 2 < [ f 1 , a 2 f 1 , i 2 ] 2 + [ f 2 , a 2 f 2 , i 2 ] 2 + [ f 3 , a 2 f 3 , i 2 ] 2 .
Summing each member of (A19) and (A20) follows:
[ f 1 , a 1 f 1 , i 2 ] 2 + [ f 2 , a 2 f 2 , i 1 ] 2 + [ f 3 , a 2 f 3 , i 1 ] 2 < [ f 1 , a 1 f 1 , i 1 ] 2 + [ f 2 , a 1 f 2 , i 1 ] 2 + [ f 3 , a 1 f 3 , i 1 ] 2 + + [ f 1 , a 2 f 1 , i 2 ] 2 + [ f 2 , a 2 f 2 , i 2 ] 2 + [ f 3 , a 2 f 3 , i 2 ] 2 0 < [ f 1 , a 1 f 1 , i 1 ] 2 + [ f 1 , a 2 f 1 , i 2 ] 2 [ f 1 , a 1 f 1 , i 2 ] 2 +     + [ f 2 , a 1 f 2 , i 1 ] 2 + [ f 2 , a 2 f 2 , i 2 ] 2 [ f 2 , a 2 f 2 , i 1 ] 2 +     + [ f 3 , a 1 f 3 , i 1 ] 2 + [ f 3 , a 2 f 3 , i 2 ] 2 [ f 3 , a 2 f 3 , i 1 ] 2 .
Note that the quantities in the brackets are not negative and hence:
0 < [ f 1 , a 1 f 1 , i 1 ] 2 + [ f 1 , a 2 f 1 , i 2 ] 2 [ f 1 , a 1 f 1 , i 2 ] 2     + [ f 2 , a 1 f 2 , i 1 ] 2 + [ f 2 , a 2 f 2 , i 2 ] 2 [ f 2 , a 2 f 2 , i 1 ] 2 +     + [ f 3 , a 1 f 3 , i 1 ] 2 + [ f 3 , a 2 f 3 , i 2 ] 2 [ f 3 , a 2 f 3 , i 1 ] 2 < < [ f 1 , a 1 f 1 , i 1 + f 1 , a 2 f 1 , i 2 ] 2 [ f 1 , a 1 f 1 , i 2 ] 2 +     + [ f 2 , a 1 f 2 , i 1 + f 2 , a 2 f 2 , i 2 ] 2 [ f 2 , a 2 f 2 , i 1 ] 2 +     + [ f 3 , a 1 f 3 , i 1 f 3 , a 2 f 3 , i 2 ] 2 [ f 3 , a 2 f 3 , i 1 ] 2 < < f 1 , a 1 f 1 , i 1 + f 1 , a 2 f 1 , i 2 f 1 , a 1 + f 1 , i 2 +     + f 2 , a 1 f 2 , i 1 + f 2 , a 2 f 2 , i 2 f 2 , a 2 + f 2 , i 1 +     + f 3 , a 1 f 3 , i 1 f 3 , a 2 f 3 , i 2 f 3 , a 2 + f 3 , i 1 = = f 1 , a 2 f 1 , i 1 + f 2 , a 1 f 2 , i 2 + f 3 , a 1 f 3 , i 2 ,
which leads to an absurdity because, according to(A17), it is:
f 1 , a 2 f 1 , i 1 < 0 f 2 , a 1 f 2 , i 2 < 0 f 3 , a 1 f 3 , i 2 < 0 .
The cases with n > 3 can be proved in a similar way.

References

  1. Zitzler, E.; Thiele, L. Multiobjective optimization using evolutionary algorithms—A comparative case study. In Parallel Problem Solving from Nature-PPSN V; Springer: Heidelberg/Berlin, Germany, 1998; Volume 1498, pp. 292–301. [Google Scholar]
  2. Lei, Y.; Gong, M.; Zhang, J.; Li, W.; Jiao, L. Resource allocation model and double-sphere crowding distance for evolutionary multi-objective optimization. Eur. J. Oper. Res. 2014, 234, 197–208. [Google Scholar] [CrossRef]
  3. Wang, R.; Purshouse, R.C.; Fleming, P.J. Preference-inspired co-evolutionary algorithms using weight vectors. Eur. J. Oper. Res. 2015, 243, 423–441. [Google Scholar] [CrossRef]
  4. Sinha, A.; Korhonen, P.; Wallenius, J.; Deb, K. An interactive evolutionary multi-objective optimization algorithm with a limited number of decision maker calls. Eur. J. Oper. Res. 2014, 233, 674–688. [Google Scholar] [CrossRef]
  5. Ghosh, A.; Dehuri, S. Evolutionary Algorithms for Multi-Criterion Optimization: A Survey. Int. J. Comput. Inf. Sci. 2004, 2, 38–57. [Google Scholar]
  6. Dilettoso, E.; Rizzo, S.A.; Salerno, N. A Parallel Version of the Self-Adaptive Low-High Evaluation Evolutionary-Algorithm for Electromagnetic Device Optimization. IEEE Trans. Magn. 2014, 50, 633–636. [Google Scholar] [CrossRef]
  7. Wilfried, J.; Blume, C. Pareto Optimization or Cascaded Weighted Sum: A Comparison of Concepts. Algorithms 2014, 7, 166–185. [Google Scholar]
  8. Deb, K. Multi-Objective Optimization using Evolutionary Algorithms; John Wiley & Sons: Hoboken, NJ, USA, 2001. [Google Scholar]
  9. Chen, B.; Zeng, W.; Lin, Y.; Zhang, D. A New Local Search-Based Multiobjective Optimization Algorithm. IEEE Trans. Evol. Comput. 2015, 19, 50–73. [Google Scholar] [CrossRef]
  10. Talbi, E.G.; Basseur, M.; Nebro, A.J.; Alba, E. Multi-objective optimization using metaheuristics: Non-standard algorithms. Int. Trans. Oper. Res. 2012, 19, 283–305. [Google Scholar] [CrossRef] [Green Version]
  11. Zitzler, E.; Thiele, L.; Laumanns, M.; Fonseca, C.M.; Fonseca, V.G. Performance assessment of multiobjective optimizers: An analysis and review. IEEE Trans. Evol. Comput. 2003, 7, 117–132. [Google Scholar] [CrossRef]
  12. Fonseca, C.M.; Fleming, P.J. On the performance assessment and comparison of stochastic multiobjective optimizers. In Parallel Problem Solving from Nature (PPSN-IV); Springer: Berlin, Germany, 1996; pp. 584–593. [Google Scholar]
  13. Zitzler, E.; Deb, K.; Thiele, L. Comparison of multiobjective evolutionary algorithms: Empirical results. Evol. Comput. 2000, 8, 173–195. [Google Scholar] [CrossRef] [PubMed]
  14. Shukla, P.K.; Deb, K. On finding multiple Pareto-optimal solutions using classical and evolutionary generating methods. Eur. J. Oper. Res. 2007, 181, 1630–1652. [Google Scholar] [CrossRef]
  15. Chen, Y.; Zou, X.; Xie, W. Convergence of multi-objective evolutionary algorithms to a uniformly distributed representation of the Pareto front. Inf. Sci. 2011, 181, 3336–3355. [Google Scholar] [CrossRef]
  16. Lizárraga, G.; Gomez, M.J.; Castañon, M.G.; Acevedo-Davila, J.; Rionda, S.B. Why Unary Quality Indicators Are Not Inferior to Binary Quality Indicators. In MICAI 2009 Advances in Artificial Intelligence; Aguirre, A.H., Borja, R.M., Garciá, C.A.R., Eds.; Springer: Berlin, Germany, 2009; Volume 5845, pp. 646–657. [Google Scholar]
  17. Dilettoso, E.; Rizzo, S.A.; Salerno, N. A new indicator to assess the quality of a Pareto approximation set applied to improve the optimization of a magnetic shield. In Proceedings of the Scientific Computing in Electrical Engineering, Zurich, Switzerland, 11–14 September 2012.
  18. Deb, K.; Pratap, A.; Agarwal, S.; Meyariva, T. A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II. IEEE Trans.Evol. Comput. 2002, 6, 182–197. [Google Scholar] [CrossRef]
  19. Schaffer, J.D. Multiple objective optimization with vector evaluated genetic algorithms. In Proceedings of the First International Conference on Genetic Algorithms, Pittsburgh, PA, USA, 24–26 July 1985; Grefensttete, J.J., Ed.; Lawrence ErlbaumAssociates Inc.: Hillsdale, NJ, USA, 1987; pp. 93–100. [Google Scholar]
  20. Fonseca, C.M.; Fleming, P.J. An overview of evolutionary algorithms in multi-objective optimization. Evol. Comput. J. 1995, 3, 1–16. [Google Scholar] [CrossRef]
  21. Poloni, C. Hybrid GA for Multiobjective aerodynamic shape optimization. In Genetic Algorithms in Engineering and Computer Science; Winter, G., Periaux, J., Galan, M., Cuesta, P., Eds.; Wiley: New York, NY, USA, 1997; pp. 397–414. [Google Scholar]
  22. Kursawe, F. A variant of evolution strategies for vector optimization. In Parallel Problem Solving from Nature; Schwefel, H.-P., Männer, R., Eds.; Springer: Berlin, Germany, 1990; pp. 193–197. [Google Scholar]
  23. Deb, K.; Thiele, L.; Laumanns, M.; Zitzler, E. Scalable multiobjective optimization test problems. In Proceedings of the IEEE Congress on Evolutionary Computation, Honolulu, HI, USA, 12–17 May 2002; pp. 825–830.
  24. Beume, N.; Naujoks, B.; Emmerich, M. SMS-EMOA: Multiobjective selection based on dominated hypervolume. Eur. J. Oper. Res. 2007, 181, 1653–1669. [Google Scholar] [CrossRef]
  25. Schütze, O.; Domínguez-Medina, C.; Cruz-Cortés, N.; de la Fraga, L.G.; Sun, J.-Q.; Toscano, G.; Landa, R. A scalar optimization approach for averaged Hausdorff approximations of the Pareto front. Eng. Optim. 2016, 48, 1593–1617. [Google Scholar] [CrossRef]
  26. Zitzler, E.; Knowles, J.D.; Thiele, L. Quality assessment of pareto set approximations. In Multiobjective Optimization; Lecture Notes in Computer Science; Springer: Berlin, Germany, 2008; Volume 5252, pp. 373–404. [Google Scholar]
  27. Laumanns, M.; Zenklusen, R. Stochastic convergence of random search methods to fixed size Pareto front approximations. Eur. J. Oper. Res. 2011, 213, 414–421. [Google Scholar] [CrossRef]
  28. Fonseca, C.M.; Knowles, J.D.; Thiele, L.; Zitzler, E. A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers. Available online: http://www.tik.ee.ethz.ch/pisa/publications/emo-tutorial-2up.pdf (accessed on 15 December 2016).
  29. Coello Coello, C.A.; Sierra, M.R. A Study of the Parallelization of a Coevolutionary Multi-objective Evolutionary Algorithm. Lect. Notes Comput. Sci. 2004, 2972, 688–697. [Google Scholar]
  30. Czyzak, P.; Jaskiewicz, A. Pareto simulated annealing—a metaheuristic technique for multi-objective combinatorial optimization. J. MultiCriteria Decis. Anal. 1998, 7, 34–47. [Google Scholar] [CrossRef]
  31. Lotov, A.V.; Kamenev, G.K.; Berezkin, V.E. Approximation and Visualization of Pareto-Efficient Frontier for Nonconvex Multiobjective Problems. Dokl. Math. 2002, 66, 260–262. [Google Scholar]
  32. Lotov, A.V.; Bushenkov, V.A.; Kamenev, G.K. Interactive Decision Maps: Approximation and Visualization of Pareto Frontier; Springer: Boston, MA, USA, 2004. [Google Scholar]
  33. Srinivas, N.; Deb, K. Multiobjective optimization using nondominated sorting in genetic algorithms. Evol. Comput. 1994, 2, 221–248. [Google Scholar] [CrossRef]
  34. Van Veldhuizen, D.A. Multiobjective Evolutionary Algorithms: Classifications, Analyzes, and New Innovations. Ph.D. Thesis, Engineering of the Air Force Institute of Technology University, Wright-Patterson AFB, OH, USA, 1999. [Google Scholar]
  35. Wu, J.; Azarm, S. Metrics for Quality Assessment of a Multiobjective Design Optimization Solution Set. J. Mech. Des. 2001, 123, 18–25. [Google Scholar] [CrossRef]
  36. Schott, J.R. Fault Tolerant Design Using Single and Multicriteria Genetic Algorithm Optimization. Mc.S. Thesis, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Boston, MA, USA, 1995. [Google Scholar]
  37. Van Veldhuizen, D.A.; Lamont, G.B. On measuring multiobjective evolutionary algorithm performance. In Proceedings of the 2000 Congress on Evolutionary Computation, La Jolla, CA, USA, 16–19 July 2000; pp. 204–211.
  38. Tan, K.C.; Khor, E.F.; Lee, T.H. Evolutionary Algorithms for Multi-Objective Optimization: Performance Assessments and Comparisons. J. Artif. Intel. Rev. 2002, 17, 253–290. [Google Scholar]
  39. Schutze, O.; Esquivel, X.; Lara, A.; Coello Coello, C.A. Using the Averaged Hausdorff Distance as a Performance Measure in Evolutionary Multi-Objective Optimization. IEEE Trans. Evol. Comput. 2012, 16, 504–522. [Google Scholar] [CrossRef]
  40. Rudolph, G.; Schütze, O.; Grimme, C.; Medina, C.D.; Trautmann, H. Optimal averaged Hausdorff archives for bi-objective problems: Theoretical and numerical results. Comput. Optim. Appl. 2016, 64, 589–618. [Google Scholar] [CrossRef]
  41. While, L. A new analysis of the LebMeasure algorithm for calculating hypervolume. In Evolutionary Multi-Criterion Optimization; Lecture Notes in Computer Science; Springer: Heidelberg/Berlin, Germany, 2005; Volume 3410, pp. 326–340. [Google Scholar]
  42. While, L.; Bradstreet, L.; Barone, L.; Hingston, P. Heuristics for optimising the calculation of hypervolume for multi-objective optimisation problems. In Proceedings of the IEEE Congress on Evolutionary Computation, Edinburgh, UK, 2–5 September 2005; pp. 2225–2232.
  43. Fonseca, C.M.; Paquete, L.; López-Ibáñez, M. An improved dimension-sweep algorithm for the hypervolume indicator. In Proceedings of the IEEE Congress on Evolutionary Computation, Vancouver, BC, Canada, 16–21 July 2006; pp. 1157–1163.
  44. Beume, N.; Rudolph, G. Faster S-Metric calculation by considering dominated hypervolume as Klee’s measure problem. In Proceedings of the 2nd IASTED Conference on Computational Intelligence, San Francisco, CA, USA, 20–22 November 2006; pp. 231–236.
  45. Knowles, J.; Corne, D.; Fleischer, M. Bounded archiving using the lebesgue measure. In Proceedings of the 2003 Congress on Evolutionary Computation, Canberra, Australia, 8–12 December 2003; Volume 4, pp. 2490–2497.
  46. While, L.; Barone, L.; Hingston, P.; Huband, S. A Faster Algorithm for Calculating Hypervolume. IEEE Trans. Evol.Comput. 2006, 10, 29–38. [Google Scholar] [CrossRef]
Figure 1. In this case A and B are incomparable in terms of Pareto dominance, but A is preferable to B because A is closer to the Pareto-optimal front (POF), more extended, more populated, and better distributed.
Figure 1. In this case A and B are incomparable in terms of Pareto dominance, but A is preferable to B because A is closer to the Pareto-optimal front (POF), more extended, more populated, and better distributed.
Mca 22 00025 g001
Figure 2. Comparison method in case of one QI (a) and in case of a vector of q QIs (b).
Figure 2. Comparison method in case of one QI (a) and in case of a vector of q QIs (b).
Mca 22 00025 g002
Figure 3. An APF (blue dots) with all good features but not close to the POF and another (red triangles) that is only close to the POF.
Figure 3. An APF (blue dots) with all good features but not close to the POF and another (red triangles) that is only close to the POF.
Mca 22 00025 g003
Figure 4. An APF (blue dots) with all good features but not uniformly distributed and another (red triangles) that is only uniformly distributed.
Figure 4. An APF (blue dots) with all good features but not uniformly distributed and another (red triangles) that is only uniformly distributed.
Mca 22 00025 g004
Figure 5. An APF blue dots) with all good features but not extended and another (red triangles) that is only extended.
Figure 5. An APF blue dots) with all good features but not extended and another (red triangles) that is only extended.
Mca 22 00025 g005
Figure 6. An APF (blue dots) with all good features but with poor cardinality and another (red triangles) that has only high cardinality.
Figure 6. An APF (blue dots) with all good features but with poor cardinality and another (red triangles) that has only high cardinality.
Mca 22 00025 g006
Figure 7. An APF with all the desired features.
Figure 7. An APF with all the desired features.
Mca 22 00025 g007
Figure 8. Di,A of a point i belonging to the POF (example with n = 2).
Figure 8. Di,A of a point i belonging to the POF (example with n = 2).
Mca 22 00025 g008
Figure 9. di,A of a point i belonging to the POF.
Figure 9. di,A of a point i belonging to the POF.
Mca 22 00025 g009
Figure 10. ri,A of a point i belonging to the POF.
Figure 10. ri,A of a point i belonging to the POF.
Mca 22 00025 g010
Figure 11. Interpretation function: pseudo-code to compare A and B by means of DOA(A) and DOA(B).
Figure 11. Interpretation function: pseudo-code to compare A and B by means of DOA(A) and DOA(B).
Mca 22 00025 g011
Figure 12. Possible scenarios for A≻≻B.(a) scenario A1: most relevant case; (b) scenario B1 or C1; (c) scenario C1: limit case; (d) scenario D1.
Figure 12. Possible scenarios for A≻≻B.(a) scenario A1: most relevant case; (b) scenario B1 or C1; (c) scenario C1: limit case; (d) scenario D1.
Mca 22 00025 g012
Figure 13. Possible scenarios for A≻B. (a) scenario A2 or C2; (b) scenario B2 or D2; (c) scenario C2: limit case; (d) scenario D2: limit case; (e) scenario E2.
Figure 13. Possible scenarios for A≻B. (a) scenario A2 or C2; (b) scenario B2 or D2; (c) scenario C2: limit case; (d) scenario D2: limit case; (e) scenario E2.
Mca 22 00025 g013
Figure 14. Convex and connected POF (solid line), APF1 (◊), APF2 (+), APF3 (○), APF4 (□) and APF5 (●).
Figure 14. Convex and connected POF (solid line), APF1 (◊), APF2 (+), APF3 (○), APF4 (□) and APF5 (●).
Mca 22 00025 g014
Figure 15. Non-convex and connected POF (solid line), APF1 (◊), APF2 (+), APF3 (○), APF4 (□) and APF5 (●).
Figure 15. Non-convex and connected POF (solid line), APF1 (◊), APF2 (+), APF3 (○), APF4 (□) and APF5 (●).
Mca 22 00025 g015
Figure 16. Convex and disconnected POF (solid line), APF1 (◊), APF2 (+), APF3 (○), APF4 (□) and APF5 (●).
Figure 16. Convex and disconnected POF (solid line), APF1 (◊), APF2 (+), APF3 (○), APF4 (□) and APF5 (●).
Mca 22 00025 g016
Table 1. Dominance relations between two solutions [11].
Table 1. Dominance relations between two solutions [11].
SymbolRelationDescription
x1≺≺x2strictly dominance x1 strictly dominates x2x1 is better than x2 with respect to each objective function
x1x2dominance x1 dominates x2x1 is not worse than x2 with respect to each objective function and x1 is better than x2 by at least one objective function
x1x2weakly dominance x1 weakly dominates x2x1 is not worse than x2 with respect to each objective function
x1||x2incomparability x1 and x2 are incomparablex1 and x2 do not weakly dominate each other
Table 2. Dominance relations between two APFs [11].
Table 2. Dominance relations between two APFs [11].
SymbolRelationDescription
A≺≺BA strictly dominates Beach solution belonging to B is strictly dominated by a solution belonging to A
ABA dominates Beach solution belonging to B is dominated by a solution belonging to A
ABA is better than Beach solution belonging to B is weakly dominated by a solution belonging to A, and AB
ABA weakly dominates Beach solution belonging to B is weakly dominated by a solution belonging to A
A||BA and B are incomparableA and B do not weakly dominate each other
Table 3. Summary of selected UQIs and features that influence their value.
Table 3. Summary of selected UQIs and features that influence their value.
IndicatorClosenessDistributionExtensionCardinality
Average Distance from Reference Set [30]
Chi-Square-Like Deviation Measure [33] Mca 22 00025 i001 Mca 22 00025 i001
Completeness Indicator [31,32]
Enclosing Hypercube [11] Mca 22 00025 i001
Generational Distance [34]
Hypervolume [1]
Inverted Generational Distance [29]
M1* [13]
M2* [13] Mca 22 00025 i001
M3* [13]
Maximum Pareto Front Error [34]
Outer Diameter [26]
Overall Nondominated Vector Generation [34]
Overall Pareto Spread [35]
Potential Function [27]
Seven Points Average Distance [36] Mca 22 00025 i001
Spacing [37]
Unary ε-Indicator [11,26]
Uniform Distribution [38]
Worst Distance from Reference Set [30]
Δ [8]
Δp [39,40]
Table 4. DOA Evaluation for some typical POF (symbol in brackets is the marker associated to the APF in Figure 14, Figure 15 and Figure 16).
Table 4. DOA Evaluation for some typical POF (symbol in brackets is the marker associated to the APF in Figure 14, Figure 15 and Figure 16).
POFAPFClosenessDistributionExtensionCardinalityDOA
Convex and connected (see Figure 14)APF1(◊)poorpoorpoorpoor0.71040
APF2(+)goodpoorpoorpoor0.16940
APF3(○)goodgoodpoorpoor0.16287
APF4(□)goodgoodgoodpoor0.10253
APF5(●)goodgoodgoodgood0.06431
Non-convex and connected (see Figure 15)APF1(◊)poorpoorpoorpoor0.79167
APF2(+)goodpoorpoorpoor0.24303
APF3(○)goodgoodpoorpoor0.23465
APF4(□)goodgoodgoodpoor0.09301
APF5(●)goodgoodgoodgood0.06993
Convex and disconnected (see Figure 16)APF1(◊)poorpoorpoorpoor0.69510
APF2(+)goodpoorpoorpoor0.16866
APF3(○)goodgoodpoorpoor0.16810
APF4(□)goodgoodgoodpoor0.07254
APF5(●)goodgoodgoodgood0.06331

Share and Cite

MDPI and ACS Style

Dilettoso, E.; Rizzo, S.A.; Salerno, N. A Weakly Pareto Compliant Quality Indicator. Math. Comput. Appl. 2017, 22, 25. https://doi.org/10.3390/mca22010025

AMA Style

Dilettoso E, Rizzo SA, Salerno N. A Weakly Pareto Compliant Quality Indicator. Mathematical and Computational Applications. 2017; 22(1):25. https://doi.org/10.3390/mca22010025

Chicago/Turabian Style

Dilettoso, Emanuele, Santi Agatino Rizzo, and Nunzio Salerno. 2017. "A Weakly Pareto Compliant Quality Indicator" Mathematical and Computational Applications 22, no. 1: 25. https://doi.org/10.3390/mca22010025

Article Metrics

Back to TopTop