Non-Dominated Sorting Manta Ray Foraging Optimization for Multi-Objective Optimal Power Flow with Wind/Solar/Small- Hydro Energy Sources

Abstract

This present study describes a novel manta ray foraging optimization approach based non-dominated sorting strategy, namely (NSMRFO), for solving the multi-objective optimization problems (MOPs). The proposed powerful optimizer can efficiently achieve good convergence and distribution in both the search and objective spaces. In the NSMRFO algorithm, the elitist non-dominated sorting mechanism is followed. Afterwards, a crowding distance with a non-dominated ranking method is integrated for the purpose of archiving the Pareto front and improving the optimal solutions coverage. To judge the NSMRFO performances, a bunch of test functions are carried out including classical unconstrained and constrained functions, a recent benchmark suite known as the completions on evolutionary computation 2020 (CEC2020) that contains twenty-four multimodal optimization problems (MMOPs), some engineering design problems, and also the modified real-world issue known as IEEE 30-bus optimal power flow involving the wind/solar/small-hydro power generations. Comparison findings with multimodal multi-objective evolutionary algorithms (MMMOEAs) and other existing multi-objective approaches with respect to performance indicators reveal the NSMRFO ability to balance between the coverage and convergence towards the true Pareto front (PF) and Pareto optimal sets (PSs). Thus, the competing algorithms fail in providing better solutions while the proposed NSMRFO optimizer is able to attain almost all the Pareto optimal solutions.

1. Introduction

Nowadays, meta-heuristics become popular in different research areas for resolving challenging optimization issues. These stochastic approaches are among the best and effective strategies in finding optimal solutions, conflicting with the classical (deterministic) optimization approaches which are devalued due to their drawbacks as local optima stagnation [1], etc. In spite of the benefits of the intelligence algorithms, they require some improvement to satisfy the diverse characteristics of complex real-world applications. Features that mostly faced in real issues are uncertainty [2], dynamicity [3], combinatorial, multiple objectives [4,5], constraints, etc. Along these lines, it is obviously seen that no approach is qualified in resolving the diverse kind of optimization problems. In that regard, the No-Free Lunch (NFL) theorem [6] validates this and opens the way for developers to create the newest approaches and enhances the quality of the existing ones.

Some of well-regarded meta-heuristic algorithms are: a genetic algorithm (GA), which is the first stochastic algorithm inspired by John Holland in 1960 [7], followed by simulated annealing (SA) in 1983 [8], particle swarm optimization (PSO) in 1995 by Kennedy [9], and more approaches that were developed later such as ant bee colony (ABC) [10], arithmetic optimization algorithm (AOA) [11], Harris hawks optimization (HHO) [12], sine cosine algorithm (SCA) [13], black widow optimization (BWO) [14], dynamic differential annealed optimization (DDAO) [15], levy flight distribution (LFD) [16], Salp swarm algorithm (SSA) [17], Henry gas solubility optimization (HGSO) [18], manta ray foraging optimization (MRFO) [19], and so on. All these developed stochastic algorithms are frequently single-objective; therefore, the researchers improve them according to the nature and complexity of their problems. Hence, in line with the aforementioned nature of applications, we have improved the new bio-inspired approach called manta ray foraging optimization (MRFO) [19] with a view to cope with the multi-objective problems (MOPs), which are the main focus of this paper.

During the two last years, several studies have guaranteed the superiority and efficiency of the MRFO algorithm in solving global optimization problems, such as: Fahd et al. [20], who applied the standard MRFO to perform the dynamic operation for connecting PV into the grid system. The authors in [21] have examined the global maximum power point of a partially shaded MJSC photovoltaic (PV) array applying the MRFO algorithm. Regarding the work of Selem et al. [22], the MRFO was applied to define the unknown electrical parameters of proton exchange membrane fuel cells stacks, which is considered as a constrained optimization problem. In addition, El-Hameed et al. [23] used MRFO to solve the solar module parameters’ identifications of a three diode equivalent model of PV. In addition, in an attempt to ameliorate the performance of this suggested approach, Dalia et al. [24] introduced a modified MRFO by using the fractional-order optimization algorithms, in order to enhance its exploitation ability. Referring to [25], a binary version of MRFO has been proposed using four S-Shaped and four V-Shaped transfer functions for the feature selection problem. In the bio-medical area, Karrupusamy utilized a hybrid MRFO to identify the issue in an existing brain tumor by using a convolutional neural network as a classifier that classifies the features and supplies optimal classification results [26], etc. The authors in [27] have used the multi-objective manta ray foraging optimization (MOMRFO) based weighted sum to handle the optimal power flow (OPF) problem for hybrid AC and multi-terminal direct current power grids. In Ref. [28], the authors applied the IMOMRFO to solve the IEEE-30 and IEEE-57 OPF issues.

In accordance with the literature review, the multi-objective algorithms (MOAs) are divided into two techniques: a priori versus a posteriori [29]. The first class converts the multi-objective problem to a single one, by aggregating all objectives in one function using a set of weights that are chosen by an expert in the problem domain (decision makers). The drawback of this method appears when we generate the Pareto optimal set, we should run the algorithm multiple times. Alternatively, the second class is the posteriori technique which does not require any addition weights. In this method, the multi-objective formulation is maintained and the Pareto optimal set is obtained in just one run; then, the decision-making occurs after the optimization. In addition, the Pareto front of all kinds of problems can be determined utilizing this posteriori technique, which is the focus of this work, in which a new multi-objective version of MRFO based non-dominated sorting approach named NSMRFO was developed.

Different shapes of fronts exist in multi-objective problems: linear, convex, concave, separated, and so on. Therefore, to obtain an accurate approximation Pareto optimal front for every multi-objective optimization issues, three fundamental challenges should be addressed: distribution of solutions (coverage), accuracy (convergence), and local fronts  [30]. Thus, an efficient algorithm is the one that has the ability to balance between them: avoid a premature convergence and extract a uniform distribution front covering the entire true Pareto optimal front. Some of the most popular multi-objective algorithms are: Non-dominated sorting genetic algorithm (NSGA) [31,32], strength Pareto evolutionary algorithm (SPEA) [33,34], multi-objective particle swarm optimization (MOPSO) [35], and multi-objective evolutionary algorithm based on decomposition (MOEA/D) [36].

Since the multi-objective optimization problem appears, the non-dominated sorting strategy with crowding distance and non-dominated ranking has been known as the efficient and most significant mechanisms in handling the algorithms for solving the multi-objective problems. The significant advantages of the NSGA-II and its borrowed MOAs motivated us to suggest a novel multi-objective variant of the MRFO approach, which is based on the NSGA-II outstanding operators. The search mechanism in MRFO is kept similar in the NSMRFO optimizer. Furthermore, in order to assess the NSMRFO success, various MMMOEAs with other MOEAs were investigated for comparisons with respect to diverse indicator metrics in search and objective spaces. Thus, with accordance to the statistical outcomes, the proposed NSMRFO outperformed its competitors and even the existing MOMRFOs for different kinds of problems.

The main contributions of this paper are as follows:

• A new multi-objective version of manta ray foraging optimization based the crowding distance and non-dominated sorting operators has been introduced.
• Various performance metrics have been employed in order to affirm the NSMRFO effectiveness.
• The suggested NSMRFO was benchmarked on the standard unconstrained, constrained multi-objective test suites, CEC2020 multi-modal multi-objective optimization functions as well as engineering design problems to verify its validity.
• The IEEE 30-bus OPF as one of the most significant real-world issues in the power system is investigated with wind/solar/small-hydro energy sources for the first time with the multi-objective case.

The remainder of this paper is arranged in four sections as follows: Section 2 summarizes the basic definitions of the multi-objective problems, and then describes the proposed algorithm MRFO and the structure of its multi-objective version NSMRFO. Simulation results, analyses, and competing algorithms are discussed in Section 3. As a final point, Section 4 concludes this work and proposes some future research directions.

2. Multi-Objective Optimization

As mentioned before, the multi-objective problem is the subject of handling problems that need optimizing more than one objective simultaneously, which are mostly in conflict. The basic mathematical formulation of the multi-objective optimization for such minimization problem can be defined as:

$$\begin{array}{cc}\hfill \mathrm{Minimize}:& F\left(\overrightarrow{x}\right)=\left\{f_1\left(\overrightarrow{x}\right),f_2\left(\overrightarrow{x}\right),\dots ,f_{N_{obj}}\left(\overrightarrow{x}\right)\right\}\hfill \\ \hfill \mathrm{Subject}\mathrm{to}:& g_i\left(\overrightarrow{x}\right)\ge 0,i=1,2,\dots ,m\hfill \\ \hfill & h_i\left(\overrightarrow{x}\right)=0,i=1,2,\dots ,p\hfill \\ \hfill & L_i\le x_i\le U_i,i=1,2,\dots ,n\hfill \end{array}$$

(1)

where $F\left(\overrightarrow{x}\right)$ is the objective function to be optimized, $h_i\left(\overrightarrow{x}\right)$ is the equality constraints, $g_i\left(\overrightarrow{x}\right)$ is the inequality constraints, $N_{obj}$, m, p, and n are the numbers of objective functions, inequality constraints, equality constraints, and variables. $L_i$ and $U_i$ are the boundaries of the ith variable.

The arithmetic relational operators cannot be effective in multi-objective optimization to compare the search space of different solutions. Alternatively, the Pareto optimal dominance concept is utilized to determine which solution is better than another. The essential definitions of dominance relation are defined as follows [37,38]:

Let us take two vectors $\overrightarrow{x}=\left(x_1,x_2,\dots ,x_n\right)$ and $\overrightarrow{y}=\left(y_1,y_2,\dots ,y_n\right)$

Definition 1

(Pareto Dominance). $\overrightarrow{x}$ is said to dominate $\overrightarrow{y}$ if and only if $\overrightarrow{x}$ is partially less than $\overrightarrow{y}$ (i.e., $\overrightarrow{x}\le \overrightarrow{y}$):

$$\forall i\in \{1,2,\dots ,N_{obj}\}:f_i\left(\overrightarrow{x}\right)\le f_i\left(\overrightarrow{y}\right)\wedge \exists i\in \{1,2,\dots ,N_{obj}\}:f_i\left(\overrightarrow{x}\right)<f_i\left(\overrightarrow{y}\right)$$

(2)

Definition 2

(Pareto Optimality). $\overrightarrow{x}\in X$ is called a Pareto-optimal solution iff:

$$\nexists \overrightarrow{y}\in X\mid F\left(\overrightarrow{y}\right)<F\left(\overrightarrow{x}\right)$$

(3)

Definition 3

(Pareto Optimal Set). The Pareto optimal set is a set that comprises all Pareto optimal solutions (neither $\overrightarrow{x}$ dominates $\overrightarrow{y}$ nor $\overrightarrow{y}$ dominates $\overrightarrow{x}$):

$$P_s:=\{x,y\in X\mid \exists F\left(\overrightarrow{y}\right)>F\left(\overrightarrow{x}\right)\}$$

(4)

Definition 4

(Pareto Optimal Front). The Pareto optimal front is defined as:

$$P_f:=\left\{F\left(\overrightarrow{x}\right)\mid \overrightarrow{x}\in P_s\right\}$$

(5)

In such multi-objective optimization problem, a solution is the set of best non-dominated solutions. Therefore, the Pareto optimal solutions projection in the objective space are kept in a set called Pareto optimal front as illustrated in Figure 1. The solutions of both spaces obviously reveal that the green shapes are better than the others, since they dominate all other colors.

Figure 1. Parameter and objective spaces.

The concept of the MRFO standard version is explained briefly in the following section.

2.1. Manta Ray Foraging Optimizer (MRFO)

MRFO is among the recent algorithms proposed in 2020, inspired by giant known critters of the sea called Manta Rays [19]. Figure 2 depicts the shape of a manta ray. To establish this algorithm, the authors mimic three feeding behaviors of Manta Rays: chain, somersault, and cyclone feeding. Furthermore, the manta rays are assumed as search agents which explore the planktons’ location and proceed towards them. Then, the planktons at significant concentration represent the best solution. The source code of MRFO is given in https://www.mathworks.com/matlabcentral/fileexchange/73130-manta-ray-foraging-optimization-mrfo (accessed on 24 May 2021).

Figure 2. Manta ray body form. (a) Manta ray in the ocean; (b) parts of a manta ray, dorsal, and ventral.

Following the population-based optimization algorithms, the steps of MRFO are randomly initialized as illustrated below:

$$x_i=L{b}_i+rand\times (U{b}_i-L{b}_i),i=1,\dots ,N$$

(6)

where $Ub$ and $Lb$ are the maximum and minimum bounds of variables in the search space, rand is a random number between 0 and 1, $rand\in [0,1]$.

The three main operators are mathematically clarified in the next subsections.

2.1.1. Chain Foraging

In this foraging strategy, about 50 Mantas line up head to tail forming an orderly line. The chain swims towards the position of intense concentration of plankton with a fully open mouth. The missing plankton by the leader (manta at the top of the chain) will be devoured by the followers. In the course of the foraging process, the position of each follower is updated towards the best source of plankton and individuals in front of it. This foraging phase is depicted in Figure 3. The mathematical updating formulas are presented as follows:

$$x_{i,j}^{t+1}=\left\{\begin{array}{cc}{x}_{i,j}^t+r_1·\left(x_{best,j}^t-x_{i,j}^t\right)+\alpha ·\left(x_{best,j}^t-x_{i,j}^t\right),\hfill & i=1\hfill \\ x_{i,j}^t+r_2·\left(x_{i-1,j}^t-x_{i,j}^t\right)+\alpha ·\left(x_{best,j}^t-x_{i,j}^t\right),\hfill & i=2,\dots ,N\hfill \end{array}\right.$$

(7)

where $x_{i,j}$ is the position of ith manta ray in the jth dimension, $r_1$ and $r_2$ are the random vector in range [0, 1], $x_{best,j}^t$ is the best plankton concentration position, and $\alpha$ is a weight coefficient that is expressed as:

$$\alpha =2·r_3·\sqrt{|log(r_4\left)\right|}$$

(8)

where $r_3$ and $r_4$ introduce the random vector in range [0, 1].

Figure 3. Simulation model of chain foraging behavior.

2.1.2. Cyclone Foraging

Cyclone foraging phase follows the feeding strategy in WOA [39] in terms of spiral movement. After discovering a significant amount of plankton in the profundity of the ocean, the mantas move one behind another towards plankton making a spiral shape. This foraging phase is illustrated in Figure 4. The manta updates its position based on its best previous position and the manta in front of it.

Figure 4. Simulation model of cyclone foraging behavior.

The spiral-shaped movement is mathematically modeled as:

$$x_{i,j}^{t+1}=\left\{\begin{array}{cc}{x}_{best,j}+r_5·\left(x_{best,j}^t-x_{i,j}^t\right)+\beta ·\left(x_{best,j}^t-x_{i,j}^t\right),\hfill & i=1\hfill \\ x_{best,j}+r_6·\left(x_{i-1,j}^t-x_{i,j}^t\right)+\beta ·\left(x_{best,j}^t-x_{i,j}^t\right),\hfill & i=2,\dots ,N\hfill \end{array}\right.$$

(9)

where $r_5$ and $r_6$ present the random value in [0, 1]; $\beta$ is the weight coefficient that is formulated as:

$$\beta =2e^{r_7\frac{T-t+1}{T}}·sin\left(2\pi r_7\right)$$

(10)

where $r_7$ denotes the random vector in range [0, 1], and T and t are the maximum and current iteration, respectively.

The cyclone foraging can be considered as the main phase in MRFO, in which it performs the intensification (exploitation) and diversification (exploration) mechanisms. The exploitation improvement is achieved based on considering the best plankton found so far as a reference point. On the other hand, the exploration phase incites MRFO to reach the overall optimal solution in accordance with the mathematical equations described below:

$$x_{rand,j}={\mathrm{Lb}}_j+r_8·\left({\mathrm{Ub}}_j-{\mathrm{Lb}}_j\right)$$

(11)

$$x_{i,j}^{t+1}=\left\{\begin{array}{c}{x}_{rand,j}+r_9\left(x_{rand,j}-x_{i,j}^t\right)+\beta \left(x_{rand,j}-x_{i,j}^t\right),i=1\hfill \\ x_{rand,j}+r_{10}\left(x_{i-1,j}^t-x_{ij}^t\right)+\beta \left(x_{rand,j}-x_{i,j}^t\right),i=2,\dots ,N\hfill \end{array}\right.$$

(12)

where $x_{rand,j}$ is the random position generated inside the search space.

2.1.3. Somersault Foraging

The last phase in MRFO is somersault feeding, wherein the manta ray swims to and fro around a pivot point and somersaults around itself to a new position. Figure 5 illustrates this feeding behavior. The manta updates its position using the following mathematical model:

$$x_{i,j}^{t+1}=x_{i,j}^t+S·\left(r_{11}·x_{best,j}-r_{12}·x_{i,j}^t)\right),i=1,\dots ,N$$

(13)

where $r_{11}$, $r_{12}$ depict the random values between 0 and 1. S is the somersault factor, $S=2$.

Figure 5. Simulation model of somersault foraging behavior.

MRFO’s diversification and intensification phases are balanced using the variations value $t/T$, which is gradually increased. The expression $t/T>rand$ denotes the exploration stage, reversibly, and the exploitation process is adopted. The main steps followed in MRFO are demonstrated in Figure 6.

Figure 6. MRFO flowchart for the minimization problem.

block = [rectangle split, draw, rectangle split parts=2, text width=10em, text centered, minimum height=4em] grnblock = [rectangle, draw, fill=green!20, text width=10em, text centered, minimum height=3em] whtblock = [rectangle, draw, fill=white!20, text width=12em, text centered, minimum height=3em] line = [draw, -latex’] cloud = [draw, ellipse,fill=orange!50, node distance=2cm, minimum height=2em] diam=[draw,diamond,fill=orange!60,text width=9em, text centered, inner sep=-2pt] container = [draw, rectangle,dashed,inner sep=0.28cm, rounded corners,fill=yellow!20,minimum height=4cm]

2.2. Proposed (NSMRFO)

As MRFO is relevant for single objective issues, we have developed a multi-objective version of MRFO to handle problems with many fitness functions by applying the Pareto dominance strategy. This variant is inspired from the non-dominated sorting genetic algorithm (NSGA-II) approach, which is the most popular and efficient algorithm in the area of multi-objective optimization in the literature. The non-dominated sorting (NDS) technique employs the crowding distance to define an ordering among individuals and preserve the diversity and the elitist mechanism. To compute all non-dominated solutions, a ranking process is applied called non-dominated ranking (NDR), in which the front that is not dominated by any solutions is assigned to rank 1, while rank 2 is in accordance with the front that is dominated by at least one of the solutions, and so on; the ranking scheme is described in Figure 7a. The crowding distance value of a particular solution is the average distance of its two neighboring solutions as illustrated in Figure 7b. Therefore, the less value of crowding distance denotes comparatively the higher crowded space and conversely. The formulation of this crowding distance mechanism is defined as below:

$$C{D}_i\left(j\right)=\frac{f_i(j+1)-f_i(j-1)}{f_i^{max}-f_i^{min}}$$

(14)

where $f_i^{min}$ and $f_i^{max}$ are the minimum and maximum values of the ith objective function.

Figure 7. Non-dominated ranking fonts (a); crowding-distance calculation (b).

It is worth discussing here that the (NDS) provides a probability to the dominated solutions to be chosen as well, which enhances the diversification of the suggested algorithm. The pseudo-code of NSMRFO is depicted in Algorithm 1. The computational space complexity of NSMRFO is the same as NSGA-II of the order of $O\left(M{N}^2\right)$, where M is the number of objectives, and N is the number of manta rays, while the computational complexity was found to be much better than that of some of the approaches such as SPEA and NSGA, which are of $O\left(M{N}^3\right)$.

Algorithm 1 Non-Dominated Sorting Manta Ray Foraging Optimization

1: Initialize the NSMRFO parameters: design variables, bounds($Ub,Lb$), population, and termination criteria. 2: Generate a uniform random initial population of mantas x with respect to $Ub$ and $Lb$. 3: Compute the fitness function of each manta and sort all of them. 4: Determine the non-dominated solutions in the initial population and save them in Pareto archive. 5: Compute crowding distance for each Pareto archive member. 6: Select a position vector based on crowding distance value. 7: Compute the position vector and update the position of mantas following the MRFO procedure. 8: Compute the fitness values of all the updated positions of mantas. 9: Determine the new non-dominated solutions in the population, save them in a Pareto archive, then remove any dominated solutions in the Pareto archive. 10: Compute the crowding distance value for each Pareto archive member and eliminate as many as necessary according to archive size with the lowest crowding distance value. 11: Perform non-dominated sorting according to the crowding distance mechanism, then select the global best solution using the ranking scheme. 12: Display Pareto optimal set.

2.3. Evaluation Criteria

The employed performance metrics are described in this section. The performance indicators are one of the techniques employed for measuring the potential of a multi-objective algorithm in terms of the diversity and coverage. In this work, various metrics are used such as generational distance (GD) [40], inverted generational distance (IGD) [41] in search [42] and objective [42] spaces, spacing (SP) [43], Pareto sets proximity (rPSP) [44] and reciprocal of hypervolume (rHV) [45], which are formulated as follows:

• Generational Distance (GD) [40]:

$$\mathrm{GD}=\frac{\sqrt{{\sum }_{\mathrm{i}=1}^{{\mathrm{n}}_{\mathrm{pf}}}{\mathrm{d}}_{\mathrm{i}}^2}}{{\mathrm{n}}_{\mathrm{pf}}}$$

(15)

where $n_{pf}$ is the number of obtained Pareto optimal solutions, and $d_i$ indicates the Euclidean distance between the ith Pareto optimal solution attained and the closest true Pareto optimal solution in the reference set:

• Inverted Generational Distance (IGD) [41]:

$$IGD=\frac{\sqrt{{\sum }_{i=1}^{n_{tpf}}{\left(d_i^{\prime }\right)}^2}}{n_{tpf}}$$

(16)

where $n_{tpf}$ is the number of true Pareto optimal solutions and $d_i^{\prime }$ indicates the Euclidean distance between the ith true Pareto optimal solution and the closest Pareto optimal solution obtained in the reference set.

$IGDX$ is the IGD in search space. $IGDF$ is the IGD in objective space:

• Spacing (SP) [43]:

$$\mathrm{SP}=\sqrt{\frac{1}{n_{pf}-1}{\sum }_{i=1}^{n_{pf}}{\left(\overline{d}-d_i\right)}^2}$$

(17)

where $n_{pf}$ is the number of Pareto optimal solutions obtained. $d_i$ indicates the Euclidean distance between the ith Pareto optimal solution attained and the closest true Pareto optimal solution in the reference set. $\overline{d}$ is the average of all $d_i$.

• Reciprocal of Pareto sets proximity (rPSP) [44]:

$$\mathrm{rPSP}=\frac{IGDX}{CR}$$

(18)

$$CR={\left(\prod _{i=1}^m{\delta }_i\right)}^{1/2D}$$

(19)

$${\delta }_i={\left(\frac{min\left(PF{e}_{\mathrm{i}}^{max},PF{t}_i^{max}\right)-max\left(PF{e}_{\mathrm{i}}^{min},PF{t}_i^{min}\right)}{PF{t}_{\mathrm{i}}^{max}-PF{t}_i^{min}}\right)}^2$$

(20)

where $CR$ is the cover rate. m is the number of objective functions. D is the number of decision variables. $PFt$, $PFe$ are the true and obtained Pareto front, respectively:

• Reciprocal of hypervolume (rHV) [45]:

$$rHV(S,w)=\frac{1}{HV(S,w)}$$

(21)

$$HV(S,w)={\lambda }_D\left(\bigcup _{z\in S}[z;w]\right)$$

(22)

where ${\lambda }_D$ is the D-dimensional Lebesgue measure.

3. Experimental Results and Analysis

In this section, the effectiveness of the proposed multi-objective approach is carried out by using 18 different unconstrained benchmark functions, the CEC2020 benchmark test that contains 24 functions, four constrained problems, four engineering design problems, and the IEEE 30-bus optimal power flow issue incorporating wind/solar/small-hydro power. These test suites have different shapes of front like linear, convex, concave, connected, disconnected, etc., as indicated in Table 1. Five analyses are investigated to prove the robustness of the developed NSMRFO algorithm, the first one aims to assess the convergence by using generation distance (GD) metric, the second evaluates the diversity by computing the spacing (SP) metric, the inverse generation distance (IGD) in the search and objective spaces metric, which intends to affirm the NSMRFO’s efficacy in balancing between convergence and diversity, and the reciprocal of Pareto sets proximity (rPSP) and hypervolume (rHV). Moreover, for evaluating the NSMRFO approach, 11 significant multi-objective optimization algorithms are re-implemented, which are named: multi-objective slime mould algorithm (MOSMA) [46], multi-objective bonobo optimizer based decomposition (MOBO/D) [47], multi-objective multi-verse optimization (MOMVO) [48], multi-objective water cycle algorithm (MOWCA) [49], non-dominated sorting grey wolf optimizer (NSGWO) [50], multi-objective manta ray foraging optimizer (MOMRFO) [51], improved multi-objective manta ray foraging optimizer (IMOMRFO) [28], non-sorting genetic algorithm II (NSGA-II) [32], double niched non sorting genetic algorithm II (DN-NSGA) [52], omni optimizer (OMNI) [53], a multi-objective particle swarm optimizer using ring topology (MO_ Ring_ PSO_ SCD) [44], and their characteristics are shown in Table 2. The MATLAB codes for these algorithms were downloaded from: https://aliasgharheidari.com/SMA.html (SMA), https://www.mathworks.com/matlabcentral/fileexchange/79843-multi-objective-bonobo-optimizer-with-decomposition-method (BO), https://seyedalimirjalili.com/mvo (MVO), https://ali-sadollah.com/water-cycle-algorithm-wca/ (WCA), https://www.mathworks.com/matlabcentral/fileexchange/75259-multi-objective-non-sorted-grey-wolf-mogwo-nsgwo?s_tid=srchtitle_nsgwo_1 (GWO), https://www.mathworks.com/matlabcentral/fileexchange/103530-momrfo-multi-objective-manta-ray-foraging-optimizer?s_tid=srchtitle_MOMRFO_1 (MRFO), https://www.mathworks.com/matlabcentral/fileexchange/103895-improved-multi-objective-manta-ray-foraging-optimization?s_tid=srchtitle_MOMRFO_2 (IMRFO), and the codes of the other algorithms and the CEC2020 test suite can be found https://www.mathworks.com/matlabcentral/fileexchange/103895-improved-multi-objective-manta-ray-foraging-optimization?s_tid=srchtitle_MOMRFO_2, respectively (all accessed on 7 August 2021). Each approach is executed on a personal computer, Windows 8.1 (64-bit), core i5 with 4GB-RAM Processor @1:8 GHz using MATLAB R2020a. All benchmark functions are executed 20 times for 1000 iterations and 100 populations, except the CEC2020 problem, which is executed 21 times for a population $pop=200$, and a maximum number of function evaluations equal to 10,000 *$pop$. In addition, the OPF problem was repeated 20 time for $pop=100$ and 200 iterations. Note that the best performing algorithm is assessed based on the mean and standard deviation outcomes. The quantitative and qualitative performance outcomes are illustrated in Table 3, Table 4, Table 5, Table 6, Table 7, Table 8, Table 9, Table 10, Table 11, Table 12, Table 13, Table 14 and Table 15 and Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15 and Figure 16, respectively. The outcomes of each set of benchmark functions are outlined and discussed in the following sections.

Table 1. Descriptions of the unconstrained, constrained, and CEC2020 benchmark functions.

Case	Name	No. Objs	Description
Classical test problems
F1	Schaffer1	2	Convex
F2	Schaffer2	2	Disconnected
F3	Kursawe	2	Degenerate, disconnected
F4	Poloni	2	Disconnected
F5	Fonseca2	2	Concave
F6	Viennet2	3	Connected
F7	Viennet3	3	Connected and asymmetric
ZDT test suite
F8	ZDT1	2	Convex
F9	ZDT2	2	Concave
F10	ZDT3	2	Disconnected
F11	ZDT4	2	Convex, many local optima
F12	ZDT6	2	Concave, nonuniform fitness landscape
YTD test suite
F13	YTD1	2	Disconnected
F14	YTD2	2	Disconnected
F15	YTD3	2	Disconnected
F16	YTD4	2	Disconnected
F17	YTD5	2	Disconnected
F18	YTD6	2	Disconnected
Constrained test suite
F19	OZY	2	Convex, mixed
F20	BNH	2	Convex
F21	SRN	2	Linear, degenerate
F22	CONSTR	2	Convex
CEC-2020 test suite
F23	MMF1	2	Convex
F24	MMF2	2	Convex
F25	MMF4	2	Concave
F26	MMF5	2	Convex
F27	MMF7	2	Convex
F28	MMF8	2	Convex
F29	MMF10	2	Convex
F30	MMF11	2	Convex
F31	MMF12	2	Convex
F32	MMF13	2	Convex
F33	MMF14	3	Concave
F34	MMF15	3	Concave
F35	MMF1_e	2	Convex
F36	MMF14_a	3	Concave
F37	MMF15_a	3	Concave
F38	MMF10_l	2	Convex
F39	MMF11_l	2	Convex
F40	MMF12_l	2	Convex
F41	MMF13_l	2	Convex
F42	MMF15_l	3	Concave
F43	MMF15_a_l	3	Concave
F44	MMF16_l1	3	Concave
F45	MMF16_l2	3	Concave
F46	MMF16_l3	3	Concave

Table 2. Parameter settings of the tested algorithms.

Algorithms	Parameters	Values
NSMRFO	Somersault factor S	2
MOSMA [46]	-	-
MOBO/D [47]	alpha-bonobo (scab)	$1.55$
	selected bonobo (scsb)	$1.4$
	probability (rcpp)	$0.004$
	Initial probability (pxgm-initial)	$0.08$
	subgroup size factor (tsgsfactor-max)	$0.07$
	Number of grids per dimension	7
	Grid inflation rate	$0.1$
	Leader selection pressure	2
	Deletion selection pressure	2
MOMVO [48]	maximum worm hole existence probability	1
	minimum worm hole existence probability	$0.2$
MOWCA [49]	number of rivers + sea (Nsr)	4
NSGWO [50]	-	-
MOMRFO [51]	Somersault factor S	2
	nGrid	30
IMOMRFO [28]	Somersault factor S	2

Table 3. GD, IGD, and SP metrics comparison based on unconstrained test suites.

Case	Name	NSMRFO	MOSMA	MOBO/D	MOMVO	MOWCA	NSGA-II	MOMRFO	IMOMRFO
GD Metric
F1	SCH1	1.5186×10${}^{-3}$	1.5408×10${}^{-3}$	1.1205×10${}^{-3}$	1.6145×10${}^{-3}$	5.0232×10${}^{-3}$	1.4988×10${}^{-3}$	1.9728×10${}^{-3}$	1.0480×10${}^{-3}$
		6.21×10${}^{-5}$	7.73×10${}^{-5}$	1.51×10${}^{-4}$	6.31×10${}^{-4}$	2.14×10${}^{-3}$	6.44×10${}^{-5}$	1.41×10${}^{-4}$	1.12×10${}^{-4}$
F2	SCH2	3.7839×10${}^{-3}$	3.1145×10${}^{-3}$	4.4024×10${}^{-3}$	3.5789×10${}^{-3}$	3.9098×10${}^{-3}$	3.0103×10${}^{+0}$	3.5790×10${}^{-3}$	3.5958×10${}^{-3}$
		1.83×10${}^{-4}$	9.21×10${}^{-4}$	2.77×10${}^{-3}$	1.23×10${}^{-4}$	9.19×10${}^{-5}$	1.0829×10${}^{+0}$	3.62×10${}^{-4}$	3.20×10${}^{-4}$
F3	KUR	1.5001×10${}^{-3}$	4.8631×10${}^{-3}$	8.4540×10${}^{-3}$	3.3798×10${}^{-3}$	6.5932×10${}^{-3}$	6.6901×10${}^{-2}$	3.1843×10${}^{-3}$	3.4414×10${}^{-3}$
		1.52×10${}^{-4}$	6.02×10${}^{-3}$	8.89×10${}^{-3}$	1.99×10${}^{-3}$	4.22×10${}^{-3}$	5.30×10${}^{-2}$	2.91×10${}^{-4}$	3.27×10${}^{-4}$
F4	POL	2.2210×10${}^{-2}$	5.6752×10${}^{-2}$	6.9886×10${}^{-2}$	1.2298×10${}^{-1}$	7.6231×10${}^{-2}$	5.8359×10${}^{-2}$	4.9070×10${}^{-2}$	7.1044×10${}^{-2}$
		1.56×10${}^{-2}$	1.99×10${}^{-2}$	4.43×10${}^{-2}$	5.05×10${}^{-2}$	1.75×10${}^{-2}$	4.06×10${}^{-3}$	1.37×10${}^{-2}$	1.24×10${}^{-2}$
F5	FON2	2.3155×10${}^{-4}$	1.4972×10${}^{-4}$	6.5563×10${}^{-4}$	1.1445×10${}^{-3}$	1.4387×10${}^{-4}$	1.9174×10${}^{-4}$	2.8196×10${}^{-4}$	2.9866×10${}^{-4}$
		2.49×10${}^{-5}$	1.82×10${}^{-5}$	9.49×10${}^{-5}$	5.71×10${}^{-4}$	3.65×10${}^{-5}$	2.61×10${}^{-5}$	4.1343×10${}^{-5}$	3.1249×10${}^{-5}$
F6	VNT2	4.0305×10${}^{-3}$	2.1878×10${}^{-2}$	1.8094×10${}^{-2}$	1.6617×10${}^{-2}$	1.9053×10${}^{-2}$	1.8746×10${}^{-2}$	1.1303×10${}^{-2}$	1.2150×10${}^{-2}$
		9.00×10${}^{-4}$	6.02×10${}^{-3}$	1.02×10${}^{-3}$	6.12×10${}^{-3}$	7.44×10${}^{-4}$	1.20×10${}^{-3}$	1.34×10${}^{-3}$	1.86×10${}^{-3}$
F7	VNT3	2.3873×10${}^{-2}$	2.3977×10${}^{-2}$	2.8189×10${}^{-2}$	2.3966×10${}^{-2}$	2.0508×10${}^{-2}$	2.3930×10${}^{-2}$	1.6262×10${}^{-2}$	2.2563×10${}^{-2}$
		1.27×10${}^{-3}$	1.12×10${}^{-3}$	3.31×10${}^{-2}$	1.06×10${}^{-3}$	6.24×10${}^{-3}$	1.49×10${}^{-3}$	1.77×10${}^{-3}$	4.08×10${}^{-3}$
F8	ZDT1	2.3319×10${}^{-4}$	3.6551×10${}^{-4}$	5.0011×10${}^{-4}$	3.8157×10${}^{-3}$	2.6413×10${}^{-4}$	3.6790×10${}^{-2}$	1.9395×10${}^{-4}$	3.7296×10${}^{-2}$
		3.23×10${}^{-5}$	4.74×10${}^{-5}$	2.51×10${}^{-5}$	3.82×10${}^{-4}$	4.24×10${}^{-5}$	2.69×10${}^{-2}$	5.12×10${}^{-5}$	1.80×10${}^{-2}$
F9	ZDT2	9.3101×10${}^{-5}$	3.0210×10${}^{-4}$	2.3881×10${}^{-4}$	5.7435×10${}^{-3}$	5.8786×10${}^{-3}$	8.8345×10${}^{-2}$	1.1687×10${}^{-4}$	4.2670×10${}^{-2}$
		4.36×10${}^{-6}$	5.52×10${}^{-5}$	8.69×10${}^{-5}$	3.93×10${}^{-4}$	9.36×10${}^{-3}$	7.92×10${}^{-2}$	3.11×10${}^{-5}$	3.74×10${}^{-2}$
F10	ZDT3	1.6294×10${}^{-4}$	2.1877×10${}^{-4}$	8.7873×10${}^{-4}$	1.9516×10${}^{-2}$	4.6262×10${}^{-4}$	2.0248×10${}^{-2}$	1.5360×10${}^{-4}$	3.6556×10${}^{-2}$
		1.07×10${}^{-5}$	2.36×10${}^{-5}$	5.25×10${}^{-4}$	4.98×10${}^{-2}$	6.01×10${}^{-4}$	1.41×10${}^{-2}$	2.23×10${}^{-5}$	3.63×10${}^{-2}$
F11	ZDT4	2.2409×10${}^{-4}$	2.7059×10${}^{-4}$	9.3797×10${}^{-2}$	1.3951×10${}^{+0}$	1.2212×10${}^{+0}$	2.9861×10${}^{+0}$	3.8600×10${}^{-4}$	3.4435×10${}^{-4}$
		4.91×10${}^{-5}$	5.52×10${}^{-5}$	4.58×10${}^{-2}$	7.42×10${}^{-1}$	1.26×10${}^{+0}$	1.04×10${}^{+0}$	5.23×10${}^{-5}$	2.54×10${}^{-4}$
F12	ZDT6	4.4226×10${}^{-2}$	1.9098×10${}^{-2}$	1.0946×10${}^{-4}$	1.2260×10${}^{-2}$	2.1251×10${}^{-1}$	1.8761×10${}^{-1}$	7.2768×10${}^{-5}$	6.5288×10${}^{-1}$
		4.93×10${}^{-2}$	2.54×10${}^{-2}$	1.01×10${}^{-4}$	5.36×10${}^{-3}$	1.09×10${}^{-1}$	1.34×10${}^{-1}$	6.46×10${}^{-6}$	1.65×10${}^{-1}$
F13	TYD1	1.0438×10${}^{-3}$	6.7872×10${}^{-2}$	4.4802×10${}^{-2}$	5.7748×10${}^{-3}$	3.1187×10${}^{-2}$	1.3143×10${}^{-2}$	2.4044×10${}^{-2}$	-
		4.20×10${}^{-4}$	7.30×10${}^{-3}$	2.77×10${}^{-2}$	7.05×10${}^{-4}$	7.32×10${}^{-3}$	6.49×10${}^{-3}$	7.55×10${}^{-3}$
F14	TYD2	7.2424×10${}^{-4}$	1.2213×10${}^{-1}$	4.8234×10${}^{-3}$	7.7742×10${}^{-3}$	2.3999×10${}^{-1}$	2.3745×10${}^{+0}$	6.5481×10${}^{-4}$	-
		1.55×10${}^{-5}$	1.11×10${}^{-1}$	5.34×10${}^{-3}$	1.47×10${}^{-3}$	4.11×10${}^{-1}$	1.71×10${}^{+0}$	5.31×10${}^{-5}$
F15	TYD3	1.3413×10${}^{-3}$	2.6726×10${}^{-2}$	2.8383×10${}^{-3}$	1.0019×10${}^{-2}$	5.7059×10${}^{-3}$	5.4824×10${}^{+0}$	1.2825×10${}^{-3}$	-
		6.72×10${}^{-5}$	2.34×10${}^{-2}$	1.39×10${}^{-3}$	1.77×10${}^{-3}$	5.16×10${}^{-3}$	2.96×10${}^{+0}$	1.43×10${}^{-4}$
F16	TYD4	6.5768×10${}^{-4}$	1.5877×10${}^{-3}$	1.4063×10${}^{-3}$	8.0617×10${}^{-3}$	1.4953×10${}^{-1}$	4.7620×10${}^{+0}$	1.1171×10${}^{-3}$	-
		1.14×10${}^{-4}$	1.14×10${}^{-4}$	9.23×10${}^{-5}$	3.18×10${}^{-3}$	3.75×10${}^{-1}$	2.91×10${}^{+0}$	5.87×10${}^{-5}$
F17	TYD5	2.8699×10${}^{-4}$	5.3955×10${}^{-4}$	6.0136×10${}^{-4}$	1.5922×10${}^{-2}$	1.4687×10${}^{-3}$	14.962×10${}^{+0}$	4.6125×10${}^{-4}$	-
		9.07×10${}^{-4}$	1.33×10${}^{-5}$	2.56×10${}^{-4}$	5.98×10${}^{-2}$	1.74×10${}^{-4}$	1.92×10${}^{+0}$	2.72×10${}^{-5}$
F18	TYD6	3.0338×10${}^{-3}$	7.3752×10${}^{-3}$	3.1964×10${}^{-3}$	5.7748×10${}^{-3}$	1.1735×10${}^{-2}$	12.040×10${}^{+0}$	2.7641×10${}^{-3}$	-
		8.18×10${}^{-4}$	3.80×10${}^{-3}$	8.86×10${}^{-5}$	7.05×10${}^{-4}$	1.34×10${}^{-2}$	2.14×10${}^{+0}$	1.81×10${}^{-4}$
IGD Metric
F1	SCH1	4.9741×10${}^{-4}$	7.0139×10${}^{-4}$	1.4727×10${}^{-3}$	5.0232×10${}^{-3}$	5.0230×10${}^{-4}$	5.4182×10${}^{-4}$	7.3050×10${}^{-4}$	7.8791×10${}^{-4}$
		5.99×10${}^{-5}$	8.08×10${}^{-5}$	5.88×10${}^{-4}$	2.14×10${}^{-3}$	3.73×10${}^{-5}$	5.72×10${}^{-5}$	8.13×10${}^{-5}$	1.06×10${}^{-4}$
F2	SCH2	4.6630×10${}^{-4}$	4.0565×10${}^{-4}$	2.4468×10${}^{-2}$	3.7958×10${}^{-3}$	4.5943×10${}^{-4}$	4.3624×10${}^{-2}$	7.9057×10${}^{-4}$	8.7927×10${}^{-4}$
		2.85×10${}^{-5}$	1.69×10${}^{-5}$	1.94×10${}^{-2}$	5.96×10${}^{-4}$	1.38×10${}^{-5}$	1.30×10${}^{-4}$	4.52×10${}^{-5}$	1.25×10${}^{-4}$
F3	KUR	1.6326×10${}^{-4}$	4.5883×10${}^{-4}$	1.1761×10${}^{-3}$	8.0353×10${}^{-4}$	3.3578×10${}^{-4}$	3.4430×10${}^{-3}$	2.5796×10${}^{-4}$	2.5193×10${}^{-4}$
		1.54×10${}^{-5}$	3.66×10${}^{-4}$	8.27×10${}^{-4}$	1.36×10${}^{-4}$	2.12×10${}^{-4}$	2.36×10${}^{-3}$	2.27×10${}^{-5}$	1.84×10${}^{-5}$
F4	POL	4.8896×10${}^{-4}$	4.9407×10${}^{-4}$	4.0539×10${}^{-2}$	5.9292×10${}^{-3}$	4.8057×10${}^{-4}$	5.5668×10${}^{-4}$	9.1231×10${}^{-4}$	8.0518×10${}^{-4}$
		3.52×10${}^{-5}$	1.43×10${}^{-4}$	6.50×10${}^{-3}$	1.61×10${}^{-3}$	3.97×10${}^{-5}$	3.42×10${}^{-5}$	2.84×10${}^{-4}$	1.27×10${}^{-4}$
F5	FON2	2.7389×10${}^{-4}$	5.0585×10${}^{-4}$	6.4536×10${}^{-4}$	1.5838×10${}^{-3}$	2.9388×10${}^{-4}$	3.0779×10${}^{-4}$	4.0938×10${}^{-4}$	4.5341×10${}^{-4}$
		9.53×10${}^{-6}$	1.45×10${}^{-4}$	1.32×10${}^{-4}$	3.27×10${}^{-4}$	3.99×10${}^{-5}$	2.15×10${}^{-5}$	5.24×10${}^{-5}$	6.94×10${}^{-5}$
F6	VNT2	3.8653×10${}^{-3}$	4.7815×10${}^{-3}$	1.2333×10${}^{-2}$	7.7099×10${}^{-3}$	4.4016×10${}^{-3}$	4.9972×10${}^{-3}$	2.7897×10${}^{-3}$	2.7698×10${}^{-3}$
		4.80×10${}^{-4}$	1.29×10${}^{-3}$	2.25×10${}^{-3}$	1.19×10${}^{-3}$	9.41×10${}^{-4}$	8.71×10${}^{-4}$	3.17×10${}^{-4}$	2.05×10${}^{-4}$
F7	VNT3	1.4552×10${}^{-3}$	1.9514×10${}^{-3}$	2.1503×10${}^{-2}$	5.2610×10${}^{-3}$	1.4331×10${}^{-3}$	1.3868×10${}^{-3}$	1.4799×10${}^{-3}$	1.6787×10${}^{-3}$
		6.37×10${}^{-4}$	2.02×10${}^{-4}$	2.33×10${}^{-2}$	6.11×10${}^{-4}$	2.42×10${}^{-4}$	1.49×10${}^{-4}$	1.79×10${}^{-4}$	1.48×10${}^{-4}$
F8	ZDT1	2.6206×10${}^{-4}$	4.7382×10${}^{-4}$	6.3507×10${}^{-4}$	2.0620×10${}^{-3}$	2.6266×10${}^{-4}$	1.5650×10${}^{-2}$	3.9012×10${}^{-4}$	1.7831×10${}^{-3}$
		1.75×10${}^{-5}$	7.07×10${}^{-5}$	9.90×10${}^{-5}$	1.54×10${}^{-4}$	1.35×10${}^{-5}$	1.08×10${}^{-2}$	3.63×10${}^{-5}$	3.78×10${}^{-4}$
F9	ZDT2	2.7851×10${}^{-4}$	4.5857×10${}^{-4}$	5.2148×10${}^{-4}$	2.8492×10${}^{-3}$	8.2051×10${}^{-3}$	4.4870×10${}^{-2}$	3.9149×10${}^{-4}$	2.7908×10${}^{-3}$
		2.10×10${}^{-5}$	2.53×10${}^{-5}$	1.18×10${}^{-4}$	1.97×10${}^{-4}$	1.13×10${}^{-2}$	4.22×10${}^{-2}$	2.89×10${}^{-5}$	7.71×10${}^{-4}$
F10	ZDT3	1.8616×10${}^{-4}$	2.8916×10${}^{-4}$	5.6483×10${}^{-4}$	2.1679×10${}^{-3}$	6.8846×10${}^{-4}$	7.7232×10${}^{-3}$	2.9073×10${}^{-4}$	1.5993×10${}^{-3}$
		7.89×10${}^{-6}$	3.92×10${}^{-5}$	1.98×10${}^{-4}$	4.23×10${}^{-4}$	1.24×10${}^{-3}$	4.26×10${}^{-3}$	2.21×10${}^{-5}$	4.07×10${}^{-4}$
F11	ZDT4	2.6414×10${}^{-5}$	5.0698×10${}^{-4}$	1.9416×10${}^{-2}$	3.9102×10${}^{-1}$	1.6733×10${}^{-2}$	1.2540×10${}^{+0}$	3.7765×10${}^{-4}$	4.1458×10${}^{-4}$
		2.55×10${}^{-5}$	8.23×10${}^{-5}$	7.71×10${}^{-3}$	2.14×10${}^{-1}$	2.36×10${}^{-2}$	4.47×10${}^{-1}$	3.03×10${}^{-5}$	3.31×10${}^{-5}$
F12	ZDT6	2.9553×10${}^{-4}$	4.6403×10${}^{-4}$	1.6486×10${}^{-3}$	6.8445×10${}^{-4}$	1.5553×10${}^{-2}$	9.3672×10${}^{-2}$	4.1577×10${}^{-4}$	5.6469×10${}^{-2}$
		4.62×10${}^{-5}$	4.88×10${}^{-5}$	1.02×10${}^{-3}$	2.30×10${}^{-4}$	1.60×10${}^{-2}$	7.90×10${}^{-2}$	3.62×10${}^{-5}$	1.14×10${}^{-2}$
F13	TYD1	9.4440×10${}^{-4}$	3.2475×10${}^{-2}$	1.5235×10${}^{-2}$	2.9073×10${}^{-3}$	2.0043×10${}^{-2}$	1.1656×10${}^{-2}$	7.3191×10${}^{-3}$	-
		4.92×10${}^{-4}$	2.83×10${}^{-3}$	7.62×10${}^{-3}$	6.13×10${}^{-3}$	6.34×10${}^{-3}$	7.15×10${}^{-3}$	1.43×10${}^{-3}$
F14	TYD2	2.9823×10${}^{-4}$	6.5423×10${}^{-3}$	1.5225×10${}^{-3}$	2.2408×10${}^{-3}$	2.1579×10${}^{-2}$	4.9072×10${}^{-1}$	6.3908×10${}^{-4}$	-
		1.12×10${}^{-5}$	4.96×10${}^{-3}$	9.52×10${}^{-4}$	1.06×10${}^{-3}$	3.01×10${}^{-2}$	3.50×10${}^{-1}$	8.77×10${}^{-5}$
F15	TYD3	3.2263×10${}^{-4}$	7.7030×10${}^{-3}$	5.0933×10${}^{-3}$	1.6855×10${}^{-2}$	4.8173×10${}^{-3}$	8.8421×10${}^{-1}$	7.9035×10${}^{-4}$	-
		9.51×10${}^{-6}$	5.96×10${}^{-3}$	3.99×10${}^{-3}$	5.85×10${}^{-3}$	4.09×10${}^{-3}$	4.68×10${}^{-1}$	8.47×10${}^{-5}$
F16	TYD4	3.4868×10${}^{-4}$	5.1097×10${}^{-4}$	3.2404×10${}^{-3}$	7.6373×10${}^{-3}$	1.7494×10${}^{-2}$	1.1066×10${}^{+0}$	6.9184×10${}^{-4}$	-
		2.58×10${}^{-5}$	3.81×10${}^{-5}$	1.74×10${}^{-3}$	8.39×10${}^{-3}$	2.48×10${}^{-2}$	6.08×10${}^{-1}$	6.53×10${}^{-5}$
F17	TYD5	4.7951×10${}^{-4}$	1.2769×10${}^{-1}$	2.0048×10${}^{-3}$	8.0911×10${}^{-3}$	1.3432×10${}^{-1}$	3.1897×10${}^{+1}$	1.0127×10${}^{-3}$	-
		6.02×10${}^{-5}$	2.09×10${}^{-2}$	3.72×10${}^{-2}$	4.25×10${}^{-3}$	2.92×10${}^{-7}$	4.19×10${}^{+0}$	1.17×10${}^{-4}$
F18	TYD6	4.3449×10${}^{-4}$	4.9312×10${}^{-2}$	1.0853×10${}^{-3}$	5.3901×10${}^{-3}$	7.1215×10${}^{-2}$	4.9147×10${}^{+0}$	7.5797×10${}^{-4}$	-
		4.29×10${}^{-5}$	2.55×10${}^{-2}$	4.40×10${}^{-4}$	9.49×10${}^{-3}$	5.31×10${}^{-3}$	8.65×10${}^{-1}$	9.46×10${}^{-5}$
SP Metric
F1	SCH1	2.9654×10${}^{-2}$	3.9539×10${}^{-2}$	6.8903×10${}^{-2}$	2.0338×10${}^{-2}$	2.9242×10${}^{-2}$	3.1513×10${}^{-2}$	4.0819×10${}^{-2}$	3.8608×10${}^{-2}$
		1.99×10${}^{-3}$	4.30×10${}^{-3}$	2.47×10${}^{-2}$	2.25×10${}^{-2}$	1.81×10${}^{-3}$	3.04×10${}^{-3}$	4.96×10${}^{-3}$	4.34×10${}^{-3}$
F2	SCH2	4.3778×10${}^{-2}$	4.2673×10${}^{-2}$	6.2204×10${}^{-1}$	5.2262×10${}^{-2}$	4.6848×10${}^{-2}$	3.3859×10${}^{-1}$	5.4329×10${}^{-2}$	6.3003×10${}^{-2}$
		2.96×10${}^{-3}$	2.42×10${}^{-3}$	6.39×10${}^{-1}$	2.96×10${}^{-2}$	2.40×10${}^{-3}$	1.01×10${}^{-1}$	9.08×10${}^{-3}$	8.34×10${}^{-3}$
F3	KUR	6.4721×10${}^{-2}$	9.2719×10${}^{-2}$	2.6929×10${}^{-1}$	1.2020×10${}^{-1}$	8.4376×10${}^{-2}$	9.4290×10${}^{-2}$	1.1223×10${}^{-1}$	9.9612×10${}^{-2}$
		2.32×10${}^{-2}$	1.45×10${}^{-2}$	1.76×10${}^{-1}$	4.12×10${}^{-2}$	2.01×10${}^{-2}$	3.80×10${}^{-2}$	1.76×10${}^{-2}$	1.93×10${}^{-2}$
F4	POL	8.1830×10${}^{-2}$	1.6050×10${}^{-1}$	2.1860×10${}^{-1}$	1.2687×10${}^{-1}$	8.1395×10${}^{-2}$	9.0758×10${}^{-2}$	2.6669×10${}^{-1}$	2.1438×10${}^{-1}$
		8.11×10${}^{-3}$	1.03×10${}^{-1}$	1.77×10${}^{-1}$	1.39×10${}^{-1}$	5.88×10${}^{-3}$	9.35×10${}^{-3}$	9.68×10${}^{-2}$	1.14×10${}^{-1}$
F5	FON2	5.9240×10${}^{-3}$	1.2015×10${}^{-2}$	1.1867×10${}^{-2}$	1.4375×10${}^{-2}$	6.5275×10${}^{-3}$	7.0641×10${}^{-3}$	8.6261×10${}^{-3}$	9.2525×10${}^{-3}$
		6.03×10${}^{-4}$	3.23×10${}^{-3}$	1.87×10${}^{-3}$	5.34×10${}^{-3}$	7.71×10${}^{-4}$	8.87×10${}^{-4}$	7.22×10${}^{-4}$	1.82×10${}^{-3}$
F6	VNT2	2.0346×10${}^{-2}$	5.7179×10${}^{-2}$	1.9270×10${}^{-2}$	2.3098×10${}^{-2}$	1.7122×10${}^{-2}$	1.4910×10${}^{-2}$	2.7402×10${}^{-2}$	2.0876×10${}^{-2}$
		8.29×10${}^{-3}$	2.91×10${}^{-2}$	4.02×10${}^{-3}$	9.53×10${}^{-3}$	2.40×10${}^{-2}$	3.02×10${}^{-3}$	8.07×10${}^{-3}$	2.95×10${}^{-3}$
F7	VNT3	6.5468×10${}^{-2}$	6.3280×10${}^{-2}$	3.5118×10${}^{-1}$	1.0089×10${}^{-1}$	5.7043×10${}^{-2}$	6.6166×10${}^{-2}$	9.3950×10${}^{-2}$	8.0997×10${}^{-2}$
		4.97×10${}^{-3}$	4.01×10${}^{-3}$	1.87×10${}^{-1}$	5.36×10${}^{-2}$	6.75×10${}^{-3}$	7.75×10${}^{-3}$	1.19×10${}^{-2}$	2.46×10${}^{-2}$
F8	ZDT1	6.8254×10${}^{-3}$	1.0333×10${}^{-2}$	1.7046×10${}^{-2}$	1.1959×10${}^{-2}$	8.3394×10${}^{-3}$	8.0300×10${}^{-3}$	9.0844×10${}^{-3}$	9.6005×10${}^{-2}$
		5.83×10${}^{-4}$	1.41×10${}^{-3}$	6.28×10${}^{-3}$	1.90×10${}^{-3}$	5.23×10${}^{-4}$	1.57×10${}^{-3}$	1.19×10${}^{-3}$	6.65×10${}^{-2}$
F9	ZDT2	6.8695×10${}^{-3}$	9.9417×10${}^{-3}$	1.1725×10${}^{-2}$	1.6606×10${}^{-2}$	1.1321×10${}^{-2}$	4.1262×10${}^{-3}$	9.8885×10${}^{-3}$	9.0712×10${}^{-2}$
		7.19×10${}^{-4}$	9.80×10${}^{-4}$	1.40×10${}^{-3}$	8.41×10${}^{-3}$	1.06×10${}^{-2}$	2.91×10${}^{-3}$	1.13×10${}^{-3}$	3.79×10${}^{-2}$
F10	ZDT3	8.1064×10${}^{-3}$	1.2409×10${}^{-2}$	1.3643×10${}^{-2}$	1.7348×10${}^{-2}$	1.0571×10${}^{-2}$	7.4131×10${}^{-3}$	1.2411×10${}^{-2}$	8.9649×10${}^{-2}$
		6.43×10${}^{-4}$	1.83×10${}^{-3}$	3.84×10${}^{-3}$	5.88×10${}^{-3}$	4.74×10${}^{-3}$	1.06×10${}^{-3}$	2.09×10${}^{-3}$	9.45×10${}^{-2}$
F11	ZDT4	7.0857×10${}^{-3}$	9.2094×10${}^{-3}$	4.5038×10${}^{-2}$	7.7476×10${}^{-3}$	1.7807×10${}^{-1}$	2.7520×10${}^{-2}$	9.0207×10${}^{-3}$	9.4750×10${}^{-3}$
		5.46×10${}^{-4}$	8.20×10${}^{-4}$	1.23×10${}^{-2}$	5.56×10${}^{-3}$	1.50×10${}^{-1}$	4.95×10${}^{-3}$	1.07×10${}^{-3}$	1.52×10${}^{-3}$
F12	ZDT6	1.5261×10${}^{-2}$	2.6212×10${}^{-2}$	1.6208×10${}^{-2}$	8.1444×10${}^{-2}$	7.5266×10${}^{-2}$	9.6715×10${}^{-2}$	8.5474×10${}^{-3}$	2.5094×10${}^{-1}$
		1.96×10${}^{-2}$	1.96×10${}^{-2}$	6.73×10${}^{-3}$	5.14×10${}^{-2}$	3.31×10${}^{-2}$	1.17×10${}^{-1}$	1.41×10${}^{-3}$	1.66×10${}^{-1}$
F13	TYD1	2.8746×10${}^{-3}$	6.2895×10${}^{-2}$	5.2384×10${}^{-2}$	4.3920×10${}^{-3}$	1.6767×10${}^{-2}$	4.5573×10${}^{-3}$	1.4521×10${}^{-2}$	-
		1.64×10${}^{-3}$	1.29×10${}^{-2}$	3.20×10${}^{-2}$	2.70×10${}^{-3}$	1.41×10${}^{-2}$	1.32×10${}^{-3}$	7.21×10${}^{-3}$
F14	TYD2	1.4202×10${}^{-2}$	1.9431×10${}^{-1}$	6.3481×10${}^{-2}$	3.6429×10${}^{-2}$	1.0145×10${}^{-1}$	8.5839×10${}^{-3}$	2.2990×10${}^{-2}$	-
		9.67×10${}^{-4}$	1.35×10${}^{-1}$	5.01×10${}^{-2}$	2.27×10${}^{-2}$	9.47×10${}^{-2}$	1.26×10${}^{-2}$	3.27×10${}^{-3}$
F15	TYD3	1.8745×10${}^{-2}$	1.1549×10${}^{-1}$	2.9293×10${}^{-1}$	1.0838×10${}^{-1}$	7.0163×10${}^{-2}$	3.6667×10${}^{-2}$	3.1919×10${}^{-2}$	-
		1.31×10${}^{-3}$	5.56×10${}^{-2}$	2.67×10${}^{-1}$	1.25×10${}^{-1}$	4.47×10${}^{-2}$	9.76×10${}^{-3}$	5.67×10${}^{-3}$
F16	TYD4	2.4070×10${}^{-2}$	2.5290×10${}^{-2}$	7.4783×10${}^{-2}$	4.1287×10${}^{-2}$	1.5601×10${}^{-1}$	2.6227×10${}^{-2}$	3.0307×10${}^{-2}$	-
		6.41×10${}^{-3}$	1.79×10${}^{-3}$	2.69×10${}^{-2}$	4.11×10${}^{-2}$	2.32×10${}^{-1}$	2.40×10${}^{-2}$	4.02×10${}^{-3}$
F17	TYD5	4.4231×10${}^{-3}$	1.8844×10${}^{-2}$	5.1214×10${}^{-2}$	3.8197×10${}^{-2}$	-	-	6.0025×10${}^{-3}$	-
		4.11×10${}^{-4}$	5.95×10${}^{-2}$	8.54×10${}^{-2}$	4.56×10${}^{-3}$	-	-	8.99×10${}^{-4}$
F18	TYD6	1.2875×10${}^{-2}$	8.2821×10${}^{-2}$	2.5104×10${}^{-2}$	4.2786×10${}^{-2}$	3.2520×10${}^{-2}$	-	1.8811×10${}^{-2}$	-
		1.15×10${}^{-3}$	4.12×10${}^{-2}$	1.02×10${}^{-2}$	1.07×10${}^{-2}$	2.80×10${}^{-2}$	-	2.33×10${}^{-3}$

Table 4. GD, IGD, and SP metrics comparison based on constrained test suites.

Problem	NSMRFO	MOSMA [46]	MOBO/D [47]	MOMVO [48]	MOWCA [49]	NSGA-II [32]	MOMRFO [51]	IMOMRFO [28]
GD Metric
OZY	4.0885×10${}^{-1}$	1.8977×10${}^{+0}$	9.6335×10${}^{-1}$	4.1926×10${}^{-1}$	1.2504×10${}^{+0}$	8.5422×10${}^{-1}$	4.8528×10${}^{-1}$	4.1772×10${}^{-1}$
	2.29×10${}^{-1}$	6.90×10${}^{-1}$	5.91×10${}^{-1}$	4.29×10${}^{-1}$	7.76×10${}^{-1}$	5.94×10${}^{-1}$	1.74×10${}^{-1}$	2.04×10${}^{-1}$
BNH	3.4178×10${}^{-1}$	3.6565×10${}^{-1}$	2.0107×10${}^{-1}$	2.8575×10${}^{-1}$	3.3953×10${}^{-1}$	2.0462×10${}^{+0}$	2.2904×10${}^{+0}$	1.1858×10${}^{+0}$
	1.75×10${}^{-2}$	1.52×10${}^{-2}$	8.17×10${}^{-2}$	2.16×10${}^{-1}$	1.46×10${}^{-2}$	8.52×10${}^{-2}$	1.78×10${}^{-1}$	1.56×10${}^{-1}$
SRN	3.7131×10${}^{-2}$	1.3030×10${}^{-1}$	8.0926×10${}^{-2}$	3.6668×10${}^{-1}$	6.5713×10${}^{-2}$	4.3051×10${}^{-2}$	6.2583×10${}^{-2}$	7.6938×10${}^{-2}$
	6.99×10${}^{-3}$	6.53×10${}^{-2}$	4.67×10${}^{-2}$	8.97×10${}^{-2}$	6.97×10${}^{-3}$	6.91×10${}^{-3}$	8.83×10${}^{-3}$	1.18×10${}^{-2}$
CONSTR	6.0167×10${}^{-4}$	1.0382×10${}^{-3}$	6.9108×10${}^{-4}$	8.1620×10${}^{-4}$	7.2413×10${}^{-4}$	7.0693×10${}^{-4}$	6.0098×10${}^{-4}$	8.0662×10${}^{-4}$
	1.50×10${}^{-4}$	5.55×10${}^{-4}$	4.14×10${}^{-5}$	4.02×10${}^{-4}$	7.28×10${}^{-5}$	4.62×10${}^{-5}$	4.88×10${}^{-5}$	6.47×10${}^{-5}$
IGD Metric
OZY	1.4014×10${}^{-3}$	9.7966×10${}^{-3}$	9.965×10${}^{-3}$	2.2981×10${}^{-3}$	6.4842×10${}^{-3}$	4.3475×10${}^{-3}$	4.2650×10${}^{-3}$	1.5178×10${}^{-3}$
	1.18×10${}^{-3}$	2.07×10${}^{-3}$	6.43×10${}^{-3}$	2.04×10${}^{-3}$	2.21×10${}^{-3}$	5.31×10${}^{-3}$	2.80×10${}^{-3}$	2.07×10${}^{-3}$
BNH	6.7915×10${}^{-4}$	1.1293×10${}^{-3}$	1.9024×10${}^{-3}$	4.4141×10${}^{-3}$	6.6386×10${}^{-4}$	1.8246×10${}^{-4}$	2.4874×10${}^{-4}$	2.0763×10${}^{-4}$
	3.39×10${}^{-5}$	9.31×10${}^{-5}$	1.22×10${}^{-3}$	7.06×10${}^{-4}$	4.39×10${}^{-5}$	1.78×10${}^{-5}$	1.16×10${}^{-5}$	2.20×10${}^{-5}$
SRN	9.7172×10${}^{-5}$	2.7928×10${}^{-4}$	1.9607×10${}^{-4}$	5.9502×10${}^{-4}$	8.9740×10${}^{-5}$	1.0583×10${}^{-4}$	1.2501×10${}^{-4}$	1.4485×10${}^{-4}$
	6.88×10${}^{-6}$	2.28×10${}^{-4}$	6.97×10${}^{-5}$	8.07×10${}^{-5}$	1.49×10${}^{-6}$	1.46×10${}^{-5}$	7.45×10${}^{-6}$	1.17×10${}^{-5}$
CONSTR	2.5255×10${}^{-4}$	1.0344×10${}^{-3}$	1.6097×10${}^{-3}$	1.7429×10${}^{-3}$	2.6802×10${}^{-4}$	2.7118×10${}^{-4}$	4.7669×10${}^{-4}$	3.5283×10${}^{-4}$
	4.44×10${}^{-5}$	2.92×10${}^{-4}$	8.93×10${}^{-4}$	4.30×10${}^{-4}$	4.31×10${}^{-5}$	1.51×10${}^{-5}$	6.21×10${}^{-5}$	4.98×10${}^{-5}$
SP Metric
OZY	1.9039×10${}^{+0}$	8.0601×10${}^{+0}$	2.3336×10${}^{+0}$	2.0649×10${}^{+0}$	2.0314×10${}^{+0}$	1.3227×10${}^{+0}$	1.6335×10${}^{+0}$	1.3983×10${}^{+0}$
	8.93×10${}^{-1}$	2.85×10${}^{+0}$	2.03×10${}^{+0}$	4.42×10${}^{-1}$	1.41×10${}^{+0}$	2.48×10${}^{-1}$	2.66×10${}^{-1}$	1.23×10${}^{-1}$
BNH	7.1873×10${}^{-1}$	8.7699×10${}^{-1}$	1.8574×10${}^{+0}$	1.2154×10${}^{+0}$	7.595×10${}^{-1}$	7.5542×10${}^{-1}$	9.4169×10${}^{-1}$	1.4736×10${}^{+0}$
	6.39×10${}^{-2}$	8.93×10${}^{-2}$	8.78×10${}^{-1}$	5.76×10${}^{-1}$	6.94×10${}^{-2}$	7.37×10${}^{-2}$	1.32×10${}^{-1}$	4.43×10${}^{-1}$
SRN	1.3558×10${}^{+0}$	3.6172×10${}^{+0}$	2.3050×10${}^{+0}$	2.9495×10${}^{+0}$	1.5216×10${}^{+0}$	1.4977×10${}^{+0}$	2.1824×10${}^{+0}$	2.2106×10${}^{+0}$
	1.16×10${}^{-1}$	3.10×10${}^{+0}$	5.83×10${}^{-1}$	4.47×10${}^{+0}$	9.96×10${}^{-2}$	1.67×10${}^{-1}$	3.49×10${}^{-1}$	4.86×10${}^{-1}$
CONSTR	1.2579×10${}^{-2}$	4.2564×10${}^{-2}$	1.7483×10${}^{-1}$	3.7418×10${}^{-2}$	4.3448×10${}^{-2}$	4.3658×10${}^{-2}$	8.4588×10${}^{-2}$	5.1093×10${}^{-2}$
	2.87×10${}^{-2}$	3.74×10${}^{-3}$	5.34×10${}^{-2}$	2.26×10${}^{-2}$	4.46×10${}^{-3}$	2.91×10${}^{-3}$	1.05×10${}^{-2}$	2.97×10${}^{-3}$

Table 5. rPSP, rHV, IGDX, and IGDF indicator metrics comparison based on CEC2020 test suites.

Case	Name	NSMRFO	NSGA-II [32]	DN-NSGAII [52]	OMNI-OPT [53]	MO_Ring_PSO_SCD [44]	MOMRFO [51]	IMOMRFO [28]
MMF1	rPSP	3.0414×10${}^{-2}$(2.0151×10${}^{-3}$)	7.6055×10${}^{-2}$(1.2369×10${}^{-2}$)	5.5900×10${}^{-2}$(1.1025×10${}^{-2}$)	5.1803×10${}^{-2}$(9.6985×10${}^{-3}$)	2.9630×10${}^{-2}$(1.0108×10${}^{-3}$)	8.4765×10${}^{-2}$(4.7454×10${}^{-3}$)	2.5834×10${}^{-2}$(1.2271×10${}^{-3}$)
	rHV	1.1437×10${}^{+0}$(2.0798×10${}^{-4}$)	1.1438×10${}^{+0}$(5.3194×10${}^{-4}$)	1.1447×10${}^{+0}$(6.7793×10${}^{-4}$)	1.1443×10${}^{+0}$(6.4982×10${}^{-4}$)	1.1449×10${}^{+0}$(2.0606×10${}^{-4}$)	1.1562×10${}^{+0}$(2.7386×10${}^{-3}$)	1.1448×10${}^{+0}$(3.0962×10${}^{-4}$)
	IGDX	3.0328×10${}^{-2}$(1.9920×10${}^{-3}$)	7.5464×10${}^{-2}$(1.2263×10${}^{-2}$)	5.5494×10${}^{-2}$(1.0934×10${}^{-2}$)	5.1242×10${}^{-2}$(9.4782×10${}^{-3}$)	2.9526×10${}^{-2}$(1.0200×10${}^{-3}$)	8.4117×10${}^{-2}$(4.5660×10${}^{-3}$)	2.5763×10${}^{-2}$(1.2162×10${}^{-3}$)
	IGDF	1.5698×10${}^{-3}$(1.0727×10${}^{-4}$)	1.5829×10${}^{-3}$(1.4770×10${}^{-4}$)	2.0530×10${}^{-3}$(3.3231×10${}^{-4}$)	1.7065×10${}^{-3}$(1.7301×10${}^{-4}$)	2.0136×10${}^{-3}$(1.2596×10${}^{-4}$)	6.9605×10${}^{-3}$(5.4121×10${}^{-4}$)	1.9755×10${}^{-3}$(1.5075×10${}^{-4}$)
MMF2	rPSP	3.9573×10${}^{-2}$(1.6761×10${}^{-2}$)	6.4752×10${}^{-2}$(4.7622×10${}^{-2}$)	5.3866×10${}^{-2}$(3.3966×10${}^{-2}$)	5.7411×10${}^{-2}$(3.0238×10${}^{-2}$)	2.5322×10${}^{-2}$(1.0985×10${}^{-2}$)	8.8438×10${}^{-2}$(6.1782×10${}^{-2}$)	1.8170×10${}^{-2}$(5.9853×10${}^{-3}$)
	rHV	1.0882×10${}^{+0}$(3.2063×10${}^{-1}$)	1.0622×10${}^{+0}$(3.1225×10${}^{-1}$)	1.0683×10${}^{+0}$(3.1418×10${}^{-1}$)	1.0630×10${}^{+0}$(3.1241×10${}^{-1}$)	1.0779×10${}^{+0}$(3.1731×10${}^{-1}$)	1.0846×10${}^{+0}$(3.1846×10${}^{-1}$)	1.0719×10${}^{+0}$(3.1533×10${}^{-1}$)
	IGDX	3.8468×10${}^{-2}$(1.5407×10${}^{-2}$)	5.9944×10${}^{-2}$(4.0824×10${}^{-2}$)	4.9750×10${}^{-2}$(2.7319×10${}^{-2}$)	5.4726×10${}^{-2}$(2.7670×10${}^{-2}$)	2.3993×10${}^{-2}$(9.7265×10${}^{-3}$)	8.8358×10${}^{-2}$(6.1801×10${}^{-2}$)	1.7759×10${}^{-2}$(5.8627×10${}^{-3}$)
	IGDF	1.7210×10${}^{-2}$(6.2329×10${}^{-3}$)	4.9329×10${}^{-3}$(5.3012×10${}^{-3}$)	8.1623×10${}^{-3}$(6.7755×10${}^{-3}$)	5.0734×10${}^{-3}$(4.0992×10${}^{-3}$)	1.2283×10${}^{-2}$(5.7738×10${}^{-3}$)	1.6374×10${}^{-2}$(5.7905×10${}^{-3}$)	7.8016×10${}^{-3}$(2.5737×10${}^{-3}$
MMF4	rPSP	2.0579×10${}^{-2}$(8.7789×10${}^{-3}$)	7.4849×10${}^{-2}$(3.0326×10${}^{-2}$)	4.4836×10${}^{-2}$(2.2676×10${}^{-2}$)	5.0017×10${}^{-2}$(2.3320×10${}^{-2}$)	1.7425×10${}^{-2}$(7.8290×10${}^{-3}$)	8.4134×10${}^{-2}$(4.6963×10${}^{-2}$)	1.4675×10${}^{-2}$(4.2218×10${}^{-3}$)
	rHV	1.6624×10${}^{+0}$(4.5992×10${}^{-1}$)	1.6596×10${}^{+0}$(4.6077×10${}^{-1}$)	1.6621×10${}^{+0}$(4.6117×10${}^{-1}$)	1.6605×10${}^{+0}$(4.6102×10${}^{-1}$)	1.6653×10${}^{+0}$(4.6154×10${}^{-1}$)	1.6909×10${}^{+0}$(4.6900×10${}^{-1}$)	1.6658×10${}^{+0}$(4.6210×10${}^{-1}$)
	IGDX	2.0563×10${}^{-2}$(8.7830×10${}^{-3}$)	7.2488×10${}^{-2}$(2.6323×10${}^{-2}$)	4.3585×10${}^{-2}$(1.9905×10${}^{-2}$)	4.9157×10${}^{-2}$(2.1735×10${}^{-2}$)	1.6995×10${}^{-2}$(6.8399×10${}^{-3}$)	8.2887×10${}^{-2}$(4.5058×10${}^{-2}$)	1.4573×10${}^{-2}$(4.2109×10${}^{-3}$)
	IGDF	3.6618×10${}^{-3}$(5.0897×10${}^{-3}$)	2.3078×10${}^{-3}$(3.6650×10${}^{-3}$)	3.1486×10${}^{-3}$(5.0146×10${}^{-3}$)	2.2140×10${}^{-3}$(2.7604×10${}^{-3}$)	3.6288×10${}^{-3}$(4.6275×10${}^{-3}$)	8.4242×10${}^{-3}$(3.9799×10${}^{-3}$)	2.7047×10${}^{-3}$(1.9362×10${}^{-3}$)
MMF5	rPSP	4.7896×10${}^{-2}$(1.3575×10${}^{-2}$)	1.2187×10${}^{-1}$(3.5342×10${}^{-2}$)	1.0176×10${}^{-1}$(2.9121×10${}^{-2}$)	1.0690×10${}^{-1}$(2.9566×10${}^{-2}$)	4.8283×10${}^{-2}$(1.3907×10${}^{-2}$)	1.3980×10${}^{-1}$(4.3140×10${}^{-2}$)	4.9105×10${}^{-2}$(1.2510×10${}^{-2}$)
	rHV	1.1471×10${}^{+0}$(2.9796×10${}^{-1}$)	1.1474×10${}^{+0}$(2.9818×10${}^{-1}$)	1.1484×10${}^{+0}$(2.9833×10${}^{-1}$)	1.1475×10${}^{+0}$(2.9805×10${}^{-1}$)	1.1489×10${}^{+0}$(2.9879×10${}^{-1}$)	1.1612×10${}^{+0}$(3.0306×10${}^{-1}$)	1.1489×10${}^{+0}$(2.9900×10${}^{-1}$)
	IGDX	4.7815×10${}^{-2}$(1.3551×10${}^{-2}$)	1.1919×10${}^{-1}$(3.3294×10${}^{-2}$)	1.0023×10${}^{-1}$(2.8573×10${}^{-2}$)	1.0573×10${}^{-1}$(2.9466×10${}^{-2}$)	4.8045×10${}^{-2}$(1.3965×10${}^{-2}$)	1.3859×10${}^{-1}$(4.3103×10${}^{-2}$)	4.1761×10${}^{-2}$(1.2468×10${}^{-2}$)
	IGDF	2.5965×10${}^{-3}$(5.0157×10${}^{-3}$)	2.0983×10${}^{-3}$(3.6123×10${}^{-3}$)	2.8638×10${}^{-3}$(4.9328×10${}^{-3}$)	2.0359×10${}^{-3}$(2.7153×10${}^{-3}$)	2.9333×10${}^{-3}$(4.5794×10${}^{-3}$)	7.2912×10${}^{-3}$(4.0541×10${}^{-3}$)	2.1336×10${}^{-3}$(1.9466×10${}^{-3}$)
MMF7	rPSP	1.9387×10${}^{-2}$(9.3975×10${}^{-3}$)	5.7247×10${}^{-2}$(3.1367×10${}^{-2}$)	3.6647×10${}^{-2}$(2.3315×10${}^{-2}$)	3.5249×10${}^{-2}$(2.2641×10${}^{-2}$)	1.9691×10${}^{-2}$(9.1406×10${}^{-3}$)	6.9463×10${}^{-2}$(4.1870×10${}^{-2}$)	1.7823×10${}^{-2}$(8.3248×10${}^{-3}$)
	rHV	1.0939×10${}^{+0}$(3.1305×10${}^{-1}$)	1.0940×10${}^{+0}$(3.1312×10${}^{-1}$)	1.0956×10${}^{+0}$(3.1351×10${}^{-1}$)	1.0945×10${}^{+0}$(3.1319×10${}^{-1}$)	1.0955×10${}^{+0}$(3.1384×10${}^{-1}$)	1.1053×10${}^{+0}$(3.1810×10${}^{-1}$)	1.0956×10${}^{+0}$(3.1410×10${}^{-1}$)
	IGDX	1.3931×10${}^{-2}$(9.3851×10${}^{-3}$)	5.5052×10${}^{-2}$(2.6287×10${}^{-2}$)	3.6114×10${}^{-2}$(2.2089×10${}^{-2}$)	3.4994×10${}^{-2}$(2.2456×10${}^{-2}$)	1.9613×10${}^{-2}$(9.0891×10${}^{-3}$)	6.5441×10${}^{-2}$(4.2268×10${}^{-2}$)	1.7762×10${}^{-2}$(8.2912×10${}^{-3}$)
	IGDF	2.4520×10${}^{-3}$(5.0404×10${}^{-3}$)	2.0851×10${}^{-3}$(3.6136×10${}^{-3}$)	2.9755×10${}^{-3}$(4.9031×10${}^{-3}$)	2.0540×10${}^{-3}$(2.7073×10${}^{-3}$)	2.8412×10${}^{-3}$(4.5950×10${}^{-3}$)	7.0477×10${}^{-3}$(4.0844×10${}^{-3}$)	2.1512×10${}^{-3}$(1.9387×10${}^{-3}$)
MMF8	rPSP	3.6319×10${}^{-2}$(1.0596×10${}^{-2}$)	6.3600×10${}^{-1}$(3.7420×10${}^{-1}$)	1.3760×10${}^{-1}$(9.5102×10${}^{-2}$)	1.1329×10${}^{-1}$(4.5487×10${}^{-2}$)	3.7333×10${}^{-2}$(1.0898×10${}^{-2}$)	4.9126×10${}^{-1}$(4.0283×10${}^{-1}$)	3.2405×10${}^{-2}$(1.0053×10${}^{-2}$)
	rHV	2.1002×10${}^{+0}$(6.0129×10${}^{-1}$)	2.0980×10${}^{+0}$(6.0054×10${}^{-1}$)	2.1022×10${}^{+0}$(6.0177×10${}^{-1}$)	2.0991×10${}^{+0}$(6.0084×10${}^{-1}$)	2.1153×10${}^{+0}$(6.0603×10${}^{-1}$)	2.1404×10${}^{+0}$(6.1291×10${}^{-1}$)	2.1146×10${}^{+0}$(6.0583×10${}^{-1}$)
	IGDX	3.2217×10${}^{-2}$(1.0008×10${}^{-2}$)	5.3568×10${}^{-1}$(2.9311×10${}^{-1}$)	1.3479×10${}^{-1}$(9.3095×10${}^{-2}$)	1.1106×10${}^{-1}$(4.4798×10${}^{-2}$)	3.7076×10${}^{-2}$(1.0809×10${}^{-2}$)	4.2621×10${}^{-1}$(2.9906×10${}^{-1}$)	3.6059×10${}^{-2}$(1.0534×10${}^{-2}$)
	IGDF	2.7154×10${}^{-3}$(4.9825×10${}^{-3}$)	2.0591×10${}^{-3}$(3.6179×10${}^{-3}$)	2.9780×10${}^{-3}$(4.9069×10${}^{-3}$)	2.1029×10${}^{-3}$(2.6977×10${}^{-3}$)	3.6962×10${}^{-3}$(4.4151×10${}^{-3}$)	7.1149×10${}^{-3}$(4.0691×10${}^{-3}$)	2.3955×10${}^{-3}$(1.9024×10${}^{-3}$)
MMF10	rPSP	8.8481×10${}^{-3}$(1.1246×10${}^{-2}$)	2.5191×10${}^{-1}$(2.7155×10${}^{-1}$)	1.5654×10${}^{-1}$(1.3180×10${}^{-1}$)	1.3713×10${}^{-1}$(1.2761×10${}^{-1}$)	2.6216×10${}^{-2}$(1.2256×10${}^{-2}$)	1.4365×10${}^{-1}$(3.1590×10${}^{-1}$)	1.4725×10${}^{-2}$(8.8940×10${}^{-3}$)
	rHV	3.4880×10${}^{-1}$(4.5879×10${}^{-1}$)	3.5165×10${}^{-1}$(4.5671×10${}^{-1}$)	3.5287×10${}^{-1}$(4.5781×10${}^{-1}$)	3.5120×10${}^{-1}$1(4.5708×10${}^{-1}$)	3.5314×10${}^{-1}$(4.6609×10${}^{-1}$)	3.5680×10${}^{-1}$(4.7447×10${}^{-1}$)	3.5280×10${}^{-1}$(4.6897×10${}^{-1}$)
	IGDX	8.8043×10${}^{-3}$(1.1229×10${}^{-2}$)	2.2546×10${}^{-1}$(2.2101×10${}^{-1}$)	1.5294×10${}^{-1}$(1.3199×10${}^{-1}$)	1.3410×10${}^{-1}$(1.2713×10${}^{-1}$)	2.5108×10${}^{-2}$(1.1186×10${}^{-2}$)	1.1967×10${}^{-1}$(2.3595×10${}^{-1}$)	1.4498×10${}^{-2}$(8.8366×10${}^{-3}$)
	IGDF	7.4909×10${}^{-3}$(4.2541×10${}^{-3}$)	1.4591×10${}^{-1}$(1.0654×10${}^{-1}$)	1.5492×10${}^{-1}$(1.0312×10${}^{-1}$)	1.2957×10${}^{-1}$(1.1531×10${}^{-1}$)	8.9074×10${}^{-2}$(3.6667×10${}^{-2}$)	9.3671×10${}^{-2}$(6.5256×10${}^{-2}$)	5.1006×10${}^{-2}$(1.9770×10${}^{-2}$)
MMF11	rPSP	6.1422×10${}^{-3}$(1.1237×10${}^{-2}$)	8.4835×10${}^{-2}$(2.7855×10${}^{-1}$)	3.5314×10${}^{-2}$(8.5845×10${}^{-2}$)	3.2982×10${}^{-2}$(7.3458×10${}^{-2}$)	9.0973×10${}^{-3}$(1.0656×10${}^{-2}$)	8.7426×10${}^{-2}$(3.1641×10${}^{-1}$)	7.3248×10${}^{-3}$(9.7250×10${}^{-3}$)
	rHV	1.8084×10${}^{-1}$(4.5516×10${}^{-1}$)	1.8085×10${}^{-1}$(4.5389×10${}^{-1}$)	1.8125×10${}^{-1}$(4.5512×10${}^{-1}$)	1.8086×10${}^{-1}$(4.5409×10${}^{-1}$)	1.8300×10${}^{-1}$(4.6294×10${}^{-1}$)	1.8522×10${}^{-1}$(4.7124×10${}^{-1}$)	1.8361×10${}^{-1}$(4.6576×10${}^{-1}$)
	IGDX	6.1330×10${}^{-3}$(1.1219×10${}^{-2}$)	6.9615×10${}^{-2}$(2.2043×10${}^{-1}$)	3.4446×10${}^{-2}$(8.4416×10${}^{-2}$)	3.2795×10${}^{-2}$(7.3430×10${}^{-2}$)	8.9742×10${}^{-3}$(1.0583×10${}^{-2}$)	6.7869×10${}^{-2}$(2.3633×10${}^{-1}$)	7.2899×10${}^{-3}$(9.6856×10${}^{-3}$)
	IGDF	9.5815×10${}^{-3}$(4.1010×10${}^{-3}$)	3.6071×10${}^{-2}$(5.9097×10${}^{-2}$)	3.9988×10${}^{-2}$(6.2797×10${}^{-2}$)	3.5967×10${}^{-2}$(5.9483×10${}^{-2}$)	2.8444×10${}^{-2}$(2.7189×10${}^{-2}$)	4.0228×10${}^{-2}$(4.7980×10${}^{-2}$)	2.2162×10${}^{-2}$(1.5078×10${}^{-2}$)
MMF12	rPSP	5.1099×10${}^{-3}$(1.1468×10${}^{-2}$)	7.4708×10${}^{-2}$(2.7902×10${}^{-1}$)	2.4964×10${}^{-2}$(8.4495×10${}^{-2}$)	2.1668×10${}^{-2}$(7.1639×10${}^{-2}$)	6.4725×10${}^{-3}$(1.0905×10${}^{-2}$)	8.1365×10${}^{-2}$(3.1734×10${}^{-1}$)	5.4530×10${}^{-3}$(1.0056×10${}^{-2}$)
	rHV	6.3158×10${}^{-1}$(3.6955×10${}^{-1}$)	6.4092×10${}^{-1}$(3.7060×10${}^{-1}$)	6.3340×10${}^{-1}$(3.6941×10${}^{-1}$)	6.4108×10${}^{-1}$(3.7080×10${}^{-1}$)	6.3646×10${}^{-1}$(3.7632×10${}^{-1}$)	6.3772×10${}^{-1}$(3.8383×10${}^{-1}$)	6.3588×10${}^{-1}$(3.7901×10${}^{-1}$)
	IGDX	5.1038×10${}^{-3}$(1.1450×10${}^{-2}$)	6.0563×10${}^{-2}$(2.2097×10${}^{-1}$)	2.4580×10${}^{-2}$(8.2986×10${}^{-2}$)	2.1384×10${}^{-2}$(7.1558×10${}^{-2}$)	6.4530×10${}^{-3}$(1.0846×10${}^{-2}$)	6.1990×10${}^{-2}$(2.3726×10${}^{-1}$)	5.4410×10${}^{-3}$(1.0013×10${}^{-2}$)
	IGDF	5.2930×10${}^{-3}$(5.9306×10${}^{-3}$)	2.0352×10${}^{-2}$(6.0080×10${}^{-2}$)	2.0688×10${}^{-2}$(6.3714×10${}^{-2}$)	2.0155×10${}^{-2}$(5.9869×10${}^{-2}$)	1.4400×10${}^{-2}$(2.8857×10${}^{-2}$)	2.0032×10${}^{-2}$(5.1241×10${}^{-2}$)	9.5889×10${}^{-3}$(1.6983×10${}^{-2}$)
MMF13	rPSP	2.6947×10${}^{-2}$(9.1879×10${}^{-3}$)	1.5088×10${}^{-1}$(2.6073×10${}^{-1}$)	8.2414×10${}^{-2}$(7.3363×10${}^{-2}$)	7.3058×10${}^{-2}$(6.0391×10${}^{-2}$)	3.3613×10${}^{-2}$(1.0230×10${}^{-2}$)	1.2363×10${}^{-1}$(3.0658×10${}^{-1}$)	2.6729×10${}^{-2}$(8.5077×10${}^{-3}$)
	rHV	1.9794×10${}^{-1}$(4.5621×10${}^{-1}$)	1.9811×10${}^{-1}$(4.5501×10${}^{-1}$)	1.9815×10${}^{-1}$(4.5619×10${}^{-1}$)	1.9819×10${}^{-1}$(4.5521×10${}^{-1}$)	2.0020×10${}^{-1}$(4.6397×10${}^{-1}$)	2.0219×10${}^{-1}$(4.7227×10${}^{-1}$)	2.0069×10${}^{-1}$(4.6680×10${}^{-1}$)
	IGDX	2.6924×10${}^{-2}$(9.1790×10${}^{-3}$)	1.2473×10${}^{-1}$(2.0416×10${}^{-1}$)	8.1734×10${}^{-2}$(7.1997×10${}^{-2}$)	7.2171×10${}^{-2}$(6.0456×10${}^{-2}$)	3.3272×10${}^{-2}$(1.0113×10${}^{-2}$)	1.0423×10${}^{-1}$(2.2665×10${}^{-1}$)	2.6498×10${}^{-2}$(8.4547×10${}^{-3}$)
	IGDF	1.1578×10${}^{-2}$(4.0956×10${}^{-3}$)	2.5969×10${}^{-2}$(5.6626×10${}^{-2}$)	3.4898×10${}^{-2}$(6.0045×10${}^{-2}$)	2.7389×10${}^{-2}$(5.6364×10${}^{-2}$)	3.3914×10${}^{-2}$(2.4986×10${}^{-2}$)	3.9737×10${}^{-2}$(4.6482×10${}^{-2}$)	2.1351×10${}^{-2}$(1.4635×10${}^{-2}$)
MMF14	rPSP	4.1910×10${}^{-2}$(1.1592×10${}^{-2}$)	1.5056×10${}^{-1}$(2.6014×10${}^{-1}$)	9.6022×10${}^{-2}$(7.0165×10${}^{-2}$)	8.6868×10${}^{-2}$(5.8512×10${}^{-2}$)	4.2862×10${}^{-2}$(1.1602×10${}^{-2}$)	2.0409×10${}^{-1}$(2.8858×10${}^{-1}$)	4.0854×10${}^{-2}$(1.1207×10${}^{-2}$)
	rHV	4.0147×10${}^{-1}$(4.0727×10${}^{-1}$)	4.0269×10${}^{-1}$(4.0293×10${}^{-1}$)	3.8556×10${}^{-1}$(4.0759×10${}^{-1}$)	3.8938×10${}^{-1}$(4.0575×10${}^{-1}$)	3.9660×10${}^{-1}$(4.1374×10${}^{-1}$)	4.1179×10${}^{-1}$(4.1982×10${}^{-1}$)	4.0802×10${}^{-1}$(4.1546×10${}^{-1}$)
	IGDX	4.1904×10${}^{-2}$(1.1593×10${}^{-2}$)	1.3567×10${}^{-1}$(2.0222×10${}^{-1}$)	9.5640×10${}^{-2}$(6.8734×10${}^{-2}$)	8.6831×10${}^{-2}$(5.8510×10${}^{-2}$)	4.2813×10${}^{-2}$(1.1597×10${}^{-2}$)	1.8455×10${}^{-1}$(2.0971×10${}^{-1}$)	4.0804×10${}^{-2}$(1.1200×10${}^{-2}$)
	IGDF	5.4539×10${}^{-2}$(1.6340×10${}^{-2}$)	9.6846×10${}^{-2}$(4.6160×10${}^{-2}$)	9.4576×10${}^{-2}$(5.0121×10${}^{-2}$)	8.4600×10${}^{-2}$(4.6496×10${}^{-2}$)	6.4696×10${}^{-2}$(2.3468×10${}^{-2}$)	1.3334×10${}^{-1}$(4.3382×10${}^{-2}$)	5.8273×10${}^{-2}$(1.6945×10${}^{-2}$)
MMF15	rPSP	4.2986×10${}^{-2}$(1.1230×10${}^{-2}$)	1.3517×10${}^{-1}$(2.6371×10${}^{-1}$)	9.1805×10${}^{-2}$(7.0885×10${}^{-2}$)	7.7907×10${}^{-2}$(6.0049×10${}^{-2}$)	4.7182×10${}^{-2}$(1.2528×10${}^{-2}$)	1.4709×10${}^{-1}$(3.0237×10${}^{-1}$)	4.405×10${}^{-2}$(1.1510×10${}^{-2}$)
	rHV	3.2743×10${}^{-1}$(4.2546×10${}^{-1}$)	3.2665×10${}^{-1}$(4.2150×10${}^{-1}$)	3.2329×10${}^{-1}$(4.2326×10${}^{-1}$)	3.2442×10${}^{-1}$(4.2196×10${}^{-1}$)	3.3365×10${}^{-1}$(4.2946×10${}^{-1}$)	3.3111×10${}^{-1}$(4.3874×10${}^{-1}$)	3.2896×10${}^{-1}$(4.3348×10${}^{-1}$)
	IGDX	4.2980×10${}^{-2}$(1.1229×10${}^{-2}$)	1.2101×10${}^{-1}$(2.0563×10${}^{-1}$)	9.1440×10${}^{-2}$(6.9444×10${}^{-2}$)	7.7906×10${}^{-2}$(6.0049×10${}^{-2}$)	4.7161×10${}^{-2}$(1.2514×10${}^{-2}$)	1.2776×10${}^{-1}$(2.2260×10${}^{-1}$)	4.3997×10${}^{-2}$(1.1490×10${}^{-2}$)
	IGDF	8.1385×10${}^{-2}$(2.2410×10${}^{-2}$)	1.3602×10${}^{-1}$(4.7501×10${}^{-2}$)	1.5589×10${}^{-1}$(5.2948×10${}^{-2}$)	1.2952×10${}^{-1}$(4.6961×10${}^{-2}$)	9.7474×10${}^{-2}$(2.8216×10${}^{-2}$)	1.4464×10${}^{-1}$(4.3814×10${}^{-2}$)	8.6563×10${}^{-2}$(2.3977×10${}^{-2}$)
MMF1_e	rPSP	2.5295×10${}^{-1}$(1.0091×10${}^{-1}$)	1.9878×10${}^{+0}$(1.6600×10${}^{+0}$)	1.3369×10${}^{+0}$(1.2026×10${}^{+0}$)	1.0495×10${}^{+0}$(5.7725×10${}^{-1}$)	3.8473×10${}^{-1}$(2.3103×10${}^{-1}$)	1.2670×10${}^{+0}$(1.0900×10${}^{+0}$)	8.0812×10${}^{-1}$(8.3709×10${}^{-1}$)
	rHV	1.0605×10${}^{+0}$(3.6559×10${}^{-1}$)	1.0577×10${}^{+0}$(3.6406×10${}^{-1}$)	1.0770×10${}^{+0}$(3.6709×10${}^{-1}$)	1.0582×10${}^{+0}$(3.6420×10${}^{-1}$)	1.0736×10${}^{+0}$(3.7092×10${}^{-1}$)	1.0935×10${}^{+0}$(3.8019×10${}^{-1}$)	1.0741×10${}^{+0}$(3.7299×10${}^{-1}$)
	IGDX	2.3873×10${}^{-1}$(9.0448×10${}^{-2}$)	1.1385×10${}^{+0}$(7.2140×10${}^{-1}$)	8.4842×10${}^{-1}$(5.3728×10${}^{-1}$)	7.7669×10${}^{-1}$1(3.6214×10${}^{-1}$)	3.3887×10${}^{-1}$(1.7606×10${}^{-1}$)	8.8705×10${}^{-1}$(5.5970×10${}^{-1}$)	5.8998×10${}^{-1}$(4.6403×10${}^{-1}$)
	IGDF	1.3848×10${}^{-2}$(1.9966×10${}^{-2}$)	2.7268×10${}^{-2}$(5.8372×10${}^{-2}$)	3.4463×10${}^{-2}$(6.2371×10${}^{-2}$)	2.7164×10${}^{-2}$(5.8417×10${}^{-2}$)	2.1051×10${}^{-2}$(2.8771×10${}^{-2}$)	2.9547×10${}^{-2}$(5.0972×10${}^{-2}$)	1.5898×10${}^{-2}$(1.9273×10${}^{-2}$)
MMF14_a	rPSP	9.8501×10${}^{-2}$(6.6172×10${}^{-2}$)	4.9059×10${}^{-1}$(1.0170×10${}^{+0}$)	3.9961×10${}^{-1}$(9.0669×10${}^{-1}$)	2.6576×10${}^{-1}$(4.1921×10${}^{-1}$)	1.1700×10${}^{-1}$(1.8135×10${}^{-1}$)	4.7431×10${}^{-1}$(7.7160×10${}^{-1}$)	2.5190×10${}^{-1}$(6.5067×10${}^{-1}$)
	rHV	4.6023×10${}^{-1}$(4.1027×10${}^{-1}$)	4.7199×10${}^{-1}$(4.0357×10${}^{-1}$)	4.4547×10${}^{-1}$(4.1014×10${}^{-1}$)	4.5422×10${}^{-1}$(4.0712×10${}^{-1}$)	4.6184×10${}^{-1}$(4.1494×10${}^{-1}$)	4.4325×10${}^{-1}$(4.2776×10${}^{-1}$)	4.5776×10${}^{-1}$(4.1893×10${}^{-1}$)
	IGDX	9.5640×10${}^{-2}$(5.8876×10${}^{-2}$)	2.9905×10${}^{-1}$(4.6208×10${}^{-1}$)	2.5536×10${}^{-1}$(4.2144×10${}^{-1}$)	2.1307×10${}^{-1}$(2.8613×10${}^{-1}$)	1.0450×10${}^{-1}$(1.4104×10${}^{-1}$)	3.5110×10${}^{-1}$(3.9516×10${}^{-1}$)	1.7050×10${}^{-1}$(3.5834×10${}^{-1}$)
	IGDF	5.8067×10${}^{-2}$(1.7432×10${}^{-2}$)	9.7959×10${}^{-2}$(4.5850×10${}^{-2}$)	1.0176×10${}^{-1}$(4.9979×10${}^{-2}$)	8.8945×10${}^{-2}$(4.6136×10${}^{-2}$)	6.2330×10${}^{-2}$(2.3163×10${}^{-2}$)	1.4429×10${}^{-1}$(4.5822×10${}^{-2}$)	5.8140×10${}^{-2}$(1.7326×10${}^{-2}$)
MMF15_a	rPSP	6.8011×10${}^{-2}$(6.8798×10${}^{-2}$)	3.4746×10${}^{-1}$(9.9411×10${}^{-1}$)	3.1338×10${}^{-1}$(8.9802×10${}^{-1}$)	1.8960×10${}^{-1}$(4.1569×10${}^{-1}$)	9.5891×10${}^{-2}$(1.7942×10${}^{-1}$)	4.9542×10${}^{-1}$(7.6995×10${}^{-1}$)	2.0756×10${}^{-1}$(6.4395×10${}^{-1}$)
	rHV	3.2551×10${}^{-1}$(4.2625×10${}^{-1}$)	3.3075×10${}^{-1}$(4.2120×10${}^{-1}$)	3.3190×10${}^{-1}$(4.2221×10${}^{-1}$)	3.2718×10${}^{-1}$(4.2192×10${}^{-1}$1)	3.3521×10${}^{-1}$(4.2949×10${}^{-1}$)	3.1747×10${}^{-1}$(4.4185×10${}^{-1}$)	3.2814×10${}^{-1}$(4.3388×10${}^{-1}$)
	IGDX	6.6158×10${}^{-2}$(6.1465×10${}^{-2}$	2.1426×10${}^{-1}$(4.5588×10${}^{-1}$)	1.9620×10${}^{-1}$(4.1929×10${}^{-1}$)	1.5713×10${}^{-1}$(2.8550×10${}^{-1}$)	8.5960×10${}^{-2}$(1.3964×10${}^{-1}$)	3.1059×10${}^{-1}$(3.9207×10${}^{-1}$)	1.3689×10${}^{-1}$(3.5454×10${}^{-1}$)
	IGDF	8.3820×10${}^{-2}$(2.2857×10${}^{-2}$)	1.5043×10${}^{-1}$(4.9843×10${}^{-2}$)	1.5674×10${}^{-1}$(5.2428×10${}^{-2}$)	1.4004×10${}^{-1}$(4.8312×10${}^{-2}$)	9.5996×10${}^{-2}$2(2.7970×10${}^{-2}$)	1.8942×10${}^{-1}$(5.3270×10${}^{-2}$)	8.9313×10${}^{-2}$(2.4917×10${}^{-2}$)
MMF10_l	rPSP	1.1118×10${}^{-1}$(7.5967×10${}^{-2}$)	5.8854×10${}^{+0}$(4.0955×10${}^{+0}$)	2.2084×10${}^{+0}$(3.1980×10${}^{+0}$)	4.7371×10${}^{+0}$(3.9776×10${}^{+0}$)	1.9439×10${}^{-1}$(1.6249×10${}^{-1}$)	1.2643×10${}^{+0}$(1.7486×10${}^{+0}$)	2.9513×10${}^{-1}$(6.2144×10${}^{-1}$)
	rHV	1.9208×10${}^{-1}$(4.5255×10${}^{-1}$)	1.9281×10${}^{-1}$(4.5108×10${}^{-1}$)	1.9435×10${}^{-1}$(4.5201×10${}^{-1}$)	1.9276×10${}^{-1}$(4.5125×10${}^{-1}$)	1.9509×10${}^{-1}$(4.6014×10${}^{-1}$)	1.9526×10${}^{-1}$(4.6861×10${}^{-1}$)	1.9500×10${}^{-1}$(4.6304×10${}^{-1}$)
	IGDX	1.0917×10${}^{-1}$(7.0632×10${}^{-2}$)	2.7230×10${}^{-1}$(4.4162×10${}^{-1}$)	2.4316×10${}^{-1}$(4.0906×10${}^{-1}$)	2.3371×10${}^{-1}$(2.6921×10${}^{-1}$)	1.7615×10${}^{-1}$(1.2264×10${}^{-1}$)	2.0963×10${}^{-1}$(4.1447×10${}^{-1}$)	2.2312×10${}^{-1}$(3.3361×10${}^{-1}$)
	IGDF	8.6794×10${}^{-2}$(3.0770×10${}^{-2}$)	1.7698×10${}^{-1}$(5.4922×10${}^{-2}$)	1.7998×10${}^{-1}$(6.6802×10${}^{-2}$)	1.8236×10${}^{-1}$(5.8248×10${}^{-2}$)	1.6032×10${}^{-1}$(4.6416×10${}^{-2}$)	2.0147×10${}^{-1}$(5.3897×10${}^{-2}$)	1.4601×10${}^{-1}$(4.1643×10${}^{-2}$)
MMF11_l	rPSP	1.4984×10${}^{+0}$(4.8986×10${}^{-1}$)	3.7271×10${}^{+0}$(1.9133×10${}^{+0}$)	2.1513×10${}^{+0}$(1.4348×10${}^{+0}$)	2.7669×10${}^{+0}$(2.0862×10${}^{+0}$)	3.5725×10${}^{-1}$(4.4515×10${}^{-1}$)	1.3821×10${}^{+0}$(6.7718×10${}^{-1}$)	7.0114×10${}^{-1}$(7.3469×10${}^{-1}$)
	rHV	1.7453×10${}^{-1}$(4.5501×10${}^{-1}$)	1.7428×10${}^{-1}$(4.5376×10${}^{-1}$)	1.7476×10${}^{-1}$(4.5500×10${}^{-1}$)	1.7435×10${}^{-1}$(4.5396×10${}^{-1}$)	1.7657×10${}^{-1}$(4.6281×10${}^{-1}$)	1.7874×10${}^{-1}$(4.7110×10${}^{-1}$)	1.7720×10${}^{-1}$(4.6564×10${}^{-1}$)
	IGDX	2.0694×10${}^{-1}$(1.2130×10${}^{-1}$)	3.3066×10${}^{-1}$(4.2895×10${}^{-1}$1)	3.1932×10${}^{-1}$(3.9145×10${}^{-1}$)	2.8816×10${}^{-1}$(2.5875×10${}^{-1}$)	2.2614×10${}^{-1}$(6.3706×10${}^{-2}$)	3.1391×10${}^{-1}$(3.8456×10${}^{-1}$)	2.7950×10${}^{-1}$(3.2344×10${}^{-1}$)
	IGDF	8.4878×10${}^{-2}$(2.6402×10${}^{-2}$)	1.0173×10${}^{-1}$(4.6038×10${}^{-2}$)	1.0402×10${}^{-1}$(5.0036×10${}^{-2}$)	1.0234×10${}^{-1}$(4.5983×10${}^{-2}$)	8.8337×10${}^{-2}$(3.0647×10${}^{-2}$)	1.1235×10${}^{-1}$(3.9813×10${}^{-2}$)	8.6340×10${}^{-2}$(2.2678×10${}^{-2}$)
MMF12_l	rPSP	1.2230×10${}^{+0}$(4.0194×10${}^{-1}$)	5.4330×10${}^{+0}$(3.6459×10${}^{+0}$)	2.7054×10${}^{+0}$(1.3762×10${}^{+0}$)	2.8187×10${}^{+0}$(2.0073×10${}^{+0}$)	3.8266×10${}^{-1}$(4.4694×10${}^{-1}$)	1.5398×10${}^{+0}$(6.9024×10${}^{-1}$)	7.6782×10${}^{-1}$(7.4197×10${}^{-1}$)
	rHV	6.3110×10${}^{-1}$(3.6957×10${}^{-1}$)	6.3081×10${}^{-1}$(3.6843×10${}^{-1}$)	6.3133×10${}^{-1}$(3.6953×10${}^{-1}$)	6.3094×10${}^{-1}$(3.6860×10${}^{-1}$)	6.3473×10${}^{-1}$(3.7649×10${}^{-1}$)	6.3796×10${}^{-1}$(3.8378×10${}^{-1}$)	6.3487×10${}^{-1}$(3.7910×10${}^{-1}$)
	IGDX	2.2733×10${}^{-1}$(6.3978×10${}^{-2}$)	3.3123×10${}^{-1}$(4.2950×10${}^{-1}$)	3.2151×10${}^{-1}$(3.9193×10${}^{-1}$)	2.8873×10${}^{-1}$(2.5933×10${}^{-1}$)	1.8958×10${}^{-1}$(1.2730×10${}^{-1}$)	3.1766×10${}^{-1}$(3.8522×10${}^{-1}$)	2.6702×10${}^{-1}$(3.2836×10${}^{-1}$)
	IGDF	7.9885×10${}^{-2}$(2.2256×10${}^{-2}$)	8.7167×10${}^{-2}$(4.6952×10${}^{-2}$)	8.8515×10${}^{-2}$(5.1303×10${}^{-2}$)	8.7240×10${}^{-2}$(4.7027×10${}^{-2}$)	6.5683×10${}^{-2}$(3.3893×10${}^{-2}$)	9.0096×10${}^{-2}$(4.0895×10${}^{-2}$)	6.3356×10${}^{-2}$(2.8488×10${}^{-2}$)
MMF13_l	rPSP	6.0336×10${}^{-1}$(3.2481×10${}^{-1}$)	1.6977×10${}^{+0}$(3.1417×10${}^{+0}$)	1.0867×10${}^{+0}$(1.6258×10${}^{+0}$)	1.2330×10${}^{+0}$(2.3150×10${}^{+0}$)	3.5849×10${}^{-1}$(2.7145×10${}^{-1}$)	7.7178×10${}^{-1}$(7.7167×10${}^{-1}$)	5.2464×10${}^{-1}$(5.8903×10${}^{-1}$)
	rHV	1.9787×10${}^{-1}$(4.5622×10${}^{-1}$)	1.9759×10${}^{-1}$(4.5498×10${}^{-1}$)	1.9793×10${}^{-1}$(4.5622×10${}^{-1}$)	1.9765×10${}^{-1}$(4.5518×10${}^{-1}$)	1.9997×10${}^{-1}$(4.6399×10${}^{-1}$)	2.0219×10${}^{-1}$(4.7227×10${}^{-1}$)	2.0055×10${}^{-1}$(4.6682×10${}^{-1}$)
	IGDX	2.3277×10${}^{-1}$(1.1833×10${}^{-1}$)	3.6932×10${}^{-1}$(4.2166×10${}^{-1}$)	3.5189×10${}^{-1}$(3.8587×10${}^{-1}$)	3.1456×10${}^{-1}$(2.5493×10${}^{-1}$)	2.3594×10${}^{-1}$(6.4516×10${}^{-2}$)	3.4039×10${}^{-1}$(3.8065×10${}^{-1}$)	2.9341×10${}^{-1}$(3.2102×10${}^{-1}$)
	IGDF	1.2932×10${}^{-1}$(3.6344×10${}^{-2}$)	1.3656×10${}^{-1}$(4.6598×10${}^{-2}$)	1.3988×10${}^{-1}$(4.9857×10${}^{-2}$)	1.3719×10${}^{-1}$(4.6702×10${}^{-2}$)	8.1117×10${}^{-2}$(3.3842×10${}^{-2}$)	1.4936×10${}^{-1}$(4.5168×10${}^{-2}$)	8.1341×10${}^{-2}$(3.1849×10${}^{-2}$)
MMF15_l	rPSP	3.5514×10${}^{-1}$(3.7112×10${}^{-1}$)	1.2183×10${}^{+0}$(3.1717×10${}^{+0}$)	6.9391×10${}^{-1}$(1.6950×10${}^{+0}$)	9.0741×10${}^{-1}$(2.3741×10${}^{+0}$)	2.2095×10${}^{-1}$(2.8990×10${}^{-1}$)	5.7247×10${}^{-1}$(8.0801×10${}^{-1}$)	3.3866×10${}^{-1}$(6.1596×10${}^{-1}$)
	rHV	3.0933×10${}^{-1}$(4.2618×10${}^{-1}$)	3.1070×10${}^{-1}$(4.2169×10${}^{-1}$)	3.0669×10${}^{-1}$(4.2401×10${}^{-1}$)	3.0778×10${}^{-1}$(4.2258×10${}^{-1}$)	3.1379×10${}^{-1}$(4.3070×10${}^{-1}$)	3.1703×10${}^{-1}$(4.3851×10${}^{-1}$)	3.0902×10${}^{-1}$(4.3463×10${}^{-1}$)
	IGDX	2.1678×10${}^{-1}$(6.3534×10${}^{-2}$	3.4746×10${}^{-1}$(4.2675×10${}^{-1}$)	3.1425×10${}^{-1}$(3.9438×10${}^{-1}$)	2.9007×10${}^{-1}$(2.5995×10${}^{-1}$)	1.7449×10${}^{-1}$(1.2642×10${}^{-1}$)	3.3589×10${}^{-1}$(3.8195×10${}^{-1}$)	2.5576×10${}^{-1}$(3.2919×10${}^{-1}$)
	IGDF	1.5949×10${}^{-1}$(4.2758×10${}^{-2}$)	1.8994×10${}^{-1}$(5.4658×10${}^{-2}$)	1.9009×10${}^{-1}$(5.5997×10${}^{-2}$)	1.7960×10${}^{-1}$(5.2395×10${}^{-2}$)	1.4914×10${}^{-1}$(4.1847×10${}^{-2}$)	2.0741×10${}^{-1}$(5.7881×10${}^{-2}$)	1.4785×10${}^{-1}$(4.1132×10${}^{-2}$)
MMF15_a_l	rPSP	3.1218×10${}^{-1}$(3.6154×10${}^{-1}$)	1.0417×10${}^{+0}$(3.2095×10${}^{+0}$)	6.2297×10${}^{-1}$(1.7046×10${}^{+0}$)	8.1088×10${}^{-1}$(2.3899×10${}^{+0}$)	2.1731×10${}^{-1}$(2.8815×10${}^{-1}$)	1.0585×10${}^{+0}$(9.6772×10${}^{-1}$)	3.1097×10${}^{-1}$(6.1937×10${}^{-1}$)
	rHV	3.1502×10${}^{-1}$(4.2673×10${}^{-1}$)	3.1895×10${}^{-1}$(4.2175×10${}^{-1}$)	3.1458×10${}^{-1}$(4.2402×10${}^{-1}$)	3.1100×10${}^{-1}$(4.2365×10${}^{-1}$)	3.2086×10${}^{-1}$(4.3083×10${}^{-1}$)	3.1889×10${}^{-1}$(4.4046×10${}^{-1}$)	3.1992×10${}^{-1}$(4.3390×10${}^{-1}$)
	IGDX	1.9875×10${}^{-1}$(5.8533×10${}^{-2}$)	3.1727×10${}^{-1}$(4.3282×10${}^{-1}$)	2.9038×10${}^{-1}$(3.9877×10${}^{-1}$)	2.6349×10${}^{-1}$(2.6421×10${}^{-1}$)	1.7466×10${}^{-1}$(1.2437×10${}^{-1}$)	4.8954×10${}^{-1}$(3.6510×10${}^{-1}$)	2.3716×10${}^{-1}$(3.3227×10${}^{-1}$)
	IGDF	1.6305×10${}^{-1}$(4.2373×10${}^{-2}$)	1.9269×10${}^{-1}$(5.3600×10${}^{-2}$)	2.0166×10${}^{-1}$(5.6775×10${}^{-2}$)	1.8704×10${}^{-1}$(5.2568×10${}^{-2}$)	1.5666×10${}^{-1}$(4.1225×10${}^{-2}$)	2.5153×10${}^{-1}$(6.8004×10${}^{-2}$)	1.5353×10${}^{-1}$(3.9899×10${}^{-2}$)
MMF16_l1	rPSP	2.1832×10${}^{-1}$(3.8126×10${}^{-1}$)	9.7190×10${}^{-1}$(3.2241×10${}^{+0}$)	5.9705×10${}^{-1}$(1.7104×10${}^{+0}$)	7.7514×10${}^{-1}$(2.3989×10${}^{+0}$)	1.7666×10${}^{-1}$(2.9706×10${}^{-1}$)	4.9735×10${}^{-1}$(8.2507×10${}^{-1}$)	2.6244×10${}^{-1}$(6.3079×10${}^{-1}$)
	rHV	3.1752×10${}^{-1}$(4.2620×10${}^{-1}$)	3.1772×10${}^{-1}$(4.2205×10${}^{-1}$)	3.1006×10${}^{-1}$(4.2497×10${}^{-1}$)	3.1207×10${}^{-1}$(4.2336×10${}^{-1}$)	3.1788×10${}^{-1}$(4.3156×10${}^{-1}$)	3.2520×10${}^{-1}$(4.3860×10${}^{-1}$)	3.1749×10${}^{-1}$(4.3464×10${}^{-1}$)
	IGDX	1.3520×10${}^{-1}$(5.4939×10${}^{-2}$)	2.7292×10${}^{-1}$(4.4222×10${}^{-1}$)	2.5740×10${}^{-1}$(4.0554×10${}^{-1}$)	2.2453×10${}^{-1}$(2.7150×10${}^{-1}$)	1.3588×10${}^{-1}$(1.3063×10${}^{-1}$)	3.1262×10${}^{-1}$(3.8910×10${}^{-1}$)	1.9225×10${}^{-1}$(3.4211×10${}^{-1}$)
	IGDF	1.2982×10${}^{-1}$(3.3061×10${}^{-2}$)	1.5922×10${}^{-1}$(4.7659×10${}^{-2}$)	1.5747×10${}^{-1}$(5.0227×10${}^{-2}$)	1.4965×10${}^{-1}$(4.6701×10${}^{-2}$)	1.2620×10${}^{-1}$(3.4123×10${}^{-2}$)	2.0144×10${}^{-1}$(5.3609×10${}^{-2}$)	1.2424×10${}^{-1}$(3.1887×10${}^{-2}$)
MMF16_l2	rPSP	3.6067×10${}^{-1}$(3.5062×10${}^{-1}$)	1.2955×10${}^{+0}$(3.1488×10${}^{+0}$)	8.1064×10${}^{-1}$(1.6661×10${}^{+0}$)	1.0402×10${}^{+0}$(2.3375×10${}^{+0}$)	2.4167×10${}^{-1}$(2.8283×10${}^{-1}$)	6.9618×10${}^{-1}$(7.8602×10${}^{-1}$)	3.5365×10${}^{-1}$(6.0898×10${}^{-1}$)
	rHV	3.1793×10${}^{-1}$(4.2615×10${}^{-1}$)	3.1758×10${}^{-1}$(4.2205×10${}^{-1}$)	3.1179×10${}^{-1}$(4.2454×10${}^{-1}$)	3.1655×10${}^{-1}$(4.2246×10${}^{-1}$)	3.1664×10${}^{-1}$(4.3174×10${}^{-1}$)	3.2816×10${}^{-1}$(4.3807×10${}^{-1}$)	3.1723×10${}^{-1}$(4.3455×10${}^{-1}$)
	IGDX	1.9989×10${}^{-1}$(1.2125×10${}^{-1}$)	3.8973×10${}^{-1}$(4.1691×10${}^{-1}$)	3.6819×10${}^{-1}$(3.8209×10${}^{-1}$)	3.4183×10${}^{-1}$(2.5016×10${}^{-1}$)	2.4602×10${}^{-1}$(7.1740×10${}^{-2}$)	3.8721×10${}^{-1}$(3.7161×10${}^{-1}$)	2.8003×10${}^{-1}$(3.2298×10${}^{-1}$)
	IGDF	1.9763×10${}^{-1}$(5.5487×10${}^{-2}$)	2.2172×10${}^{-1}$(6.1833×10${}^{-2}$)	2.1677×10${}^{-1}$(6.1381×10${}^{-2}$)	2.1156×10${}^{-1}$(5.9398×10${}^{-2}$)	1.8187×10${}^{-1}$(5.0529×10${}^{-2}$)	2.4685×10${}^{-1}$(6.7163×10${}^{-2}$)	1.8583×10${}^{-1}$(5.2109×10${}^{-2}$)
MMF16_l3	rPSP	3.0848×10${}^{-1}$(3.6371×10${}^{-1}$)	1.0667×10${}^{+0}$(3.2049×10${}^{+0}$)	6.7322×10${}^{-1}$(1.6931×10${}^{+0}$)	8.5319×10${}^{-1}$(2.3823×10${}^{+0}$)	2.0408×10${}^{-1}$(2.9122×10${}^{-1}$)	5.5665×10${}^{-1}$(8.0336×10${}^{-1}$)	2.9495×10${}^{-1}$(6.2354×10${}^{-1}$)
	rHV	3.1707×10${}^{-1}$(4.2633×10${}^{-1}$)	3.1789×10${}^{-1}$(4.2199×10${}^{-1}$)	3.0774×10${}^{-1}$(4.2541×10${}^{-1}$)	3.1153×10${}^{-1}$(4.2355×10${}^{-1}$)	3.1741×10${}^{-1}$(4.3158×10${}^{-1}$)	3.2256×10${}^{-1}$(4.3922×10${}^{-1}$)	3.1719×10${}^{-1}$(4.3455×10${}^{-1}$)
	IGDX	1.6339×10${}^{-1}$(1.2622×10${}^{-1}$)	3.1668×10${}^{-1}$(4.3368×10${}^{-1}$)	3.0231×10${}^{-1}$(3.9676×10${}^{-1}$)	2.6735×10${}^{-1}$(2.6418×10${}^{-1}$)	1.9272×10${}^{-1}$(5.6531×10${}^{-2}$)	3.4539×10${}^{-1}$(3.8096×10${}^{-1}$)	2.2430×10${}^{-1}$(3.3554×10${}^{-1}$)
	IGDF	1.6970×10${}^{-1}$(4.2863×10${}^{-2}$)	1.9249×10${}^{-1}$(5.1843×10${}^{-2}$)	1.8509×10${}^{-1}$(5.2703×10${}^{-2}$)	1.8207×10${}^{-1}$(5.0213×10${}^{-2}$)	1.5570×10${}^{-1}$(3.9720×10${}^{-2}$)	2.2536×10${}^{-1}$(5.8444×10${}^{-2}$)	1.5506×10${}^{-1}$(3.9332×10${}^{-2}$)
Rank		47/96	9/96	6/96	4/96	8/96	2/96	20/96
rPSP		10/24	0/24	0/24	0/24	5/24	0/24	9/24
rHV		8/24	7/24	6/24	1/24	0/24	2/24	0/24
IGDX		17/24	0/24	0/24	0/24	1/24	0/24	6/24
IGDF		12/24	2/24	0/24	3/24	2/24	0/24	5/24

Table 6. GD, IGD, and SP metrics comparison based on engineering design test suites.

Problem	NSMRFO	MOSMA [46]	MOBO/D [47]	MOMVO [48]	MOWCA [49]	NSGA-II [32]	MOMRFO [51]	IMOMRFO [28]
GD Metric
4-bar truss	3.3030×10${}^{-1}$	1.6635×10${}^{-1}$	1.9677×10${}^{+0}$	1.7194×10${}^{-1}$	1.3286×10${}^{+0}$	1.8236×10${}^{-1}$	1.4895×10${}^{-1}$	2.4210×10${}^{+0}$
	4.13×10${}^{-1}$	1.21×10${}^{-2}$	3.06×10${}^{+0}$	2.72×10${}^{-2}$	3.64×10${}^{+0}$	8.66×10${}^{-3}$	7.21×10${}^{-3}$	1.94×10${}^{+0}$
Disk brake	1.4345×10${}^{-3}$	2.3597×10${}^{-2}$	2.9302×10${}^{-3}$	8.0972×10${}^{-2}$	5.1287×10${}^{-3}$	7.0487×10${}^{-3}$	3.0258×10${}^{-3}$	4.7383×10${}^{-2}$
	2.72×10${}^{-4}$	2.70×10${}^{-2}$	1.45×10${}^{-3}$	1.45×10${}^{-1}$	3.22×10${}^{-3}$	1.26×10${}^{-2}$	5.62×10${}^{-4}$	7.82×10${}^{-2}$
Welded beam	5.2346×10${}^{-3}$	6.0605×10${}^{-1}$	3.7817×10${}^{-1}$	2.0160×10${}^{-2}$	5.9606×10${}^{-3}$	4.2063×10${}^{-3}$	3.9257×10${}^{-3}$	1.1569×10${}^{-1}$
	1.68×10${}^{-3}$	1.25×10${}^{+0}$	9.64×10${}^{-1}$	5.68×10${}^{-2}$	2.01×10${}^{-3}$	1.04×10${}^{-3}$	2.54×10${}^{-3}$	9.92×10${}^{-2}$
Speed reducer	7.2541×10${}^{+0}$	3.4412×10${}^{+1}$	1.2252×10${}^{+1}$	1.3694×10${}^{+1}$	1.7352×10${}^{+1}$	1.3368×10${}^{+1}$	1.4421×10${}^{+1}$	9.2079×10${}^{+0}$
	1.87×10${}^{+0}$	3.33×10${}^{+1}$	1.29×10${}^{+0}$	3.22×10${}^{+0}$	1.93×10${}^{+0}$	9.81×10${}^{-1}$	1.21×10${}^{+0}$	1.09×10${}^{+0}$
IGD Metric
4-bar truss	5.2306×10${}^{-4}$	1.2645×10${}^{-3}$	1.4296×10${}^{-2}$	3.4156×10${}^{-3}$	6.6312×10${}^{-4}$	5.9489×10${}^{-4}$	9.9940×10${}^{-4}$	1.1208×10${}^{-3}$
	2.99×10${}^{-5}$	1.21×10${}^{-3}$	4.93×10${}^{-3}$	6.11×10${}^{-4}$	3.30×10${}^{-4}$	4.06×10${}^{-5}$	8.66×10${}^{-5}$	6.15×10${}^{-5}$
Disk brake	2.4597×10${}^{-4}$	2.6790×10${}^{-3}$	1.7209×10${}^{-3}$	1.0017×10${}^{-3}$	5.7433×10${}^{-4}$	1.2484×10${}^{-3}$	4.0983×10${}^{-4}$	6.1796×10${}^{-4}$
	5.77×10${}^{-5}$	7.85×10${}^{-4}$	6.99×10${}^{-4}$	2.48×10${}^{-4}$	4.21×10${}^{-4}$	1.29×10${}^{-3}$	3.14×10${}^{-5}$	1.57×10${}^{-4}$
Welded beam	7.7497×10${}^{-4}$	7.0596×10${}^{-3}$	6.9524×10${}^{-3}$	1.3979×10${}^{-3}$	1.2635×10${}^{-3}$	3.0953×10${}^{-4}$	3.7374×10${}^{-4}$	4.7932×10${}^{-4}$
	4.75×10${}^{-4}$	1.66×10${}^{-3}$	4.25×10${}^{-3}$	7.84×10${}^{-4}$	1.75×10${}^{-3}$	4.33×10${}^{-5}$	4.88×10${}^{-5}$	1.29×10${}^{-4}$
Speed reducer	1.7887×10${}^{-3}$	1.9012×10${}^{-2}$	1.0443×10${}^{-2}$	5.9622×10${}^{-3}$	3.3948×10${}^{-3}$	2.7998×10${}^{-3}$	2.9813×10${}^{-3}$	3.3957×10${}^{-3}$
	2.16×10${}^{-5}$	2.68×10${}^{-3}$	8.99×10${}^{-3}$	5.54×10${}^{-3}$	1.44×10${}^{-3}$	4.73×10${}^{-5}$	8.81×10${}^{-5}$	7.20×10${}^{-4}$
SP Metric
4-bar truss	1.8155×10${}^{+0}$	4.8453×10${}^{+0}$	2.1874×10${}^{+1}$	3.9299×10${}^{+0}$	2.7300×10${}^{+0}$	2.5462×10${}^{+0}$	4.7759×10${}^{+0}$	1.4586×10${}^{+1}$
	7.79×10${}^{-1}$	3.81×10${}^{+0}$	1.67×10${}^{+1}$	3.77×10${}^{+0}$	3.77×10${}^{+0}$	2.21×10${}^{-1}$	6.71×10${}^{-1}$	1.45×10${}^{+1}$
Disk brake	9.3521×10${}^{-2}$	2.2820×10${}^{-1}$	1.7934×10${}^{-1}$	4.0540×10${}^{-1}$	8.0842×10${}^{-2}$	7.9256×10${}^{-2}$	1.1040×10${}^{-1}$	4.3313×10${}^{-1}$
	9.79×10${}^{-3}$	9.04×10${}^{-2}$	8.42×10${}^{-2}$	9.16×10${}^{-1}$	1.06×10${}^{-2}$	1.40×10${}^{-2}$	1.33×10${}^{-2}$	7.58×10${}^{-1}$
Welded beam	1.9678×10${}^{-1}$	6.4688×10${}^{+0}$	2.6550×10${}^{+0}$	2.3160×10${}^{-1}$	2.2191×10${}^{-1}$	2.3184×10${}^{-1}$	2.1139×10${}^{-1}$	5.4636×10${}^{-1}$
	4.31×10${}^{-2}$	1.23×10${}^{+1}$	4.69×10${}^{+0}$	1.22×10${}^{-1}$	3.93×10${}^{-2}$	2.34×10${}^{-2}$	3.02×10${}^{-2}$	4.81×10${}^{-1}$
Speed reducer	2.1976×10${}^{+1}$	2.5008×10${}^{+1}$	3.0519×10${}^{+1}$	4.0828×10${}^{+1}$	2.2979×10${}^{+1}$	2.3112×10${}^{+1}$	2.8427×10${}^{+1}$	6.3713×10${}^{+1}$
	2.17×10${}^{+0}$	6.28×10${}^{+0}$	1.34×10${}^{+1}$	1.33×10${}^{+1}$	2.60×10${}^{+0}$	1.66×10${}^{+0}$	3.91×10${}^{+0}$	1.95×10${}^{+1}$

Table 7. The characteristics of the system.

Systems	30-Bus [54]
Characteristics	Value	Details
Buses	30	-
Branches	41	-
Generators	3	Buses: 1, 2, and 8
Slack bus	1	Buse: 1
Wind generators	1	Buses: 5
Solar generators	2	Buse: 11 and 13
Small-hydro generators	1	Buse: 13
Active and reactive power	-	283.4 MW, 126.2 MVAr
Control variables	11	-

Table 8. Cost and emission coefficients of thermal generators [54].

	Generator	Bus	a	b	c	d	e	$\mathit{\alpha }$	$\mathit{\beta }$	$\mathit{\gamma }$	$\mathit{\xi }$	$\mathit{\lambda }$
IEEE-30	$P_{g1}$	1	30	2	0.00375	18	0.037	0.04091	−0.05554	0.06490	0.0002	6.667
	$P_{g2}$	2	25	1.75	0.0175	16	0.038	0.02543	−0.06047	0.05638	0.0005	3.333
	$P_{g3}$	8	20	3.25	0.00834	12	0.045	0.05326	−0.03550	0.03380	0.002	2

Table 9. Characteristic details of wind/solar/small-hydro generators [54].

	Wind Power (bus 5)			Solar Power (bus 11)		Combined Solar/Small-Hydro Power (bus 13)
Test Systems	No. of Turbines	Pwr (MW)	Parameters of Weibull PDF	Psr (MW)	Parameters of Lognormal PDF	Psr (MW)	Parameters of Lognormal PDF	Phr (MW)	Parameters of Gumbel PDF
IEEE-30	25	75	k = 2	50	$\mu$ = 5.2	45	$\mu$ = 5	5	$\lambda$ = 15
			c = 9		$\sigma$ = 0.6		$\sigma$ = 0.6		$\gamma$ = 1.2

Table 10. Findings of best objectives for case 1.

Case 1			NSMRFO		MOMVO [48]		MOWCA [49]		NSGWO [50]		MOMRFO [51]		IMOMRFO [28]
Control Variables	Min	Max	Cost	E	Cost	E	Cost	E	Cost	E	Cost	E	Cost	E
P${}_{tg2}$	20	80	53.108	60.441	53.813	79.882	65.933	52.442	54.421	56.651	59.090	64.799	43.141	64.982
P${}_{wg}$	0	75	51.903	73.597	56.252	74.961	35.136	74.937	50.662	15.858	47.240	66.735	55.886	67.336
P${}_{g3}$	10	35	16.476	31.077	13.194	27.016	24.800	24.722	13.128	24.091	11.981	32.089	15.660	21.897
P${}_{sg}$	0	50	17.407	21.171	16.889	25.689	13.209	50.000	17.505	10.181	21.577	46.457	18.109	40.445
P${}_{shg}$	0	50	15.878	45.724	15.115	22.804	16.516	34.040	17.273	37.732	16.048	25.442	15.018	37.884
V${}_{g1}$	0.95	1.1	1.0805	1.0684	1.1000	1.0874	1.0793	1.0732	1.0746	1.0259	1.0700	1.0655	1.0746	1.0838
V${}_{g2}$	0.95	1.1	1.0625	1.0577	1.0802	1.0705	1.0708	1.0557	1.0610	1.0287	1.0548	1.0570	1.0666	1.0753
V${}_{g5}$	0.95	1.1	1.0406	1.0360	1.0413	1.0556	1.0240	1.0307	1.0355	1.0407	1.0302	1.0296	1.0462	1.0272
V${}_{g8}$	0.95	1.1	1.0280	1.0215	1.0210	1.0252	1.0324	1.0244	1.0300	1.0031	1.0219	1.0267	1.0286	1.0323
V${}_{g11}$	0.95	1.1	1.0635	1.0574	1.0497	1.0614	1.0560	1.0709	1.0658	1.0789	1.0704	1.0551	1.0387	1.0523
V${}_{g13}$	0.95	1.1	1.0372	1.0320	0.9999	1.0089	1.0194	1.0594	1.0473	0.9886	1.0353	1.0423	1.0128	0.9982
Q${}_{tg1}$	−50	140	16.159	21.010	29.660	38.554	−2.3346	33.953	4.2530	29.911	7.9031	18.958	−6.1844	19.924
Q${}_{tg2}$	−20	60	22.069	24.129	48.350	13.470	59.861	9.4168	30.142	28.823	27.160	19.936	50.169	54.470
Q${}_{wg}$	−30	35	30.615	23.786	21.343	33.759	14.518	16.921	26.313	25.518	28.830	26.924	33.653	3.1405
Q${}_{tg3}$	−15	40	31.337	20.742	17.553	15.561	39.202	18.831	36.317	3.6417	32.787	18.327	36.748	30.527
Q${}_{sg}$	−20	25	20.441	20.458	18.014	20.990	18.491	24.584	20.608	23.828	24.787	28.416	13.772	19.339
Q${}_{shg}$	−20	25	15.616	15.216	2.8992	4.9409	9.2576	23.816	19.108	18.060	16.755	14.588	8.716	1.2780
Total Cost ($/h)	-	-	893.815	1012.83	895.799	998.532	904.351	1021.33	894.215	1024.58	895.589	1009.6	897.961	1010.91
Emission (t/h)	-	-	1.7686	0.0985	1.7594	0.1089	1.758	0.0969	1.9969	0.0968	2.1716	0.0987	2.7944	0.1019
VD (p.u.)	-	-	0.4561	0.5291	0.6419	0.5618	0.5177	0.4471	0.4326	0.4931	0.4699	0.4621	0.5982	0.6145
$P_{Loss}$ (MW)	-	-	6.3699	3.1979	6.7693	3.7273	7.0872	2.7821	6.5379	3.2890	5.7357	2.8983	6.5780	3.4661
P${}_{tg1}$	50	140	134.99	50.028	134.907	56.774	134.893	50.041	136.948	51.854	138.290	50.576	142.164	54.304

Table 11. Findings of best compromise solutions for case 1.

Case 1			NSMRFO	MOMVO [48]	MOWCA [49]	NSGWO [50]	MOMRFO [51]	IMOMRFO [28]
Control Variables	Min	Max	BCS	BCS	BCS	BCS	BCS	BCS
P${}_{tg2}$	20	80	60.658	79.791	65.943	64.856	64.438	65.828
P${}_{wg}$	0	75	55.620	55.825	54.797	56.854	54.207	57.585
P${}_{tg3}$	10	35	23.791	10.265	24.559	21.437	23.046	12.331
P${}_{sg}$	0	50	18.720	17.384	18.199	19.870	21.799	22.762
P${}_{shg}$	0	50	16.333	16.890	15.653	16.109	14.859	19.657
V${}_{g1}$	0.95	1.1	1.0749	1.0729	1.0795	1.0784	1.0701	1.0857
V${}_{g2}$	0.95	1.1	1.0642	1.0638	1.0710	1.0638	1.0593	1.0675
V${}_{g5}$	0.95	1.1	1.0405	1.0347	1.0245	1.0471	1.0406	1.0057
V${}_{g8}$	0.95	1.1	1.0331	1.0267	1.0285	1.0335	1.0248	1.0311
V${}_{g11}$	0.95	1.1	1.0624	1.0549	1.0445	1.0614	1.0582	1.0504
Control Variables	Min	Max	BCS	BCS	BCS	BCS	BCS	BCS
V${}_{g13}$	0.95	1.1	1.0331	1.0302	1.0208	1.0379	1.0313	1.0569
Q${}_{tg1}$	−50	140	5.6755	4.2337	5.0900	13.966	5.9788	24.335
Q${}_{tg2}$	−20	60	30.809	37.568	58.617	17.758	29.104	41.067
Q${}_{wg}$	−30	35	26.973	23.646	9.2601	32.981	33.386	−9.445
Q${}_{tg3}$	−15	40	35.741	36.326	34.291	33.961	29.605	40.288
Q${}_{sg}$	−20	25	19.544	18.581	15.261	18.622	19.976	15.144
Q${}_{shg}$	−20	25	13.604	14.277	10.954	14.964	15.157	22.829
Total Cost ($/h)	-	-	930.948	934.021	931.355	931.249	931.347	932.258
Emission (t/h)	-	-	0.4112	0.4190	0.4239	0.4158	0.4253	0.4602
VD (p.u.)	-	-	0.4542	0.5026	0.5474	0.4467	0.4959	0.4353
$P_L$ (MW)	-	-	5.2144	5.8737	5.5104	5.2299	5.3406	5.8467
P${}_{tg1}$	50	140	109.150	109.119	109.759	109.361	109.829	111.084

Table 12. Findings of best objectives for case 2.

Case 2			NSMRFO		MOMVO [48]		MOWCA [49]		NSGWO [50]		MOMRFO [51]		IMOMRFO [28]
Control Variables	Min	Max	Cost	VD	Cost	VD	Cost	VD	Cost	VD	Cost	VD	Cost	VD
P${}_{tg2}$	20	80	52.024	54.112	52.557	52.387	47.316	35.038	46.047	63.371	45.578	45.641	37.922	65.929
P${}_{wg}$	0	75	49.449	48.886	49.826	53.907	50.814	50.727	50.504	50.746	50.397	59.982	47.082	37.443
P${}_{g3}$	10	35	21.458	34.578	17.580	24.853	24.778	25.296	25.700	32.668	23.375	27.656	26.664	30.457
P${}_{sg}$	0	50	15.549	14.878	18.054	33.311	16.061	29.283	17.568	18.363	19.150	20.280	22.056	29.289
P${}_{shg}$	0	50	13.863	2.4460	14.652	3.8207	15.203	4.2886	13.752	9.6771	13.465	5.7879	18.004	1.0118
V${}_{g1}$	0.95	1.1	1.0770	1.0777	1.0800	1.0736	1.0731	1.0615	1.0734	1.0680	1.0777	1.0743	1.0841	1.0780
V${}_{g2}$	0.95	1.1	1.0589	1.0623	1.0679	1.0677	1.0598	1.0587	1.0576	1.0578	1.0631	1.0631	1.0756	1.0646
V${}_{g5}$	0.95	1.1	1.0236	1.0062	1.0329	1.0190	1.0408	1.0364	1.0195	1.0115	1.0257	1.0200	1.0418	1.0062
V${}_{g8}$	0.95	1.1	1.0297	1.0324	1.0355	1.0367	1.0357	1.0352	1.0297	1.0305	1.0311	1.0351	1.0336	1.0335
V${}_{g11}$	0.95	1.1	1.0781	1.0806	1.0769	1.0793	1.0707	1.0813	1.0754	1.0776	1.0741	1.0742	1.0690	1.0749
V${}_{g13}$	0.95	1.1	1.0586	1.0613	1.0461	1.0462	1.0587	1.0556	1.0535	1.0581	1.0595	1.0590	1.0235	1.0480
Q${}_{tg1}$	−50	140	13.538	9.7512	1.8904	−8.1601	1.8042	−23.430	8.5189	2.2257	6.3704	0.4094	−3.2325	8.4710
Q${}_{tg2}$	−20	60	25.623	44.482	40.716	59.259	23.322	48.275	32.641	41.144	37.354	44.935	58.645	48.064
Q${}_{wg}$	−30	35	15.823	−3.1762	17.560	2.4995	30.249	26.829	12.628	4.6642	14.424	4.9202	23.406	−0.8900
Q${}_{tg3}$	−15	40	34.891	38.425	36.648	39.452	37.310	38.764	36.776	38.255	32.672	39.733	26.422	39.927
Q${}_{sg}$	−20	25	24.310	24.842	23.003	24.592	20.470	24.999	23.981	24.619	22.406	22.098	21.381	23.793
Q${}_{shg}$	−20	25	23.001	24.408	16.671	18.017	22.157	22.211	21.697	23.860	22.938	23.213	8.442	20.075
Total Cost ($/h)	-	-	894.77	921.23	893.85	940.66	895.07	918.36	895.72	931.70	895.53	916.70	898.95	948.61
Control Variables	Min	Max	Cost	VD	Cost	VD	Cost	VD	Cost	VD	Cost	VD	Cost	VD
Emission (t/h)	-	-	2.0652	1.7635	2.0184	0.7516	1.8018	2.4120	1.8782	0.4839	2.0838	1.2817	1.841	1.0063
VD (p.u.)	-	-	0.4169	0.3975	0.4222	0.4054	0.4227	0.4041	0.4157	0.4044	0.4195	0.4064	0.4659	0.4033
$P_L$ (MW)	-	-	6.4450	6.4705	6.3981	5.4931	6.0562	6.1743	6.1556	5.4798	6.2143	5.6924	6.0657	6.4642
P${}_{tg1}$	50	140	137.50	134.97	137.13	120.61	135.31	139.98	135.98	122.42	6.3704	129.75	135.66	125.644

Table 13. Findings of best compromise solutions for case 2.

Case 2			NSMRFO	MOMVO [48]	MOWCA [49]	NSGWO [50]	MOMRFO [51]	IMOMRFO [28]
Control Variables	Min	Max	BCS	BCS	BCS	BCS	BCS	BCS
P${}_{tg2}$	20	80	53.594	52.415	41.134	45.147	45.137	39.393
P${}_{wg}$	0	75	49.649	50.911	48.375	50.559	51.718	45.047
P${}_{tg3}$	10	35	24.446	23.020	24.425	28.827	27.402	29.832
P${}_{sg}$	0	50	15.036	19.166	22.582	17.427	19.076	24.502
P${}_{shg}$	0	50	10.658	9.0253	13.720	10.795	10.006	15.466
V${}_{g1}$	0.95	1.1	1.0778	1.0753	1.0612	1.0679	1.0713	1.0752
V${}_{g2}$	0.95	1.1	1.0625	1.0678	1.0596	1.0577	1.0631	1.0536
V${}_{g5}$	0.95	1.1	1.0067	1.0212	1.0295	1.0113	1.0201	1.0248
V${}_{g8}$	0.95	1.1	1.0325	1.0358	1.0353	1.0306	1.0351	1.0223
V${}_{g11}$	0.95	1.1	1.0813	1.0814	1.0821	1.0785	1.0744	1.0654
V${}_{g13}$	0.95	1.1	1.0620	1.0462	1.0583	1.0582	1.0597	1.0545
Q${}_{tg1}$	−50	140	8.6578	−7.9539	−25.938	−4.4517	−8.3054	21.981
Q${}_{tg2}$	−20	60	43.962	57.301	55.280	48.401	51.7371	18.117
Q${}_{wg}$	−30	35	−3.3246	6.0463	20.146	4.5803	7.7620	24.266
Q${}_{tg3}$	−15	40	39.981	39.509	39.906	39.580	39.807	25.829
Q${}_{sg}$	−20	25	24.989	24.909	24.995	24.937	22.156	21.912
Q${}_{shg}$	−20	25	23.981	17.364	22.278	23.748	23.146	23.681
Total Cost ($/h)	-	-	897.98	899.55	898.24	898.43	916.70	900.87
Emission (t/h)	-	-	1.9546	1.7885	2.3326	1.9872	1.9026	1.7214
VD (p.u.)	-	-	0.4043	0.4087	0.4073	0.4049	0.4064	0.4330
$P_L$ (MW)	-	-	6.6032	6.3273	6.2834	6.2478	6.1310	6.0237
P${}_{tg1}$	50	140	136.62	135.19	139.45	136.89	136.19	134.58

Table 14. Findings of best objectives for case 3.

Case 3			NSMRFO		MOMVO [48]		MOWCA [49]		NSGWO [50]		MOMRFO [51]		IMOMRFO [28]
Control Variables	Min	Max	Cost	${\mathbf{P}}_{\mathbf{Loss}}$	Cost	${\mathbf{P}}_{\mathbf{Loss}}$	Cost	${\mathbf{P}}_{\mathbf{Loss}}$	Cost	${\mathbf{P}}_{\mathbf{Loss}}$	Cost	${\mathbf{P}}_{\mathbf{Loss}}$	Cost	${\mathbf{P}}_{\mathbf{Loss}}$
P${}_{tg2}$	20	80	48.838	49.192	44.767	45.204	40.985	45.256	40.474	63.806	38.9739	61.303	59.511	33.250
P${}_{wg}$	0	75	49.909	72.932	48.257	74.977	51.787	75.000	55.135	71.267	59.6321	74.869	56.777	72.769
P${}_{g3}$	10	35	21.138	32.741	27.145	26.376	22.983	35.000	22.511	29.912	16.5629	30.789	10.571	34.292
P${}_{sg}$	0	50	21.451	45.874	19.084	50.000	20.603	49.377	18.490	40.095	18.9793	44.802	18.489	49.458
P${}_{shg}$	0	50	13.105	26.710	15.521	13.398	16.772	31.261	16.845	25.878	16.6080	26.851	14.919	42.827
V${}_{g1}$	0.95	1.1	1.0738	1.0779	1.0842	1.0900	1.0585	1.0689	1.0769	1.0693	1.0713	1.0720	1.0873	1.0673
V${}_{g2}$	0.95	1.1	1.0646	1.0669	1.0606	1.0653	1.0507	1.0575	1.0678	1.0646	1.0585	1.0609	1.0593	1.0642
V${}_{g5}$	0.95	1.1	1.0473	1.0432	1.0276	1.0426	1.0248	1.0474	1.0387	1.0428	1.03330	1.0377	1.0415	1.0192
V${}_{g8}$	0.95	1.1	1.0374	1.0405	1.0267	1.0444	1.0205	1.0466	1.0363	1.0410	1.0273	1.0386	1.0147	1.0435
V${}_{g11}$	0.95	1.1	1.0598	1.0620	1.0376	1.0800	1.0570	1.0673	1.0679	1.0792	1.0562	1.0717	1.0564	1.0808
V${}_{g13}$	0.95	1.1	1.0414	1.0372	1.0316	1.0333	1.0468	1.0565	1.0499	1.0429	1.0290	1.0486	1.0239	1.0780
Q${}_{tg1}$	−50	140	−5.909	17.678	29.918	44.051	−9.9749	15.696	−5.4244	3.7508	2.6613	20.722	42.179	−3.7032
Q${}_{tg2}$	−20	60	34.404	24.895	21.033	−4.4405	44.003	3.8259	45.267	28.455	36.900	14.327	5.2565	50.301
Q${}_{wg}$	−30	35	34.193	18.916	21.373	16.885	25.371	26.239	21.516	20.238	23.887	16.920	36.104	−4.7289
Q${}_{tg3}$	−15	40	39.358	32.193	34.422	37.147	33.541	40.000	35.084	34.503	38.279	33.550	19.654	34.754
Q${}_{sg}$	−20	25	17.404	19.090	13.108	24.809	20.429	19.558	19.555	24.034	19.323	22.276	21.064	23.587
Q${}_{shg}$	−20	25	16.122	13.081	15.802	10.553	22.646	19.488	18.249	14.437	14.207	17.537	13.433	26.827
Total Cost ($/h)	-	-	896.04	1008.19	896.96	997.31	897.15	1021.53	896.12	1004.17	896.753	1033.83	898.125	1033.62
Emission (t/h)	-	-	1.7812	0.1017	1.7489	0.1313	1.9311	0.0958	1.8630	0.1008	2.0913	0.0963	1.7021	0.1018
VD (p.u.)	-	-	0.4338	0.4436	0.5350	0.4380	0.4654	0.4492	0.4286	0.4317	0.5059	0.4322	0.5660	0.5197
$P_{Loss}$ (MW)	-	-	6.1629	2.7309	6.2089	3.3192	6.1567	2.3448	5.9016	2.7980	6.0776	2.6358	6.7354	2.7088
P${}_{tg1}$	50	140	135.12	58.671	134.83	76.701	136.43	49.842	135.85	54.046	137.69	43.655	134.36	56.710

Table 15. Findings of best compromise solutions for case 3.

Case 3			NSMRFO	MOMVO [48]	MOWCA [49]	NSGWO [50]	MOMRFO [51]	IMOMRFO [28]
Control Variables	Min	Max	BCS	BCS	BCS	BCS	BCS	BCS
P${}_{tg2}$	20	80	34.493	31.895	40.145	23.841	39.028	37.708
P${}_{wg}$	0	75	66.511	71.867	58.895	68.321	64.085	66.952
P${}_{tg3}$	10	35	32.323	34.515	26.639	30.326	27.308	24.467
P${}_{sg}$	0	50	22.452	19.084	29.953	24.143	26.355	28.624
P${}_{shg}$	0	50	16.640	14.602	20.914	18.749	16.761	9.3818
V${}_{g1}$	0.95	1.1	1.0777	1.0858	1.0611	1.0782	1.0776	1.0829
V${}_{g2}$	0.95	1.1	1.0653	1.0606	1.0528	1.0623	1.0672	1.0693
V${}_{g5}$	0.95	1.1	1.0453	1.0506	1.0323	1.0337	1.0411	1.0524
V${}_{g8}$	0.95	1.1	1.0400	1.0363	1.0281	1.0379	1.0351	1.0335
V${}_{g11}$	0.95	1.1	1.0705	1.0305	1.0606	1.0775	1.0578	1.0698
V${}_{g13}$	0.95	1.1	1.0418	1.0387	1.0495	1.0494	1.0497	1.0602
Q${}_{tg1}$	−50	140	5.5834	35.083	−4.2225	10.982	2.9712	9.3361
Q${}_{tg2}$	−20	60	28.456	−0.7926	32.816	28.018	38.769	27.986
Q${}_{wg}$	−30	35	25.306	33.461	26.109	14.683	21.568	30.466
Q${}_{tg3}$	−15	40	35.318	37.243	34.519	35.900	31.389	22.263
Q${}_{sg}$	−20	25	20.664	8.3323	20.123	22.938	16.581	19.608
Q${}_{shg}$	−20	25	14.609	16.692	21.494	17.543	18.581	22.011
Total Cost ($/h)	-	-	926.22	931.32	936.50	927.55	931.87	931.34
Emission (t/h)	-	-	0.7300	0.6514	0.4596	0.8453	0.5106	0.6806
VD (p.u.)	-	-	0.4306	0.4855	0.4380	0.4233	0.4357	0.4373
$P_{Loss}$ (MW)	-	-	4.6000	4.6938	4.6349	4.6027	4.6366	4.8748
P${}_{tg1}$	50	140	120.07	118.03	111.49	122.62	113.52	118.84

Figure 8. Obtained Pareto front of NSMRFO on classical test suites.

Figure 9. Obtained Pareto front of NSMRFO on ZDT test suites.

Figure 10. Obtained Pareto front of NSMRFO on TYD test suites.

Figure 11. Obtained Pareto front of NSMRFO on constrained test suites.

Figure 12. Cont.

Figure 13. Obtained Pareto front of NSMRFO on engineering design test suites. (a) 4-bar truss; (b) disk brake; (c) welded beam; (d) speed reducer.

Figure 14. Optimal Pareto fronts of all the algorithms for case 1. (a) Pareto front of optimal solutions; (b) voltage profile of PQ buses.

Figure 15. Optimal Pareto fronts of all the algorithms for case 2. (a) Pareto front of optimal solutions; (b) voltage profile of PQ buses.

Figure 16. Optimal Pareto fronts of all the algorithms for case 3. (a) Pareto front of optimal solutions; (b) voltage profile of PQ buses.

3.1. Evaluation on Unconstrained Benchmark Functions

As mentioned above, the proposed approach is tested firstly on the classical unconstrained test problems with two and three objectives. The achieved mean and STD values of 20 runs of each parameter metrics from NSMRFO and different approaches are presented in Table 3 and indicated in bold. It is worth noting here that the better algorithm is the one with the lower metric value. The suggested approach NSMRFO managed to outperform the MOSMA [46], MOBO/D [47], MOMVO [48], MOWCA [49], MOMRFO [51], (IMOMRFO) [28], and (NSGA-II) [32] optimizer significantly on eight functions out of 18 cases for GD, 14 out of 10 cases for IGD, and 14 out of 18 cases for SP metrics. By comparison, MOSMA is better on SCH2 for all metrics, the MOWCA is best on FON2 for GD, on POL for IGD, on POL, and VNT3 for SP, the MOMVO is best on just SCH1 for SP metric, the IMOMRFO is best on SCH1 for GD, and VNT2 for IGD; however, the NSGA-II and MOMRFO offered a good solution on four and eight functions, respectively. By contrast, the MOBO/D optimizer provides the worst results. Therefore, it may be observed from this table that the NSMRFO approach is able to outperform all competitors on most cases. Furthermore, it is also evident from Figure 8, Figure 9 and Figure 10 that NSMRFO converges better toward a true Pareto front with different features from diverse perspectives. In addition, the Pareto optimal solutions have been well distributed over the true PF on the classical functions.

3.2. Evaluation on Constrained Benchmark Functions

To evaluate the accuracy of the developed NSMRFO approach, four constrained test functions with different Pareto optimal fronts and three analysis metrics were investigated. Inspecting the obtained Pareto fronts in Figure 11, and the outcomes in Table 4, it is clearly seen that the suggested NSMRFO yields a higher convergence and coverage toward the true PF on all constrained benchmark functions. For the numerical results, NSMRFO ranks first for the majority of functions except on BNH and CONSTR for GD, and BNH, SRN for IGD in which it ranks second compared to the aforementioned well-known competitive techniques, and OZY for SP. Note that the optimal findings are marked in boldface and underlined.

3.3. Evaluation on CEC2020 Benchmark Functions

This subsection presents the performance of the suggested technique NSMRFO in CEC2020 multimodal multi-objective optimization (MMO) problems using four indicator metrics, the $rPSP$ and $IGDX$ that reflect the quality of the Pareto set in the search space, and the $rHV$ and $IGDF$ that reflect the quality of the Pareto front in the objective space. The functions of the remaining MMO problems are characterized by different geometries’ linear and nonlinear concave and convex functions. To illustrate the effectiveness of the multi-objective MRFO version, six well-known competitors are adopted for comparison such as: NSGA-II [32], DN-NSGAII [52], OMNI-OPT [53], MO_Ring_PSO_SCD [44], MOMRFO [51], IMOMRFO [28]. The numerical statistical results of the obtained parameter indexes in search and objective spaces by each approach are summarized in Table 5. It is worth noting that the optimal results of each indicator are the lowest values. The underlined bold solutions indicate the algorithms’ optimum result. Additionally, the last five rows posted in this table present the results score of each approach, in which the NSMRFO ranks first by providing 47 optimal solutions out of a 96 benchmark suite, which means twenty-four functions times four indicators. By contrast, the MOMRFO, OMNI-OPT, and DN-NSGAII competitor approaches show the worst scores. It can be clearly observed from the perspective of the search space values that the crowding distance mechanism has the capability to efficiently increase the PS convergence and diversity of the optimization algorithms. However, In spite of the same strategy used in NSGAII, DN-NSGAII, and the proposed optimizer, they offered a significant different performance, which means that the NSMRFO diversity and convergence are improved. The IMOMRFO was the closest approach of NSMRFO, where it ranks second by having 20 best values out of 96, and it offered good search space results and poor objective space results. The NSGAII performed a little better on the objective space, especially the $rHV$ metric. The CEC2020 corresponding box plots of the four metrics rPSP, rHV, IGDX, and IGDF are depicted in Figure 12. According to this figure, the NSMRFO achieved the indicator minimum values in most MMFs such as MMF1, MMF4, MMF5, MMF7, MMF8, MMF11, MMF12, MMF13, MMF14, MMF15, MMF1_e, MMF14_a, MMF15_a, MMF10_l, rHV on MMF11_l, rPSP and rHV on MMF12_l, rHV on MMF13_l, rPSP and rHV on MMF15_a_l. In addition, the NSMRFO is more stable compared to its competitor approaches. To sum up, the suggested optimizer achieves the best rank performance in terms of all indicator metrics compared to its competing algorithms, and have significant stability.

3.4. Evaluation on Engineering Design Problems

For examining the applicability of an algorithm, the engineering design problems can be very beneficial. In this subsection, four engineering functions are considered in order to assess the capability of NSMRFO in dealing with the real-word problems. The first one is 4-bar truss, and it aims to optimize the volume and deflection with four dimensions. The disk brake consists of minimizing the stopping time and weight of a brake system with four dimensions. The third engineering problem is the welded beam that tends to decrease the vertical deflection and cost of fabrication with four dimensions. The speed reducer as a last function attempts to reduce its stress and total weight with seven dimensions. For results verification, seven well-known approaches are applied. The statistical results are summarized in Table 6, and it is evident that NSMRFO can outrank the other algorithms on the most problems except the 4-bar truss for the GD metric, the welded beam for GD and SP metrics, and disk brake for SP metric; it ranks first in 8 out of 12 test suites. Accordingly, the NSGA-II and MOMRFO are the closest competitors where they provide good estimations on two functions over 12. However, concerning the other algorithms MOSMA, MOBO/D, MOMVO and IMOMRFO, they have the lowest results. As illustrated in Figure 13, the NSMRFO Pareto front shows higher approximations toward the true PFs in terms of coverage and convergence.

3.5. Evaluation on OPF Incorporating Wind/Solar/Small-Hydro Energy

3.5.1. Problem Methodology

 Wind Power

The wind cost can be expressed as below:

$$C_{Tw}=C_{dw}+C_{uew}+C_{oew}$$

(23)

with

$$C_{dw}=d_w{P}_{ws}$$

(24)

$$C_{uew}=K_{uew}\left(P_{wav}-P_{ws}\right)=K_{uew}{\int }_{P_{ws}}^{P_{wr}}\left(p_w-P_{ws}\right)f_w\left(p_w\right)d{p}_w$$

(25)

$$C_{oew}=K_{oew}\left(P_{ws}-P_{wav}\right)=K_{oew}{\int }_0^{P_{ws}}\left(P_{ws}-p_w\right)f_w\left(p_w\right)d{p}_w$$

(26)

where $d_w$, $K_{oew}$, and $K_{uew}$ are the coefficients of direct, over, and under estimation cost, respectively. $P_{ws,i}$, $P_{wav}$ are the scheduled and actual available power, respectively. $P_{wr}$ is rated power output from the plant. $f_w\left(p_w\right)$ is the probability density function of the wind power plant.

 Solar power

The total solar cost can be formulated as follows:

$$C_{Ts}=C_{ds}+C_{ues}+C_{oes}$$

(27)

with

$$C_{ds}=d_s{P}_{ss}$$

(28)

$$C_{ues}=K_{ues}\left(P_{sav}-P_{ss}\right)=K_{ues}*f_s\left(P_{sav}>P_{ss}\right)*\left[E\left(P_{sav}>P_{ss}\right)-P_{ss}\right]$$

(29)

$$C_{oes}=K_{oes}\left(P_{ss}-P_{sav}\right)=K_{oeS}*f_s\left(P_{sav}<P_{ss}\right)*\left[P_{ss}-E\left(P_{sav}<P_{ss}\right)\right]$$

(30)

where $d_s$, $K_{oes}$ and $K_{ues}$ are the coefficients of direct, over and under estimation cost of solar power generator. $P_{ss}$, $P_{sav}$ are the scheduled and actual available power, respectively. $f_s\left(P_{sav}<P_{ss}\right)$ The probability of occurrence of solar power shortage. $E\left(P_{sav}>P_{ss}\right)$ and $E\left(P_{sav}<P_{ss}\right)$ are the expectations of solar power above and below $P_{ss}$.

 Small-hydro power

The Small-hydro power is defined as follows:

$$C_{Tsh}=C_{dsh}+C_{uesh}+C_{oesh}$$

(31)

with

$$C_{ds}=d_s{P}_{ss}+d_h{P}_{hs}$$

(32)

$$C_{uesh}=K_{uesh}\left(P_{\mathrm{shav}}-P_{\mathrm{ssh}}\right)=K_{uesh}*f_{\mathrm{sh}}\left(P_{\mathrm{shav}}>P_{\mathrm{ssh}}\right)*\left[E\left(P_{\mathrm{shav}}>P_{\mathrm{ssh}}\right)-P_{\mathrm{ssh}}\right]$$

(33)

$$C_{oesh}=K_{oesh}\left(P_{shs}-P_{shav}\right)=K_{oesh}*f_{sh}\left(P_{shav}<P_{shs}\right)*\left[P_{shs}-E\left(P_{shav}<P_{shs}\right)\right]$$

(34)

where $d_h$ is the small-hydro direct cost coefficient. $K_{oesh}$ and $K_{uesh}$ are the coefficients of over and under estimation cost of combined solar and small-hydro power generator. $P_{shs}$, $P_{shav}$ are the scheduled and actual available power, respectively. $E\left(P_{shav}>P_{shs}\right)$ and $E\left(P_{shav}<P_{shs}\right)$ are the expectations of combined system power above and below $P_{shs}$.

 Objective functions

• Total cost

The network total cost including the thermal/wind/solar/small-hydro generators is modeled as follows:

$$F_1=min\left\{F_T+C_{Tw}+C_{Ts}+C_{Tsh}\right\}.$$

(35)

where

$$F_T=\begin{array}{c}\sum _{i=1}^{Ng}{a}_i+b_i{P}_{tgi}+c_i{P}_{tgi}^2+|d_i*sin(e_i*(P_{tgi}^{min}-P_{tgi}))|\hfill \end{array}$$

(36)

$a_i$, $b_i$ and $c_i$ are the conventional generators cost coefficients. $d_i$ and $e_i$ are the coefficients for the valve-point loading effect.

• Emission

The emission function is formulated using an exponential function as shown below [54]:

$$F_2=E=min\left\{\sum _{i=1}^{Ng}{10}^{-2}\left({\alpha }_i+{\beta }_i{P}_{gi}+{\gamma }_i{P}_{gi}^2\right)+{\xi }_{i}exp\left({\lambda }_i{P}_{gi}\right)\right\}$$

(37)

where ${\alpha }_i$, ${\beta }_i$, ${\gamma }_i$, ${\xi }_i$, and ${\lambda }_i$ are the emission coefficients of the power plant.

• Voltage deviation

The voltage deviation is calculated by:

$$F_3=VD=min\left\{\begin{array}{c}\sum _{i=1}^{Npq}|V_{Li}-1.0|\hfill \end{array}\right\}$$

(38)

• Power loss

The Power loss is calculated by:

$$F_4=P_{loss}=min\left\{\begin{array}{c}\sum _{l=1}^{N_l}{G}_{l(i,j)}(V_i^2+V_j^2-2V_i{V}_{j}cos\left({\delta }_{ij}\right))\hfill \end{array}\right\}$$

(39)

where $G_{l(i,j)}$ represents the conductance of line l. ${\delta }_{ij}={\delta }_i-{\delta }_j$ represents the voltage angle difference between bus i and bus j.

 Constraints

• Equality constraints

The power flow equations are assumed as equality constraints that are represented by:

$$\left\{\begin{array}{c}{P}_{gi}-P_{di}-|V_i|\sum _{j=1}^{Nb}\left|V_j\right|[G_{ij}cos\left({\theta }_{ij}\right)+B_{ij}sin\left({\theta }_{ij}\right)]=0\hfill \\ Q_{gi}-Q_{di}-|V_i|\sum _{j=1}^{Nb}\left|V_j\right|[G_{ij}sin\left({\theta }_{ij}\right)-B_{ij}cos\left({\theta }_{ij}\right)]=0\hfill \end{array}\right.$$

(40)

where $Nb$ is the number of buses. $Q_{gi}$ and $P_{gi}$ are generated reactive and active power, respectively. $Q_{di}$ and $P_{di}$ are reactive and active power demand, respectively. $G_{ij}$ and $B_{ij}$ represent the admittance matrix components $Y_{ij}=G_{ij}+j{B}_{ij}$ called the conductance and susceptance.

• Inequality constraints

The inequality constraints are given as below:

−

Generator constraints:

$$V_{gi}^{min}\le V_{gi}\le V_{gi}^{max}i=1,...,Ng$$

(41)

$$P_{tgi}^{min}\le P_{tgi}\le P_{tgi}^{max}i=1,...,Ntg$$

(42)

$$P_{ws,i}^{min}\le P_{ws,i}\le P_{ws,i}^{max}i=1,...,Nwg$$

(43)

$$P_{ss,i}^{min}\le P_{ss,i}\le P_{ss,i}^{max}i=1,...,Nsg$$

(44)

$$P_{shs,i}^{min}\le P_{shs,i}\le P_{shs,i}^{max}i=1,...,Nshg$$

(45)

$$Q_{tgi}^{min}\le Q_{tgi}\le Q_{tgi}^{max}i=1,...,Ntg$$

(46)

$$Q_{ws,i}^{min}\le Q_{ws,i}\le Q_{ws,i}^{max}i=1,...,Nwg$$

(47)

$$Q_{ss,i}^{min}\le Q_{ss,i}\le Q_{ss,i}^{max}i=1,...,Nsg$$

(48)

$$Q_{shs,i}^{min}\le Q_{shs,i}\le Q_{shs,i}^{max}i=1,...,Nshg$$

(49)

−

Security constraints:

$$V_{Li}^{min}\le V_{Li}\le V_{Li}^{max}i=1,...,Npq$$

(50)

$$S_{li}\le S_{li}^{max}i=1,...,Nl$$

(51)

where $Nl$ is the number of transmission lines. $S_{li}$ and $S_{li}^{max}$ indicate the maximum limit of the transmission line.

3.5.2. Results of the OPF Problem

To assess performance of the suggested algorithm NSMRFO against other approaches, several cases related to the modified IEEE 30-bus optimal power flow problem integrating wind/solar/small-hydro power are investigated. This test system comprises 41 branches, 6 generating units in which 3 thermal generators at buses 1, 2, and 8, the wind and solar plants at buses 5 and 11, respectively, while combined solar and small-hydro generators are connected to bus 13 as summarized in Table 7. The detailed input data for the considered IEEE 30-bus system are given in [54]. The thermal generators’ coefficients are provided in Table 8. Solar irradiance, wind distribution, and small-hydro river flow rate are modeled using Lognormal, Weibull, and Gumbel probability density function (PDF), respectively [54]. These PDF parameters are listed in Table 9. Additionally, in terms of the optimization issue, the system has 11 control variables, with various constraints and objective functions for a total active and reactive power demands of 283.4 MW and 126.2 MVAR, respectively.

To validate the suggested approach, five well-known stochastic algorithms are employed as competitors, namely MOMVO [48], MOWCA [49], NSGWO [50], MOMRFO [51], and IMOMRFO [28]. The test system under study is examined via three case studies defined as follows:

• Minimizing the total cost and emission;
• Minimizing the total cost and voltage deviation;
• Minimizing the total cost and power loss.

The optimum settings of the control variables, their allowable ranges, the numerical best outcomes of each objective, and the best compromise solutions (BCS) are depicted in Table 10, Table 11, Table 12, Table 13, Table 14 and Table 15. Furthermore, their optimal Pareto fronts are illustrated in Figure 14a–Figure 16a. It is worth noting that all the findings are generated after twenty independent runs for a population size of 100 and 200 iterations. According to the aforementioned tables, it is obviously seen that the NSMRFO’ results are remarkably better than the competitor approaches in all cases, notably the best compromise solutions’ tables. In addition, it is clearly observed from the figures that the suggested NSMRFO can generate the superior Pareto non-dominated solutions with good distribution and good diversification front in comparison to other algorithms. As shown in Figure 14b–Figure 16b, the BCS’ voltage profile PQ of load buses do not exceed their limits, and remained within the minimum and maximum bounds.

3.6. Discussion

As previously stated, the main difference between the suggested approach and its competitors is the better diversity and accuracy for the majority of the problems, in which the NSMRFO ranks first, followed by the MOMRFO on the unconstrained problems, the NSGA-II on the constrained test suite, the NSGA-II on the engineering problems, and the IMOMRFO on the CEC2020 benchmark MMO functions, while the MOWCA gives a little better score. By contrast, the MOSMA, MOMVO and MOBO/D provide the worst rank. However, on the other hand, the NSMRFO generates very challenging and competitive solutions on most benchmark test suites. In summary, all quantitative and qualitative outcomes and analysis reveal the higher accuracy and significant diversity of NSMRFO in dealing with different unconstrained, constrained, CEC 2020 multimodal multi-objective, and engineering benchmark functions. This comes from the strong ability of MRFO in exploitation and exploration as long as the NSMRFO employs similar mechanisms as its single approach, and inherits its high convergence. In addition, the crowding distance and archive selection methodologies also contribute to the NSMRFO high coverage and convergence.

4. Conclusions

In this work, the ability of a suggested multi-objective manta ray foraging optimizer known as NSMRFO to handle problems with different characteristics has been tested. The NSMRFO optimizer has been developed on the basis of NSGA-II operators as crowding distance, elitist non-dominated sorting, and an archive mechanism. A set of test functions have been employed to benchmark the performance of the NSMRFO approach from different perspectives that include: seven classical, ZDT, TYD, four constrained, twenty-four CEC2020, four problems for engineering design, and the IEEE 30-bus OPF with renewable sources wind/solar/small-hydro power. Additionally, to qualitatively affirm the achieved solutions, the original true Pareto fronts have been compared to those obtained. Thereby, for performance assessment, various performance metrics in search and objective spaces have utilized such generational distance (GD), inverted generational distance (IGDX and IGDF), spacing metric, reciprocal Pareto sets proximity (rPSP), and reciprocal hypervolume (rHV). Thus, NSMRFO can relatively provide an accurate estimation shape with closer distance to the true PF compared to the multimodal multi-objective evolutionary approaches and some recent competitive algorithms. The NSMRFO impressive performance leads to handling challenging real-world problems in various engineering fields for future works.