How to Assess Different Algorithms Using Intuitionistic Fuzzy Logic

: Intuitionistic fuzzy logic is the main tool in the recently developed step-wise “cross-evaluation” procedure that aims at the assessment of different optimization algorithms. In this investigation, the procedure previously applied to compare the effectiveness of two or three algorithms has been signiﬁcantly upgraded to evaluate the performance of a set of four algorithms. For the ﬁrst time, the procedure applied here has been tested in the evaluation of the effectiveness of ge-netic algorithms (GAs), which are proven as very promising and successful optimization techniques for solving hard non-linear optimization tasks. As a case study exempliﬁed with the parameter iden-tiﬁcation of a S. cerevisiae fed-batch fermentation process model, the cross-evaluation procedure has been executed to compare four different types of GAs, and more speciﬁcally, multi-population genetic algorithms (MGAs), which differ in the order of application of the three genetic operators: Selection, crossover and mutation. The results obtained from the implementation of the upgraded intuitionistic fuzzy logic-based procedure for MGA performance assessment have been analyzed, and the standard MGA has been outlined as the fastest and most reliable one among the four investigated algorithms.


Introduction
Model parameter identification of non-linear fermentation processes (FP) is an important step for adequate modeling.Frequently, conventional optimization methods fail and do not lead to a satisfactory solution [1].Then, stochastic algorithms appear as a reliable alternative.Genetic algorithms (GAs) [2], based on Darwin's theory of evolution and "survival of the fittest" concept, are a stochastic technique for global optimization broadly applied to various complicated problems in different areas [3].GAs find the global optimal solution by simultaneously evaluating multiple points in the parameter search space.Properties of GAs like noise tolerance, easy interface interaction and hybridization make them a suitable and reliable tool to handle hard problems like FP parameter identification [3][4][5].
The standard simple genetic algorithm (SGA) [2] searches a global optimal solution applying the main genetic operators in the following order: Selection (denoted by S in the abbreviated algorithms' names), crossover (denoted by C in the abbreviated algorithms' names) and mutation (denoted by M in the abbreviated algorithms' names).SGA starts with a choice of chromosomes representing better possible solutions according to a preferred selection method.Afterwards, the crossover operator proceeds to form new offspring.Then mutation is applied with a determinate probability.SGA in which many sub-populations proceed independently from each other is known as a multi-population genetic algorithm (MGA).The standard MGA as originally presented in [2] is here denoted as MGA_SCM.Given that the underlying idea of GA is to imitate the mechanics of natural selection and genetics, and since our aim is to simulate processes occurring in the nature, we can note that the probability of mutation occurring before crossover is comparable to the probability of both processes taking place in the reverse order, or comparable to the probability of having the selection performed after crossover and mutation, regardless of their order.In this line of reasoning, eight modified SGAs and eight modified MGAs have been developed to improve the accuracy and the convergence time of the algorithms applied for parameter identification of a S. cerevisiae fed-batch cultivation process [5].All modified GAs follow the algorithm's logic, where in two out of eight algorithms, the mutation operator is omitted, while the rest six only differ from one another in the order of execution of the three basic genetic operators.
From both an engineering and a mathematical perspective, among the significant criteria for assessing the algorithms' performance quality, the most representative ones are the objective function value (J) and the time for the algorithm's convergence (CPU time).Intuitionistic fuzzy logic (IFL) acts as an alternative when the quality of different algorithms is being assessed for various purposes.For the very first time, in [6], the procedure to assess the algorithm quality performance (AAQP) implementing IFL has been proposed and successfully applied for comparison of the performance of standard SGA before and after the application of the procedure for purposeful model parameters genesis (PMPG) [7] as well as for three different values of generation gap (GGAP) that has been determined as the most sensitive GA parameter.Similarly, the AAQP procedure has been applied for the standard MGA, again before and after PMPG procedure and again at three values of GGAP [8].In [9], two kinds of SGA, namely SGA_SCM and SGA_MCS, have been assessed applying AAQP before and after PMPG procedure at three values of GGAP.Later on, in [10], AAQP was implemented aiming to derive intuitionistic fuzzy estimations in order to compare the standard SGA and standard MGA, again before and after the PMPG procedure and for three values of GGAP.The promising results obtained provoked searching for other applications of the elaborated AAQP procedure, as well as its further developing.Thus, in [11], SGA and MGA as well as their modifications have been examined, again before and after the PMPG procedure implementation, while the procedure for AAQP has been successfully applied to assess the performance of three modified SGAs, two modified MGAs, SGA towards MGA in their standard implementations, as well as to assess the performance of SGA at three values of GGAP.In addition, the step-wise AAQP procedure implementing IFL has been successfully applied to assess the quality performance of three kinds of MGA, namely MGA_SCM, MGA_CMS and MGA_MCS [12] and, respectively, of MGA_SCM, MGA_SMC and MGA_SC [13].As noted above, all of these applications of SGA and MGA and their modifications had been executed for the parameter identification of a fed-batch fermentation process model of S. cerevisiae.
The AAQP implementing IFL has instigated many improvements in both theoretical and practical aspects of the IFL.In [14], the authors reported on the orders of intuitionistic fuzzy numbers, while in [15], type-2 fuzzy sets and intuitionistic fuzzy sets were used to construct the fuzzy logic rules aiming to reflect the complexity and uncertainty of an urban rail system, thus to provide theoretical support for the urban rail transit security region decision and control.In [16], the authors developed a new operator for real-coded evolutionary algorithms, while in [17], a self-tuning linear adaptive-genetic algorithm for feature selection.A mathematical programming model and an enhanced simulated annealing algorithm for the school timetabling problem were elaborated in [18].In [19], the authors presented optimization under uncertainty using an intuitionistic fuzzy expected value model, while in [20], the authors used MGA for automatic generation of test cases.
The PMPG procedure itself attracts the interest of the researchers in the field of development of different optimization techniques as well as their application to a vast variety of objects.In recent years, PMPG has provoked the authors of [21] to develop and present an efficient hybrid of continuous ant colony optimization and weighted crossover GA, the authors of [22] to construct a new compound arithmetic crossover-based GA and the authors of [23] to elaborate an early warning system framework based on structured analytical techniques and a fuzzy expert system.From an application point of view, PMPG has given an impetus for structural optimization of a bespoke single-layer cable-net [24], for small enterprises' default prediction with empirical evidence [25], for non-deterministic polynomial problems solved by an improved hybrid ant colony optimization algorithm [26], for the evaluation of GA as learning system in rigid space interpretation [27] and to analyze mutation vectors of the selection mechanism in differential evolution [28].
In the present leg of this ongoing research, we are assessing the three modified MGA_SCM algorithms, in which the crossover and mutation operators-or only the crossover-are performed prior to the selection operator.In particular, our investigation focuses on the MGA_CMS, MGA_MCS and MGA_CS algorithm variants, since they have not been yet assessed by AAQP by now.All the three modifications are compared to the standard algorithm MGA_SCM.As a result, an upgraded procedure for the algorithms' performance assessment using IFL will be implemented for the first time for quality assessment of four algorithms, namely MGA_MCS, MGA_CMS and MGA_CS versus MGA_SCM.

Intuitionistic Fuzzy Estimations
IFL, proposed by Atanassov [29,30], provides another degree of freedom compared to fuzzy logic, rendering an account of both the membership and non-membership of an element to a set, thus allowing better handling of uncertainty.In IFL, the truth-value of the variable p is the ordered couple where . These values might be obtained using different formula considering the problem specificity.Following the previous author's investigations, in this research, the degrees of validity and non-validity are determined using the following formulas: and where m is the lower boundary of the so-called narrow range; u is the upper boundary of the so-called broad range; n is the upper boundary of the so-called narrow range.The broad range of the model parameters is based on referent data, while the narrow one is based on the preliminary evaluations of the algorithm.The manner of determination of the "narrow" range and the "broad" range is to be explained in the next section.
If there is a collected database with elements in the form p, M(p), N(p) , a number of new values for the variables can be obtained.In the case of 2 records in the database, 3 new values can be defined, e.g., optimistic (opt), average (aver) and pessimistic (pes).In the case of 3 records in the database, 5 new values can be defined, e.g., strong optimistic (strong_opt), optimistic (opt), average (aver), pessimistic (pes) and strong pessimistic (strong_pes), as shown in most of the previous implementations of the AAQP procedure [6,[8][9][10][11][12][13].
In the present study, the significant improvement of the proposed AAQP procedure is in its upgrade in such a way to allow the comparison of four different algorithms.For that purpose, the following seven new values are introduced in the case of four records in the database, namely: • optimistic • less optimistic • average • less pessimistic • pessimistic • strong pessimistic Therefore, for each p The second, third, fourth and fifth inequalities from the above chain of inequalities are obvious, while the validity of the first and the sixth inequalities follow from the inequality for every a 1 , a 2 , . . ., a n ∈ [0, 1], which is proved by induction.

Procedure for Assessment of Algorithms Quality Performance Implementing IFL
Introducing IFL in the assessment of the quality of GA performance requires that the degrees of validity and non-validity are defined within two distinct intervals of model parameter variation.One of these intervals, determined here as the "broad" range, is based on referent data, available in the literature [31].The other interval might be defined applying certain criteria for range shrinking, for instance, using some evaluations of model parameters based on the algorithms' performance in the past, or might be obtained using the PMPG procedure, developed by previous authors [7].The IFL-based AAQP procedure, used so far for the evaluation of a maximum of three GA/values of a GA parameter, is here upgraded to handle the assessment of the quality performance of four different algorithms: Step 1.For each of the investigated algorithms, performance of the number of runs in the "broad" and "narrow" ranges of model parameters.
Step 2. For each of the investigated algorithms, evaluation of the average values of J, CPU time and model parameters in the "broad" and "narrow" ranges.
Step 3.For each of the investigated algorithms, evaluation of the degrees of validity and non-validity, applying Formulas (2) and (3).
Step 5.For each of the model parameters, assignment of the aforementioned values of considered algorithms in "broad" and "narrow" ranges.Step 6. Assessment of the quality of performance of considered algorithms on the basis of the values, obtained in Step 5.
The stepwise procedure consistently performs all of the above listed six steps, with no cycles or skipping any of the steps.

Multipopulation Genetic Algorithms
MGA is more natural than SGA since within MGA, a number of populations (also subpopulations) evolve independently from one another over time.After a fixed number of generations, a period called isolation time, some of the individuals migrate between the subpopulations.The working principle of the standard MGA, namely MGA_SCM, can be found in detail in [2,32].
Three modifications of MGA are the focus of this investigation.In all of them, the selection is the last-performed operator.In two of modifications, crossover and mutation are performed interchangeably, while in the third modification, the mutation operator is skipped.Thus the resultant algorithms are denoted as MGA_CMS, MGA_MCS and MGA_CS.

Case Study of AAQP over Four MGA
The AAQP procedure is applied to parameter identification of S. cerevisiae fed-batch cultivation using four types of MGA altogether.The experimental data of S. cerevisiae fedbatch cultivation are obtained from the Institute of Technical Chemistry at the University of Hannover, Germany, and the detailed description of process conditions is given in [1].
The mathematical model of a fed-batch cultivation of S. cerevisiae is commonly described according to the mass balance [1], as follows: where X is the biomass concentration, [g/L]; S is the substrate (glucose) concentration, [g/L]; E is the ethanol concentration, [g/L]; O 2 is the dissolved oxygen concentration, [%]; O * 2 is the saturation concentration of dissolved oxygen, [%]; F is the rate of feeding, [l/h]; V is the volume of the bioreactor, [L]; k O 2 L a is the volumetric oxygen transfer coefficient, [1/h]; S in is the initial glucose concentration, [g/L]; µ 2S and µ 2E are the maximum growth rates, of substrate and ethanol, respectively [1/h]; k S and k E are the saturation constants, of substrate and ethanol, respectively [g/L]; Y ij is the yield coefficients, [g/g]; and all model parameters fulfill the non-zero division requirement.The functions are continuous and differentiable.
Mean square deviation between the experimental data Y = [X, S, E, O 2 ] and the model output Y* has been used as an optimization criterion: Following the S. cerevisiae fed-batch cultivation model ( 11)-( 16), 9 model parameters were estimated, while consequently applying MGA_SCM, MGA_CMS, MGA_MCS and MGA_CS.The parameter identification of the model was conducted in the Matlab 7 environment utilizing the Genetic Algorithm Toolbox [33].All computations were performed using a PC Intel Pentium 4 (2.4 GHz).
Again, it is worth noting that this is the first time the procedure for quality assessment of genetic algorithms' performance has been performed over a total number of four algorithms.Three modifications of MGA_SCM will be evaluated, in which the selection is the last operator.As such, MGA_CMS, MGA_MCS and MGA_CS are the object of investigation, together with the standard MGA_SCM.
The values of MGA parameters and operators have been thoroughly discussed and accepted according to previous authors' investigations [5,11].The quality of the performance of the considered MGA has been evaluated before after applying the PMPG procedure [7].For each of the algorithms considered here, 30 runs have been performed.Table 1 presents the objective functions and levels of performance of the four types considered here of MGA with respect to the PMPG procedure.
Table 2 presents the minimum, maximum and average values of each model parameter only for the top levels, according to Table 1.
Following the PMPG procedure, the new boundaries of the model parameters for the so-called "narrow" range are obtained in the following manner: The new minimum is lower but still closer to the TL lowest value, and the new maximum is higher but still closest to the TL highest value.Table 3 presents the "broad" boundaries according to [31] and new boundaries proposed following the PMPG procedure for the investigated MGA.In addition, IFL estimations, obtained by applying Formulas (2) and (3), are presented in Table 3.  Employing the AAQP procedure, outlined in Section 2.2, seven prognoses varying from strongly optimistic to strongly pessimistic are constructed for the performance of the investigated MGA, based on IFL estimations and Formulas (2)- (10).Table 4 presents the lower and upper boundaries for the identified parameters for each of the considered here four types of MGA.The four types of MGA investigated here were again applied for parameter identification of fed-batch cultivation of S. cerevisiae using the boundaries proposed in Table 3. Again, 30 runs of the algorithms were conducted to secure reliable results.In Table 5, we present the average values of J, CPU time and the nine model parameters when MGA_SCM, MGA_CMS, MGA_MCS and MGA_CS were applied, prior to and after the implementation of PMPG.Again, it is noteworthy that for all four types of MGA investigated here, running in the "narrow" range leads to an expected decrease of the CPU time, while preserving the high accuracy of the model in all of the cases.Additionally, specifically running MGA_CS in the "narrow" range reduces the CPU time by about 1.12 times compared to the "broad" range.The results obtained after the PMPG procedure score the highest accuracy, thus demonstrating the procedure's effectiveness across all four investigated types of MGA.
Table 6 presents the assigned estimations for all model parameters of the investigated MGA before and after PMPG application, with respect to the data in Table 4.
Table 6 shows no particularly strong_pes and pes prognoses, thus concluding all investigated MGA as highly reliable.The demonstrated leader is MGA_SCM before PMPG, featuring three strong_opt prognoses, five less_opt prognoses and one aver prognosis.This is the only algorithm with three strong_opt prognoses, followed by five others exhibiting two strong_opt prognoses each.Among them, there are three algorithms with very similar performances: MGA_CMS after PMPG with two strong_opt prognoses, six less_opt prognoses and one aver prognosis, and two other algorithms, MGA_SCM and MGA_MCS, both after PMPG, with less_pes instead of the aver prognosis.All four cases will be considered the most reliable ones since they exhibit the highest degree of accuracy (the lowest achieved values of J).Only one algorithm is distinguished from the rest by its evidently worse prognoses, and this is MGA_CS.If CPU time is considered alone, the only undisputable leader is MGA_SCM.Comparing MGA_SCM before and after PMPG, the appearance of MGA_SCM before PMPG is evaluated with one additional strong_opt prognosis versus one more less_opt prognosis in the MGA_SCM after PMPG, and one aver versus one less_pes prognosis, respectively.Therefore, it is up to the decision maker to choose either the higher evaluated but slightly slower MGA_SCM before PMPG, or the slightly lower evaluated, but overall fastest MGA_SCM after PMPG.
Thus, based on the implementation of the AAQP procedure for the assessment of four MGAs, an undisputable leader has been distinguished, namely MGA_SCM.As it has been just mentioned, it is the user's choice to decide which one of the algorithms to apply-the one before PMPG for a slightly higher accuracy at the expense of more CPU time, or the one after PMPG, for expeditiousness in the conditions of a slightly worse prognosis.
Thus, the application of the AAQP procedure presented here demonstrates its effectiveness as an appropriate and attractive tool in evaluating the performance of different algorithms.

Discussion
In this investigation, intuitionistic fuzzy logic has been applied as a main tool when assessing the quality of different algorithms performance for parameter estimation of a fed-batch fermentation process model.For that purpose, a step-wise procedure has been applied here for quality assessment of three modifications of the standard MGA_SCM, namely MGA_CMS, MGA_MCS and MGA_CS, in comparison to the standard MGA_SCM.These algorithms have been investigated for the purposes of parameter identification of S. cerevisiae fed-batch cultivation.Aiming to retain the promising results achieved in previous legs of this research, namely less convergence time at preserved model accuracy, the AAQP procedure overbuilds the results obtained after the application of the recently developed PMPG procedure.After implementing IFL, first to obtain intuitionistic fuzzy estimations of the model parameters and then to construct different prognoses (seven in this investigation), the results for the four investigated MGA variants have been analyzed.As a result of the application of both PMPG and AAQP procedures for the assessment of the four algorithms, MGA_SCM after PMPG has been distinguished as the fastest one although not the top ranked by the AAQP procedure, while MGA_SCM before PMPG has been estimated the top in terms of AAQP, although not the fastest one.Among the distinguished two "leaders", MGA_SCM in the "narrow" range is 1.12 times faster than MGA_SCM in the "broad" range, preserving (and even slightly improving) the highest achieved values of model accuracy.
The results of the presented investigation confirm the following conclusions: (1) All modifications of MGA considered here are effective, and while MGA_SCM has been chosen as the most reliable one, quite similar results obtained support the workability and effectiveness of all three modifications of MGA investigated here, with the selection operator applied after the mutation and crossover.
(2) The applied procedure for purposeful model parameter genesis is effective, which in all types of MGA considered here to an expected decrease in the CPU time while preserving the model accuracy.
(3) The AAQP procedure has been effectively upgraded here for the first time to handle the concurrent assessment of the performance quality of four algorithms.

Conclusions
In this study, the step-wise "cross-evaluation" procedure that aims at assessment of different optimization algorithms and previously applied to compare the effectiveness of two or three algorithms has been significantly upgraded to evaluate the performance of four algorithms.The "cross-evaluation" of four different MGA presented here confirms both the workability and effectiveness of all three modifications of MGA investigated here, the effectiveness of the applied procedure for purposeful model parameter genesis, as well as the workability of IFL estimations to assist in algorithms' quality performance assessment.The proposed cross-evaluation procedure has been executed to compare four different types of MGA, exemplified with the parameter identification of a S. cerevisiae fed-batch fermentation process model.The upgraded procedure presented here depends neither on the objects to be assessed, nor on the type of parameters to be evaluated.In general, the AAQP may be applied to n objects, however at the expense of research complexity.As such, the proposed procedure can be considered as a multipurpose one-an appropriate tool for reliable assessment of other optimization algorithms, for their respective parameters, as well as for other optimization tasks.As such, the AAQP procedure based on intuitionistic fuzzy logic for "cross-evaluation" of different algorithms/objects/parameters is a very promising tool from both mathematical and engineering perspectives.

Table 1 .
Performance of the four investigated types of MGA.
Note: TL is the top level, ML-middle level, LL-low level, LB-lower boundary, UB-upper boundary.

Table 2 .
Values of the model parameters for the top levels of investigated MGA.

Table 3 .
Boundaries of model parameters for investigated MGA.

Table 4 .
Evaluated prognoses for MGA performance.

Table 6 .
Estimations of model parameter.