This article is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

Interpretability and accuracy are two important features of fuzzy systems which are conflicting in their nature. One can be improved at the cost of the other and this situation is identified as “Interpretability-Accuracy Trade-Off”. To deal with this trade-off Multi-Objective Evolutionary Algorithms (MOEA) are frequently applied in the design of fuzzy systems. Several novel MOEA have been proposed and invented for this purpose, more specifically, Non-Dominated Sorting Genetic Algorithms (NSGA-II), Strength Pareto Evolutionary Algorithm 2 (SPEA2), Fuzzy Genetics-Based Machine Learning (FGBML), (2 + 2) Pareto Archived Evolutionary Strategy ((2 + 2) PAES), (2 + 2) Memetic- Pareto Archived Evolutionary Strategy ((2 + 2) M-PAES),

Interpretability [

Interpretability and accuracy are contradictory issues in the design of a fuzzy system. An increment in either feature can only be done at the cost of the other. This situation is called Interpretability-Accuracy (I-A) Trade-Off [

The identification of fuzzy systems from data samples for specific functions associates different tasks, like input selection, rule selection, rule generation, fuzzy partition, membership function tuning

To deal with Interpretability-Accuracy Trade-Off in Fuzzy Systems, Multi-Objective Evolutionary Algorithms (MOEAs) have been used, leading to the next generation of GFSs named Evolutionary Multi-Objective Fuzzy Systems (EMOFS) [

A recent discussion and review on the existing approaches of EMOFS has been given in [

The paper is divided into four sections. In

Evolutionary algorithms are stochastic optimization techniques, which simulate the concept of natural evolution. Evolutionary approaches consist of methodologies, genetic algorithms, evolutionary programming and evolutionary strategies. These techniques have been proven to be a robust and powerful search mechanism. In Evolutionary Multi-Objective Optimization, the objectives conflict with each other. These approaches are capable of tackling the problems of (1) large, complex and high dimensional search space, and (2) multiple conflicting objectives. In these techniques, no optimal, ideal and single solution can be derived, instead, a set of solutions are produced because the improvement in one objective leads to degradation in the remaining objectives. These solutions are called Pareto-Optimal Solutions. These Pareto Optimal Solutions in terms of objective function are called Pareto Front.

For example, two objective maximization problems can be formulated as: maximize _{1} (_{2} (

In

(

Multi-objective optimization problems are solved by using evolutionary algorithms, like Genetic Algorithms (GA) and result in a new area called EMO [

The conflicting nature of the objectives leads to many problems, like dominance resistance and speciation. In [

In the early 1990s, the work in the area of EMOFS was oriented towards the development of accurate fuzzy systems, with less concentration on interpretability. However, in the late 1990s, the interpretability became an important issue along with accuracy.

Interpretability and Accuracy Related Work in the 1990s.

Approaches developed | Focus | References |
---|---|---|

Maximization of the number of correctly classified patterns along with minimization of the number of rules and fuzzy rule selection represented as a combinatorial optimization problem | Accuracy improvement & complexity minimization | [ |

Association of rule weights in rules also called certainty factor | Accuracy improvement | [ |

Multiple consequents in a rule | Accuracy improvement | [ |

Use of fine fuzzy partition (over-fitting), multiple fuzzy grid approach | Accuracy Improvement | [ |

Applying independent membership functions | Accuracy & Scalability improvement | [ |

Use of multi-dimensional fuzzy membership function | Accuracy and scalability improvement | [ |

Use of tree-type fuzzy partitions | Accuracy & Scalability Improvement | [ |

Scalability and hierarchical fuzzy systems | Accuracy & Scalability Improvement | [ |

Use of don’t care conditions/ scalability improvement/input selection for each rule (rule wise input selection) | Complexity minimization | [ |

Many non-dominated fuzzy systems can be obtained along the trade-off surface (

The most commonly used MOEAs are NSGA-II [

Pareto front non-dominated fuzzy systems.

During the search for non-dominated fuzzy systems in an EMO environment, accuracy and complexity are the two important factors to be considered with the objectives of accuracy maximization and interpretability maximization (complexity minimization). Initially, an aggregation approach is used for this purpose.

After that the MOEAs are well adapted for this issue and it is represented as,

To obtain these objectives, several criteria, like number of selected fuzzy rules, number of correctly classified rules, tuning of membership, granularity of the uniform partition,

A two objective approach, considering maximization of the number of correctly classified training patterns and minimization of the number of selected fuzzy rules, is proposed in [

A multi-objective genetic procedure has been proposed in [

Interpretability-accuracy trade-off analysis was done in [

In [

A Pareto based multi-objective evolutionary approach has been proposed in [

A rule selection and a tuning of the membership functions of an initial set of candidate linguistic fuzzy rules were performed in [

A new post processing method is developed in [

A multi-objective genetic algorithm is proposed in [

A brief review on the state of the art on the use of multi-objective genetic algorithms to obtain the compact fuzzy rule-based systems under rule selection and parameter tuning has been done in [

A set of linguistic fuzzy rule-based systems with different trade-offs between accuracy and interpretability has been generated in [

A multi-objective evolutionary algorithm for tuning fuzzy rule-based systems has been proposed in [

A Mamdani fuzzy rule-based system with different good trade-offs between complexity and accuracy has been developed by using multi-objective evolutionary algorithm in [

Six different MOEA are used to obtain simpler and still accurate linguistic fuzzy models by performing rule selection and tuning of membership functions in [_{CC} [_{ACC}^{2}, NSGA-II_{A}, NSGA-II_{U}. These algorithms use two objectives, systems error and number of rules. A new post processing approach has been developed in [

A deep-tuned fuzzy rule-based classifier system (FRBCS) from examples has been designed in [

A method for generating single granularity based fuzzy classification rules and lateral tuning of membership functions has been proposed in [

A multi-objective evolutionary framework applied to regression problem has been proposed in [

A post processing approach is developed to reduce complexity of data-driven linguistic fuzzy models in [

In [

The Pareto optimum set of fuzzy systems with different I-A trade-off has been generated in [

The MOEA for searching the Pareto optimal fuzzy rules are discussed in [

An approach for an evolutionary training set selection in the framework of multi-objective evolutionary learning of Mamdani fuzzy rule-based systems (MFRBS) has been proposed in [

A MOEA has been proposed in [

Several approaches have used three objectives to deal with the interpretability and accuracy trade-off issue in developing fuzzy systems.

A three-objective approach is proposed in [

In [

A rule selection criterion for prescreening a candidate as fuzzy has been proposed in [

A multi-objective evolutionary algorithm (MOEA) is proposed in [

The NSGA-II algorithm has been used to create multiple Pareto optimal fuzzy systems in [

An approach for improving the interpretability of linguistic fuzzy rule-based systems has been proposed in [

A three-objective evolutionary algorithm has been proposed in [

A multi-objective genetic fuzzy system has been proposed in [

HILK (Highly Interpretable Linguistic Knowledge) in [

An index is proposed to preserve the semantic interpretability of linguistic fuzzy models in [

Three types of interpretability measures are introduced in [

A Pareto Multi-Objective Cooperative Co-Evolutionary Algorithm (PMOCCA) is proposed in [

A multi-objective fuzzy genetics-based machine-learning (GBML) algorithm is developed for fuzzy rule-based classifiers for examining the Interpretability-Accuracy Trade-Off in [

The capability of MOEA to find a variety of FRBS with different trade-offs between complexity and interpretability is called search ability. The improvement in the search ability of any MOEA is a critical research issue.

In [

An improvement in the search ability has been proposed in [

In MOGFS approaches, a set of non-dominated solutions has been generated, which makes it very difficult to choose one. A double cross-validation approach is used to do this task in [

The comparison between GBML and Genetic Rule Selection has been done in [

The search ability of MOEA in Pareto-optimal or near Pareto optimal fuzzy rule-based systems for classification problems has been discussed in [

The design of reliable classifiers by integrating multiple classifiers into a single one resulted in the development of ensemble classifiers. The generation of ensemble classifiers with high diversity using MOEA is an important research issue.

In [

Three objective-based multi-objective formulations of fuzzy rule selection have been discussed in [

Optimization of scalarizing functions and fine fuzzy partitions in EMOFRBS is a crucial research issue.

An approach to optimize scalarizing functions using EMO has been developed in [

The fine fuzzy partitions are used in the evolutionary multi-objective optimization for designing the fuzzy rule-based classifiers in [

User preferences [

An iterative fuzzy modeling has been performed in [

High dimensionality in fuzzy systems can be handled by using Evolutionary Multi-Objective Optimization (EMO), and it is an important research issue. High dimensional and large data sets lead to expansion in search space and affect the performance of evolutionary algorithms in the form of solution quality and convergence.

A MOEA is proposed for knowledge extraction from numerical data for high dimensional pattern classification problems with many continuous attributes in [

In [

A MOEA has been proposed in [

Explicit semantics (fuzzy sets, operators, inference engine) and implicit semantics (knowledge gathered by user) are compared using a co-intension approach called Semantic Co-intension. A novel index has been proposed in [

Context adaptation is the approach to develop context-free models for creating context adapted FRBS so as to increase the accuracy. In [

The EMO has been used in developing data mining approaches addressing different issues, like sub-group discovery, rule mining

A three-objective based multi-objective genetic rule selection has been introduced in [

In [

Using EMO in fuzzy systems, several applications have been developed.

A genetic fuzzy framework has been proposed for financial prediction in [

A fine tuned fuzzy logic controller for heating, ventilating and air conditioning systems has been developed using multi-objective evolutionary algorithms in [

The accuracy-complexity relationship has been analyzed in [

Several research issues have been identified in EMOFS while considering the issue of Interpretability-Accuracy Trade-Off. Some of these are listed below:

Formulation and quantification of interpretability along with the identification of its global definition [

Improvement in the interpretability of a system by selecting parameters like number of inputs, number of rules, rule length, fuzzy partition granularity, membership function separability, linguistic modifiers, linguistic hedges

Handling Interpretability-Accuracy (I-A) Trade–Off using EMO [

An increment in the number of objectives degrades the performance of any EMO algorithm. Hence, improvement of the performance of MOEA when the numbers of objectives are high is a big research line. It helps to deal with the High Dimensional Problems [

Integration of user preferences [

Handling large and multi-dimensional data sets [

Improvement in the search ability [

Generation of mechanisms for interpretable explanations for fuzzy reasoning and inference mechanism, quantification of explanation ability of FRBS [

The EMO algorithms applied in developing Fuzzy Systems need improvement in order to deal with problems like high dimensionality, exponentially populated solutions, Interpretability-Accuracy Trade-Off, quantification of interpretability and explanation ability of the fuzzy systems,

Evolutionary Multi-Objective Optimization (EMO) Algorithms used in Multi-objective Fuzzy Systems.

S. No. | EMO Used | References |
---|---|---|

1 | SPEA2 | [ |

2 | NSGA-II | [ |

3 | SPEA2_{ACC} |
[ |

4 | (2 + 2) PAES | [ |

5 | (2 + 2) M-PAES | [ |

6 | HILK EMO | [ |

7 | Fuzzy GBML | [ |

8 | PMOCCA | [ |

Many types of problems are also considered for fuzzy systems with their multi-objective development, listed in

Types of problems identified and discussed in the literature.

S. No. | Type of the problem identified | References |
---|---|---|

1 | Classification of Problems | [ |

2 | Regression | [ |

3 | Linguistic FRBS | [ |

4 | Function Approximation Problems | [ |

5 | TS Type FRBS | [ |

In the future, the authors are interested to develop efficient and robust MOEA, applicable for the development of accurate and interpretable fuzzy systems. Focus would also be dedicated to invent new indexes for measuring the interpretability of EMOFS and new EMO approaches for managing Interpretability-Accuracy Trade-off.