Stochastic Configuration Based Fuzzy Inference System with Interpretable Fuzzy Rules and Intelligence Search Process

Abstract

In this paper, a stochastic configuration based fuzzy inference system with interpretable fuzzy rules (SCFS-IFRs) is proposed to improve the interpretability and performance of the fuzzy inference system and determine autonomously an appropriate model structure. The proposed SCFS-IFR first accomplishes a fuzzy system through interpretable linguistic fuzzy rules (ILFRs), which endows the system with clear semantic interpretability. Meanwhile, using an incremental learning method based on stochastic configuration, the appropriate architecture of the system is determined by incremental generation of ILFRs under a supervision mechanism. In addition, the particle swarm optimization (PSO) algorithm, an intelligence search technique, is used in the incremental learning process of ILFRs to obtain better random parameters and improve approximation accuracy. The performance of SCFS-IFRs is verified by regression and classification benchmark datasets. Regression experiments show that the proposed SCFS-IFRs perform best on 10 of the 20 data sets, statistically significantly outperforming the other eight state-of-the-art algorithms. Classification experiments show that, compared with the other six fuzzy classifiers, SCFS-IFRs achieve higher classification accuracy and better interpretation with fewer rules.

1. Introduction

Neuro-fuzzy systems [1,2], which inherit the advantages of both fuzzy inference systems and neural networks, have attracted sustained attention. The fuzzy rules in neuro-fuzzy systems make the system interpretable and the trainable parameters make the precision of the system improvable via learning.

Interpretability is a very important research direction for neuro-fuzzy systems. Since interpretability is considered as the ability to express the meaning of a model in an understandable way, it is of great help to understand the function of the model and what is learned from it [3]. Although we have not yet found a way to measure interpretability mathematically, a fuzzy inference system with high interpretability should have comprehensible fuzzy partition and simple fuzzy rules. We should use fuzzy sets that are distinguishable and semantically clear and use as few fuzzy rules as possible to build an interpretable rule base. In [4], a fuzzy extreme learning machine (FELM) uses ILFRs to improve the interpretability of an extreme learning machine (ELM). ILFRs use several fuzzy sets to equally divide the input universe of discourse in order to better endow the fuzzy sets with semantic labels and generate random matrices to select features and to combine rules. ILFRs have been concerned in neuro-fuzzy modeling because of their advantages in interpretability. In [5,6], two deep fuzzy classifiers which adopt ILFRs are proposed. In order to improve the model transparency of the fuzzy broad learning system (FBLS), a Compact FBLS (CFBLS) using ILFRs has been improved in [7]. Although there are a large number of neuro-fuzzy systems which have been developed to improve interpretability and approximation accuracy, there is still room for further improvement. It is still a significant research problem to achieve good approximation accuracy with an appropriate number of fuzzy rules and determine the number of these fuzzy rules autonomously rather than through trial and error.

The most commonly used fuzzy system in the neural fuzzy model is the Takagi-Sugeno-Kang (TSK), which has been widely applied in many fields, such as regression [8], classification [9], model approximation [10], adaptive control [11], business [12], etc. Neural-fuzzy systems adopt the form of a neural network to represent the fuzzy system, and the training algorithm in the neural network is used to realize the extraction of fuzzy rules and the establishment of the fuzzy system without tedious manual intervention. The back propagation (BP) algorithm, a commonly used training algorithm in neural network, has been used for training neuro-fuzzy systems [13]. However, as datasets grow in size and dimension, the training algorithm of the neuro-fuzzy system has encountered many difficulties, such as training rate, overfitting, low interpretability, etc.

Because of the effectiveness and efficiency of randomized learning techniques, they have great potential in developing fast learning algorithms for neuro-fuzzy systems. In [14], the proposed Bernstein fuzzy system uses the ELM training system to simultaneously approximate smooth functions and their derivatives. In the generalized Bernstein fuzzy system, the premise parameters are randomly generated and the consequent parameters are determined by solving a linear optimization problem. In [15], a regularized extreme learning adaptive neuro-fuzzy inference system (R-ELANFIS) is designed to improve the generalization performance of a neuro-fuzzy system. In [16], a broad learning system (BLS) is proposed for fast learning of a large volume of data, which randomly assign the values of input parameters and expand the neurons in a broad manner without a retraining process. In [17], a TSK fuzzy system is combined with a broad learning system to establish a fuzzy broad learning system.

The scope of random parameters plays an important role in the training process of randomized learning approaches and has been proved to affect approximation capability [18]. A stochastic configuration network (SCN) is proposed in [19] to improve randomized learning techniques. SCN is built incrementally by assigning the random parameters under a supervisory mechanism and adaptively setting their range. The supervisory mechanism of SCN is helpful for ensuring universal approximation and good performance. Various SCN models have been developed over the past decade. In [20], a deep SCN is developed to randomly assign the wights of each layer under inequality constraints. In [21], a deep stacked SCN is established to automatically extract hidden units from data streams. In [22], the driving amount is introduced into the structure of SCN to improve the generalization approximation ability.

In this paper, a novel stochastic configuration based fuzzy inference system called SCFS-IFRs is proposed to improve the interpretability and performance of the fuzzy system, which uses an adaptive incremental learning approach based on the stochastic configuration algorithm to determine an appropriate architecture autonomously and ensure universal approximation and continuous performance improvement of SCFS-IFRs. The contribution of SCFS-IFRs can be summarized as follows:

(1)

For high interpretability, SCFS-IFRs implement a fuzzy system through ILFRs and determine the appropriate number of fuzzy rules by an adaptive incremental learning algorithm. ILFRs better endow fuzzy sets with semantic labels by equally dividing the input universe of discourse and the rule combination matrix is used to select activated fuzzy sets. They help to improve the semantic expression of fuzzy rules. Moreover, the incremental learning method of SCFS-IFRs which determines a suitable number of fuzzy rules, is also beneficial for improving interpretability and avoiding the problem of “rule explosion”.

(2)

For good performance, a kind of supervisory mechanism and an intelligence search process are to be considered in the adaptive incremental learning algorithm of SCFS-IFRs. The inequality constraints that the premise parameters of fuzzy rules need to satisfy are given, which can ensure that the performance of SCFS-IFRs will be improved along with the increase in rules. The PSO algorithm, an intelligence search technique, is used in the process of stochastic configuration to obtain better random parameters.

The rest of this article is organized as follows. In Section 2, we review the TSK fuzzy system and SCN. In Section 3, we elaborate the proposed SCFS-IFRs. In Section 4, numerical experiments verify the performance of the proposed SCFS-IFRs. Some concluding remarks are given in Section 5.

The abbreviations and variables used in this paper are listed in

Table 1. Table of abbreviations.

Abbreviations	Full Name
ILFRs	interpretable linguistic fuzzy rules
PSO	particle swarm optimization
ELM	extreme learning machine
FELM	fuzzy extreme learning machine
BLS	broad learning system
FBLS	fuzzy broad learning system
CFBLS	compact fuzzy broad learning system
TSK	Takagi-Sugeno-Kang
TSK-FC	TSK fuzzy classifier
BP	back propagation
R-ELANFIS	regularized extreme learning adaptive neuro-fuzzy inference system
SCN	stochastic configuration network
RVFLN	random vector functional link neural network
HPFNN	hierarchical polynomial-based fuzzy neural networks
BL-DFIS	broad learning based dynamic fuzzy inference system
SCFS-IFRs	stochastic configuration based fuzzy inference system with interpretable fuzzy rules

and

Table 2. Table of variables.

Variables	Interpretation
$u_k$	center of the $k$th fuzzy set, predefined constant
${\sigma }_L$	width of fuzzy set in the $L$th fuzzy rule, probabilistic variables
${\mathit{C}}_{\mathit{L}}$	rule combination matrix of the $L$th rule, probabilistic variables
${\mathit{a}}_{\mathit{L}}$	weight parameters of the $L$th rule
$e_L$	residual error of system with $L$ rule
$\epsilon$	predefined tolerance level
$r$	real variable with adaptively values between 0 and 1
$\left\{{\mu }_L\right\}$	real sequence with a limit of zero and adaptively values
$\lambda$	predefined regular coefficient
$\mathit{I}$	identity matrix
$L_{max}$	predefined maximum number of rules
$K$	predefined number of fuzzy sets for each feature
$b_{\sigma }$	predefined upper bound of the width parameter ${\sigma }_L$
$\left({\mathit{V}}_C^t,V_{\sigma }^t\right)$	velocity vectors of population of PSO
$w$	predefined inertia wight factor of PSO
$c_1,c_2$	predefined positive acceleration constants of PSO
$r_1,r_2$	random factors of PSO, probabilistic variables
$n_{max}$	predefined the maximum number of iterations of PSO
$T$	predefined the population size of PSO

.

2. Preliminaries

2.1. Takagi-Sugeno-Kang Fuzzy Inference System

The Takagi-Sugeno-Kang (TSK) fuzzy system proposed in [23], as one of the most important fuzzy systems, has been applied extensively in many different fields [24]. The consequent part of the fuzzy rule is the function of the input variables rather than the fuzzy set. Specifically, given an input vector $x={\left(x_1,x_2\cdots ,x_m\right)}^T$, the typical fuzzy rules are of the form:

$$R_l:\mathrm{If}{x}_1\mathrm{is}{A}_{l1}\mathrm{and}\cdots \mathrm{and}{x}_m\mathrm{is}{A}_{lm},\mathrm{then}y=f_l\left(x\right),l=1,2,\cdots ,L$$

(1)

where $A_{lj}$ is an antecedent fuzzy set of variable $x_j$ for the rule $R_l$, $f_l\left(x\right)$ is the consequent function of the rule $R_l$ and $L$ is the number of fuzzy rules.

Although the popular first-order TSK fuzzy system sets the consequent function $f_l\left(x\right)$ as linear function of the input variables, $f_l\left(x\right)$ can be any form function. The linear combination of a subset of the Chebyshev polynomial has been adopted by the rule consequent for improving the non-linear expression ability of TSK fuzzy system in [25], which is of the form.

$$f_l\left(x\right)=\mathsf{\varphi }{\left(x\right)}^T{\mathit{a}}_l$$

(2)

where $\mathsf{\varphi }\left(x\right)={\left(1,T_1\left(x_1\right),\cdots ,T_1\left(x_m\right),T_2\left(x_1\right),\cdots ,T_n\left(x_m\right)\right)}^T$ is the expanded input vector of form, ${\mathit{a}}_l\in {ℝ}^{nm+1}$ is the weight of the linear combination, $T_{k+1}\left(x_j\right)$ is the Chebyshev polynomial of the form $T_{k+1}\left(x_j\right)=2x_j{T}_k\left(x_j\right)-T_{k-1}\left(x_j\right)$ with $T_0\left(x_j\right)=1,T_1\left(x_j\right)=x_j$ and $n$ is the highest degree of the used Chebyshev polynomial.

For the $l$th fuzzy rule $R_l$, the activation level is:

$${\tau }_l\left(x\right)={\displaystyle \prod _{j=1}^m}{\mu }_{A_{lj}}\left(x_j\right)$$

(3)

where ${\mu }_{A_{lj}}\left(x_j\right)$ is the membership function of $A_{lj}$.

As pointed out in [26,27,28], the output system $\left(1\right)$ can be calculated as

$$y={\displaystyle \sum _{l=1}^L}{\tau }_l\left(x\right)f_l\left(x\right)={\displaystyle \sum _{l=1}^L}{\tau }_l\left(x\right)\mathsf{\varphi }{\left(x\right)}^T{\mathit{a}}_l$$

(4)

In the above TSK fuzzy system, the parameters of membership function ${\mu }_{A_{lj}}$ and the consequent weight ${\mathit{a}}_l$ need to be determined. Traditional gradient descent algorithms can be used to identify these parameters, but is usually time-consuming with a great deal of input data. Many learning algorithms [29,30,31] have been proposed for accelerating the parameter identification procedure and developing the approximation performance. For example, the premise parameters can be generated randomly and the consequent parameters can be quickly calculated using a pseudoinverse.

2.2. Stochastic Configuration Network

The stochastic configuration network (SCN) [19], which develops a randomized method, generates random parameters under a supervisory mechanism. It can guarantee the approximation capability and determine an appropriate architecture.

For a giving target function $f:{ℝ}^m\to ℝ$, an SCN with $L-1$ hidden nodes is denoted as

$$y={\displaystyle \sum _{j=1}^{L-1}}{\beta }_{j}g\left(w_j^{T}x+b_j\right)$$

(5)

where $g\left(w_j^T\mathrm{x}+b_j\right)$ is the activation function with random parameters $w_j\in {ℝ}^m$ and $b_j\in ℝ$, ${\beta }_j\in ℝ$ is the output weight of the hidden node. The residual error is denoted as

$$e_{L-1}=f-{\displaystyle \sum _{j=1}^{L-1}}{\beta }_{j}g\left(w_j^{T}x+b_j\right)$$

(6)

If $e_{L-1}$ is bigger than a predefined error tolerance $\epsilon$, the SCN framework incrementally adds hidden nodes until the residual error reaches $\epsilon$. The supervisory mechanism of an inequality constraint is of the form

$$〈e_{L-1},g_L〉{}^2\ge \left(1-r-{\mu }_L\right)\Vert e_{L-1}{}^2\Vert$$

(7)

where $r\in \left(0,1\right)$ and $\left\{{\mu }_L\right\}$ is a sequence with $\underset{L\to +\infty }{\lim }{\mu }_L=0$ and $0<{\mu }_L\le \left(1-r\right)$ and $g_L\left(x\right)=g\left({\mathit{w}}_L^{T}x+b_L\right)$ is the random basis function of the new hidden node.

The output parameters ${\beta }_j,j=1,2,\cdots ,L$ are calculated by

$$\left[{\beta }_1^{*},{\beta }_2^{*},\cdots ,{\beta }_L^{*}\right]=\arg \underset{\beta }{\min }\Vert f-{\displaystyle {\sum }_{j=1}^L}{\beta }_j{g}_j\Vert {}^2$$

(8)

The leading model $y={\displaystyle {\sum }_{j=1}^L}{\beta }_j{g}_j$ shows continuous performance improvement and the universal approximation of SCN have been proved in [19].

Although the existence of random parameters satisfying the constraint conditions $\left(7\right)$ has been proved theoretically, how to find better parameters among these satisfying constraint conditions is still a meaningful problem. In this paper, the PSO algorithm is applied to find better parameters to further improve the performance of the system.

3. Stochastic Configuration Based Fuzzy Inference System with Interpretable Fuzzy Rules and Intelligence Search Process

The SCFS-IFRs model establishes a TSK fuzzy system of form (1) by adopting the following new method.

(1)

The ILFRs with the rule combination matrix are used to construct the TSK fuzzy system to guarantee model interpretability and avoid the “rule explosion” problem.

(2)

An adaptive incremental learning algorithm based on stochastic configuration is proposed to train the SCFS-IFR model and fix the suitable number of fuzzy rules.

(3)

The PSO algorithm is adopted to search for better parameters to improve the performance of the system in the process of stochastic configuration.

This section elaborates the proposed SCFS-IFRs. Firstly, the general structure of the SCFS-IFRs is introduced and then their adaptive learning algorithm is presented.

3.1. General Structure of SCFS-IFRs

The details of SCFS-IFRs shown in Figure 1 are given as follows. Given a dataset with inputs $\mathit{X}={\left(x_1,x_2,\cdots ,x_N\right)}^T$ and outputs $\mathit{Y}={\left(y_1,y_2,\cdots ,y_N\right)}^T$, where $x_i=\left(x_{i1},x_{i2},\cdots ,x_{im}\right)$ and $y_i=\left(y_{i1},y_{i2},\cdots ,y_{it}\right),j=1,2,\cdots ,N$ and suppose all input features are normalized to $\left[0,1\right]$. We equally divide $\left(0,1\right)$ into $K-1$ parts, and correspondingly, each feature is transformed into $K$ fuzzy sets ${\left\{A_k\right\}}_{k=1}^K$ which are defined as follows

$${\mu }_{A_k}\left(x_{ij}\right)=\exp \left(-\frac{{\left(x_{ij}-u_k\right)}^2}{2{\sigma }^2}\right),k=1,2,\cdots ,K,j=1,2,\cdots ,m$$

(9)

where ${\mu }_{A_k}\left(\cdot \right)$ is the Gaussian membership function of the $k$th fuzzy set, $x_{ij}$ is the $j$th dimension of input feature $x_i$, $u_k=\frac{k-1}{K}$ is the center of fuzzy set $A_k$ and $\sigma$ is the randomly generated width. Here, the $K$ fuzzy sets can be assigned $K$ explicit linguistic labels. For instance, when $K=5$, the five fuzzy sets can be assigned 5 linguistic labels: very low, low, medium, high, and very high. Compared with the random allocation of center parameters, the traditional equidistant partition method makes the fuzzy set more interpretable but may lead to “rule explosion”.

Motivated by [6], the rule combination matrix $\mathit{C}\in {\left\{0,1\right\}}^{m\times K\times L}$ is used to determine which fuzzy sets of the input features are used to generate the $L$ fuzzy rules, where $C\left(j,k,l\right)=1$ represents the $k$th fuzzy set of the $j$th feature in which the $l$th fuzzy rule will be activated; conversely, it will not be activated. To ensure that at least one fuzzy set is activated for each feature in each rule, we require that ${\displaystyle {\sum }_{k=1}^K}C\left(j,k,l\right)\ge 1,j=1,2,\cdots ,m,l=1,2,\cdots ,L$. $B_{lj}$ denotes the union of all activated fuzzy sets of the $j$th feature in the $l$th fuzzy rule, i.e.,

$$B_{lj}={\displaystyle {\bigcup }_{C\left(j,k,l\right)=1}}{A}_k,j=1,2,\cdots ,m$$

(10)

The SCFS-IFRs with $L$ ILFRs take the following form:

$$\begin{gathered} R_l:\mathrm{if}{x}_{i1}\mathrm{is}{B}_{l1}\mathrm{and}{x}_{i2}\mathrm{is}{B}_{l2}\mathrm{and}\cdots \mathrm{and}{x}_{im}\mathrm{is}{B}_{lm},\mathrm{then}y=\mathsf{\varphi }{\left(x_{\mathrm{i}}\right)}^T{a}_l, \\ l=1,2,\cdots ,L \end{gathered}$$

(11)

We can adopt the operator of algebraic sum $s\left(a,b\right)=a+b-ab$ to calculate the membership function of $B_{lj}$ as

$${\mu }_{B_{lj}}\left(x_{ij}\right)=1-{{\displaystyle \prod }}_{k=1}^K\left(1-C\left(j,k,l\right){\mu }_{A_k}\left(x_{ij}\right)\right),j=1,2,\cdots ,m,l=1,2,\cdots ,L$$

(12)

Therefore, the output and residual error of the SCFS-IFRs $\left(11\right)$ on dataset $\left\{\mathit{X},\mathit{Y}\right\}$ can be calculated as

$${\mathit{S}}_L={\displaystyle \sum _{l=1}^L}\left(\begin{array}{c}{\tau }_l\left(x_1\right)\mathsf{\varphi }{\left(x_1\right)}^T\\ ⋮\\ {\tau }_l\left(x_N\right)\mathsf{\varphi }{\left(x_N\right)}^T\end{array}\right){\mathit{a}}_l={\displaystyle \sum _{l=1}^L}{\mathit{\eta }}_l{\mathit{a}}_l$$

(13)

$$e_L=\mathit{Y}-{\mathit{S}}_L$$

(14)

where ${\tau }_l\left(x_i\right)={{\displaystyle \prod }}_{j=1}^m{B}_{lj}\left(x_{ij}\right)={{\displaystyle \prod }}_{j=1}^m\left(1-{{\displaystyle \prod }}_{k=1}^K\left(1-C\left(j,k,l\right){\mu }_{A_k}\left(x_{ij}\right)\right)\right)$ is the activation level of the $l$th fuzzy rule, $a_l$ is the weight of the linear combination of the expanded input vector $\mathsf{\varphi }\left(x_i\right)$ and denotes

$${\eta }_l=\left(\begin{array}{c}{\tau }_l\left(x_1\right)\mathsf{\varphi }{\left(x_1\right)}^T\\ ⋮\\ {\tau }_l\left(x_{\mathrm{N}}\right)\mathsf{\varphi }{\left(x_{\mathrm{N}}\right)}^T\end{array}\right)$$

(15)

Remark 1.

A SCFS-IFRs model is implemented in the framework of the TSK fuzzy system. The semantically clear fuzzy sets generated by equidistant partition and the rule combination matrix used to generate fuzzy rules contribute to the high interpretability of SCFS-IFRs.

For instance, let us have $L=2$ fuzzy rules, $m=2$ features for each input sample ${\mathrm{x}}_i$, $K=5$ fuzzy sets with linguistic labels: very low, low, medium, high, and very high. Given

$$C\left(:,:,1\right)=\left(\begin{array}{ccc}\begin{array}{cc}\begin{array}{cc}\begin{array}{c}1\\ \begin{array}{c}0\\ 0\end{array}\end{array}& \begin{array}{c}1\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}& \begin{array}{c}0\\ \begin{array}{c}1\\ 0\end{array}\end{array}\end{array}& \begin{array}{c}0\\ \begin{array}{c}0\\ 1\end{array}\end{array}& \begin{array}{c}0\\ \begin{array}{c}0\\ 1\end{array}\end{array}\end{array}\right), C\left(:,:,2\right)=\left(\begin{array}{ccc}\begin{array}{cc}\begin{array}{cc}\begin{array}{c}0\\ \begin{array}{c}1\\ 0\end{array}\end{array}& \begin{array}{c}0\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}& \begin{array}{c}0\\ \begin{array}{c}0\\ 1\end{array}\end{array}\end{array}& \begin{array}{c}0\\ \begin{array}{c}0\\ 1\end{array}\end{array}& \begin{array}{c}1\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\right)$$

The two ILFRs can be expressed as

$$\begin{gathered} {\mathrm{R}}_1:{\mathrm{if}\mathrm{x}}_{\mathrm{i}1}{\mathrm{is}\mathrm{very}\mathrm{low}\mathrm{or}\mathrm{low}\mathrm{and}\mathrm{x}}_{\mathrm{i}2}{\mathrm{is}\mathrm{medium}\mathrm{and}\mathrm{x}}_{\mathrm{i}3}\mathrm{is}\mathrm{high}\mathrm{or}\mathrm{very}\mathrm{high},\mathrm{then}\mathrm{y}=\mathsf{\varphi }{\left(x_{\mathrm{i}}\right)}^{\mathrm{T}}{a}_1 \\ {\mathrm{R}}_2:{\mathrm{if}\mathrm{x}}_{\mathrm{i}1}{\mathrm{is}\mathrm{very}\mathrm{high}\mathrm{and}\mathrm{x}}_{\mathrm{i}2}{\mathrm{is}\mathrm{very}\mathrm{low}\mathrm{and}\mathrm{x}}_{\mathrm{i}3}\mathrm{is}\mathrm{medium}\mathrm{or}\mathrm{high},\mathrm{then}\mathrm{y}=\mathsf{\varphi }{\left(x_{\mathrm{i}}\right)}^{\mathrm{T}}{a}_2 \end{gathered}$$

In addition, the rule combination matrix defined in this paper allows multiple fuzzy sets corresponding to each feature in a fuzzy rule to be activated, which improves the semantic expression of fuzzy rules and makes it possible to express more complex relations with fewer rules.

3.2. Adaptive Incremental Learning for SCFS-IFRs

We propose an adaptive incremental learning algorithm to train the proposed SCFS-IFRs, which can be helpful in determining an appropriate model structure autonomously and ensure the approximation capability of SCFS-IFRs. In order to further improve the approximation accuracy, the PSO algorithm is adopted to search for better random parameters in the process of stochastic configuration.

The adaptive incremental learning algorithm starts with an initial system, which can be either an empty system or a system with $L_0$ rules generated by other traditional methods. Given training dataset $\left\{\mathit{X},\mathit{Y}\right\}$, new fuzzy rules are increased incrementally in the light of a supervisory mechanism of stochastic configuration for improving the approximation performance until a predefined tolerance level $\epsilon$ is met.

Suppose that there is SCFS-IFRs ${\mathit{S}}_{L-1}$ with $L-1$ fuzzy rules of the form $\left(11\right)$. The output and residual error of ${\mathit{S}}_{L-1}$ on dataset $\left\{\mathit{X},\mathit{Y}\right\}$ are expressed in the form

$${\mathit{S}}_{L-1}={\displaystyle \sum _{l=1}^{L-1}}{\mathit{\eta }}_l{\mathit{a}}_l,{\mathit{e}}_{L-1}=\mathit{Y}-{\mathit{S}}_{L-1}$$

If $\Vert {\mathit{e}}_{L-1}\Vert$ does not reach the predefined tolerance level $\epsilon$, i.e., $\Vert {\mathit{e}}_{L-1}\Vert \ge \epsilon$, a new fuzzy rule $R_L$ should be generated to construct the new SCFS-IFRs ${\mathit{S}}_L$ with $L$ fuzzy rules. There are three parameters that should be determined for generating rule $R_L$: the rule combination matrix (denoted by ${\mathit{C}}_L=\mathit{C}\left(:,:,L\right)$), the width of gaussian membership function (denoted by ${\sigma }_L$) and the weight of the linear combination in the consequent part (denoted by $a_L$).

The premise parameters ${\mathit{C}}_L$ and ${\sigma }_L$ are randomly produced to satisfy the following inequalities

$$\Vert {\mathit{\eta }}_L{\left({\mathit{\eta }}_L^T{\mathit{\eta }}_L+\lambda {\mathit{I}}_M\right)}^{-1}{\left({\mathit{\eta }}_L^T{\mathit{e}}_{L-1}\right)\Vert }^2\ge \left(1-r-{\mu }_L\right)\Vert {\mathit{e}}_{L-1}\Vert {}^2$$

(16)

and the consequent parameter ${\mathit{a}}_L$ can be evaluated by

$${\mathit{a}}_L={\left({\mathit{\eta }}_L^T{\mathit{\eta }}_L+\lambda \mathit{I}\right)}^{-1}\left({\mathit{\eta }}_L^T{\mathit{e}}_{L-1}\right)$$

(17)

where $\lambda$ is the regular coefficient, $\mathit{I}$ is the identity matrix, ${\mathit{\eta }}_L$ is of form $\left(15\right)$ and $r\in \left(0,1\right)$ and $\left\{{\mu }_L\right\}$ is a sequence with $\underset{L\to \infty }{\lim }{\mu }_L=0$ and $0<{\mu }_L\le \left(1-r\right)$.

The output and the residual error of the ${\mathit{S}}_L$ can be represented as

$${\mathit{S}}_L={\displaystyle \sum _{l=1}^L}{\mathit{\eta }}_l{\mathit{a}}_l,{\mathit{e}}_L=\mathit{Y}-{\mathit{S}}_L={\mathit{e}}_{L-1}-{\mathit{\eta }}_L{\mathit{a}}_L$$

and the improvement brought about from the new rule $R_L$ can be calculated as

$$P_L=\Vert {\mathit{e}}_L\Vert {}^2-\Vert {\mathit{e}}_{L-1}\Vert {}^2=\Vert {\mathit{e}}_{L-1}-{\mathit{\eta }}_L{\mathit{a}}_L\Vert {}^2-\Vert {\mathit{e}}_{L-1}\Vert {}^2=-\Vert {\mathit{\eta }}_L{\left({\mathit{\eta }}_L^T{\mathit{\eta }}_L+\lambda {\mathit{I}}_M\right)}^{-1}{\left({\mathit{\eta }}_L^T{\mathit{e}}_{L-1}\right)\Vert }^2.$$

(18)

The improvement $P_L$ provides the target fucntion for the choice of parameters ${\mathit{C}}_L$ and ${\sigma }_L$. There may be many values of parameters ${\mathit{C}}_L$ and ${\sigma }_L$ that satisfy the inequality constraint $\left(16\right)$. We adopt the PSO algorithm to search for better random parameters ${\mathit{C}}_L$ and ${\sigma }_L$ which satisfy the inequality constraint $\left(16\right)$ and have smaller values of improvement $P_L$, in order to improve the system performance.

In the process of stochastic configuration using the PSO algorithm, firstly $T$ pairs of parameters ${\left\{\left({\mathit{C}}_L^t,{\sigma }_L^t\right)\right\}}_{t=1}^T$ within ${\left\{0,1\right\}}^{m\times K}\times \left(0,b_{\sigma }\right)$. are randomly initialized as the position of initial population; correspondingly, $T$ pairs of parameters ${\left\{\left({\mathit{V}}_C^t,V_{\sigma }^t\right)\right\}}_{t=1}^T$ within ${\left(-1,1\right)}^{m\times K}\times \left(-b_{\sigma },b_{\sigma }\right)$ are randomly initialized as the velocity vectors of the initial population. The fitness value $P_L^t$ of particle $t$ can be calculated by Equation $\left(18\right)$.

At each iteration, the positions and velocity vectors can be updated as

$$\begin{gathered} {\left({\mathit{V}}_C^t,V_{\sigma }^t\right)}_{new}=w{\left({\mathit{V}}_C^t,V_{\sigma }^t\right)}_{old}+c_1{r}_1\left({\left({\mathit{C}}_L^t,{\sigma }_L^t\right)}_p-{\left({\mathit{C}}_L^t,{\sigma }_L^t\right)}_{old}\right)+ \\ {c}_2{r}_2\left(\left({\mathit{C}}_L^g,{\sigma }_L^g\right)-{\left({\mathit{C}}_L^t,{\sigma }_L^t\right)}_{old}\right), \end{gathered}$$

(19)

$${\left({\mathit{C}}_L^t,{\sigma }_L^t\right)}_{temp}={\left({\mathit{C}}_L^t,{\sigma }_L^t\right)}_{old}+{\left({\mathit{V}}_C^t,V_{\sigma }^t\right)}_{new}$$

(20)

where $w$ is the inertia weight factor, $c_1$ and $c_2$ are positive acceleration constants, $r_1$ and $r_2$ are random factors sampled from a uniform distribution $U\left(0,1\right)$, ${\left({\mathit{C}}_L^t,{\sigma }_L^t\right)}_p$ are $\left({\mathit{C}}_L^g,{\sigma }_L^g\right)$ represent the best position in the history of particle $t$ and the whole population, respectively. In this paper, we set the parameters of PSO as $w=0.9,c_1=c_2=2$. Through $n_{max}$ iterations of PSO, we can obtain better parameters to generate new fuzzy rules to improve the performance of the system.

In this way, we gradually generate new fuzzy rules one by one and add them to the system until the residual error reaches the predefined error tolerance $\epsilon$ or the number of rules exceeds the given maximum rules number $L_{max}$. The adaptive incremental learning algorithm can be summarized as follows.

Adaptive incremental learning algorithm of SCFS-IFRs:

Given dataset with inputs $\mathit{X}={\left(x_1,x_2,\cdots ,x_N\right)}^T$ and outputs $\mathit{Y}={\left(y_1,y_2,\cdots ,y_N\right)}^T$, the predefined maximum number of rules $L_{max}$, the error tolerance $\epsilon$, the population size $T$, the maximum number of iterations of the PSO $n_{max}$, the number of fuzzy sets $K$ for each feature, the upper bound $b_{\sigma }$ of the width parameter and the regularization parameter $\lambda$.

• Initialize $\mathit{e}=\mathit{Y},L=0$, $0<r<1$;
• Set center of Gaussian membership function $u_k=\frac{k-1}{K},k=1,2,\cdots ,K$;
• While $L<L_{max}$ and $\Vert \mathit{e}\Vert >\epsilon$, Do
• Set ${\mu }_L=\left(1-r\right)/\left(L+1\right)$;
• Randomly initialize $T$ particles with position and velocity vectors as ${\left\{\left({\mathit{C}}_L^t,{\sigma }_L^t\right)\right\}}_{t=1}^T$ and ${\left\{\left({\mathit{V}}_C^t,V_{\sigma }^t\right)\right\}}_{t=1}^T$
• For $i=1$ to $n_{max}$ do
• Calculate the fitness value ${\left\{P_L^t\right\}}_{t=1}^T$ according to Equation $\left(18\right)$;
• Update the positions ${\left\{\left({\mathit{C}}_L^t,{\sigma }_L^t\right)\right\}}_{t=1}^T$ and velocity vectors ${\left\{\left({\mathit{V}}_C^t,V_{\sigma }^t\right)\right\}}_{t=1}^T$ using Equations (19) and (20);
• End for
• Set the best position as $\left({\mathit{C}}_L^g,{\sigma }_L^g\right)$ as $\left({\mathit{C}}_L^{},{\sigma }_L^{}\right)$;
• Calculate $P_L$ according to Equation $\left(18\right)$;
• if $P_L\le -r\Vert {\mathit{e}}_{L-1}\Vert {}^2$ Calculate ${\mathit{\eta }}_L\mathrm{and}{\mathit{a}}_L$ according to Equations $\left(15\right)$ and $\left(17\right),$ respectively;
• Else randomly take $\tau \in \left(0,r\right)$, renew $r:=r+\tau ,$ return to Step 4;
• End if
• Update $\mathit{e}≔\mathit{e}-{\mathit{\eta }}_L{\mathit{a}}_L$;
• End while
• Returen ${\left\{\left({\mathit{C}}_l^{},{\sigma }_l^{}\right),{\mathit{a}}_l\right\}}_{l=1}^L$.

In this way, the adaptive incremental learning algorithm determines the structure and parameters of SCFS-IFRs. The random parameters $\left({\mathit{C}}_L^{},{\sigma }_L^{}\right)$ are adaptively determined according to the inequality constraint $\left(16\right)$ and the appropriate architecture is determined. In the adaptive incremental learning algorithm, while the random parameters $\left({\mathit{C}}_L^{},{\sigma }_L^{}\right)$ are optimized, the number of fuzzy sets $K$ for each feature, the regularization parameter $\lambda$, and the degree of Chebyshev polynomial $n$ are predefined and do not need to be optimized.

Remark 2.

The approximation capability of SCFS-IFRs can be analyzed as follows. If the random parameters$\left({\mathit{C}}_L^{},{\sigma }_L^{}\right)$of the new fuzzy rule satisfies the inequality constraint$\left(16\right)$and the consequent parameter${\mathit{a}}_L$is evaluated by Equation (17), we can find that$\Vert {\mathit{e}}_L\Vert {}^2-\Vert {\mathit{e}}_{L-1}\Vert {}^2\le -\left(1-r-{\mu }_L\right)\Vert {\mathit{e}}_{L-1}\Vert {}^2$according to Equation (18). This means that$\Vert {\mathit{e}}_L\Vert {}^2\le \left(r+{\mu }_L\right)\Vert {\mathit{e}}_{L-1}\Vert {}^2$. Since$\underset{L\to \infty }{\lim }{\mu }_L=0$, it is obvious that$\underset{L\to \infty }{\lim }\Vert {\mathit{e}}_L\Vert {}^2=0$, which proves the approximation capability of SCFS-IFRs.

4. Performance Evaluation

In this section, the performance of the proposed SCFS-IFRs is investigated. The proposed SCFS-IFRs is compared with some representative models such as RVFLN [32], ELM [33], RVFL-RS [34], HPFNN [35], SCN [19], SCBLS [36], BLS [16], FBLS [7], TSK-FC [37], L2TSK-FC [37], HID-TSK [6], CFBLS [7], FELM [4] and BL-DFIS [38] through numerical experiments on 20 regression datasets and 10 classification datasets.

For a fair comparison, we use the same training strategies for SCFS-IFRs as [6,7], i.e., a ten-fold cross validation. We repeat every experiment 30 times and report the average result. The optimal settings of parameters are obtained by grid search method. The number of fuzzy sets $K$ for each feature is chosen from {3,5,7}, while the regularization parameter $\lambda$ is chosen from {${10}^{-3},{10}^{-2},{10}^{-1}$}, and the degree of Chebyshev polynomial $n$ is chosen form {1,2,3,4}. The predefined maximum number of rules $L_{max}$ is set as 25, the population size $T$ is set as 50 and the maximum number of iterations of the PSO $n_{max}$ is set as 10.

4.1. Regression

20 regression datasets which are selected from the UCI [39] and KEEL[40] repositories are employed to evaluate the performance of the SCFS-IFRs in regression tasks. The relevant details of the 20 datasets are summarized in

Table 3. Details of 20 regression datasets from UCI and KEEL.

Dataset	No. of Features	No. of Data	Dataset	No. of Features	No. of Data
Airfoil	5	1503	Baseball	16	337
Concrete	8	1030	Izmir	9	1461
Auto MPG	7	392	Ankara	9	1609
Yacht	6	308	Electrical	4	1056
QSAR	6	908	Computer	21	8192
Treasury	15	1049	Laser	4	993
Abalone	8	4177	Prices	13	506
Categorical	7	4052	Cool	8	768
Auto MPG6	5	392	Wine	11	1599
Energy	6	365	Heat	8	786

.

Table 4. Comparisons of testing RMSE on 20 datasets.

Dataset	RVFLN	ELM	RVFL-RS	HPFNN	SCN	SCBLS	BLS	FBLS	SCFS-IFRs
Airfoil	3.0120	2.9575	3.0120	3.6479	5.2468	4.9016	5.1255	5.2006	2.7497
Concrete	6.5454	6.8064	6.7564	6.7854	7.2041	6.7368	7.1660	7.0735	5.5644
Auto MPG	3.0211	2.8833	2.9047	2.5723	3.1761	3.1633	3.1780	3.1271	2.6370
Yacht	3.4028	3.3144	4.7810	0.8783	9.3073	4.3551	6.9055	5.2876	0.7443
QSAR	0.9410	0.9469	0.9274	0.9166	0.8986	0.8763	0.8697	0.8798	0.9160
Treasury	0.2086	0.2020	0.1968	0.2305	0.2250	0.2149	0.2328	0.2272	0.1815
Abalone	2.1506	2.1263	2.1260	2.0966	2.2502	2.2228	2.1830	2.1872	2.0994
Categorical	0.3260	0.3865	0.2012	0.0780	0.1130	0.7924	1.0201	0.9321	0.0804
Auto MPG6	3.0642	2.9866	3.9217	2.6323	2.0850	2.1359	2.1748	2.0788	2.6878
Energy	0.3898	0.3980	0.3905	0.3883	0.3881	0.3791	0.3811	0.3714	0.3930
Baseball	723.70	833.72	717.55	615.14	711.36	678.40	666.68	642.84	693.55
Izmir	1.2032	1.3781	1.2186	1.1028	1.1402	1.1389	1.1426	1.1511	1.0810
Ankara	1.4364	1.7054	1.4122	1.2754	1.2453	1.2825	1.3104	1.2811	1.1982
Electrical	102.40	107.16	83.215	115.26	110.75	99.984	113.691	111.974	80.311
Computer	4.9992	5.5109	4.9039	2.9284	4.2178	3.4578	2.9228	4.6363	2.4164
Laser	6.5630	6.2108	4.9542	7.0169	6.3307	6.3058	6.2107	6.1545	5.8172
Prices	4.7295	4.1906	4.0259	3.6176	6.1395	4.7395	5.7109	5.7305	3.0717
Cool	2.0263	2.1077	1.6723	1.7730	3.0224	2.9430	2.8301	2.0390	1.3136
Wine	0.6440	0.6483	0.6448	0.6290	0.6600	0.6552	0.6571	0.6616	0.6294
Heat	1.1406	1.3795	0.5085	0.7662	2.3875	2.3985	2.4324	1.2139	0.5349

compares the testing RMSE of the proposed models with another eight algorithms: RVFLN [32], ELM [33], RVFL-RS [34], HPFNN [35], SCN [19], SCBLS [36], BLS [16] and FBLS [7]. The results of RVFLN, ELM, RVFL-RS and HPFNN are borrowed from [35], and the results of SCN, SCBLS, BLS, FBLS are obtained from the source code provided by the cited author. It should be noted that, in order to be consistent with the results in [35], only the input features are normalized in this experiment. In Table 4, the bold items represent the smallest testing RMSE. Our SCFS-IFRs shows the smallest testing RMSE in 10 of 20 datasets while HPFNN shows the smallest testing RMSE in five datasets. It can be seen that SCFS-IFRs have better generalization performance than the other eight algorithms.

In order to observe the statistical differences between SCFS-IFRs and the other eight algorithms in the 20 datasets, the Friedman test and the Holm post-hoc test are applied. The result of the Friedman test is displayed in

Table 5. Result of Friedman tests (significance level of 0.10).

Algorithm	SCFS-IFRs	HPFNN	RVFL-RS	SCBLS	RVFLN	FBLS	ELM	BLS	SCN
Rank	2.300	3.750	4.575	5.050	5.475	5.500	5.950	6.100	6.300
p-value	0.00001

. We can see that the SCFS-IFRs has the top rank and the $p$-value is less than the significance level 0.1 which implies that there are significant differences in the ranking of generalization performance of these algorithms.

Then we compare SCFS-IFRs with other models using the Holm post-hoc test and the result is shown in

Table 6. Result of Holm post-hoc tests (significance level of 0.10).

Comparison	Statistic	Adjusted p-Value	Result
SCFS-IFRs vs SCN	4.6188	0.00003	H0 is rejected
SCFS-IFRs vs BLS	4.3878	0.00008	H0 is rejected
SCFS-IFRs vs ELM	4.2146	0.00015	H0 is rejected
SCFS-IFRs vs FBLS	3.6950	0.0011	H0 is rejected
SCFS-IFRs vs RVFLN	3.6661	0.0011	H0 is rejected
SCFS-IFRs vs SCBLS	3.1754	0.00449	H0 is rejected
SCFS-IFRs vs RVFL-RS	2.6269	0.01723	H0 is rejected
SCFS-IFRs vs HPFNN	1.6743	0.09407	H0 is rejected

. The null hypothesis ${\mathrm{H}}_0$ in the Holm test is that the ranking of the two comparative methods has no significant difference. The significance level $\alpha$ is set as 0.1. If the adjusted p-value of the Holm test is less than the significance level $\alpha$, this means we should reject the null hypothesis ${\mathrm{H}}_0$ and there are remarkable differences between the two comparative methods. From the Holm test results in Table 6, we can see that the proposed SCFS-IFRs are significantly better than RVFLN, ELM, RVFL-RS, HPFNN, SCN, BLS, FBLS and SCBLS.

4.2. Classification

We select the following 10 datasets to evaluate the performance of the proposed SCFS-IFRs in classification tasks. The relevant details of the 10 classification datasets are summarized in

Table 7. Details of 10 classification datasets from UCI and KEEL.

Dataset	No. of Data	No. of Features	No. of Classes
Diabetes	768	8	2
Australia	690	14	2
Balance	625	4	3
Magic04	19,020	10	2
Monk2	432	6	2
Pageblocks	5472	10	5
Sonar	208	60	2
Specfheart	267	44	2
Breast	699	9	2
Wine	178	13	3

.

The average classification accuracies for the proposed SCFS-IFRs and the other six fuzzy models, TSK-FC [37], L2TSK-FC [37], HID-TSK [6], CFBLS [7], FELM [4] and BL-DFIS [38], for the 10 classification datasets are reported in

Table 8. Comparisons of classification accuracies on 10 datasets.

Dataset	TSK-FC	L2TSK-FC	HID-TSK	CFBLS	FELM	BL-DFIS	SCFS-IFRs
Diabetes	0.6687	0.6432	0.7182	0.7764	0.7512	0.7724	0.7738
Australia	0.7298	0.7768	0.8337	0.8750	0.8477	0.8657	0.8594
Balance	0.7079	0.7114	0.8416	0.9066	0.8720	0.9141	0.9152
Magic04	0.6487	0.7424	0.8160	0.8533	0.8260	0.8629	0.8631
Monk2	0.6332	0.6062	0.6599	0.9353	0.6598	0.7482	0.9720
Pageblocks	0.9025	0.8938	0.9067	0.9550	0.9355	0.9592	0.9603
Sonar	0.4930	0.6620	0.6585	0.7591	0.6895	0.8223	0.8243
Specfheart	0.7516	0.7450	0.7520	0.8335	0.7921	0.8130	0.7622
Breast	0.8846	0.9469	0.9512	0.9750	0.9669	0.9674	0.9691
Wine	0.9037	0.9183	0.9076	0.9740	0.9676	0.9878	0.9885
Average	0.7324	0.7646	0.8045	0.8843	0.8308	0.8713	0.8888

. The results of TSK-FC [37], L2TSK-FC, HID-TSK and CFBLS are borrowed from [7] and the results of FELM and BL-DFIS are borrowed from [38]. The bold items represent the highest testing classification accuracy. The comparisons of classification accuracy in Table 8 show that, SCFS-IFRs have the highest average classification accuracy. In terms of the classification accuracy of each data set, SCFS-IFRs have the highest classification accuracy in six of 10 data sets, while the accuracy of classification of SCFS-IFRs in the other four data sets is still comparable.

The incremental learning algorithm is another advantage of SCFS-IFRs. This helps to achieve a compact architecture while maintaining good performance.

Table 9. Comparisons of average number of fuzzy rules on 10 datasets.

Dataset	TSK-FC	L2TSK-FC	HID-TSK	CFBLS	FELM	BL-DFIS	SCFS-IFRs
Diabetes	3.1	2.0	9.0	8.4	18	1.1	3.5
Australia	7.0	10.0	9.0	15.7	19	2.3	1.7
Balance	4.3	3.9	9.3	9.2	20	2.2	1.2
Magic04	13.4	10.8	36.3	18.9	20	14.3	10.4
Monk2	8.5	7.6	7.0	5.9	18	8.9	3.8
Pageblocks	7.2	7.0	16.2	18.9	17	12.6	12.0
Sonar	8.5	9.3	24.0	8.1	20	2.2	4.6
Specfheart	7.1	7.5	9.3	9.2	20	1.0	1.3
Breast	14.6	13.9	12.0	7.6	20	3.3	5.75
Wine	10.0	5.7	42.0	13.7	20	1.2	2.4
Average	8.37	7.77	17.41	11.56	19.2	4.91	4.67

displays the comparisons of the average number of fuzzy rules that all seven models need to use in order to achieve the accuracy shown in Table 8. The bold items represent the minimum number of rules. From Table 9, It can be observed that the proposed SCFS-IFRs can achieve good classification accuracy with few rules.

Finally, similar to Section 4.2, we use the Friedman test and the Holm post-hoc test to observe the statistical differences between SCFS-IFRs and the other six algorithms on the 10 classification datasets. The result of the Friedman test is displayed in

Table 10. Result of Friedman tests (significance level of 0.10).

Algorithm	SCFS-IFRs	CFBLS	BL-DFIS	FELM	HID-TSK	L2TSK-FC	TSK-FC
Rank	2.300	3.750	4.575	5.050	5.475	5.500	5.950
p-value	1.339 × 10−9

. We can see that the SCFS-IFR has the highest ranking and the $p$-value is 1.339 × ${10}^{-9}\ll 0.10$. This implies that there are significant differences in the ranking of these algorithms.

The result of the Holm post-hoc test is listed in

Table 11. Result of Holm post-hoc tests (significance level of 0.10).

Comparison	Statistic	Adjusted p-Value	Result
SCFS-IFRs vs TSK-FC	5.07198	0	H0 is rejected
SCFS-IFRs vs L2TSK-FC	4.65794	0.00002	H0 is rejected
SCFS-IFRs vs HID-TSK	3.51933	0.00173	H0 is rejected
SCFS-IFRs vs FELM	2.38073	0.05184	H0 is rejected
SCFS-IFRs vs BL-DFIS	0.62106	1	H0 is accepted
SCFS-IFRs vs CFBLS	0.41404	1	H0 is accepted

. The null hypothesis ${\mathrm{H}}_0$ in the Holm test is that the rankings of the two comparative methods have no significant difference. From Table 11, we can see that the proposed SCFS-IFRs is significantly better than TSK-FC, L2TSK-FC, HID-TSK and FELM, but there is no significant difference in the comparison results between SCFS-IFRs, BL-DFIS and CFBLS.

However, SCFS-IFRs has an advantage over BL-DFIS and CFBLS in terms of interpretability. First, it can be seen from Table 9 that the average number of rules of SCFS-IFRs is 4.67, far less than the average number of rules for CFBLS, 11.56. The fewer the number of rules, the better the interpretability. Then, BL-DFIS use mapped features by ELM-AE as the premise variable of fuzzy rules, which is not conducive to the interpretation of the rule.

4.3. The Interpretability of SCFS-IFRs

Although a mathematical measure of interpretability has not been found, a fuzzy inference system with high interpretability should have clear semantic expression and simple fuzzy rules. We discuss the interpretability of SCFS-IFRs through one experimental result on the Monk2 task.

In this experiment, the input universe is divided into four equal parts, which implies five fuzzy subsets with five linguistic labels as shown in Section 3.1. The five fuzzy subsets have five fixed centers which are $\left\{0,0.25,0.5,0.75,1\right\}$. In this experiment, two fuzzy rules generated by the proposed SCFS-IFRs are presented in

Table 12. Rule description for the Monk2 dataset.

IF		THEN
$R_1$	$x_1$ is medium or high and $x_2$ is high and $x_3$ is very low and $x_4$ is low or medium or high and $x_5$ is low and $x_5$ is medium or very high	$y_1=0.761+0.011x_1-0.608x_2-0.038x_3+0.051x_4-0.026x_5+0.016x_6$
		$y_2=0.219+0.020x_1-0.892x_2-0.102x_3+0.125x_4-0.015x_5-0.043x_6$
$R_2$	$x_1$ is very low or low and $x_2$ is very low and $x_3$ is high or very high and $x_4$ is very low or medium and $x_5$ is very high and $x_5$ is very low and low	$y_1=0.279+0.060x_1+0.082x_2+0.187x_3+0.174x_4-0.547x_5-0.134x_6$
		$y_2=-0.024-0.067x_1-0.342x_2-0.164x_3-0.079x_4+0.774x_5+0.260x_6$

. The widths of fuzzy subsets and the rule combination matrix are randomly assigned under the supervisory mechanism $\left(16\right)$ and optimized by the proposed adaptive incremental learning algorithm of SCFS-IFRs. The widths of the two fuzzy rules are 0.8139 and 0.4826. The rule combination matrixes of two fuzzy rules are:

$${\mathit{C}}_1=\left(\begin{array}{ccccc}0& 0& 1& 1& 0\\ 0& 0& 0& 1& 0\\ 1& 0& 0& 0& 0\\ 0& 1& 1& 1& 0\\ 0& 1& 0& 0& 0\\ 0& 0& 1& 0& 1\end{array}\right)$$

and

$${\mathit{C}}_2=\left(\begin{array}{ccccc}1& 1& 0& 0& 0\\ 1& 0& 0& 0& 0\\ 0& 0& 0& 1& 1\\ 1& 0& 1& 0& 0\\ 0& 0& 0& 0& 1\\ 1& 1& 0& 0& 0\end{array}\right)$$

where ${\mathit{C}}_l\left(j,k\right)=1$ represents the $k$the fuzzy set of the $j$th input variable in the $l$th fuzzy rule will be activated; conversely, it will not be activated.

The activated fuzzy sets and the polynomial consequent functions are listed in the two parts of Table 12, respectively. Since the highest degree of the used Chebyshev polynomial is set as 1, the polynomial consequent functions are linear functions.

In SCFS-IFRs, the fixed language labels of the antecedents make the intuitive interpretation of the antecedents possible, and the rule combination matrixes allow activation of more than one fuzzy set, corresponding to each feature in a fuzzy rule, which improves the semantic expression of fuzzy rules. This shortens the length of fuzzy rules while maintaining the performance of the model and avoids the “rule explosion” problem.

5. Conclusions

By incorporating stochastic configuration and ILFRs into a TSK fuzzy system, a novel neuro-fuzzy system named SCFS-IFRs is established for classification and regression data modeling. The SCFS-IFRs uses a stochastic configuration based incremental learning approach to determine the system architecture and the random parameters. Interpretable fuzzy rules are incrementally generated and the premise parameters are randomly selected based on stochastic configuration strategy. In addition, the PSO algorithm, an intelligence search technique, is used in the process of stochastic configuration to obtain better random parameters and further improve the approximation accuracy.

Hence, the proposed SCFS-IFRs improves the interpretability and approximation accuracy of the neuro-fuzzy system at the same time. We tested the model’s performance in both the regression task and the classification task. In the regression experiments, the results of 20 regression datasets which are selected from UCI [39] and KEEL [40] repositories show that the proposed SCFS-IFRs performs best on 10 of the 20 data sets while statistically significantly outperforming the other eight state-of-the-art algorithms. In the classification experiments, SCFS-IFRs achieves higher classification accuracy and better interpretation with fewer rules, compared with the other six fuzzy classifiers. In the future, a broad learning system can be considered to combine with SCFS-IFRs in order to improve approximation performance and decrease training time. In addition, it would be an interesting problem to analyze the approximation bound of SCFS-IFRs theoretically.