NARX Deep Convolutional Fuzzy System for Modelling Nonlinear Dynamic Processes

Abstract

This paper presents a new approach for modelling nonlinear dynamic processes (NDP). It is based on a nonlinear autoregressive with exogenous (NARX) inputs model structure and a deep convolutional fuzzy system (DCFS). The DCFS is a hierarchical fuzzy structure, which can overcome the deficiency of general fuzzy systems when facing high dimensional data. For relieving the curse of dimensionality, as well as improving approximation performance of fuzzy models, we propose combining the NARX with the DCFS to provide a good approximation of the complex nonlinear dynamic behavior and a fast-training algorithm with ensured convergence. There are three NARX DCFS structures proposed, and the appropriate training algorithm is adapted. Evaluations were performed on a popular benchmark—Box and Jenkin’s gas furnace data set and the four nonlinear dynamic test systems. The experiments show that the proposed NARX DCFS method can be successfully used to identify nonlinear dynamic systems based on external dynamics structures and nonlinear static approximators.

1. Introduction

Many industrial processes are nonlinear with uncertainty and have time-varying properties. In industrial process modelling, model structure and parameters often change over time. Among the known structural time variations in process plants is the change in the process delay. Obtaining an accurate mathematical model of industrial processes is not an easy task and presents a challenge for system identification. An alternative approach is to develop data-driven (DD) models that do not require a detailed physical understanding of the process and can be extracted from data streams at the same time as data collection. The concept, which assumes the adaptation of both the model structure and the model parameters, is called an evolving system. In [1], an overview of the development of evolving intelligent regression and classification systems is presented, focusing on fuzzy and neuro-fuzzy methods.

In the past, various DD linear model structures have been established, such as state space (SS) models, autoregressive exogenous (ARX) models, Box–Jenkins (BJ) models, and output error (OE) models [2]. These structures can be easily extended to the nonlinear structure, for example, ARX to nonlinear ARX (NARX). However, rule-based (RB) systems, artificial neural networks (ANN) based systems, and a combination of both neuro-fuzzy systems (NFS) can be used to approximate a non-linear static characteristic or an input–output transition function. Fuzzy modelling, as a representative of RB systems, provides fuzzy inference to manage uncertainty and better understand modelling results than ANN or deep learning (DL) methods. It is now well established that fuzzy systems (FS), ANN, and NFS are approximators of all different types of non-linear functions within a compact domain.

In the initial phase of the investigations, linguistic modelling [3] and fuzzy relation equation-based methods [4] were suggested as the basic methods for fuzzy model identification. The Takagi–Sugeno–Kang (TSK) models [5,6,7] have enabled the evolution of more complex rule-based systems where the rules are supported by local regression models. TSK training algorithms are characterised by a division into separate model structure identification and parameter estimation, where structure identification is assumed to be the determination of the number of rules and the parameters of the membership functions of the fuzzy sets in the rules. On the consequent side of the rules are the parameters of the linear models, which are usually determined by the least squares method. Later, researchers started to pay attention to the integration of a non-linear autoregressive model with exogenous input (NARX) into an intelligent model [8,9,10,11,12]. Intelligent models with NARX structure have good predictive properties, and they are widely used for the modelling and identification of nonlinear dynamical systems [13]. Paper [14] reviews hybrid models that combine the advantages of FS and ANN using training algorithms based on supervised learning methods. When modelling complex non-linear systems, these methods have the disadvantage of a large rule base with many rules. Rule generation is complex and often not understandable. In the case of a large training dataset, it is difficult to process it efficiently with iterative learning algorithms, which are characterised by multiple passes over the same pieces of data.

The applicability of FS or NFS is limited by the number of input variables and the number of fuzzy sets defined on the domains of the input and output variables. Increasing these numbers leads to a problem known in the literature as the “curse of high dimensionality”. Researchers have proposed some feature reduction approaches to be used before applying the training algorithm. In our early work [15], we addressed this problem by proposing a method to decompose a single large FS into smaller subsystems. By following the decomposition mechanism of the fuzzy inference procedure, we have applied several simple single-input-single-output (SISO) decomposed fuzzy inference systems (FIS) to model dynamic systems. The proposed decomposition was based on a FS represented by a fuzzy relational equation. In order to improve the performance we later presented the structure of a decomposed NFS ARX model [16] for modelling unknown dynamical systems.

Today, we are dealing with a huge amount of data and many features when modelling complex dynamic processes. As a result, the performance of traditional neuro-fuzzy classification and prediction systems is reduced. A hierarchical fuzzy system (HFS), first proposed by Raju et al. [17], shows better performance than general FS for big data prediction [18,19,20,21,22]. The training algorithms for HFS are mostly of the same types as those for ANN or deep convolutional neural networks (DCNN) [23]. Those algorithms are mainly iterative in their nature and, therefore, they are computationally intensive when applied to problems with a huge amount of data and many features. The deep convolutional fuzzy system (DCFS), proposed in [24], is a hierarchical fuzzy structure, which can overcome the deficiency of general FS when facing high dimensional data. The DCFS hierarchical structure has the form of a pyramid made up of several smaller simple fuzzy subsystems (FSs). A high dimensional data set input into the DCFS is split into several small datasets and delivered to FSs in the first structure layer. The first layer FSs are designed using the WM training method [25,26] and may be represented as the ordinary week estimators [25]. By passing the training data through FSs of the first level, a data set for the second level is generated, and the training procedure may be repeated.

In this paper, we propose to use DCFS as approximators of unknown nonlinear functions, in the NARX structure, to model nonlinear dynamic processes (NDPs). We investigated the significance of the appropriate selection of the structure of the FSs at the first level of the DCFS. We test the proper FSs input–output signal selection at the first level of the DCFS, which has an impact on the final approximation properties. An important result is the definition of three different NARX DCFS structures. For the proposed NARX DCFS, we have adapted the WM algorithm to be suitable for training on the input/output data of different processes.

This paper consists of five sections. In Section 2, we present the basics of nonlinear dynamic input/output modelling with an external dynamic’s principle, the concept and implementation of the Wang–Mendel FS, and the idea of the DCFS. We continue by presenting the main contribution of the research, which is the adaptation of the DCFS for modelling the nonlinear static approximators of the nonlinear time-invariant dynamic systems. There are three different NARX DCFS structures proposed, and the training algorithm is adapted. Section 3 presents the experimental results tested on different data sets to validate the proposed method. The modelling results are evaluated and are analysed in Section 4, where we summarise the strengths and weaknesses of the proposed modelling method. Finally, the last section concludes the paper.

2. Materials and Methods

In this section, we will give details on how to apply a NARX DCFS for nonlinear system modelling. The non-linear behaviour of time-varying dynamical systems is often modelled by approximating static non-linearities in combination with linear dynamical systems. Such models are used to predict the output of a process, which can either be one-step or long-term predictions. In the case of one-step forecasting, a nonlinear static function approximator (NSFA) is used to predict the output of the model based on the past inputs and outputs of the process. Such a model is often used in predictive and adaptive control systems. Alternatively, a NSFA can be used to predict the output of a process over a long-time horizon, in which case a multi-step prediction with a longer prediction horizon is performed.

In the following subsections, we present models, methods, and notations needed to implement and understand the proposed NARX DCFS model and its application for modelling NDPs.

2.1. Nonlinear Dynamic Input/Output Models Based on External Dynamics

A nonlinear SISO system in the continuous time space is defined as

$$y\left(t\right)=H\left(u\left(t\right)\right)$$

(1)

where $u\left(t\right), y\left(t\right)\in R$ are the input variable and output variable, and $H\left(t\right)$ is a bounded continuous nonlinear function. By performing discretization with sampling time $T_s, t=k·T_s$, a discrete-time SISO nonlinear system is given as follows

$$y\left(k\right)=f\left(y\left(k-1\right),\cdots y\left(k-n_y\right),u\left(k-\tau -1\right),\cdots u\left(k-\tau -n_u\right)\right)$$

(2)

where $u\left(k\right)$ and $y\left(k\right)$ are the system input and output at a discrete time $k$, respectively,$n_u$ and $n_y$ are the discrete lags for the input and output signals depending on the unknown system orders, τ is a delay, and $f(·)$ is an unknown nonlinear function.

The external dynamics (ES) strategy is a popular and widely used structure for nonlinear dynamic system (NDS) modelling. ES means that the nonlinear dynamic model can be divided into two parts: a static approximator of the nonlinear function and a delay mechanism for the input and output signals, as shown in Figure 1.

The structure in the Figure 1 is also well known from linear dynamic models. Most nonlinear dynamic input/output models can be expressed in the following form:

$$\widehat{y}\left(k\right)=f\left(\phi \left(k\right)\right)$$

(3)

where $\phi \left(k\right)$ is a regression vector that may include the prior and, optionally, the actual input process data, the previous output process (or model) data, and the prior prediction errors. $e\left(k-i\right), i=1,2\cdots n_y$. By choosing a regression vector, we choose a structure familiar from linear dynamic models. If we only have lagged outputs in the regression vector, then we model nonlinear time series (NTS). Other popular nonlinear input/output models are summarised in

Table 1. Structures of regression vectors for common input/output models.

Model	Regression Vector φ(k)
NTS	$\phi \left(k\right)=\left[y\left(k-1\right),\cdots y\left(k-n_y\right)\right]$
NARX	$\phi \left(k\right)=\left[y\left(k-1\right),\cdots y\left(k-n_y\right),u\left(k-1\right),\cdots u\left(k-n_u\right)\right]$
NARMAX	$\phi \left(k\right)=\left[y\left(k-1\right),\cdots y\left(k-n_y\right),u\left(k-1\right),\cdots u\left(k-n_u\right),e\left(k-1\right),\cdots e\left(k-n_y\right)\right]$
OE	$\phi \left(k\right)=\left[\widehat{y}\left(k-1\right),\cdots \widehat{y}\left(k-n_y\right),u\left(k-1\right),\cdots u\left(k-n_u\right)\right]$

.

Note that, when modelling NDPs, complexity usually increases significantly as the dimensionality of the input space increases. This is the reason why lower dimensionality models—NTS, NARX, and OE—are more widespread.

Over the last three decades, regression techniques, as well as ANN, FIS, and ANFIS [27], have been widely employed for modelling the nonlinear static approximators of the nonlinear time-invariant dynamic systems. ANNs allow accurate predictions, but the resulting model structures are not useful for explaining the physical background of the process being modelled. FSs and NFSs are inherently better suited for interpreting physical behaviour, as their properties are usually described by if-then rules. A simple FS and a corresponding DD learning method, called the Wang–Mendel (WM) method, proposed in [25], was one of the first methods to model FSs from data and is still a strong and applicable method for DD learning. As WM is the FS used in DCFS, we describe it, in detail, in the next subsection.

2.2. WM Fuzzy System

The WM fuzzy system is standard FS designed with fuzzy rules. Following fuzzy IF-THEN rules allow for modelling the behaviour of the output variable $y$ depending on the input variables $x={\left(x_1,\cdots ,x_n\right)}^T:$

$$\mathrm{IF} x_1 \mathrm{is} A_1^{\left(q\right)} \mathrm{and}\cdots \mathrm{and} x_m \mathrm{is} A_m^{\left(q\right)} \mathrm{THEN} y \mathrm{is} B^{\left(q\right)}$$

(4)

where $A_j^{\left(q\right)}$ and $B^{\left(q\right)}$ are fuzzy sets defined in rule base R, $q$ is the index of the rule, and $\left(j=1,\cdots ,m\right)$ is a subset of $\left(1,\cdots ,n\right)$. The selection of the m input variables in (4) with $m\le n$ means that the rules can consist of a limited selection of input variables. Fuzzy sets $A_j^{\left(q\right)}$ and $B^{\left(q\right)}$ are defined as shown in Figure 2, where the centers of the fuzzy sets are equidistant between $\min x_j$ and $\max x_j$. For simplicity, assume that $M_1=M_j=M_m$ in the following notation. Therefore, the maximum number of rules is $M=M_i^m$.

A fuzzy inference mechanism aggregates the rules from a fuzzy rule base into a mapping from fuzzy input set $A^{\prime }$ in the input space $R^m$ to fuzzy set $B^{\prime }$ in the output space $R$. The fuzzifier $fuzz\left(x\right)$ converts the crisp input $x$, into the fuzzy set $A^{\prime }$, and the defuzzifier $defuzz\left(B^{\prime }\right)$ determines a crisp single point $y$ in the output space $R$ that best represents the fuzzy set $B^{\prime }$. For fuzzy reasoning, the product fuzzy inference mechanism is

$${\mu }_{B^{\prime }}\left(y\right)=\underset{q=1,M}{\max }\left\{\underset{x\in R^m}{\sup }\left[{\mu }_{A^{\prime }}\left(x\right){{\displaystyle \prod }}_{j=1}^m{\mu }_{A_j^{\left(q\right)}}\left(x_j\right){\mu }_{B^{\left(q\right)}}\left(y\right)\right]\right\},$$

(5)

where m is number of inputs $x_{i1},\cdots ,x_{im}$ we used following singleton fuzzifier

$${\mu }_{A^{\prime }}\left(x^{\prime }\right)=fuzz\left(x^{\prime }\right)=\{\begin{array}{c}1,\mathrm{if}{x}^{\prime }=x\\ 0,\mathrm{otherwise}\end{array}$$

(6)

and center–average defuzzifier

$$y=defuzz\left(B^{\prime }\right)=\frac{{{\displaystyle \sum }}_{q=1}^M{y}_c^{\left(q\right)}\left({{\displaystyle \prod }}_{j=1}^m{\mu }_{A_j^{\left(q\right)}}\left(x_j\right)\right)}{{{\displaystyle \sum }}_{q=1}^M\left({{\displaystyle \prod }}_{j=1}^m{\mu }_{A_j^{\left(q\right)}}\left(x_j\right)\right)},$$

(7)

where $y_c^{\left(q\right)}$ are the midpoints of the symmetric triangular membership functions of the output fuzzy sets $B^{\left(q\right)}$. Note that, for the membership functions presented in Figure 2, we simplify the expression by recognizing that ${\displaystyle \sum }_{q=1}^M\left({\displaystyle \prod }_{j=1}^m{\mu }_{A_j^{\left(q\right)}}\left(x_j\right)\right)=1$, and the denominator of the FS (7) is equal to 1, and FS is simplified to

$$y={\displaystyle \sum }_{q=1}^M{y}_c^{\left(q\right)}\left({\displaystyle \prod }_{j=1}^m{\mu }_{A_j^{\left(q\right)}}\left(x_j\right)\right).$$

(8)

Note that the FS (8) is considered from the $M$ fuzzy rules (4), with each rule covering a cell $\left(j_1,\cdots ,j_m\right)$ in the m dimensional input space, which means that FS implements local inference with one rule representing one input–output relationship. As a result, this relationship can be represented by the parameter $y_c^{\left(q\right)}$.

The mapping of the non-linear function of the dynamical system (3) is created using only the information, which is written into the appropriate regression vector depending on the structure of the chosen nonlinear input/output model from Table 1. The inputs to the FS (8) for modelling a nonlinear static approximator are, therefore, $x\left(k\right)=\phi \left(k\right)$ and the dimension of the input vector can be quite large. Both FSs and non-FSs methods based on fuzzy rules (4) are characterised by the problem of the curse of dimensionality. This means that, in the case of many input variables $m$ and a large number of fuzzy sets, $M_i$ of fuzzy sets per input variable greatly increases the number of rules $M$. In our previous research [28], the problem of too many fuzzy rules to model more complex dynamical systems was solved by decomposing the FS into several simple FSs with fewer input variables. The principle is based on the decomposition of the inference mechanism. As an example, the approximation of the nonlinear function with the decomposed FS having $n_u+n_y$ input variables would be replaced by $n_u+n_y$ FSs with one input and one output, as well as with two-dimensional fuzzy relations $R_i$. The output of the simplified decomposed fuzzy model is

$$y=\frac{1}{n_u+n_y}·\left\{defuzz\left[fuzz\left(u\left(k-\tau -1\right)\right)°R_{b_1}\right]+\cdots +defuzz\left[fuzz\left(u\left(k-\tau -n_u\right)\right)°R_{b_{nu}}\right]+defuzz\left[fuzz\left(y\left(k-1\right)\right)°R_{a_1}\right]+\cdots +defuzz\left[fuzz\left(y\left(k-n_y\right)\right)°R_{a_{ny}}\right]\right\},$$

(9)

where ° represents a set of composition operators (i.e., max–min, max–prod, sum–prod, etc.) and R is the relational matrix. The structure of the dynamical system (9) used to model the non-linear process is like that of the discrete linear ARX model. The difference is that the linear parameters $a_i$ and $b_i$ of the linear ARX model are substituted with simple FSs. In [16], we analyze the advantages of this approach and outline the drawbacks. Advantages of the simplified decomposed fuzzy model are fewer rules in rule base; two-dimensional fuzzy relations; code optimization and hardware inference realization are possible; similarity between the structure of the suggested model and the discrete linear ARX model. However, the analysis of the results also showed certain weaknesses resulting from the decomposition of a FS into several simple ones. In the case of approximation of complex nonlinear dynamic function, the trained model is worse at predicting the output of the process than in the case of an undecomposed FS. We have concluded that simplifying the structure of a FS into several simple FSs connected at a single level brings limitations in the approximation of complex non-linear functions. We started to look for a solution in the multilevel connection of simple FSs, which led us to study the hierarchical FSs, which include DCFS.

2.3. Deep Convolutional Fuzzy System

The key idea behind deep modelling is mostly in the multi-layer nature of the method, the transformation of features in the model, and the sufficient complexity of the model. DCFS, proposed in [24], represents a new principle of development—a hierarchical FS based on the WM method—in particular, for high-dimensional mappings. The DCFS is based on the use of the WM method of learning from data to determine simple multilevel multidimensional FSs. The design proceeds from lower levels to higher levels using the WM method [25,26]. The number of inputs of simple FSs is determined by a so-called moving window, which acts as a convolution operator.

In [24], DCFS was developed for stock index prediction. The structure of the DCFS is presented in Figure 3. The DCFS contains n inputs mapped to the input vector $\left(x_1^0,x_2^0,\cdots ,x_n^0\right)$, and the scalar output $x^L$. Level $l \left(l=1,2, \cdots ,L-1\right)$is composed of $n^l$ fuzzy systems $F{S}_i^l \left(i=1,2, \cdots ,n^l\right),$ the outputs of which are marked as $x_i^l$, and represents inputs to higher level $l+1$ FSs. At the last level, there is only one $F{S}^L$, which processes the $n^{L-1}$ outputs from level $L-1$ and gives the final DCFS output $x^L$. The fuzzy systems $F{S}_1^l, F{S}_2^l, \cdots ,F{S}_{n^l}^l \left(l=1,2, \cdots ,L-1\right)$ and their inputs were obtained as outputs from the lower level $F{S}_i^{l-1}$ outputs $x_1^{l-1},x_2^{l-1},\cdots ,x_{n^{l-1}}^{l-1}$. Those using a moving window are grouped into the input sets $I_1^l,I_2^l,\cdots ,I_{n^l}^l$. The purpose of the moving window is to determine the number of FSs in the layers and the number of inputs to a particular $F{S}_i^l$. It performs a double task. Namely, it determines the size $m^l$ of the moving window, and it determines the moving pattern, which refers to the number of inputs that are moved in each step.

.

The choice of the length m of the moving window is important and can be different for $F{S}_i^l$ at different levels. The number should not be too large, as this will lose the advantages of DCFS over a conventional FS. At the same time, it should not be too small. It is usually chosen between 2, 3, 4, or 5, and it may move one input at a time or more than one input at a time to cover all variables in the level. The strategy of moving inputs to individual $F{S}_i^l$ can also be changed. This provides a variety of DCFS structure options.

In [24], the author argues that the DCFS learned from the data with the WM learning method has a better model interpretation capability compared to the DCNN results. The WM training method determines the parameters $y_c^{\left(l\right)}$, $l=1, 2,\cdots ,M$ of the FS (8), which is used for all $F{S}_i^l$ ($i=1, 2,\cdots ,n^l$) and $\left(l=1, 2,\cdots ,L-1\right)$ in the DCFS. Details on the adaptation of the WM training method [26] for DCFS training are detailed in [24].

In next section, we will give details on how to apply a DCFS for system modelling of the nonlinear dynamic input/output models based on a nonlinear static approximator and an external tapped delay line.

2.4. Nonlinear Dynamic Input/Output Models Based NARX DCFS

The main contribution of this research is the adaptation of DCFS for modelling the nonlinear static approximators of the nonlinear time-invariant dynamic systems, which are based on ED structures, such as the NARX structure. First, it is necessary to investigate the appropriate structure of the DCFS input vector fitted to the input regression vector of the NARX structure (Table 1). There are many possibilities, but several structures make more sense, which we describe below.

2.4.1. Different NARX DCFS Structures

NARX DCFS structures differ in the number of l, (l = 1, 2,$\cdots ,$L) hierarchical levels of the DCFS, the number of n^l fuzzy subsystems $F{S}_i^l$ in each level, the selection and number of inputs to the fuzzy subsystems (moving window $I_i^l$), and the way in which all the fuzzy subsystems are implemented. The fuzzy subsystems $F{S}_i^l$ are defined by the number of inputs, as well as the number of fuzzy sets on the input and output variables. The FSs may be equal at all levels, or they may be different. In the following, we assume the same number $M=M_1=M_m$ of fuzzy sets on the inputs and outputs of all fuzzy subsystems $F{S}_i^l$.

First, let’s start by defining the general structure of the order n.

Definition 1.

The General NARX DCFS structure of the order n_u, n_y maps the input regression vector $\phi \left(k\right)=\left[y\left(k-1\right),\cdots y\left(k-n_y\right),u\left(k-\tau -1\right),\cdots u\left(k-\tau -n_u\right)\right]$directly to the DCFS input vector $\left(x_1^0,x_2^0,\cdots ,x_n^0\right)$, where n = n_u + n_y, and τ is a dead-time. The input sets $I_1^l,I_2^l,\cdots ,I_{n^l}^l$to the fuzzy systems $F{S}_1^l, F{S}_2^l, \cdots ,F{S}_{n^l}^l$(l = 1, 2,$\cdots ,L-1$) are selected from the previous level’s outputs $x_1^{l-1},x_2^{l-1},\cdots ,x_{n^{l-1}}^{l-1}$ using a moving window of the length m with moving scheme where it may move one variable at a time starting from $x_1^{l-1}$until $x_{n^{l-1}}^{l-1}$.

Note that the general NARX DCFS has FSs in the first level where the inputs in the moving window $I_1^1,I_2^1,\cdots ,I_{n^1}^1$ are not arranged according to any rule. The inputs are ordered sequentially, as they are written in the regression vector. An example of general NARX DCFS with n_u = 3, n_y = 3, n₁ = 4, $m^1=m^2=3, m^3=2$, and L = 3 is presented in Figure 4a. When selecting the inputs for the $F{S}_i^1$ at the first level, it is reasonable to select the inputs from the regressor vector $\phi \left(k\right)$ in a way that makes sense. The next NARX DCFS structure choice is based on combining the delayed inputs $u\left(k-\tau -1\right),\cdots u\left(k-\tau -n_u\right)$ together as inputs for the moving windows of the first half of $F{S}_i^1$ at the first level. Similarly, the combined delayed outputs $y\left(k-1\right),\cdots y\left(k-n_y\right)$ of the regression vector are inputs for the second half of $F{S}_i^1$ at the first level. This principle can be continued at higher levels up to the last level. Due to the division of FSs into input and output parts, we called this structure the Input-output NARX DCFS structure.

Definition 2.

The Input-output NARX DCFS structure of the order n_u, n_y maps the input regression vector $\phi \left(k\right)=\left[y\left(k-1\right),\cdots y\left(k-n_y\right),u\left(k-\tau -1\right),\cdots u\left(k-\tau -n_u\right)\right]$to the DCFS input vector $\left(x_1^0,x_2^0,\cdots ,x_n^0\right)$, where n = n_u + n_y, and τ is a dead-time. At the first level, FSs are divided into input part consisting of $F{S}_{ui}^1$with the moving windows $I_{ui}^1$ which contain input regressors $u\left(k-\tau -1\right),\cdots u\left(k-\tau -n_u\right)$, and output part consisting of $F{S}_{yi}^1$ with the moving windows $I_{yi}^1$ which contain input regressors $y\left(k-1\right),\cdots y\left(k-n_y\right)$.

An example of input–output NARX DCFS with n_u = 3, n_y = 3, n₁ = 4, m¹ = 2, m² = 3, m³ = 2, and L = 3 is presented in Figure 4b. The first two FSs at the first level, $F{S}_1^1$ and $F{S}_2^1$, process delayed inputs, and the second two, $F{S}_3^1$ and $F{S}_4^1$, process delayed outputs.

The next option for selecting the inputs for the $F{S}_i^1$ at the first level, which is frequently used in hierarchical FSs-based dynamical models, is the selection of lower-order dynamic sub-models.

Definition 3.

The Sub-model NARX DCFS structure of the order n_u, n_y maps the input regression vector $\phi \left(k\right)=\left[y\left(k-1\right),\cdots y\left(k-n_y\right),u\left(k-\tau -1\right),\cdots u\left(k-\tau -n_u\right)\right]$to the DCFS input vector $\left(x_1^0,x_2^0,\cdots ,x_n^0\right)$, where n = n_u + n_y, and τ is a dead-time. At the first level, FSs are organized as lower order sub-models with the moving windows$I_i^1$which contain input regressors $u\left(k-\tau -i\right),\cdots u\left(k-\tau -i-n_{ui}^1\right)$,$y\left(k-i\right),\cdots y\left(k-i-n_{yi}^1\right)$, where $n_{ui}^1\le n_u$and $n_{yi}^1\le n_y$are lags for the output and input signals if sub-models.

An example of input–output NARX DCFS with n_u = 3, n_y = 3, n₁ = 3, $n_{u1}^1=n_{u2}^1=n_{u3}^1=1$, $n_{y1}^1=n_{y2}^1=n_{y3}^1=1$, $m^1=m^2=m^3=2$, and L = 3 is presented in Figure 4c. We see that, at the first level, we have three FSs that approximate the dynamic behaviour of the first order. Similarly, we could have higher-order systems. Structures of each of the FSs are not necessarily always the same, and the possibilities are almost unlimited.

Remark 1.

With the proper selection of parameters, practically any FS may be presented as a DCFS. The selection of parameters l = 1, n¹ = n_u + n_y, m¹ = n_u + n_y fuzzy subsystem $F{S}_1^1$gives a structure which is the same as general fuzzy system (GFS) based on the ARX model structure presented in [4]. In a similar way, the choice of parameters l = L, n¹ = n_u + n_y, m¹ = 1, m^l = 2, $l=2,3\cdots L$can be combined to give a structure like the simple decomposed fuzzy ARX model of the SISO dynamic system (9) proposed in detail in reference [16]. Instead of a multi-level structure, the outputs of the first level can be averaged, resulting in exactly same structure.

The system identification procedure of a nonlinear dynamic input–output model involves constructing an appropriate model from input–output data. In the next subsection, we present a training algorithm that enables fast parameter training of $F{S}_i^l$ based on input–output data.

2.5. Fast Training Algorithm for NARX DCFS Structures

The main training algorithm of DCFS is the Wang–Mendel (WM) method [25,26]. The original MATLAB code of the DCFS training algorithm [24] is published in a text file at: https://ieeexplore.ieee.org/ielx7/91/9130783/8788632/code_training_algorithm.pdf?arnumber=8788632&tag=1, accessed on 27 December 2022.

We have adapted the training algorithm code to the proposed NARX DCFS structures. It is not an iterative method; rather, it is based on a rule extraction approach from data. Given the N input–output data pairs $u\left(k\right), y\left(k\right)$, $k=1,2\cdots N$, where $y\left(k\right)$ and $u\left(k\right)$ are the at time $k$, the task of the training algorithm is to design a NARX DCFS parameter to match this input–output pair. The training method for DCFS proposed in [24] has been modified to allow effective training of the NARX DCFS structures. We continue with the description of the extended training method, which we present as an Algorithem 1, as follows.

Algorithm 1
Step 1. Choose the structure of the NARX DCFS.      Define the $n_y$, $n_u$, $\tau$ of regression vector $\phi \left(k\right)=\left[y\left(k-1\right),\cdots y\left(k-n_y\right),u\left(k-\tau -1\right),\cdots u\left(k-\tau -n_u\right)\right]$.      Define the NARX DCFS structure and the way the NARX regression vector $\phi \left(k\right)$ is mapped to the DCFS input vector $\left(x_1^0,x_2^0,\cdots ,x_n^0\right)$.      Create data pair for DCFS training $\left[x_1^0\left(k\right),\cdots ,x_n^0\left(k\right); y^0\left(k\right)\right], k=1,2\cdots N$. Step 2. Choose the structure of the DCFS.      Define the number hierarchical levels of the DCFS of L, the number of nl fuzzy subsystems $F{S}_i^l$ in each level, and the number of inputs to the fuzzy subsystems $m^l$, $l=1,2\cdots L$ (moving window $I_i^l$).      Choose the moving scheme $I_i^l$.      Choose the number $M$ of fuzzy sets for all inputs and all $FSs.$ Step 3. Design the first level $F{S}_i^1$ (i = 1, 2,$\cdots ,n^1$) in the form of (8), using the WM method [24,25,26], where the data pairs are $\left[x_i^0\left(k\right),\cdots ,x_{m+i-1}^0\left(k\right); y^0\left(k\right)\right], k=1,2\cdots N$.     Step 3.1. For each cell $\left(j_1, j_2,\cdots ,j_m\right)$ with $j_1, j_2,\cdots ,j_m=1,2,\cdots ,M$ set the initial values of the weight parameter $w^{j_{1,}\cdots ,j_m}$ and the weight output parameter $u^{j_{1,}\cdots ,j_m}$ equal to zero.     Step 3.2. For each input $x_i^0,\cdots ,x_{i+m-1}^0$ to $F{S}_i^1$, consider the $M$ fuzzy sets $A^1, \cdots ,A^M$, with membership functions of triangular shape, symmetrically arranged as in the Figure 2, and choose the endpoints as
$\begin{array}{c}\min x_i^0=\min \left(x_i^0\left(k\right)|k=1,2\cdots N\right)\\ \max x_i^0=\max \left(x_i^0\left(k\right)|k=1,2\cdots N\right)\\ ⋮\\ \min x_{m+i-1}^0=\min \left(x_{m+i-1}^0\left(k\right)|k=1,2\cdots N\right)\\ \max x_{m+i-1}^0=\max \left(x_{m+i-1}^0\left(k\right)|k=1,2\cdots N\right)\end{array}$	(10)
Step 3.3. For each data pair $\left[x_i^0\left(k\right),\cdots ,x_{m+i-1}^0\left(k\right); y^0\left(k\right)\right], k=1,2,\cdots ,N$ determine the fuzzy sets $A^{j_1^{*}}, \cdots ,A^{j_m^{*}}$ that achieve the maximum membership values among the $M$ fuzzy sets $A^1, \cdots ,A^M$ at $x_i^0\left(k\right),\cdots ,x_{i+m-1}^0\left(k\right)$. Determine
$\begin{array}{c}{j}_1^{*}=\underset{j\in \left\{1,2,\cdots M\right\}}{\arg \max }\left(A^j\left(x_i^0\left(k\right)\right) \right)\\ ⋮\\ j_m^{*}=\underset{j\in \left\{1,2,\cdots M\right\}}{\arg \max }\left(A^j\left(x_{m+i-1}^0\left(k\right)\right) \right)\end{array}$		(11)
Step 3.4. Update the weight parameters $w^{j_1^{*}, \cdots ,j_m^{*}}$ and weight output parameters $u^{j_1^{*}, \cdots ,j_m^{*}}$ for cell $j_1^{*}, \cdots ,j_m^{*}$
$\begin{array}{c}{w}^{j_1^{*}, \cdots ,j_m^{*}}=w^{j_1^{*}, \cdots ,j_m^{*}}+A^{j_1^{*}}\left(x_i^0\left(k\right)\right)\cdots A^{j_m^{*}}\left(x_{m+i-1}^0\left(k\right)\right)\\ u^{j_1^{*}, \cdots ,j_m^{*}}=u^{j_1^{*}, \cdots ,j_m^{*}}+\left(A^{j_1^{*}}\left(x_i^0\left(k\right)\right)\cdots A^{j_m^{*}}\left(x_{m+i-1}^0\left(k\right)\right)\right)·y^0\left(k\right)\end{array}$		(12)
Step 3.5. Repeat Steps 3.3 and 3.4 for $k=1,2\cdots N$. For the cells $\left(j_1,\cdots ,j_m\right)$ with $w^{j_{1,}\cdots ,j_m}\ne 0$, determine the parameters $y_c^{j_{1,}\cdots ,j_m}$ in the $F{S}_i^1$ of (8) as
$y_c^{j_{1,}\cdots ,j_m}=\frac{u_c^{j_{1,}\cdots ,j_m}}{w_c^{j_{1,}\cdots ,j_m}}$.		(13)
Cells $\left(j_1,\cdots ,j_m\right)$ with $w^{j_{1,}\cdots ,j_m}\ne 0$ are covered by data, and define
$Y_c\left(0\right)=\left\{\left(j_1,\cdots ,j_m\right)|\left(j_1,\cdots ,j_m\right)\mathrm{where}\mathrm{is}{w}^{j_{1,}\cdots ,j_m}\ne 0\right\}$.		(14)
Step 3.6. For each cell $\left(j_1,\cdots ,j_m\right)$ not in $Y_c\left(0\right)$, search its neighbors to see whether they are in $Y_c\left(0\right)$, where two cells, $\left(j_1,\cdots ,j_m\right)$ and $\left(j_1^{\prime },\cdots ,j_m^{\prime }\right)$, are neighbours to each other if $j_i=j_i^{\prime }$ for all $i=1,2,\cdots ,m$ except at one location r, such that the $j_r=j_r^{\prime }+1$ or $j_r=j_r^{\prime }-1$. For the cells $\left(j_1,\cdots ,j_m\right)$ not in $Y_c\left(0\right)$ that have at least one neighbor in $Y_c\left(0\right)$, determine the $y_c^{j_{1,}\cdots ,j_m}$ as the average of the $y_c^{j_{1,}\cdots ,j_m}$ ’s of its neighbors in $Y_c\left(0\right)$. Define
$Y_c\left(1\right)=Y_c\left(0\right) \oplus \left\{\left(j_1,\cdots ,j_m\right)| y_c^{j_{1,}\cdots ,j_m}\mathrm{is}\mathrm{determined}\mathrm{in}\mathrm{this}\mathrm{step}\right\}$		(15)
Step 3.7. Repeat Step 3.6 with $Y_c\left(0\right)$ replaced by $Y_c\left(1\right)$, and $Y_c\left(1\right)$ replaced by $Y_c\left(2\right)$, and continue this process to get $Y_c\left(3\right), Y_c\left(4\right)$, $\cdots$, $Y_c\left(p\right)$ that contain the cells $\left(j_1,\cdots ,j_m\right)$ with $j_1,\cdots ,j_m=$ $1,2,\cdots ,M$. For more features of the Wang–Mendel (WM) method and a detailed description, refer to [25,26].
Step 4. Repeat Step 2 for l = 2 up to L, designing the FSs at all levels. For example, for level l we design the s $F{S}_i^l$, $i=1,2,\cdots ,n^l$, assuming that, in the previous step, we have designed all FSs from level l-1.
Step 4.1. For each $k=1,2\cdots N$ put $x_1^0\left(k\right)$, $x_2^0\left(k\right)$,$\cdots$, $x_n^0\left(k\right)$ input to DCFS level 1 and compute upwards along the DCFS to get the outputs of level l-1. The outputs of FSs of level l-1 are denoted by $x_1^{l-1}\left(k\right),x_2^{l-1}\left(k\right),\cdots ,x_{n^{l-1}}^{l-1}\left(k\right)$. We create new input–output data pairs $\left[x_1^{l-1}\left(k\right),x_2^{l-1}\left(k\right),\cdots ,x_{n^{l-1}}^{l-1}\left(k\right), y^0\left(k\right)\right],$ $k=1,2\cdots N$ for designing the level l $F{S}_i^l$.
Step 4.2. Repeat Step 3, and train the level l $F{S}_i^l$, $i=1,2,\cdots ,n^l$ with the new input–output data pairs $\left[x_1^{l-1}\left(k\right),x_2^{l-1}\left(k\right),\cdots ,x_{n^{l-1}}^{l-1}\left(k\right), y^0\left(k\right)\right]$, with k = 1,2⋯N.
Step 4.3. Set the l = l +1 and repeat Steps 4.1 and 4.2 until the last $F{S}_{}^L$ on the level L is designed.

Note that the proposed training algorithm is much faster than the standard gradient descent-based algorithms, such as back propagation (BP). All the $F{S}_i^l$ in the DCFS are designed with N data, which are applied once in the training procedure, rather than iteratively several times, as is necessary for gradient-based and other iterative learning methods. In the following, we present the proposed NARX DCFS structure for the prediction of nonlinear dynamical systems.

3. Experimental Studies

With the aim of evaluating the applicability of the NARX DCFS for modelling non-linear dynamic systems, all three structures defined in Section 2.4.1 were tested on different sets of benchmark test data in ordering, gas furnace system input–output modelling, and several nonlinear dynamic test processes. All experiments were performed in MATLAB R2021b on a PC with Intel^® Core™ i7 CPU 870 @ 2.93 GHz 2.93 GHz and Windows 10 Pro operating system. The program code for all experiments in the study and the necessary data sets are available as Supplemental Material.

3.1. Gas Furnace Model Identification

We selected Box and Jenkin’s benchmark [29] to test, analyse, and validate the modelling results. The data in this dataset were obtained as measurements from the combustion process in an industrial furnace. The input data is the methane flow rate, while the output data is the CO2 concentration in the combustion gases. The modelling data represent 296 measurements that were sampled with a sampling time of 9s. The modelling objective is to predict y(k) at a fixed iteration k based on the available knowledge of the behaviour of the system at previous time instances. The objective of the modelling is to predict the output based on the past input–output data and, since it is a dynamic system, a regression vector of appropriate dimensions and with appropriate delay is first chosen. The new y(k) is affected by the following input–output variables [y(k-1), ⋯y(k-n_y), u(k-$\tau$-1), ⋯u(k-$\tau$-n_u)]. In the above examples, $n_u=3$ and $n_y=3$ were chosen, and the delay $\tau$ was varied between 1 and 4.

For comparing the quality of performance, a mean square error (MSE) was chosen as a measure of the prediction quality

$$MSE=\frac{1}{N-\tau -n_u+1}{\displaystyle \sum }_{k=\tau +n_u}^N{\left[y\left(k\right)-\widehat{y}\left(k\right)\right]}^2$$

(16)

Calculating the MSE of the output variable tells us what the average estimation error of our model is. For example, $MSE=0.5$ represents an average estimation error of about $1.3\%$. From this, we conclude that the $MSE\le 0.5$ values for this benchmark are already a reasonably good result. In this study, we take the MSE index as a measure to compare the output prediction results of different fuzzy models against the predictions of the proposed NARX DCFS model structures.

In our first experiment, we used a WM FS (7) proposed and the WM system identification method. When modelling with FSs, some standard steps must be implemented at the start, such as the choice of the number of fuzzy sets M on each input and output variable, the shape of the membership functions, and the placement of these fuzzy sets on the definition ranges of the input and output variables. Triangular shapes of the membership functions, which were placed symmetrically to equally cover the entire definition ranges of the variables, were chosen. Another important step in modelling dynamical systems is the correct choice of the delay τ and orders n_u, n_y, of the system, which have important impacts on the structure of the regressor vector that is the input to the FS. Setting a suitable delay is related to prior knowledge of the process and is often chosen empirically. In our research, we set the delay between 0 and 4, and we mostly found a suitable result with a value of τ = 3.

We simulated five cases where we varied the number of M fuzzy sets on each input variable from 5, 7, 11, 13, and 15. Since a WM FS (7) has six input variables, the number of possible rules increases exponentially as the number of fuzzy sets M increases. Accuracy rises with larger M, but at the same time, the processing time t_CPU of the training algorithm increases dramatically. The performance result of this fuzzy model is shown in Figure 5, where Figure 5a depicts the original input u of the system, namely gas flow rate, and Figure 5b compares the output of the process y with the predicted output y_m of the identified model.

Figure 5b shows the error e between the model output and the predicted process output for the data used for training and for the input test signal. The MSE obtained of our model is 0.08962 by train input signal and 0.62777 by test input signal. By selecting a relatively large number of fuzzy sets M = 11 on the six inputs, we were faced with a large processing time of the training algorithm t_CPU = 29.32 s. An important conclusion from testing with WM FS is that, as the number of fuzzy sets M increases, the running time t_CPU of the training algorithm increases exponentially. Figure 5b,c shows that the MSE_test of the prediction with the test input signal is much larger than the MSE_train of the prediction with the training input.

Over the same set of input–output data, all three proposed NARX DCFS structures for output prediction were tested. The results are summarised and presented in

Table 2. Comparison of the performance of the proposed methods in Box–Jenkins system identification problem.

Model	$\mathit{\tau }$	M	MSEtrain	MSEtest	tCPU [s]
	3	5	0.28235	0.76069	1.10
	3	7	0.17843	0.54243	3.21
WM FS (7) [26]	3	11	0.08962	0.62777	29.32
	3	13	0.05681	0.55120	94.25
	3	15	0.04051	0.90677	261.52
	3	5	0.49303	0.38133	0.79
General NARX DCFS	3	15	0.11042	0.32413	1.83
	3	30	0.02411	0.28357	5.07
	3	5	0.45535	0.50022	0.77
Input-output NARX DCFS	3	15	0.12208	0.24639	1.18
	3	30	0.04537	0.4484	3.58
	3	5	0.38298	0.59655	0.50
Sub-model NARX DCFS	3	15	0.13069	0.34904	0.85
	3	30	0.03697	0.58240	1.11

. We first tested out the general NARX DCFS as defined in Definition 1, and with selected parameters, n_u = 3, n_y = 3, n₁ = 4, $m^1=m^2=3, m^3=2$, and L = 3 are presented in Figure 4a. Such a DCFS structure is characterised by seven FSs with three input variables and one output variable, which are organised into three levels. The inputs are ordered sequentially, as they are written, in the regression vector. For comparison purposes, we have kept the same delay τ but varied the number of fuzzy sets M. As expected, accuracy improved as M increased. Note that the processing time t_CPU of the training algorithm increased more slowly with increasing M than in the case of the WM FS, although M is much larger (30 compared to 11). Similarly, we tested the other two proposed NARX DCFS structures, as defined by Definitions 2 and 3 and presented in Figure 4b,c. We observe that the processing time t_CPU of the training algorithm is even smaller for the same number of fuzzy sets M. This is the result of fewer inputs to FSs at some levels, and it is a consequence of fewer FSs inputs at some levels, which, in turn, results in a slightly worse MSE performance measure.

The prediction results are given in Table 2. The results for the General NARX DCFS with M = 30, where we achieved the best performance measures MSE_train = 0.02411 and MSE_test = 0.28357, are highlighted. The prediction result of General NARX DCFS-based model is shown in Figure 6. The figure indicates that the testing error of the model is larger than the training model. Although we have chosen a relatively large number of fuzzy sets M = 30, the processing time of the training algorithm was relatively small t_CPU = 5.07s.

To evaluate the results, we made a comparative analysis of the results of the proposed NARX DCFS structure with other identification methods known from the literature. The methods are different by structure, by the fuzz systems used, and by the methods of learning from the data. We have selected some identification methods [3,4,15,30,31] based on fuzzy relational matrix identification to describe the inference mechanism of the FS. In these methods, we are dealing with comparable learning algorithms, which belong to non-iterative learning methods. We also selected two results based on Sugeno’s fuzzy inference [5,6]. One method is from the field of classical modelling [29]. The three methods described in papers [15,16] are characterised by the use of the FS decomposition principle. The results of these methods compared to NARX DCFS are presented in

Table 3. Identification results for the Box–Jenkins gas furnace.

Reference	No. of Variables	MSE
Box Jenkins [29]	-	0.71
Tong [3]	y(k-1), u(k-4)	0.469
Pedrycz [4]	y(k-1), u(k-4)	0.320
Xu [30]	y(k-1), u(k-4)	0.328
Costa Branco [31]	y(k-1), u(k-4)	0.312
Sugeno-Yasukawa [6]	y(k-1), y(k-2), y(k-3) u(k-1), u(k-2), u(k-3)	0.190
Takagi Sugeno [5]	y(k-1), u(k-3), u(k-4)	0.068
Golob ARX min-max [15]	y(k-1), u(k-4)	0.73
Golob ARX sum-prod [15]	y(k-1), u(k-4)	0.57
Golob DNF ARX [16]	y(k-1), u(k-4)	0.196
General NARX DCFS	y(k-1), y(k-2), y(k-3), u(k-4), u(k-5), u(k-6)	0.024

.

The table shows that the Sugeno-type FS [5] model is the closest to the General NARX DCFS model in terms of MSE. It should be noted that Sugeno FS is based on rules with linear functions on the consequent side of the rule. Verification result MSE = 0.024 of proposed General NARX DCFS indicates that identification methods based on hierarchical structures, such as the proposed NARX DCFSs, allow for avoiding the use of gradient identification methods and enable the efficient use of identification methods based on a rule extraction approach from data.

3.2. Nonlinear Dynamic Test Processes Identification

The four nonlinear dynamic test systems presented below serve as examples to illustrate the suitability of NARX DCFS for modelling various non-linear processes [2]. They cover different types of non-linear behaviour to demonstrate the universality of the approach.

• For a Hammerstein system, which is the typical coupling of a static non-linear function and a dynamic linear system, the example is given by a differential equation:

$$y\left(k\right)=0.01867·arctan\left[u\left(k-1\right)\right]+0.01746·arctan\left[u\left(k-2\right)\right]+1.7826·y\left(k-1\right)-0.8187·y\left(k-2\right)$$

(17)

• As the opposite, a Wiener system is a linear dynamic system in series with a static, non-linear function, and the example is given by a differential equation:

$$y\left(k\right)=arctan[0.01867·u\left(k-1\right)+0.01746·u\left(k-2\right)+1.7826·tan\left(y\left(k-1\right)\right)-0.8187·tan\left(y\left(k-2\right)\right)]$$

(18)

• A nonlinear differential equation (NDE) system is the approximation of a non-minimum phase system of a second order with parameters: gain 1, time constants 4 s and 10 s, and a zero at 0.25 s. Output feedback is a parabolic nonlinearity:

$$y\left(k\right)=-0.07289·\left[u\left(k-1\right)-0.2·y^2\left(k-1\right)\right]+0.09394·\left[u\left(k-2\right)-0.2·y^2\left(k-2\right)\right]+1.68364·y\left(k-1\right)-0.70469·y\left(k-2\right)$$

(19)

• A not separable dynamic (NSD) system has a nonlinearity which cannot be divided into a static non-linear part and a dynamic linear part. The behaviour of the system depends on the input variable:

$$y\left(k\right)=0.133·u\left(k-1\right)-0.0667·u\left(k-2\right)+1.5·y\left(k-1\right)-0.7·y\left(k-2\right)+u\left(k\right)·\left[0.1·y\left(k-1\right)-0.2·y\left(k-2\right)\right]$$

(20)

To identify NARX DCFSs, we excited the non-linear processes with an amplitude modulated pseudo random binary signal (APRBS), which is shown in Figure 7a. APRBS are designed to ensure good excitation of the process at different operating points and are, therefore, suitable for the excitation of non-linear processes. To test the NARX DCFSs, we generated the test signal presented in Figure 7b.

The test signal consists of step functions over the entire range of the input signal, followed by a ramp, and finally ending with a period of the sinusoidal signal.

Figure 8 shows the results of training the model with the General NARX DCFS structure to model the first of the four test non-linear processes, namely the Hammerstein system with the parameters given in Equation (17). Figure 8a shows the input signal used for training in a discrete time space from k = 0 to approximately k = 5000, as well as for model testing from k = 5000 to the end. The training and testing results are good, as confirmed by the acceptable MSE_train = 0.021 and MSE_test = 0.023 values. A comparison of the model output signal $y_m\left(k\right)$ and the output $y\left(k\right)$ of the process is given in Figure 8b, and the corresponding error $e\left(k\right)$ is given in Figure 8c. We continued testing with the test signal shown in Figure 9a. Comparison of the results in Figure 9b, where the model output and the process output are presented, shows a good match between the model and the process. The error in the figure is also within acceptable limits. In addition to errors due to transients in step excitation, the largest deviations of the model output from the process output are in the region k = 3400 to k = 3800 and in the k = 6000 to k = 6200.

The training and testing results of the other three nonlinear dynamic test processes implemented by Equations (18)–(20) were similar. A slightly worse prediction model was obtained in the case of NDE system identification, which resulted in a larger acceptable MSE_train and MSE_test. Comparison between all four NDPs and identified NARX DCFS model outputs on training data and test data for all four NDPs is presented in Figure 10.

4. Discussion

We were interested in the modelling results with different combinations of input signals into DCFS and, therefore, we were defined three different families of possible NARX DCFS structures. The first experiment showed, as illustrated by the modelling results presented in Table 2, that all three structures are useful. The results are comparable to the similar WM FS, which, however, requires much more CPU time to execute the training algorithm. Note that, in the case of the presented results we have chosen the simple structures presented in Figure 4, especially in the case of the input–output NARX DCFS structure and the sub-model NARX DCFS structure. Better prediction accuracy is obtained by increasing the number of M fuzzy sets on the input variables of fuzzy systems. In contrast to WM FS, increasing the number of M results in a reasonable increase in the time t_CPU required to execute the training algorithm. This is an important advantage of NARX DCFS over WM FS, as M can be scaled up to large numbers, much higher than 30, improving the accuracy of the model prediction.

Figure 8, Figure 9 and Figure 10 show that the results of the second experiment, namely modelling four different types of NDSs for prediction purposes, are also encouraging. The results also show some weaknesses. From the model output responses in Figure 9b and the associated error, an increase in the deviation is seen in the interval from k = 3400 to k = 3800 and, similarly, in the interval from k = 6000 to k = 6200. The error varies depending on the range of the input signal, and in this range, the dynamic system was not sufficiently excited during training. This is due to two reasons. The first is in the input excitation signal, which did not provide enough information in this region to successfully generate FSs’ rules to cover this area. The results can be improved by a more appropriate distribution of the amplitude levels of the excitation input signal presented in Figure 7a. The second reason is an inappropriate choice of the parameter M. If too small of a value is chosen, the training algorithm generates fewer rules, and the result is expressed in worse approximation properties. Comparison of the modelling results of the Hammerstein NDS in Figure 10a and the Wiener NDS in Figure 10b, obtained with the General NARX DCFS structure, shows similar properties. Small deviations are present, which differ from the operating point within the process definition range. Where the process has been adequately excited by the training procedure, the results are better, while others show a slight increase in the deviation between the model and the output of the process. The feature of these two NDSs, a separate non-linear static system and a linear dynamic system, proves to be suitable for modelling with NARX DCFS structures. The order of nonlinearity before the linear system (Hammerstein) or the nonlinearity after the linear system (Wiener) does not significantly affect the result. However, the decoupling of the nonlinear and linear system is not significant for the third and fourth test NDSs. Comparatively, the MSE results are slightly worse, but the deviation is not significant. From Figure 10c,d, we observe slightly more oscillations when tracking the model to the ramp excitation and harmonic signal. Either way, the results are useful for applications with single-step output predictions.

In Table 3, we compare the results of the NARX DCFS modelling with other similar methods. Some are based on fuzzy relational systems with non-iterative training algorithms, and others use FIS, where a combination of linear function rules and training from the data is implemented iteratively. There are several studies that also address neuro-fuzzy modelling of non-linear dynamical systems. We have reviewed recent publications, and the following is a brief overview. In [32], a review of fuzzy and neural prediction interval modelling of nonlinear dynamical systems is presented. The difference between the methods summarised in this review article and our method is that these methods predict a prediction interval, which has the advantage of specifying an interval of model output values. Most of these methods are based on various more-complex FIS structures, such as Type-2 fuzzy system, and use complex iterative training algorithms. Such a FIS was proposed in [33] for predicting the short-term wind power. The advantage of this method over the NARX DCFS is its ability to predict very precisely. The disadvantage lies in the complexity of the learning methods, as it uses the gravitational search algorithm (GSA) for an optimization interval Type-2 fuzzy system. The most similar in concept to our NARX DCFS model is the prediction model proposed in the paper [34] and called the deep neural network-based fuzzy cognitive maps model. It is characterised by both good prediction properties and good possibilities for linguistic interpretation of the results. An alternate function gradient descent, combined with a BP method, is used for a training algorithm. A partial similarity with the NARX DCFS structure can be detected in the study [35], where WM FSs and a combined learning method, consisting of the WM method and a complex-valued neural network, are used to predict dynamic nonlinear systems. The training algorithm is non-iterative. The study [36] presents a similar DCFS with an improved WM method adapted to solve complex classification problems. The use of ANNs to model and predict the outputs of nonlinear systems is a hot research area with many published studies. We highlight this research [37], which presents a new ANN training algorithm based on a hybrid artificial bee colony algorithm that has been used to efficiently approximate nonlinear static systems as well as dynamic systems. Due to the complexity, it is difficult to compare the results with our method. We also reviewed more recent studies [38,39,40,41] that used the same Box and Jenkin’s benchmark in experimental studies. Compared to NARX DCFS, the results are better or comparable. This is a consequence of the proposed more time-consuming training algorithms and optimization methods.

To conclude the discussion, let us summarise the advantages and disadvantages of the proposed method. The good features of the NARX DCFS models are:

• The hierarchical multilevel structure of DCFS allows the approximator of a non-linear function to be implemented with many more inputs in the regressor vector and without exponential growth in the number of rules.
• There are many different possibilities when creating a DCFS input vector from a regression vector. This allows us to have different structures. In this work, we present three of them.
• The training algorithm remains non-iterative, as is typical for DCFS. The N input–output data are processed only once.
• The ability to predict the output of the nonlinear system is satisfactory.

However, the following drawbacks are also present:

• The excitation signal must be chosen appropriately, providing both an adequate frequency bandwidth and a large variety of amplitude levels.
• The results of testing with a training signal are often much better than the results with a test signal, such as in Figure 6b.
• Once the NARX DCFS parameters and the input signal have been selected, the prediction results cannot be improved by repeating the training procedure.

The above features require further improvements.

5. Conclusions

This study provides a new NARX DCFS model for modelling dynamic nonlinear processes that is simple, allows fast training from input–output data, and does not suffer from the problem of rule explosion in the case when a nonlinear function approximator has many inputs. The main contributions of our study are the definitions of three structures, which differ in the number of hierarchical levels of the DCFS, the number of fuzzy subsystems at each level, the choice of the moving window, and the way in which all fuzzy subsystems are implemented. With the appropriate choice of parameters, practically any FS can be represented as a DCFS. Only minor changes were needed to adapt the DCFS training algorithm to model the NARX DCFS. The experimental results show that NARX DCFS structures are useful for modelling nonlinear dynamical systems where there are many inputs to the static nonlinearity approximator and the speed and convergence of the training algorithm are important, while the accuracy of the model prediction is less relevant.

Our future work will be oriented to research the impact of NARX DCFS system structure on prediction accuracy, as well as applying NARX DCFS models to real process modelling, identification, and control problems.