The Modeling of Time Series Based on Least Square Fuzzy Cognitive Map

Abstract

A novel design method for time series modeling and prediction with fuzzy cognitive maps (FCM) is proposed in this paper. The developed model exploits the least square method to learn the weight matrix of FCM derived from the given historical data of time series. A fuzzy c-means clustering algorithm is used to construct the concepts of the FCM. Compared with the traditional FCM, the least square fuzzy cognitive map (LSFCM) is a direct solution procedure without iterative calculations. LSFCM model is a straightforward, robust and rapid learning method, owing to its reliable and efficient. In addition, the structure of the LSFCM can be further optimized with refinements the position of the concepts for the higher prediction precision, in which the evolutionary optimization algorithm is used to find the optimal concepts. Withal, we discussed in detail the number of concepts and the parameters of activation function on the impact of FCM models. The publicly available time series data sets with different statistical characteristics coming from different areas are applied to evaluate the proposed modeling approach. The obtained results clearly show the effectiveness of the approach.

1. Introduction

The modeling and prediction of time series have been classic issues. Over the past few decades, researchers have developed many classical numeric models of time series such as standard exponential smoothing, Holt–Winters, autoregressive integrated moving average model (ARIMA) etc. These models of time series have made great progress in dealing with numerical forecasting problems. These models are difficult to use to solve the prediction problems with uncertain circumstances and lack of interpretability, which is difficult for people to intuitively understand. Fuzzy set theory can tolerate uncertainty and approximation [1], which has also been involved into the modeling of time series. Consequently, the fuzzy time series models have high interpretability, which gives the detailed numerical data some semantic meaning. Song and Chissom [2] presented the concept of fuzzy time series based on the fuzzy set theory. In recent years, some methods based on fuzzy time series have been presented to make predictions in many areas, such as stock price, university enrollments, economic growth, etc. [3,4].

As a soft computing tool, fuzzy cognitive maps (FCM) proposed by Kosko [5], can be used to capture the dynamic behaviors of a given system and implements the reasoning process based on knowledge representation [6]. A fuzzy cognitive map is a directed weighted graph and consists of concept nodes and directed weights between nodes, which demonstrate knowledge-based representation and inference process. Since each concept and weight of the FCM have semantic meaning [7,8,9], the FCM model is easy to comprehend for humans, due to their high interpretability for the complex system. The concepts of the FCM play a key factor when we want to construct an FCM for time series forecasting. The concepts of an FCM are either achieved by mechanism or clustering of the given time series data set [10,11,12]. The interrelationships between all concepts corresponding to weights are confirmed during the learning process. Hebbian-based [13,14,15] and population-based [16,17,18,19] methods are the mainstream for small-scale FCM learning problems. Moreover, the population-based methods outperform Hebbian-based methods in terms of time series prediction [20]. For large-scale FCM learning problems, Wu et al. [21] introduced Lasso regularization as the sparsity penalty term into the objective function to ensure the sparse structure of the resulting FCM. Lu et al. [22,23] transformed FCM learning into a convex optimization problem with constraints, and the maximum entropy terms were invoked to solve the optimization problem. The obvious problem in the above-mentioned FCM learning methods of time series is the need for iterative calculation and intensive computing to perform inference. These methods are time consuming.

Time series modeling with a fuzzy cognitive map has been applied in a range of quite diverse fields. Stach [24] proposed a method that combines FCM with granular to time series prediction realized both at the linguistic and numerical level. They take advantage of real-coded genetic algorithms to learn FCM. Papageorgiou and Froelich [25,26] used evolutionary-based and multi-step enhancement of the evolutionary algorithm to learn the FCM to cope with the forecasting of patient states in the case of pulmonary infections. Yang [27] resorted to wavelet transform to decompose original non-stationary time series into multivariate time series, then the high-order FCMs were applied to model and predict multivariate time series. Lu [28] proposed a high-order fuzzy cognitive map (HFCM) to model and predict time series. The structure of the HFCM generated works in an automatic fashion. Lu [29] proposed a hybrid algorithm based on an FCM for fuzzy time series prediction, in which a fuzzy C-means clustering algorithm was used to construct the framework of the FCM and a genetic algorithm was applied to learn the weights of the FCM. Homenda [30] adopted simplified fuzzy cognitive maps to construct the framework of time series modeling and introduced the selection criteria of concepts. In order to achieve a reasonable balance between complexity and accuracy, some simplification strategies which a posteriori remove nodes and weights were presented. Salmeron [31] proposed a dynamic optimization of the fuzzy cognitive maps for univariate time series forecasting. In this model, the concept of a sliding window was applied to train the predictive model. In order to improve the effectiveness of long-term prediction, an improved evolutionary approach for learning of the FCM model was proposed [32]. Froelich [7] proposed fuzzy grey cognitive maps (FGCMs) as a nonlinear predictive model to predict the multivariate interval-valued time series, in which evolutionary algorithm was also applied for learning FGCMs. Homenda [33] developed a methodology that joins a fuzzy cognitive map and moving window approach for time series prediction, in which the FCMs were optimized using a particle swarm optimization technique.

In this paper we investigate the FCM model and propose a novel method to construct the time series model based on FCM. The aim of this research is to improve the efficiency and accuracy of the time series model with FCM. Following are the major steps of the proposed model processing flow. First, fuzzy c-means clustering is applied to fuzzify the time series and form concepts from the given time series, i.e., each clustering center serves as a concept. Then, the weights of the FCM are learned based on historical data with the least square method. After that, in order to improve the model accuracy, the concepts are further refinement. The concepts obtained by fuzzy c-means clustering may not be the best representatives; therefore, we use a population-based optimization algorithm to further adjust the concepts. Finally, the developed FCM model makes the numerical prediction.

The exposure of the material is structured as follows. Section 2 gives a brief overview of the fuzzy c-means clustering and fuzzy cognitive map. Section 3 presents a learning of LSFCM. Subsequently, in Section 4, the outline of the proposed method along with its essential functional modules is thoroughly explained, in particular the refinement of concepts. Modeling and forecasting of time series are presented in detail in this section. In Section 5, we exploit six publicly available datasets to verify the validity and feasibility of the proposed method and the effect of parameters in the proposed model is discussed, based on accuracy of the constructed FCM prediction model. Finally, Section 6 provides some conclusions.

2. Preliminaries

2.1. Fuzzy c-Means Clustering

Fuzzy c-means [34], a fuzzy clustering method, allows one piece of data to belong to two or more clusters. The objective function of fuzzy c-means is as follows:

$$\arg minJ={{\displaystyle \sum }}_{i=1}^N{{\displaystyle \sum }}_{j=1}^n{u}_{ij}^m\Vert x_i-c_j{}^2\Vert$$

(1)

where $N$ is the number of data points. $n$ is the number of clusters. $m$ is a fuzzification coefficient, which is commonly set to 2. $x_i$is the ith data point. $c_j$ is the center of the $j$th cluster. $u_{ij}$is the degree of membership of $x_i$in the $j$th cluster, $u_{ij}\in \left[0,1\right]$, ${{\displaystyle \sum }}_{j=1}^n{u}_{ij}=1$.

Fuzzy c-means performs through an iterative optimization of the objective function with the update of membership $u_{ij}$ and the cluster centers $c_j$.

$$u_{ij}=\frac{1}{{{\displaystyle \sum }}_{k=1}^n{\left(\frac{\Vert x_i-c_j\Vert }{\Vert x_i-c_k\Vert }\right)}^{\frac{2}{m-1}}}$$

(2)

$$c_j=\frac{{{\displaystyle \sum }}_{i=1}^N{u}_{ij}^m{x}_i}{{{\displaystyle \sum }}_{i=1}^N{u}_{ij}^m}$$

(3)

This iteration will stop when it reaches a termination criterion, or until after a specified maximum number of iterations.

2.2. Fuzzy Cognitive Map

FCM can be understood as a graphical representation of the knowledge. One FCM consists of concepts (nodes) and directed weights. An example of an FCM with three concepts is illustrated in Figure 1. Concepts, also called nodes, $C_1,C_2,\cdots ,C_c$ represent the main features of the mapped system. The directed weights $w_{ij}$ labeled with fuzzy values show the strength of the causal conditions between the concepts. The square matrix $W=\left[w_{ij}\right]$ is the weight matrix and $i,j=1,2,\dots n$, where $w_{ij}\in \left[-1,1\right]$. If $w_{ij}>0$, it means that the increase in value of concept $C_j$ leads to an increase in concept $C_i$, and vice versa. If $w_{ij}<0$, it means that the increase in value of concept $C_j$ leads to a decrease in concept $C_i$ and vice versa. If $w_{ij}=0$, it means that there is no relationship between $C_j$ and $C_i$.

Given an FCM including $c$ concepts. The activation state values $A$ and the weight matrix $W$ are expressed as

$$A=\left[\begin{array}{c}{A}_{11} A_{12} \cdots A_{1c}\\ \begin{array}{c}{A}_{21} A_{22} \cdots A_{2c}\\ ⋮ ⋮ ⋮\\ A_{n1} A_{n2} \cdots A_{nc}\end{array}\end{array}\right],W=\left[\begin{array}{c}{w}_{11} w_{12} \cdots w_{1c}\\ \begin{array}{c}{w}_{21} w_{22} \cdots w_{2c}\\ ⋮ ⋮ ⋮\\ w_{c1} w_{c2} \cdots w_{cc}\end{array}\end{array}\right]$$

with $c$ being the number of concepts and $n$ being the number of samples. The elements in each row of the matrix $A$ are the state values of concepts at the corresponding time. The primary mission for constructing an FCM focuses on learning the weight matrix.

The reasoning process of the FCM is generally described as following,

$$A_{\left(t+1\right)i}=f\left({{\displaystyle \sum }}_{j=1}^c{A}_{tj}{w}_{ji}\right)$$

(4)

where $A_{tj}$ is the state value of $j$th concept at time $t$, $A_{\left(t+1\right)i}$ is the state value of $i$th concept at time $t+1$. In this equation, $w_{ji}$ is the fuzzy weight which shows the value of the influential intensity from concept $C_j$ to $C_i$. The activation function $f\left(·\right)$ is a nonlinear monotonically increasing function which squashes the weighted sum of the concepts’ states into a certain interval. One of the most widely used activation functions is the unipolar sigmoid function as given in following,

$$f\left(\mathrm{x}\right)=\frac{1}{1+e^{-\lambda x}}$$

(5)

where $\lambda >0$ is the shape parameter of the function. The state value of the sigmoid function is affected depending on these parameters.

3. The Learning of Least Square Fuzzy Cognitive Map

Different from traditional learning approaches for FCMs, the fuzzy weight matrix is obtained with the least square method in this study. Compared with the traditional method, learning the FCM with least square method is a one-time solution of the matrix equation rather than multi-iteration stochastic searching. The learning of the FCM least square method is abbreviated to LSFCM.

The activation function $f\left(x\right)=\frac{1}{1+e^{-\lambda x}}$ is a sigmoid function, let us consider the function,

$$A_{\left(t+1\right)i}=\frac{1}{1+e^{-\lambda {{\displaystyle \sum }}_{j=1}^c{A}_{tj}{w}_{ji}}}$$

(6)

$$\Rightarrow e^{-\lambda {\displaystyle \sum }_{j=1}^c{A}_{tj}{w}_{ji}}={(A_{\left(t+1\right)i})}^{-1}-1$$

$$\Rightarrow -\lambda {{\displaystyle \sum }}_{j=1}^c{A}_{tj}{w}_{ji}=\ln \left({(A_{\left(t+1\right)i})}^{-1}-1\right)$$

(7)

where $t$ is the discrete time, $t=1,2,3,\dots n-1$, and $c$ is the number of concepts. We have,

$${{\displaystyle \sum }}_{j=1}^c{A}_{tj}{w}_{ji}=-{\lambda }^{-1}\ln \left({(A_{\left(t+1\right)i})}^{-1}-1\right)$$

(8)

$0<A_{i\left(t+1\right)}<$. Let $Y_{\left(t+1\right)i}\triangleq -{\lambda }^{-1}\ln \left({(A_{\left(t+1\right)i})}^{-1}-1\right)$, then,

$${{\displaystyle \sum }}_{j=1}^c{A}_{tj}{w}_{ji}=Y_{\left(t+1\right)i}$$

(9)

The primitive function is$\left(A_t,A_{\left(t+1\right)i}\right)\to A_{i\left(t+1\right)}=f\left(A_{\left(t\right)}\right)$ and now we have $\left(A_t,Y_{\left(t+1\right)i}\right)\to$ $Y_{\left(t+1\right)i}=\varphi \left(A_t\right)$; the least square method is used to estimate $w_{ji}$. The sum of squares to be minimized, the fitness function, is described as follows,

$$J=arg\underset{w}{\min }\frac{1}{2}{{\displaystyle \sum }}_{i=1}^c{{\displaystyle \sum }}_{t=1}^{n-1}{(Y_{\left(t+1\right)i}-{{\displaystyle \sum }}_{j=1}^c{A}_{tj}{w}_{ji})}^2$$

(10)

The state values of concepts at the current time moment are described by X, X =${\left[A_1 A_2 \cdots A_{n-1}\right]}^T$; it results in a set of concepts’ state values at the next time moment $\overline{Y}={\left[A_2 A_3 \cdots A_n\right]}^T$, and $Y_i=-{\lambda }^{-1}\ln \left(1./{\overline{Y}}_i-1\right)$. Then, the fitness function can be written in matrix form,

$$\min :\frac{1}{2}{{\displaystyle \sum }}_{i=1}^c{(Y_i-X{W}_i)}^2$$

(11)

where $Y$ is a $\left(n-1\right)\times c$ matrix, $Y_i$is the $i$th column of $Y$, $W_i$ is the $i$th column of $W$. Then we can solve the estimated $W_i$ as

$$W_i^{*}=arg \underset{W_i}{min}\frac{1}{2}\Vert Y_i-X{W}_i{\Vert }^2+\beta \Vert W_i{\Vert }^2$$

(12)

The term $\Vert W_i{\Vert }^2$ is the L₂-norm of $W_i$ which reduces the collinearity of data, $\beta \ge 0$.

Let $\frac{\partial J_i}{\partial W_i}=0$, then $-X^{\mathrm{T}}\left(Y_i-X{W}_i\right)=0$, we obtain the solution,

$$W_i^{*}={(X^{\mathrm{T}}X+\beta I)}^{-1}{X}^{\mathrm{T}}{Y}_i$$

(13)

The vector $W_i$ is one column of weight matrix $W$. We the estimated $W$ as,

$$W^{*}={(X^{\mathrm{T}}X+\beta I)}^{-1}{X}^{\mathrm{T}}Y$$

(14)

We can obtain the estimated values of $W$ by solving (14); there is no iteration and result obtainable at one stroke.

Note that the values of the $W^{*}$ may not be all in the interval $\left[-1,1\right]$, when the least square method is used to learn the FCM. Owing to $Y=-{\lambda }^{-1}\ln \left(1./\overline{Y}-1\right)$, (14) can be rewritten as follows,

$$W^{*}={\lambda }^{-1}{(X^{\mathrm{T}}X+\beta I)}^{-1}{X}^{\mathrm{T}}\left(-\ln \left(1./\overline{Y}-1\right)\right)$$

(15)

It is shown from the formulas that the values of $W^{*}$ are linearly proportional to the shape parameter $\lambda$, viz., $W^{*}\propto {\lambda }^{-1}$. Suppose $\lambda =1$, then $W^1={(X^{\mathrm{T}}X+\beta I)}^{-1}{X}^{\mathrm{T}}\left(-\ln \left(1./\overline{Y}-1\right)\right)$; it is not sure that the values of elements of the matrix are all in the interval $\left[-1,1\right]$. In order to obtain the suitable estimated weight matrix, let ${\lambda }_1=\max \left\{\left|w_{ij}^1\right|\right\},i,j=1,2,\cdots c$ and $\lambda \ge {\lambda }_1$, then $W^{*}$ can be calculated by (15) and the values of elements of the matrix are all in the interval $\left[-1,1\right]$. Algorithm 1 explains the procedure of the LSFCM.

Algorithm 1: LSFCM

Input: Concepts $\mathit{C}$ and its state values matrix $\mathit{A}$.

1 X = ${\left[A_1 A_2 \cdots A_{n-1}\right]}^T$;

2 $\overline{Y}={\left[A_2 A_3 \cdots A_n\right]}^T$;

3 If $\mathrm{rank}\left(X^{\mathrm{T}}X\right)\ge c$ Then

4  $\beta =0$;

5 Else If $\mathrm{rank}\left(X^{\mathrm{T}}X\right)<c$ Then

6  $\beta >0$;

7 End If

8 $W^1={(X^{\mathrm{T}}X+\beta I)}^{-1}{X}^{\mathrm{T}}\left(-\ln \left(1./\overline{Y}-1\right)\right)$;

9 ${\lambda }_1=max\left\{\left|w_{ij}^1\right|\right\},i,j=1,2,\cdots c$;

10  $W^{*}={\lambda }^{-1}{(X^{\mathrm{T}}X+\beta I)}^{-1}{X}^{\mathrm{T}}\left(-\ln \left(1./\overline{Y}-1\right)\right)$, where $\lambda \ge {\lambda }_1$;

Return the estimated weight matrix $W^{*}$.

4. Modeling Time Series Using LSFCM

This section covers how to use the LSFCM to model time series. The entire process of the modeling is outlined in Figure 2. First, the data are normalized and divided into the training set and test set in chronological order. Second, fuzzy time series are constructed from numerical data based on a fuzzy c-means clustering algorithm. Third, the FCM is learned using the least square method to efficiently obtain the weight matrix of the FCM according to the historical data. After the FCM model is established, we can compute the forecasted values using the LSFCM model.

In order to improve the prediction accuracy of the developed LSFCM model, the model is further optimized by refinement of the concepts. The evolutionary optimization algorithm serves as the optimization vehicle to refine the concepts which have been developed by the fuzzy c-means clustering. The degree of fuzzy membership of each datum to the clusters will be altered and the ensuing LSFCM process is carried out again. The optimization process is driven by the minimization of the reconstruction error. Through these measures, the quality of the FCM model improves. The optimization procedure which is called LSFCM-ref is an iterative process and is repeated until the desired number of iterations is reached. In what follows, the proposed model of forecasting time series will be presented in detail.

4.1. Constructing the LSFCM Model

In this section, the scheme of the proposed time series modeling and prediction approach based on LSFCM and fuzzy c-means clustering is detailed.

When we attempt to model time series with FCMs, the numeric time series should be mapped to a fuzzy time series first. Suppose that the FCM has c concepts and $\mathit{X}=\{\mathit{x}(t\left)\right\}, t=1,2,..n$ is a numeric time series, then one numerical datum $\mathit{x}\left(t\right)$ is mapped to a fuzzy set which has c elements in the domain, i.e., $f_{\mathrm{c}}:\mathit{x}\to <\mathit{v},\mathit{u}>$, where $\mathit{v}=\left[v_1,v_2,\cdots ,v_{\mathrm{c}}\right]$ is the matrix of the concepts and $\mathit{u}=[u_1,u_2,\cdots ,u_{\mathrm{c}}]$ is the fuzzy membership degree of the $\mathit{x}$ corresponding to $c$ concepts. Then the numeric datum can be present in the fuzzy set form,

$$U_i=\frac{{\mathit{u}}_{i1}}{v_1}+\frac{{\mathit{u}}_{i2}}{v_2}+\dots +\frac{{\mathit{u}}_{ic}}{v_c}$$

(16)

Let the given time series$\mathit{X}=\{\mathit{x}(t\left)\right\}, t=1,2,..n$ normalize to [0,1]. The normalized time series is redefined as $\overline{\mathit{X}}=\left\{\overline{\mathit{x}}\left(t\right)\right|\overline{\mathit{x}}\left(t\right)\in \left[\mathbf{0},\mathbf{1}\right]\},t=1,2,..n$. The time series $\overline{X}$ is divided into the training set and test set. The training set containing the $L$ datum is used to establish the FCM model; the test set containing the $\mathrm{n}-L$ datum is used to estimate the models.

In order to fuzzify the time series, a fuzzy c-means clustering algorithm is used to obtain clustering centers and the fuzzy memberships, the number of the cluster center c is predefined, and the clustering centers are taken as the concepts. We have the concepts’ vectors $\mathit{V}={[{\mathit{v}}_1,{\mathit{v}}_2,\cdots ,{\mathit{v}}_{\mathrm{c}}]}^{\mathrm{T}}$ and the corresponding fuzzy membership matrix $\mathit{U}=[{\mathit{u}}_{1i},{\mathit{u}}_{2i},\cdots ,{\mathit{u}}_{\mathrm{c}i}],i=1,2,\dots n$. Then the fuzzy membership matrix is used to construct the FCM. There is an internal fuzzy logical relationship between the neighbors of the partition matrix U, viz. $U\left(\mathrm{t}\right)\to U\left(\mathrm{t}+1\right)$. Accordingly, we have n − 1 input–output data pairs, as shown,

$${\left(\begin{array}{c}\begin{array}{c}{u}_{11}\\ u_{21}\end{array}\\ ⋮\\ u_{\mathrm{c}1}\end{array}\right)}^T\to {\left(\begin{array}{c}\begin{array}{c}{u}_{12}\\ u_{22}\end{array}\\ ⋮\\ u_{\mathrm{c}2}\end{array}\right)}^T;{\left(\begin{array}{c}\begin{array}{c}{u}_{12}\\ u_{22}\end{array}\\ ⋮\\ u_{\mathrm{c}2}\end{array}\right)}^T\to {\left(\begin{array}{c}\begin{array}{c}{u}_{13}\\ u_{23}\end{array}\\ ⋮\\ u_{\mathrm{c}3}\end{array}\right)}^T;\cdots ;{\left(\begin{array}{c}\begin{array}{c}{u}_{1\left(n-1\right)}\\ u_{2\left(n-1\right)}\end{array}\\ ⋮\\ u_{\mathrm{c}\left(n-1\right)}\end{array}\right)}^T\to {\left(\begin{array}{c}\begin{array}{c}{u}_{1n}\\ u_{2n}\end{array}\\ ⋮\\ u_{\mathrm{c}n}\end{array}\right)}^T$$

(17)

Referring to (4), the fuzzy logical relationship between$U\left(\mathrm{t}+1\right)$ and $U\left(\mathrm{t}\right)$ can be described as follows,

$$U_{\left(t+1\right)i}=f\left({{\displaystyle \sum }}_{j=1}^c{U}_{tj}{w}_{ji}\right);t=1,2,\cdots n-1,i=1,2,\cdots c$$

(18)

The formula can be expressed in vector form,

$$U_{t+1}=f\left(U_t\mathrm{W}\right)$$

(19)

According to the LSFCM construction process, the concepts $V$ and the fuzzy membership matrix $U$ are used as raw material to construct FCM and the least square method is used to learn the FCM. According to Algorithm 1, the weight matrix $W$ can be calculated by the following equation,

$$W={({\overline{U}}^{\mathrm{T}}\overline{U})}^{-1}{\overline{U}}^{\mathrm{T}}\overline{Y},$$

(20)

where $\overline{U}={\left[U_1,U_2, \cdots U_{n-1}\right]}^T$ and ${\overline{Y}}_i=-{\lambda }^{-1}\ln \left(U_{i\left(t+1\right)}{}^{-1}-1\right), t=1,2,\cdots n-1$. The elements of the weight matrix can be restricted to the given interval by adjusting the values of $\lambda$. After the processes above, we can obtain a fuzzy cognitive map model of the given time series.

Any datum $\overline{\mathit{x}}\left(t\right)\left(t=1,2,..n\right)$ belonging to the time series can be transformed into the form of fuzzy membership values $u_{it}\left(i=1,2,\cdots c;t=1,2,\cdots n\right)$ by the fuzzy c-means algorithm. When the structure of the LSFCM is formed, the dynamic characteristics of the given time series can be interpreted or predicted by the LSFCM. We can predict future fuzzy membership values based on the LSFCM model. According to (19), the forecast membership value ${\widehat{U}}_t=f\left(U_{t-1}\mathrm{W}\right)$, then the forecast datum, can be reconstructed as the following,

$${\widehat{\mathit{x}}}_t=\frac{{{\displaystyle \sum }}_{i=1}^c{\widehat{u}}_{ti}^m{v}_i}{{{\displaystyle \sum }}_{i=1}^c{\widehat{u}}_{ti}^m},$$

(21)

where $v_i$ is the concept of FCM, which is calculated by the fuzzy c-means clustering algorithm.

For the quantitative evaluation of LSFCM model quality, the performance index root mean squared error (RMSE) is defined by the following,

$$\mathrm{RMSE}=\sqrt{\frac{1}{n}{{\displaystyle \sum }}_{t=1}^n\Vert {\mathit{x}}_t-{\widehat{\mathit{x}}}_t{\Vert }^2},$$

(22)

where $n$ is the number of time series data, ${\mathit{x}}_t$ is the actual value and ${\widehat{\mathit{x}}}_{t }$ is the predictive value of time series at the time t. Obviously, the smaller the RMSE is, the higher the quality of the model is.

4.2. Refinements of LSFCM Model

The concepts of the LSFCM are obtained by the fuzzy c-means algorithm; however the performance index is prediction error. The initial concepts do not necessarily lead to the minimum prediction error. Thus, the idea is to optimize the LSFCM model by adjusting the position of the concepts (cluster centers) for a better performance index. The partition matrix $\mathit{U}$ includes the fuzzy membership of each sample to the concepts. Supposing that the position of the concepts is relocated in the domain of definition, on this basis, there will be acting in response to cause a new partition matrix. The structure of the LSFCM will be relearned based on the new partition matrix and concepts, then that impacts directly on the prediction error. The procedure of migration of concepts is depicted in Figure 3. The initial/updated population of concepts is used to construct the LSFCM model. Then, the reconstruction errors are calculated and the optimal concept whose LSFCM model has minimum errors is selected. The best concept is output after the iteration, which is used to construct the optimal LSFCM model. The refinements of concepts will be detailed discussed in the following.

How to change the position of concepts plays a pivotal role in the process. In this study, the stochastic seeking strategy is considered to adjust the coordinate of all the concepts. It can be believed that the clustering centers of the fuzzy c-means algorithm are already close to the best concepts of the FCM but still make further progress. The position of the clustering center is in the neighborhood of the optimum position which makes the LSFCM model show a better performance index. To refine the FCM model, the initial concepts obtained by fuzzy c-means are used as the starting point. To modify the concepts, a sub-defined interval is introduced for the coordinates of the concepts. The radius of the sub-defined interval is described as follows,

$$r=\frac{\max \left(\mathrm{data}\right)-\min \left(\mathrm{data}\right)}{2c}$$

(23)

Each concept moves to a new position from its current position in a particular range rather than the entire domain and the new position $v_i^{\prime }\in \left[v_i-r, v_i+r\right]$. Hence, the efficiency of adjustment is higher. The probability of a new concept position in the domain is a normal distribution. In other words, the closer the position to the initial concept, the higher the probability selected. The strategy of concept adjustment is illustrated in Figure 4. The initial population of concepts is a normal distribution. ${\mathit{\phi }}_{\mathrm{i}}\left({\mathit{v}}_{\mathrm{i}}\right)$ $\left(i=1,2\cdots c\right)$ is the probability density function that the position in the domain selects as the new concept. Each concept can move to the best position around the initial position. Although the movement of each concept in each coordinate is random, the movement of all concepts has a preference or tendency. The final result is to minimize the performance index RMSE, viz. $\frac{1}{N}{{\displaystyle \sum }}_{t=1}^N\Vert x_t-{\widehat{x}}_t{\Vert }^2\to \min$.

The positions of the concepts are adjusted by the evolutionary algorithm Particle swarm optimization (PSO). The fuzzy membership matrix $\mathit{U}$ is refreshed corresponding to the new concepts. Once the optimal concepts are obtained, the final LSFCM model (LSFCM-ref) is formed.

5. Results

In this section, publicly available real-world time series are used to demonstrate the effectiveness of the proposed method. All FCM learning methods used for comparative purposes were tested under the same conditions. As previously described, the FCM model is governed mainly by the number of concepts. The shape parameter λ of the sigmoid function also has effect on the performance of the FCM model. In order to evaluate the quality of the proposed methods, there were two purposes of these experiments involving the time series. The first one was that the quantitative evaluation of the impact on the prediction accuracy of the proposed approach being brought by the number of concepts c and the parameter of the sigmoid function. The second one was comparison with other FCM learning methods and the classical forecasting models.

For all the time series, the normalized time series data were split into two samples: training set and test set. The fuzzification coefficient $m$ was set to two. There eight time series data sets are applied to evaluate and analyze the developed LSFCM and LSFCM-ref model of time series. The eight time series are given in

Table 1. Experimental data.

NO.	Time Series	Size
(a)	Mean daily flow, Oldman River near Brocket, Jan 01, 1988 to Dec 31, 1991	1461
(b)	Monthly Boston armed robberies Jan.1966-Oct.1975	118
(c)	Nigeria power consumption	123
(d)	Annual water use in New York city, liters per capita per day, 1898–1968	71
(e)	Annual sheep population (1000s) in England and Wales 1867–1939	73
(f)	Births per 10,000 of 23-year-old women, U.S., 1917–1975	59
(g)	Daily open prices of the S&P 500 stock index, May 16, 2017 to May 15, 2020	756
(h)	Daily close prices of the Dow Jones industrial index, May 1, 2019 to April 29, 2020	252

and plotted in Figure 5. In each time series, the first 80% of data were used for model training, the last 20% of data were left for testing purposes.

5.1. The Influence of the Parameters of the Proposed Model

The number of the concepts is highly sensitive to the performance of FCM models [35]. The parameters $c$ and $\lambda$ are particular discussed in the experiments. Optimal values of the parameters are established by inspecting the predictive error (RMSE) of the training set. The values of $c$ range from 2, 3,..., 20, while several representative values of $\lambda$ are selected. Figure 6 plots line chart groups of RMSE of corresponding different parameter values. Examination of the data shown in Figure 6 leads us to the conclusion that the value of RMSE becomes substantially lower when increasing the number of concepts; however, the value becomes slightly higher or even higher still when the number of concepts exceeds a certain number. In other words, the predictive accuracy does not continuous increase with the increasing number of $c$. For example, regarding the Oldman time series, the optimal value of $c$ is 10. Whereas for the Annual water time series, once the number of $c$ moves past eight, there will be no substantial improvement of the reconstruction error, or it will become even worse. The topology of the FCM is more complex with the growth of the number of concepts. Therefore, the optimal value of $c$ is selected under comprehensive consideration of the topological complexity of the FCM and the predictive accuracy.

Letting $\lambda =1$, we can calculate the weight matrix,

$$W^1=\left[\begin{array}{cccccc}5.0174& -8.6692& -7.2166& -5.1458& -8.3879& -7.8711\\ -9.1504& 4.0895& -7.9115& -8.6434& -4.0537& -7.1306\\ -5.7369& -6.1585 & 3.5081& -3.8613 & -5.6261 & -4.4168\\ -3.3991& -6.2223& -2.6043 & 1.4834 & -5.8400& -5.0940 \\ -7.1786 & -3.1379& -5.1858& -6.4852& -0.3340& -2.2041\\ -6.1414& -4.5083& -4.1489& -5.3633& -2.6671& 2.4939 \end{array}\right].$$

Then, letting $\lambda =\max \left|W^1\right|=9.1504$, we have the weight matrix conforming to the definition

$$W^{9.1504}=\left[\begin{array}{cccccc}0.5483 & -0.9474& -0.7887& -0.5624& -0.9167& -0.8602\\ -1.0000& 0.4469& -0.8646& -0.9446& -0.4430& -0.7793\\ -0.6270& -0.6730 & 0.3834& -0.4220 & -0.6148 & -0.4827\\ -0.3715& -0.6800& -0.2846 & 0.1621 & -0.6382& -0.5567\\ -0.7845 & -0.3429& -0.5667& -0.7087& -0.0365& -0.2409\\ -0.6712& -0.4927& -0.4534& -0.5861& -0.2915& 0.2725\end{array}\right].$$

If setting $\lambda =10$, we can also calculate the weight matrix

$$W^{10}=\left[\begin{array}{cccccc}0.5017 & -0.8669& -0.7217& -0.5146& -0.8388& -0.7871\\ -0.9150& 0.4090& -0.7912& -0.8643& -0.4054& -0.7131\\ -0.5737& -0.6158 & 0.3508& -0.3861 & -0.5626 & -0.4417\\ -0.3399& -0.6222& -0.2604 & 0.1483 & -0.5840& -0.5094\\ -0.7179 & -0.3138& -0.5186& -0.6485& -0.0334& -0.2204\\ -0.6141& -0.4508& -0.4149& -0.5363& -0.2667& 0.2494\end{array}\right].$$

5.2. Comparison with Other Methods

For comparison, a subset of classical forecasting models was selected for comparison included the Naive, Standard Exponential Smoothing (SES), Holt–Winters and ARIMA models. The details are described in the following sections. The prediction accuracy of the conventional FCMs models was calculated. The prediction accuracy was calculated and listed in

Table 2. Comparison with other FCM models.

Data	PSO-FCM	GA-FCM	LSFCM	LSFCM-Ref
(a) (c = 10)	11.66	12.4	11.48	10.47
(b) (c = 10)	38.5	38.4	41.6	37.46
(c) (c = 7)	8328	8386	8999	7753
(d) (c = 7)	23.5	23.6	26.1	22.8
(e) (c = 7)	79.28	80.39	80.55	73.23
(f)(c = 6)	11.8	13.1	12.89	10.51
(g)(c = 9)	34.14	35.02	34.58	33.21
(h)(c = 10)	543.82	545.83	542.79	518.88

. As can be seen from Table 2, the prediction precision of the time series with FCM learning by the Particle swarm optimization (PSO) or Genetic algorithm (GA) methods is slightly higher than the developed LSFCM prediction model, but less than the LSFCM-ref model. There are no significant differences for all the time series in these models: PSO-FCM, GA-FCM and LSFCM. However, the results with LSFCM-ref model are more accurate than others.

With refinement of concepts, we can further improve the accuracy of the LSFCM model. The comparison shows that the developed FCM prediction model can produce satisfactory quality for the time series. It can be seen in Figure 7 that the location of the concepts changes with refinements. In the coordinates, the black star points are the initial position of the concepts, and the red circles are the final position of optimized concepts.

Furthermore, the comparison between the proposed approach and the classical prediction approach (Naive, SES, Holt–Winters and ARIMA) is presented in

Table 3. Comparison with the classic prediction models.

Data	Naive	ARIMA	SES	Holt–Winters	LSFCM	LSFCM-Ref
(a)	13.12	10.51	22.13	13.14	11.48	10.47
(b)	61.92	60.67	62.79	64.02	41.6	37.46
(c)	11,435	10945	14320	10605	8999	7753
(d)	27.31	28.50	28.33	28.78	26.1	22.8
(e)	82.34	82.65	82.34	105.27	80.55	73.23
(f)	13.14	12.63	13.15	13.65	12.89	10.51
(g)	46.13	46.49	96.55	46.75	34.58	33.21
(h)	949.61	939.64	1464.15	1042.99	542.79	518.88

. As can be noted, the LSFCM approach outperformed the classical prediction models and the LSFCM has the minimum RMSE compared with the other prediction methods.

6. Conclusions

A novel FCM learning approach for time series was proposed in this study. The model contains two stages. The first one is to construct an FCM model with the least square method, viz. LSFCM. The second one is to optimize the LSFCM with refinement of concepts to improve the prediction accuracy. Fuzzy c-means clustering is applied to fuzzify the given time series data set to automatically extract the FCM’s concepts and fuzzy membership matrix. Two important contributions of the proposed method are learning the FCM with the least square method and refinement of concepts. The former can help the FCM learning eliminate the strenuous iterative computation. The latter can help the FCM obtain the optimal concepts by relocating the position of concepts. In addition, the stochastic strategy is applied to refine the concepts of the LSFCM. The influence of the parameters of the FCM on the prediction accuracy is analyzed. The number of concepts significantly impacts the prediction accuracy of the LSFCM prediction model. When the number of clusters is assigned to the optimal value, the ideal numerical prediction accuracy can be obtained. The parameters of activation function have no great effect on the prediction accuracy of the LSFCM model but have a big impact on the weight matrix. From the results of experimentation, LSFCM is a very competitive model for time series modeling and forecasting.