Forecasting Based on High-Order Fuzzy-Fluctuation Trends and Particle Swarm Optimization Machine Learning

: Most existing fuzzy forecasting models partition historical training time series into fuzzy time series and build fuzzy-trend logical relationship groups to generate forecasting rules. The determination process of intervals is complex and uncertain. In this paper, we present a novel fuzzy forecasting model based on high-order fuzzy-ﬂuctuation trends and the fuzzy-ﬂuctuation logical relationships of the training time series. Firstly, we compare each piece of data with the data of theprevious day in a historical training time series to generate a new ﬂuctuation trend time series (FTTS). Then, we fuzzify the FTTS into a fuzzy-ﬂuctuation time series (FFTS) according to the up, equal, or down range and orientation of the ﬂuctuations. Since the relationship between historical FFTS and the ﬂuctuation trend of the future is nonlinear, a particle swarm optimization (PSO) algorithm is employed to estimate the proportions for the lagged variables of the fuzzy AR (n) model. Finally, we use the acquired parameters to forecast future ﬂuctuations. In order to compare the performance of the proposed model with that of the other models, we apply the proposed method to forecast the Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX) time series datasets. The experimental results and the comparison results show that the proposed method can be successfully applied in stock market forecasting or similarkinds of time series. We also apply the proposed method to forecast Shanghai Stock Exchange Composite Index (SHSECI) and DAX30 index to verify its effectiveness and universality.


Introduction
It is well known that historic time series imply the behavior rules of a given phenomenon and can be used to forecast the future of the same event [1].Many researchers have developed time series models to predict the future of a complex system, e.g., regression analysis [2], the autoregressive moving average (ARIMA) model [3], the autoregressive conditional heteroscedasticity (ARCH) model [4], the generalized ARCH (GARCH) model [5], and so on.However, these methods require some premise hypotheses,such as a normality postulate [6], etc.Meanwhile, models that satisfied the constraints precisely can miss the true optimum design within the confines of practical and realistic approximations.Therefore, Song and Chissom proposed the fuzzy time series forecasting model [7][8][9].Since then, the FTS model has been applied for forecasting in many nonlinear and complicated forecasting problems, e.g., stock market [10][11][12][13], electricity load demand [14,15], project cost [16], and the enrollment at Alabama University [17,18], etc.A vast majority of FTS models are first-order and high-order fuzzy AR (autoregressive) models.These models can be considered as an equivalent version of AR (n) based on fuzzy lagged variables of time series.Most of these fuzzy time series models follow the basic steps as Chen proposed [19]: Step 1: Define the universe U and the number and length of the intervals; Step 2: Fuzzify the historical training time series into fuzzy time series; Step 3: Establish fuzzy logical relationships (FLR) according to the historical fuzzy time series and generate forecasting rules based on fuzzy logical groups (FLG); Step 4: Calculate the forecast values according to the FLG rules and the right-hand side (RHS) of the forecasted point.
In order to improve the accuracy of such kinds of FTS models, researchers have proposed other improved models based on Chen's model.For example, concerning the determination of suitable intervals, Huarng [20] proposed averages and distribution methods to determine the optimal interval length.Huarng and Yu [21] proposed an unequal interval length method based on ratios of data.Since then, many studies [20,[22][23][24][25][26][27] have been carried out for the determination of the optimal interval length using statistical theory.Some authors even employed PSO techniques to determine the length of the intervals [12].In fact, in addition to the determination of intervals, the definition of the universe of discourse also has an effect on the accuracy of the forecasting results.In these models, minimum data value, maximum data value, and two suitable positive numbers must be determined to make a proper bound of the universe of discourse.
Concerning the establishment of fuzzy logical relationships, many researchers utilize artificial neural networks to determine fuzzy relations [28][29][30].The study of Aladag et al. [28] is considered as a basic high-order method for forecasting based on artificial neural networks.Meanwhile, fuzzy AR models are also widely used in many fuzzy time series forecasting studies [11,12,[31][32][33][34][35].In order to reflect the recurrence and the weights of different FLR in fuzzy AR models, Yu [36] used a chronologically-determined weight in the defuzzification process.Cheng et al. [37] used the frequencies of different right-hand sides (RHS) of FLG rules to determine the weight of each LHS.Furthermore, many studies employed the adaptive fuzzy inference system (ANFIS) method [38] for time series forecasting.For example, Primoz and Bojan [39] defined soft constraints based on ANFIS to discrete optimization for obtaining optimal solutions.Egrioglu et al. [40] proposed a model named the modified adaptive network based fuzzy inference system (MANFIS).Sarica et al. [41] developed a model based on an autoregressive adaptive network-based fuzzy inference system (AR-ANFIS), etc.Since 2013, considering the impacts of specification errors, fuzzy auto regressive and moving average (ARMA) time series forecasting models were proposed [42,43].The initial first-order ARMA fuzzy time series forecasting model was proposed by Egrioglu et al. [42] based on the particle swarm optimization method.Kocak [43] developed a high-order ARMA fuzzy time series model based on artificial neural networks.Kocak [44] used both fuzzy AR variables and fuzzy MA variables to increase the performance of the forecasting models.
A forecasting model is used to predict the future fluctuation of a time series based on current values.Therefore, we present a novel method to forecast the fluctuation of a stock market based on a high-order AR (n) fuzzy time series model and particle swarm optimization (PSO) arithmetic.Unlike existing models, the proposed model is based on the fluctuation values instead of the exact values of the time series.Firstly, we calculate the fluctuation for each datum by comparing it with the data of its previous day in a historical training time series to generate a new fluctuation trend time series (FTTS).Then, we fuzzify the FTTS into fuzzy-fluctuation time series (FFTS) according to the up, equal, or down range of each fluctuation data value.Since the relationship between historical FFTS and future fluctuation trends is nonlinear, a PSO algorithm is employed to estimate the proportion of each AR and MA parameter in the model.Finally, we use these acquired parameters to forecast future fluctuations.The advantages provided by the proposed method are as follows.
The remaining content of this paper is organized as follows: Section 2 introduces some preliminaries of fuzzy-fluctuation time series based on Song and Chissom's fuzzy time series [7][8][9].Section 4 introduces the process of the PSO machine learning method.Section 4 describes a novel approach for forecasting based on high-order fuzzy-fluctuation trends and the PSO heuristic learning process.In Section 5, the proposed model is usedto forecast the stock market using TAIEX datasets from 1997 to 2005, SHSECI from 2007 to 2015, and the year 2015 of the DAX30 index.Conclusions and potential issues for future research are summarized in Section 6.

Preliminaries
Song and Chissom [7][8][9] combined fuzzy set theory with time series and presented the following definitions of fuzzy time series.In this section, we will extend fuzzy time series to fuzzy-fluctuation time series (FFTS) and propose the related concepts.Definition 1.Let L = l 1 , l 2 , ..., l g be a fuzzy set in the universe of discourse U ; it can be defined by its membership function, µ L : U → [0, 1] , where µ L (u i ) denotes the grade of membership of u i , U = {u 1 , u 2 , ...u i , ..., u l }.
The fluctuation trends of a stock market can be expressed by a linguistic set L = l 1 , l 2 , ..., l g , e.g., let g = 3, L = {l 1 , l 2 , l 3 } = {down, equal, up}.The element l i and its subscript i is strictly monotonically increasing [45], so the function can be defined as follows: f : l i = f (i).To preserve all of the given information, the discrete L = l 1 , l 2 , ..., l g also can be extended to a continuous label L = {l a |a ∈ R}, which satisfies the above characteristics.Definition 2. Let X(t)(t = 1, 2, ..., T) be a time series of real numbers, where T is the number of the time series.Y(t) is defined as a fluctuation time series, where Y(t) = X(t) − X(t − 1), (t = 2, 3, ..., T).Each element of Y(t) can be represented by a fuzzy set S(t)(t = 2, 3, ..., T) as defined in Definition 1. Then we call time series Y(t) to befuzzified into a fuzzy-fluctuation time series (FFTS) S(t).Definition 3. Let S(t) (t = 2, 3, ..., T) be a FFTS.If S(t) is determined by S(t − 1), S(t − 2), ..., S(t − n), then the fuzzy-fluctuation logical relationship is represented by: S(t) ← S(t − 1), S(t − 2), ..., S(t − n) (1 and it is called the nth-order fuzzy-fluctuation logical relationship (FFLR) of the fuzzy-fluctuation time series, where S(t) is called the left-hand side(LHS) and S(t − n), ..., S(t − 2)S(t − 1) is called the right-hand side(RHS) of the FFLR.This model can be considered as an equivalent of the auto-regressive model of AR (n), defined in Equation (2): where φ k (k = 1, 2, .., n) represents the portion of S(t − k) for calculating the forecast is φ k , ε t is the calculation error, and S(t) is introduced to preserve more information, as described in Definition 1.

PSO-Based Machine Learning Method
In this paper, the particle swarm optimization (PSO) is employed to estimate the parameters in Equation (2).The PSO method was introduced as an optimization method for continuous nonlinear functions [46].It is a stochastic optimization technique, which is similar to social models, such as birds flocking or fish schooling.During the optimization process, particles are distributed randomly in the design space and their location and velocities are modified according to their personal best and global best solutions.Let m+1 represent the current time step, x i,m+1 , v i,m+1 , x i,m , v i,m indicate the current position, current velocity, previous position, and previous velocity of particle i, respectively.The position and velocity of particle i are manipulated according to the following equations: where w is an inertia weight which determines how much the previous velocity is preserved [47], c 1 and c 2 are the self-confidence coefficient and social confidence coefficient, respectively, Rand() ∈ [0, 1] is a random number, and p i,m and p g,m are the personal best position found by particle i and the global best position found by all particles in the swarm up to time step m, respectively.Let the design space be defined by [x min , x max ].If the position of particle i exceeds the boundary, then v i,m+1 is modified as follows:

A Novel Forecasting Model Based on High-Order Fuzzy-Fluctuation Trends
In this paper, we propose a novel forecasting model based on high-order fuzzy-fluctuationtrends and a PSO machine learning algorithm.In order to compare the forecasting results with other researchers' work [10,11,27,36,[48][49][50][51], the authentic TAIEX (Taiwan Stock Exchange Capitalization Weighted Stock Index) is employed to illustrate the forecasting process.The data from January 1999 to October 1999 are used as training time series and the data from November 1999 to December 1999 are used as testing dataset.The basic steps of the proposed model are shown in Figure 1.
Symmetry 2017, 9, 124 4 of 15 the current position, current velocity, previous position, and previous velocity of particle i, respectively.The position and velocity of particle i are manipulated according to the following equations: where w is an inertia weight which determines how much the previous velocity is preserved [47], c1 and c2 are the self-confidence coefficient and social confidence coefficient, respectively, Let the design space be defined by [ ] min max x ,x . If the position of particle i exceeds the boundary, then , 1

A Novel Forecasting Model Based on High-Order Fuzzy-Fluctuation Trends
In this paper, we propose a novel forecasting model based on high-order fuzzy-fluctuationtrends and a PSO machine learning algorithm.In order to compare the forecasting results with other researchers' work [10,11,27,36,[48][49][50][51], the authentic TAIEX (Taiwan Stock Exchange Capitalization Weighted Stock Index) is employed to illustrate the forecasting process.The data from January 1999 to October 1999 are used as training time series and the data from November 1999 to December 1999 are used as testing dataset.The basic steps of the proposed model are shown in Figure 1.
Step 2: Establish nth-order FFLRs for the forecasting model According to Equation ( 2), each S(t)(t ≥ n + 2) can be represented by its previous n days' fuzzy-fluctuation number.Therefore, the total of FFLRs for historical training data is pn = T − n − 1.
Step 3: Determine the parameters for the forecasting model based on the PSO machine learning algorithm In this paper, the PSO method is employed to determine the parameters and a general error ε in Equation ( 2).The personal best position and global best position are determined by minimizing the root of the mean squared error (RMSE) in the training process: where n denotes the number of values forecasted, forecast(t) and actual(t) denote the forecasting value and actual valueat time tin the training process, respectively.For determined φ k (k = 1, 2, .., n) and ε, the forecast value at time t is as follows: The pseudo-code for the PSO-based machine learning algorithm is shown in Appendix A.
Step 4: Forecast test time series For each data in the test time series, its future number can be forecasted according to Equation ( 7), based on the observed data point X (t − 1), its n-order fuzzy-fluctuation trends, and the parameters generated from the training dataset.
Step 1: Calculate the fluctuation trend for each element in the historical training dataset of TAIEX1999.Then, we use the whole mean of the fluctuation numbers of the training dataset to fuzzify the fluctuation trends into FFTS.For example, the whole mean of the historical dataset of TAIEX1999 from January to October is 85.That is to say, len = 85.For X(1) = 6152.43and X(2) = 6199.91,Y(2 ) = 47.48,S(2) = 3, and S(2) ≈ 2.5586.On the other hand, based on the previous data X(1) and the accurate fuzzy number S(2), X(2) can be obtained by: X(1) Step 2: Based on the FFTS from 5 January 1999 to 30 October shown in Table 1, establish the nth-order FFLRs for the forecasting model.For example, suppose n=6, the following FFLRs of FFTS can be generated: Step 3: Replace each error ε t in Equation ( 8) with one and the same ε.Let the number of iterations itern = 100, the inertia weight w = 0.7298, the self-confidence coefficient and social confidence coefficient c 1 = c 2 = 1.4962, and use the PSO algorithm listed in Figure 1 to determine the parameters and ε.In the PSO process, each element in the generalizedEquation (8) is a particle and their personal best and global best positions are determined by the RMSE of the actual values and forecast values.The obtained global best parameters are shown in Table 1.
Step 4: Use the obtained global best parameters in Table 1 to forecast the test dataset from 1 November 1999 to 30 December.For example, the forecasting value of the TAIEX on 8 November 1999 is calculated asfollows: Firstly, according to the fuzzy-fluctuation trends (2,1,1,1,2,1) and the parameters in Table 1, the forecasted continuous labeled fuzzy-fluctuation number is: The other forecasting results are shown in Table 2 and Figure 2.
The forecasting performance can be assessed by comparing the difference between the forecasted valuesand the actual values.The widely used indicators in time series models comparisons are the mean squared error (MSE), root of the mean squared error (RMSE), mean absolute error (MAE), mean percentage error (MPE), etc.To compare the performance of different forecasting results, the Diebold-Mariano test statistic (S) is also widely used.These indicators are defined by Equations ( 9)-( 13): Step 3: Replace each error t ε in Equation ( 8) with one and the same ε .Let the number of iterations itern = 100, the inertia weight w = 0.7298, the self-confidence coefficient and social confidence coefficient c1 = c2 =1.4962, and use the PSO algorithm listed in Figure 1 to determine the parameters Step 4: Use the obtained global best parameters in Table 1to forecast the test dataset from 1 November 1999 to 30 December.For example, the forecasting value of the TAIEX on 8 November 1999 is calculated asfollows: Firstly, according to the fuzzy-fluctuation trends (2,1,1,1,2,1) and the parameters in Table 1, the forecasted continuous labeled fuzzy-fluctuation number is: The other forecasting results are shown in Table 2 and Figure 2.
where n denotes the number of values forecasted, forecast(t) and actual(t) denote the predicted value and actual valueat time t, respectively.S is a test statistic of the Diebold method which is used to compare predictive accuracy of two forecasts obtained by different methods.Forecast1 represents the dataset obtained by method 1, and Forecast2 represents another dataset from method 2. If S>0 and |S| > Z = 1.64, at the 0.05 significant level, Forecast2 has better predictive accuracy than Forecast1.In order to compare the forecasting results with different parameters such as the number n of the nth-order and the element number g of linguistic set used in the fluctuation fuzzifying process, different experiments under different parameters were carried out.Each typeof experiment was repeated 30 times.The forecasting errors of the averages for the experiments are shown in Tables 3  and 4. In Table 4, g = 3 represents that the linguistic set is {down, equal, up}, g = 5 means {greatly down, slightly down, equal, slightly up, greatly up}, g = 7 means {very greatly down, greatly down, slightly down, equal, slightly up, greatly up, very greatly up}, and "none" means that the fluctuation values will not be fuzzified at all.
From Tables 3 and 4, we can see that the RMSEs are lower when n is equal to six or more.With respect to the parameter g, obviously, the fuzzified fluctuation trends perform better than none fuzzified ones, and it is proper to let g = 3.
Letting n=6 and g=3, we employ the proposed method to forecast the TAIEX from 1997 to 2005.The forecasting results and errors are shown in Figure 3 and Table 5.
Where n denotes the number of values forecasted, forecast(t) and actual(t) denote the predicted value and actual valueat time t, respectively.S is a test statistic of the Diebold method which is used to compare predictive accuracy of two forecasts obtained by different methods.Forecast1 represents the dataset obtained by method 1, and Forecast2 represents another dataset from method 2. If S>0 and

S > Z =
, at the 0.05 significant level, Forecast2 has better predictive accuracy than Forecast1.With respect to the proposed method for the sixth-order, the MSE, RMSE, MAE, and MPE are 9862.33,99.31, 75.22, and 0.01, respectively.In order to compare the forecasting results with different parameters such as the number n of the nth-order and the element number g of linguistic set used in the fluctuation fuzzifying process, different experiments under different parameters were carried out.Each typeof experiment was repeated 30 times.The forecasting errors of the averages for the experiments are shown in Tables 3  and 4.  In Table 4, g = 3 represents that the linguistic set is {down, equal, up}, g = 5 means {greatly down, slightly down, equal, slightly up, greatly up}, g = 7 means {very greatly down, greatly down, slightly down, equal, slightly up, greatly up, very greatly up}, and "none" means that the fluctuation values will not be fuzzified at all.
From Tables 3 and 4, we can see that the RMSEs are lower when n is equal to six or more.With respect to the parameter g, obviously, the fuzzified fluctuation trends perform better than none fuzzified ones, and it is proper to let g = 3.
Lettingn=6 and g=3, we employ the proposed method to forecast the TAIEX from 1997 to 2005.The forecasting results and errors are shown in Figure 3 and Table 5.Table 6 shows a comparison of RMSEs for different methods for forecasting the TAIEX1999.From this table, we can see that the performance of the proposed method is acceptable.The greatest advantage of the proposed method is that it puts forward a method relying completely on the machine learning mechanism.Though RMSEs of some of the other methods outperform the proposed method, they oftenneed to determine complex discretization partitioning rulesor use adaptive expectation models to justify the final forecasting results.The method proposed in this paper is simpler and easily realized by a computer program.The Proposed Method 99.12 -**The proposed method has better predictive accuracy than the method at the 5% significance level.

Forecasting DAX30
The German DAX30 index is an important stock index in Germany.The RMSEs of different models forecastingyear 2015 of DAX30 are shown in Table 7. ** The proposed method has better predictive accuracy than the method at the 5% significancelevel.
From Table 7, we can see that the proposed method can successfully predict the DAX30 index.Table 6 shows a comparison of RMSEs for different methods for forecasting the TAIEX1999.From this table, we can see that the performance of the proposed method is acceptable.The greatest advantage of the proposed method is that it puts forward a method relying completely on the machine learning mechanism.Though RMSEs of some of the other methods outperform the proposed method, they oftenneed to determine complex discretization partitioning rulesor use adaptive expectation models to justify the final forecasting results.The method proposed in this paper is simpler and easily realized by a computer program.[49] 102.11 0.21 Chen and Chen's Method(2015) [11] 103.9 0.36 Chen and Chen's Method(2015) [10] 92 −0.42 Zhao et al.'s Method(2016) [27] 110.85 1.08 The Proposed Method 99.12 -** The proposed method has better predictive accuracy than the method at the 5% significance level.

Forecasting DAX30
The German DAX30 index is an important stock index in Germany.The RMSEs of different models forecastingyear 2015 of DAX30 are shown in Table 7. From Table 7, we can see that the proposed method can successfully predict the DAX30 index.

Forecasting SHSECI
The SHSECI (Shanghai Stock Exchange Composite Index) is the most famous stock market index in China.In the following, we apply the proposed method to forecast the SHSECI from 2007 to 2015.For each year, the authentic datasets of historical daily SHSECI closing prices from January to October are used as the training data, and the datasets from November to December are used as the testing data.The RMSEs of forecast errors are shown in Table 8.From Table 8, we can see that the proposed method can successfully predict the SHSECI stock market.

Conclusions
In this paper, a novel forecasting model is proposed based on high-order fuzzy-fluctuation logical trends and the PSO machine learning method.The proposed method is based on the fluctuations of the time series.The PSO method is employed to look for the best parameters to minimize the RMSE of a historical training dataset.Experiments show that the parameters generated from the training dataset can be successfully used for future datasets as well.In order to compare the performance with that of other methods, we take the TAIEX1999 as an example.We also forecasted TAIEX1997-2005, DAX30 2015 and SHSECI 2007-2015 to verify its effectiveness and universality.In the future, we will consider other factors which might affect the fluctuation of the stock market, such as the trade volume, the beginning value, the end value, etc.We will also consider the influence of other stock markets, such as the Dow Jones, the NASDAQ, the M1b, and so on.
is a random number, and , i m p and g,m p are the personal best position found by particle i and the global best position found by all particles in the swarm up to time step m, respectively.

Figure 1 .
Figure 1.Flowchart of our proposed forecasting model.Figure 1. Flowchart of our proposed forecasting model.
that is 6152.43+ (2.5588 − 2) × 85 ≈ 6199.91.In this way, the historical training dataset can be represented by a fuzzified fluctuation dataset as shown in Appendix B.

Table 1 .
Global best parameters obtained using PSO for training dataset.

Table 1 .
2,...,n) φ and ε .In the PSO process, each element in the generalizedEquation (8) is a particle and their personal best and global best positions are determined by the RMSE of the actual values and forecast values.The obtained global best parameters are shown in Table 1.Global best parameters obtained using PSO for training dataset.
With respect to the proposed method for the sixth-order, the MSE, RMSE, MAE, and MPE are 9862.33,99.31, 75.22, and 0.01, respectively.

Table 4 .
Comparison of forecasting errors for different linguistic sets (n=6).

Table 4 .
Comparison of forecasting errors for different linguistic sets (n=6).

Table 6 .
A comparison of RMSEs for different methods for forecasting the TAIEX1999.

Table 7 .
RMSEs of the forecast errors for year 2015 of DAX30.

Table 6 .
A comparison of RMSEs for different methods for forecasting the TAIEX1999.

Table 7 .
RMSEs of the forecast errors for year 2015 of DAX30.
** The proposed method has better predictive accuracy than the method at the 5% significancelevel.

Table 8 .
RMSEs of forecast errors for SHSECI from 2007 to 2015.

Table A2 .
Historical training data and fuzzified fluctuation data of TAIEX1999.