An Improvement of GM (1, N) Model Based on Support Vector Machine Regression with Nonlinear Cross Effects

Abstract

This paper presents GM (1, N) models with linear cross effect and nonlinear cross effect, and discusses the difference of driving factors between these two types of models to solve the cross effects of GM (1, N) model. The model with a linear cross effect in this paper preserves the solution of whitenization in the GM (1, 1) model. While the model with nonlinear cross effect integrates the sequences of systemic features, driving factors, and the cross effect of these driving factors. While applying support vector machine (SVM) regression, it transfers the nonlinear relationship among these sequences to a linear relationship. To test the GM (1, N) model that is based on support vector machine (SVM) with nonlinear effect, the study applies it to forecast the total output of the pharmaceutical industry. The range of the data is selected from 2005–2017, which the data from 2005–2013 are used to fit into the model. The GM (1, N) model based on SVM with nonlinear cross effect achieves 0.6566 and 0.2956 in its fitted total of relative error and the forecast total of relative error, respectively. The new model presents a more accurate analysis on fitting and forecast precision than the classic GM (1, N) model and GM (1, N) with the linear cross effect model.

1. Introduction

Grey system theory is an interdisciplinary scientific area that Deng first introduced in the early 1980s [1,2]. Since then, the theory has become quite popular with its ability to deal with the systems that have partially unknown parameters [3]. Presenting superiority to conventional statistical models, grey models only require a limited amount of data to estimate the behaviour of unknown systems [4]. As an important part of grey system theory, grey forecasting modeling method has been widely used in recent years because of its advantages in small sample data modeling, simplicity, and easy calculation. In theoretical and methodological research, scholars at home and abroad have improved the single variable grey forecasting model that is represented by GM (1, 1) model. The problems of expansion and optimization have been studied and abundant achievements have been achieved. Deng proposed GM (1, N) model on the basis of GM (1, 1) model, and applied it to analyze the coordination of economic, social, and scientific and technological systems in order to analyze the influence of a set of related factors on the system characteristic sequence and to predict the change trend of the system characteristic sequence [5]. Although the GM (1, N) model has dynamic characteristics that are similar to the GM (1, 1) model, it is easy to cause large errors in prediction due to the missing of a practical precise solution in the grey differential equation [6]. When comparing with the GM (1, 1) model, the GM (1, N) model has new features as: It implies the driving effects among each influencing factor.

Regarding the feature of GM (1, N) model above, a question has been raised as to how the driving effects of influencing factors can be quantified, Xiao and Deng (2001) modified GM (1, N) model with applying linear regression on definition formulas [7]. In this basis, a study by Mao and Chirwa (2006) attempted to assume $\sum b_i{x}_i^{\left(1\right)}$, which is the driving factor as a grey constant. A solution is conducted by applying the albino differential equation [8]. However, the solution that is based on albino differential equation is pointed as being complex in GM (1, N) (Hsu, 2009; Wu et al., 2013) [9,10]. Tien (2009) applied the Simpson integration rule to simplify the albino differential equation in GM (1, N) model [11].

Apart from the problem of quantified driving effects, some studies also addressed the time-lag problem in the current GM (1, N) model [12,13] (Jones et al., 2004; Wu and Chen, 2005). The study that was conducted by Hao et al. (2011) developed a new grey GM (1, N) model with the features of multi-variable and hysteresis [14]. This model is able to find the optimal lagging parameter via applying particle swarm optimization (PSO). PSO benefits a more accurate prediction in GM (1, N) model, but Han et al. (2013) improved the application of PSO in GM (1, N) model, and created another time-lagging model [15]. This model is able to determine the order of GM (1, N) while applying grey correlation and an average absolute relative error. More studies continue to improve GM (1, N).

Wu et al. (2013) applied fractional order and generated operators into GM (1, N), and improved the model to GM (1, N, T), which has a better accuracy in prediction [16]. While Wu et al. (2015) addressed a certain situation where the existing accumulative sequence of correlated variables is too large, the influencing factors might not be able to be transferred as grey constant [17]. To improve this weakness, (Yuan et al., 2016) attempted to improve initial value and background value in GM (1, N) model, in order to reduce the impact of the correlated variable on prediction [18].

Current studies are keen to improve the feasibility of GM (1, N) model in reality. Feng et al. (2014) mentioned that the coefficient of the albinism differential equation should be dynamic, thus their study developed the MGM (1, N) model [19]. It enables dynamically considering the coefficient of accumulative sequence. Ma and Liu (2015) analysed the discrete GM (1, N) model, and applied a convolution integral method to improve the model [20]. The improved model is able to solve the GM (1, N) model with multiple variables. To be more realistic, Kayacan et al. (2010) considered the relationship among accumulative sequences as non-linearization [21]. However, it has not discussed the cross effect among the driving factors. Wang et al. (2011) also improved the linear relationship in accumulative sequence to non-linear relationship [22]. When comparing with the study by Kayacan et al. (2010), this study considered the cross effect among the driving factors, but it still has a strong constraint that assumes the relationship among the accumulative sequence is linearization.

In reality, accumulative sequences in systems, driving factors, and cross effects may be linearization or non-linearization. It is more common to present a non-linear relationship [23,24,25,26,27,28]. Therefore, current GM (1, N) models are hardly difficult to address this. Our study aims to apply support vector machine (SVM) regression to discuss the non-linear cross effect in GM (1, N) model. The study is structured, as follows: Section 2 presents the GM (1, N) model with cross effect. Section 3 improved GM (1, N) model to the new GM (1, N) model with the non-linear cross effect. Section 4 applies a numerical example to test the new model. The final chapter concludes the entire study.

2. GM (1, N) Model with Cross Effect

2.1. Classic GM (1, N) Model

Definition 1.

Assume$X_1^{(0)}=\left(x_1^{(0)}(1),x_1^{(0)}(1),\dots ,x_1^{(0)}(n)\right)$is the sequence of the data in a system, and the correlation factor sequences are as follows:

$$\begin{array}{c}{X}_2^{(0)}=\left(x_2^{(0)}(1),x_2^{(0)}(2),\dots ,x_2^{(0)}(n)\right)\\ X_3^{(0)}=\left(x_3^{(0)}(1),x_3^{(0)}(2),\dots ,x_3^{(0)}(n)\right)\\ ⋮\\ X_N^{(0)}=\left(x_N^{(0)}(1),x_N^{(0)}(2),\dots ,x_N^{(0)}(n)\right)\end{array}$$

(1)

$X_i^{(1)}$ denotes the first accumulated sequence of $X_i^{(0)}\left(i=1,2,\dots ,N\right)$ and “first accumulated sequence” means $X_i^{(1)}={\displaystyle \sum _{k=1}^n}{X}_{ik}^{(0)}$. $Z_1^{(1)}$ denotes the sequence of mean consecutive neighbours generation of $X_1^{(1)}$, which is:

$$x_1^{(0)}(k)+a{z}_1^{(1)}(k)={\displaystyle \sum _{i=2}^N{b}_i{x}_i^{(1)}}(k)$$

(2)

Equation (2) is the classic GM (1, N) model.

2.2. GM (1, N) Model with Cross Effect

Definition 2.

Assume that the cross effect of the system feature sequence in driving factors, exists and the cross effect of driving factors is the correlation coefficients of the regression, thus, the GM (1, N) model with cross effect can be presented as follows:

$$\begin{array}{l}{x}_1^{(0)}(k)+a{z}_1^{(1)}(k)={\displaystyle \sum _{i=2}^N{b}_i{x}_i^{(1)}}(k)+{\displaystyle \sum _{r,s\in I,r\ne s}f(b_{rs})x_r^{(1)}(k)x_s^{(1)}(k)}\\ \end{array}$$

(3)

In Equation (3), $z_1^{(1)}(k)=\frac{x_1^{(1)}(k)+x_1^{(1)}(k-1)}{2}$, $a$ is the evolution coefficient of the system feature sequence, and ${\displaystyle \sum _{i=2}^N{b}_i}{x}_i^{(1)}(k)$ is the sum of individual influential effect of each driving factor. ${\displaystyle \sum _{r,s\in I,r\ne s}f(b_{rs})x_r^{(1)}(k)x_s^{(1)}(k)}$ is the cross effect of driving factors that have relative relationship; where $r$ and $s$ refer to two different influence factors with cross-effect, and $I$ is a set of all influence factors in the GM (1, N) model. $f(b_{rs})$ is the cross-effect regression of relative coefficients, which $f(0)=0$. If $b_{rs}=0$, each driving factor is uncorrelated. This is defined as the GM (1, N) model. While $f(b_{rs})$ has two situations, it may be a linear regression, but it may be the nonlinear regression in most of the scenarios. Therefore, we firstly consider $f(b_{rs})$ as a linear regression, and secondly consider it as a non-linearization.

Definition 3.

GM (1, N) model with linear cross effect can be presented as Equation (4), which is a whitenization equation of GM (1, N) model with cross-effect.

$$\begin{array}{l}\frac{d{x}_1^{(1)}(t)}{dt}+a{z}_1^{(1)}(t)={\displaystyle \sum _{i=2}^N{b}_i}{x}_i^{(1)}(t)+{\displaystyle \sum _{r,s\in I,r\ne s}f(b_{rs})x_r^{(1)}}(t)x_s^{(1)}(t)\\ \end{array}$$

(4)

Lemma 1.

The solution of GM (1, N) model with linear cross effect can be presented, as follows.

$$x_1^{(1)}(t)=e^{-at}(x_1^{(1)}(0)-t({\displaystyle \sum _{i=2}^N{b}_i}{x}_i^{(1)}(0)+{\displaystyle \sum _{r,s\in I,r\ne s}f(b_{rs})x_r^{(1)}(0)x_s^{(1)}(0))}+{\displaystyle \sum _{i=2}^N{\displaystyle \int b_i{x}_i^{(1)}{e}^{at}dt}}+{\displaystyle \sum _{i=2}^N{\displaystyle \int f(b_{rs})x_r^{(1)}(t)}}{x}_s^{(1)}(t)e^{at}dt)\infty$$

(5)

Proof. 

Assume that $X_1^{(0)}$ is the system feature sequence, $X_i^{(0)}(i=2,3,\dots ,N)$ is the sequence of driving factors, $X_i^{(1)}$ is the 1-AGO sequence of $X_i^{(0)}$, and $Z_1^{(1)}$ is the sequence of mean consecutive neighbours generation of $X_1^{(1)}$.

$$B=\left[\begin{array}{ccccc}-z_1^{(1)}(2)& x_2^{(1)}(2)& \cdots & x_N^{(1)}(2)& f(b_{rs})x_r^{(1)}(2)x_s^{(1)}(2)\\ -z_1^{(1)}(3)& x_2^{(1)}(3)& \cdots & x_N^{(1)}(3)& f(b_{rs})x_r^{(1)}(3)x_s^{(1)}(3)\\ ⋮& ⋮& \cdots & ⋮& ⋮\\ -z_1^{(1)}(n)& x_2^{(1)}(n)& \cdots & x_N^{(1)}& f(b_{rs})x_r^{(1)}(n)x_s^{(1)}(n)\end{array}\right]$$

$$Y=\left[\begin{array}{c}{x}_1^{(0)}(2)\\ x_1^{(0)}(3)\\ ⋮\\ x_1^{(0)}(n)\end{array}\right]$$

Assume that $f(b_{rs})=b_{rs}{r}_{rs}+c$, where $r_{rs}$ is the relative coefficient of the driving factors $r$ and $s$, and $b_{rs}$ and $c$ are the undetermined parameters.

There is $\tilde{B}=\left[b_{2,}{b}_3,\dots ,b_N,b_{rs},c\right]$.

Thus, the least square estimation of the parameter list of $\widehat{a}={\left[a,b_2,b_3,\dots ,b_N,b_{rs},c\right]}^T$ should satisfy:

• When $n=N+3$, $\left|\tilde{B}\right|\ne 0$, $\widehat{a}={\tilde{B}}^{-1}Y$;
• When $n>N+3$, and $\left|{\tilde{B}}^T\tilde{B}\right|\ne 0$, $\widehat{a}={({\tilde{B}}^{T}B)}^{-1}{B}^{T}Y$;
• When $n<N+3$, and $\tilde{B}$ is line non-singular matrix, thus, the non-singular matrix of $\tilde{B}$ is $\tilde{B}=IC$, where the generalized inverse matrix of $\tilde{B}$ is ${\tilde{B}}^{+}=C^T{(C{C}^T)}^{-1}I$.

It will have $\widehat{a}=C^T{(C^{T}C)}^{-1}Y={\tilde{B}}^T{(\tilde{B}{\tilde{B}}^T)}^{-1}Y$. □

Due to $x_i^{(1)}(k)$ and $x_r^{(1)}(k)x_s^{(1)}(k)$ being related to time ‘$k$’, thus the following step aims to solve ${\displaystyle \sum _{i=2}^N{\displaystyle \int b_i{x}_i^{(1)}{e}^{at}dt}}$ and ${\displaystyle \sum _{i=2}^N{\displaystyle \int f(b_{rs})x_r^{(1)}(t)}}{x}_s^{(1)}(t)e^{at}dt$. To simplify this step, we assume that the change of $x_i^{(1)}(k)$ is slight, and ${\displaystyle \sum _{i=2}^N{b}_i{x}_i^{(1)}(k)}$ and ${\displaystyle \sum _{r,s\in I,r\ne s}f(b_{rs})x_r^{(1)}(k)x_s^{(1)}(k)}$ are grey constant. By following the solution in GM (1, 1) model, we can achieve a similar time response of GM (1, N) model with a linear cross effect, which is:

$$\begin{array}{l}{\widehat{x}}_1^{(1)}(k+1)=(x_1^{(1)}(1)-\frac{1}{a}({\displaystyle \sum _{i=2}^N{b}_i}{x}_i^{(1)}(k+1)+{\displaystyle \sum _{r,s\in I,r\ne s}f(b_{rs})x_r^{(1)}(k+1)x_s^{(1)}(k+1)})e^{-ak}+\frac{1}{a}({\displaystyle \sum _{i=2}^N{b}_i}{x}_i^{(1)}(k+1)\\ +{\displaystyle \sum _{r,s\in I,r\ne s}f(b_{rs}})x_r^{(1)}(k+1)x_s^{(1)}(k+1))\end{array}$$

(6)

Generally, the change of accumulate sequence in $x_i^{(1)}(k)$ is random and enormous. This leads the above solution to be unrealistic. Therefore, we develop another solution in the following chapter.

3. GM (1, N) Model with Nonlinear Cross Effect

As the assumption in the previous chapter, where we assumed linearization is the relationship among $x_1^{(0)}(k)$, $x_1^{(1)}(k-1)$, $x_i^{(1)}(k),i=2,\dots ,N$, and $x_r^{(1)}(k)x_s^{(1)}(k),r,s\in I,r\ne s$. This assumption is unrealistic, as, in reality, non-linearization is more common. Therefore, we attempt to improve the linearization to non-linearization in the relationship among the accumulate sequence of system feature, accumulate sequence of driving factors, and accumulate cross effect of driving factors. Subsequently, we apply SVM regression to determine the nonlinear mapping relations among $x_1^{(0)}(k)$, $x_i^{(1)}(k),i=2,\dots ,N$ and $x_r^{(1)}(k)x_s^{(1)}(k),r,s\in I,r\ne s$.

SVM is a type of supervised machining learning models with the associated certain algorithms through regression analysis (Sebald and Bucklew, 2001). It applies VC dimension theory and structural risk minimization. These two theories enable SVM to better solve problems with a small sample size, non-linearization, high dimension, and local minimum (Cortes and Vapnik, 1995). Therefore, applying SVM can solve the new GM (1, N) model with a nonlinear cross effect, where it transfers the nonlinear relationship of driving factors to linearization. To address this point, we assume that a sample is $\left\{x_i,y_i\right\}$, where $x_i\in R^m$ is the input variable and $y_i\in R$ is the output variable, $y_i=f(x_i),i=1,2,\dots ,m$. Additionally, $f(x)$ is an unknown function.

If the regression function $f(x_i)$ is non-linearization, to solve this problem, it can create a nonlinear mapping $\phi (x)$ to map the sample from the input space to the high-dimensional feature space $H$, and then develop linear SVM regression in feature space $H$. The sample should satisfy the criteria in order to operate this procedure, where $y_i(x_i\cdot \omega +b)-1+{\zeta }_i\ge 0$. $\zeta$ is slack variable and $\zeta \ge 0$. These criteria can guarantee nonlinear problems that are to be transferred as an optimal function that aimed to find the maximum boundary of SVM, and it can be presented as the following equation.

$$\{\begin{array}{l}\min \frac{1}{2}{\Vert \omega \Vert }^2+c{\displaystyle \sum _{i=1}^n{\zeta }_i}\\ y_i(x_i\cdot \omega +b-1+{\zeta }_i\ge 0,{\zeta }_i\ge 0,i=1,2,\dots ,n\end{array}$$

(7)

In Equation (7), $c>0$ is penalty parameter that is used to control the complexity of models. Through solving the above equation, we can achieve a discriminant function, which is: $f(x)=\mathrm{sgn}({\displaystyle \sum _{i=1}^n{a}_i}{y}_{i}K(x_i,x)+b)$, where $K(x_i,x)$ is kernel function.

To develop a solution, our study applies radial basis function and also assumes some input variables include the accumulation of $(k-1)$ and $x_1^{(1)}(k-1)$ in system feature series, $x_i^{(1)}(k),i=2,\dots ,N$ in driving factors, and $x_r^{(1)}(k)x_s^{(1)}(k),r,s\in I,r\ne s$ in driving factors with the cross effect. Additionally, the output variable is determined as $x_1^{(0)}(k)$.

4. Empirical Analysis

To test the feasibility of the GM (1, N) model with a non-linear cross effect in prediction, our study applies it to forecast the total output of the pharmaceutical industry. In detail, the influencing factors that we select include investment in industrial capital, total product sales revenue, total profit from product sales, total health costs, gross domestic product, total factor productivity, average disposable income of urban residents, and consumer price index. The data are drawn from Chinese Industrial Yearbook, Industrial Statistics Yearbook, Chinese Medical Yearbook, and China Statistical Yearbook. The range of the data is selected from 2005–2017, where the data from 2005–2013 are used to fit into models, and the data from 2014–2017 are used to develop the prediction in the fitted model. $X_1$ denotes the number of total output of the pharmaceutical industry in this numerical example. $X_2$ is total investment in the pharmaceutical industry. $X_3$ is total product sales revenue. $X_4$ is total profit from product sales. $X_5$ is total health costs. $X_6$ is gross domestic product. $X_7$ is average disposable income of residents. $X_8$ is consumer price index. The results of the models fitting and prediction are presented in the following tables.

In

Table 1. Original data of the choice variables from 2005 to 2017 in China.

Year	${\mathit{X}}_1/\mathbf{billions}$	${\mathit{X}}_2/\mathbf{billions}$	${\mathit{X}}_3/\mathbf{billions}$	${\mathit{X}}_4/\mathbf{billions}$	${\mathit{X}}_5/\mathbf{billions}$	${\mathit{X}}_6/\mathbf{billions}$	${\mathit{X}}_7/\mathbf{RMB}$	${\mathit{X}}_8$
2005	4250	661.8	258,442	13,065	8659.91	184,937.4	10,493	101.8
2006	5019	769.0	313,592	19,504	9843.34	216,314.4	11,759	101.5
2007	6362	885.0	399,717	27,155	11,573.97	265,810.3	13,786	104.8
2008	7875	1155.6	500,020	30,562	14,535.40	314,045.4	15,781	105.9
2009	9443	1858.6	547,000	45,333	17,541.92	340,902.8	17,175	99.3
2010	12,350	2119.0	697,744	53,050	19,980.39	401,512.8	19,109	103.3
2011	15,624	2330.3	841,830	61,396	24,345.91	473,104.0	21,810	105.4
2012	18,770	2617.1	929,292	61,910	28,119.00	519,470.1	24,565	102.6
2013	22,297	3139.3	1,029,150	62,831	31,668.95	568,845.2	18,310.8	102.6
2014	25,798	3991.5	1,107,033	68,155	35,312.40	636,138.7	20,167.1	102.0
2015	29,038	5175.6	1,109,853	66,187	40,974.64	689,052.1	21,966.2	101.4
2016	31,676	6282.1	1,158,999	71,921	46,344.88	744,127.2	23,821.0	102
2017	32,395	7327.9	1,133,161	74,916	52,598.28	827,121.7	25,973.8	101.6

, the values of $X_3$ and $X_4$ in 2009 are missing in the original dataset. Therefore, the imputation of the missing data is operated to guarantee that the data sample is still representative of the population. The application of the fitting function by using non-linear regression in this study allows for giving estimated values to $X_3$ and $X_4$, which are 547,000 and 45,333, respectively. Table 1 presents the dimensions of original data from 2005–2017. It is distinct in that the dimensions are inconsistent. The original data sequence needs to conduct initial value processing in order to avoid the drift of multivariate grey modelling data Matrix. After the prediction sequence is obtained, the initial inverse transformation is applied, and the dimension and magnitude of the characteristic sequence of the system are restored. Classic GM (1, N) model, GM (1, N) with a linear cross effect and GM (1, N) that are based on SVM with non-linear cross effect are established based on the data series after initial value processing from 2005–2013.

Table 2. Comparison of fitted values and relative errors in three models.

Year	Actual Value	Classic GM (1, N)		GM (1, N) with Linear Cross Effect		GM (1, N) Based on SVM with Non-Linear Cross Effect
		Fitted Value	Relative Error/%	Fitted Value	Relative Error/%	Fitted Value	Relative Error/%
2005	4250	4250	-	4250	-	4250	-
2006	5019	5227.8	4.16	5204	3.69	5122	2.06
2007	6362	6707	5.43	6646	4.46	6586	3.52
2008	7875	8385	6.48	8291.5	5.29	8181	3.89
2009	9443	10,590	12.15	10,461	10.78	9873.6	4.56
2010	12,350	14,696.5	19.01	13,898.7	12.54	12,867.5	4.19
2011	15,624	16,509.8	5.67	17,378.6	11.23	16,527	5.78
2012	18,770	20,587	9.68	20,292	8.11	19,731	5.12
2013	22,297	25,737	15.43	24,428.6	9.56	23,342.7	4.69
Fitted total of Relative Error	-	-	78.01	-	65.66	-	34.52

and

Table 3. Comparison of the prediction and relative errors in three models.

Year	Actual Value	Classic GM (1, N)		GM (1, N) with Linear Cross Effect		GM (1, N) Based on SVM with Non-Linear Cross Effect
		Predicted Value	Relative Error/%	Predicted Value	Relative Error	Predicted Value	Relative Error
2014	25,798	27,193.6	5.41	26,884	4.21	26,744.8	3.67
2015	29,038	32,043.4	10.35	30,853	6.25	30,434.7	4.81
2016	31,676	35,822	13.09	34,251	8.13	33,237.6	4.93
2017	32,395	37,361	15.33	35,949	10.97	33,409	3.13
Forecast total of Relative Error	-	-	50.18	-	29.56	-	16.54

show the simulation results and prediction results obtained after the initial value inverse transformation.

Based on Table 2 and Table 3, it can be seen that the GM (1, N) model based on SVM with non-liner cross effect has better performance in fitting precision and predicted accuracy. Specifically, the fitted total of relative error that was developed by classic GM (1, N) model is 0.7801, the forecast total of relative error is 0.5018, and GM (1, N) with linear cross effect model has more accurate values on these two aspects, which are 0.6566 and 0.2956, respectively. The reason of the better accuracy in GM (1, N) with a linear cross effect is that it considers the linear cross effect among the three influencing factors, including the crowding-out effect among the factors. In this basis, our improved GM (1, N) model is able to consider the integrated driving effect includes non-linear cross effect including crowding-out effects. Therefore, it develops a better fitting precision and predicted accuracy than the previous two models.

5. Conclusions

Our study develops a new GM (1, N) model that discusses the difference of linearization and non-linearization in the cross effect. For the GM (1, N) model with linear cross effect, the solution can be developed by applying albino differential equation in traditional GM (1, 1) model. While for the GM (1, N) model with nonlinear cross effect, which presented in our study, it can firstly use SVM regression to transfer non-linearization to linearization, and then develop a solution. Both of the methods have followed the principle of accumulation in GM (1, 1) model, but our new GM (1, N) model addresses the non-linear relationship between the sequence in system feature, accumulate sequence in driving factors and cross effect, and find a solution by using SVM regression. Through the test on fitting and prediction by adopting the data regarding medical service in China, we find that the GM (1, N) model based on SVM regression with a nonlinear cross effect has better performance on these two indicators than the Classic GM (1, N) model and GM (1, N) with a linear cross effect model.