Forecasting Economic Growth of the Group of Seven via Fractional-Order Gradient Descent Approach

Abstract

This paper establishes a model of economic growth for all the G7 countries from 1973 to 2016, in which the gross domestic product (GDP) is related to land area, arable land, population, school attendance, gross capital formation, exports of goods and services, general government, final consumer spending and broad money. The fractional-order gradient descent and integer-order gradient descent are used to estimate the model parameters to fit the GDP and forecast GDP from 2017 to 2019. The results show that the convergence rate of the fractional-order gradient descent is faster and has a better fitting accuracy and prediction effect.

1. Introduction

In recent years, fractional model has become a research hotspot because of its advantages. Fractional calculus has developed rapidly in academic circles, and its achievements in the fields include [1,2,3,4,5,6,7,8,9,10].

Gradient descent is generally used as a method of solving the unconstrained optimization problems, and is widely used in evaluation and in other aspects. The rise in fractional calculus provides a new idea for advances in the gradient descent method. Although numerous achievements have been made in the two fields of fractional calculus and gradient descent, the research results combining the two are still in their infancy. Recently, ref. [11] applied the fractional order gradient descent to image processing and solved the problem of blurring image edges and texture details using a traditional denoising method, based on integer order. Next, ref. [12] improved the fractional-order gradient descent method and used it to identify the parameters of the discrete deterministic system in advance. Thereafter, ref. [13] applied the fractional-order gradient descent to the training of neural networks’ backpropagation (BP), which proves the monotony and convergence of the method.

Compared with the traditional integer-order gradient descent, the combination of fractional calculus and gradient descent provides more freedom of order; adjusting the order can provide new possibilities for the algorithm. In this paper, economic growth models of seven countries are established, and their cost functions are trained by gradient descent (fractional- and integer-order). To compare the performance of fractional- and integer-order gradient descent, we visualize the rate of convergence of the cost function, evaluate the model with $MSE$, $MAD$ and $R^2$ indicators and predict the GDP of the seven countries in 2017–2019 according to the trained parameters.

The Group of Seven (G7)

The G6 was set up by France after western countries were hit by the first oil shock. In 1976, Canada’s accession marked the birth of the G7, whose members are the United States, the United Kingdom, France, Germany, Japan, Italy and Canada seven developed countries. The annual summit mechanism of the G7 focuses on major issues of common interest, such as inclusive economic growth, world peace and security, climate change and oceans, which have had a profound impact on global, economic and political governance. In addition to the G7 members, there are a number of developing countries with large economies, such as China, India and Brazil. In the context of economic globalization, the study of G7 economic trends and economic-related factors can provide a useful reference for these countries’ development.

The economic crisis broke out in western countries in 1973, so the data in this paper cover the period from 1973 to 2016, and data for the seven countries are available since then. Some G7 members (France, Germany, Italy and the United States) were members of the European Union (EU) during this period, so this paper also establishes the economic growth model of the EU. Data for this article are from the World Bank.

2. Model Describes

The prediction of variables generally uses time series models [14] (for example, ARIMA and SARIMA), or artificial neural networks [15,16], which have been very popular in recent years. The time series model mainly predicts the future trend in variables, but it is difficult to reflect the change in unexpected factors in the model. Additionally, the neural network model needs to adjust more parameters, the network structure selection is too large, the training efficiency is not high enough, and easy to overfit.

Although the linear model is simple in form and easy to model, its weight can intuitively express the importance of each attribute, so the linear model has a good explanatory ability. It is reasonable to build a linear regression model of economic growth, which can clearly learn which factors have an impact on the economy.

Next, we chose eight explanatory variables to describe the economic growth in this paper. The explained variable is y, where y refers to GDP and is a function. The expression for y is as follows:

$$y\left(t\right)=\sum _{j=1,2,3,4,5,6,7,8}{\theta }_j{x}_j\left(t\right)+{\theta }_0+ϵ,$$

(1)

where t is year ($t=44$), ${\theta }_0$ is the intercept. $ϵ$ is an unobservable term of random error. ${\theta }_j$ represents the weight of each variable. The eight explanatory variables are:

• $x_1$: land area (km${}^2$)
• $x_2$: arable land (hm${}^2$)
• $x_3$: population
• $x_4$: school attendance (years)
• $x_5$: gross capital formation (in 2010 US$)
• $x_6$: exports of goods and services (in 2010 US$)
• $x_7$: general government final consumer spending (in 2010 US$)
• $x_8$: broad money (in 2010 US$)

3. Fractional-Order Derivative

Due to the differing conditions, there are different forms of fractional calculus definition, the most common of which are Gr$\ddot{\mathrm{u}}$nwald–Letnikov, Riemann–Liouville, and Caputo. In this article, we chose the definition of fractional-order derivative in terms of the Caputo form. Given the function $f\left(t\right)$, the Caputo fractional-order derivative of order $\alpha$ is defined as follows:

$${}^{\mathit{Caputo}}\mathit{D}_t^{\alpha }f\left(t\right)=\frac{1}{\Gamma (1-\alpha )}{\int }_c^t{(t-\tau )}^{-\alpha }{f}^{{}^{\prime }}\left(\tau \right)d\tau ,$$

where ${}^{\mathit{Caputo}}\mathit{D}_t^{\alpha }$ is the Caputo derivative operator. $\alpha$ is the fractional order, and the interval is $\alpha \in (0,1)$. $\Gamma (·)$ is the gamma function. c is the initial value. For simplicity, ${}_c\mathit{D}_t^{\alpha }$ is used in this paper to represent the Caputo fractional derivative operator instead ${}^{\mathit{Caputo}}{\mathit{D}}_{\mathrm{t}}^{\alpha }$.

Caputo fractional differential has good properties. For example, we provide the Laplace transform of Caputo operator as follows:

$$L\left\{\mathit{D}^{\alpha }f\left(t\right)\right\}=s^{\alpha }F\left(s\right)-\sum _{k=0}^{n-1}{f}^{\left(k\right)}\left(0\right)s^{\alpha -k-1},$$

where $F\left(s\right)$ is a generalized integral with a complex parameter s, $F\left(s\right)={\int }_0^{\infty }f\left(t\right)e^{-st}dt$. $n=:\left[\alpha \right]$ is the $\alpha$ rounded up to the nearest integer. It can be seen from the Laplace transform that the definition of the initial value of Caputo differentiation is consistent with that of integer-order differential equations and has a definite physical meaning. Therefore, Caputo fractional differentiation has a wide range of applications.

4. Gradient Descent Method

4.1. The Cost Function

The cost function (also known as the loss function) is essential for a majority of algorithms in machine learning. The model’s optimization is the process of training the cost function, and the partial derivative of the cost function with respect to each parameter is the gradient mentioned in gradient descent. To select the appropriate parameters $\theta$ for the model (1) and minimize the modeling error, we introduce the cost function:

$$C\left(\theta \right)=\frac{1}{2m}\sum _{i=1}^m{(h_{\theta }\left(x^{\left(i\right)}\right)-y^{\left(i\right)})}^2,$$

(2)

where $h_{\theta }\left(x^{\left(i\right)}\right)$ is a modification of model (1), $h_{\theta }\left(x\right)={\theta }_0+{\theta }_1{x}_1+\cdots +{\theta }_j{x}_j$, which represents the output value of the model. $x^{\left(i\right)}$ are the sample features. $y^{\left(i\right)}$ is the true data, and t represents the number of samples ($m=44$).

4.2. The Integer-Order Gradient Descent

The first step of the integer-order gradient descent is to take the partial derivative of the cost function $C\left(\theta \right)$:

$$\frac{\partial C\left(\theta \right)}{\partial {\theta }_j}=\frac{1}{m}\sum _{i=1}^m(h_{\theta }\left(x^{\left(i\right)}\right)-y^{\left(i\right)})x_j^{\left(i\right)},j=1,2,\cdots ,8,$$

(3)

and the update function is as follows:

$${\theta }_{j+1}={\theta }_j-\eta \frac{1}{m}\sum _{i=1}^m(h_{\theta }\left(x^{\left(i\right)}\right)-y^{\left(i\right)})x_j^{\left(i\right)},$$

(4)

where $\eta$ is learning rate, $\eta >0$.

4.3. The Fractional-Order Gradient Descent

The first step of fractional-order gradient descent is to find the fractional derivative of the cost function $C\left(\theta \right)$. According to Caputo’s definition of fractional derivative, from [17] we know that if $g\left(h\right(t\left)\right)$ is a compound function of t, then the fractional derivation of $\alpha$ with respect to t is

$${}_c\mathit{D}_t^{\alpha }g\left(h\right)=\frac{\partial \left(g\right(h\left)\right)}{\partial h}·{}_c\mathit{D}_t^{\alpha }h\left(t\right).$$

(5)

It can be known from (5) that the fractional derivative of a composite function can be expressed as the product of integral and fractional derivatives. Therefore, the calculation for ${}_c\mathit{D}_{{\theta }_j}^{\alpha }C\left(\theta \right)$ is as follows:

$$\begin{array}{cc}\hfill {}_c\mathit{D}_{{\theta }_j}^{\alpha }C\left(\theta \right)& =\frac{1}{m}\sum _{i=1}^m(h_{\theta }\left(x^{\left(i\right)}\right)-y^{\left(i\right)})\Gamma (1-\alpha ){\int }_c^{{\theta }_j}{({\theta }_j-\tau )}^{-\alpha }\frac{\partial [h_{\theta }\left(x^{\left(i\right)}\right)-y^{\left(i\right)}]}{\partial {\theta }_j}d\tau \hfill \\ \hfill & =\frac{1}{m}\sum _{i=1}^m(h_{\theta }\left(x^{\left(i\right)}\right)-y^{\left(i\right)})x_j^{\left(i\right)}\Gamma (1-\alpha ){\int }_c^{{\theta }_j}{({\theta }_j-\tau )}^{-\alpha }d\tau \hfill \\ \hfill & =\frac{1}{m(1-\alpha )\Gamma (1-\alpha )}{({\theta }_j-c)}^{(1-\alpha )}\sum _{i=1}^m(h_{\theta }\left(x^{\left(i\right)}\right)-y^{\left(i\right)})x_j^{\left(i\right)},\hfill \end{array}$$

and the update function is as follows:

$${\theta }_{j+1}={\theta }_j-\eta \frac{1}{m(1-\alpha )\Gamma (1-\alpha )}{({\theta }_j-c)}^{(1-\alpha )}\sum _{i=1}^m(h_{\theta }\left(x^{\left(i\right)}\right)-y^{\left(i\right)})x_j^{\left(i\right)},j=1,2,\cdots ,8$$

(6)

where $\eta$ is the learning rate, $\eta >0$. $\alpha$ is the fractional order, $0<\alpha <1$. c is the initial value of Caputo’s fractional derivative, and $c<min\left\{{\theta }_j\right\}$.

5. Model Evaluation Indexes

We use the absolute relative error ($ARE$) to measure the prediction error:

$$AR{E}_i=\frac{\left|y_i-{\widehat{y}}_i\right|}{y_i}.$$

To evaluate the fitting quality of gradient descent on the model, the following three indicators can be calculated:

The mean square error ($MSE$):

$$MSE=\frac{1}{n}\sum _{i=1}^n{(y_i-\widehat{y_i})}^2.$$

The coefficient of determination ($R^2$):

$$R^2=1-\frac{{\displaystyle \sum _{i=1}^n}{(y_i-\widehat{y_i})}^2}{{\displaystyle \sum _{i=1}^n}{(y_i-\overline{y_i})}^2}.$$

The mean absolute deviation ($MAD$):

$$MAD=\frac{{\displaystyle \sum _{i=1}^n}\left|y_i-\widehat{y_i}\right|}{n}.$$

In these formulas, n is the number of years ($n=44$). $y_i$ and $\widehat{y_i}$ are the real value and the model output, respectively. $\overline{y_i}$ is the mean of the GDP.

6. Main Results

In this article, we standardize the data for each country before running the algorithm, and each iteration to update $\theta$ uses m samples. The grid search method was used to select the appropriate learning rate and initial weight interval, and the effects of different fractional orders are compared to select the best order (see

Table 1. Parameters for different countries.

Country	$\mathit{\alpha }$	Learning Rate	Initial Interval
Canada	0.8	0.03	$(-0.5,0.5)$
France	0.8	0.03	$(-0.8,0.8)$
Germany	0.8	0.03	$(-0.1,0.1)$
Italy	0.8	0.03	$(-0.5,0.5)$
Japan	0.8	0.03	$(-0.1,0.1)$
The United Kingdom	0.8	0.03	$(-0.5,0.5)$
The United States	0.8	0.03	$(-0.1,0.1)$
European Union	0.8	0.03	$(-0.5,0.5)$

).The learning rate and the initial weight interval are applicable to both fractional-order gradient descent and integer-order gradient descent.

6.1. Comparison of Convergence Rate of Fractional and Integer Order Gradient Descent

In order to facilitate visual comparison, (4) and (6) are iterated 50 times, respectively, as well as their convergence rates (see Figure 1).

As shown in Figure 1, for each dataset, after the same number of iterations, the convergence rate of fractional-order gradient descent is faster than that of integer-order gradient descent, which indicates that the method combining fractional-order and gradient descent is better than the traditional integer-order gradient descent in the convergence rate of update equation.

6.2. Fitting Result

Then, we fit GDP with integer-order gradient descent and fractional-order gradient descent, respectively. Start by setting a threshold and stop iterating when the gradient is less than this threshold. The fitting effect diagram is shown in Figure 2, and the performance evaluation of the model is shown in

Table 2. Performance of integer order and fractional order gradient descent.

	Canada		France		Germany		Italy
Index	Integer (4)	Fractional (6)	Integer (4)	Fractional (6)	Integer (4)	Fractional (6)	Integer (4)	Fractional (6)
$MSE$ ($\times {10}^{20}$)	2.2548	1.5689	7.3396	4.3851	7.6262	6.8976	3.2521	2.701
$R^2$	0.9984	0.9989	0.9971	0.9983	0.9981	0.9983	0.9974	0.9978
$MAD$ ($\times {10}^{10}$)	1.1015	0.9066	2.2076	1.68	2.2824	2.0203	1.4947	1.3146
	Japan		The United Kingdom		The United States		European Union
Index	Integer (4)	Fractional (6)	Integer (4)	Fractional (6)	Integer (4)	Fractional (6)	Integer (4)	Fractional (6)
$MSE$ ($\times {10}^{20}$)	19.9103	16.6656	15.2421	13.7876	98.4201	60.7402	197.9143	90.5717
$R^2$	0.9986	0.9989	0.9946	0.9951	0.9993	0.9995	0.9983	0.9992
$MAD$ ($\times {10}^{10}$)	3.8663	3.2745	3.1182	2.9489	7.8593	5.714	11.8393	7.2684

.

It can be seen from Table 2 that the $MSE$, $R^2$ and $MAD$ results of GDP fitted by fractional-order gradient descent are better than that fitted by integer-order gradient descent, which indicates that, under the same iteration number, learning rate and initial weight interval, the fitting performance of the data fitted by fractional-order gradient descent is better than that of integer-order.

6.3. Predicted Results

Finally, in order to test the prediction effect of fractional- and integer-order gradient descent on GDP, we forecast the GDP from 2017 to 2019, and used the $ARE$ index to measure the prediction error (see

Table 3. Integer-order and fractional-order gradient descent for G7 countries’ GDP data from 2017 to 2019.

Country	Year	Actual Value	Predicted Value		$\mathit{A}\mathit{R}\mathit{E}$
			Integer	Fractional	Integer	Fractional
	2017	1869939124387.55	1851176120948	1865471720455.36	0.01003	0.00239
Canada	2018	1907592951375.51	1885635961969.18	1897212921116.78	0.01151	0.00544
	2019	1939183469806.34	1913183323405.81	1924536620147.11	0.01341	0.00755
	2017	2876185347152.35	2945583296625	2913853765393.87	0.02313	0.01211
France	2018	2927751436718.37	2987173241226.19	2955215192748.21	0.0193	0.0084
	2019	2971919320115.83	3052414282679.98	3007640733954.07	0.02608	0.01103
	2017	3873475897139.37	3992089822476.93	3987473981388.45	0.03062	0.02943
Germany	2018	3922591386837.48	4035516755191.92	4019973502352.35	0.02879	0.02483
	2019	3944379455526.15	4007551577032.44	3942199462068.09	0.01602	0.00055
	2017	2124019926800.66	2152553322306.66	2148504123256.22	0.01343	0.01053
Italy	2018	2144072575240.17	2184791916115.44	2178336024841.51	0.01899	0.01598
	2019	2151420719257.08	1694388219398.54	1946816137097.53	0.21243	0.0951
	2017	6150456276847.65	6246751221623.44	6217262375879.73	0.01566	0.01086
Japan	2018	6170335002849.18	6302599251651.13	6266099914852.53	0.02144	0.01552
	2019	6210698351093.34	6274298653661.42	6272342082178.18	0.01411	0.01379
	2017	2841238185971.41	2714332507299.13	2737032647202.61	0.04467	0.03668
The United Kingdom	2018	2879331251695.23	2735833239476.11	2760916583838.65	0.04984	0.041126
	2019	2921446026408.24	2784137398857.08	2812534141119.14	0.047	0.03728
	2017	17403783207186.7	17154216039682.2	17344565695242.3	0.01434	0.0034
The United States	2018	17913248631409.5	17681187933498.6	17835485270334.4	0.01725	0.00434
	2019	18300385513295.6	18004286468803.1	18168502346487.9	0.01618	0.00721
	2017	16012037378199.3	17983491434848.4	18072460558164	0.04479	0.04006
European Union	2018	16351210756244.2	18105516308926.6	18296349316535.8	0.05715	0.04272
	2019	16605351894524	18828265531889.1	19241290506759	0.03446	0.01328

).

7. Conclusions

In this paper, the gradient descent method is used to study the linear model problems which is different from [18,19]. The results show that, in addition to the least square estimation, the gradient descent method can also solve the regression analysis problem by iterating the cost function, and obtain good results, a without complicating the model. It also improves the interpretability of explanatory variables. We apply the fractional differential to gradient descent, and compare the performance of fractional-order gradient descent with that of integer-order gradient descent. It was found that the fractional-order has a faster convergence rate, higher fitting accuracy and lower prediction error than the integer-order. This provides an alternative method for fitting and forecasting GDP and has a certain reference value.