An Efficient Deep Learning Based Model to Predict Interest Rate Using Twitter Sentiment

: In macroeconomics, decision making is highly sensitive and significantly influences the financial and business world, where the interest rate is a crucial factor. In addition, the interest rate is used by the governments to manage the monetary policy. There is a need to design an efficient algorithm for interest rate prediction. The analysis of the social media sentiment impact on financial decision making is also an open research area. In this study, we deploy a deep learning model for the accurate forecasting of the interest rate for the UK, Turkey, China, Hong Kong, and Mexico. For this purpose, daily data of the interest rate and exchange rate covering the period from Jan 2010 to Oct 2019 is used for all the mentioned countries. We also incorporate the input of the twitter sentiments of six mega-events, namely the US election 2012, Mexican election 2012, Gaza under attack 2014, Hong Kong protest 2014, Refugee Welcome 2015, and Brexit 2016. Our results provide evidence that the error of the deep learning model significantly decreases when event sentiment is incorporated. A notable improvement has been observed in the case of the Hong Kong interest rate, i.e., a 266% decline in the error after incorporating event sentiments as an input in the deep learning model.


Introduction:
In economics and business applications, the accurate forecasting of time series data has always been a critical issue, including wind speed prediction [1,2], portfolio risk [3,4], volatility of the energy market [5,6], exchange rate [7], inflation [8], and stock market index [9,10]. In this context, the forecasting of the interest rate has a crucial importance while making decisions in business and economics. It affects the valuation of derivatives and fixed income securities. In addition, the interest rate is a part of monetary policy and the government, to control the fluctuation in overall price levels. It also impacts asset prices and business cycles. In the Capital asset pricing model (CAPM), the return on risky securities is elucidated in terms of excess returns relative to the risk-free rate. Therefore, in the existing body of knowledge, a number of studies have proposed different statistical and soft computing models for a more accurate forecasting, which is helpful for investors and business managers when making decisions.
Several mathematical finance models based on the theory of market equilibrium and noarbitrage assumption have been proposed in the last few decades to capture the changes incurred in the interest rate. Furthermore, models based on stochastic processes are also common in the literature [11,12]. These models are based on economic theories, but they fail to predict the dynamics of the interest rate during the crises regime [11]. Therefore, the demand for a more practical and adaptive model has increased, in order to model the dynamics of the interest rate [11]. There are a number of studies in which different regression models, like vector autoregressive (VAR), autoregressive conditionally heteroskedastic (ARCH), generalized autoregressive conditionally heteroskedastic (GARCH), and autoregressive moving average, are used to forecast the time series interest rate [13,14]. All these regression models only capture linear structures and generally fail to provide accurate prediction metrics [15]. These models follow the assumption of normality of disturbance terms and the linear relationship among variables. However, these assumptions are usually not valid for real-time data. Therefore, the use of machine learning models to get accurate and robust results becomes inevitable [15,16]. These models also capture non-linear patterns in time series data, which makes them more advantageous and useful. There is very little work on social media relations with interest rate prediction. In particular, there is no work where important local and global events are considered to check the effects on interest rate prediction.
In this study, we use advanced models of machine learning, i.e., linear regression, support vector regression (SVR), and Deep Learning (DL) to predict the daily interest rates of the UK, Mexico, Turkey, Hong Kong, and China. We used these models because of their ability to capture the nonlinear patterns in time series datasets and also because they provide more accurate results. We used the exchange rate for each respective country and the twitter sentiment of multiple events happening around the globe as input. The twitter events used in this research are the US election 2012, Mexican election 2012, Gaza under attack 2014, Hong Kong protest 2014, Refugee Welcome 2015, and Brexit 2016. These events are categorized as local and global events with respect to each country. The sentiment of each event is calculated and then used as a parameter for the interest rate prediction. We processed almost 9.6 million tweets to calculate the daily event sentiment. The proposed approach is evaluated on the interest rate dataset collected for almost 9 years for each country. The results showed a significant improvement when the twitter sentiment is used along with deep learning. The proposed work contributes to the literature in multiple ways. We used a deep learning-based algorithm for the efficient and accurate prediction of interest rates by incorporating the twitter sentiment of local and global events for each country. The system also uses the exchange rate as a parameter to predict the interest rate. Moreover, we also compared our findings of a deep learning model with state-of-the-art forecasting methods and showed the superiority of the proposed work.
Our research study is organized as follows: the second section is a precise literature review, Section 3 presents the detail of data and methodology, Section 4 discusses the results and discussion, and this is followed by the last section, which presents the conclusions.

Literature Review
There are a number of studies that focus on the mining of public opinions and emotions, which is normally referred to as social media, where twitter is considered as one of the widely used social platforms. In addition, these opinions, in the form of sentiments, have numerous applications in market services and investment settings [17][18][19][20]. Social media plays a vital role in the return prediction of Chief Executive Officer (CEO) through inside trading [21]. In the literature, there are studies that focus on stock prediction using social the media sentiment of mega-events [22] and exchange rate prediction in a similar context [23]. In terms of model comparison, there has been a formal comparison of single-factor models [24]. The study was conducted on the data of the US and rejected the commonly used square root diffusion model [25].
The modeling of short-term interest rate prediction has been a crucial point in the literature because it has a fundamental importance in fixed income securities and the risk associated with holding such securities. In recent times, researchers have focused on those models that are capable of handling issues of a complex nature without considering the structural assumption of time series datasets. These assumptions are normality, non-heteroskedasticity, and linearity of variables. In this context, models that can capture nonlinear patterns have become very common in recent research. In a study, US 3-months Treasury bills were forecasted using a Markov switching model by Hamilton [26], and the model outperformed the linear model. Non-linearity exists in the interest rate because of the stochastic swings [27]. This can also be understood by considering the business cycle [28]. Moreover, the term structure of interest rate is also studied by considering nonlinearity in the time series datasets [29,30]. Artificial neural network (ANN) models outperform the state-of-the-art econometric models because they provide robust results even if the time series data have violated the assumption of non-linearity. These ANN models yield accurate results even if the sample size is low [31]. Contrary to this, the traditional Box Jenkins (ARIMA) approach of forecasting provides reliable results only when the sample size is above 50. Moreover, this technique is only suitable when there is a lack of information about the data generating process [32].
From the empirical perspective, the existing literature is divided into four different streams. The first stream uses the change in the yield curve in order to predict the price level and GDP. Several studies prevail in this stream and follow both linear and nonlinear models to use the spread among short term and long term bond rates as an input factor for explaining different macroeconomic factors like recession indices, price level, economic growth, and industrial production [33][34][35][36][37][38]. These econometric models have great forecasting tendencies across different rejoins and time [39].
The second part of the literature focuses on the "data-driven models". In these studies, mathematical models that are appropriate for focusing on spot rates are used for example splines [40] and parsimonious [41,42]. However, these models focus on the future behavior of interest rates. In addition, their focus is on the interpolation of the current shape and term structure. The third stream focuses on "dynamic models" such as equilibrium models [11,43] and arbitrage-free models [44,45].
The fourth stream of literature focuses on those data-driven models which rely on knowledge discovery techniques to predict the interest rate. In addition, these models cope with complexities like non-linearity, structural breaks, and seasonality issues laying in variables. Most of the research focuses on the prediction of interest rates considering the time dynamics [46][47][48][49][50]. There are several studies which follow machine learning models to predict the interest rate. Neural networks and casebased reasoning (CBR) are used in the literature [47]. In the case of the US, the performance of the neural network is better than that of CBR. In addition, it is important to incorporate the structural changes in the economy in the neural network to get more refined results [48]. Furthermore, the performance of the neural network improves when internet news of a positive or negative nature are including as inputs in the neural network model. This news is related to a macroeconomic policy like monetary policy, price levels, unemployment, etc. Another model called a fuzzy inference neural network, dealing with non-linearity and the complex nature of interest rates, yield a better prediction as compared to neural networks [51].
In the existing literature, the deep learning model has not been used before to model the interest rate. Moreover, the twitter sentiment on mega-events of a dynamic nature has never been incorporated to predict the interest rate, which is a highly volatile and sensitive macroeconomic variable.

Data and Methodology
In order to predict the interest rate, we follow a set of approaches that includes a state-of-the-art econometric technique of linear regression, support vector regression, and deep learning. In the first step, we predict the interest rate for different countries: the UK as a developed economy, and Turkey, China, Hong Kong, and Mexico as emerging economies. We use the twitter sentiment of the relevant events to predict the interest rate in the second step. Figure 1 shows the detailed flow of the process we followed for the interest rate prediction. First, we use the Twitter dataset of different events and calculate the sentiment for global and local events [52]. Second, we incorporate this social media sentiment to predict the interest rate of the UK, Turkey, China, Hong Kong, and Mexico by using multiple techniques, i.e., linear regression, Support Vector Regression, and deep learning (DL). Table 1 shows the details of the dataset we used for the interest rate prediction.

Sentiment Analysis of Social Media Tweet for Prediction of CC:
In order to calculate the sentiment from the textual data, we used the Alex Davies word list [53]. In this approach, the list consists of almost five thousand words categorized into positive, negative, and neutral sets. Tweets are tokenized, and the word list is prepared in order to remove whitespace, emotions, punctuations, and URL. Tweets are classified as positive, negative, and neutral categories by using an efficient and accurate dictionary. We replaced the word list by adding our own list of 4000 words to improve the results because this list considers the relationship of each word and multiword expressions. At first, the daily textual data of positive, negative, and neutral tweets were represented in percentages, after which the net daily sentiment was calculated.
We processed the textual dataset of twitter of multiple events for the sentiment analysis. We considered six mega-events of a dynamic nature for the sentiment analysis, the details of which are mentioned in Table 2. The dataset of the events was taken from twitter, which contains almost 9.6 million tweets. After the sentiment analysis, we predicted the interest rate by using a state-of-the-art econometric technique of linear regression and advanced robust techniques of SVM and DL.

Forecasting Models:
In this section, we discuss the details of the models we applied for the interest rate prediction. Support Vector Regression: The original SVM follows the theory of statistical learning algorithms. Furthermore, it manages the structural risks [53,54]. In recent literature, the augmented form of SVM was used [54].
Let us suppose the time series data is given as follows: where Xi represents the Exogenous Variable at the time i, and yi represents the Endogenous Variable.
The regression for the proposed dataset can be written as: In the above Equation (2), weight and bias are represented by W and b. The input vector X can be mapped by φ(X) into a higher dimensional space. We solve the optimization problem represented in Equations (3) and (4) for values of W and b: Subject to: In the above mathematical expressions, c represents the parameter that acts as a trade-off between simplicity and generalizability, whereas ξi and ξ * i the slack variables which are used for the cost of errors. We used the kernel mapping approach to map the non-linear data from the original vector space to higher-dimensional space. By this mapping, the regression form of SVM is obtained as follows in Equation (5): αi and α * I show the Lagrange multipliers. Meanwhile, the Gaussian Radial Function is represented by: Linear Regression: Linear models are simple in nature, and therefore the time series data is predicted using these simple models. We used multiple a linear regression in our modeling that considers a single variable explained by multiple explanatory variables. We suppose y as the explained variable, which has a linear relationship with k explanatory variables X1, X2, X3 . . . Xk: Furthermore, β1, β2, β3 . . . βk are the slope coefficients of X1, X2, X3 . . . Xk respectively. ε is the stochastic disturbance term, which is the difference between the fitted and observed values.
The jth slope coefficient βj can be defined as the anticipated change in y due to the 1 unit change in the jth explanatory variable Xj. Assuming E(ε) = 0, the equation can be written as:

Deep Learning Model:
Deep learning is used to achieve more efficient results. There are hidden layers in neural networks, and in each layer the net network learns new feature space through the linear transformation of the given inputs. The continual non-linear function is applied, leading the process to the output layer. Neural networks are defined as the process to get the output by flowing the information through hidden layers. In our study, we apply the DL model, which contains a large number of hidden layers and neurons which are interconnected and operate in a parallel way. Moreover, this approach is also common for solving regression problems. The performance of the DL model improves when the frequency of the data increases [55]. In this study, we use a DL model based on the Convolutional Neural Network with the standard-setting.

Evaluation Metrics:
In time series prediction, there are multiple evaluation metrics, such as Absolute Error (AE) and Root Mean Squared Error (RMSE). By subtracting the actual values from predicted values we obtain these metrics [56,57].

Root Mean Squared Error-RMSE
The following mathematical expression is used to calculate RMSE. It shows the average magnitude of the estimated disturbance term of predicted values:

Mean Absolute Error-MAE
MAE is obtained by taking the average of the estimated disturbance and avoiding the sign of the predicted values. Equal weights are given to the estimated disturbance terms. All the estimated errors are given equal weights. We use the following formula to calculate MAE: Here, n shows the total number of estimated values. The predicted values are represented by ( ), which shows the forecasted values, whereas ( ) shows the actual values.

Results and Discussion:
This part presents the details of the results and experimentation.  *, **, *** represent the significance at 10%, 5%, and 1%.

Descriptive Statistics:
The interest rate of all the five countries follows a non-normal distribution because the normality test statistic of JB (Jarque Bera) is highly significant at 1%. The interest rate of the UK has the lowest mean (0.499), median (0.500), and standard deviation (0.125), as compared to the rest of the countries. The Turkish interest rate shows the highest mean (8.925), median (7.50), and standard deviation (5.349).

Unit Root:
The results of the regression are only considered reliable when the time series data have the problem of time dependence (Unit Root). For this reason, we use the Augmented Dicky Fuller (ADF) test. The ADF test checks the problem of stationarity by testing the null hypothesis of "Unit root exists" against the alternate of "No unit root". The results are reported in Table 4. We check the stationarity of the interest rate and exchange rate of all the countries in our dataset. The results show that both the variables are stationary at a 1st difference in the case of all the five countries. Critical values are also reported in Table 4.

Interest Rate Prediction without Sentiment:
In this part, we show the results of the interest rate prediction without incorporating the event sentiment as input in our models. We used three different techniques, which are Linear Regression (LR), SVR, and DL for the interest rate prediction. We also used the exchange rate as input or independent variable for the interest rate prediction.
We used the daily data interest rate and exchange rate of all the countries, ranging from 1st of January 2010 to October 2019. The prediction results are reported in Table 5. It is evident from the results of both the evaluation metrics (AE and RMSE) that the DL model yields more accurate results as compared to SVR and LR for all the countries except the UK. The results of DL in the case of China and Hong Kong are approximately 78% accurate as compared to LR. Similarly, DL in the case of Mexico and Turkey yields approximately 64.5% and 52.1% better results as compared to LR. Both AE and RMSE provide the same results.

Interest Rate Prediction with the Sentiment:
In this part, we consider the sentiment of different events as input for the interest rate prediction. The details of the events is mentioned in Table 2. We processed the textual data of almost 9.6 million tweets of different events in order to calculate the event sentiment. We used the sentiment of all the events separately for all the countries as inputs in our data set.
The results of the interest rate prediction of Turkey are presented in Table 6. Our results provide evidence that the event sentiment is a significant input in predicting the interest rate of Turkey because the results have significantly improved in the case of all events except Brexit and Ghaza Attack. The mean error of the DL model has decreased by almost 60% when the event sentiment of Hong Kong Protest, Mexican Election, Refugees Welcome, and US election are incorporated as inputs. In addition, the event of the US election stands out as being the most important event because the error of the DL model has decreased by approximately 90% when we have used the sentiment of the US Election.  Figure 2 shows the plot of 500 actual and forecasted values of the interest rate of Turkey when we incorporated the event sentiment of Brexit and Refugees Welcome as input in the LR, SVR, and DL models. The graph shows that the forecasted values of the DL model are close to their actual values as compared to LR and SVM.     The results of the interest rate prediction of China are presented in Table 7. Our results provide evidence that event sentiment is a significant input in predicting the interest rate of China because the results have significantly improved in the case of all events except Refugees Welcome. The mean error of the DL model has decreased by almost 34% when the event sentiment of Brexit, Ghaza Attack, Hong Kong Protest, Mexican Election, and US election are incorporated as inputs.       The results of the interest rate prediction of the UK are presented in Table 8. Our results provide evidence that event sentiment is a significant input to predict the interest rate of the UK because the results have significantly improved in the case of the Mexican election and US election. The mean error of the DL model has decreased by almost 33% when the event sentiment of Mexican Election and US Election are incorporated as inputs. However, the interest rate of the UK remains insensitive towards both the local event of Brexit and global events like the Ghaza Attack, Hong Kong Protest and Refugees Welcome. These results endorsed the fact that the economy of the UK is relatively insensitive to the events (Brexit, Hong Kong Protest) that are significant for the interest rate of emerging economies like China, Turkey, and Hong Kong.
When we incorporated the event sentiment of all the events to plot the graph of forecasted and actual values, there was no fluctuation in the spread because there was no significant variation or trend in the data of the interest rate of the UK. Therefore, the graph is not shown here.
The results of the interest rate prediction of Mexico are presented in Table 9. Our results provide evidence that event sentiment is a significant input to predict the interest rate of Turkey because the results have significantly improved in the case of all events. The mean error of the DL model has decreased by almost 74% when the event sentiment of Hong Kong Protest, Mexican Election, Refugees Welcome, and US Election were incorporated as inputs. In addition, the event of the US election stands out as being the most important event because the error of the DL model has decreased by approximately 98.65% when we have used the sentiment of US Election.  Figure 8 shows the plot of 500 actual and forecasted values of the interest rate of Mexico when we incorporated the events of Refugee Welcome and Brexit as input in the LR, SVM, and DL models. The graph shows that the forecasted values of the SVM and DL models are close to their actual values as compared to LR.   When we incorporated the event sentiment of the US election and Mexico election to plot the graph, we could not see any fluctuation in the spread because there was no significant variation or trend in the data of the interest rate of Mexico. Therefore, the graph of these two events is not shown here.
The results of the interest rate prediction of Hong Kong are presented in Table 10. Our results provide evidence that event sentiment is a significant input to predict the interest rate of Hong Kong because the results have significantly improved in the case of all the events except Brexit. The mean error of the DL model has decreased by almost 266% when event sentiments were incorporated as inputs.  When we incorporated the event sentiment of all the events to plot the graph of forecasted and actual values, we could not see any fluctuation in the spread because there was no significant variation or trend in the data of the interest rate of Hong Kong. Therefore, the graph is not shown here.

Conclusions
This research study demonstrates the deep learning technique to predict the interest rate of the UK, Turkey, China, Hong Kong, and Mexico. We used the daily data of the interest rate, ranging from 1st of Jan 2010 to 23rd of Oct 2019. The exchange rate of the relevant country and twitter-based sentiment of some mega-events like the US election 2012, Mexican election 2012, Gaza under attack 2014, Hong Kong protest 2014, Refugee Welcome 2015, and Brexit 2016 were used as input parameters of the model. We calculated the sentiment using the twitter dataset of almost 9.6 million tweets in response to these six mega-events. Our results suggest that the deep learning model outperforms the support vector model and linear regression. Furthermore, the events sentiment has improved the predictive power of the deep learning model. Since the interest rate is highly volatile and complex in nature, it is also responsive towards mega-events happening across the globe. Therefore, we argue that the interest rate is sensitive towards the social media response of investors, which may also affect financial inclusion in the economy. Therefore, the social media sentiment can be used along with deep learning models to improve the performance of financial predictions, as these time series data are dependent on general opinion. These models can be further extended to design intelligent business plans which can guide investors to invest and gain maximum profit. Moreover, these results are also useful for policymakers when implementing monetary policy because the interest rate is one of the major tools for managing money supply in the economy.