An Integrative System Based on Signal Processing and Tuned Regression Gaussian Process by Grey Wolf Optimization Algorithm for Bitcoin Price Forecasting

Abstract

We propose various hybrid predictive systems to forecast the Bitcoin next-day price. In particular, we combine the decomposition methods based on signal processing techniques including maximum overlap discrete wavelet transform (MODWT), empirical wavelet transform (EWT), empirical mode decomposition (EMD), and variational mode decomposition (VMD) for feature extraction from original price series. Then, the extracted features are fed to the machine learning models for training and forecasting. We implemented five machine learning models, including regression Gaussian process (RGP), support vector regression (SVR), k-nearest neighbors algorithm (kNN), regression trees (RT), and feedforward neural networks (FFNN). The grey wolf optimization (GWO) algorithm is employed for hyperparameter optimization of the machine learning models. The root mean squared error (RMSE) is used for the evaluation and comparison of 20 hybrid predictive systems. The simulation results show that the RGP-GWO-VMD hybrid predictive system achieved the lowest forecasting error. In addition, RGP-GWO yielded on average the lowest forecasting error across all of the machine learning systems. Furthermore, among signal decomposition methods, the lowest forecasting error is generally achieved under the EWT. Hence, we presented the best results in forecasting Bitcoin prices from 20 hybrid prediction systems to serve as the baseline for future work and to guide traders, investors, and portfolio managers.

1. Introduction

Cryptocurrency is a digital currency traded in computer networks, and it is not subjected to government or any central authority. Nowadays, trading cryptocurrencies is attracting many investors and scholars, as they offer a high level of openness and high returns but with a high risk and volatility. Bitcoin is broadly acknowledged as one of the most prominent and traded cryptocurrencies. Therefore, investors and traders have shifted their attention to the trading of cryptocurrencies, especially Bitcoin. In this regard, forecasting Bitcoin prices is crucial for investors and traders, primarily for better portfolio management, specifically in the short term to generate high profits from active trading. However, it is challenging to accurately predict Bitcoin price since it is particularly volatile. Therefore, scholars have proposed various models to predict Bitcoin price in recent years.

For instance, Atsalakis et al. [1] employed a hybrid neuro-fuzzy controller to forecast the direction of the change in the daily price of Bitcoin. They concluded that it outperformed the adaptive neuro-fuzzy system (ANFIS) and the standard artificial neural network. Lahmiri and Bekiros [2] found that the predictability of long-short-term memory neural network topologies (LSTM) is significantly higher when compared to the generalized regression neural network (GRNN). Mallqui and Fernandes [3] proposed a predictive system based on the combination of recurrent neural networks (RNN) and decision tree classifiers and found that their system obtained the best results to predict the Bitcoin price direction compared to LSTM and recurrent neural networks (RNN). In addition, the support vector machines (SVM) algorithm obtained the best results to forecast the Bitcoin price compared to the standard artificial neural networks and RNN. Lahmiri and Bekiros [4] implemented and compared the performance of support vector regression (SVR), Gaussian Poisson regressions (GRP), regression trees (RT), k-nearest neighbours (kNN), feedforward neural networks (FFNN), Bayesian regularization neural networks (BRNN), and radial basis function networks (RBFNN). The optimal parameters of the SVR, GRP, and kNN were determined by using the Bayesian optimization method. They found that the BRNN achieved superior accuracy compared to the remaining predictive models. Guo et al. [5] combined the wavelet transform (WT) with a casual multi-head attention temporal convolutional network and found that their proposed model improves the price forecasting performance compared to the autoregressive integrated moving average (ARIMA), ARIMA with additional explanatory variables (ARIMAX), convolutional neural networks (CNN), multilayer perceptron (MLP), LSTM, sequence-to-sequence (Seq2Seq) network, Bayesian neural networks (BNN), and state-frequency memory (SFM) recurrent neural networks. Koo and Kim [6] found that the LSTM yielded the highest accuracy in predicting price direction movement, followed by RNN and standard MLP. In addition, it was shown that the flattening distribution strategy (FDS), used to eliminate the difference between outliers and non-outliers, significantly improves the accuracy of the LSTM, RNN, and MLP. Lahmiri and Bekiros [7] investigated the effect of standard numerical training algorithms, including conjugate gradient with Powell-Beale restarts, the resilient algorithm, and Levenberg–Marquardt algorithm on the accuracy of the deep feedforward neural network (DFFNN) and concluded that the DFFNN trained with the Levenberg–Marquardt algorithm outperforms the DFFNN trained with Powell-Beale restarts algorithm and DFFNN trained with the resilient algorithm in forecasting intraday prices of Bitcoin. Rajabi et al. [8] concluded that the learnable window size-MLP (LWS-MLP) is superior to the SVR, ARIMA, random forests (RF), LSTM, MLP, and WaveNet. Rathore et al. [9] found that Facebook’s Prophet performed better than the Naïve model by using open, high, low, volume, and market capitalization as inputs. Hajek et al. [10] concluded that the bagged support vector regression (BSVR) trained with sentiment index outperformed ARIMA, feedforward neural network, random forest (RF), radial basis function neural network (RBFNN), stacked artificial neural network (SANN), and bidirectional long short-term memory (Bi-LSTM). Zou and Herremans [11] proposed a hybrid multimodal model that consists of a support vector machine (SVM) trained with price data, which is fused with a text-based CNN. They concluded that their model can be used to build a profitable trading strategy with a reduced risk over a hold or moving average strategy. Cheng et al. [12] found that LSTM has a noticeable improvement compared to seasonal autoregressive integrated moving average (SARIMA) and Facebook Prophet. Abul Basher et al. [13] concluded that RF predicts Bitcoin price directions with a higher degree of accuracy than logit models.

A common limitation across the above studies is the absence of directly integrating signal processing algorithms to extract intrinsic information in Bitcoin data and tuned machine learning to improve accuracy into the forecasting framework. Instead, they rely on direct inputs such as historical observations or exogenous variables to train machine learning systems. This classical forecasting process may introduce noisy data, which is not appropriate to improve accuracy. Also, other studies used pretrained deep learning models on very large datasets composed of different and various signals and images. Such pretrained deep learning models are not necessarily suitable to analyze and predict the underlying Bitcoin data. In addition, previous works have not considered optimization of the machine learning models to improve their effectiveness.

Given the complexity of the dynamics underlying Bitcoin prices and the inherent uncertainty in forecasting such volatile prices, this paper puts forth novel hybrid predictive systems for the purpose of forecasting Bitcoin’s next-day price. Specifically, this study proposes the hybridization of signal processing techniques and machine learning to forecast Bitcoin’s future price. First, signal decomposition methods are used for decomposing the original price time series to fully leverage the information contained within the data. Second, machine learning models are trained with decomposition-based information to predict the Bitcoin next-day price, and the grey wolf optimization (GWO) [14] algorithm is employed to optimize the hyper-parameters of each machine learning model, thereby enhancing its performance. Finally, the performance of each hybrid predictive system is evaluated based on the root mean of squared errors (RMSE). The signal decomposition methods include maximum overlap discrete wavelet transform (MODWT) [15], empirical wavelet transform (EWT) [16], empirical mode decomposition (EMD) [17], and variational mode decomposition (VMD) [18]. The machine learning models include the Gaussian process regression (GPR) [19], support vector regression (SVR) [20], k-nearest neighbors algorithm (kNN) [21], regression trees (RT) [22], and feedforward neural networks (FFNN) [23].

We rely on signal processing/decomposition methods, as they are basically multiresolution techniques used to decompose the original signal into intrinsic components to provide its representation in time-frequency space. Such components provide a meaningful description of the oscillations in the original signal. In addition, machine learning has become a revolutionary tool in time series forecasting. It includes various algorithms that allow computers to learn patterns from data and make predictions with no prior assumptions. For the hybrid system tuning, the grey wolf optimization (GWO) algorithm is chosen. Indeed, the GWO is a swarm intelligence optimization algorithm inspired by the hunting behaviour of grey wolves in nature. It is simple, as it uses only one operator to determine the positions of the solutions (wolves) for problem-solving [14]. Figure 1 shows the flow chart of the hybrid predictive systems.

The main contributions of our paper are as follows:

• We implement various signal decomposition algorithms to highlight multiresolution components of the original data.
• We implement various machine learning models for forecasting purposes.
• We design 20 predictive systems that integrate signal decomposition and machine learning models.
• Heuristic optimization is employed to tune all integrative predictive systems based on the GWO algorithm.
• To the best of our knowledge, this is the first time a comprehensive set of hybrid predictive systems has been designed and implemented to forecast Bitcoin’s next-day price.

The paper is structured as follows. Section 2 introduces the signal processing techniques, machine learning models, and grey wolf optimization algorithm. Section 3 presents the data and the simulation results from each hybrid predictive system. Lastly, Section 4 concludes the paper.

2. Methods

To extract intrinsic characteristics of Bitcoin price data, we implement various signal processing algorithms, including MODWT, EWT, EMD, and VMD. Thanks to their ability to analyze signals, they were applied to various engineering applications. For instance, the MODWT was successful in various applications, including the prediction of pan evaporation [24], solar radiation [25], irrigation flow [26], and wave height [27]. EMD was successful in diverse tasks, including prediction of tourism demand [28], CO₂ concentration [29], water quality [30], and oil production [31]. The EWT was effective in different problems, including the prediction of wave height [32], wind speed [33], wave power [34], and fault in the wheelset-bearing system [35]. The VMD was used in numerous problems, including the prediction of ship motion attitude [36], pumped storage hydropower unit [37], fault in cryogenic rolling bearing fault [38], and wind speed [39]. In addition, the GWO algorithm is adopted in our study to fine-tune machine learning systems as it was found to be useful in tuning the hyperparameters of the MLP [40,41], support vector machines [42], and AdaBoost, Bagging, and backpropagation neural networks [43]. The variational mode decomposition, Gaussian process regression, and grey wolf optimization algorithm are described next.

2.1. Variational Mode Decomposition (VMD)

The variational mode decomposition (VMD) [18] is an adaptive signal processing technique used to decompose the original signal into variational modes functions (VMFs) where each VMF is centred on a specific frequency s_k estimated during the decomposition process. The determination of centre frequencies and bandwidths for the component signals requires seeking K modal functions such that the total bandwidth of all decomposed VMFs is minimized and their sum equals the original signal. The amplitude and frequency of each VMF at any given point in time are determined using the Hilbert transform, which generates a unilateral spectrum. The bandwidth of each VMF is calculated based on the following steps: (a) apply the Hilbert transform to each VMF to obtain its spectrum, (b) shift the spectral distribution of each VMF to its corresponding frequency baseband, and (c) determine the signal bandwidth using the Wiener filter. The decomposition algorithm is represented in the form of a constrained optimization problem:

$${min}_{u_k{s}_k}=\left\{{\sum }_{k=1}^K{‖{\delta }_t\left[\left(\delta \left(t\right)+{\displaystyle \frac{j}{\pi t}}\right)\times d_k\left(t\right)\right]e^{-j{\omega }_{k}t}‖}_2^2\right\}$$

(1)

Subject to:

$${\sum }_{k=1}^K{d}_k\left(t\right)=x\left(t\right)$$

(2)

where s_k is centre frequency and d_k(t) is the kth mode of the signal. In this work, the number of modes is set to five.

2.2. Regression Gaussian Process

The regression Gaussian process (RGP) [19] is a non-parametric model useful to model uncertainty in the data. The implicit function f(x) is assumed to follow the Gaussian distribution as follows:

$$f\left(x\right)~GP\left(m\left(x\right),k_f\left(x,x^{\prime }\right)\right)$$

(3)

where the m(x) is the mean; the k_f(x,x′) represents the kernel function. In this work the square exponential function is chosen:

$$k_f\left(x,x^{\prime }\right)~{\sigma }_f^{2}exp\left(-{\displaystyle \frac{{\left(x-x^{\prime }\right)}^2}{2l^2}}\right)$$

(4)

where ${\sigma }_f^2$ is the variance of the kernel function and l is the length. Then, the implicit function f(x) can be expressed as:

$$y=f\left(x_n\right)+{\xi }_n$$

(5)

where ξ_n∼N(0, ${\sigma }_n^2$) is the Gaussian noise.

2.3. Grey Wolf Optimization

The grey wolf optimization (GWO) [14] is an evolutionary metaheuristic optimization algorithm based on swarm intelligence to simulate the hunting behaviour and social leadership of grey wolves. To replicate the leadership hierarchy, the GWO algorithm includes four types of grey wolves: namely, α, β, δ, and ω. In grey wolves, the social hierarchy of wolves, α is considered as the fittest solution, and β and δ are described as the second and third solutions, respectively, and ω is after these three wolves. Specifically, hunting operations are commonly led by α and β, and δ wolves could intermittently hunt. The GWO algorithm consists of 3 main steps: encircling, hunting, and attacking prey.

The grey wolves encircle the prey, where their positions are specified by:

$$\overrightarrow{X}\left(t+1\right)={\overrightarrow{X}}_p\left(t\right)-\overrightarrow{A}·\overrightarrow{D}$$

(6)

$$\overrightarrow{D}={\overrightarrow{C}·\overrightarrow{X}}_p\left(t\right)-\overrightarrow{X}\left(t\right)$$

(7)

where t is iteration number, and:

$$\overrightarrow{A}=2\overrightarrow{a}·\overrightarrow{r_1}-\overrightarrow{a}$$

(8)

$$\overrightarrow{C}=2·\overrightarrow{r_2}$$

(9)

where $\overrightarrow{a}$ is linearly decreasing from 2 to 0 during iterations and $\overrightarrow{r_1}$ and $\overrightarrow{r_2}$ are random vectors in [0, 1]. The Grey wolf hunting is based on update of the prey position with the help of the three wolves as follows:

$$\overrightarrow{D_{\alpha }}=\left|\overrightarrow{C_1}·\overrightarrow{X_{\alpha }}-\overrightarrow{X}\right|$$

(10)

$$\overrightarrow{D_{\beta }}=\left|\overrightarrow{C_2}·\overrightarrow{X_{\beta }}-\overrightarrow{X}\right|$$

(11)

$$\overrightarrow{D_{\delta }}=\left|\overrightarrow{C_{31}}·\overrightarrow{X_{\delta }}-\overrightarrow{X}\right|$$

(12)

Finally, depending on the estimated positions of α, β, and δ, the solutions are updated for every iteration as follows:

$$\overrightarrow{X_1}=\overrightarrow{X_{\alpha }}-\overrightarrow{A_1}·\left(\overrightarrow{D_{\alpha }}\right)$$

(13)

$$\overrightarrow{X_2}=\overrightarrow{X_{\beta }}-\overrightarrow{A_2}·\left(\overrightarrow{D_{\beta }}\right)$$

(14)

$$\overrightarrow{X_3}=\overrightarrow{X_{\delta }}-\overrightarrow{A_3}·\left(\overrightarrow{D_{\delta }}\right)$$

(15)

$$\overrightarrow{X}\left(t+1\right)={\displaystyle \frac{\overrightarrow{X_1}+\overrightarrow{X_2}+\overrightarrow{X_3}}{3}}$$

(16)

In this study, the GWO is adopted to optimize the parameters and kernels of the predictive systems.

In this study, the GWO algorithm is employed to optimize each single predictive system. For GRP, it is used to optimize the length of the kernel function. For SVR, it is used to optimize the spatial distribution status parameter of the radial basis function kernel. For RT, it is employed to determine the optimal number of splits (divisions). For kNN, GWO is used to determine the optimal distance metric and the number of neighbors. Finally, for FFNN, it is employed to determine the number of hidden layers, the number of neurons in hidden layers, and the type of activation function, for instance, sigmoid, hyperbolic, Relu, or soft plus.

2.4. Performance Measures

For performance evaluation, three common performance metrics are used including the root mean squared error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE). The RMSE calculates the error magnitude that is in the same units as the target variable; MAE offers a straightforward measure of prediction accuracy; and MAPE facilitates interpretation of the performance in terms of relative error.

The RMSE computes the square root of the average of squared differences between predicted $\widehat{y}$ and actual values y over n observations. It is given by:

$$\mathrm{RMSE} = \sqrt{{\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{\left.{\left(y\right.}_i-\widehat{y_i}\right)}^2}$$

(17)

The MAE calculates the average of absolute differences between predicted and actual values, focusing on the average magnitude of errors, regardless of their direction. It is expressed as follows:

$$\mathrm{MAE} = {\displaystyle \frac{1}{n}}{\sum }_{i=1}^n\left|y_i-\widehat{y_i} \right|$$

(18)

The MAPE computes the average percentage difference between predicted and actual values.

$$\mathrm{MAPE}={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n\left|{\displaystyle \frac{y_i-\widehat{y_i} }{y_i}}\right|\times 100$$

(19)

The lower the performance metric, the better the accuracy of the predictive system.

3. Results

We used Bitcoin’s daily price data in US dollars from 27 March 2018 to 27 March 2023 from Yahoo Finance. Figure 2 displays the price series of Bitcoin. One could observe that Bitcoin price data is not stationary and noisy. We used the first 80% of data for training the models and the remaining 20% for testing. As indicated previously, the standard root mean of squared errors (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) are employed to evaluate the performance of each predictive system. In this regard, RMSE and MAE are expressed in US dollars.

Table 1. RMSE obtained by each predictive system.

	MODWT	EWT	EMD	VMD
FFNN-GWO	23.7585	26.9270	35.7726	25.5540
RGP-GWO	9.5656	0.0004	0.8004	0.0003
kNN-GWO	8904.5	24,488	27,983	19,195
SVR-GWO	10.8910	0.0129	0.2843	0.0079
RT-GWO	30.2288	1.9575	24.5190	4.5349

provides the obtained RMSE for each forecasting approach used to combine the signal processing technique with machine learning. As indicated, the RGP model trained with VMD-resulting VMFs and optimized the GWO algorithm (RGP-VMD-GWO) achieved the lowest forecasting error measured by RMSE; for instance, 0.0003. In contrast, the kNN trained with EMD-resulting IMFs (kNN-EMD-GWO) yielded the highest forecasting error; for instance, RMSE = 27,983. In addition,

Table 2. MAE obtained by each predictive system.

	MODWT	EWT	EMD	VMD
FFNN-GWO	21,404.8508	25,120.2785	31,256.6425	22,502.4740
RGP-GWO	8127.6220	0.2500	684.7072	0.1980
kNN-GWO	8876.8000	22,850	19,071	16,114
SVR-GWO	8245.3294	12.0220	262.1148	6.8629
RT-GWO	25,916.7054	1473.6921	23,708.1446	3737.2589

provides the obtained MAE by each proposed prediction system. As shown, the RGP-GWO-VMD system yielded the lowest prediction error measured by MAE; for instance, 0.1980. Conversely, the FFNN-GWO-EMD obtained the highest prediction error; for instance, MAE = 31,256.6425. Also,

Table 3. MAPE obtained by each predictive system.

	MODWT	EWT	EMD	VMD
FFNN-GWO	71.6620	83.7710	115.2839	78.3467
RGP-GWO	29.3046	0.0008	2.3804	0.0006
kNN-GWO	7176.3	71.2246	19,183	36,839
SVR-GWO	27.0891	0.0417	0.8715	0.0233
RT-GWO	90.0347	5.0136	84.4070	13.1088

shows the MAPE obtained by each integrative forecasting system. The lowest MAPE (0.0006) is obtained by the RGP-GWO-VMD system, whilst the highest MAPE (36,839) is obtained by the kNN-GWO-VMD. From Table 1, Table 2 and Table 3, we conclude that all three performance measures yield the same conclusions: the RGP-GWO-VMD is the best predictive integrative system to forecast Bitcoin price based on all performance measures, whilst the hybrid kNN-GWO-EMD is the worst in terms of RMSE and MAPE, and the FFNN-GWO-EMD is the worst in terms of MAE.

In addition, we performed the Diebold-Mariano (DM) test to verify the null hypothesis of equal forecast accuracy between each baseline hybrid predictive system and RGP-GWO-VMD at a 5% statistical significance level. The results are provided in

Table 4. Results from D-M test: RGP-GWO-VMD versus reference models.

Reference Model	D-M Statistic	p-Value	Reference Model	D-M Statistic	p-Value
FFNN-GWO-CWT	5.5243	6.31 × 10−8	kNN-GWO-CWT	27.9533	1.2439 × 10−92
FFNN-GWO-MODWT	−6.9929	1.30 × 10−11	kNN-GWO-MODWT	27.7543	7.3203 × 10−92
FFNN-GWO-EMD	−28.1139	2.98 × 10−93	kNN-GWO-EMD	3.4482	6.3042 × 10−4
FFNN-GWO-VMD	−23.9177	1.18 × 10−76	kNN-GWO-VMD	23.5887	2.5358 × 10−75
SVR-GWO-CWT	43.0342	7.74 × 10−145	RT-GWO-CWT	15.4041	1.611 × 10−41
SVR-GWO-MODWT	−5.2386	2.74 × 10−7	RT-GWO-MODWT	−16.876	1.30 × 10−47
SVR-GWO-EMD	−15.7695	4.57 × 10−43	RT-GWO-EMD	−76.3537	4.14 × 10−226
SVR-GWO-VMD	19.2096	2.75 × 10−57	RT-GWO-VMD	−5.7176	2.25 × 10−8

The null hypothesis is that both RGP-GWO-VMD (the overall best model) and the reference model have equal predictive accuracy. The test is performed at 5% statistical significance.

. As shown, the null hypothesis is rejected for all pairs of comparisons, as the probability values of the test (p-value) are always below 5%. Thus, the forecasts from the RGP-GWO-VMD are consistently different from those of the other integrative predictive systems.

Finally, to summarize the results from simulations, we computed the average RMSE obtained by each predictive system. We found that the lowest average RMSE across signal processing algorithms was obtained by RGP-GWO (2.59), followed by SVR-GWO (2.8), RT-GWO (15.31), ANN-GWO (28.00), and kNN-GWO (20,142.63). Also, we computed the average RMSE for each signal processing algorithm. It is found that the lowest average RMSE was obtained by MODWT (1795.7887), followed by VMD (3845.0194), EWT (4903.3795), and EMD (5608.8752).

4. Conclusions

Forecasting cryptocurrencies is receiving growing attention as they represent alternative investments with high profits compared to traditional equities [44,45,46,47,48,49,50]. In this regard, various studies implemented statistical and machine learning models to forecast prices [51,52,53,54,55,56] and volatility of Bitcoin [57,58,59,60] which is the largest traded cryptocurrency.

In this study, we proposed novel integrative predictive systems to forecast Bitcoin’s next-day price. Indeed, we designed new integrative forecasting systems based on combinations of signal processing algorithms (for instance, MODWT, EWT, EMD, and VMD) and machine learning systems (for instance, RGP, SVR, kNN, RT, and FFNN). In addition, grey wolf optimization was adopted to find the best hyperparameter combinations to further enhance the performance of the integrative prediction systems and alleviate the risk of overfitting. In total, twenty integrative systems are implemented and tested.

The experimental results showed that the RGP-GWO-VMD hybrid predictive system was found to be the best predictive integrative system based on three performance measures including RMSE, MAE, and MAPE. In addition, according to Diebold-Mariano test, we concluded that the forecasts from the RGP-GWO-VMD are consistently different from those of the other integrative predictive systems. In this regard, the superiority of the RGP-GWO-VMD can be explained by the fact that the RGP is effective in modelling uncertainties inherent in the Bitcoin generating process. In addition, based on Wiener filtering and Hilbert transform, and by imposing band-limited limits on the decomposition process, VMD leads to better representation of complex temporal patterns like trends, noise, and seasonality in the original sequence. Also, the integration of GWO further enhances the RGP-VMD by optimizing the length scale parameter of RGP which leads to reduced uncertainty in predictions.

Furthermore, the MODWT multiresolution algorithm yielded the lowest average forecasting error measured by RMSE across machine learning systems. However, if the average RMSE is calculated for each signal processing algorithm across all machine learning systems when the least performer kNN-GWO is excluded, the MODWT algorithm obtains the highest value of RMSE (18.6109). The EWT, EMD, and VMD obtain 7.22445, 15.3440, and 7.5242, respectively. In other words, the performance of the least performer kNN-GWO improves with MODWT as it offers better time-frequency localization characteristics that can be exploited by kNN-GWO.

In summary, the empirical results showed that the proposed integrative forecasting system, RGP-GWO-VMD, achieved significant performance. Our study has the merit to shed light on the optimal design of an integrative predictive system used to forecast the Bitcoin next-day price based on the integration of signal processing, machine learning, and grey wolf optimization algorithm.

Future works would consider extending the study to other cryptocurrencies, stocks, and commodities.