Comparative Analysis of Machine Learning, Hybrid, and Deep Learning Forecasting Models: Evidence from European Financial Markets and Bitcoins
Abstract
:1. Introduction
2. Research Background
3. Data and Methodology
3.1. Data Analysis
3.2. Methodology
3.2.1. Data Pre-Processing
3.2.2. Autoregressive Integrated Moving Average (ARIMA)
3.2.3. Hybrid ETS-ANN Model
3.2.4. k-Nearest Neighbor Classifier (kNN)
3.2.5. Forecasting Performance Measures
4. Discussion
5. Conclusions
Funding
Data Availability Statement
Conflicts of Interest
Abbreviations
ACF | autocorrelation function |
ADF | augmented Dickey–Fuller |
AIC | Akaike information criterion |
ANN | artificial neural network |
ARCH | autoregressive conditional heteroskedasticity |
ARIMA | autoregressive integrated moving average |
AvgRelMAE | average relative mean absolute error |
ETS | exponential smoothing |
h | horizon |
kNN | k-nearest neighbor classifier |
MAE | mean absolute error |
MAPE | mean absolute percentage error |
RMSE | root mean square error |
Appendix A. Forecasting the Model Parameters
Year | ||||||
---|---|---|---|---|---|---|
Asset | Model | 2018–2021 | 2018 | 2019 | 2020 | 2021 |
BTC-USD | ARIMA | (4, 1, 1) | (2, 1, 2) | (1, 0, 5) | (4, 1, 4) | (3, 0, 3) |
GDAXI | (1, 1, 1) | (3, 1, 2) | (1, 1, 5) | (3, 0, 4) | (1, 1, 4) | |
FTSE | (4, 1, 3) | (5, 0, 2) | (5, 1, 1) | (3, 0, 5) | (2, 1, 4) | |
N100 | (5, 1, 0) | (3, 1, 2) | (2, 1, 1) | (1, 0, 0) | (1, 1, 5) | |
FCHI | (5, 1, 4) | (3, 0, 4) | (2, 1, 2) | (3, 0, 2) | (1, 1, 4) | |
SSMI | (1, 1, 5) | (2, 1, 5) | (5, 1, 0) | (4, 0, 2) | (4, 1, 5) | |
Note: optimal parameters based on information criterion ‘AIC’. | ||||||
BTC-USD | kNN | k = 26 | k = 27 | k = 26 | k = 9 | k = 27 |
GDAXI | k = 19 | k = 27 | k = 18 | k = 21 | k = 18 | |
FTSE | k = 25 | k = 27 | k = 27 | k = 16 | k = 25 | |
N100 | k = 27 | k = 20 | k = 23 | k = 27 | k = 21 | |
FCHI | k = 14 | k = 24 | k = 20 | k = 6 | k = 18 | |
SSMI | k = 27 | k = 22 | k = 15 | k = 14 | k = 27 | |
Note: weights: uniform; algorithm: brute; p: 2; k: optimal k. | ||||||
ETS-NN | ETS | NN | ||||
ETS (A, Ad, N) | LSTM layers: 50 | |||||
Dropout rate: 0.2 | ||||||
Optimizer: Adam | ||||||
Loss Function: Mean Squared Error | ||||||
Number of Epochs: 100 | ||||||
Batch Size: 32 |
References
- Stolberg, S.G. Biden Says the Pandemic Is Over. But at Least 400 People Are Dying Daily. The New York Times, 19 September 2022. Available online: https://www.nytimes.com/2022/09/19/us/politics/biden-covid-pandemic-over.html (accessed on 28 September 2022).
- Fisher, A.; Roberts, A.; McKinlay, A.R.; Fancourt, D.; Burton, A. The impact of the COVID-19 pandemic on mental health and well-being of people living with a long-term physical health condition: A qualitative study. BMC Public Health 2021, 21, 1801. [Google Scholar] [CrossRef]
- Ashraf, B.N. Economic impact of government interventions during the COVID-19 pandemic: International evidence from financial markets. J. Behav. Exp. Financ. 2020, 27, 100371. [Google Scholar] [CrossRef]
- Watson, R.; Popescu, G.H. Will the COVID-19 pandemic lead to long-term consumer perceptions, behavioral intentions, and acquisition decisions? Econ. Manag. Financ. Mark. 2021, 16, 70–83. [Google Scholar]
- Mazur, M.; Dang, M.; Vega, M. COVID-19 and the march 2020 stock market crash. Evidence from S&P500. Financ. Res. Lett. 2021, 38, 101690. [Google Scholar] [CrossRef]
- Goldstein, I.; Koijen, R.S.; Mueller, H.M. COVID-19 and its impact on financial markets and the real economy. Rev. Financ. Stud. 2021, 34, 5135–5148. [Google Scholar] [CrossRef]
- Di, M.; Xu, K. COVID-19 vaccine and post-pandemic recovery: Evidence from Bitcoin cross-asset implied volatility spillover. Financ. Res. Lett. 2022, 50, 103289. [Google Scholar] [CrossRef]
- Aslam, F.; Mohti, W.; Ferreira, P. Evidence of intraday multifractality in European stock markets during the recent coronavirus (COVID-19) outbreak. Int. J. Financ. Stud. 2020, 8, 31. [Google Scholar] [CrossRef]
- Khattak, M.A.; Ali, M.; Rizvi, S.A.R. Predicting the European stock market during COVID-19: A machine learning approach. MethodsX 2021, 8, 101198. [Google Scholar] [CrossRef] [PubMed]
- Su, C.W.; Rizvi, S.K.A.; Naqvi, B.; Mirza, N.; Umar, M. COVID19: A blessing in disguise for European stock markets? Financ. Res. Lett. 2022, 49, 103135. [Google Scholar] [CrossRef] [PubMed]
- Lachaab, M.; Omri, A. Machine and deep learning-based stock price prediction during the COVID-19 pandemic: The case of CAC 40 index. EuroMed J. Bus. 2023. ahead-of-print. [Google Scholar] [CrossRef]
- Topcu, M.; Gulal, O.S. The impact of COVID-19 on emerging stock markets. Financ. Res. Lett. 2020, 36, 101691. [Google Scholar] [CrossRef] [PubMed]
- Azimli, A. The impact of COVID-19 on the degree of dependence and structure of risk-return relationship: A quantile regression approach. Financ. Res. Lett. 2020, 36, 101648. [Google Scholar] [CrossRef] [PubMed]
- Goodell, J.W.; Goutte, S. Co-movement of COVID-19 and Bitcoin: Evidence from wavelet coherence analysis. Financ. Res. Lett. 2021, 38, 101625. [Google Scholar] [CrossRef] [PubMed]
- Akhtaruzzaman, M.; Boubaker, S.; Nguyen, D.K.; Rahman, M.R. Systemic risk-sharing framework of cryptocurrencies in the COVID-19 crisis. Financ. Res. Lett. 2022, 47, 102787. [Google Scholar] [CrossRef]
- Uddin, G.S.; Yahya, M.; Goswami, G.G.; Lucey, B.; Ahmed, A. Stock market contagion during the COVID-19 pandemic in emerging economies. Int. Rev. Econ. Financ. 2022, 79, 302–309. [Google Scholar] [CrossRef]
- Ampountolas, A. The Effect of COVID-19 on Cryptocurrencies and the Stock Market Volatility: A Two-Stage DCC-EGARCH Model Analysis. J. Risk Financ. Manag. 2023, 16, 25. [Google Scholar] [CrossRef]
- Hyndman, R.J.; Khandakar, Y. Automatic time series forecasting: The forecast package for R. J. Stat. Softw. 2008, 27, 1–22. [Google Scholar] [CrossRef] [Green Version]
- Panigrahi, S.; Behera, H.S. A hybrid ETS–ANN model for time series forecasting. Eng. Appl. Artif. Intell. 2017, 66, 49–59. [Google Scholar] [CrossRef]
- Ampountolas, A. Cryptocurrencies Intraday High-Frequency Volatility Spillover Effects Using Univariate and Multivariate GARCH Models. Int. J. Financ. Stud. 2022, 10, 51. [Google Scholar] [CrossRef]
- Leippold, M.; Wang, Q.; Zhou, W. Machine learning in the Chinese stock market. J. Financ. Econ. 2022, 145, 64–82. [Google Scholar] [CrossRef]
- Ciner, C. Stock return predictability in the time of COVID-19. Financ. Res. Lett. 2021, 38, 101705. [Google Scholar] [CrossRef] [PubMed]
- Al-Awadhi, A.M.; Alsaifi, K.; Al-Awadhi, A.; Alhammadi, S. Death and contagious infectious diseases: Impact of the COVID-19 virus on stock market returns. J. Behav. Exp. Financ. 2020, 27, 100326. [Google Scholar] [CrossRef] [PubMed]
- Liu, H.; Manzoor, A.; Wang, C.; Zhang, L.; Manzoor, Z. The COVID-19 outbreak and affected countries stock markets response. Int. J. Environ. Res. Public Health 2020, 17, 2800. [Google Scholar] [CrossRef] [Green Version]
- Barker, J. Machine learning in M4: What makes a good unstructured model? Int. J. Forecast. 2020, 36, 150–155. [Google Scholar] [CrossRef]
- Shanker, M.; Hu, M.Y.; Hung, M.S. Effect of data standardization on neural network training. Omega 1996, 24, 385–397. [Google Scholar] [CrossRef]
- Shankar, K.H.; Singh, I.; Howard, M.W. Neural mechanism to simulate a scale-invariant future. Neural Comput. 2016, 28, 2594–2627. [Google Scholar] [CrossRef]
- Hyndman, R.J.; Athanasopoulos, G. Forecasting: Principles and Practice; OTexts: Melbourne, Australia, 2018. [Google Scholar]
- Dickey, D.A.; Fuller, W.A. Distribution of the estimators for autoregressive time series with a unit root. J. Am. Stat. Assoc. 1979, 74, 427–431. [Google Scholar]
- Zhang, G.; Patuwo, B.E.; Hu, M.Y. Forecasting with artificial neural networks:: The state of the art. Int. J. Forecast. 1998, 14, 35–62. [Google Scholar] [CrossRef]
- Zhang, G.P. Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 2003, 50, 159–175. [Google Scholar] [CrossRef]
- Babu, C.N.; Reddy, B.E. A moving-average filter based hybrid ARIMA–ANN model for forecasting time series data. Appl. Soft Comput. 2014, 23, 27–38. [Google Scholar] [CrossRef]
- Khandelwal, I.; Adhikari, R.; Verma, G. Time series forecasting using hybrid ARIMA and ANN models based on DWT decomposition. Procedia Comput. Sci. 2015, 48, 173–179. [Google Scholar] [CrossRef] [Green Version]
- Fix, E.; Hodges, J.L., Jr. Discriminatory Analysis-Nonparametric Discrimination: Small Sample Performance; Technical Report; University of California, Berkeley: Berkeley, CA, USA, 1952. [Google Scholar]
- Neath, R.C.; Johnson, M.S.; Baker, E.; McGaw, B.; Peterson, P. Discrimination and Classification. In International Encyclopedia of Education, 3rd ed.; Baker, E., McGaw, B., Peterson, P., Eds.; Elsevier Ltd.: London, UK, 2010; Volume 1, pp. 135–141. [Google Scholar]
- Weinberger, K.Q.; Saul, L.K. Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 2009, 10, 207–244. [Google Scholar]
- Dornaika, F.; Bosaghzadeh, A.; Salmane, H.; Ruichek, Y. Object Categorization Using Adaptive Graph-Based Semi-supervised Learning. In Handbook of Neural Computation; Elsevier: Amsterdam, The Netherlands, 2017; pp. 167–179. [Google Scholar]
- Kim, S.; Kim, H. A new metric of absolute percentage error for intermittent demand forecasts. Int. J. Forecast. 2016, 32, 669–679. [Google Scholar] [CrossRef]
Index | Mean | Std Dev | Min | Max | SR | SE | ADF Stat | JB-Stat |
---|---|---|---|---|---|---|---|---|
Panel A: (1 January 2018–31 December 2018) | ||||||||
BTC-USD | −0.0030 | 0.0422 | −0.1685 | 0.1322 | −0.4289 | 0.0022 | −6.75 *** | 50.58 *** |
FCHI | −0.0003 | 0.0073 | −0.0332 | 0.0262 | −0.8446 | 0.0004 | −18.51 *** | 99.73 *** |
FTSE | −0.0003 | 0.0067 | −0.0315 | 0.0235 | −0.9122 | 0.0004 | −19.81 *** | 124.61 *** |
GDAXI | −0.0005 | 0.0081 | −0.0348 | 0.0290 | −0.8578 | 0.0004 | −20.11 *** | 57.25 *** |
N100 | −0.0003 | 0.0069 | −0.0342 | 0.0252 | −0.9059 | 0.0004 | −18.75 *** | 172.41 *** |
SSMI | −0.0003 | 0.0075 | −0.0313 | 0.0285 | −0.8031 | 0.0004 | −20.21 *** | 138.34 *** |
Panel B: (1 January 2019–31 December 2019) | ||||||||
BTC-USD | 0.0024 | 0.0356 | −0.1409 | 0.1736 | 0.1426 | 0.0019 | −19.63 *** | 343.33 *** |
FCHI | 0.0007 | 0.0070 | −0.0357 | 0.0272 | −0.2569 | 0.0004 | −19.23 *** | 346.17 *** |
FTSE | 0.0003 | 0.0062 | −0.0323 | 0.0225 | −0.5346 | 0.0003 | −18.52 *** | 286.89 *** |
GDAXI | 0.0006 | 0.0073 | −0.0311 | 0.0337 | −0.2735 | 0.0004 | −19.44 *** | 253.62 *** |
N100 | 0.0006 | 0.0066 | −0.0328 | 0.0266 | −0.2951 | 0.0003 | −19.42 *** | 323.99 *** |
SSMI | 0.0006 | 0.0055 | −0.0208 | 0.0228 | −0.3609 | 0.0003 | −18.11 *** | 97.09 *** |
Panel C: (1 January 2020–31 December 2020) | ||||||||
BTC-USD | 0.0046 | 0.0377 | −0.3717 | 0.1819 | 0.3790 | 0.0020 | −8.88 *** | 12,224.20 *** |
FCHI | −0.0001 | 0.0171 | −0.1228 | 0.0839 | −0.2841 | 0.0009 | −5.35 *** | 2144.66 *** |
FTSE | −0.0003 | 0.0153 | −0.1087 | 0.0905 | −0.3855 | 0.0008 | −5.36 *** | 2139.88 *** |
GDAXI | 0.0002 | 0.0173 | −0.1224 | 0.1098 | −0.2139 | 0.0009 | −4.80 *** | 2507.75 *** |
N100 | 0.0000 | 0.0159 | −0.1197 | 0.0818 | −0.2837 | 0.0008 | −5.33 *** | 2671.55 *** |
SSMI | 0.0001 | 0.0125 | −0.0964 | 0.0702 | −0.3435 | 0.0007 | −5.14 *** | 3246.57 *** |
Panel D: (1 January 2021–31 December 2021) | ||||||||
BTC-USD | 0.0022 | 0.0421 | −0.1377 | 0.1875 | 0.4751 | 0.0022 | −20.06 *** | 35.57 *** |
FCHI | 0.0007 | 0.0074 | −0.0475 | 0.0291 | −1.2687 | 0.0004 | −21.08 *** | 652.70 *** |
FTSE | 0.0004 | 0.0067 | −0.0364 | 0.0347 | −0.8552 | 0.0004 | −10.36 *** | 535.81 *** |
GDAXI | 0.0004 | 0.0076 | −0.0415 | 0.0331 | −0.8858 | 0.0004 | −22.11 *** | 351.68 *** |
N100 | 0.0006 | 0.0072 | −0.0428 | 0.0318 | −1.1399 | 0.0004 | −20.95 *** | 428.27 *** |
SSMI | 0.0005 | 0.0056 | −0.0238 | 0.0210 | −1.1433 | 0.0003 | −5.38 *** | 125.07 *** |
BTC-USD | FCHI | FTSE | GDAXI | N100 | SSMI | |
---|---|---|---|---|---|---|
BTC-USD | 1 | |||||
FCHI | 0.276 | 1 | ||||
FTSE | 0.263 | 0.900 | 1 | |||
GDAXI | 0.274 | 0.944 | 0.871 | 1 | ||
N100 | 0.284 | 0.989 | 0.913 | 0.948 | 1 | |
SSMI | 0.274 | 0.830 | 0.816 | 0.814 | 0.851 | 1 |
ARIMA | ETS-NN | kNN | |||||||
---|---|---|---|---|---|---|---|---|---|
MAE | RMSE | MAPE | MAE | RMSE | MAPE | MAE | RMSE | MAPE | |
Year 2018–2021 | |||||||||
BTC-USD | 13,471.98 | 15,702.97 | 25.44 | 6701.90 | 7862.06 | 15.48 | 22,518.47 | 23,801.17 | 28.45 |
GDAXI | 386.32 | 458.67 | 2.45 | 1290.97 | 1319.07 | 8.27 | 1253.96 | 1639.35 | 9.52 |
FTSE | 171.47 | 205.32 | 2.38 | 453.42 | 469.57 | 6.35 | 461.69 | 564.05 | 6.88 |
N100 | 58.16 | 69.84 | 4.40 | 121.69 | 127.14 | 9.34 | 127.21 | 147.98 | 12.37 |
FCHI | 357.04 | 425.17 | 5.24 | 755.48 | 786.81 | 11.27 | 503.26 | 646.80 | 9.15 |
SSMI | 859.91 | 983.97 | 7.03 | 944.07 | 1022.29 | 7.78 | 884.45 | 1110.99 | 8.54 |
Year 2018 | |||||||||
BTC-USD | 893.39 | 1297.50 | 19.42 | 2906.60 | 3030.60 | 72.02 | 1983.54 | 2480.08 | 37.19 |
GDAXI | 529.96 | 689.04 | 4.54 | 716.69 | 753.22 | 6.51 | 522.10 | 682.16 | 4.53 |
FTSE | 261.75 | 309.93 | 3.59 | 115.51 | 156.82 | 1.70 | 255.81 | 300.29 | 3.59 |
N100 | 37.81 | 47.12 | 3.78 | 35.38 | 41.67 | 3.76 | 44.13 | 52.73 | 4.47 |
FCHI | 188.01 | 233.41 | 3.65 | 182.92 | 212.41 | 3.78 | 232.37 | 294.40 | 4.61 |
SSMI | 158.89 | 202.98 | 1.81 | 158.99 | 187.17 | 1.81 | 172.57 | 204.05 | 1.94 |
Year 2019 | |||||||||
BTC-USD | 1468.02 | 1659.53 | 17.50 | 644.74 | 717.56 | 8.58 | 3895.68 | 4603.57 | 44.08 |
GDAXI | 459.71 | 605.76 | 3.61 | 777.87 | 784.44 | 5.89 | 594.54 | 701.98 | 4.76 |
FTSE | 123.80 | 148.80 | 1.70 | 138.75 | 194.79 | 1.85 | 113.94 | 141.66 | 1.56 |
N100 | 32.24 | 39.62 | 2.93 | 38.34 | 40.08 | 3.38 | 42.75 | 48.54 | 4.00 |
FCHI | 202.94 | 255.31 | 3.55 | 269.60 | 277.26 | 4.55 | 267.13 | 320.31 | 4.72 |
SSMI | 344.25 | 438.14 | 3.38 | 493.55 | 509.50 | 4.72 | 371.20 | 423.61 | 3.73 |
Year 2020 | |||||||||
BTC-USD | 4412.44 | 6286.24 | 26.43 | 7256.93 | 7863.49 | 37.23 | 3067.44 | 4958.43 | 22.51 |
GDAXI | 818.08 | 909.35 | 6.42 | 175.36 | 257.79 | 1.35 | 723.74 | 1137.54 | 6.42 |
FTSE | 230.45 | 303.38 | 3.86 | 389.48 | 411.95 | 6.05 | 399.32 | 517.23 | 6.34 |
N100 | 35.59 | 43.82 | 3.55 | 86.94 | 90.54 | 7.97 | 78.33 | 97.47 | 7.71 |
FCHI | 254.47 | 291.15 | 5.20 | 450.81 | 472.97 | 8.18 | 455.62 | 529.93 | 9.01 |
SSMI | 200.22 | 239.08 | 1.96 | 101.48 | 131.54 | 0.97 | 288.88 | 507.73 | 2.98 |
Year 2021 | |||||||||
BTC-USD | 6944.45 | 8983.94 | 13.41 | 4425.80 | 5468.44 | 8.39 | 7181.29 | 9254.76 | 17.71 |
GDAXI | 389.88 | 462.52 | 2.48 | 284.25 | 344.78 | 1.81 | 611.14 | 911.19 | 4.24 |
FTSE | 199.91 | 230.71 | 2.78 | 166.57 | 186.08 | 2.28 | 227.05 | 259.67 | 3.24 |
N100 | 59.84 | 71.61 | 4.53 | 39.93 | 45.18 | 2.94 | 74.28 | 93.97 | 6.14 |
FCHI | 336.90 | 405.34 | 4.94 | 336.55 | 366.36 | 4.78 | 426.25 | 540.26 | 6.91 |
SSMI | 861.34 | 985.47 | 7.04 | 446.78 | 497.18 | 3.55 | 527.14 | 725.35 | 4.69 |
Model | BTC-USD | GDAXI | FTSE | N100 | FCHI | SSMI |
---|---|---|---|---|---|---|
Year 2018–2021 | ||||||
ETS-NN | 0.497 | 3.342 | 2.644 | 2.092 | 2.116 | 1.098 |
kNN | 1.672 | 3.246 | 2.693 | 2.187 | 1.410 | 1.029 |
ARIMA | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
Year 2018 | ||||||
ETS-NN | 3.253 | 1.352 | 0.441 | 0.936 | 0.973 | 1.001 |
kNN | 2.220 | 0.985 | 0.977 | 1.167 | 1.236 | 1.086 |
ARIMA | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
Year 2019 | ||||||
ETS-NN | 0.063 | 1.557 | 0.955 | 1.198 | 1.325 | 1.252 |
kNN | 0.382 | 1.190 | 0.784 | 1.336 | 1.313 | 0.942 |
ARIMA | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
Year 2020 | ||||||
ETS-NN | 1.634 | 0.082 | 0.768 | 0.807 | 0.904 | 0.159 |
kNN | 0.690 | 0.339 | 0.788 | 0.727 | 0.913 | 0.452 |
ARIMA | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
Year 2021 | ||||||
ETS-NN | 0.321 | 0.729 | 0.851 | 0.644 | 0.880 | 0.511 |
kNN | 0.521 | 1.566 | 1.160 | 1.197 | 1.115 | 0.603 |
ARIMA | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ampountolas, A. Comparative Analysis of Machine Learning, Hybrid, and Deep Learning Forecasting Models: Evidence from European Financial Markets and Bitcoins. Forecasting 2023, 5, 472-486. https://doi.org/10.3390/forecast5020026
Ampountolas A. Comparative Analysis of Machine Learning, Hybrid, and Deep Learning Forecasting Models: Evidence from European Financial Markets and Bitcoins. Forecasting. 2023; 5(2):472-486. https://doi.org/10.3390/forecast5020026
Chicago/Turabian StyleAmpountolas, Apostolos. 2023. "Comparative Analysis of Machine Learning, Hybrid, and Deep Learning Forecasting Models: Evidence from European Financial Markets and Bitcoins" Forecasting 5, no. 2: 472-486. https://doi.org/10.3390/forecast5020026
APA StyleAmpountolas, A. (2023). Comparative Analysis of Machine Learning, Hybrid, and Deep Learning Forecasting Models: Evidence from European Financial Markets and Bitcoins. Forecasting, 5(2), 472-486. https://doi.org/10.3390/forecast5020026