^{*}

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (

This study proposes a multiple kernel learning (MKL)-based regression model for crude oil spot price forecasting and trading. We used a well-known trend-following technical analysis indicator, the moving average convergence and divergence (MACD) indicator, for extracting features from original spot prices. Additionally, we factored in the possibility that movements of target crude oil prices may be related to other important crude oil markets besides the target market for the prediction time horizon since traders may find price movement information within other relevant crude oil markets useful. We also considered multiple timeframes in this study since trends may differ across different timeframes and, in fact, traders may use their own timeframes. Therefore, for forecasting target crude oil prices, this study emphasizes on features pertaining to other important crude oil markets and different timeframes in addition to features of the target crude oil market and target timeframe. Moreover, the MKL framework has been used to fuse information extracted from different sources and timeframes of the same data source. Experimental results show that out-of-sample forecasting using the MKL method is superior to benchmark methods in terms of root mean square error (RMSE) and average percentage profit (

Crude oil is the world's most actively traded commodity, accounting for over 10% of total world trade [

West Texas Intermediate (WTI) and Brent Crude oil market are two of the world's most important crude oil markets. While Brent Crude oil is sourced from the North Sea and is primarily used in Europe, while WTI crude oil is refined mostly in the Midwest and Gulf Coast regions in the United States of America, and is mainly supplied to the North American market. Although crude oil prices in these two markets have a significant interrelationships, for instance, price fluctuations in one market impact prices in the other, price movements in these markets are not always similar because of differing crude oil quality characteristics and the diverse locations they cater to.

A fluctuation in crude oil prices may significantly impact a nation's economy. Forecasts assist in minimizing such risks arising from the uncertainty surrounding future crude oil prices. To this end, it is critical to engage in prediction exercises modeled for forecasting crude oil prices. Although many business practitioners and researchers have attempted to develop various forecasting methods to predict crude oil prices, it is extremely difficult to design a model that captures the various dimensions affecting future crude oil prices. Crude oil prices are strongly influenced by several factors, including gross domestic product (GDP) growth, political events, conflicts and wars, and financial policies relating to the US dollar (since crude oil is priced in US dollars), among others. Additionally, since crude oil sourced from different locations have varying qualities and transport costs at different rates are involved in shipping crude oil from one location to another, crude oil prices vary in different parts of the world. All these factors together contribute to strong fluctuations in the world market for crude oil, which has subsequently acquired the characteristics of complex nonlinearity, dynamic variation, and high irregularity.

Technical analysis is a way to forecast market prices of securities such as stocks based solely on the past prices and traded volumes, and technical indicators are usually used to do technical analysis. In the last few decades, numerous researchers [

Other researchers have used econometric models or traditional time series analysis methods such as co-integration analysis and autoregressive integrated moving average (ARIMA) for forecasting prices. For example, Huntington [

Since these models based on the linearity assumptions are not suitable for approximation of nonlinear patterns hidden in crude oil price series, this study has applied nonlinear models to predict crude oil prices. Some machine learning methods such as artificial neural networks (ANNs) and support vector machines (SVM) were proposed to solve the nonlinearity problems of time series and gave better results than conventional methods. For example, many researchers applied ANN based models [

In recent years, some researchers have applied the multiple kernel learning (MKL) [

For the purpose of forecasting crude oil prices by considering features from different sources and different representations, we propose to extract and use the features from two main crude oil spot markets and three different timeframes. The two markets in this context are WTI and Brent Crude oil markets, the two largest crude oil markets in the world. Although WTI crude oil is mainly supplied to North America and Brent Crude oil is mainly used in Europe, some interrelationship between these two markets cannot be ruled out, given the interdependence of worldwide oil markets in the highly integrated contemporary global economic system. For instance, the fluctuations in one market do not go unnoticed in the other market. Therefore, there is a strong case for referring to price movements in the other market for predicting crude oil prices in a particular market. In addition to extracting features from two different crude oil markets, the features of different timeframes are also considered as useful information for prediction.

In order to predict crude oil prices (WTI or Brent) in the target market, this study uses features from other crude oil markets besides features of the target market, and examines features from two time horizons other than the target timeframe. Features from different sources or features of different time representations may have different properties and quality characteristics. Given its efficient prediction performance observed in studies mentioned earlier [

The remainder of the paper is arranged as follows: Section 2 describes the methods for this research. Details of the prediction model are described in Section 3. Section 4 describes the experimental design. The experimental results and discussions thereof are reported in Section 5. Finally, study conclusions and problems we encountered in the course of research, and potential for future work in this area are outlined in Section 6.

In this section, we first introduce the technical indicators used in this research. Thereafter, SVM regression model and MKL regression model are presented.

The moving average (MA) is a trend-following index used to understand present trends. Moving averages are used to emphasize the direction of a trend and to smooth out price fluctuations. Depending on how past prices are weighted, there are different types of moving averages such as simple moving average (SMA) and exponential moving average (EMA), which have different ways of calculating moving average prices. The SMA is a simple mean value with identical weights used for past prices:
_{n}

The MACD provides two indicators: MACD and MACD signal. MACD shows the difference between a fast and a slow EMA of closing prices. “Fast” means a short-period average and “slow” means a long-period average. When MACD(t) (MACD at the time period

MACD signal is equal to the 9-period EMA of the MACD:

The default values (12, 26, and 9) of MACD parameters can be adjusted according to the needs of the traders. In this study, we have simply used the default values of MACD parameters since this value set is widely recognized and used worldwide.

SVR is a version of the SVM [

Generally, in a regression problem, suppose we are given a set of training examples:
_{i}^{n}, y_{i}_{i}_{i}

SVR is a kernel based regression method, which tries to locate a regression hyperplane with small risk in high-dimensional feature space. It possesses good function approximation and generalization capabilities. The ε-insensitive SVR is the most commonly used SVR. It finds a regression hyperplane with an ε-insensitive band. In the ε-insensitive SVR, its loss function is described by:

The image of the input data need not lie strictly on or inside the ε-insensitive band for making the method robust. Instead, images that lie outside the ε-insensitive band are penalized and slack variables are introduced to account for these penalties. In the following equations, SVR has been used to refer to ε-insensitive SVR. The objective function and constraints for SVR are as follows:
_{i}_{i}

To solve _{i}_{i}_{i},x_{j}_{i}_{j}

_{i}_{i}

The SVR method uses a single mapping function φ and, hence, a single kernel function

Furthermore, we use individual kernels for fusing the features from different sources or different representations. In addition to learning the coefficients
_{j}_{j}

A normal SVM is applied to a single feature type. In our experiments, we used one Gaussian kernel and one linear kernel for each feature set, and we used MKL to integrate the features of different crude oil markets and timeframes. With MKL, we trained an SVM with an adaptively weighted combined kernel, which fuses different kinds of features.

Sonnenburg

The proposed model is composed of three components as shown in

Feature extraction (FE) component;

Multiple kernel regression/prediction (MKRP) component;

Performance evaluation (PE) component.

The FE component first transforms crude oil spot price to MACD and MACD signals, following which it extracts features (historical n-days MACD features, see

The MKRP component then predicts the crude oil price by fusing information from the two crude oil markets and three different timeframes. In this research, we have tested the forecasting ability of the proposed model on the basis of one-day, two-day, and three-day ahead predictions, while previous studies usually focused only on one-day ahead prediction.

For MKR, the input features are extracted from two different sources: WTI and Brent Crude oil prices. Since WTI and Brent Crude oil are the two biggest crude oil markets in the world, we selected these two sources.We transformed the original spot prices to MACD and MACD signal. For each kernel, the inputs are 4-period MACD values and MACD signals calculated from different timeframes. Details are presented in

Finally, the PE component evaluates the prediction and trading results based on the two evaluation criteria. This aspect is discussed in detail in Section 4.4.

There are a number of crude oil price series. Of these various series, two main crude oil price series, WTI crude oil spot price and Brent Crude oil spot price, are chosen as experimental samples. There are two primary reasons why these two are chosen as crude oil price sources for our study. First, these two crude oil markets have maximum impact on the world economy; hence, these forecasts would be useful for many countries in the world. Second, since fluctuations in one market could be an important reference for the other, both these markets have been used for the experiment. This study uses daily spot prices obtained from the energy information administration (EIA) website of the US Department of Energy (DOE) [

The WTI crude oil spot price information we obtained from the website is from 2 January 1986 through 2 January 2012, and the Brent Crude oil spot price information is from 2 January 1987 to 2 January 2012. The difference in duration for which data was collected is because of reasons pertaining to data availability—the EIA website provides Brent data from 20 May 1987. Moreover, since this study uses information from both sources for prediction of price series, for reasons of convenience, we have considered data for the period ranging from 2 January 1990 to 31 December 2011, in both cases. Details of training and testing period are described in the Section 4.2.

In addition, in Section 5.4, we will show the experiment results by using information from crude oil markets (WTI and Brent crude oil markets), but also from two types of conventional gasoline market: “New York Harbor Regular” and “US Golf Coast Regular”. We selected these two additional oil markets because their history data is as early as that of Brent and WTI (from year 1986). Their data were downloaded from EIA website. The platform for experiment is Ubuntu, R language. MKL shogun package [

We used a rolling window method to separate the training and testing period. Since we want to have about 10 to 20 pairs of training and testing data and about one year for testing in the experiments, we decided to perform regression on data relating to 2048 trading days (around eight years) and obtained predicted values for 256 trading days (around one year). Further, for each subsequent experiment, we moved both the training and testing period forward by 256 trading days (around one year). There is a total of 14 training and testing periods for WTI and Brent crude oil price prediction from the beginning of 1990 to the end of 2011. The training and testing period and their relations are shown in

Additionally, we tested the prediction ability of the proposed model by conducting forecasts for three different days: one-day, two-day, and three-day ahead predictions. For example, for testing two-day ahead prediction, we predict the crude oil price two days later; for trading, we hold the trading position for two days and close the position after two days.

A list of proposed and benchmark methods is shown in

To judge the forecasting performances and evaluate accuracy of prediction, two evaluation measures are used: root mean square error (RMSE) and average percentage profit (

RMSE is a frequently-used measure to calculate differences between the values predicted by a model or a predictor and the values actually observed. It is defined by the formula:
_{i}_{i}

In addition to the magnitude measures, we also measure the usefulness of the prediction for making profits in trading. Suppose the predictor predicts the oil price will go up (_{i}_{i}_{i}_{i}

Note that m denotes the steps of

From average RMSE/mean results of the total 14 experiments conducted (“mean” is the average crude oil price in the corresponding testing periods), the results of experiments based on the MKR model (MKR-S-3 and MKR-M-3) showed the best prediction results (the lower the value, better the score in RMSE/mean). Additionally, we found that although the SVR-S-3 and SVR-M-3 use more features from more sources than SVR-S-1, forecasts of these methods is less accurate than SVR-S-1. In fact, SVR-S-3 has additional features of two more timeframes and SVR-M-3 has additional features of another crude oil market and two more timeframes. The reason for this could be that SVR has failed to fuse the information from different sources and/or different timeframes. On the contrary, the MKR based methods (MKR-S-3 and MKR-M-3) yield better results than SVR-based methods (SVR-S-1, SVR-S-3, and SVR-M-3), indicating that information from different sources and different representations are useful for predictions. Moreover, this also indicates that, the MKR-based prediction methods fused the entire information more effectively than SVR-based methods.

Furthermore, since average RMSE/mean of MKR-M-3 and MKR-S-3 are not very different, we can conclude that the additional data source other than the target is not very useful. Moreover, on observing the standard deviation results, we find that the results of MKR-S-3 and MKR-M-3 are smaller than that of SVR-S-1, SVR-S-3, and SVR-M-3 for different prediction days (one-day, two-day, and three-day ahead). This indicates that the MKR-based model not only outperforms the SVR based model in terms of the magnitude of prediction, but it also attains low volatility. Similar conclusions could be derived for WTI crude oil price prediction from results shown in

We now focus on the results for varying time horizons. From

Next, we focus on the results of different trading horizons for the same method. Note that since

Finally, we compare the results of our proposed method for WTI and Brent. For WTI, proposed method MKR-M-3 yields about 1.30%, 0.55%, and 0.34% per day for one-day, two-day, and three-day ahead predictions, respectively. For Brent, it yields about 1.05%, 0.46%, and 0.28% per day for one-day, two-day, and three-day ahead predictions, respectively. This indicates that for each prediction based on time horizon, the proposed method produced better results when applied to WTI spot price, rather than Brent spot price.

The coefficients for sub-kernels (β_{j}

For WTI one-day ahead prediction, we find that in most of the MKR training periods, the coefficients of Brent-2, Brent-3, WTI-2, and WTI-3 are larger than those of the others. Thus, it can be concluded that for predicting WTI crude oil price with one-day time horizon, features of Brent-2, Brent-3, WTI-2, and WTI-3 can be considered as more influential references than Brent-1 and WTI-1.

For two-day ahead prediction, the coefficient of WTI-2 drastically changes between training periods 4 and 5: while WTI-2 accounts for almost 35% in the 4th training period, it accounts for only 15% in the 5th training period, which is about a 20% point difference. The sudden coefficient change of WTI-2 indicates that the importance of WTI-2 features may have suddenly changed from the 4th to the 5th period for two-day ahead WTI prediction. We also observe that while coefficients of one-day timeframe features (Brent-2 and WTI-2) and three-day timeframe features (Brent-3 and WTI-3) decrease training period 4 onwards, the coefficients of one-day timeframe (Brent-1 and WTI-1), on the other hand, register an increase. This pattern indicates that the one-day MACD indicator feature gains relatively more importance for all subsequent training periods, after the 4th period.

For three-day ahead prediction, the coefficients of each feature set are not stable and coefficients of some feature sets, such as those of Brent-2 and WTI-2, change rapidly. This indicates that in case of predictions for longer time horizon, it is difficult to measure the relative importance of one feature over another as they demonstrate an unstable and dynamic character.

For two-day ahead prediction, coefficients of some feature sets, especially Brent-1 and Brent-2, are not stable. As observed in the results for WTI, the coefficient of WTI-1 shows a sudden change from the 4th training period to the 5th training period.

For three-day ahead prediction, coefficients of many feature sets are not stable, and coefficients of more feature sets (than those of one-day and two-day ahead predictions), such as Brent-1, Brent-2, WTI-1, and WTI-2, change rapidly. This indicates, as it did in the previous case, that it is difficult to measure the relative importance of one feature over another in case of longer timeframes.

Moreover, from

In addition to the coefficients shown for consecutive experiment periods,

Additionally, we observed that for each prediction, the sum of coefficients of Brent (Brent-1, Brent-2, and Brent-3) and sum of coefficients of WTI (WTI-1, WTI-2, and WTI-3) are close to 50%. This indicates that for each prediction, the importance of both the crude oil sources is almost equal.

From the average RMSE results shown in

From the average

In this study, we have proposed an MKL-based crude oil prediction method, which includes three components: feature extraction (FE), multiple kernel regression for prediction (MKRP), and performance evaluation (PE). In this study, the FE component first extracts features as MACD indicator from two crude oil sources and three different timeframes. Second, the MKRP component predicts the crude oil prices by employing MKR. Finally, the PE component evaluates the prediction results by using RMSE and

Experimental results show that prediction method based on the MKR framework yields better results than those obtained from SVR. Our study also detected that in case information is extracted from more than one source and/or different representations, SVR fails to effectively fuse the information, resulting in even more inaccurate results than those produced by employing the SVR method that used information from only a single source, pertaining to a single timeframe. On the contrary, methods based on the MKR framework effectively fused information from different sources and different representations, and produced better results than the benchmark methods, with the exception that the additional data source did not add to the effectiveness of the forecast. However, we first believed that the knowledge of another market price movements is beneficial for a trader (therefore we conducted experiments) but in fact, if the knowledge of one market price movement is highly utilized, the knowledge of another market price movement one day ago is not useful at least for the case we experimented. The reason might be that the two markets are correlated almost in real time.

The coefficients that we obtained from the MKL regression function for crude oil price prediction, using data from different crude oil markets and timeframes, demonstrated a possible correlation between our target crude oil market (WTI or Brent) and its target prediction time horizons (one-day, two-day, or three-day ahead), with other crude markets or other timeframes. The relative value of coefficients of the kernels in MKL results could be utilized to see possible correlations between reference crude oil markets with reference timeframes and the target crude oil market with the target prediction time horizon. As the time horizon goes on extending, coefficients of each feature set become unstable and the average percentage profit (

Future work in this field may take several interesting directions. For example, other than crude oil prices, stock prices of USA, main European stock markets, exchange rates of EUR/USD and USD/JPY are considered as useful information for predicting crude oil prices. Besides exploring some of these determinants of crude oil prices, possibility of incorporating features from more than three timeframes and including more stages in the step-ahead prediction model, could be investigated by future studies.

This research has been partially supported by Global-COE program (Symbiotic, Safe and Secure System Design program) of Keio University, Japan. The authors would like to thank their sponsors. Additionally, the authors would like to thank the EIA for making its crude oil spot prices data available, and authors of the MKL Shogun package, which greatly assisted in this research.

The authors declare no conflict of interest.

Structure of proposed model.

Rolling window method for training and forecasting (

Coefficients of MKR of one-day, two-day, and three-day ahead predictions for WTI crude oil price (14 experiments for each time horizon).

Coefficients of MKR of one-day, two-day, and three-day ahead predictions for Brent Crude oil price (14 experiments for each time horizon).

Features for each kernel in the MKL framework.

1 | MACD value at time |
5 | MACD value at time ( |

2 | MACD signal at time |
6 | MACD signal at time ( |

3 | MACD value at time ( |
7 | MACD value at time ( |

4 | MACD signal at time ( |
8 | MACD signal at time ( |

A list of proposed and benchmark methods

1 | SVR-S-1 | Only the target market | Only the target timeframe | Support vector regression (SVR) |

2 | SVR-S-3 | Only the target market | Three timeframes | Support vector regression (SVR) |

3 | SVR-M-3 | Both the markets | Three timeframes | Support vector regression (SVR) |

4 | MKR-S-3 | Only the target market | Three timeframes | Multiple kernel regression (MKR) |

5 | MKR-M-3 | Both the markets | Three timeframes | Multiple kernel regression (MKR) |

Average RMSE/mean results for Brent Crude oil price prediction (total 14 experiments).

SVR-S-1 | 0.02796 | 1.99121 | 2.29711 |

SVR-S-3 | 0.05688 | 3.34386 | 4.46337 |

SVR-M-3 | 0.06629 | 5.17562 | 6.16460 |

MKR-S-3 | 0.02316 | 1.60254 | 1.97642 |

MKR-M-3 | 0.02316 | 1.60211 | 1.97584 |

Standard deviation of RMSE/mean for Brent Crude oil price prediction (total 14 experiments).

SVR-S-1 | 0.01011 | 1.61699 | 1.76454 | |

SVR-S-3 | 0.06187 | 2.68442 | 3.99079 | |

SVR-M-3 | 0.02494 | 3.71164 | 4.43922 | |

MKR-S-3 | 0.00447 | 0.95943 | 1.18787 | |

MKR-M-3 | 0.00447 | 0.95845 | 1.18666 |

Average RMSE/mean results for WTI Crude oil price prediction (total 14 experiments).

SVR-S-1 | 0.02856 | 2.02755 | 2.33543 |

SVR-S-3 | 0.04904 | 3.49764 | 4.14796 |

SVR-M-3 | 0.07788 | 5.83853 | 6.71285 |

MKR-S-3 | 0.02450 | 1.70738 | 2.06678 |

MKR-M-3 | 0.02450 | 1.70724 | 2.06662 |

Standard deviation of RMSE/mean for WTI Crude oil price prediction (total 14 experiments).

SVR-S-1 | 0.01050 | 1.47208 | 1.57962 |

SVR-S-3 | 0.03186 | 3.20480 | 3.45841 |

SVR-M-3 | 0.03384 | 4.58162 | 5.72177 |

MKR-S-3 | 0.00510 | 1.05566 | 1.24878 |

MKR-M-3 | 0.00510 | 1.05577 | 1.24895 |

Average

SVR-S-1 | 0.00242 | 0.00207 | 0.00138 |

SVR-S-3 | −0.00132 | −0.00057 | −0.00036 |

SVR-M-3 | −0.00154 | −0.00071 | −0.00040 |

MKR-S-3 | 0.01038 | 0.00469 | 0.00282 |

MKR-M-3 | 0.01054 | 0.00464 | 0.00280 |

Average

SVR-S-1 | 0.00367 | 0.00176 | 0.00117 |

SVR-S-3 | 0.00057 | −0.00005 | 0.00009 |

SVR-M-3 | −0.00143 | −0.00088 | −0.00044 |

MKR-S-3 | 0.01390 | 0.00546 | 0.00344 |

MKR-M-3 | 0.01309 | 0.00556 | 0.00348 |

Average Coefficients of MKR for one-day, two-day, and three-day ahead predictions (14 experiments for each time horizon).

0.10458 | 0.18613 | 0.21962 | 0.10458 | 0.20034 | 0.18471 | |

0.16146 | 0.18567 | 0.17001 | 0.16146 | 0.18291 | 0.13846 | |

0.21943 | 0.19276 | 0.09234 | 0.21943 | 0.18439 | 0.09163 | |

0.08025 | 0.20840 | 0.21438 | 0.08025 | 0.21686 | 0.19983 | |

0.12662 | 0.22418 | 0.16391 | 0.12662 | 0.20971 | 0.14893 | |

0.12008 | 0.27768 | 0.11531 | 0.12008 | 0.25153 | 0.11529 |

Standard deviation of Coefficients of MKR for one-day, two-day, and three-day ahead predictions (14 experiments for each time horizon).

0.031287 | 0.019009 | 0.031188 | 0.031287 | 0.018160 | 0.012432 | |

0.066042 | 0.087793 | 0.048502 | 0.066042 | 0.029628 | 0.048715 | |

0.094199 | 0.131373 | 0.047001 | 0.094199 | 0.055894 | 0.053051 | |

0.043781 | 0.034358 | 0.035105 | 0.043781 | 0.050984 | 0.038125 | |

0.043023 | 0.057368 | 0.012079 | 0.043023 | 0.073336 | 0.034651 | |

0.043395 | 0.108578 | 0.035963 | 0.043395 | 0.100807 | 0.035932 |

Average RMSE results for WTI and Brent crude oil price prediction (14 experiments in total).

MKR-M-3 (proposed method) | 1.22309 | 1.70724 | 2.06662 | 1.11768 | 1.60210 | 1.97584 |

MKR-F-3 | 1.2223 | 1.70230 | 2.05287 | 1.11587 | 1.59464 | 1.95926 |

Average

MKR-M-3 (proposed method) | 0.01309 | 0.00556 | 0.00348 | 0.01054 | 0.00464 | 0.00280 |

MKR-F-3 | 0.00921 | 0.00578 | 0.00316 | 0.00778 | 0.00662 | 0.00211 |