The Use of Singular Spectrum Analysis and K-Means Clustering-Based Bootstrap to Improve Multistep Ahead Load Forecasting

Abstract

In general, studies on short-term hourly electricity load modeling and forecasting do not investigate in detail the sources of uncertainty in forecasting. This study aims to evaluate the impact and benefits of applying bootstrap aggregation in overcoming the uncertainty in time series forecasting, thereby increasing the accuracy of multistep ahead point forecasts. We implemented the existing and proposed clustering-based bootstrapping methods to generate new electricity load time series. In the proposed method, we use singular spectrum analysis to decompose the series between signal and noise to reduce the variance of the bootstrapped series. The noise is then bootstrapped by K-means clustering-based generation of Gaussian normal distribution (KM.N) before adding it back to the signal, resulting in the bootstrapped series. We apply the benchmark models for electricity load forecasting, SARIMA, NNAR, TBATS, and DSHW, to model all new bootstrapped series and determine the multistep ahead point forecasts. The forecast values obtained from the original series are compared with the mean and median across all forecasts calculated from the bootstrapped series using the Malaysian, Polish, and Indonesian hourly load series for 12, 24, and 36 steps ahead. We conclude that, in this case, the proposed bootstrapping method improves the accuracy of multistep-ahead forecast values, especially when considering the SARIMA and NNAR models.

1. Introduction

Electricity load forecasting plays a critical role in controlling the balance between power demand and supply. Sometimes, the energy demand exceeds the energy supply and vice versa, which results in financial losses. An important aspect of a smart grid system is determining an accurate load forecasting model. Electricity load forecasting provides information that will simplify the work of planning consumption, generation, distribution, and other essential tasks of the smart grid system [1,2].

Much work has been performed to develop models and strategies to improve the electricity load forecasting accuracy. Generally, an hourly load series shows three relationships, i.e., between the observations for consecutive hours on a particular day, between the observations for the same hour on consecutive days, and between the observations for the same hour on the same day in successive weeks. In certain countries, the hourly load series may become more complex due to calendar variations [3]. The effect of calendar variation is usually considered by including a dummy variable in the model [4,5,6,7]. In countries with four seasons, the temperature is often included in the load forecasting model [8,9]. For countries with two seasons, such as Malaysia, it is also possible to include temperature information to improve the forecasts’ accuracy [10]. Many models, from simple to complex, have been proposed and developed by researchers and practitioners around the world to improve the accuracy of electricity load forecasting, e.g., regression and seasonal autoregressive integrated moving average (SARIMA) models [4,11,12], exponential smoothing [3,6,13,14,15], neural network (NN) [16,17,18,19], singular spectrum analysis (SSA) [20,21,22], wavelets [23,24], fuzzy systems [10,25,26], support vector machine [1,21,27,28], among others. However, the most suitable model for electricity load forecasting in a given country may not be the best to model the data in another country because of different consumption and behavioral characteristics.

In this study, we discuss the implementation of the bootstrap aggregating method to improve the accuracy of multistep ahead load forecast. Bootstrap aggregation, which is known by the acronym “bagging”, was proposed by [29] to reduce the variance of the predictor. It works by generating replicated bootstrap samples of the training data and using them to obtain the aggregated predictor. Bagging aims at improving the point forecast by considering sources of uncertainty, namely, the parameter estimates, the appropriate model determination, and the noise. In 2016, [30] successfully applied the development of this method in the field of time series forecasting by using the moving block bootstrap (MBB). Further, [31] explored how bagging improves point forecasts and showed that model selection as a solution to model uncertainty was the most influential on the success of bagging in time series. As described in [30], MBB bagging methods first apply the Box–Cox transform to the original series and then decompose it into a trend, seasonal, and noise using STL (Seasonal and Trend decomposition using Loess). STL is a decomposition method developed by [32]. In MBB, the noise is bootstrapped and added back to the trend and seasonal components. The new transformed bootstrapped series are then inverted and modeled. However, MBB is more appropriate for bootstrapping stationary time series. When the original data are not stationary, the bootstrapped series may be very noisy and do not fluctuate as the original series [30].

Recently, [33] proposed three clustering-based bootstrap aggregating methods, i.e., Smoothed MBB (S.MBB), K-means clustering based (KM), and K-means clustering based-generated from Gaussian normal distribution (KM.N), which perform better under noisy and fluctuating data. In adapting the fluctuating data, the S.MBB method smooths the noise using simple exponential smoothing before applying MBB. Meanwhile, KM and KM.N methods adapt to the noisy series by first implementing the K-means cluster. The original series clusters into K groups and then creates new time series based on the clusters. The difference between KM and KM.N is in how they generate the bootstrap series. In KM, a new time series is created directly by sampling values of clusters, while in KM.N, it is created by generating values based on the parameters of the Gaussian normal distribution of clusters. Both the KM and KM.N methods succeeded in making the bootstrapped time series have low variance between each other. Based on the experimental study of the electricity load series with multiple seasonal and calendar effects, KM.N performed better than the KM method [33]. However, this method creates a bootstrapped series based on the original data without sorting out signal and noise. Thus, in more complex series where the calendar effect may not be visible clearly in the original data, it will produce a noisier bootstrapped series at specific points, especially at times affected by calendar variation.

Inspired by [30,33], this study proposes an SSA–clustering-based method named SSA.KM.N as a modification of KM.N. Our proposed method combines singular spectrum analysis (SSA) as an alternative to the STL method in MBB and KM.N to generate new series from the remainder of the SSA decomposition. Literature shows that SSA is powerful in decomposing time series with complex seasonal patterns ([5,20]). SSA plays a role in breaking down time series, which have trends, multiple seasonal components, and are affected by calendar variation, into signal and noise, which generally contain extreme values representing calendar effects in more detail. By taking advantage of the unique strengths of SSA and KMN, our methodology can better adapt to fluctuating time series related to the effects of calendar variation. Bootstrapping the noise using KM.N and adding it to the signal is expected to produce a bootstrapped time series with low variance and values around the original series.

In this work, the proposed method compares with KM.N in its application to bootstrap two Malaysian load time series with different sample sizes and different time periods; one from Poland and another from Indonesia. We evaluate the impact and benefits of applying SSA.KM.N and KM.N in overcoming the source of uncertainty in time series forecasting and their success in increasing the accuracy of multistep ahead point forecast obtained by standard models such as SARIMA, NNAR, TBATS, and DSHW models.

The rest of the paper is organized as follows. Section 2 describes the methods used in this paper, starting from forecasting methods, decomposition, bagging, and ensemble methods. We also present the procedure of our proposed approach in this section. Section 3 reports the application of KM.N and SSA.KM.N to the four electricity load time series and shows the error evaluation for 12, 24, and 36 steps-ahead point forecasts obtained from SARIMA, NNAR, DSHW, and TBATS for further investigation and assessment. Conclusions are found in Section 4.

2. Materials and Methods

This section contains a brief overview of the methods used for time series modeling and forecasting, the decomposition method, the ensemble learning, and the proposed approach.

2.1. Forecasting Methods

SARIMA, and exponential smoothing (i.e., TBATS and DSHW), are popular approaches to forecast trend and seasonal time series. On the other hand, NNAR is a powerful method for capturing nonlinear relationships in time series data. These four methods are frequently used in modeling load series, and their forecast accuracy is used as benchmarks for other proposed methods [4,10,22]. For example, the Spanish Transmission System Operator uses autoregressive (AR) and NN models [7].

The seasonal ARIMA model, notated as SARIMA $\left(p,d,q\right){\left(P,D,Q\right)}_S$, is an extension of ARIMA model that accommodates the seasonal component of the time series [34,35], and can be written as follows:

$${\varphi }_p\left(\mathrm{B}\right){\psi }_P\left({\mathrm{B}}^S\right){\nabla }^d{\nabla }_S^D{z}_t={\theta }_q\left(\mathrm{B}\right){\vartheta }_Q\left({\mathrm{B}}^S\right)a_t$$

(1)

where $z_t$ is observation at time $t$, $S$ is seasonal period, $p$, $P$, $q$, and $Q$ are the orders of autoregressive, seasonal autoregressive, moving average, and seasonal moving average, respectively. Superscript $d$ and $D$ notate the regular and seasonal differentiation, while ${\nabla }_S=1-{\mathrm{B}}^S$ is a backshift operator, and ${\varphi }_p\left(\mathrm{B}\right)$, and ${\theta }_q\left(\mathrm{B}\right)$ are polynomials in B of degree $p$ and $q$, respectively. Notations ${\psi }_P\left({\mathrm{B}}^S\right)$, and ${\vartheta }_Q\left({\mathrm{B}}^S\right)$ are polynomials in ${\mathrm{B}}^S$ of degrees $P$ and $Q$, and $a_t$ is white noise. The orders of $p$, $q$, $P$, and $Q$ can be determined from the correlogram and partial correlogram. Oftentimes, the identification of these orders is not an easy task and the user experience is required [36]. The automatic algorithms discussed in [37] with the “auto.arima” function of the R software can be used to help handle this problem [31]. However, other researchers may prefer to estimate the parameters manually instead of using automated packages [38]. In this case, we use “Arima” function included in the package “forecast” in the software R [39].

NNAR is a feedforward neural network that consists of lagged input neurons, one hidden layer with nonlinear function, and one output neuron [40,41]. NNAR ${\left(p,P,k\right)}_S$ model can be represented as in Equation (2),

$$z_t=b_1+{\displaystyle \sum }_{j=1}^k{v}_j{f}_j\left(b_0+{\displaystyle \sum }_{i=1}^p{z}_{t-i}{w}_{ij}+{\displaystyle \sum }_{m=1}^P{z}_{t-mS}{w}_{mj}\right)$$

(2)

where $b_0$ and $b_1$ are biased, $k$ is the number of neurons in the hidden layer, $p$ is the order of the non-seasonal component, while P is the order of the seasonal component. The sigmoid function at the jth neuron in the hidden layer, $f_j$, is defined in Equation (3),

$$f_j\left(u\right)=\frac{1}{\left[1+\exp \left(-u\right)\right]},$$

(3)

where $u=b_0+{{\displaystyle \sum }}_{i=1}^p{z}_{t-i}{w}_{ij}+{{\displaystyle \sum }}_{m=1}^P{z}_{t-mS}{w}_{mj}$. In its implementation we use the “nnetar” function in the R package “forecast” ([39]). Later in the experimental study, $P$ is set to be 1, p is the optimal number of lags for the linear model fitted to the seasonally adjusted data, and k is determined by the rounded value of $\left(P+p\right)/2$. The final forecast values are obtained by averaging 20 networks with different random starting weights.

TBATS and DSHW are modifications of the exponential smoothing to handle trends and multiple seasonal patterns in time series forecasting [42]. DSHW, proposed by [13], accommodates two seasonal patterns where one cycle may be nested within another. Meanwhile, TBATS, proposed by [3], can handle a more complex seasonal pattern in time series forecasting. The term “complex” means that the time series has a trend, and multiple seasonal patterns with integer or non-integer periods, and this may include dual-calendar seasonal effects. Success studies of the use of DSHW and TBATS in modeling electricity load time series can be found in [3,13,43]. Detail of these models can be found in [3,13,42,44]. In this paper, TBATS and DSHW are fitted by using the “tbats” and “dshw” functions included in the R package “forecast” ([39]).

2.2. SSA Decomposition Method

SSA is a technique in the field of time series analysis that has a vast range of applicability in decomposition [20], missing value imputation [45], and forecasting [46]. In this study, we focus on the use of the SSA algorithm to decompose time series into the following two components: signal and unstructured noise. SSA consists of four steps, namely, embedding, singular value decomposition (SVD), grouping, and diagonal averaging [47,48].

In embedding, we transform a time series $z_t = \{z_1,z_2,\dots ,z_{n_{\mathit{tr}}}\}$ into a trajectory matrix as in (4) as follows:

$$Z=\left[\begin{array}{ccccc}{z}_1& z_2& z_3& \dots & z_{c_2}\\ z_2& z_3& z_4& \dots & z_{c_2+1}\\ ⋮& ⋮& ⋮& \cdots & ⋮\\ z_{c_1}& z_{c_1+1}& z_{c_1+2}& \dots & z_{n_{tr}}\end{array}\right]$$

(4)

where $c_1$ is the window length, $n_{tr}$ the number of training data sets, $c_2=n_{tr}-c_1+1$ and $c_1\le c_2$.

The matrix Z is then decomposed by SVD and expressed as follows:

$$Z=Z^{\left(1\right)}+Z^{\left(2\right)}={\displaystyle \sum }_{l=1}^{r_1}\sqrt{{\lambda }_l}{u}_l{v}_l^{’}+{\displaystyle \sum }_{l=r_1+1}^{r_2}\sqrt{{\lambda }_l}{u}_l{v}_l^{’}$$

(5)

where ${\lambda }_l$ are the eigenvalues of matrix $Z$, $u_l$ and $v_l$ are left and right singular vectors of the matrix $Z$ corresponding to the eigenvalues ${\lambda }_l$, respectively. We need to determine $r_1$, the number of signal components used for reconstruction in the grouping stage. Finally, we can obtain signal and noise by the diagonal averaging procedure, i.e., the anti-diagonals that map the matrices of the signal and noise components back to time series. The original time series can then be expressed as follows:

$$z_t={\tilde{z}}_t^{\left(1\right)}+{\tilde{z}}_t^{\left(2\right)}$$

(6)

where ${\tilde{z}}_t^{\left(1\right)}$ is the signal and ${\tilde{z}}_t^{\left(2\right)}$ is the noise.

2.3. Bagging and Ensemble Methods

The K-means clustering-based bagging method, proposed by [33], clusters a univariate time series into K groups by the K-means method. Each cluster has its average value and standard deviation as parameters of the cluster. A Gaussian normal distribution according to these parameters is then used to generate a random number as a new value of the bootstrap time series. For example, suppose $z_t$ is a value of the original time series at time $t$ that belongs to the ith cluster. In that case, we can obtain the new value of the bootstrap time series at time $t$ by generating a random value based on the Gaussian normal distribution of the particular cluster. The code for bootstrapping time series by K-means clustering-based-generated from Gaussian normal distribution (KM.N) can be found in [49].

2.4. Proposed Approach

The proposed SSA.KM.N bagging method is a modification of KM.N where the first stage decomposes the original time series using SSA (Figure 1; blue cells). As shown in [20,50], SSA can be used to decompose complex time series into several simple pattern components.

Step 1.

Divide the series into the following two parts: training and testing datasets;

Step 2.

Generate $\mathcal{B}$ new series from the original training data by the KM.N method;

Step 3.

Generate $\mathcal{B}$ new series from the original training data by the SSA.KM.N method;

a.

Apply SSA to the original training data to define the signal and the irregular component;

b.

Generate $\mathcal{B}$ new series from the irregular component obtained in Step 3a by the KM.N method;

c.

Sum each new irregular component obtained from Step 3b with the signal component so that we obtain $\mathcal{B}$ bootstrapped series of the original training data;

Step 4.

Model the original training data, and each bootstrapped series by SARIMA, NNAR, TBATS, and DSHW;

Step 5.

Calculate up to M-steps-ahead forecast values by each model obtained in Step 4;

a.

Define the M-steps-ahead forecast values from the SARIMA, NNAR, TBATS, and DSHW models obtained from the original training data series;

b.

Apply mean and median to calculate the final forecast of the bootstrap series determined in Step 2 and Step 3 for the first $n_{\mathcal{B}}$ bootstrap series;

Step 6.

Evaluate the forecast accuracy based on root mean square error (RMSE) and mean absolute percentage error (MAPE).

The two accuracy measures considered in this study can be defined as follows. MAPE, calculated by Equation (7), is frequently used in evaluating load forecasting accuracy since it is a scale-independent error that may compare forecast performance between different data sets [44,51]. Meanwhile, RMSE, calculated using Equation (8), is a scale-dependent error that can be used to compare the accuracy performance of several models on the same data set [51].

$$\mathrm{MAPE}=\frac{100\%}{H}{\displaystyle \sum }_{h=1}^H|e_{n_{tr}+h}/y_{n_{tr}+h}|$$

(7)

$$\mathrm{RMSE}={\left(\frac{{{\displaystyle \sum }}_{h=1}^H{e}_{n_{tr}+h}^2}{H}\right)}^{1/2}$$

(8)

In addition, we also evaluate the model using mean bias error (MBE) as defined in Equation (9). It provides information whether there is a positive or negative bias [52]. We can calculate MBE by the following:

$$\mathrm{MBE}=\frac{{{\displaystyle \sum }}_{h=1}^H{e}_{n_{tr}+h}}{H}$$

(9)

where

$$e_{n_{tr}+h}=y_{n_{tr}+h}-{\widehat{y}}_{n_{tr}+h}$$

${\widehat{y}}_{n_{tr}+h}$ and $y_{n_{tr}+h}$ are the predicted value and the actual value at time ($n_{tr}+h)$, respectively. H is the number of observations included in the calculation and $n_{tr}$ is the size of training data.

In this study, each bootstrapped series is modeled separately, and the final forecast for time t is obtained by the following two ensemble methods: the mean and the median across all forecast values at time t, calculated from the bootstrap series. In this study, we obtain the mean and the median of the predicted values generated from the first $n_{\mathcal{B}}$ (between 10 to 100) bootstrapped series to investigate whether the number of generated time series affects the accuracy of the forecast results.

3. Results and Discussion

In this section, we discuss two hourly electricity loads in Johor, Malaysia, and two other electricity load datasets from Poland and Indonesia. We decided to use these four data sets to show the generality of our work for electricity load forecasting.

3.1. Application to the Hourly Electricity Load of Johor, Malaysia

This subsection focuses on short-term forecasting of hourly electricity load with application to Malaysian data. We consider two datasets with different sizes that can be accessed in [53]. The first is the hourly load series from 1 January to 31 December 2009, and the second is the hourly load series from 1 January to 31 July 2010, which are depicted in Figure 2 (see Figure 2a,b, respectively).

The period from 1 January, time 00:00, to 30 November 2009, time 23:00, and the period from 1 January, time 00:00, to 30 June 2010, time 23:00, were used for estimation purposes as the training data. The remainder was used to evaluate the forecast performance of the models. These periods are summarized in

Table 1. Training and testing datasets of hourly load series used in the experimental study.

	1st Data Set	2nd Data Set
Training	01/01/2009 00:00–30/11/2009 23:00	01/01/2010 00:00–30/06/2010 23:00
Testing (for up to h-step ahead):
$h$ = 1	01/12/2009 00:00–31/12/2009 23:00	01/07/2010 00:00–31/07/2010 23:00
$h$ = 12	01/12/2009 00:00–01/12/2009 11:00	01/07/2010 00:00–01/07/2020 11:00
$h$ = 24	01/12/2009 00:00–01/12/2009 23:00	01/07/2010 00:00–01/07/2020 23:00
$h$ = 36	01/12/2009 00:00–01/12/2009 11:00	01/07/2010 00:00–02/07/2020 11:00

.

Our analysis generated 100 bootstrap time series using the KM.N and the SSA.KM.N methods. Note that the original series is included in those 100-bootstrap series. Figure 3 shows the original time series and a realization of the bootstrap series by each of the two methods.

From Figure 3, we can see that both the KM.N and SSA.KM.N methods produce bootstrap series (in red) with almost the same pattern as the original series (in black). Even in certain parts, where the data have lower or higher values than other times as a result of the calendar variation, the SSA.KM.N can generate series closer to the original time series than the KM.N, visually. As illustrations, we zoom the load on the time period influenced by the Prophet’s birthday (Figure 4a) and Eid al-Fitr (Figure 4b). Figure 4 shows that the variance of bootstrapped series obtained by KM.N (left) is larger than those obtained by SSA.KM.N (right).

The performance of the one-step ahead forecast accuracy of SARIMA, NNAR, TBATS, and DSHW is shown in

Table 2. RMSE and MAPE for one-step-ahead forecasts for the testing data of the two data sets of hourly electricity load in Malaysia, obtained by SARIMA, NNAR, TBATS, and DSHW.

Method	Testing of 1st Data Set (1–31 December 2009)		Testing of 2nd Data Set (1–31 July 2010)
	RMSE	MAPE	RMSE	MAPE
SARIMA	1168.02	1.90	1719.68	2.54
NNAR	749.92	1.21	1182.68	1.75
TBATS	1584.47	2.91	1466.33	2.37
DSHW	839.18	1.29	1035.14	1.46

. All these calculations were performed in the R software. From the analysis of the correlogram and the partial correlogram, the model SARIMA(2,0,3)(2,1,2)₂₄ was chosen for the first data set and the SARIMA(2,0,0)(3,1,0)₂₄ for the second data set. The most appropriate NNAR, TBATS, and DSHW models were reconstructed and chosen automatically by the “nnetar”, “tbats”, and “dshw” functions in R. Based on Table 2, we can see that both for the first and second data sets, the NNAR and the DSHW produce smaller RMSE and MAPE than SARIMA and TBATS in the case of one-step ahead forecasting.

Furthermore, we investigate how these four models work for multistep ahead load forecasting with and without bagging implementation. The comparative values of RMSE and MAPE for 12, 24, and 36 steps ahead for SARIMA, NNAR, TBATS, and DSHW are presented in

Table 3. RMSE and MAPE of $h$-step ahead forecast obtained by SARIMA model from the original series, KM.N and SSA.KM.N bootstrap series.

	1st Data Set						2nd Data Set
	RMSE			MAPE			RMSE			MAPE
$\mathit{h}$	12	24	36	12	24	36	12	24	36	12	24	36
original	466.51	1408.20	2118.07	0.88	1.94	3.13	1139.08	1268.58	1324.19	2.27	2.48	2.97
Bagging + Mean
KM.N
$n_{\mathcal{B}}$ = 25	512.57	1404.68	2035.79	0.97	1.94	3.00	1345.29	1405.54	1366.41	2.25	2.57	2.87
$n_{\mathcal{B}}$ = 50	530.99	1419.45	2049.64	0.99	1.96	3.03	1320.97	1393.69	1366.42	2.28	2.59	2.91
$n_{\mathcal{B}}$ = 75	544.23	1429.02	2053.97	1.02	1.99	3.05	1323.97	1394.82	1367.79	2.29	2.60	2.92
$n_{\mathcal{B}}$ = 100	556.73	1438.17	2062.04	1.09	2.03	3.08	1329.88	1397.27	1371.85	2.32	2.61	2.93
SSA. KM.N
$n_{\mathcal{B}}$ = 25	490.52	1368.94	2029.14	1.04	1.94	2.96	1067.12	1234.85	1286.55	2.11	2.40	2.87
$n_{\mathcal{B}}$ = 50	494.71	1357.47	2018.93	1.05	1.94	2.95	1069.56	1235.92	1288.94	2.13	2.41	2.88
$n_{\mathcal{B}}$ = 75	498.23	1356.26	2015.31	1.07	1.94	2.95	1068.79	1235.36	1288.80	2.13	2.41	2.88
$n_{\mathcal{B}}$ = 100	495.76	1360.24	2018.48	1.06	1.95	2.95	1072.37	1236.66	1289.22	2.14	2.41	2.88
Bagging + Median
KM.N
$n_{\mathcal{B}}$ = 25	548.32	1415.38	2045.61	1.03	1.98	3.04	1330.61	1388.67	1348.25	2.12	2.49	2.80
$n_{\mathcal{B}}$ = 50	543.85	1424.97	2055.74	1.02	1.98	3.05	1306.42	1383.85	1355.37	2.18	2.53	2.86
$n_{\mathcal{B}}$ = 75	549.99	1430.50	2056.88	1.05	2.00	3.06	1322.84	1390.97	1360.40	2.22	2.55	2.88
$n_{\mathcal{B}}$ = 100	566.06	1444.64	2066.11	1.09	2.04	3.09	1325.92	1392.35	1365.12	2.27	2.58	2.90
SSA.KM.N
$n_{\mathcal{B}}$ = 25	501.40	1372.29	2029.04	1.10	1.98	2.98	1072.91	1239.46	1290.89	2.13	2.41	2.88
$n_{\mathcal{B}}$ = 50	514.83	1355.26	2012.78	1.13	1.97	2.95	1076.47	1239.43	1292.00	2.15	2.42	2.89
$n_{\mathcal{B}}$ = 75	514.08	1355.95	2011.37	1.13	1.97	2.95	1069.09	1234.93	1289.34	2.13	2.41	2.88
$n_{\mathcal{B}}$ = 100	508.08	1365.38	2018.37	1.11	1.97	2.96	1074.03	1238.00	1290.43	2.15	2.42	2.88

Green cells represent the RMSE and MAPE values of the SARIMA model obtained from the bootstrap series lower than those obtained from the original series. Bold values represent the lowest value in a column of each bagging method with green cells.

,

Table 4. RMSE and MAPE of $h$-step ahead forecast obtained by NNAR model from the original series, KM.N and SSA.KM.N bootstrap series.

	1st Data Set						2nd Data Set
	RMSE			MAPE			RMSE			MAPE
$\mathit{h}$	12	24	36	12	24	36	12	24	36	12	24	36
Original	1761.72	2365.26	4393.21	2.12	3.18	5.81	2474.07	2797.86	5179.86	3.62	4.24	6.15
Bagging + Mean
KM.N
$n_{\mathcal{B}}$ = 25	960.09	1723.56	2881.12	1.10	2.35	4.01	2367.77	2062.45	4128.65	3.50	3.32	5.19
$n_{\mathcal{B}}$ = 50	964.93	1730.89	2894.54	1.12	2.37	4.03	2433.15	2096.23	4154.82	3.58	3.34	5.22
$n_{\mathcal{B}}$ = 75	996.61	1747.33	2949.97	1.16	2.39	4.08	2358.62	2044.60	4084.19	3.48	3.28	5.13
$n_{\mathcal{B}}$ = 100	1007.27	1766.48	2972.19	1.16	2.42	4.12	2388.40	2067.72	4120.69	3.51	3.30	5.17
SSA. KM.N
$n_{\mathcal{B}}$ = 25	973.69	1661.79	2756.24	1.11	2.29	3.93	2279.29	1958.57	5344.05	3.41	3.15	6.04
$n_{\mathcal{B}}$ = 50	954.97	1618.73	2701.08	1.11	2.23	3.84	2349.54	1997.22	5451.24	3.52	3.18	6.14
$n_{\mathcal{B}}$ = 75	971.00	1634.74	2732.76	1.11	2.24	3.86	2325.34	1980.90	5443.80	3.47	3.16	6.13
$n_{\mathcal{B}}$ = 100	968.26	1628.32	2716.57	1.11	2.24	3.85	2311.58	1974.44	5452.81	3.45	3.16	6.13
Bagging + Median
KM.N
$n_{\mathcal{B}}$ = 25	913.18	1695.23	2836.71	1.07	2.34	3.96	2364.00	2069.78	4029.04	3.52	3.35	5.14
$n_{\mathcal{B}}$ = 50	912.72	1690.94	2861.26	1.05	2.32	3.98	2469.39	2110.04	4136.53	3.63	3.36	5.21
$n_{\mathcal{B}}$ = 75	948.08	1697.71	2883.83	1.13	2.35	4.01	2399.13	2068.57	4035.98	3.53	3.31	5.11
$n_{\mathcal{B}}$ = 100	964.86	1713.66	2892.47	1.12	2.36	4.02	2418.19	2080.95	4088.09	3.55	3.32	5.15
SSA.KM.N
$n_{\mathcal{B}}$ = 25	978.42	1654.41	2785.53	1.13	2.28	3.95	2248.30	1952.46	5361.38	3.37	3.14	6.05
$n_{\mathcal{B}}$ = 50	979.69	1633.47	2657.16	1.13	2.26	3.81	2270.89	1954.88	5411.12	3.39	3.14	6.09
$n_{\mathcal{B}}$ = 75	975.65	1642.36	2683.66	1.12	2.26	3.83	2315.22	1980.65	5475.96	3.44	3.16	6.15
$n_{\mathcal{B}}$ = 100	959.00	1612.46	2631.46	1.13	2.23	3.78	2307.14	1977.87	5438.96	3.43	3.15	6.12

Green cells represent the RMSE and MAPE values of the SARIMA model obtained from the bootstrap series lower than those obtained from the original series. Bold values represent the lowest value in a column of each bagging method with green cells.

,

Table 5. RMSE and MAPE of $h$-step ahead forecast obtained by TBATS model from the original series, KM.N and SSA.KM.N bootstrap series.

	1st Data Set						2nd Data Set
	RMSE			MAPE			RMSE			MAPE
$\mathit{h}$	12	24	36	12	24	36	12	24	36	12	24	36
original	1613.36	2057.21	2146.92	4.03	3.72	4.08	1574.78	2771.37	2685.71	3.87	5.54	5.54
Bagging + Mean
KM.N
$n_{\mathcal{B}}$ = 25	1740.64	2333.25	2360.33	3.99	3.98	3.91	1410.82	3122.42	2864.93	3.50	5.93	5.86
$n_{\mathcal{B}}$ = 50	1668.46	2245.73	2282.53	3.89	3.86	3.84	1296.39	3045.17	2805.20	3.18	5.67	5.63
$n_{\mathcal{B}}$ = 75	1664.15	2246.18	2290.12	3.86	3.85	3.84	1280.05	3047.59	2806.13	3.14	5.67	5.62
$n_{\mathcal{B}}$ = 100	1732.41	2325.22	2370.24	3.91	3.93	3.90	1255.27	3002.58	2775.60	3.10	5.59	5.55
SSA. KM.N
$n_{\mathcal{B}}$ = 25	1966.14	2547.93	2560.87	4.61	4.44	4.26	1371.73	1974.32	2286.35	3.43	4.03	4.61
$n_{\mathcal{B}}$ = 50	2023.97	2609.35	2634.36	4.60	4.48	4.28	1447.05	1939.14	2316.25	3.51	3.99	4.68
$n_{\mathcal{B}}$ = 75	2011.41	2580.08	2615.16	4.60	4.45	4.28	1503.40	1861.19	2247.76	3.56	3.87	4.61
$n_{\mathcal{B}}$ = 100	1994.50	2572.82	2597.21	4.59	4.45	4.27	1468.42	1879.36	2259.16	3.52	3.90	4.60
Bagging + Median
KM.N
$n_{\mathcal{B}}$ = 25	1937.56	2611.75	2574.41	4.19	4.27	4.16	1336.97	3256.25	2953.00	3.30	6.03	5.91
$n_{\mathcal{B}}$ = 50	1719.88	2336.30	2337.81	3.86	3.90	3.87	1297.49	3240.04	2933.74	3.19	5.96	5.83
$n_{\mathcal{B}}$ = 75	1818.78	2406.98	2410.71	4.01	4.03	4.00	1297.06	3239.03	2934.09	3.19	5.96	5.83
$n_{\mathcal{B}}$ = 100	2010.25	2642.92	2639.85	4.19	4.31	4.21	1287.17	3203.36	2909.66	3.16	5.90	5.78
SSA.KM.N
$n_{\mathcal{B}}$ = 25	1980.32	2604.85	2589.40	4.56	4.46	4.16	1594.16	2542.15	2846.89	3.99	5.13	5.72
$n_{\mathcal{B}}$ = 50	2021.70	2674.56	2675.42	4.51	4.50	4.19	1482.30	2157.74	2548.26	3.71	4.46	5.16
$n_{\mathcal{B}}$ = 75	2043.60	2661.86	2678.06	4.59	4.53	4.27	1476.01	2070.70	2462.00	3.65	4.28	5.01
$n_{\mathcal{B}}$ = 100	2040.19	2629.41	2651.92	4.59	4.50	4.25	1415.00	2012.09	2390.52	3.54	4.14	4.88

Green cells represent the RMSE and MAPE values of the SARIMA model obtained from the bootstrap series lower than those obtained from the original series. Bold values represent the lowest value in a column of each bagging method with green cells.

and

Table 6. RMSE and MAPE of $h$-step ahead forecast obtained by DSHW model from the original series, KM.N and SSA.KM.N bootstrap series.

	1st Data Set						2nd Data Set
	RMSE			MAPE			RMSE			MAPE
$\mathit{h}$	12	24	36	12	24	36	12	24	36	12	24	36
original	3448.13	4802.00	4207.31	4.19	6.97	6.39	1458.59	1231.29	1148.92	2.77	2.05	2.00
Bagging + Mean
KM.N
$n_{\mathcal{B}}$ = 25	2183.44	3644.90	3236.11	2.67	5.39	5.32	1809.37	1902.75	1718.99	3.29	3.06	2.75
$n_{\mathcal{B}}$ = 50	2230.95	3640.88	3233.82	2.83	5.46	5.40	1890.92	2012.37	1801.60	3.43	3.24	2.89
$n_{\mathcal{B}}$ = 75	2194.68	3590.45	3183.48	2.81	5.40	5.34	1918.15	2065.24	1842.82	3.47	3.33	2.96
$n_{\mathcal{B}}$ = 100	2268.22	3660.82	3241.71	2.91	5.50	5.40	1913.90	2052.31	1828.07	3.49	3.32	2.96
SSA. KM.N
$n_{\mathcal{B}}$ = 25	3168.62	4611.09	4018.48	3.76	6.62	5.99	2361.26	2782.86	2349.97	3.68	4.33	3.82
$n_{\mathcal{B}}$ = 50	3157.42	4596.48	4006.13	3.74	6.60	5.97	2552.00	2945.08	2475.15	4.06	4.56	3.95
$n_{\mathcal{B}}$ = 75	3156.84	4598.52	4007.59	3.74	6.60	5.97	2544.55	2946.87	2487.05	4.03	4.59	4.05
$n_{\mathcal{B}}$ = 100	3153.79	4597.77	4005.71	3.73	6.60	5.97	2521.97	2911.70	2455.59	4.02	4.55	3.99
Bagging + Median
KM.N
$n_{\mathcal{B}}$ = 25	2556.48	4295.01	3778.13	3.18	6.27	5.93	1860.06	1967.00	1771.43	3.33	3.12	2.79
$n_{\mathcal{B}}$ = 50	2542.87	4273.27	3758.95	3.12	6.23	5.90	1922.97	2026.09	1817.49	3.47	3.23	2.88
$n_{\mathcal{B}}$ = 75	2574.91	4263.89	3750.11	3.15	6.21	5.88	1934.91	2060.74	1843.21	4.48	3.30	2.93
$n_{\mathcal{B}}$ = 100	2658.87	4320.54	3801.11	3.21	6.28	5.94	1935.22	2046.94	1830.22	3.52	3.29	2.92
SSA.KM.N
$n_{\mathcal{B}}$ = 25	3184.94	4621.30	4025.46	3.77	6.63	5.99	2372.52	2855.73	2427.66	3.58	4.50	4.06
$n_{\mathcal{B}}$ = 50	3183.66	4607.60	4014.75	3.77	6.62	5.98	2602.98	2938.16	2472.20	4.11	4.46	3.93
$n_{\mathcal{B}}$ = 75	3178.47	4606.99	4014.94	3.76	6.62	5.98	2517.25	2841.47	2404.81	4.01	4.38	3.91
$n_{\mathcal{B}}$ = 100	3184.52	4610.00	4016.52	3.77	6.62	5.98	2521.24	2812.68	2374.88	4.03	4.33	3.84

Green cells represent the RMSE and MAPE values of the SARIMA model obtained from the bootstrap series lower than those obtained from the original series. Bold values represent the lowest value in a column of each bagging method with green cells.

, respectively. We also present in Table 3, Table 4, Table 5 and Table 6 the RMSE and MAPE obtained from the forecast values of each model with four different numbers of bootstrap time series, to infer whether the number of bootstrap samples interferes with the accuracy of the forecasts.

Based on Table 3, we can see that the SSA.KM.N performed better than the KM.N in reducing the RMSE and MAPE of forecasts for 24- and 36 steps-ahead, respectively, obtained by the SARIMA model. The green cells in Table 3 represent the RMSE and MAPE values of the SARIMA model obtained from the bootstrap series, which are lower than those obtained from the original series. Bold values represent the lowest value in a column of each bagging method with green cells.

Moreover, in Table 3, it can be seen that for the first dataset, SARIMA provided high accuracy values for forecasting one day ahead (next 24 h), indicated by the MAPE values of less than two. For the second dataset, the MAPE values were between two and three. For each bagging method, there is no significant difference between the forecast results obtained by the mean and the median ensemble.

Based on the analysis for multistep-ahead forecasting by the SARIMA model with the bagging methods, it cannot be concluded that the more bootstrapped series used in the calculation, the more accurate the forecasting results will be. As we can see, the values in bold (see Table 3) are not in the $n_{\mathcal{B}}$ = 100 row, being some in the $n_{\mathcal{B}}$ = 25 row.

The comparative forecast results obtained by the NNAR model reconstructed from the original and bootstrap series are presented in Table 4. Based on the analysis of this table, the NNAR tends to produce larger RMSE than SARIMA. This is not in line with the results for predicting one step ahead (see Table 2). However, both the KM.N and SSA.KM.N methods can improve the accuracy performance of the forecasts. The 36-steps-ahead forecast values obtained from the bootstrap series using SSA.KM.N produced a larger RMSE than those obtained from the original time series, but the MAPE value showed the opposite direction. Similar to the results shown in Table 3, in this case, a greater number of bagging samples does not necessarily result in a better performance in terms of forecasting accuracy.

Table 5 shows that, for the first dataset, bagging did not improve the forecast accuracy obtained by the TBATS model. For the second dataset, the SSA.KM.N enhanced the performance of forecasting accuracy, but this did not apply to the KM.N. Based on the RMSE, the forecast values calculated from the TBATS model were more accurate than those obtained from the NNAR model.

Contrary to the results shown in Table 5, bagging implementation improved the forecasting accuracy of the DSHW model for the first dataset but not for the second dataset (see Table 6). In this case, the KM.N bagging performed better than the SSA.KM.N in reducing the forecasting error. Table 6 shows that the MAPE values obtained by the DSHW model for the first dataset are on average 2–3 times higher than those obtained by the DSHW model for the second dataset. However, in this case, the application of KM.N was able to reduce the MAPE value for 12-step ahead by approximately 36%.

By implementing the SSA.KM.N in the hourly load forecasting of Malaysia up to 36 steps ahead, the RMSE was able to be reduced by 4.97% and 40% when using SARIMA and NNAR, respectively. Meanwhile, KM.N was able to reduce the RMSE value by up to 3.8% for SARIMA and up to 35.43% for NNAR. Furthermore, although in one case, the SSA.KM.N bagging implementation for predicting up to 36 steps ahead using TBATS and DSHW can decrease the RMSE by more than 10%, in another case, it may behave differently. Similar conclusions were obtained when analyzing the MAPE.

For further evaluation, we consider MBE to see the direction of the models and present the results in

Table 7. MBEs of $h$-step ahead forecast for the first data set obtained by SARIMA, NNAR, TBATS, and DSHW models.

	SARIMA			NNAR			TBATS			DSHW
$\mathit{h}$	12	24	36	12	24	36	12	24	36	12	24	36
original	145.99	907.89	1446.04	751.37	1574.98	2308.25	−69.78	598.89	607.95	2233.79	3829.69	3311.54
Bagging + Mean
KM.N
$n_{\mathcal{B}}$ = 25	167.58	905.80	1390.65	443.29	1232.64	1968.24	290.35	1200.32	1246.45	1356.39	2890.41	2618.15
$n_{\mathcal{B}}$ = 50	214.87	937.02	8881.84	441.63	1236.06	1269.93	384.09	1287.13	470.02	1447.27	2924.45	1012.76
$n_{\mathcal{B}}$ = 75	230.20	950.19	1425.00	460.41	1246.86	2003.83	371.32	1252.58	1318.35	1436.04	2891.67	2610.35
$n_{\mathcal{B}}$ = 100	257.88	969.21	1441.16	473.65	1266.20	2025.98	348.74	1242.51	1298.32	1490.21	2951.16	2654.25
SSA. KM.N
$n_{\mathcal{B}}$ = 25	−137.62	728.93	1264.851	411.07	1170.29	1895.15	269.68	1112.43	1132.33	1921.13	3601.54	3093.45
$n_{\mathcal{B}}$ = 50	−150.17	714.81	869.88	386.29	1130.32	1210.20	221.89	1042.36	334.78	1919.84	3593.18	1010.49
$n_{\mathcal{B}}$ = 75	−164.26	705.98	1243.53	402.64	1142.61	1863.60	238.15	1053.85	1069.63	1919.89	3595.02	3087.34
$n_{\mathcal{B}}$ = 100	−154.71	714.48	1250.64	391.24	1133.87	1851.27	323.68	1139.93	1162.73	1913.61	3591.93	3083.82
Bagging + Median
KM.N
$n_{\mathcal{B}}$ = 25	198.26	925.21	1407.78	413.57	1213.51	1938.24	447.49	1340.14	1307.19	1533.13	3350.10	2965.00
$n_{\mathcal{B}}$ = 50	230.70	946.82	1426.14	413.63	1209.42	1948.60	297.71	1130.41	1117.38	1553.86	3350.17	2963.54
$n_{\mathcal{B}}$ = 75	234.08	951.99	1428.00	420.36	1207.34	1954.58	375.90	1203.43	1178.55	1593.23	3355.65	2964.73
$n_{\mathcal{B}}$ = 100	260.12	972.47	1444.30	440.88	1227.32	1970.68	558.25	1415.63	1394.91	1645.62	3403.65	3003.21
SSA.KM.N
$n_{\mathcal{B}}$ = 25	−152.57	722.76	1258.23	387.98	1150.80	1894.71	343.25	1289.54	1343.09	1932.04	3610.62	3097.90
$n_{\mathcal{B}}$ = 50	−172.75	699.00	1237.00	378.60	1130.51	1820.84	431.02	1376.17	1441.36	1935.88	3603.03	3092.51
$n_{\mathcal{B}}$ = 75	−177.18	696.20	1233.93	382.15	1136.42	1835.54	421.09	1348.28	1416.42	1934.80	3603.84	3094.52
$n_{\mathcal{B}}$ = 100	−168.17	708.37	1244.94	367.05	1113.81	1801.46	413.72	1325.61	1396.90	1939.26	3606.19	3095.18

Green cells represent the MBE values obtained from the bootstrap series for the first data set lower (in absolute values) than those obtained from the original series. Bold values represent the lowest value in a column of each bagging method with green cells.

and

Table 8. MBEs of $h$-step ahead forecast for the second data set obtained by SARIMA, NNAR, TBATS, and DSHW models.

	SARIMA			NNAR			TBATS			DSHW
$\mathit{h}$	12	24	36	12	24	36	12	24	36	12	24	36
original	287.06	−228.44	−591.45	2319.17	1146.12	2903.47	−1266.71	−2379.66	−1457.27	−1178.78	−805.60	−534.08
Bagging + Mean
KM.N
$n_{\mathcal{B}}$ = 25	616.40	−3.69	−389.82	1733.21	472.40	1818.49	−1191.85	−2641.61	−1815.55	−1496.71	−1538.48	−935.39
$n_{\mathcal{B}}$ = 50	568.38	−38.61	−582.74	1784.60	530.70	1375.18	−1071.33	−2541.56	−431.21	−1568.82	−1638.40	10.61
$n_{\mathcal{B}}$ = 75	566.62	−40.03	−423.90	1729.60	476.55	1807.17	−1041.47	−2524.44	−1688.93	−1591.59	−1687.65	−1044.28
$n_{\mathcal{B}}$ = 100	559.38	−43.89	−428.30	1744.63	493.21	1827.51	−1009.37	−2480.44	−1640.25	−1592.14	−1679.05	−1047.71
SSA. KM.N
$n_{\mathcal{B}}$ = 25	291.41	−224.24	−571.59	1672.25	481.19	2374.84	−97.56	−869.941	−314.26	−1783.46	−2256.37	−1713.42
$n_{\mathcal{B}}$ = 50	283.98	−225.54	−606.41	1728.23	557.90	1976.58	−149.71	−914.66	−18.19	−1952.89	−2375.25	−335.45
$n_{\mathcal{B}}$ = 75	283.69	−226.25	−574.95	1708.84	537.57	2457.88	−125.375	−832.99	−348.08	−1943.21	−2389.85	−1849.06
$n_{\mathcal{B}}$ = 100	285.39	−222.95	−571.83	1695.97	518.81	2448.55	−209.48	−940.32	−418.47	−1931.83	−2363.00	−1804.90
Bagging + Median
KM.N
$n_{\mathcal{B}}$ = 25	653.55	10.43	−376.27	1730.47	422.54	1737.09	−1138.90	−2723.60	−1888.97	−1529.41	−1578.47	−957.24
$n_{\mathcal{B}}$ = 50	600.71	−16.70	−403.8	1811.03	532.83	1586.62	−1102.34	−2699.53	−1860.87	−1591.57	−1636.36	−995.47
$n_{\mathcal{B}}$ = 75	604.67	−19.03	−405.87	1751.48	487.22	1788.34	−1095.80	−2690.14	−1849.37	−1600.02	−1674.19	−1024.61
$n_{\mathcal{B}}$ = 100	578.09	−35.40	−419.54	1763.27	504.38	1817.09	−1083.54	−2654.90	−1816.41	−1609.89	−1663.54	−1023.59
SSA.KM.N
$n_{\mathcal{B}}$ = 25	289.37	−221.65	−571.52	1652.67	470.11	2375.43	−3.50	−755.85	−200.04	−1746.97	−2341.34	−1867.79
$n_{\mathcal{B}}$ = 50	285.59	−223.41	−574.00	1668.23	499.54	2419.34	−227.46	−1074.75	−432.69	−1977.57	−2334.53	−1784.67
$n_{\mathcal{B}}$ = 75	283.23	−224.74	−575.07	1694.84	517.58	2458.64	−267.19	−1001.17	−421.91	−1921.88	−2271.71	−1797.42
$n_{\mathcal{B}}$ = 100	284.76	−221.12	−571.49	1690.07	514.03	2440.18	−350.40	−1103.52	−506.35	−1925.08	−2246.15	−1737.53

Green cells represent the MBE values obtained from the bootstrap series for the first data set lower (in absolute values) than those obtained from the original series. Bold values represent the lowest value in a column of each bagging method with green cells.

. The MBE is supposed to provide information on the long-term performance of the model. Based on Table 7 and Table 8, the interpretation of the model performance is consistent with that based on RMSE and MAPE. The directions of the bias generated by the models with and without bagging are the same, except for the forecasting of 12 steps ahead by SARIMA (see Table 7). It may be related to the weakness of MBE, where the positive and negative errors can cancel each other, and high individual errors can result in low MBE values. However, we can see that bagging methods, both KM.N and SSA.KM.N, reduces MBE values obtained from the NNAR model compared with those obtained from the original time series. In the case of modeling the second data set by TBATS, SSA, KM.N yields lower MBEs than the KM.N bagging method.

3.2. Application to the Hourly Electricity Load of Poland

Figure 5 shows the hourly electricity load of Poland, in Megawatts (Mwh), from 26 October, at 01:00 to 16 December 2020 at 00:00. The data were accessed from https://www.pse.pl/obszary-dzialalnosci/krajowy-system-elektroenergetyczny/zapotrzebowanie-kse (accessed on 21 January 2021). This data set contains the linear trend and multiple seasonal patterns with daily and weekly periods. There was a slight pattern change around time index 400 (11 November 2020) due to the influence of the National Independence Day holiday (shown by the orange rectangle in Figure 5). We fit the model using the first 1212 observations and evaluated the forecasting accuracy performance using the last 36 observations.

In this experimental study, we generate 50 bootstrapped series from the original electricity load of Poland using KM.N and SSA.KM.N. We then model each generated time series by SARIMA, NNAR, TBATS, and NNAR, in the same way as for the Malaysian data. The accuracy evaluation was based on RMSE, MAPE, and MBE for 12, 24, and 36 steps ahead, and is summarized in

Table 9. RMSEs of $h$-step ahead forecast for the hourly electricity load of Poland obtained by SARIMA, NNAR, TBATS, and DSHW models.

	SARIMA			NNAR			TBATS			DSHW
$\mathit{h}$	12	24	36	12	24	36	12	24	36	12	24	36
Original	2120.28	2436.07	2540.46	107.18	308.45	382.81	259.27	437.82	466.29	351.51	289.03	284.54
Bagging + Mean
KM.N
$n_{\mathcal{B}}$ = 10	298.01	254.32	229.17	133.53	343.25	416.31	309.15	456.14	468.23	323.29	258.16	254.41
$n_{\mathcal{B}}$ = 20	276.62	237.69	220.48	135.79	319.32	390.97	307.57	457.72	467.18	326.77	255.35	255.16
$n_{\mathcal{B}}$ = 25	266.20	231,54	218.07	132.76	305.93	371.25	308.52	459.61	469.26	305.33	240.89	240.29
$n_{\mathcal{B}}$ = 50	300.58	257.61	230.87	146.32	316.44	383.40	304.79	455.58	467.42	318.51	246.98	243.43
SSA. KM.N
$n_{\mathcal{B}}$ = 10	361.10	304.85	256.78	130.42	254.16	290.95	291.24	433.47	441.28	365.75	298.55	291.71
$n_{\mathcal{B}}$ = 20	356.84	302.42	254.80	128.66	228.89	262.25	296.73	439.51	444.58	369.77	298.33	291.90
$n_{\mathcal{B}}$ = 25	353.73	299.78	252.83	132.14	234.19	268.70	296.91	442.01	447.02	370.41	300.02	291.14
$n_{\mathcal{B}}$ = 50	353.14	299.24	252.50	131.97	231.66	267.86	299.30	435.43	437.14	374.79	300.84	290.49
Bagging + Median
KM.N
$n_{\mathcal{B}}$ = 10	323.16	270.39	243.86	127.12	330.51	401.92	319.28	463.28	470.74	324.95	261.77	253.61
$n_{\mathcal{B}}$ = 20	287.77	243.37	222.76	135.81	312.82	385.15	312.11	461.49	470.19	330.34	257.77	256.02
$n_{\mathcal{B}}$ = 25	265.14	227.19	213.62	136.70	298.21	370.86	312.14	461.90	470.74	316.96	249.81	245.81
$n_{\mathcal{B}}$ = 50	302.26	257.68	228.44	150.87	313.86	381.42	310.11	460.61	470.76	336.98	261.51	257.76
SSA.KM.N
$n_{\mathcal{B}}$ = 10	364.58	306.80	258.27	137.93	286.20	326.78	296.29	448.13	448.54	366.90	303.00	297.67
$n_{\mathcal{B}}$ = 20	362.98	307.99	259.25	132.48	245.93	287.50	297.94	449.62	451.81	376.13	303.83	298.76
$n_{\mathcal{B}}$ = 25	364.04	308.75	260.17	137.58	247.58	284.35	300.09	453.73	456.32	373.93	304.04	295.95
$n_{\mathcal{B}}$ = 50	306.64	258.38	1.44	133.83	232.00	267.26	299.78	450.28	450.92	378.22	305.12	294.39

Green cells represent the RMSE values obtained from the bootstrap series for the Polish data set lower than those obtained from the original series.

,

Table 10. MAPEs of $h$-step ahead forecast for the hourly electricity load of Poland obtained by SARIMA, NNAR, TBATS, and DSHW models.

	SARIMA			NNAR			TBATS			DSHW
$\mathit{h}$	12	24	36	12	24	36	12	24	36	12	24	36
original	7.09	9.99	10.20	0.36	0.84	1.19	1.01	1.52	1.59	1.34	1.13	1.11
Bagging + Mean
KM.N
$n_{\mathcal{B}}$ = 10	1.14	0.99	0.86	0.42	0.98	1.29	1.15	1.89	1.84	1.23	0.98	0.97
$n_{\mathcal{B}}$ = 20	1.02	0.90	0.82	0.43	0.91	1.21	1.13	1.89	1.84	1.19	0.91	0.93
$n_{\mathcal{B}}$ = 25	0.97	0.88	0.81	0.41	0.86	1.40	1.13	1.90	1.85	1.13	0.87	0.89
$n_{\mathcal{B}}$ = 50	1.12	0.99	0.86	0.48	0.92	1.20	1.14	1.88	1.84	1.14	0.86	0.87
SSA. KM.N
$n_{\mathcal{B}}$ = 10	1.44	1.22	0.86	0.42	0.75	0.91	1.10	1.76	1.71	1.37	1.16	1.13
$n_{\mathcal{B}}$ = 20	1.42	1.21	0.86	0.42	0.69	0.83	1.10	1.80	1.73	1.38	1.15	1.12
$n_{\mathcal{B}}$ = 25	1.41	1.21	0.86	0.43	0.71	0.84	1.10	1.81	1.75	1.38	1.16	1.12
$n_{\mathcal{B}}$ = 50	1.41	1.20	0.86	0.43	0.70	0.84	1.10	1.79	1.71	1.39	1.15	1.11
Bagging + Median
KM.N
$n_{\mathcal{B}}$ = 10	1.22	1.05	0.91	0.39	0.93	1.24	1.18	1.93	1.87	1.22	1.01	0.98
$n_{\mathcal{B}}$ = 20	1.06	0.92	0.83	0.411	0.90	1.20	1.15	1.91	1.86	1.18	0.90	0.92
$n_{\mathcal{B}}$ = 25	0.96	0.86	0.79	0.41	0.84	1.14	1.15	1.91	1.86	1.14	0.89	0.90
$n_{\mathcal{B}}$ = 50	1.12	0.98	0.84	0.49	0.93	1.20	1.16	1.91	1.86	1.20	0.91	0.92
SSA.KM.N
$n_{\mathcal{B}}$ = 10	1.45	1.23	0.96	0.46	0.83	1.01	1.10	1.84	1.75	1.38	1.19	1.17
$n_{\mathcal{B}}$ = 20	1.44	1.24	0.96	0.44	0.74	0.90	1.09	1.85	1.77	1.40	1.17	1.15
$n_{\mathcal{B}}$ = 25	1.45	1.24	0.97	0.46	0.74	0.89	1.09	1.87	1.79	1.39	1.17	1.14
$n_{\mathcal{B}}$ = 50	1.44	1.22	0.86	0.44	0.70	0.84	1.09	1.85	1.76	1.40	1.18	1.13

Green cells represent the MAPE values obtained from the bootstrap series for the Polish data set lower than those obtained from the original series.

and

Table 11. MBEs of $h$-step ahead forecast for the hourly electricity load of Poland obtained by SARIMA, NNAR, TBATS, and DSHW models.

	SARIMA			NNAR			TBATS			DSHW
$\mathit{h}$	12	24	36	12	24	36	12	24	36	12	24	36
original	1561.97	411.79	962.12	86.73	198.18	286.42	−117.26	79.96	170.71	135.94	0.51	40.35
Bagging + Mean
KM.N
$n_{\mathcal{B}}$ = 10	273.31	174.83	76.82	104.49	227.21	309.50	−120.87	−39.71	66.39	154.18	44.32	67.89
$n_{\mathcal{B}}$ = 20	244.56	147.74	53.40	104.45	211.54	289.94	−129.99	−49.57	53.46	187.92	84.30	107.71
$n_{\mathcal{B}}$ = 25	232.10	136.46	42.70	97.04	198.57	270.37	−132.55	−51.01	53.80	168.66	72.60	95.20
$n_{\mathcal{B}}$ = 50	268.19	174.37	81.24	118.23	215.67	287.11	−118.26	−35.53	66.27	195.48	92.75	110.85
SSA. KM.N
$n_{\mathcal{B}}$ = 10	344.43	247.28	163.03	99.88	163.18	202.15	−150.01	−41.64	45.34	171.18	15.69	58.17
$n_{\mathcal{B}}$ = 20	340.06	234.96	158.87	101.06	147.35	178.70	−150.37	−52.88	33.91	178.98	23.50	65.33
$n_{\mathcal{B}}$ = 25	337.43	241.26	155.77	105.18	152.51	183.96	−147.70	−54.21	33.98	179.56	23.77	63.67
$n_{\mathcal{B}}$ = 50	336.97	240.48	154.86	104.43	150.84	183.92	−156.03	−64.71	16.50	189.36	25.20	65.02
Bagging + Median
KM.N
$n_{\mathcal{B}}$ = 10	290.28	194.92	91.57	92.97	214.79	−288.01	−136.15	−61.41	47.86	140.73	33.83	50.60
$n_{\mathcal{B}}$ = 20	253.80	158.19	66.38	100.62	207.48	−228.01	−131.90	−56.81	48.96	195.85	91.18	109.55
$n_{\mathcal{B}}$ = 25	231.23	138.56	14.34	97.07	193.80	−228.01	−129.80	−53.97	50.75	180.31	76.99	96.29
$n_{\mathcal{B}}$ = 50	267.03	173.02	85.13	121.50	216.16	−228.01	−120.45	−44.30	59.69	206.92	101.26	120.60
SSA.KM.N
$n_{\mathcal{B}}$ = 10	347.60	248.26	162.68	111.64	189.09	231.51	−146.64	−61.85	28.45	154.73	−3.77	45.28
$n_{\mathcal{B}}$ = 20	346.04	249.75	164.58	107.36	160.93	201.89	−145.21	−62.12	30.76	174.09	15.25	61.85
$n_{\mathcal{B}}$ = 25	347.75	250.10	166.04	112.56	162.74	198.52	−147.18	−67.27	27.96	173.74	18.04	59.33
$n_{\mathcal{B}}$ = 50	345.41	248.56	162.74	106.97	151.23	182.94	−151.43	−72.11	18.52	187.02	19.27	61.72

Green cells represent the MBE values obtained from the bootstrap series for the Polish data set lower (in absolute values) than those obtained from the original series.

.

Based on Table 9 and Table 10, we can see that SSA.KM.N can improve the accuracy of 24 and 36 steps ahead of electricity load forecasting for the Polish data using the NNAR model, while KM.N fails to improve forecasting accuracy for this model. On the other hand, the KM.N bagging method works well on the DSHW model, while SSA.KM.N does not perform so well. However, both of them succeeded in increasing the accuracy of forecasting for the ARIMA model.

We can see from Table 10 that implementing the bagging method on Polish data does not reduce the MAPE values in the case of the TBATS model. Still, it makes the MBEs smaller (in absolute values) than those obtained from the original data (Table 11). Furthermore, although the RMSE and MAPE values of the DSHW model decreased with bagging, the results were the opposite when analyzing the MBE values. SSA.KM.N gives better outcomes for the NNAR model, while KM.N is better for the DSHW model.

3.3. Application to the Hourly Electricity Load of Java-Bali, Indonesia

To show the generality of the implementation of bagging methods in electricity load forecasting, we also discuss the hourly electricity load of Java-Bali, Indonesia. The data consists of 1464 observations, from 1 October to 30 November 2015. Figure 6 shows that the data has no trend but has double seasonal patterns. It is relatively stable except at time points around 312–336 (14 October 2015) due to the influence of the Hijriyah New Year holiday (shown by the orange rectangle in Figure 6). Moreover, this data set was also discussed in [54].

We generate 50 bootstrapped time series for this case based on 1428 observations (1 October at 01.00 to 29 November at 12.00). The error evaluation in terms of RMSE, MAPE, and MBE obtained from the SARIMA, NNAR, TBATS, and DSHW models are summarized in

Table 12. RMSE of $h$-step ahead forecast for the hourly electricity load of Indonesia obtained by SARIMA, NNAR, TBATS, and DSHW models.

	SARIMA			NNAR			TBATS			DSHW
$\mathit{h}$	12	24	36	12	24	36	12	24	36	12	24	36
original	1236.72	2440.28	3119.10	193.12	216.95	234.57	611.23	702.52	609.34	167.51	265.62	351.18
Bagging + Mean
KM.N
$n_{\mathcal{B}}$ = 10	140.46	148.61	182.96	192.32	200.15	211.26	581.86	681.02	593.49	138.18	231.80	339.36
$n_{\mathcal{B}}$ = 20	137.65	148.79	182.42	201.59	201.79	207.81	581.67	678.01	592.20	137.31	224.45	318.91
$n_{\mathcal{B}}$ = 25	136.02	149.93	184.64	197.91	198.36	204.31	581.49	677.87	591.87	139.20	227.65	327.53
$n_{\mathcal{B}}$ = 50	136.02	149.93	184.64	194.08	200.23	208.27	580.00	677.27	590.68	147.81	245.81	362.49
SSA. KM.N
$n_{\mathcal{B}}$ = 10	181.30	178.87	194.99	136.43	177.76	208.49	605.75	704.10	608.36	170.08	270.61	364.78
$n_{\mathcal{B}}$ = 20	186.56	184.79	199.14	135.42	177.59	208.45	600.50	700.37	605.79	167.54	277.51	385.43
$n_{\mathcal{B}}$ = 25	185.58	183.79	198.45	134.86	178.64	209.78	597.76	698.29	604.43	167.74	277.13	383.10
$n_{\mathcal{B}}$ = 50	193.68	189.13	201.59	146.47	184.91	213.14	602.30	699.76	605.76	150.49	260.36	370.83
Bagging + Median
KM.N
$n_{\mathcal{B}}$ = 10	139.51	144.76	183.00	190.81	199.99	210.46	581.67	680.87	593.92	153.44	245.70	352.49
$n_{\mathcal{B}}$ = 20	138.10	146.75	183.39	198.30	200.90	205.40	581.72	680.30	593.77	162.20	247.44	344.32
$n_{\mathcal{B}}$ = 25	139.17	150.71	187.80	195.26	196.90	202.43	584.21	681.03	594.27	160.89	242.23	344.98
$n_{\mathcal{B}}$ = 50	136.72	146.71	184.83	192.91	199.18	206.05	585.00	682.61	595.21	161.88	258.46	372.81
SSA.KM.N
$n_{\mathcal{B}}$ = 10	180.44	178.15	194.23	135.73	174.27	206.11	605.40	700.65	606.85	166.57	268.12	358.32
$n_{\mathcal{B}}$ = 20	187.25	184.52	198.52	132.52	176.76	207.01	604.35	700.16	606.58	168.90	267.43	360.99
$n_{\mathcal{B}}$ = 25	186.72	183.84	198.25	135.54	178.50	208.79	598.77	696.99	604.16	168.29	268.13	366.47
$n_{\mathcal{B}}$ = 50	193.01	188.59	200.85	149.27	186.48	213.25	607.15	701.46	607.49	155.88	258.63	359.15

Green cells represent the RMSE values obtained from the bootstrap series for the Indonesian data set lower than those obtained from the original series.

,

Table 13. MAPE of $h$-step ahead forecast for the hourly electricity load of Indonesia obtained by SARIMA, NNAR, TBATS, and DSHW models.

	SARIMA			NNAR			TBATS			DSHW
$\mathit{h}$	12	24	36	12	24	36	12	24	36	12	24	36
original	5.83	10.87	13.45	0.95	1.03	1.00	2.78	3.25	2.59	0.77	1.10	1.40
Bagging + Mean
KM.N
$n_{\mathcal{B}}$ = 10	0.67	0.70	0.76	0.96	0.96	0.92	2.67	3.15	2.53	0.64	0.92	1.29
$n_{\mathcal{B}}$ = 20	0.65	0.71	0.76	1.01	0.98	0.92	2.66	3.15	2.53	0.61	0.90	1.22
$n_{\mathcal{B}}$ = 25	0.63	0.71	0.77	0.99	0.96	0.91	2.66	3.14	2.53	0.63	0.91	1.25
$n_{\mathcal{B}}$ = 50	0.63	0.71	0.77	0.97	0.96	0.91	2.65	3.14	2.53	0.68	0.96	1.37
SSA. KM.N
$n_{\mathcal{B}}$ = 10	0.87	0.86	0.85	0.66	0.78	0.84	2.76	3.24	2.58	0.79	1.13	1.45
$n_{\mathcal{B}}$ = 20	0.91	0.90	0.88	0.66	0.78	0.83	2.73	3.23	2.57	0.78	1.12	1.51
$n_{\mathcal{B}}$ = 25	0.90	0.89	0.88	0.66	0.78	0.84	2.71	3.22	2.56	0.79	1.12	1.50
$n_{\mathcal{B}}$ = 50	0.94	0.92	0.89	0.73	0.83	0.87	2.73	3.22	2.57	0.68	1.03	1.43
Bagging + Median
KM.N
$n_{\mathcal{B}}$ = 10	0.67	0.69	0.75	0.95	0.96	0.92	2.66	3.15	2.54	0.71	0.98	1.35
$n_{\mathcal{B}}$ = 20	0.65	0.70	0.76	0.99	0.97	0.91	2.66	3.26	2.54	0.75	1.00	1.33
$n_{\mathcal{B}}$ = 25	0.66	0.72	0.78	0.97	0.94	0.89	2.68	3.16	2.54	0.74	0.97	1.32
$n_{\mathcal{B}}$ = 50	0.65	0.71	0.76	0.96	0.95	0.90	2.68	3.17	2.55	0.76	1.02	1.43
SSA.KM.N
$n_{\mathcal{B}}$ = 10	0.87	0.86	0.85	0.67	0.78	0.83	2.73	3.24	2.58	0.77	1.12	1.43
$n_{\mathcal{B}}$ = 20	0.91	0.90	0.88	0.65	0.78	0.83	2.75	3.23	2.58	0.76	1.11	1.43
$n_{\mathcal{B}}$ = 25	0.90	0.89	0.87	0.67	0.79	0.84	2.72	3.22	2.57	0.77	1.10	1.44
$n_{\mathcal{B}}$ = 50	0.95	0.89	0.92	0.75	0.84	0.88	2.76	3.23	2.58	0.70	1.06	1.41

Green cells represent the MAPE values obtained from the bootstrap series for the Indonesian data set lower than those obtained from the original series.

and

Table 14. MBE of $h$-step ahead forecast for the hourly electricity load of Indonesia obtained by SARIMA, NNAR, TBATS, and DSHW models.

	SARIMA			NNAR			TBATS			DSHW
$\mathit{h}$	12	24	36	12	24	36	12	24	36	12	24	36
original	760.77	1914.50	2658.76	−171.70	−73.39	−7.27	−247.66	−394.84	−210.28	13.77	53.09	184.57
Bagging + Mean
KM.N
$n_{\mathcal{B}}$ = 10	−110.53	−107.27	−74.87	−176.59	−90.30	−17.23	−196.70	−361.33	−186.34	30.98	63.46	194.51
$n_{\mathcal{B}}$ = 20	−107.48	−109.13	−77.58	−185.87	−104.32	−29.48	−192.40	−354.91	−178.92	14.41	46.96	170.86
$n_{\mathcal{B}}$ = 25	−104.71	−110.04	−79.76	−181.50	−104.71	−30.37	−192.71	−355.37	−179.71	25.91	58.99	184.20
$n_{\mathcal{B}}$ = 50	−104.71	−110.04	−79.76	−176.85	−99.07	−24.80	−199.37	−358.92	−184.30	52.43	89.04	224.39
SSA. KM.N
$n_{\mathcal{B}}$ = 10	−157.22	−147.24	−104.51	−121.14	−33.90	26.25	−242.95	−399.82	−220.51	9.29	53.06	193.08
$n_{\mathcal{B}}$ = 20	−163.43	−154.65	−111.92	−120.07	−32.97	27.19	−237.07	−394.13	−214.88	12.78	68.00	216.31
$n_{\mathcal{B}}$ = 25	−162.16	−153.24	−110.25	−119.85	−32.31	28.96	−232.08	−390.43	−211.79	14.10	68.99	215.21
$n_{\mathcal{B}}$ = 50	−170.78	−158.66	−113.53	−133.25	−41.03	22.80	−241.80	−395.38	−215.15	8.55	60.63	206.89
Bagging + Median
KM.N
$n_{\mathcal{B}}$ = 10	−109.90	−105.26	−73.05	−174.03	−91.36	−18.07	−197.11	−361.31	−185.10	36.11	72.84	204.91
$n_{\mathcal{B}}$ = 20	−109.00	−107.40	−75.75	−182.41	−105.51	−31.60	−196.47	−359.60	−182.80	29.83	62.95	191.74
$n_{\mathcal{B}}$ = 25	−108.90	−111.56	−80.54	−177.65	−105.33	−31.90	−199.31	−360.76	−183.54	32.22	71.68	199.89
$n_{\mathcal{B}}$ = 50	−105.43	−105.82	−73.99	−174.42	−100.32	−26.06	−203.93	−364.34	−187.13	57.59	99.10	234.13
SSA.KM.N
$n_{\mathcal{B}}$ = 10	−156.02	−146.64	−104.14	−119.95	−35.59	25.79	−239.17	−392.13	−211.83	5.88	46.37	185.00
$n_{\mathcal{B}}$ = 20	−164.52	−153.32	−109.83	−118.08	−32.99	27.89	−239.19	−391.83	−211.09	−1.97	46.70	186.84
$n_{\mathcal{B}}$ = 25	−162.60	−152.57	−109.06	−121.50	−34.02	28.08	−231.34	−387.20	−207.83	1.60	52.27	193.65
$n_{\mathcal{B}}$ = 50	−171.08	−158.53	−113.30	−136.04	−42.54	21.33	−242.47	−393.92	−212.83	4.19	48.66	190.21

Green cells represent the MBE values obtained from the bootstrap series for the Indonesian data set lower (in absolute values) than those obtained from the original series.

, respectively. The overall results shown in Table 12, Table 13 and Table 14 for this application are similar to those of the previous applications for Malaysian and Polish electricity load data.

Table 13 shows that SSA.KM.N produces lower MAPE than KM.N for the NNAR model. Compared with that obtained from the original data, the MAPE of the NNAR model was able to be reduced by up to 31.38%, 24.27%, and 17% for 12, 24, and 36 steps ahead of forecast values, respectively. Meanwhile, KM.N failed to lower the MAPE value for 12 steps-ahead, and it only declined approximately 8.74% and 11% for 24 and 36 steps-ahead forecast values, respectively.

In addition, for the NNAR model, MBE presented in Table 14 shows the application of the SSA.KM.Ns bagging method provides less bias for 12 and 24 steps-ahead forecast values than without bagging. However, this does not apply to KM.N.

Based on the experimental findings of the four data sets, bagging implementation can work well to improve the forecasting accuracy of the SARIMA and NNAR models. However, the TBATS and DSHW did not yield the same behavior. The success of this implementation is thought to be influenced by the uncertainty of the models. In this experimental study, we found that some bootstrapped series failed to be modeled by TBATS and DSHW, affecting the final forecast results calculated based on the mean and median across all the forecast values.

The number of bootstrap series does not seem to affect the forecasting accuracy calculated by the mean and median ensemble. In some cases, the SSA.KM.N was able to improve the multistep-ahead forecasting accuracy, but in other cases, the KM.N provided better results.

In this case, the selection of the model is an important step to be considered. The application of bagging with the right forecasting model will increase the accuracy of multistep ahead forecast values. Further development of the hybrid model, i.e., FFORMA [55] and exponential smoothing-neural network [56] or other combinations depending on the pattern of the data, can be considered to help overcome the uncertainty of the models [30].

4. Conclusions

In this study, we evaluated the impact and benefit of applying the existing KM.N and our proposed clustering-based bootstrap method, SSA.KM.N, in overcoming the uncertainty in time series multistep-ahead point forecasts. We focused on time series with a trend, seasonal, and affected by calendar variation and considered two Malaysian, one Polish, and one Indonesian electricity load time series as illustrative examples.

KM.N is considered an appropriate method for bootstrapping data with complex seasonal patterns, such as electrical load data. In the proposed method, we combined SSA and KM.N with the hope of producing bootstrap values that are more similar to the original data. We considered the SSA method to decompose the load series into signal and noise. By SSA, the observed values influenced by the calendar variation appear more clearly in the noise component than in the original data. Bootstrapping this residual value and adding it to the signal will result in the bootstrap series values around the original data.

Furthermore, we applied the following four models, usually used as benchmark models in forecasting electricity load time series: SARIMA, NNAR, TBATS, and DSHW. These four models are applied to all bootstrapped series to obtain up to 36-steps ahead of forecast values. The final forecast at time t is obtained by the following two ensemble methods: the mean and the median across all forecast values at time t. Based on the experimental results, we note that the number of bootstrapped series does not seem to affect the forecasting accuracy calculated by the mean and the median ensemble. We also found that the model suitable for the original series is not necessarily good for all bootstrapped series. We note that the accuracy of multiple-step-ahead forecasting values can be improved when the model, with different parameters, is appropriate for both the original and bootstrap data. Thus, combining several models and ensemble learning methods can be the direction of future research.