Recurrent Neural Network Based Short-Term Load Forecast with Spline Bases and Real-Time Adaptation

Abstract

Short-term load forecast (STLF) plays an important role in power system operations. This paper proposes a spline bases-assisted Recurrent Neural Network (RNN) for STLF with a semi-parametric model being adopted to determine the suitable spline bases for constructing the RNN model. To reduce the exposure to real-time uncertainties, interpolation is achieved by an adapted mean adjustment and exponentially weighted moving average (EWMA) scheme for finer time interval forecast adjustment. To circumvent the effects of forecasted apparent temperature bias, the forecasted temperatures issued by the weather bureau are adjusted using the average of the forecast errors over the preceding 28 days. The proposed RNN model is trained using 15-min interval load data from the Taiwan Power Company (TPC) and has been used by system operators since 2019. Forecast results show that the spline bases-assisted RNN-STLF method accurately predicts the short-term variations in power demand over the studied time period. The proposed real-time short-term load calibration scheme can help accommodate unexpected changes in load patterns and shows great potential for real-time applications.

1. Introduction

Short Term Load Forecasting (STLF) can be used to obtain the most economical way to commit power generation sources while fulfilling policies requirements, ensuring reliability and meeting the security, environmental, and equipment constraints of the power system [1].

The daily load profile generally follows cyclic and seasonal patterns related to both the climate and human activities, and is intrinsically a univariate time series. Many general forecasting methods based on regression or time-series models can be used for load forecasting (e.g., a semi-parametric additive model [2] or an autoregressive integrated moving average (ARIMA) [3]). These methods assume, however, a linear relationship between the observed and future time series. This assumption makes them less effective for time series with significant nonlinear characteristics, such as those associated with energy demand. Chen et al. [4] considers a more complicated time series model with a functional trend curve to improve the forecast results.

Due to their nonlinear fitting ability, machine learning techniques have been applied to many forecasting problems. The Artificial Neural Network (ANN) [5] is a typical machine learning method. ANNs learn regularities and patterns automatically from past recorded data and produce generalized results with the ability to be self-adaptive. Feed forward Multilayer Perceptron (MLP) [6,7] and Generalized Regression Neural Network (GRNN) [8] are among numerous ANN-based STLF techniques used in many control centers [9,10]. A systematic review on the techniques of electricity demand forecasting methods is available [11]. This overview includes regression, time series analysis, ANN, support vector machines and certain bottom-up approaches.

Load demand time series are nonlinear and non-stationary, with time-dependent dynamic behaviors. Thus, in our view, the Recurrent Neural Network (RNN) is a more suitable model for this purpose, as it can consider the temporal context in the feedback connections and as it has a nonlinear property [12]. Several RNN models for STLF have been proposed. In [13], system net load profiles are modeled by RNNs as the outputs of dynamic systems influenced by weather, time and other environmental factors. A pooling-based deep RNN for household load forecasts is proposed in [14], where the focus is to provide more suitable training samples to the network.

Besides the cyclic patterns, the load demand is also affected by several exogenous variables, including the prevailing weather conditions, the calendar effect, and the general randomness inherent in individual’s behaviors. With climate change and the increase of behind-the-meter renewable integrations, it is challenging to effectively integrate the various exogenous variables into the STLF model and provide accurate load forecasts. In [15], an ensemble of radial basis function neural networks (RBFNNs) is proposed where exogenous features and features extracted from load series, through long short-term memory (LSTM) networks and multi-resolution wavelet transforms in various timescales, are used to train the RBFNNs. While in [16], a hybrid algorithm that combines similar day selection, empirical mode decomposition, and LSTM neural networks to construct a prediction model for STLF.

This study develops a RNN-based short-term (i.e., over the next few hours, the next day, or the next week) load forecast model for the Taiwan Power Company (TPC) system. It is implemented through a semi-parametric regression framework to catch general daily demand behaviors. The STLF method uses the least absolute shrinkage and selection operator (Lasso) regression [17] and an adaptive Lasso [18] to select suitable variables. It includes multi-resolution basis functions [19] and cubic B-spline basis functions in the RNN model, in order to catch useful patterns, reduce the overfitting problem, and increase the forecast accuracy. An intuitive approach for calibrating the temperature forecasts is also adopted to account for the trend of past temperature forecast errors.

To accommodate real-time operation purposes and deal with any unexpected changes in system net load, a real-time short-term load adjustment tool is also developed. An interpolation technique [20] based on the historical load values over the previous 15 min and their differences with the current STLF values is employed to achieve a real-time forecast adaptation in which the load forecast is updated every 5 min, for real time operation applications.

The main contribution of this work is to develop a systematic procedure to combine the methodologies of statistical modeling and machine learning in two consecutive stages, so as to provide accurate forecasts of the load demand. To address overfitting in the forecast model used in the statistical modelling stage, the important and effective factors are identified by means of a variable selection procedure, together with an error-reducing calibration of the forecasted temperature using a suitable non-linear transformation. RNN machine learning models are then constructed, to capture the uncertainties of the complex and non-linear load patterns. The procedure for real-time short-term load demand adjustment further enhances the real-time forecast accuracy. As a result of this improved forecasting, the system operator can begin earlier preparations to reduce the impact of net load intermittency, particularly in systems with higher renewable penetration.

2. Methodology

The proposed RNN-based STLF procedure, with selected bases through a semi-parametric model and a real-time load forecast adjustment scheme, is shown in Figure 1. In the first stage, forecasts of daily load patterns up to the next seven days are obtained using the apparent temperatures predicted by the Taiwan Central Weather Bureau (TCWB). In the second stage, based on past-forecasted results, real-time adapted forecasting load sequences are generated through the interpolation method using real-time adaptation and an exponentially weighted moving average (EWMA). Figure 1 presents the data flow of the proposed RNN-based ${\mathrm{RNN}}_{adp}$ model.

2.1. Apparent Temperature

The system net-loads are greatly affected by external factors such as temperature, humidity, wind speed and seasonal events that change over time. The apparent temperature index [21] equivalent to the temperature felt is used to evaluate its effect upon the load. The apparent temperature $T_a$is defined as:

$$T_a=1.04*T_c+0.2*e-0.65V-2.7$$

(1)

where $T_c$ is the temperature in Celsius, $e=6.105*\left(\frac{RH}{100}\right)*\mathrm{Exp}\left[\frac{17.27T_c}{237.7+T_c}\right]$, e is the pressure in hPa, V is the wind speed in m/sec, and RH is the relative humidity in percent. The apparent temperature for the next 48 h (with a 3 h resolution) is provided by the TCWB. Throughout this work, references to “temperature” refer to the apparent temperature.

Different regions have different weather patterns. The total system load is a combination of the loads from the north, central, south and east regions of Taiwan (whose average load proportions are 38%, 28%, 33% and 1%, respectively). Since the goal is to forecast the total system load, the temperatures of the four regions are merged into one value by taking the weighted average of their temperatures with weights equal to their load proportions; namely:

$${\tilde{T}}_a={\displaystyle \sum }_{i=1}^4{\ell }_i{T}_{a,i}$$

(2)

where $i=1,\dots ,4$ corresponds to the north, central, south and east regions respectively, $T_{a,i}$ is the temperature in the ith region and ${\ell }_{\mathrm{i}}$ is the proportion of regional load in system total energy demand.

2.2. Spline Basis Functions

To catch the general behaviors of the daily load patterns, we consider the class of multi-resolution basis functions proposed by Tzeng and Huang [19], which are ordered in the direction of increasing resolution detail, with the number of bases, K, being chosen to be large enough to represent the general 24-h patterns. On the other hand, due to the fact that the daily load patterns may change rapidly between the peak and off-peak periods, we also include the cubic B-spline basis functions to accommodate load patterns that change substantially within relatively short periods of time. Two sets of bases functions are used: (1). the multi-resolution bases $\left\{f_1,\dots ,f_n\right\}$ defined on n control points $\left\{s_1,\dots ,s_n\right\}$ and (2). the B-spline bases of order d,$\left\{B_{i,d}, i=1,\dots ,n\right\}$ with knots at $\left\{s_1,\dots ,s_n\right\}$. Details about the spline basis functions can be found in [19].

2.3. Semi-Parametric Model

A semi-parametric (SPM) model is adopted for STLF under the framework of additive models [22], by a suitable combination of the two aforementioned sets of spline basis functions, together with the nonlinear function of the temperature.

Now let the sequence of daily load random vectors at time t be $y_t=\left(y\left(t,1\right),\dots ,y\left(t,n\right)\right)\prime$, $t=1,\dots ,T$, with y(t,s) denoting the load at time t and local time grid location (control point) $s, s=1,\dots ,n.$

The model for STLF is assumed to have the following form:

$$\tilde{y}\left(t,s\right)=\log \left(y\left(t,s\right)\right)=\mu \left(t,s\right)+ϵ\left(t,s\right)$$

(3)

where $\mu \left(t,s\right)$ represents the mean function of the logarithm of the daily load $\tilde{y}\left(t,s\right)$, and $ϵ\left(t,s\right)$ is the corresponding random error at time $t=1,\cdots ,T$and period $s=1,\dots ,n$, assuming that ${ϵ}_t={\left(ϵ\left(t,1\right),\dots ,ϵ\left(t,n\right)\right)}^{’}~N\left(0,{\mathsf{\Sigma }}_n\left(t\right)\right),$ with ${\mathsf{\Sigma }}_n\left(t\right)$ being the covariance matrix of the random error vector ${ϵ}_t$ at time t. In the case of an independent error at time t and assuming ${\mathsf{\Sigma }}_n\left(t\right)={\sigma }^2{I}_n$, the time series errors sequence ${ϵ}_t, t=1,\dots , T,$ may yield a different covariance matrix estimate.

Specific patterns of the response variable (i.e., the system net load) of the model for forecasts are described below, under the following explanatory variables.

• It is observed that there are patterns depicted by the intra-daily, intra-weekly, peak and off-peak effects to be modeled by the multi-resolution and cubic B-spline bases.
• It is clear that the temperature significantly affects the load pattern. The weighted average of the temperature at different periods each day, and similarly the daily highest and lowest and the weighted average of temperatures in the different regions, are included as important predictors.
• The interaction effects of the period with the day type within each week are also crucial.

To accommodate these three effects, and also the temperature effect, let the mean function at time t and point s be given by:

$$\mu \left(t,s\right)=W\left(t\right)+D\left(s\right)+DW\left(t,s\right)+T\left(t,s\right)$$

(4)

with:

1

intra-weekly effect $W\left(t\right)$

2

intra-daily effect $D\left(s\right)$

3

interaction effect among the intra-daily and intra-weekly $DW\left(t,s\right)$

4

apparent temperature effect $T\left(t,s\right)$

To model the intra-daily effect, we use a combination of the first 24 multi-resolution bases $\left\{f_1,\dots ,f_{24}\right\}$ for 96 control points with $s_i=i, i=1,\dots ,96,$ and 96 cubic B-spline bases $\left\{ B_{j,4}^d, j=1,\dots ,96\right\}$ with knots at $s_j, j=1,\dots , 96$.

Similarly, the intra-weekly effect is modeled by 7 cubic B-spline bases $B_{k,}^w$ with knots at $w_k,$ $k=1,\dots ,7$.

The interaction effects among the intra-daily and intra-weekly are modeled by the products of the corresponding intra-daily and intra-weekly bases functions, namely: $\left\{B_{k,4}^w\cdot f_i, k=1,\dots , 7, i=1,\dots ,24, \right\}$, $\left\{B_{k,4}^w\cdot B_{j,4}^d, k=1,\dots ,7, j=1,\dots ,96\right\}$.

A detailed description of the spline bases functions is provided in Appendix A. The corresponding SPM is a linear combination of the above basis functions. The explicit expression of the model is given in Appendix B.

2.4. Model Bases Selection

Many standard estimators can be improved by shrinkage methods, such as ridge regression [23] and Lasso regression. This study adopts Lasso regression to obtain sparse solutions from the model bases selection.

In a Lasso regression, the value of the parameter controls both the size and the number of coefficients. Cross-validation is a resampling technique which can find a parameter value that ensures a proper balance between bias and variance. In this case, cross-validation considers the best tuning parameter value to be the one that minimizes the estimated test error rate of the forecasting results. More details about the Lasso estimate and adaptive Lasso can be found in [17,18].

2.5. Temperature Forecast Adjustment

Currently, TCWB provides three-hour temperature forecasts for the present day (D-day), the next day (D + 1), and the maximum and minimum temperature forecasts for the days (D + 2) to (D + 7).

• Calibration of temperature forecasts

To calibrate the day-ahead temperature forecasts at the eight time points $s_1,\dots ,s_8$ provided by TCWB before D-day (the first day that the temperature forecasts are to be calibrated), the errors between the historical apparent temperatures $T_a\left(t,s\right)$, and the recorded day-ahead temperature forecasts of the 28 days before D-day, ${\widehat{T}}_a\left(t,s\right), t=D-1,\dots ,D-28,$ $s=s_1,\dots ,s_8$, are used for the calibration. Define both the historical and the forecasted mean temperatures of the 28 days before D-day as $\overline{T}\left(D,s\right)\mathrm{and}\overline{\widehat{T}}\left(D,s\right)$ respectively, where $s=s_1,\dots ,s_8$:

$${\overline{T}}_a\left(D,s\right)=\frac{1}{28}{\displaystyle \sum }_{t=D-28}^{D-1}{T}_a\left(t,s\right)$$

(5)

$${\overline{\widehat{T}}}_a\left(D,s\right)=\frac{1}{28}{\displaystyle \sum }_{t=D-28}^{D-1}{\widehat{T}}_a\left(t,s\right)$$

(6)

For D-day, let the error between the mean temperatures and the calibrated temperature forecasts be, respectively:

$$T_e\left(D,s\right)={\overline{T}}_a\left(D,s\right)-{\overline{\widehat{T}}}_a\left(D,s\right)$$

(7)

$$T_c\left(D,s\right)={\widehat{T}}_a\left(D,s\right)+T_e\left(D,s\right)$$

(8)

The calibrations and (D + 1)-day’s eight temperature forecast time points can be found similarly. Note that samples from historical days with unusual temperature pattern are treated as outliers and thus deleted beforehand.

2.

Refined temperature forecasts

As we need to make load forecasts at 15-min intervals, we first interpolate the provided three-hour forecast data into a 15-min resolution, which will lead to smaller biases versus the real 15-min interval temperatures. We use the well-established Cubic Hermite interpolation method [20] and present that interpolation formula below.

Let the sequence ${\left\{u_i\right\}}_{i=1}^k$ be a partition $u_1<u_2<\cdots <u_k$ of the interval $\left[u_1,u_k\right]$, and let $\left\{T_i \right\},T_i=h\left(u_i\right)$, be the corresponding data points. The local grid spacing is $\Delta u_{\left(i+0.5\right)}=u_{\left(i+1\right)}-u_i$, and the slope of the piecewise linear interpolant between the data points is $S_{i+0.5}=\frac{\Delta h_{i+0.5}}{\Delta u_{i+0.5}}$.

The cubic Hermite interpolant polynomial defined for $u_i<x<u_{i+1}$ is:

$$H\left(u\right)=c_3{\left(u-u_i\right)}^3+c_2{\left(u-u_i\right)}^2+c_1\left(u-u_i\right)+c_0$$

(9)

where

$$c_0=T_i, c_1={\dot{h}}_i,$$

$$c_2=\frac{3S_{i+0.5}-{\dot{h}}_{i+1}-2{\dot{h}}_i}{\Delta u_{\left(i+0.5\right)}}, c_3=-\frac{2S_{i+0.5}-{\dot{h}}_{i+1}-{\dot{h}}_i}{\Delta h_{i+0.5}^2},$$

$${\dot{h}}_i=-\frac{\Delta u_{\left(i-0.5\right)}{S}_{i+0.5}+\Delta u_{\left(i+0.5\right)}{S}_{i-0.5}}{\Delta u_{\left(i+0.5\right)}-\Delta u_{\left(i-0.5\right)}},$$

Then the interpolant method produces as its output a sequence of temperature forecasts at 15-min intervals.

3.

Transformed temperature forecasts

It is noted that the effect of temperature to the load is nonlinear as shown in Figure 2 and upon examination it is observed that the load is approximately linearly related to the logistic sigmoid transformation of the temperature through

$$g\left(T\right)=\frac{1}{1+e^{-c_1\left(T-c_0\right)}}$$

(10)

where $c_0=30$ represents the location of the reflection from concave upward to concave downward,$c_1=0.85$ represents the scale parameter controlling slope changes.

2.6. Recurrent Neural Network with Selected Bases

RNN introduces loops in the network and allow internal connections among hidden units to enable exploration of the temporal relationships among the data [24]. The RNN structure with selected model bases taken from the resulting SPM described above is presented in the following.

• General Structure of RNN

Each of the RNN layers uses a loop to iterate over the time steps of the sequence. An RNN with a single hidden layer is illustrated below.

The input training data is the adaptive lasso estimator effects, it is given as:

$$\mathbf{x}=\left\{x^{\left(1\right)},x^{\left(2\right)},\dots ,x^{\left(t\right)},\dots ,x^{\left(T\right)}\right\},x^{\left(t\right)}={\left(x_1^{\left(t\right)},x_2^{\left(t\right)},\dots ,x_p^{\left(t\right)}\right)}^{⊺},t=1,\dots ,T.$$

(11)

The mapping of the output $o^{\left(t\right)}$ can be represented as:

$$o^{\left(t\right)}=\mathsf{\varphi }\left(u{x}^{\left(t\right)}+w{o}^{\left(t-1\right)}+b\right)$$

(12)

where $o^{\left(0\right)}=0$, t = 1,..., T, $\mathsf{\varphi }$ are the activation functions, u is the input weight for$x^{\left(t\right)}$, w is the input weight for $o^{\left(t-1\right)}$, where both are the same for all time points t, and where b is a parameter in the model representing the bias of the hidden layer and the output layer. The RNN uses the hidden state$o^{\left(\mathrm{t}\right)}$ at time step t to memorize the network. When t = 1, an RNN is the same as an ANN.

Consider a RNN model with k multilayer perceptrons, where in the ith hidden layer there are m neurons and the time step is T. Then the ith layer vector for these m neurons in the (i−1)th layer at time t is expressed as: $h_i^{\left(t\right)}={\left(h_{i,1}^{\left(t\right)},\dots ,h_{i,m}^{\left(t\right)}\right)}^{⊺},t=1,\dots ,T,$ which can be obtained through the following equation:

$$h_{\mathrm{i}}^{\left(\mathrm{t}\right)}=\mathsf{\varphi }\left(U_i{h}_{i-1}^{\left(t\right)}+W_i{h}_i^{\left(t-1\right)}+b_i\right),i=1,\dots ,k,$$

(13)

where $h_0^{\left(t\right)}=x^{\left(t\right)}$, $h_i^{\left(0\right)}=0$,$W_i \mathrm{and} U_i$ are the transition and input weight matrices. The mapping of the output $o^{\left(t\right)}$ can be represented as:

$$o^{\left(t\right)}=\mathsf{\varphi }\left(V{h}_k^{\left(t\right)}+b_{\mathrm{o}}\right), t=1,\dots ,T,$$

(14)

where V is output weight matrices and $b_0$ is a parameter in the model representing the bias of the hidden layer and the output layer.

With suitable choices for the parameters, such as the number of layers k, number of neurons in each layer m, and time steps of a sequence T, the RNN is expected to perform better than a more general model structure considering time effects in the neural network framework for STLF problems. We build an RNN model with k multilayer perceptrons in Python using the tensorflow library.

2.

Configuration Architecture

The RNN training process is heavily influenced by the choice of hyper parameters: sequence size, number of hidden layers and number of nodes per hidden layer. Efforts were made to search for a hyper parameter space to test different parameter combinations most suitable for TPC system. The experiment was conducted using standard RNN network to provide a best set of hyper-parameters. The results shown in

Table 1. Performance accuracy of various RNN architectures.

Time Steps	Units	Layers	(D + 1)-Day MAPE	(D + 1)-Day Max of APE at 95%	Time Steps	Units	Layers	(D + 1)-Day MAPE	(D + 1)-Day Max of APE at 95%
4	14	3	1.862	4.188	8	14	3	1.863	3.993
		4	1.876	4.383			4	1.872	4.162
		5	1.867	4.381			5	1.883	4.262
4	21	3	1.876	4.318	8	21	3	1.896	4.321
		4	1.906	4.271			3	1.863	3.993
		5	1.904	4.220			4	1.872	4.162
4	28	3	1.900	4.354	8	28	3	1.916	4.287
		4	1.882	4.336			4	1.908	4.405
		5	1.920	4.171			5	1.925	4.349

indicate that the best number of hyper-parameters units is 14, Layers is 3, and Time steps is 4.

Figure 3 shows the structure of the RNN used in this work, where the input $\{X^{\left(1\right)},\dots ,X^{\left(\mathrm{t}\right)}\}$ are values obtained from the selected model bases in the SPM model.

2.7. Real-Time Adapted Forecasting

The increasing use of renewable power sources has produced an increase in intermittency and a ramping in the net load profile that requires additional control efforts to maintain frequency quality. For a complete treatment for STLF, we also provide a real-time adaptive STLF procedure to help system operators with a detailed view into the real-time power system condition, so as to aid in their decision making. A quasi-real time RNN-based forecasting model (${\mathrm{RNN}}_{adp}$) with the objective of providing short-term load forecasts is described below.

• Load Forecasts Interpolation

In the first stage, the STLF results are interpolated to be a sequence with values in every 5-min period. The Cubic Hermite interpolation method is used to produce the load forecasts at 5-min intervals. This real time load data at 5-min intervals is then used in the second step to adaptively adjust the forecasting results.

2.

Adaptive Load Forecasting

The correction value is the average difference between the actual and the forecasted load values in the past 15-min interval. In other words, it is the average of the three differences of the actual and forecasted load values calculated at 5-min intervals.

3.

Exponentially Weighted Average

Finally, we use the EWMA to smooth the correction result. The exponential smoothing is given by the formula:

$${\tilde{y}}_i=0.8y_i+0.2{\tilde{y}}_{i-1}$$

(15)

where $y_i$ is the ith corrected value.

3. Test Results

TPC system load data from January 2012 to December 2019 are used for testing the proposed method. The load data used here is the net load (the power served by all generators minus the TPC’s pumped storage load). The days in each year are divided into two classes: general days and special days. Special days refer to exceptional days that have their own load patterns (e.g., holidays, days experiencing a typhoon, etc.). General days refer to either typical working days or weekends. The main goal of this study is to provide STLF method for general days.

3.1. Training Data Selection

For the future day loads to be predicted, the training samples are chosen from historical days with a similar load pattern. The input-target pairs are the historical temperature (predicted and actual) and load data recorded during the corresponding days in the previous 28 days (4 weeks), together with the 6 weeks around the same period of the previous year and the predicted temperatures of the future days from TCWB. To select a subset of model bases as predictors for estimating the future day loads, a training set that has 70 daily loads and temperature data corresponding to the time period shown in Figure 4 is used. The forecasting process begins every morning at 9:00 a.m. to forecast demand up to next 7 days with 15 min resolution. The test results obtained when applying the method to forecast the load in year 2018–2019 are presented. STLF performance indices, such as the mean absolute mean error (MAE), root mean square error (RMSE), absolute performance error (APE) and mean absolute percentage error (MAPE), are used to evaluate the forecasting accuracy of the model used [25].

3.2. Comparison of Test Results Obtained from the Semi-Parametric Model and the RNN Model

The MAE, RMSE and MAPE of the accuracies of the load forecasts for every month from 2018–2019 are provided. The forecasting accuracies for the two models based on the historical temperatures serve as a baseline for comparison. In

Table 2. Comparisons of the Model Performances of the next (D + 1)-day Forecasting Accuracies of the SPM and RNN Models with Historical Temperatures.

	SPM			RNN			Daily Load
Year	MAE	RMSE	MAPE	MAE	RMSE	MAPE	Average
2018	605.01	792.93	2.31	530.2	719.98	2.03	26,382.82
2019	466.39	632.22	1.76	448.99	601.02	1.7	26,441.99

the corresponding (D + 1)-day monthly MAE, RMSE and MAPE of the two models with historical temperatures are given in details. It can be seen that with the actual temperature, both models have good accuracies on the (D + 1)-day forecasts and the performances of the RNN model is especially outstanding on most of the months from 2018–2019 with annual averages of MAPE at 2.03, 1.70 respectively.

3.3. Forecasts with Temperature Calibration

The actual temperature data indicate that temperature forecasting biases increase rapidly with large values (around 3 degrees). Figure 5 presents the MAPE time plots of two STLF models using original and adjusted temperature forecasts as inputs. From Figure 5 and

Table 3. Performance Comparisons on mean absolute percentage errors (MAPEs) of semi-parametric (SPM) and RNN Models with the Historical, Forecasted and Calibrated Temperatures, for the next (D + 1)-day Load Patterns.

	SPM			RNN
Month	Historical	Forecasted	Calibrated	Historical	Forecasted	Calibrated
201801	2.6	2.64	2.6	1.62	1.43	1.41
201802	4.13	4.13	4.09	3.57	3.64	3.64
201803	2.52	2.16	2.25	1.88	1.71	1.68
201804	2.51	2.23	2.45	2.14	2.92	2.40
201805	3.58	3.32	3.77	2.32	2.68	2.41
201806	2.23	2.38	2.15	2.20	3.77	3.20
201807	1.43	2.00	1.87	1.49	2.95	2.64
201808	1.86	2.96	2.00	1.98	4.49	3.47
201809	2.5	2.85	2.33	2.89	3.87	2.57
201810	1.87	2.53	2.2	1.81	2.51	2.11
201811	1.57	1.54	1.7	1.62	2.01	1.75
201812	1.35	1.39	1.32	1.40	1.52	1.43
Average	2.32	2.48	2.37	2.03	2.74	2.34
201901	2.00	2.06	2.05	1.98	2.14	2.09
201902	1.83	1.84	1.87	1.88	2.05	1.97
201903	1.26	1.26	1.48	1.29	1.34	1.31
201904	2.10	2.52	2.63	1.94	2.6	2.38
201905	2.35	2.85	2.49	2.32	3.12	2.73
201906	2.02	2.98	3.12	1.57	3.02	2.57
201907	1.30	1.95	1.89	1.5	2.09	2.00
201908	1.91	2.55	2.6	1.9	2.87	2.80
201909	2.24	2.61	2.16	2.06	2.76	2.26
201910	1.21	1.77	1.88	1.32	2.04	1.72
201911	1.26	1.41	1.47	1.16	1.49	1.45
201912	1.71	1.73	1.73	1.56	1.64	1.60
Average	1.76	2.13	2.11	1.7	2.26	2.08

, it can be seen that, after calibrating the forecasted temperature through bias correction based on the previous 28 days’ temperature forecasting biases, the forecasting accuracies are significantly improved.

3.4. Real-Time Forecast Performance

The monthly performance comparisons for 2018 on the MAPE of the real-time D-day forecast for the next 6 h are given in

Table 4. Performance Evaluation on MAPE of the Real-Time D-day Forecast Over the Next 6 Hours.

Month	15 Min	30 Min	60 Min	120 Min	180 Min	360 Min
201801	0.488	0.49	0.494	0.502	0.509	0.529
201802	0.468	0.469	0.471	0.477	0.483	0.499
201803	0.481	0.482	0.485	0.495	0.506	0.528
201804	0.488	0.493	0.502	0.515	0.526	0.55
201805	0.47	0.475	0.483	0.496	0.506	0.554
201806	0.456	0.459	0.466	0.477	0.488	0.542
201807	0.434	0.438	0.445	0.457	0.465	0.496
201808	0.447	0.451	0.457	0.465	0.474	0.513
201809	0.485	0.486	0.492	0.503	0.513	0.557
201810	0.525	0.529	0.535	0.546	0.558	0.582
201811	0.473	0.475	0.478	0.484	0.493	0.515
201812	0.452	0.453	0.455	0.460	0.470	0.490
Average	0.473	0.475	0.481	0.490	0.500	0.530

. As the table shows, the annual averages of the MAPE for the RNN-based ${\mathrm{RNN}}_{\mathrm{adp}}$ model are below 1%.

Figure 6 shows the performance of the model for a typical day of the studied period, June 22. As can be seen, the real time load pattern and the forecasting load pattern of the RNN model have similar trend patterns, but the forecasted curve is much lower. With the adjusted model, ${\mathrm{RNN}}_{adp}$, however, the forecast accuracy improves significantly, obtaining an average error of 0.567% across the entire day.

3.5. Comparison of ANN, MIX, SPM and RNN Model Performance

In this sub-section, performance of different STLF methods are compared, including a two-stage Artificial Neural Network (ANN) model and an STLF model developed previously by TPC with special attention being given to the adjustments of the peak and nadir load forecasts [26], and a mixed model (MIX) with a weighted average of the ANN and against our basic RNN without Lasso variable selection or temperature calibration, where the weights are inversely proportional to the MAPEs of the previous day.

The performances of the next (D + 1)-day monthly MAPEs for these four forecasting models are presented in

Table 5. Performance Comparisons of the Monthly MAPE (Daily Load Average, Peak Load and Nadir Load) for the (D + 1)-day Forecasts with the Artificial Neural Network (ANN), mixed model (MIX), SPM and RNN Models.

	ANN			MIX			SPM			RNN
Month	Daily	Peak	Nadir	Daily	Peak	Nadir	Daily	Peak	Nadir	Daily	Peak	Nadir
201801	1.77	1.33	2.30	1.95	1.61	2.78	2.60	3.18	2.48	1.41	1.21	0.96
201802	2.19	2.63	2.21	2.38	2.82	2.43	4.09	4.51	3.18	3.64	3.81	2.94
201803	2.40	2.18	2.88	1.98	2.05	2.65	2.25	3.20	1.79	1.68	1.75	1.81
201804	2.78	3.01	2.59	2.12	2.33	2.21	2.45	2.93	2.23	2.40	2.31	2.33
201805	2.85	2.92	3.25	2.75	3.26	2.35	3.77	4.58	3.39	2.41	2.79	2.32
201806	7.25	6.48	8.29	7.59	6.60	9.02	2.15	2.30	2.87	3.20	3.04	2.96
201807	2.09	2.07	2.09	2.38	2.51	2.04	1.87	2.15	1.62	2.64	2.37	2.55
201808	1.89	1.91	1.68	2.13	2.59	1.78	2.00	2.05	1.80	3.47	3.30	3.73
201809	2.67	2.46	2.70	3.29	3.14	3.06	2.33	1.61	2.23	2.57	1.96	2.94
201810	1.94	2.02	1.68	1.31	1.41	1.26	2.20	1.96	1.94	2.11	1.81	2.28
201811	3.31	3.78	3.23	1.57	2.24	1.54	1.70	2.45	1.59	1.75	1.74	2.13
201812	1.49	1.47	1.16	1.25	1.19	0.79	1.32	1.06	1.17	1.43	1.24	1.60
Average	2.72	2.68	2.86	2.54	2.63	2.64	2.37	2.66	2.18	2.34	2.23	2.33

, for the year 2018. As the table shows, each model has its own advantages and disadvantages in daily load profile and max/min load forecasts. For example, the RNN model has the best overall yearly average performance for monthly MAPEs: 2.34 for the daily loads and 2.23 for daily peak loads. The SPM model, however, performs the best for nadir loads, with an average monthly MAPE of 2.18.

The RNN performance is the best in the spring seasons and fairly good in the winter, when daily load patterns are stable. The SPM performance is the best in the summer season when the weather varies more. The MIX model has its best accuracy in the winter season. The ANN does very well in February and August, when there is the most uncertainty in the load pattern. Both the ANN and MIX models have large biases in June, however, especially for days adjacent to special days.

In more closely examining the daily MAPEs of the four models, we find that there are only 2 days for which all four models have MAPEs greater than 4, thus failing to catch the real load patterns: the day before Chinese New Year’s Eve in February and the “nine in one” election day in November. Figure 7 shows the actual and (D + 1)-day forecasts for the four models made on the previous D-day of these two days. All four models have similar forecasts with large biases to the real load on these two days. This indicates that these two days should be considered as special days in the future, so as to avoid large biases being included in the training samples for future forecasts, thereby helping improving the biases, particularly in February, for the SPM and RNN models.

Figure 8 presents the boxplots of daily MAPEs in 2018, after deleting the two days mentioned above. The performances of SPM and RNN are shown to be generally more robust, with fewer extremely large MAPEs. Figure 9 provides two subfigures with the actual and forecasted loads in two of the days where both ANN and MIX have large biased forecasts.

One of those days is 13 June 2018, where the MIX and ANN have similar daily forecasts, while the SPM and RNN perform reasonably well. The other day is 11 June 2018, where the MIX model is slightly better than the ANN, with smaller biases.

Another indication that the four models complement each other well is that only about 5% of forecasting days have MAPEs greater than 2.5 for all four models. This 5% compares to the 18% of days where the MAPEs of both ANN and MIX are both greater than 2.5, and the 14% of days where this is true for both SPM and RNN–a reduction of more than 9% in these cases. Among those days where all four models had large errors, about two thirds had explainable unexpected circumstances that caused the forecasting errors, such as TPC executing electric demand bidding, extreme weather conditions, or special events.

A new model can, in fact, be created by using an optimal weighted average of the ANN, SPM and RNN forecasts as such a hybrid model might further reduce the forecasting errors with proper time-varying weightings. How to choose appropriate optimal weights is a topic worthy of further investigation.

3.6. Performances of the (D + 2) to (D + 7) Day Forecasting Accuracies of the RNN Models

Figure 10 presents the monthly averages of the (D + 2)-day to (D + 7)-day forecasting MAPEs of the RNN models with forecasting temperatures from 2018–2019. Note that the seasonal patterns appear in both years, and, as they are the (D + 2)-day to (D + 7)-day forecasts, the higher MAPEs due to longer-range forecasts are to be expected.

4. Conclusions

An STLF method using a semi-parametric model and RNN with selected bases is presented. This tool has been adopted by TPC Operation Department for daily operation purposes since 2019. Due to the weather characteristics in Taiwan, test results indicate that STLF is especially challenging at season transitions: from spring to summer and from summer to autumn. The main advantage of an RNN-based STLF proposed here is that with the calibrated forecasted temperatures and features extracted from the load series, through ensembles of B-spline and multi-resolution bases after statistical variable selection approach, it can avoid the overfitting problems in the deep learning stage and adapt to these weather changing patterns earlier than other methods. Noticeable improvements of the MAPEs in 2019 for the RNN model with calibrating temperatures as compared with other methods are observed. However, the intra-day load forecasts are sometimes far off due to unexpected meteorologic factors. The real-time adaptive load forecasts for the next one hour with every 5-min interval and helps the system operator to adjust the ancillary service requirement to meet the electricity demand changes. In [15,16], both have used LSTM as the deep learning methodology; we have also tried the LSTM model, where results for the STLF show that the improvements on the accuracies of the forecasts are limited and the model is more complicated and takes much more time to compute, which is not that feasible for daily use in practice. In [16], a similar-days selection procedure is adopted, which is worthy of more studies to see the advantages and shortcomings of this approach for our dataset with fast changing weather patterns. Besides the techniques presented, an optimal model averaging various load forecasting models is a topic for further investigation, as is how to extend the training samples to special day load forecasts.