Wind Power Forecasting Based on LSTM Improved by EMD-PCA-RF

Abstract

Improving the accuracy of wind power forecasting can guarantee the stable dispatch and safe operation of the grid system. Here, we propose an EMD-PCA-RF-LSTM wind power forecasting model to solve problems in traditional wind power forecasting such as incomplete consideration of influencing factors, inaccurate feature identification, and complex space–time relationships between variables. The proposed model incorporates Empirical Mode Decomposition (EMD), Principal Component Analysis (PCA), Random Forest (RF), and Long Short-Term Memory (LSTM) neural networks, And environmental factors are filtered by the Least Absolute Shrinkage and Selection Operator (LASSO) algorithm when pre-processing the data. First, the environmental factors are extended by the EMD algorithm to reduce the non-stationarity of the series. Second, the key influence series are extracted by the PCA algorithm in order to remove noisy information, which can seriously interfere with the data regression analysis. The data are then subjected to further feature extraction by calculating feature importance through the RF algorithm. Finally, the LSTM algorithm is used to perform dynamic time modeling of multivariate feature series for wind power forecasting. The above combined model is beneficial for analyzing the effects of different environmental factors on wind power and for obtaining more accurate prediction results. In a case study, the proposed combined forecasting model was verified using actual measured data from a power station. The results indicate that the proposed model provides the most accurate results when compared to benchmark models: MSE 7.26711 MW, RMSE 2.69576 MW, MAE 1.73981 MW, and adj-R² 0.9699203s.

1. Introduction

The issue of climate warming is currently a global concern. One of the major contributors to the problem is the use of fossil fuels. In recent years, countries around the world have been actively promoting a clean and low-carbon energy transition. Various countries have formed a new global energy pattern by formulating energy transition policies and by increasing investment in renewable energy and other programs. In this context, China has taken a series of measures to promote a low-carbon transition in its energy supply system. It is foreseeable that with the deepening of this energy transition, wind power will keep developing rapidly and become an important part of China’s new power system construction in the future. However, as wind power is affected by wind speed, temperature and other environmental factors and has high randomness and fluctuation, large-scale wind power installation presents great challenges with respect to the safe and stable operation of the power system. Thus, in order to improve wind power consumption capacity and realize the optimal dispatch of the power system, it is important to improve the accuracy of short-term wind power forecasting. The current wind power forecasting models are, on the one hand, constrained by the source data of wind farms, and tend to ignore the influence of various environmental factors on wind power, resulting in the multivariate series of environmental factors not being used effectively. On the other hand, due to the nonlinear variation of the wind power and multivariate environmental information series [1,2,3], the convergence of forecasting models gradually slows down, and problems with over-fitting problem can occur with increased in input variables.

The main current wind power forecasting methods are the physical, statistical, and machine learning methods. The physical method involves the construction of a physical model to forecast from meteorological data and the surface information around the electric field [4], represented by NWP (Numerical Weather Prediction). The main statistical methods are the time series method [5], gray forecasting method [6], autoregressive moving average method [7], and a few others which refer to the construction of a nonlinear relationship between historical data and wind power data to make forecasts. The main machine learning methods are neural networks [8], support vector machines [9], extreme learning machines [10], etc. Due to the ability of neural networks to mine the nonlinear relationships and deep features in training data, they generally have better forecasting performance [11,12], and are now widely used in forecasting for wind power generation [13]. Comparisons in the literature [14,15,16,17] between LSTM (Long Short-Term Memory neural network) and other forecasting models have shown LSTM models to be better than other forecasting models for both long-term and short-term forecasting; however, the LSTM model has the problems of model complexity and long training time. Thus, proper data processing can enhance the learning effect of the LSTM model. In [18], the authors proposed a PCA (Principal Component Analysis)-based LSTM forecasting model which was able to effectively reduce the dimensionality of the input variables of the LSTM. In [19], the authors proposed an LSTM wind power forecasting method based on RR (Robust Regression) and VMD (Variational Mode Decomposition) which improved the LSTM forecasting accuracy by decomposing the wind power series to eliminate the noise. In [20], the authors proposed a multiple variable LSTM algorithm and performed dimensionality reduction on the original data using the wavelet noise reduction method, then reconstructed pre-selected input data using chaos analysis and the classification forecast tree method, showing that dimensionality reduction can effectively improve LSTM forecasting accuracy. Both [21,22] improved the LSTM model by introducing an attention mechanism which was able to allocate limited computational resources to more important tasks while solving the information overload problem. In [23,24], the authors similarly introduced an attention mechanism into the CNN-LSTM forecasting model to improve forecasting accuracy. The above examples illustrate that improving the quality of input data is beneficial to the forecasting accuracy of the LSTM model.

To improve the accuracy of wind power forecasting, the data quality first needs to be improved from several perspectives, including data decomposition and dimensionality reduction. Wind power series and multivariate environmental factor series are often non-stationary [25], and forecasting the original series directly may result in large errors. Decomposing the signal of the original series can improve forecasting accuracy while reducing the complexity of the data [26,27]. The most commonly used signal decomposition methods are FFT (Fast Fourier Transform), WT (Wavelet Transform), and EMD (Empirical Mode Decomposition). In [28,29], the authors optimized the signal time domain waveform using FFT, although they did not consider the local features of the time frequency. In [30,31], the authors used Wavelet Transform to decompose the wind speed time series, which has the drawback of requiring manual setting of the parameters. Compared to the two methods mentioned above, EMD [32] can actively identify the features of the input data with self-adaptability and multi-resolution. In [33], the authors proposed an EMD-MODBN (Multi-objective Deep Belief Network) model, and their experimental results proved that decomposition of data using EMD methods is beneficial in helping neural networks to capture the potential connections between data. In [34], the authors decomposed the wind speed data series using EMD to remove the noise from the original data, leading to improved forecasting accuracy. In [35], the authors decomposed the residential load signal using EMD; their results proved that more data peaks correlates to better EMD decomposition. The above examples illustrate the superiority of the EMD algorithm in data decomposition. Thus, in this paper, we chose to decompose the environmental factor series using the EMD algorithm.

In addition to the accuracy and stationarity of the data, the dimensionality plays an important role in the forecasting accuracy of the model. Introducing a dimensionality reduction algorithm into the forecasting model can improve its calculation efficiency. PCA (Principal Component Analysis) is an effective method for reducing the dimensionality of data by analyzing the covariance structure of multivariate data series, calculating the contribution of each series, and selecting the primary series to be expressed in order to achieve the purpose of data dimensionality reduction [36,37,38]. In [39], the authors used PCA based on high-frequency data to forecast the high-dimensional covariance matrix, which can well characterize the long memory behavior of realized eigenvalue series and be easily estimated by OLS. In [40], the authors used the PCA-BP method to filter the model input data, eliminating redundant and irrelevant information, reducing the complexity of the model, and improving the forecasting performance of the subsequent model. In [41], the authors preprocessed the data using PCA in order to achieve dimensionality reduction, which reduced the computation time of the subsequent model. RF (Random Forest) is a supervised learning algorithm which represents an effective method of dimensionality reduction. In [42], the authors used the RF algorithm to rank the importance of individual features based on the Gini index for feature filtering, and experimental results showed that the random forest algorithm-based feature importance ranking was able to filter features and improve the accuracy of subsequent models. In [43], the authors compared a multiple linear regression model with a random forest model and found that the RF model was far superior to the multiple linear regression model in terms of both goodness-of-fit and other evaluation indicators. In [44], the authors preferentially selected features via RF algorithm, and the experimental results showed that random forest feature selection can effectively reduce data redundancy, eliminate features with poor differentiability, and improve the recognition effect of the model. In [45,46,47,48], the authors computed feature importance with RF algorithms in different fields, all of which improved the subsequent model effects. Thus, in this paper, we chose to reduce the dimensionality of the EMD decomposed series using a combined PCA-RF algorithm.

To date, data decomposition-based and data dimensionality reduction-based wind power forecasting has been studied by several scholars. In [49], the authors used complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) to divide the volatility into several fluctuation components with different frequency characteristics; their experimental results demonstrated the advantages of data decomposition. In [50], the authors used VMD to decompose wind speed into the nonlinear part, the linear part, and the noise part, and experiment results showed that the model proposed in their paper well and truly reflected the characteristics of the wind speed. In [51], the authors extracted principal components from high-dimensional raw data using PCA results as input variables for subsequent models; their experimental results demonstrated improvement in forecasting accuracy. In [52], the authors processed the input variables through VMD decomposition and PCA dimensionality reduction, and the experimental results showed that the proposed method had higher forecasting accuracy than the traditional methods. It can be seen from the above examples that both data decomposition and data dimensionality reduction are widely used in the field of wind power forecasting, although there are fewer studies that combine decomposition and dimensionality reduction.

In summary, to solve the problems of inaccurate feature identification and slow convergence in traditional wind power forecasting models, this paper makes full use of the environmental factor series that affect wind power, mines the features of wind power and environmental factors over time, and constructs an EMD-PCA-RF-LSTM forecasting model to improve the forecasting accuracy of wind power in order to provide certain technical support for the safe dispatch of power systems and enhance power system’s abilities with respect to the consumption of wind power. In this paper, the environmental factors are filtered by the LASSO (Least Absolute Shrinkage and Selection Operator) algorithm at the very beginning. Second, the environmental factor series are decomposed by the EMD algorithm to reduce non-stationarity. Third, the key influencing factor series are extracted and feature importance is calculated by the PCA-RF algorithm for further feature extraction from the data. Finally, dynamic time modeling of multivariate feature series is performed by the LSTM algorithm to forecast the wind power. A comparative analysis including a traditional BP neural network, SVM, EMD-PCA-RF-BP, EMD-PCA-RF-SVM and single LSTM, EMD-PCA-LSTM, and EMD-RF-LSTM showed that the combined forecasting model proposed in this paper had the highest accuracy.

The rest of the sections in this paper are arranged as follows: the relevant theories of each algorithm are described in Section 2; data preprocessing, and the design of the combined model, and the evaluation indicators are described in Section 3; in Section 4, the performance of proposed model is analyzed according to the results of five combined models and three benchmark models on a case study; finally, the conclusions of this study are presented in Section 5.

2. Methods

2.1. LASSO Algorithm

The LASSO algorithm, first proposed by Robert Tibshirani [53], is a regression analysis method that enables simultaneous variable selection and parameter estimation. The LASSO method is used to compress the regression coefficients by constructing a penalty function. Coefficients with small absolute values are compressed to zero, thus achieving the purpose of variable selection. The process is as follows.

A general linear regression model can be expressed by the following formula:

$$y_i=\alpha +{\displaystyle \sum _{j=1}^p{\beta }_j{x}_{ij}+{\epsilon }_i,}{\epsilon }_i\sim N\left(0,{\sigma }^2\right)$$

(1)

In Formula (1), $y_i$ is the dependent variable, $i=1,2,\cdots ,n$; $\alpha$ is a constant term; ${\beta }_1,{\beta }_{2,}\cdots ,{\beta }_p$ are the regression parameters; $x_{ij}$ is the independent variable that affects the dependent variable; and ${\epsilon }_i$ is a normal distribution with zero mean.

The estimator $\left(\widehat{\alpha },\widehat{\beta }\right)$ of LASSO can be expressed by the following formula:

$$\left(\widehat{\alpha },\widehat{\beta }\right)=\arg \min \left\{{\displaystyle \sum _{i=1}^n{\left(y_i-\alpha -{\displaystyle \sum _{j=1}^p{\beta }_j{x}_{ij}}\right)}^2}\right\}s.t.{\displaystyle \sum _{j=1}^p\left|{\beta }_j\right|\le t}$$

(2)

In Formula (2), $t$ is an adjustable parameter which satisfies $t\ge 0$ for all the estimated values of $t$ and $\alpha$ satisfying $\widehat{\alpha }=\overline{y}$. Without loss of generality, assume that $\overline{y}=0$; then, $\alpha =0$. Then, Formula (1) can be written as follows:

$$\widehat{\beta }=\arg \min \left\{{{\displaystyle \sum _{i=1}^n\left(y_i-{\displaystyle \sum _{j=1}^p{\beta }_j{x}_{ij}}\right)}}^2\right\}s.t.{\displaystyle \sum _{j=1}^p\left|{\beta }_j\right|\le t}$$

(3)

Let $t_0={\displaystyle \sum _{j=1}^p\left|{\widehat{\beta }}_j^0\right|}$, where ${\widehat{\beta }}_j^0$ is the least squares estimation of the regression coefficient. When $t<t_0$, the regression coefficients are compressed and certain of the regression coefficients converge to 0 or even equal 0. Then, Formula (3) can be written as follows:

$$\widehat{\beta }=\arg \min \left\{{\displaystyle \sum _{i=1}^n{\left(y_i-{\displaystyle \sum _{j=1}^p{\beta }_j{x}_{ij}}\right)}^2}+\lambda {\displaystyle \sum _{j=1}^p\left|{\beta }_j\right|}\right\}$$

(4)

where ${\displaystyle \sum _{i=1}^n{\left(y_i-{\displaystyle \sum _{j=1}^p{\beta }_j{x}_{ij}}\right)}^2}$ is the loss function reflecting the fitting effect of the model, $\lambda {\displaystyle \sum _{j=1}^p\left|{\beta }_j\right|}$ is the penalty function reflecting the model’s compression strength of the coefficients, and $\lambda$ is the penalty parameter subject to $\lambda \in \left[0,\infty \right)$, which determines the compression strength of the coefficient. As $\lambda$ increases, the coefficients of the variables in the model are gradually compressed, and when $\lambda$ reaches a certain value, the coefficients of certain variables are compressed to zero, achieving the purpose of variable selection.

In this paper, K-fold Cross-Validation is used to determine the penalty parameter, $\lambda$. The sample datasets are divided into $K$ subsets. One subset is used as the dataset for validating the model, while the other $K-1$ subsets are used to construct the model. The cross-validation repeats $K$ times, with each subset validated once. The process is as follows.

Step 1. Divide the sample datasets $T$ into $K$ subsets, $T=\left\{T_1,T_2,\cdots ,T_K\right\}$.

Step 2. Use one of the subsets as the validation set and the remaining $K-1$ subsets as the training set, that is, train $K$ times and validate $K$ times.

Step 3. For each penalty parameter $\lambda$, use the training set to find the estimator ${\widehat{\beta }}^{\left(t\right)}\left(\lambda \right)$ of $\beta$; the statistic for cross-validation is

$$CV\left(\lambda \right)={{\displaystyle \sum _{t=1}^K\left\{y_i-{\displaystyle \sum _{j=1}^p{\widehat{\beta }}^{\left(t\right)}\left(\lambda \right)x_{ij}}\right\}}}^2$$

(5)

Step 4. The corresponding penalty parameter is obtained by minimizing $CV\left(\lambda \right)$:

$${\widehat{\lambda }}_{cv}=\underset{\lambda >0}{\arg \min }CV\left(\lambda \right)$$

(6)

2.2. EMD Algorithm

In this paper, the EMD algorithm [34] is used to decompose the environmental factor series. The EMD algorithm can be used to obtain the local features of the environmental factor series that affect the wind power at different time scales, which results in a more detailed, though larger, number of data series.

EMD, as a data-driven adaptive nonlinear time-varying signal decomposition method, is based on Fourier analysis and Wavelet analysis, and is applicable for smoothing nonlinear non-stationary signals in a step-by-step process. The filtering process of the EMD algorithm decomposes complex time series data into a finite number of Intrinsic Mode Functions (IMFs); these decomposed IMFs contain the fluctuating information of the original data on different time scales [54]. The process is as follows.

Step 1. For any processed signal $x\left(t\right)$, determine its local maximum and minimum values and record the difference between the signal data, $x\left(t\right)$, and the mean value of the upper and lower envelope, $m_1\left(t\right)$, as $h_1\left(t\right)$, which can be expressed as follows:

$$h_1\left(t\right)=x\left(t\right)-m_1\left(t\right)$$

(7)

Step 2. Repeating the above process, the first-stage IMF $h_1\left(t\right)$ filtered from the original signal usually contains the highest frequency component of the signal. Separate $h_1\left(t\right)$ from $x\left(t\right)$ to obtain the difference signal, $r_1\left(t\right)$, with the high-frequency component removed, and repeat the above filtering steps with $r_1\left(t\right)$ as the new signal until the residual signal in the n-th stage is a monotonic function that no longer filters out the $r_1\left(t\right)$ of the IMF. The expression is as follows:

$$\{\begin{array}{l}{r}_1\left(t\right)=x\left(t\right)-h_1\left(t\right)\\ r_2\left(t\right)=r_1\left(t\right)-h_2\left(t\right)\\ ⋮\hspace{1em}\hspace{1em}\hspace{1em}⋮\\ r_n\left(t\right)=r_{n-1}\left(t\right)-h_n\left(t\right)\end{array}$$

(8)

According to the decomposition algorithm, $x\left(t\right)$ can be expressed as a sum of n IMFs and a single residual, with the following expression:

$$x(t)={\displaystyle \sum _{j=1}^n{h}_j\left(t\right)}+r_n\left(t\right)$$

(9)

In Formula (9), $r_n\left(t\right)$ is the residual, indicating the average trend in the signal, while $h_j\left(t\right)$ is the j-th IMF, j = 1, 2,…, n, which denote the different components of the signal from high to low frequencies, respectively.

The end of the filtering process is mainly based on a Cauchy-like convergence criterion [55], and the standard deviation (SD) is usually set from 0.2 to 0.3, with the following expression:

$$SD={\displaystyle \sum _{j=1}^n\frac{{\left|h_j\left(t\right)-h_{j-1}\left(t\right)\right|}^2}{{\left|h_{j-1}\left(t\right)\right|}^2}}$$

(10)

2.3. PCA-RF Combined Algorithm

In this paper, the PCA-RF combined algorithm is used to reduce the feature dimensions without losing the original data information. The results are then used as the input of the LSTM forecasting model in order to improve the calculation efficiency and accuracy of feature extraction.

2.3.1. PCA Algorithm

The data series obtained by EMD decomposition enriches the number of feature series, although the dimensionality of the input variables increases as well. PCA was first proposed by Karl Pearson in 1901, and has been widely used to reduce the number of feature vectors; thus, it can achieve the dimensionality reduction of data. It can be used to reduce the computation times of artificial neural networks as well as to increase their calculation speed. The process is as follows [56].

Assuming an original dataset $X=\left\{x_{11},x_{12},x_{ij},\cdots ,x_{mn}\right\}$ where $i$ is the time observation point and $j$ is the series of environmental factors constituting the dataset matrix:

$$X=\left[\begin{array}{llll}{x}_{11}& x_{12}& \cdots & x_{1n}\\ x_{21}& x_{22}& \cdots & x_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ x_{m1}& x_{m2}& \cdots & x_{mn}\end{array}\right]$$

(11)

Step 1. Normalize the data series to obtain the normalization matrix, $X^{\ast }$:

$$X^{\ast }=\frac{x-x_{\min }}{x_{\max }-x_{\min }}$$

(12)

Step 2. Use linear transformation to obtain the covariance matrix, $R$:

$$R=\frac{1}{n}{\left(X^{\ast }\right)}^T{X}^{\ast }$$

(13)

Step 3. Solve $\left|\lambda I-R\right|=0$ to obtain the feature matrix, $\lambda$; finally, calculate the accumulated contribution rate, ${\beta }_i$:

$${\beta }_i={\displaystyle \sum _{i=1}^k\frac{{\lambda }_i}{{\displaystyle \sum _{i=1}^n{\lambda }_i}}}$$

(14)

Usually, the first $k$ principal components, with an accumulated contribution of 75% to 95%, are able to contain most of the information that the $n$ original variables can provide.

2.3.2. RF Algorithm

Following PCA dimensionality reduction, the optimal feature series are selected via RF by evaluating the feature importance of the series. The RF algorithm, first proposed by Breiman in 2001 [57], is an integrated algorithm based on decision trees. The sampling method of RF sets about 1/3 of the sample as not selected, and this part of the sample becomes out-of-bag (OOB) data.

RF obtains the feature importance by perturbing the out-of-bag data that are not involved in decision tree training and calculating their classification accuracy difference. The process is as follows [44].

Step 1. RF carries out Bootstrap sampling by taking $K$ sample datasets to generate $K$ decision trees, each of which is generated independently.

Step 2. Let $k=1$ to train decision tree $T_k$. The training input is the $k$-th dataset; calculate the accuracy, $L_k$, for the $k$-th out-of-bag data.

Step 3. Rearrange the feature, $f$, in the out-of-bag dataset and calculate the accuracy, $L_k^f$.

Step 4. Repeat Steps 2 and 3 for all sample datasets $k=2,3,\cdots ,K$.

Step 5. Calculate the classification accuracy error after feature rearrangement, which can be expressed as

$$e_k^f=L_k-L_k^f,k=2,3,\cdots ,K$$

(15)

Step 6. From Formula (15), we can obtain the influence of features on the accuracy of the out-of-bag data:

$$e^f=\frac{1}{K}{\displaystyle \sum _{k=1}^K{e}_k^f}$$

(16)

where the variance of $e^f$ is as follows:

$$S^2=\frac{1}{K-1}{\displaystyle \sum _{k=1}^K{\left(e_k^f-e^f\right)}^2}$$

(17)

Step 7. From Formulas (16) and (17), the importance of a feature $f$ can be calculated.

$$f_{VI}=\frac{e^f}{S}$$

(18)

Step 8. From Formula (18), the $f_{VI}$ of all the features can be obtained.

To select the optimal feature subset, the feature subset is generated by removing one feature at a time from the sorted feature set, calculating the accuracy of the feature subset, and finally selecting the one with the highest accuracy as the most optimal feature set.

2.4. LSTM Algorithm

A Long Short-Term Memory (LSTM) network is an extension of a Recurrent Neural Network (RNN) [58]. The LSTM network contains one input layer, one output layer, and several hidden layers; its structure is shown in Figure 1. The key to the LSTM network is the cell state; $C_{t-1}$ and $C_t$ are the old state and new state of the cell, respectively. In an LSTM, the information about the cell state is censored by the gate structure, which allows for selective passage of information. It consists of a sigmoid neural network layer and a pairwise multiplication operation. The sigmoid is a nonlinear activation function and is contained in the gate structure. The gate structure output ranges from 0 to 1 and defines the degree of information passing through. The tanh layer in Figure 1 is an activation function that maps a real number input to the range [−1, 1].

The Cell of an LSTM includes an input gate, output gate, and forget gate, which is the key to controlling the information flow. In the following formulas, $i_t$, $o_t$, and $f_t$ denote the state values of the input gate, output gate, and forget gate, respectively [24].

Step 1. The sigmoid layer by the forget gate decides whether the information is forgotten from the old cell state, $C_{t-1}$, with the input, $x_t$, of the current layer and the output, $h_{t-1}$, of the previous layer; then, the current cell state output is as follows:

$$f_t=\sigma \left(W_1^f\cdot x_t+W_h^f\cdot h_{t-1}+b_f\right)$$

(19)

Step 2. In order to generate the information that needs to be updated and stored in the cell state, first, the input gate updates the information using the result, $i_t$, of the sigmoid layer. Then, the new candidate value, $C_t$, generated by the tanh layer is added to the cell state. $C_t$ is obtained from multiplying the old cell state by $f_t$ to forget the unwanted information and adding it to the new candidate information, $i_t\cdot {\tilde{C}}_t$:

$$i_t=\sigma \left(W_1^i\cdot x_t+W_h^i\cdot h_{t-1}+b_i\right)$$

(20)

$${\widehat{C}}_t=\tanh \left(W_1^C\cdot x_t+W_h^C\cdot h_{t-1}+b_C\right)$$

(21)

$$C_t=i_t\cdot {\tilde{C}}_t+f_t\cdot C_{t-1}$$

(22)

Step 3. To determine the information output by the output gate, the initial output is first obtained through the sigmoid layer, the value of the cell state is scaled between [−1, 1] using the tanh layer, and the output obtained by sigmoid is multiplied pair by pair to obtain the output, $h_t$:

$$o_t=\sigma \left(W_1^o\cdot x_t+W_h^o\cdot h_{t-1}+b_o\right)$$

(23)

$$h_t=o_t\times \tanh \left(C_t\right)$$

(24)

In Formulas (19)–(23), $W_1^i$, $W_1^f$, $W_1^o$, $W_1^C$ are the weight matrices communicating $x_t$ with the input gate, forget gate, output gate, and cell input, respectively; $W_h^i$, $W_h^f$, $W_h^o$, $W_h^C$ are the weight matrices connecting $h_{t-1}$ with the input gate, forget gate, output gate, and cell input, respectively; $b_i$, $b_f$, $b_o$, $b_C$ are the biases of the input gate, forget gate, output gate, and cell input, respectively; and $\sigma$ is the sigmoid activation function.

3. Data Pre-Processing and Model Design

3.1. Data Sources

The experimental sample dataset used in this paper consisted of time series data of wind power from 1 October 2019 to 30 October 2020 provided by a domestic power station. The series of environmental factors has six columns, including the weather forecast data (humidity, temperature, air pressure, air density) obtained by the corresponding environmental monitors and the actual data (wind direction, wind speed) measured by the wind tower in the actual field. The observation points are forecasted at 15 min intervals, which is in line with actual production operation changes. Due to observation errors and communication failures in the actual operation of the power station, the observation dataset needed to be cleaned; “bad data” with null values and abnormal values were eliminated on a per-day basis to obtain the experimental observation dataset. The observation time period was 24 h throughout the day, the observation interval was 15 min, and there were 96 data observation points per day. The variables included in each dataset were the 24-h weather forecast data, the field wind tower actual measurement data, and the actual wind power data, as shown in

Table 1. Data field names and explanations.

Data Field Name	Field Name Translation	Data Field Explanation
DATE	observation time point	From 0:00 on 1 October 2019 to 17:00 on 30 October 2020, observe every 15 min
humidity	humidity	Weather forecast humidity data (RH)
temp	temperature	Weather forecast temperature data (°C)
pressure	air pressure	Weather Forecast Air Pressure Data (KPa)
air_density	air density	Weather forecast air density data (kg/m3)
wind_dir	wind direction	Actual wind direction data from the wind measurement tower in the field (°)
wind_speed	wind speed	Actual wind speed data measured by wind measurement tower in the field (m/s)
power	wind power	Wind power output (MW)

. Finally, 35,586 sets of time section data were obtained. The original wind power time series and the environmental factors (in terms of wind speed) time series are shown in Figure 2 and Figure 3.

3.2. Environmental Factors Selection Based on LASSO

First, the input series of humidity, temperature, pressure, air_density, wind_dir, wind_speed, and power were normalized by their maximum–minimum values according to Formula (12). In this paper, the penalty parameter, $\lambda =0.0005$, in the LASSO method was determined by three-fold cross-validation, at which value LASSO kept five variables and filtered one variable with a coefficient of zero, as shown in Figure 4.

The exact variable coefficients are shown in

Table 2. Coefficients of variables at $\lambda =0.0005$.

Variable	Coefficient	Variable	Coefficient
wind_speed	1.614955	air_density	0
wind_dir	0.097354	humidity	−0.11514
pressure	0.006481	temp	−0.12377

.

The regression coefficient of the air_density variable was zero per the LASSO calculation, and was thus eliminated. From the perspective of variable coefficients, the wind speed coefficient reached 1.614955, which was highly positively correlated with the wind power. When the wind speed was less than the rated wind speed, the wind turbine output power was less than the rated power, and when the wind speed was greater than the rated wind speed, the wind turbine output power was stable at the rated power. Therefore, there was a strong correlation between wind speed and wind power. The coefficient of the wind direction was 0.097354, which was positively correlated with wind power. The wind direction represents the angle between the wind speed and the fan, which determines the effective wind speed acting vertically on the fan. The coefficient of air pressure was 0.006481, which was positively correlated with the output power. Low air pressure affects wind power converting efficiency, and may lead to a corona phenomenon on the wind turbine. The coefficient of humidity was −0.11514, which was negatively correlated with wind power. High humidity leads to corrosion of the external components of wind power generation equipment, such as the tower and blades, as well as to a reduction in the performance of various components inside the wind turbine. The coefficient of temperature was −0.12377, which was negatively correlated with wind power. High temperatures can lead to softening and oxidation of the parts and materials used in wind turbine equipment, resulting in reduced performance and lower wind power converting efficiency.

Through the above analysis, wind_speed, wind_dir, pressure, humidity, temperature, and power were taken as the input variables of the follow-up models for decomposition, dimensionality reduction, and forecasting in this paper.

3.3. Construction of Wind Power Forecasting Model

3.3.1. Wind Power Forecasting Model Based on EMD-PCA-RF-LSTM

In this paper, we constructed a wind power forecasting model based on EMD-PCA-RF-LSTM; in the actual forecasting process, the dataset that was applied for training the LSTM model was constructed first. The flowchart of the EMD-PCA-RF-LSTM-based wind power forecasting model is shown in Figure 5. Algorithm 1 is the step corresponding to the flowchart.

Algorithm 1: EMD-PCA-RF-LSTM-based wind power forecasting algorithm.

Input: DATE, humidity, temp, pressure, wind_dir, wind_speed, power

Output: Forecasting results, model evaluation indicators (MSE, RMSE, MAE, R2/adj-R2).

(1) Data cleaning: Cleaning the wind power series and environmental factor series data. The “bad data” due to observation errors, communication failures, etc. are eliminated on a per-day basis.  (2) Through the EMD algorithm, the environmental factor series are decomposed into intrinsic mode functions $\left\{IM{F}_1,IM{F}_2,\cdots ,IM{F}_m\right\}$ of different frequencies and the residual $r_n$, thus decomposing the different scale fluctuations or trends present in the original environmental signal level by level.  (3) The data series decomposed in Step 2 have their dimensionality reduced by PCA and the key series affecting the wind power are filtered out, with an accumulated contribution rate of 85% as the threshold to eliminate the redundancy and correlation of the different time series obtained by EMD decomposition.  (4) The key series obtained by dimensionality reduction in Step 3 are further subjected to feature extracted. A new feature set is obtained through the RF algorithm by eliminating a certain proportion of features based on feature importance.  (5) The dataset obtained in Step 4 and the wind power history data are normalized and transformed into an applicable dataset for LSTM network training. The dataset is then divided into a training set and a test set.  (6) The parameters of the LSTM model are initialized and the training set input into the LSTM model for training until achieving the target accuracy.  (7) Following model training, the training file is saved and input to the test set for testing.  (8) The forecasting results and model evaluation indicators (MSE, RMSE, MAE, R2/adj-R2) are output and the algorithm ends.

3.3.2. Model Evaluation Indicators

As expressed in Formulas (25) to (28), the mean absolute error ($MAE$), root mean square error ($RMSE$), R-Square ($R^2$), and adjusted R-Square ($adj-R^2$) were chosen to evaluate the accuracy of the proposed forecasting model. R-Square ($R^2$) is used to evaluate different models with same number of features, and $adj-R^2$ is used to evaluate different models with a different number of features. The smaller the $MAE$ and $RMSE$, and the closer the value of $R^2$/$adj-R^2$ is to 1, the higher the accuracy of the forecasting results:

$$MAE=\frac{1}{m}{\displaystyle \sum _{i=1}^m\left|\left(y_i-{\widehat{y}}_i\right)\right|}$$

(25)

$$RMSE=\sqrt{\frac{1}{m}{\displaystyle \sum _{i=1}^m{\left(y_i-{\widehat{y}}_i\right)}^2}}$$

(26)

$$R^2=1-\frac{{\displaystyle \sum _{i=1}^m{\left({\widehat{y}}_i-y_i\right)}^2}}{{\displaystyle \sum _{i=1}^m{\left({\overline{y}}_i-y_i\right)}^2}}$$

(27)

$$adj-R^2=1-\frac{\left(1-R^2\right)\left(m-1\right)}{m-p-1}$$

(28)

where $y_i$ is the true value of wind power, ${\widehat{y}}_i$ is the forecasted value of wind power, ${\overline{y}}_i$ is the mean value of wind power, $p$ is the number of features, and $m$ is the number of test sets.

4. Case Study

The experimental process described in this Section was performed with Python 3.7, AMD Ryzen 7 5800H with Radeon Graphics 3.20 GHz, a 64-bit operating system, and 16.00 GB of RAM. PyCharm Community Edition 2021.3.2 and IBM SPSS Statistics 25 were used for all experiments.

4.1. Decomposition by EMD

The environmental series in the experimental sample were non-stationary signals and had a certain randomness and sudden variability due to the influence of weather changes. The EMD algorithm was used to decompose the original environmental series data with a maximum iteration number of 1000 to obtain the IMFs and residual of each series, which could then be used to highlight the local features of the original environmental series.

Table 3. The number of IMFs and the residuals.

Variable	IMFs	Residuals
wind_speed	14	1
wind_dir	17	1
pressure	10	1
temp	11	1
humidity	14	1

shows the number of IMFs and the residuals obtained after EMD decomposition. The decomposition resulted in 66 IMFs and five residuals, for a total of 71 dimensional feature series as the new set of feature series.

The decomposition results of each series are shown in Figure 6.

4.2. Dimensionality Reduction by PCA

In order to reduce the redundancy and correlation of the feature series, the experiments were performed using IBM SPSS Statistics 25 to implement PCA and reduce the dimensionality of the previously obtained feature series. The IMFs were standardized and then input to PCA for a total of 71 dimensions. The calculated principal component eigenvalues and the accumulated contribution rates are shown in

Table 4. The principal component eigenvalues and their accumulated contribution rates.

Component	Initial Eigenvalue			Component	Initial Eigenvalue
	Total	PV 1	Accumulated %		Total	PV 1	Accumulated %
1	6.485	9.133	9.133	27	1.023	1.440	60.500
2	2.561	3.607	12.740	28	1.014	1.428	61.928
3	2.166	3.051	15.791	29	1.012	1.425	63.353
4	1.983	2.793	18.584	30	0.998	1.405	64.758
5	1.921	2.706	21.290	31	0.995	1.401	66.159
6	1.843	2.596	23.885	32	0.991	1.396	67.556
7	1.705	2.402	26.287	33	0.983	1.384	68.940
8	1.561	2.198	28.485	34	0.970	1.366	70.306
9	1.466	2.065	30.550	35	0.950	1.339	71.644
10	1.461	2.058	32.608	36	0.949	1.336	72.981
11	1.400	1.972	34.580	37	0.929	1.309	74.290
12	1.345	1.894	36.474	38	0.924	1.301	75.591
13	1.309	1.843	38.317	39	0.922	1.298	76.889
14	1.282	1.806	40.124	40	0.897	1.263	78.152
15	1.225	1.725	41.849	41	0.893	1.258	79.410
16	1.214	1.710	43.558	42	0.864	1.217	80.627
17	1.190	1.677	45.235	43	0.851	1.198	81.824
18	1.159	1.633	46.868	44	0.835	1.176	83.001
19	1.146	1.615	48.482	45	0.832	1.171	84.172
20	1.108	1.561	50.043	46	0.807	1.137	85.309
21	1.094	1.540	51.583	47	0.785	1.105	86.414
22	1.087	1.530	53.114	48	0.774	1.091	87.505
23	1.075	1.514	54.628	…	…	…	…
24	1.057	1.488	56.116	69	0.001	0.002	99.999
25	1.053	1.483	57.599	70	0.000	0.001	100.000
26	1.037	1.461	59.060	71	7.980 × 10−5	0.000	100.000

¹ PV means Percentage of Variance.

. As can be seen from the table, the accumulated contribution rate η of the first 46 feature series reaches 85%; the filtered principal components strongly represent the original feature series and have high comprehensive information capability. Thus, the first 46 feature series were selected to substitute for the original series.

4.3. Feature Extraction by RF

The remaining series following EMD decomposition and PCA dimensionality reduction were named “FAC1”, “FAC2”, …, “FAC45” and “FAC46” and used as the input features of the RF algorithm. The training set was selected as the first 70% of the data, the test set as the last 30%, and the number of decision trees was 1000. The results are shown in

Table 5. Importance value of each feature.

Variable	Importance	Variable	Importance
FAC7	0.032787	FAC17	0.022616
FAC1	0.028421	FAC21	0.022217
FAC5	0.026995	FAC13	0.022152
FAC2	0.025839	FAC23	0.022139
FAC6	0.024718	FAC15	0.022078
FAC41	0.024562	FAC43	0.021977
FAC22	0.023467	…	…
FAC8	0.023178	FAC29	0.019019
FAC14	0.022976	FAC30	0.018696
FAC19	0.022931	FAC31	0.01834

.

The intuitive view is shown in Figure 7.

Considering the feature importance ranking and the data dimension, we set three thresholds of importance, 0.025/0.024/0.023, and used the filtered variables as input variables for the follow-up LSTM model; and the model with the best effect was selected and saved.

4.4. Results and Comparison

4.4.1. Model Parameters

The LSTM model parameters are shown in

Table 6. The LSTM model parameters.

Parameter	Value of Parameter
train_size	0.9
epochs	100
batch_size	48
lr	0.0001
activation	relu
optimizer	adam
dense	1/2/3
train_size	0.9
epochs	100
batch_size	48

. For the LSTM, the first 90% of the input data were used as the training set and the last 10% as the test set. The determination of dense numbers is described in Section 4.4.2.

4.4.2. Determination of RF Threshold

The forecasting error evaluation results with the RF threshold set as importance = 0.025 are shown in

Table 7. Error evaluation of different dense numbers at importance = 0.025.

	MSE	RMSE	MAE	R2
dense = 1	11.33006	3.36602	2.43616	0.953208
dense = 2	7.30723	2.70319	1.70896	0.969822
dense = 3	7.28741	2.69952	1.73744	0.969904

.

Table 8. Error evaluation of different dense numbers at importance = 0.024.

	MSE	RMSE	MAE	R2
dense = 1	8.64442	2.94014	2.15888	0.964299777
dense = 2	7.35007	2.7111	1.72885	0.969645278
dense = 3	7.69203	2.77345	1.83434	0.968233029

shows the error with the RF threshold set as importance = 0.024.

Table 9. Error evaluation of different dense numbers at importance = 0.023.

	MSE	RMSE	MAE	R2
dense = 1	8.4885	2.9135	2.0978	0.964943737
dense = 2	7.64559	2.76507	1.8047	0.968424821
dense = 3	7.26711	2.69576	1.73981	0.969987898

shows the error with the RF threshold set as importance = 0.023:

According to Table 7, Table 8 and Table 9, the forecasting effect is optimal at importance = 0.023 and dense = 3, which results in the lowest MSE and RMSE and the highest R². Thus, the RF threshold was set to 0.023 and the dense parameter of the LSTM model was set to 3.

4.4.3. Analysis of Experimental Results

In order to verify the superiority of the EMD-PCA-RF-LSTM wind power forecasting model proposed in this paper, the effectiveness of EMD decomposition, and the necessity of PCA-RF dimensionality reduction, as well as to show that the combined forecasting model proposed in this paper can significantly enhance and improve the accuracy of wind power forecasting, three forecasting models, namely, LSTM, EMD-PCA-LSTM and EMD-RF-LSTM, were used for forecasting. In addition, the model proposed in this paper was compared with two classical time series forecasting models, namely, a BP (back propagation) neural network and an SVM (Support Vector Machine), while using EMD-PCA-RF for data decomposition and dimensionality reduction. The comparison of the models is shown in Figure 8.

The absolute error values of the different models are shown in Figure 9.

The error evaluation results of the models are shown in

Table 10. The error evaluation results of the models.

	MSE	RMSE	MAE	adj-R2	The Percentage Enhancement of R2
SVM	119.6383	10.93793	8.37387	0.5052151	/
BP	74.04614	8.60501	6.512	0.6937693	/
LSTM	12.42231	3.52453	2.6655	0.9486257	/
EMD-PCA-RF-SVM	68.24442	8.26102	5.02623	0.7175243	42.02% (Compared to SVM)
EMD-PCA-RF-BP	35.44227	5.95334	3.73575	0.8532989	22.99% (Compared to BP)
EMD-RF-LSTM	8.18422	2.86081	1.87817	0.9655112	1.77% (Compared to LSTM)
EMD-PCA-LSTM	9.56465	3.09268	2.20952	0.9599812	1.19% (Compared to LSTM)
EMD-PCA-RF-LSTM	7.26711	2.69576	1.73981	0.9699203	2.24% (Compared to LSTM)

.

According to Figure 6 and Figure 7, the combined EMD-PCA-RF-LSTM forecasting model outperforms the other models and has the smallest error compared to the actual wind power historical data.

According to Table 10, using SVM and BP alone is less effective, with R² of 0.5052151 and 0.6937693, respectively. Compared with SVM and BP, the R² of LSTM improved by 0.443411 and 0.254856, respectively, reaching 0.9486257. This is because LSTM has a long-time memory function, which improves the gradient disappearance problem, and has more nonlinear and generalization ability than BP.

After the data were decomposed and dimensionality was reduced by EMD-PCA-RF, the R² of SVM and BP improved by 0.212309 and 0.15953, and the improvement rates were 42.02% and 22.99%, respectively, which indicates that EMD-PCA-RF can process the data effectively and enhance the comprehensive information capability of the input variables while reducing the dimensionality.

Compared with LSTM, the R² of EMD-RF-LSTM reached 0.9655112, and that of EMD-PCA-LSTM reached 0.9599812. Both improved in R² compared to the benchmark model LSTM. The R² of EMD-PCA-RF-LSTM was the highest at 0.9699203, and the improvement rate was 2.24%.

In comparing EMD-RF-LSTM, EMD-PCA-LSTM and EMD-PCA-RF-LSTM, it can be seen that EMD-PCA-RF-LSTM has the best forecasting effect, illustrating the superiority of the combined forecasting model.

5. Conclusions

In this paper, we have proposed a combined EMD-PCA-RF-LSTM forecasting model for addressing with the fluctuation and randomness of wind power which fully considers the five main environmental factors that affect wind power generation, namely, temperature, air pressure, humidity, wind direction, and wind speed. Our main conclusions are as follows.

The environmental factors were filtered using the Least Absolute Shrinkage and Selection Operator (LASSO) method; variables with regression coefficients of 0 were eliminated in order to achieve variable selection. The time series of five columns of environmental factors were decomposed using the Empirical Modal Decomposition (EMD) method to obtain IMFs, and the key influencing series affecting wind power were filtered out by Principal Component Analysis (PCA) to reduce the dimensionality. The key influencing factor series after PCA dimensionality reduction were subjected to further feature extraction by Random Forest (RF), a certain percentage of features were eliminated according to the feature importance ranking, and a new feature set was ultimately obtained as the input variable of the LSTM model. Finally, the non-linear relationship between environmental factor time series and wind power series was modeled dynamically by a long and short-term memory (LSTM) neural network to construct a forecasting model to forecast wind power.

By comparing SVM, BP, LSTM, EMD-PCA-RF-SVM, EMD-PCA-RF-BP, EMD-RF-LSTM, EMD-PCA-LSTM, and EMD-PCA-RF-LSTM, it was determined that the combined forecasting model proposed in this paper works optimally. With the combined EMD-PCA-RF decomposition reduction method, the MSE, RMSE, and MAE of SVM, BP, and LSTM all decreased, the adj-R² of SVM improved by 42.02%, the adj-R² of BP improved by 22.99%, and the adj-R² of LSTM improved by 2.24% to 0.9699203, achieving the best results among all of the models.

This paper verifies the practicality of the EMD-PCA-RF-LSTM model in the field of wind power forecasting and extends the application scope of deep learning techniques. The proposed forecasting method provides a new perspective for in-depth discussion of the economic operation and scheduling of wind power grid-connected systems which has good application prospects and engineering application value in reality. Related research is already in progress.