A Hybrid Algorithm for Short-Term Wind Power Prediction

Abstract

Accurate and effective wind power prediction plays an important role in wind power generation, distribution, and management. Inthis paper, a hybrid algorithm based on gradient descent and meta-heuristic optimization is designed to improve the accuracy of prediction and reduce the computational burden. The hybrid algorithm includes three steps: in the first step, we use the gradient descent algorithm to get the initial parameters. Secondly, we input the initial parameters into the meta-heuristic optimization algorithm to search for the “best parameters” (high-quality inferior solutions). Finally, we input optimized parameters into the RMSProp optimization algorithm and conduct gradient descent again to find a better solution. We used 2021 wind power data from Guangxi, China for the experiment. The results show that the hybrid prediction algorithm has better performance than the traditional Back Propagation (BP) in accuracy, stability, and efficiency.

1. Introduction

In the 21st century, with the rapid economic development, the world’s demand for energy is growing. As traditional energy sources can no longer satisfy the needs of green economic development, a consensus among countries has emerged on how to reduce carbon emissions and develop renewable energy. Among renewable energy, wind power is widely concerned and rapidly developing due to its advantages of non-pollution, renewable and relatively mature utilization technology. Wind power has evolved into the most mature power generation technology among all renewable energy sources as time has passed [1]. According to the Global Wind Energy Council (GWEC), 2020 is a record year for the global wind power industry, with a global installed capacity of 93 GW, a year-on-year increase of 53%. China and the United States are already the two largest wind energy markets in the world [2]. However, wind energy also faces unavoidable problems. The inherent randomness, volatility, and intermittent power generation may seriously impact the grid system. Wind power prediction is important to reduce grid dispatching costs and improve system performance. Wind power prediction is divided into three classes based on the time scale: long-term, medium-term, and short-term. Long-term forecasting is for wind farm planning and annual power generation plans, and medium-term forecasting is utilized to make a weekly or monthly maintenance plan. These two prediction algorithms have less accuracy, and less study has been done in this field [3]. On the other hand, short-term forecasts are for the next few minutes to a few hours. They can aid in frequency regulation, rotating reserve capacity, and cost-effective load dispatch, making them a popular topic in the wind energy business. As a result, the majority of recent research has concentrated on predicting short time series of wind power generation.

The mainstream wind power forecasting methods nowadays can be divided into four categories: physical analysis method, statistical analysis method, machine learning methods, and the combined methods [4,5,6,7]. Among them, the physical analysis method is based on numerical weather prediction (NWP) and geographic terrain information to obtain current wind speed and wind direction data at the hub height of the wind turbine, and then calculate the output power based on the wind turbine power feature curve [8] to establish the wind power forecasting model. This method has the advantage of not relying on historical data, but the complex calculation model leads to its poor performance in real-time or short-term tasks [9,10,11], and the numerical weather forecast accuracy and update frequency can affect the forecast accuracy and speed [12].

In contrast to physical analysis, the statistical method only requires past wind speed data. The wind power prediction model is established using pattern recognition and parameter estimation. Many wind speed prediction models and wind power prediction models have been proposed using statistical methods, such as the moving average model [13], Markov model [14], and Bayesian model [15] et al. In conclusion, short-term wind power forecasting can be accomplished using statistical methods that can develop an ideal link between predicted wind power and historical data. However, it has to require a large number of historical samples with the same trend to construct an accurate model [16].

The statistical analysis method, which uses historical wind speed data as a single input, ignores the strong nonlinearity of wind power data, while the machine learning method considers the relationship between the output and multiple variables. The overall trend also shows that the machine learning method is more accurate [17]. Under the condition of sufficient historical samples, it usually has high prediction accuracy and strong generalization ability. Moreover, machine learning method has a lot of different machine learning models, including ANN model [18], support vector machine (SVM) model [19], BPNN model [20], and extreme learning machines (ELM) model [21], et al. They have been utilized for wind power prediction or wind speed prediction. Among them, the results of SVM and neural networks have been satisfactory in terms of prediction [22]. Ning and Liu utilized the particle swarm algorithm to optimize the penalty factor and kernel parameters of SVM and then used historical data as training samples to train the optimized SVM model [23]. On the other hand Liu and Wang proposed a short-term forecasting model of BPNN with the combined loss function for wind power generation [24]. Kassa et al. [25] proposed a hybrid GA-BP neural network wind power generation prediction method. However, these popular methods also have some problems. Firstly, the training time of SVM is too long, and it is very difficult to realize large-scale training samples; Secondly, BPNN is a gradient attenuation (GD) optimization algorithm [26]. Although the convergence speed is fast, the disadvantage of using the GD algorithm is that they converge easily to the local minimum value and it is sensitive to the initial value of parameters. Because the problem is not a convex programming problem, the local minimum is not the global optimal solution [27]. Instead of using the BP approach to calculate the neural network parameters, we may utilize a meta-heuristic optimization algorithm to solve this problem. Meta-heuristic algorithm does not ensure that the best solution can be found, as discussed in [28], they can, nevertheless, come up with high-quality inferior solutions. The proposed SFLA-ANN in [29] is found to be more accurate than the traditional BPNN and GA-ANN, but SFLA also has its unavoidable problem, SFLA will bring a lot of computational burdens. Therefore, in [30], BP will be used to find the initial parameters of SFLA, then through optimization to obtain better parameters, these parameters have faster convergence and higher accuracy than SFLA-ANN. However, the BP-SFLA-ANN algorithm [30] only discusses the possibility of prediction in short-term solar energy, and it ignores the crucial point that after the sub-optimal parameters obtained by SFLA, further optimization can still be sought by gradient descent. In the theory we have just described, BP enables the algorithm to find an optimum faster, but it gets stuck in a local optimum, whereas SFLA enables the algorithm to provide a certain amount of randomness that allows it to jump out of the local optimum. Can we assume that after SFLA we perform a fast gradient descent optimization algorithm to get better parameters?

In this paper, to resolve these problems, we need to reasonably use SFLA and BP. Therefore a new hybrid algorithm model (i.e., BP-SFLA-RMSprop-ANN) is proposed to achieve better performance, and we will discuss the reliability of BP-SFLA-RMSprop-ANN and whether it can be applied in short-term wind power prediction. The remainder of this paper is as follows: The proposed forecasting algorithm theory and its implementation are in Section 2. The simulation results and discussion are in Section 3. Finally, in Section 4, the conclusions are presented.

2. Algorithm Theories

2.1. Shuffled Frog Leaping Algorithm

SFLA is a meta-heuristic search algorithm that simulates the memetic evolution of a group of hungry frogs. The algorithm’s concept is based on the behavior of a group of frogs who live in a swamp with numerous stones. These frogs’ ultimate goal is to discover the place with the most food. To find and arrive at this, the frogs communicate with each other while searching, allowing them to enhance each other’s places in the swamp. We can find the SFLA combines two completely different logics, “certainty” and “randomness” to simulate communication and realize SFLA. Through the deterministic strategy, frogs only care about their local improvement. In the random strategy, all frogs in the swamp complete the global information exchange and use a flexible random model to avoid converging to the local minimum. The algorithm is divided into three primary steps, which are as follows:

• Classification process: In this step, we want to obtain different class frogs (as frogs in the same class, we will use them for local search in the next step). To this end, frogs should be sorted according to their fitness values (training error), and then the sorted first set member, that is, the best frog in the swamp should be placed in the first memeplex, the second frog in the second memeplex, the M frog in the M memeplex (M = the numbers of memeplexes), the (M + 1) frog in the first memeplex, and so on.
• Local search process: In this step, the worst frog in each memeplex communicates with the best frog in each memeplex and adjusts the frog’s leaping step and direction to increase its fitness value. During each iteration, this step will be repeated some times (epochs). The equations explaining this step are expressed in (1) and (2):

  $$D=rand\left(\right)·\left(X_{\mathrm{b}}-X_{\mathrm{w},\mathrm{old}}\right)$$

  (1)

  $$\begin{array}{c}{X}_{\mathrm{w},\mathrm{new}}=X_{\mathrm{w},\mathrm{old}}+D;\hfill \\ -D_{max}\le D\le D_{max}\hfill \end{array}$$

  (2)

  where $X_{\mathrm{b}}$, $X_{\mathrm{w},\mathrm{old}}$, and $X_{\mathrm{w},\mathrm{new}}$ respectively represent the location of the local best frog in each class, the current worst frog in each class, and the generated new frog. D and $D_{\max }$ represent the transition step size and the maximum transition step size, respectively. Rand () represents a positive random value between 0 and 1. It will take the place of the old frog if it is more fit than previously. Otherwise, this operation will be repeated for this frog. The difference is that this time, the global best frog ($X_{\mathrm{g},\mathrm{b}}$) will be compared instead of the best local ($X_{\mathrm{b}}$). Equation (3):

  $$D=rand\left(\right)·\left(X_{\mathrm{g},\mathrm{b}}-X_{\mathrm{w},\mathrm{old}}\right)$$

  (3)
  If this strategy still does not produce better fitness values for frogs, then we will make the frog (the local worst frog) jump randomly once in the solution space. In each epoch, each frog jumps at least once and at most three times in each generation. After the epoch is completed, it is necessary to calculate the fitness value of local frogs (i.e., frogs in the same memeplex) to determine the current local best and worst frogs in the memeplex.
• Shuffling process: After each repetition, each iteration, all frogs are integrated together.

These three operations will be repeated as different iterations of SFLA until they meet the requirements and stop running. Figure 1 shows the flowchart of SFLA. More details about the concept of the hybrid algorithm and its implementation will be described in detail later.

2.2. BP-SFLA-RMSprop-ANN Model

The logic of the hybrid prediction model proposed in this paper adopts meta-heuristic algorithm (SFLA) to adjust the parameters of typical feedforward ANN. While using the SFLA, we do not give up using the classical BP algorithm. This algorithm first provides high-quality initial parameters for the SFLA and prevents the meta-heuristic algorithm from starting the search process from too random areas (that is, the advantages of this algorithm compared with the model proposed in [29]), and in [30], the parameters optimized by SFLA are directly input into ANN to start predicting. However, this model (i.e., BP-SFLA-ANN) ignores the drawback of the “randomness” strategy in SFLA. When the number of SFLA’s iterations reaches a certain number, subsequent iterations will be hard to effectively optimize the parameters. As shown in Figure 2, the number of iterations of SFLA reaches more than 50, its training error will not decrease. So in the last step, the parameters obtained from SFLA should be optimized by RMSprop algorithm again (RMSprop is more purposeful than SFLA and faster than classic BP), Figure 3 well demonstrates that using RMSprop for parameter optimization in the second half can make the training error show a fluctuating downward trend. The minimum value in this trend is the parameter we need. Figure 4 shows a structure diagram of the model. Firstly, the weights and biases in BPNN are used to form the initial frog population. The number of frogs, memeplexes, iterations, epochs (local search iterations), and the maximum jumping distance of frog individuals are set; Secondly optimization is carried out until the defined convergence conditions are met. Finally, the parameters obtained by SFLA are optimized by RMSprop. The new algorithm is verified by combined with training and test samples. The specific steps are as follows:

• BPNN initialization. Set the structure of the BPNN, design the number of neurons in each layer, and initialize the weights and biases.
• SFLA parameter initialization. We set the number of frogs, memeplexes, iterations, and epochs in the algorithm.
• Calculate the fitness value. The initial parameters obtained by the BP algorithm are used as the initial frog population.
• Perform the classification step in the SFLA mentioned above.
• Perform the local search step in the SFLA mentioned above.
• Perform the shuffling step in the SFLA mentioned above.
• If the maximum number of iterations is satisfied, the optimal solution is output, otherwise, return to step 4.
• Optimized again. The optimal parameters input into RMSprop and they will be optimized again.
• Verify the algorithm. After the training, diagnose the test samples, analyze the results and verify them.

2.3. Overfitting

In the training process of neural networks, we need to solve the overfitting problem. ANN is based on its prediction performance to change its prediction efficiency, and its training goal is to minimize training error. Therefore, if the training process does not stop at the correct point, it will start to “remember” the training samples rather than “summarize” the training samples. In this paper, cross validation [31] will be used to solve this problem. The data set is divided into three subsets: training set, test set, and validation set. While training with available training samples, ANN compares the error of the training set with the error of the validation set after each iteration. If the validation error continues to increase by a certain number (the threshold is set as five times in this paper), the training process will stop immediately, and the solution with “minimum validation error” will be used to predict.

3. Case Study

3.1. Wind Speed Data Description

The wind power data sets are provided by China Southern Power Grid. The data sets record wind speed (m/s), wind direction (°), temperature (°C), humidity (RH), air pressure (HPA) and active power (MW) of the wind turbine in the past year. Each measurement interval is 15 min, and there are 52,429 sets of valid data in these wind farms. The data of the first five groups are shown in

Table 1. The dataset’s first five groups of data.

Sample	Wind Speed (m/s)	Wind Direction (°)	Temperature (°C)	Humidity (RH)	Air Pressure (HPA)	Active Power (MW)
1	5.35	41.52	16.51	85.41	964.94	20.36
2	5.06	41.44	16.60	84.58	965.19	20.36
3	4.82	39.80	16.97	82.49	965.36	16.28
4	5.01	25.67	16.50	84.01	966.25	16.88
5	4.15	56.50	18.64	85.00	965.39	9.31

, and Figure 5 shows the correlation between the individual data features, we should ensure that the absolute value of the correlation of the data features for model training must be higher than the threshold (i.e., it is set 0.1 in this paper).

Although the presented model is forecasting the short-term wind power data (i.e., 5-min, 10-min, 15-min, …, 25-min, and 30-min), the optimization-prediction process will be repeated dozens of times to cover various types of changes in wind power output in a day or a week. The daily data must be sufficiently detailed to ensure model training, testing, and validation, Figure 6 depicts the someday wind power trend.

3.2. Data Process

Wind speed, wind direction, temperature, humidity and air pressure are used as the features, and wind power is used as output data. Firstly, we need to eliminate outliers in the data set. Secondly, in order to solve the problem of overfitting, through cross validation, the data set is divided into three parts: training set, test set and validation set to realize model construction, performance evaluation and parameter optimization respectively, with a ratio of 5:2.5:2.5. We employ min-max normalization to handle the data in order to increase prediction accuracy and speed up model optimization. The normalization function is to scale the data to [0, 1], the formula is:

$$x_k^{\prime }=\frac{x_k-x_{min}}{x_{max}-x_{min}}$$

(4)

3.3. Simulation Results

We study the SFLA to optimize the weights and biases of BPNN. We set the number of frogs, memeplexes, iterations and epochs in the SFLA to 30, 3, 100 and 10 respectively. Then we use mean squared error (MSE), mean absolute error (MAE), mean absolute percentage error (MAPE) and $R^2$ as evaluation criteria. The calculation formulas are:

$$\mathrm{MSE}=\frac{1}{N}\sum _{i=1}^N{\left({\mathrm{pretict}}_i-{\mathrm{test}}_i\right)}^2$$

(5)

$$\mathrm{MAE}=\frac{1}{N}\sum _{i=1}^N\mid {\mathrm{predict}}_i-{\mathrm{test}}_i\mid$$

(6)

$$\mathrm{MAPE}=\frac{100\%}{n}\sum _{i=1}^n\left|\frac{{\mathrm{test}}_i-{\mathrm{predict}}_i}{{\mathrm{test}}_i}\right|$$

(7)

$$R^2=1-\frac{{\sum }_i{\left({\mathrm{test}}_i-{\mathrm{predict}}_i\right)}^2}{{\sum }_i{\left({\overline{\mathrm{test}}}_i-{\mathrm{test}}_i\right)}^2}$$

(8)

We will acquire 30 training errors after each training process. The best training error of each iteration is the output value. Figure 7a illustrates the downward trend of the best training error. In order to clarify the continuous reduction of the loss function in the presentation optimization process, we describe three different intervals during 0–100 iterations. b, c, and d in Figure 7 describe the decreasing trend of training error of 0–30 iterations, 30–60 iterations, and 60–100 iterations respectively.

To avoid overfitting, the threshold is set to five times. If the validation error continues to rise for five times, the training process will stop. In Figure 8, we can observe this problem occurs when iteration is #60, this is why the training does not reach its maximum number of iterations (i.e., Iteration #100) and stops right at the #65.

Despite the proposed algorithm focusing on short-term wind power forecasting, the optimization-forecasting process will be performed six times per week to account for different oscillations. Figure 9 shows the results of comparing the predicted wind power data with the actual data for the test week of 1 July to 7 July.

In the comparison experiments, we selected two similar models (i.e., BP-ANN, BP-SFLA-ANN) and two classical models (i.e., LSTM [32], GRU [33]) to compare with the proposed model. All models are in the same circumstances, with the same number of iterations, the same learning rate, and the same division of data to evaluate the efficiency of the proposed models. Furthermore, we use seconds to unify their running time units for easy comparison.

Table 2. The comparison of similar algorithms.

Prediction Models	Performance Metrics
	MSE	MAE	MAPE	${\mathit{R}}^2$	Run Time (s)
Hybrid Algorithm(BP-SFLA-RMSprop-ANN)	0.3754	0.4083	18.41%	0.9539	107
BP-ANN	0.8361	0.7536	32.63%	0.9106	87
BP-SFLA-ANN	0.5276	0.5079	22.13%	0.9374	136
LSTM	1.1708	0.7854	38.86%	0.8834	206
GRU	1.5904	0.9216	46.75%	0.8547	146

presents the performance and time-consuming comparison among the proposed model, BP-ANN, BP-SFLA-ANN, LSTM, and GRU. As shown in the Table, the proposed hybrid method delivers the best accurate prediction and the second shortest running time. With the same number of iterations, the RMSprop algorithm can effectively improve the accuracy and reduce the high running time brought by the shuffled frog leaping algorithm. It successfully proves that the theory is feasible.

4. Conclusions

Accurate, efficient, and reliable wind power prediction can greatly help power plants to carry out distribution management. The paper innovatively proposes a hybrid algorithm-based meta-heuristic optimization method and gradient descent algorithm. In this paper, the BPNN is firstly used to determine the initial parameters of SFLA. Then the SFLA is used to optimize its initial parameters, which are finally fed back to the RMSprop for further convergence. The algorithm has three advantages: 1. High-quality random initial parameters are obtained quickly from the BP algorithm. 2. Combines the strategies of randomness and determinism from the frog-hopping algorithm. 3. Go through RMSprop for a fast descent gradient. The proposed BP-SFLA-RMSprop-ANN in this paper predicts short-term wind power in the Guangxi province of China. The simulation results show that the algorithm’s prediction performance is the best among these similar algorithms, and it has a good convergence speed. In future work, we intend to optimize the algorithm further to improve convergence speed and prediction performance. To make it more suitable for the application of short-term wind power forecasting.