A Short-Term Wind Power Forecasting Approach Based on Model Configuration Optimization via Prequential-Cross Cooperative Validation Estimation

Abstract

Efficient utilization of sustainable energy is imperative for supporting the globally escalating electricity demand. Because the unstable wind energy makes the wind power access challenging for power systems, the wind power forecasting becomes the critical part of the power dispatch. In this paper, a short-term wind power forecasting approach based on model configuration optimization via prequential-cross cooperative validation estimation (PCCVE) is proposed. It enables the hybrid ANN including the convolutional neural network, bidirectional long short-term memory network, and multi-head attention mechanism (CNN-BiLSTM-MHA) to better construct the wind speed–power mapping relationship for improving forecasting performance. Firstly, the box-plot local detection–correction combining the spatial–temporal optimal-weighted fuzzy clustering and the sliding window connected box-plot is proposed to reasonably detect and correct local outlier wind speed points. It prevents CNN-BiLSTM-MHA from being interfered with local outlier wind speed points. Secondly, PCCVE based on the prequential-validation estimation and cross-validation estimation is proposed to more accurately give the estimated error of CNN-BiLSTM-MHA, thus better assisting the optimization of the values of CNN-BiLSTM-MHA’s hyperparameters. It enables CNN-BiLSTM-MHA to efficiently construct the wind speed–power mapping relationship. By comparing different approaches on the actual wind farm dataset, the effectiveness and advantages of the proposed approach are demonstrated.

1. Introduction

In recent decades, to solve the increasingly serious energy shortage and environmental pollution, global energy use is shifting away from fossil fuels to sustainable energies, including wind, solar, nuclear, hydropower, and geothermal energy [1]. Among them, wind energy is more low-carbon, more economical, and more extensive, which has attracted more attention and development [2]. According to the statistics released by the World Wind Energy Association, in 2024, the total installed capacity of wind turbines in the world has reached 1174 GW, 121 GW more than that in 2023 [3]. However, wind energy is a kind of unstable energy with fluctuation and intermittence, which makes large-scale wind power access bring great challenges to the stable operation of the power system [4]. To timely and reasonably dispatch wind power to ensure the stable operation of the power system, the development and application of short-term (30 min to 6 h in advance) wind power forecasting is very important [5].

With the development of computer science, many artificial neural network models (ANNs), such as the back propagation neural network [6], long-term and short-term memory network [7], and convolutional neural network [8], are proposed and widely used in short-term wind power forecasting. Compared with the traditional physical models [9,10] and statistical models [11,12], ANN is better at constructing the complex wind speed–power mapping relationship to provide more accurate short-term wind power forecasting. To enable ANN to better construct the wind speed–power mapping relationship, model configuration optimization and data preprocessing are extensively studied [13].

Model configuration optimization aims at optimizing the values of hyperparameters of ANN to efficiently construct the wind speed–power mapping relationship [14]. Usually, the error estimation method is used to give the estimated error of ANN. According to the estimated error, the hyperparameters of ANN are reconfigured with better values by the optimization algorithm [15]. At present, there are two commonly used error estimation methods: (a) prequential-validation estimation (PVE) [16] and (b) cross-validation estimation (CVE) [17]. Both PVE and CVE cut the time series (such as the wind speed series) into several equal-size and orderly subsets for multiple operations. Each operation is to select some subsets to train and verify ANN in a certain way to produce a verification error. After multiple operations, verification errors are averaged as the estimated error. For the error estimation method, the less the estimated error deviates from the actual error of ANN obtained in practical application, the more accurate it is. However, the first few operations of PVE have the data under-utilization problem [18], which makes the estimated error deviate from the actual error and limits the accuracy of PVE. Furthermore, most operations of CVE have the data leakage problem [19], which also makes the estimated error deviate from the actual error and limits the accuracy of CVE. Even worse, inaccurate PVE and CVE sometimes mislead the optimization algorithm to finally configure hyperparameters with suboptimal value, which limits ANN from providing better wind power forecasting. Compared with previous researches that only blindly used PVE or CVE, Ref. [20] reasonably made the comprehensive use of PVE and CVE. Specifically, Ref. [20] studied the deviation distance characteristic of PVE and CVE, which finds that if the non-stationary degree of the time series is higher, estimated errors given by PVE and CVE tend to be closer to and further away from the actual error, respectively; if the non-stationary degree is lower, estimated errors given by PVE and CVE tend to be further away and closer to the actual error, respectively. So, Ref. [20] used PVE and CVE when the non-stationary degree is high and low, respectively. However, the essence of this comprehensive use of PVE and CVE is a simple selection of PVE and CVE, and its accuracy is still not high enough. To our knowledge, more and more deviation characteristics of PVE and CVE are studied, such as the deviation direction characteristic [21,22]. It finds that estimated errors given by PVE and CVE tend to be higher and lower than the actual error, respectively. Therefore, deeply studying the deviation characteristics of PVE and CVE to realize more accurate comprehensive use of PVE and CVE is the first motivation of this paper.

Data preprocessing often focuses on preventing outlier wind speed points from interfering with the construction of the wind speed–power mapping relationship [23]. Box-plot [24], as a popular statistical-based detection method with good effect, is often used to preprocess outlier data points. Compared with distance/density-based detection methods [25] that rely on complex abstract calculation, the box-plot uses concise statistical quantiles to perform detection, which is more interpretable and reliable. In addition, the z-score is also a statistical-based detection method, but it relies on statistical standard deviation and can only be applied to data that obey normal distribution [26]. Better than the z-score, the box-plot has no requirement for data distribution, which is more universal. Since wind speed data usually obey non-strict normal distribution, the box-plot is more used than the z-score to preprocess outlier wind speed points. For example, Ref. [27] adopted the traditional box-plot detection, which uses the quartiles of all wind speed points to construct box-plot boundaries, thus identifying wind speed points exceeding box-plot boundaries as outlier wind speed points. However, the outlier detection strategy [28] points out that if global time data points are used as the detection context, the global outlier points are detected; if local time data points are used as the detection context, the local outlier points are detected. Obviously, the traditional box-plot detection only identifies global outlier wind speed points but ignores local outlier wind speed points. To identify local outlier points, Ref. [29] used the sliding window with the relatively small width to capture local time data points. However, the sliding window only captures and provides few local time data points, which makes it difficult for the box-plot to reasonably detect local outlier points. This is because when local time data points are few, they contain fewer reasonable time data points, which hardly ensures their quartiles are accurate enough under the influence of local outlier wind speed points. This further makes box-plot boundaries not accurate enough to reasonably detect these local outlier points. Therefore, how to capture and input a sufficient number of reasonable local wind speed points to the box-plot to more reasonably detect and correct local outlier wind speed points is the second motivation of this paper.

Inspired by the above work, a short-term wind power forecasting approach based on model configuration optimization via prequential-cross cooperative validation estimation (PCCVE) is proposed. Firstly, the historical wind speed series from the SCADA system of the wind turbine is provided. Secondly, in the data preprocessing of historical wind speed series, the box-plot local detection–correction combining the spatial–temporal optimal-weighted fuzzy clustering and the sliding window connected box-plot is proposed to reasonably detect and correct local outlier wind speed points to produce the available historical wind speed series. Spatial–temporal optimal-weighted fuzzy clustering, which adopts the traditional similarity objective function, the novel balance objective function, and non-dominated sorting [30], is proposed to effectively transform the historical wind speed series into several clusters of similar wind speed segments, thus further providing them to the sliding window connected box-plot. Then, in the model configuration optimization of the hybrid deep ANN including the convolutional neural network, bidirectional long short-term memory network, and multi-head attention mechanism (CNN-BiLSTM-MHA) [31], and PCCVE based on PVE and CVE is proposed to more accurately give the estimated error of CNN-BiLSTM-MHA, thus better assisting the dual fitness particle swarm optimization [32] to optimize the values of CNN-BiLSTM-MHA’s hyperparameters. Finally, CNN-BiLSTM-MHA with the optimal values of main hyperparameters is trained with the available historical wind speed series, and the real-time wind speed series is input to the trained CNN-BiLSTM-MHA to receive the forecasted wind power series.

The innovations of this approach are as follows:

(1) Prequential-cross cooperative validation estimation (PCCVE) based on PVE and CVE is proposed for the first time to more accurately give the estimated error of ANN. By synthesizing the deviation direction and distance characteristics of PVE and CVE, the relative position relationship of the actual error of ANN to the estimated errors given by PVE and CVE is confirmed. This relative position relationship indicates that in the value space, the actual error of ANN tends to be between the estimation errors given by PVE and CVE, and it further tends to be closer to one of those two estimation errors due to the influence of the non-stationary degree of the time series used in PVE and CVE. According to this relative position relationship, the estimated errors given by PVE and CVE can be used to give a better estimated error that is closer to the actual error of ANN. Therefore, in model configuration optimization, PCCVE is more effective than PVE and CVE at guiding the optimization algorithm to finally configure hyperparameters with optimal values instead of suboptimal values, which promotes ANN to provide better wind power forecasting.

(2) Box-plot local detection–correction adopting the spatial–temporal optimal-weighted fuzzy clustering, the sliding window, and the box-plot is proposed for the first time to more reasonably detect and correct local outlier wind speed points. Due to the regular behaviors of wind [33], the clustering is used to transform the wind speed series into similar wind speed segments. When the sliding window is slid on similar wind speed segments to capture and input local wind speed points to the box-plot, local outlier wind speed points are more reasonably detected, because the number of local wind speed points captured by each sliding on similar wind speed segments is significantly more than that captured by each sliding on the wind speed series. Accordingly, there is a larger number of reasonable local wind speed points to resist the influence of local outlier wind speed points on the accuracy of quartiles, which further makes box-plot boundaries accurate enough.

(3) Due to the cosine measure being more sensitive to time series data than the Euclidean measure, the traditional similarity objective function based on these two measures in the spatial–temporal clustering [34] often causes the cosine measure to have an overly large weight, which further makes similar wind speed segments have overly significant temporal variation similarity and insignificant spatial position similarity, that is, insignificant spatial–temporal similarity. Therefore, the spatial–temporal optimal-weighted fuzzy clustering, which uses the novel balance objective function besides the traditional similarity objective function, is proposed for the first time to produce similar wind speed segments with more significant spatial–temporal similarity. The balance objective function is designed to evaluate the balance degree of temporal variation similarity and spatial position similarity under current weights, which further guides non-dominated sorting to select more appropriate weights for cosine measure and Euclidean measure, thus balancing the improvements of temporal variation similarity and spatial position similarity.

The remainder of this paper is organized as follows. Section 2 introduces fundamental methods and related theories. Section 3 describes datasets and evaluation indices. Section 4 presents experiments and analysis. Section 5 draws the conclusion.

2. Proposed Approach

The flowchart of the proposed approach can be illustrated in Figure 1, and the procedure can be described as follows:

Step I: Obtain the historical wind speed series that comes from the SCADA system of the local wind turbine.

Step II: Apply the traditional box-plot detection–correction [27] and the proposed box-plot local detection–correction to the historical wind speed series in turn, which deals with the global outlier wind speed points and the local outlier wind speed points, respectively, thus getting the available historical wind speed series.

Step III: Configure the hyperparameters of the CNN-BiLSTM-MHA with random values.

Step IV: Utilize the proposed prequential-cross cooperative validation estimation to train and validate the CNN-BiLSTM-MHA with the available historical wind speed series to give the estimated error of the CNN-BiLSTM-MHA.

Step V: Use the estimated error to guide the dual fitness particle swarm optimization [32] to reconfigure the hyperparameters of the CNN-BiLSTM-MHA with better values that may further reduce the estimated error.

Step VI: Repeat Step IV and Step V until the maximum number of optimizations is reached. The optimal values of hyperparameters that minimize the estimated error of the CNN-BiLSTM-MHA are obtained. The CNN-BiLSTM-MHA with the optimal values of hyperparameters is finally trained with the whole available historical wind speed series to produce the trained CNN-BiLSTM-MHA.

Step VII: Perform the same detection–corrections as in Step II to the real-time wind speed series in turn to produce the available real-time wind speed series. The box-plot boundaries and cluster centers of the detection–corrections in this step do not need to be calculated but come from Step II.

Step VIII: Input the real-time wind speed series that comes from the SCADA system of the local wind turbine to the trained CNN-BiLSTM-MHA to output the forecasted wind power series.

2.1. Data Preprocessing Based on Box-Plot Local Detection–Correction

The spatial–temporal optimal-weighted fuzzy clustering and the box-plot local detection–correction are described in the following two parts, respectively:

2.1.1. Spatial–Temporal Optimal-Weighted Fuzzy Clustering

The historical wind speed series is cut into M wind speed segments with equal width and chronological order. Define the jth wind speed segment as $s_j$, $j=1,2,\dots ,M$, the number of clusters as K, the kth clustering center as $c_k$, $k=1,2,\dots ,K$, the weights of Euclidean and cosine measures as w and $1-w$, respectively, $0<w<1$, the spatial–temporal fuzzy clustering based on Euclidean measure, cosine measure, and fuzzy c-means clustering, which can be described as follows:

Step I: Randomly initialize the spatial position membership degree $u_{k,j}^e$ and the temporal variation membership degree $u_{k,j}^c$ of $s_j$ to $c_k$, $0<u_{k,j}^e<1$, $0<u_{k,j}^c<1$, ${\sum }_{k=1}^K{u}_{k,j}^e=1$, ${\sum }_{k=1}^K{u}_{k,j}^c=1$.

Step II: Calculate the spatial–temporal membership degree ${\overline{u}}_{k,j}$ by

$$\begin{array}{c}{\overline{u}}_{k,j}=w\times u_{k,j}^e+(1-w)\times u_{k,j}^c\end{array}$$

(1)

Step III: Update the cluster center $c_k$ by

$$\begin{array}{c}{c}_k={\displaystyle \frac{{\sum }_{j=1}^N\left({\left({\overline{u}}_{k,j}\right)}^m\times s_j\right)}{{\sum }_{j=1}^N{\left({\overline{u}}_{k,j}\right)}^m}}\end{array}$$

(2)

Step IV: Update $u_{k,j}^e$ and $u_{k,j}^c$ by

$$\begin{array}{c}{u}_{k,j}^e={\displaystyle \frac{1}{{\sum }_{\overline{k}=1}^K\left({\left(M_e\left(c_k,s_j\right)/M_e\left(c_{\overline{k}},s_j\right)\right)}^{\frac{2}{m-1}}\right)}}\end{array}$$

(3)

$$\begin{array}{c}{u}_{k,j}^c={\displaystyle \frac{1}{{\sum }_{\overline{k}=1}^K\left({\left(M_c\left(c_k,s_j\right)/M_c\left(c_{\overline{k}},s_j\right)\right)}^{\frac{2}{m-1}}\right)}}\end{array}$$

(4)

where m is the fuzzy coefficient, and $M_e(·)$ and $M_c(·)$ are Euclidean measure and cosine measure in [34], respectively.

Step V: Repeat Step II to Step V until the maximum number of iterations is reached.

Step VI: Divide $s_j$ to the ${\widehat{k}}_j$ th cluster if ${\overline{u}}_{{\widehat{k}}_j,j}$ is the largest in $\left\{{\overline{u}}_{1,j},{\overline{u}}_{2,j},\dots ,{\overline{u}}_{K,j}\right\}$, ${\widehat{k}}_j=1,2,\dots ,K$.

Step VII: Output K clusters of similar wind speed segments.

To ensure similar wind speed segments have significant spatial–temporal similarity, w needs to be optimized based on the similarity objective function $E^s(·)$ and the novel balance objective function $E^b(·)$. Like the traditional similarity objective function in [34], $E^s(·)$ also makes w optimized to improve the spatial–temporal similarity of similar wind speed segments in the same cluster, which is designed as follows:

$$\begin{array}{c}{E}^s\left(w\right)=1-{\displaystyle \frac{1}{M}}\times {\displaystyle \sum _{j=1}^M}\left(w\times u_{{\widehat{k}}_j,j}^e+(1-w)\times u_{{\widehat{k}}_j,j}^c\right)\end{array}$$

(5)

where the smaller $E^s\left(w\right)$, the more improvement of spatial–temporal similarity. Because $M_c(·)$ is more sensitive to time series data, such as wind speed segments, the greater $1-w$ is than w, the smaller $E^s\left(w\right)$. However, it leads to the improvement of spatial–temporal similarity realized by overly improving temporal variation similarity and slightly improving spatial position similarity, which further makes similar wind speed segments have overly significant temporal variation similarity and insignificant spatial position similarity, that is, insignificant spatial–temporal similarity.

Therefore, $E^b(·)$ is used to balance the improvements of temporal variation similarity and spatial position similarity, which is designed as follows:

$$\begin{array}{c}{E}^b\left(w\right)={\displaystyle \frac{1}{M}}\times {\displaystyle \sum _{j=1}^M}\left|w\times u_{{\widehat{k}}_j,j}^e-(1-w)\times u_{{\widehat{k}}_j,j}^c\right|\end{array}$$

(6)

where the smaller $E^b\left(w\right)$, the more balanced improvements of temporal variation similarity and spatial position similarity. The closer w and $1-w$ are, the smaller $E^b\left(w\right)$ is.

Obviously, it is conflicting to reduce $E^s(·)$ and $E^b(·)$ at the same time, which indicates that the optimization of w is a multi-objective optimization problem. Well-known, non-dominated sorting genetic algorithm II, as a multi-objective optimization method composed of genetic algorithm and non-dominated sorting [29], is widely used. However, the value range of w is relatively small, which makes the use of the genetic algorithm overqualified. Therefore, grid search is used to replace the genetic algorithm, and its combination with non-dominated sorting is used to optimize w. The spatial–temporal optimal-weighted fuzzy clustering can be described as follows:

Step I: Use grid search to select P different values with D decimal places from (0,1) as weights $\left\{w_1,w_2,\dots ,w_P\right\}$, $P={10}^D-1$.

Step II: Apply the spatial–temporal fuzzy clustering under each weight of $\left\{w_1,w_2,\dots ,\right.$$\left.w_P\right\}$ to receive the corresponding $\left\{\left(E_1^s,E_1^b\right),\left(E_2^s,E_2^b\right),\dots ,\left(E_P^s,E_P^b\right)\right\}$.

Step III: Utilize $\left\{\left(E_1^s,E_1^b\right),\left(E_2^s,E_2^b\right),\dots ,\left(E_P^s,E_P^b\right)\right\}$ and non-dominant sorting to sort $\left\{w_1,w_2,\dots ,w_P\right\}$ and receive the pareto front of non-dominated weights, as shown in Figure 2.

In Figure 2, the non-dominated weight $w_p$ on the pareto front, which is closest to $\left(0,0\right)$, is selected as the best weight, $p=1,2,\dots ,P$. Because $E_p^s$ and $E_p^b$ of $w_p$ are moderate in $\left\{E_1^s,E_2^s,\dots ,E_P^s\right\}$ and $\left\{E_1^b,E_2^b,\dots ,E_P^b\right\}$, respectively, it indicates $w_p$ has makes the compromise between improving spatial–temporal similarity and balancing spatial–temporal similarity, which enables K clusters of similar wind speed segments to have not only significant temporal variation similarity but also significant spatial position similarity, that is, significant spatial–temporal similarity.

Step IV: Output K clusters of similar wind speed segments obtained under $w_p$.

2.1.2. Box-Plot Local Detection–Correction

Suppose there is a periodic time data series, which is shown in Figure 3.

In Figure 3, there is the local outlier point O with the value of $3.17$ in the 2nd period. When the sliding window is slid at area (1), local time data points $\left\{4.17\right.$, $5.03$, $3.17$, $5.12$, $\left.3.86\right\}$ are captured and input to the box-plot. The lower quartile $Q1$, upper quartile $Q3$, and quartile distance $IQR$ of those local time data points are 3.69, 5.05, and 1.36, respectively. By $Q1-1.5\times IQR$ and $Q3+1.5\times IQR$, box-plot boundaries $lb$ and $ub$ are calculated as 3.01 and 7.1, respectively. However, $lb$ is lower than O, which makes O not detected. The reason for the detection failure is that so few local time data points are captured that each of them significantly influences the accuracy of quartiles and box-plot boundaries. Among them, O is overly far below other points, which makes $Q1$ too small, $IQR$ too big, and $lb$ too low. Generally, periods in the periodic time data series are similar, which makes local outlier points in one period more reasonably detected by referring to a sufficient number of reasonable local time data points in other periods. When local time data points in area (1) are input to the box-plot together with local time data points $\left\{3.85\right.$, $5.12$, $6.26$, $4.83$, $\left.3.78\right\}$ in area (2), local time data points $\left\{3.96\right.$, $4.98$, $5.97$, $5.14$, $\left.3.97\right\}$ in area (3), and local time data points $\left\{4.07\right.$, $5.2$, $5.88$, $4.96$, $\left.3.82\right\}$ in area (4), $\widehat{Q1}$, $\widehat{Q3}$, and $\widehat{IQR}$ are 3.91, 5.13, and 1.22, respectively. $\widehat{lb}$ and $\widehat{ub}$ are constructed as 3.3 and 6.96, respectively. Obviously, the use of local time data points in areas (2), (3), and (4) resists the influence of O on the accuracy of quartiles and box-plot boundaries, which makes $\widehat{Q1}$ bigger than $Q1$, $\widehat{IQR}$ smaller than $IQR$, and $\widehat{lb}$ higher than $lb$. Importantly, $\widehat{lb}$ is higher than O, which makes O detected. Like similar periods in the periodic time data series, there are similar wind speed segments in the historical wind speed series due to the regular behaviors of wind. Generally, the clustering is used to identify similar wind speed segments and put them into the same cluster. In the same cluster, local outlier wind speed points in one wind speed segment can also be more reasonably detected by referring to a sufficient number of reasonable local wind speed points in other wind speed segments.

Define the local wind speed point at time i of the zth wind speed segment in the kth cluster as $w{s}_{i,z,k}$, $i=\left\{1,2,\dots ,W\right\}$, the width of the wind speed segment as W, $z=\left\{1,2,\dots ,Z_k\right\}$, the number of wind speed segments in the kth cluster as $Z_k$, and the box-plot local detection–correction can be illustrated in Figure 4, and the procedure can be described as follows:

Step I: Perform the spatial–temporal optimal-weighted fuzzy clustering to the historical wind speed series to receive K clusters of similar wind speed segments. Set $k=1$ and $i=2$.

Step II: Apply the sliding window at time i of the k th cluster to capture and input local wind speed points not only $w{s}_{i-1,1,k}$, $w{s}_{i,1,k}$, $w{s}_{i+1,1,k}$, but also $w{s}_{i-1,2,k}$, $w{s}_{i,2,k}$, $w{s}_{i+1,2,k}$, …, $w{s}_{i-1,K_z,k}$, $w{s}_{i,K_z,k}$, $w{s}_{i+1,K_z,k}$ of similar wind speed segments to the box-plot.

Step III: Use the lower quartile $Q{1}_{i,k}$, the upper quartile $Q{3}_{i,k}$, and the quartile distance $IQ{R}_{i,k}$ of local wind speed points to construct the box-plot upper boundary $u{b}_{i,k}$ and box-plot lower boundary $l{b}_{i,k}$ by $u{b}_{i,k}=Q{3}_{i,k}+1.5\times IQ{R}_{i,k}$ and $l{b}_{i,k}=Q{1}_{i,k}-1.5\times IQ{R}_{i,k}$.

Step IV: Utilize $u{b}_{i,k}$ and $l{b}_{i,k}$ to detect and correct each local wind speed point in turn. If the local wind speed point $w{s}_{i,z,k}$ is higher than $u{b}_{i,k}$ or lower than $l{b}_{i,k}$, it is detected as a local outlier wind speed point, and it is further moved down to $u{b}_{i,k}$ or up to $l{b}_{i,k}$ to produce the corrected local wind speed point $w{s}_{i,z,k}^c$. Otherwise, it is detected as a reasonable local wind speed point.

Step V: Set $i=i+1$ and repeat Step II to Step IV until $i=W-1$.

Step VI: Set $k=k+1$ and $i=2$. Repeat Step II to Step V until $k=K$.

Step VII: Use the chronological order in Section 2.1.1. to sort all wind speed segments that have no local outlier wind speed points to produce the available historical wind speed series.

2.2. Model Configuration Optimization Based on Prequential-Cross Cooperative Validation Estimation

PCCVE and CNN-BiLSTM-MHA are described in the following two parts, respectively:

2.2.1. Prequential-Cross Cooperative Validation Estimation

Both PVE and CVE cut the time series into several equal-size and orderly subsets for multiple operations. Each operation is to select some subsets to train and verify ANN in a certain way to produce a verification error. After multiple operations, verification errors are averaged as the estimated error of ANN.

For PVE, the subsets used for training are strictly earlier than the subset used for verification, which follows the forecasting logic of “forecasting the future data with the historical data”. However, in the first few operations, only a few subsets are used for training, which easily leads to the data under-utilization problem [18], resulting in verification errors of ANN being overly high. These verification errors further make the estimated error $E_p$ tend to deviate from and be higher than the actual error $E_a$ of ANN [21], which is the deviation direction characteristic of PVE.

Different with PVE, CVE makes full use of subsets to avoid the data under-utilization problem to some extent. In each operation, one subset is used for verification, and other subsets are used for training. However, in most operations, subsets used for training are later than subsets used for verification, which does not follow the forecasting logic and easily leads to the data leakage problem [19], resulting in verification errors of ANN being overly low. These verification errors further make estimated error $E_c$ tend to deviate from and be lower than $E_a$ [22], which is the deviation direction characteristic of CVE.

Importantly, PVE and CVE are sensitive to the non-stationarity degree of the time series [20]. When the non-stationarity degree is low, the consistency between historical data and future data is strong, which makes the influence of the data leakage problem on CVE less. In addition, unlike PVE, CVE is not influenced by the data under-utilization problem. Therefore, compared with $E_p$, $E_c$ deviates less from $E_a$. On the contrary, when the non-stationarity degree is high, the data leakage problem has more influence on CVE. Unlike CVE, PVE is not influenced by the data leakage problem. Furthermore, the influence of the data under-utilization problem on PVE is limited. Therefore, compared with $E_c$, $E_p$ deviates less from $E_a$. In summary, if the non-stationary degree of the time series becomes higher, $E_p$ and $E_c$ tend to be closer to and further away from $E_a$, respectively; if the non-stationary degree becomes lower, $E_p$ and $E_c$ tend to be further away and closer to $E_a$, respectively. This is the deviation distance characteristic of PVE and CVE.

By synthesizing the deviation direction characteristics of PVE and CVE, the relative position relationship of $E_a$ to $E_p$ and $E_c$ in the value space is pointed out. It finds that $E_a$ tends to be higher than $E_c$ and lower than $E_p$, that is, between $E_c$ and $E_p$. By further introducing the deviation distance characteristics of PVE and CVE, this relative position relationship is improved. It further finds that if the non-stationary degree $D_{ns}$, $0\le D_{ns}\le 1$ of the time series becomes higher, $E_a$ tends to be closer to $E_p$ than $E_c$, and if $D_{ns}$ becomes lower, $E_a$ tends to be closer to $E_c$ than $E_p$, which is illustrated in Figure 5.

According to this relative position relationship, $E_a$ can be dynamically approximated by $\left(E_p-E_c\right)\times D_{ns}+E_c$, which is illustrated in Figure 6.

In Figure 6, it can be seen that $\left(E_p-E_c\right)\times D_{ns}+E_c$ is also between $E_c$ and $E_p$, like $E_a$. If $D_{ns}$ becomes higher, $\left(E_p-E_c\right)\times D_{ns}+E_c$ tends to be closer to $E_p$ than $E_c$, like $E_a$; if $D_{ns}$ becomes lower, $\left(E_p-E_c\right)\times D_{ns}+E_c$ tends to be closer to $E_c$ than $E_p$, like $E_a$. Compared with $E_p$ and $E_c$, $\left(E_p-E_c\right)\times D_{ns}+E_c$ deviates less from $E_a$. Therefore, it is feasible for PVE and CVE to cooperate to more accurately give $\left(E_p-E_c\right)\times D_{ns}+E_c$ as the estimated error of ANN.

Define the nth subset of the available historical wind speed series as $S_n$, $n=1,2,\dots ,N$, and the prequential-cross cooperative validation estimation based on PVE and CVE can be illustrated in Figure 7, and the procedure can be described as follows:

Step I: Partition the available historical wind speed series into N equal-size and orderly subsets $\left\{S_1,S_2,\dots ,S_N\right\}$. Set $i=1$ and $j=1$.

Step II: Train CNN-BiLSTM-MHA with $\left\{S_1,S_2,\dots ,S_i\right\}$ and verify CNN-BiLSTM-MHA with $S_{i+1}$ to produce the verification error $V_i$.

Step III: Set $i=i+1$ and repeat Step II until $i=N-1$.

Step IV: Average $\left\{V_1,V_2,\dots ,V_{N-1}\right\}$ as the estimated error $E_p$ given by PVE.

Step V: Train CNN-BiLSTM-MHA with $\left\{S_1,S_2,\dots ,S_{j-1}\right\}$ and $\left\{S_{j+1},S_{j+2},\dots ,S_N\right\}$. Verify CNN-BiLSTM-MHA with $S_j$ to produce the verification error $V_j$.

Step VI: Set $j=j+1$ and repeat Step V until $j=N$.

Step VII: Average $\left\{V_1,V_2,\dots ,V_N\right\}$ as the estimated error $E_c$ given by CVE.

Step VIII: Use the weighted permutation entropy evaluation [35] to evaluate the available historical wind speed series to receive the non-stationary degree $D_{ns}$, $0\le D_{ns}\le 1$.

Step IX: Use $E_p$, $E_c$, and $D_{ns}$ to produce the estimated error $E_{pc}$ of CNN-BiLSTM-MHA. In most cases of $E_c$ < $E_p$, $E_{pc}$ = $(E_p-E_c)\ast D_{ns}+E_c$, due to the interference of the random error, $E_c$ < $E_p$ is sometimes not satisfied. In a few cases of $E_c$ > $E_p$, according to the deviation distance characteristics of PVE and CVE [20], if $D_{ns}$ > $0.5$, PVE is considered more accurate than CVE, so $E_{pc}=E_p$; if $D_{ns}<0.5$, CVE is considered more accurate than PVE, so $E_{pc}=E_c$; and if $D_{ns}=0.5$, PVE and CVE are considered equally accurate, so $E_{pc}=\left(E_p+E_c\right)/2$. In rare cases of $E_c=E_p$, $E_{pc}=E_c$.

Compared with PVE and CVE, PCCVE more accurately estimates the quality of values of hyperparameters. That is to say, when PCCVE is used to guide the optimization algorithm, the optimal value is easier to select, which ensures ANN accurately forecasts wind power.

Suppose there are two sets of values of hyperparameters, $V_o$ and $V_{so}$, which are optimal and suboptimal, respectively. The actual errors of ANN under $V_o$ and $V_{so}$ are $E_a^o$ and $E_a^{so}$, respectively, and $E_a^o$ < $E_a^{so}$. When PVE is used to estimate the quality of $V_o$ and $V_{so}$, the estimated errors of $E_a^o$ and $E_a^{so}$ are $E_p^o$ and $E_p^{so}$, respectively. Due to the accuracy of PVE being insufficient, the situation of $E_p^o$ > $E_p^{so}$ appears with high probability. If so, $V_{so}$ is configured to the hyperparameters of ANN. $E_a^{so}$ is obtained and means that ANN inaccurately forecasts wind power. Like PVE, CVE faces the above situation. When PCCVE is used to estimate the quality of $V_o$ and $V_{so}$, the estimated errors of $E_a^o$ and $E_a^{so}$ are $E_{pc}^o$ and $E_{pc}^{so}$, respectively. Due to the accuracy of PCCVE being relatively sufficient, the situation of $E_{pc}^o$ < $E_{pc}^{so}$ appears with high probability. If so, $V_o$ is configured to the hyperparameters of ANN. $E_a^o$ is obtained and means that ANN accurately forecasts wind power.

According to $E_{pc}$ given by PCCVE, the dual fitness particle swarm optimization [32] is used to reconfigure the hyperparameters of the CNN-BiLSTM-MHA with better values that may further reduce $E_{pc}$. Until the maximum number of optimizations is reached, the CNN-BiLSTM-MHA with the optimal values of hyperparameters is finally trained with the whole available historical wind speed series. The real-time wind speed series is further input to the trained CNN-BiLSTM-MHA to output the forecasted wind power series.

2.2.2. CNN-BiLSTM-MHA

CNN-BiLSTM-MHA, as a hybrid deep ANN with a well-developed ability of feature extraction, is used to construct the wind speed–power mapping relationship and forecast wind power. Define the ith input and ith output of CNN-BiLSTM-MHA in the training process as available historical wind speed points $\left\{w{s}_i^a,w{s}_{i-1}^a,\dots ,w{s}_{i-g}^a\right\}$ and the available historical wind power point $w{p}_{i+h}^a$, respectively, the lag step as g, and the forecast horizon as h, and the training process of CNN-BiLSTM-MHA can be illustrated in Figure 8.

In Figure 8, the CNN layer is used to extract local features on small regions of $\left\{w{s}_i^a,w{s}_{i-1}^a,\dots ,w{s}_{i-g}^a\right\}$ with the convolutional kernel, which outputs the feature maps with spatial information. The BiLSTM layer is used to extract the contextual features of the feature map from both forward and reverse directions with the forward LSTM layer and the backward LSTM layer, which outputs hidden state vectors with spatial–temporal information. The MHA layer is used to correlate all the features and positional information of hidden state vectors with $w{p}_{i+h}^a$ through multiple self-attention layers, which outputs attention vectors with more important spatial–temporal information. The fully connected layer is used to linearly transform attention vectors to approximate $w{p}_{i+h}^a$ to construct the wind speed–power mapping relationship.

When the trained CNN-BiLSTM-MHA is in the verification process, the jth input is available historical wind speed points $\left\{w{s}_j^a,w{s}_{j-1}^a,\dots ,w{s}_{j-g}^a\right\}$, and the jth output is the forecasted wind power point $w{p}_{j+h}^f$ of the available historical wind power point $w{p}_{j+h}^a$. When the trained CNN-BiLSTM-MHA is in the testing process, the zth input is real-time wind speed points $\left\{w{s}_z^r,w{s}_{z-1}^r,\dots ,w{s}_{z-g}^r\right\}$, and the zth output is the forecasted wind power point $w{p}_{z+h}^f$ of the future wind power at time $z+h$.

3. Application

The proposed approach is built using MATLAB version R2023a. All calculations are performed on the computer with the hardware device of Intel(R) Xeon(R) Silver 4210 CPU (20 cores, 40 threads, and 2.20 GHz base frequency), as well as the Windows 10 operating system. To be mentioned, the proposed approach contains a large number of loop operations, and each operation includes the training and verification of a deep learning network, which is relatively time-consuming. Since there is no information dependence among operations, the parallel computing toolbox of MATLAB is used to execute operations on 20 cores in parallel to accelerate the running of the proposed approach.

3.1. Datasets Description

The layout of the kelmarsh wind farm in the United Kingdom is shown in Figure 9. In the wind farm, the actual wind dataset of the 6th wind turbine (rated capacity is 2050 kW) is collected by the SCADA system (sampling from 1 January 2020 to 31 December 2021 with a 10 min interval) and used for experiments.

In the dataset, there are wind speed points and wind power points, which are, respectively, processed by the min–max normalization method:

$$\begin{array}{c}{y}_{norm}={\displaystyle \frac{x-x_{min}}{x_{max}-x_{min}}}\end{array}$$

(7)

where $y_{norm}$ is the normalized data point, x is the original data point, and $x_{max}$ and $x_{min}$ are the maximum data points and minimum data points, respectively. The dataset is resampled with the 1 h interval and then divided into training/validation set and testing set. These two sets contain 8708 and 8669 normalized wind speed–power points, respectively.

3.2. Evaluation Indices

Normalized root mean square error (NRMSE) and normalized mean absolute error (NMAE) are utilized.

$$\begin{array}{c}NRMSE={\displaystyle \frac{\sqrt{\frac{1}{L}{\sum }_{t=1}^L{\left(w{p}_t^f-w{p}_t\right)}^2}}{Cap}}\end{array}$$

(8)

$$\begin{array}{c}NMAE={\displaystyle \frac{\frac{1}{L}{\sum }_{t=1}^L\left|w{p}_t^f-w{p}_t\right|}{Cap}}\end{array}$$

(9)

where $w{p}_t^f$ and $w{p}_t$ are the forecasted and actual wind power. L is the length of the actual wind power series. $Cap$ is the rated capacity of the wind turbine. The smaller NRMSE and NMAE, the better the forecasting performance.

4. Experiments and Analysis

4.1. Effectiveness Analysis of Spatial–Temporal Optimal-Weighted Fuzzy Clustering

To verify the spatial–temporal optimal-weighted fuzzy clustering (SOFC), optimally weighted Euclidean measures and cosine measures to enable similar wind speed segments to have good spatial–temporal similarity, clusters output by the spatial–temporal empirical-weighted k-means clustering (EKC) [34], and SOFC are compared. The clusters output by EKC are shown in Figure 10.

In Figure 10, it can be clearly seen that similar wind speed segments in the same cluster have similar curve shapes but different curve heights, which indicates that similar wind speed segments output by EKC have significant temporal variation similarity but insignificant spatial position similarity, that is, insignificant spatial–temporal similarity.

The pareto front of weights of SOFC is shown in Figure 11.

The clusters output by SOFC under $w=0.24$ are shown in Figure 12.

In Figure 12, it can be clearly seen that similar wind speed segments in the same cluster have not only close curve heights but also similar curve shapes, which indicates that similar wind speed segments output by SOFC have significant temporal variation similarity and significant spatial position similarity, that is, significant spatial–temporal similarity.

4.2. Effectiveness Analysis of Box-Plot Local Detection–Correction

To verify that similar wind speed segments give accurate enough quartiles, including the lower quartile $Q1$, upper quartile $Q3$, and quartile distance $IQR$ to construct box-plot boundaries, bootstrap [36] is used. Bootstrap, as a popular statistical method to estimate the point distribution characteristics by repeatedly and randomly sampling, gives the 95% confidence interval (CI) of the point distribution characteristics of the original point set. The 95% CI can be further used to reflect the accurate degree of point distribution characteristics. The 95% CIs of $Q1$, $Q3$, and $IQR$ given by different numbers of similar wind speed segments are shown in Figure 13.

In Figure 13, it can be clearly seen that the more the number of similar wind speed segments, the narrower the 95% CIs of $Q1$, $Q3$, and $IQR$, which means $Q1$, $Q3$, and $IQR$ are more accurate. Until the number of similar wind speed series is higher than 20, the 95% CIs are no longer obviously narrow and tend to be stable, which means $Q1$, $Q3$, and $IQR$ are accurate enough.

Importantly, the number of similar wind speed segments in each cluster in SOFC is much higher than 20. It ensures that $Q1$, $Q3$, and $IQR$ are accurate enough and can be used to construct box-plot upper and lower boundaries in the sliding window connected box-plot (SWBP).

Then, the detection and correction of local outlier wind speed points by SWBP on K clusters are shown in Figure 14.

In Figure 14, box-plot upper and lower boundaries are reasonably constructed in each cluster, and they well surround similar wind speed segments that have good spatial–temporal similarity. However, local wind speed points exceeding these boundaries are inconsistent with the spatial–temporal similarity, so they are detected as local outlier wind speed points and further corrected. For example, in Figure 14b, wind speed segments follow the spatial–temporal similarity, that is, the spatial position of “Around wind speed = 9 m/s” and temporal variation of “Low to high to low”. However, local wind speed points in area (1) distribute too low and rise too slowly, which is inconsistent with the spatial–temporal similarity and exceed the box-plot low boundary. So, these local wind speed points are detected as local outlier wind speed points and further moved to the box-plot lower boundary to produce corrected local outlier wind speed points in area (2).

To intuitively illustrate the effectiveness of BLDC, the detection labels output by the z-score (Z), box-plot (B), and BLDC are compared, as shown in Figure 15. Note that the sliding window (S) is also used to assist Z and B to detect local outlier wind speed points.

In Figure 15, it can be clearly seen that the behaviors of the wind speed points labeled by Z+S, B+S, and BLDC are relatively abnormal to that of their neighboring points, which belong to local outlier wind speed points. This indicates that Z+S, B+S, and BLDC are all effective. However, both Z+S and B+S only observe a few neighboring points to judge whether a wind speed point should be labeled, which faces the problem that the number of normal wind speed points is insufficient to effectively highlight the local outlier wind speed points. Therefore, there are many local outlier wind speed points that are only labeled by BLDC, especially when the wind speed is low, such as between time 5400 and time 5480. This indicates BLDC is more effective than Z+S and B+S.

To further quantify the effectiveness of BLDC, the accuracy of the approaches based on different combinations of Z+S, B+S, BLDC, and TCN-LSTM (T) are compared. The forecasted wind power series of different approaches are shown in Figure 16. Note that local outlier wind speed points detected by Z and B are removed and filled by the cubic spline interpolation (I).

In Figure 16, it can be clearly seen that the forecasted wind power series of Ap T+BLDC more approximate the actual wind power series than that of Ap T+Z+S+I and Ap T+B+S+I, especially between time 170 and time 200. The reason is that BLDC more effectively removes abnormal spatial–temporal characteristics caused by local outlier wind speed points, which enables T to better construct the wind speed–power mapping relationship to forecast wind power. The forecasting errors of different approaches are shown in

Table 1. Forecasting errors of different approaches.

Approaches	Ap T+Z+S+I	Ap T+B+S+I	Ap T+BLDC
NRMSE $(\%)$	5.93	5.86	5.12
NMAE $(\%)$	5.24	5.02	4.49

.

In Table 1, it can be clearly seen that NRMSE and NMAE of Ap T+BLDC are significantly reduced compared with Ap T+Z+S+I and Ap T+B+S+I.

To verify the significance of the forecasting improvements of BLDC relative to Z+S+I and B+S+I, bootstrap is applied for significance testing. Firstly, repeated sampling with replacement is carried out from the testing set to generate many bootstrap testing sets. Then, on each bootstrap testing set, NRMSE under BLDC is used to subtract NRMSE under Z+S+I, and the difference value is recorded. Finally, a 95 % confidence interval is constructed based on the distribution of these difference values, which is [−0.93, −0.44]. Obviously, [−0.93, −0.44] is completely less than 0, which proves that the forecasting improvement of BLDC relative to Z+S+I is significant and stable. A 95 % confidence interval of the forecasting improvement of BLDC relative to B+S+I is also calculated, which is [−0.86, −0.31].

In addition, the runtimes of Z+S, B+S, and BLDC under the wind speed series with different lengths are shown in

Table 2. Runtimes of different detection methods.

Approaches	Z+S	B+S	BLDC
2000	0.02 s	0.05 s	26.21 s
8000	0.09 s	0.18 s	76.14 s
16,000	0.17 s	0.41 s	131.26 s

Here, 2000, 8000, and 16,000 are the lengths of the wind speed series.

.

In Table 2, it can be clearly seen that the runtimes of Z+S, B+S, and BLDC all increase with the increase in the length of wind speed series. However, the runtimes of BLDC are much longer than that of Z+S and B+S, because there is the spatial–temporal optimal-weighted fuzzy clustering in BLDC, which is relatively time-consuming.

4.3. Effectiveness Analysis of Prequential-Cross Cooperative Validation Estimation

To verify the accuracy of the prequential-cross cooperative validation estimation (PCCVE), the estimation deviations of PVE, CVE, and PCCVE are compared. However, multilayer perceptron (MLP) and elman neural network (ELMAN) need to be trained by a sufficient amount of data. So, MLP and ELMAN are sensitive to the data under-utilization problem, which easily make the estimation deviations of PVE large. Long short-term memory (LSTM) and temporal convolutional network (TCN) need to capture long-term temporal information. So, LSTM and TCN are more sensitive to the data leakage problem, which more easily make the estimation deviations of CVE large. Since the construction of PCCVE is based on PVE and CVE, the estimation deviation of PCCVE is also influenced by the type of ANN. Therefore, the estimation deviations of PVE, CVE, and PCCVE should be compared on different types of ANNs to more strictly verify the accuracy of PCCVE. The main hyperparameters of four different types of ANNs are shown in

Table 3. Main hyperparameters of four different types of ANNs.

ANNs	MLP	ELMAN	LSTM	TCN
Dimensions of input ($DI$)	✓	✓	✓	✓
Number of hidden neurons (HN) a	✓	✓	✓
Number of filters (FS)				✓
Size of convolution kernel $\left(CK\right)$				✓
Momentum factor (MF)	✓	✓
Dilation factor (DF)				✓
Initial learn rates (IL)	✓	✓	✓	✓

^a HN = Multiplier (MU) × DI.

.

The relatively effective values of these main hyperparameters are determined as ${DI}_{1\times 6}$ = $4,8,12,16,18,24$, ${MU}_{1\times 4}$ = $0.5,1,1.5,2$, ${FS}_{1\times 4}$ = $16,32,48,64$, ${CK}_{1\times 4}$ = $3,5,7,9$, ${MF}_{1\times 5}$ = $0.5,0.6,0.7,0.8,0.9$, ${DF}_{1\times 4}$ = $1,2,4,8$, and ${IL}_{1\times 7}$ = $0.005,0.02,0.035,0.05,0.065,0.08,0.095$. By traversing all combinations of the values of $D{I}_{1\times 6}$, $M{U}_{1\times 4}$, $M{F}_{1\times 5}$, and $I{L}_{1\times 7}$ to configure MLP, 840 different MLPs are obtained. By defining the estimated error minus the actual error as the estimation deviation, the estimation deviations of PVE, CVE and PCCVE on 840 different MLPs are obtained. So on, the estimation deviations of PVE, CVE, and PCCVE on 840 ELMANs, 168 LSTMs, and 2688 TCNs are obtained. Violin plot [37], as a popular data visualization method, can be used to reflect the distribution differences between different data. Violin plots of estimation deviations of PVE, CVE, and PCCVE on four different types of ANNs are shown in Figure 17.

In Figure 17, the violin plots of estimation deviations of PVE, CVE, and PCCVE on 840 MLPs, 840 ELMANs, 168 LSTMs, and 2688 TCNs are shown in Figure 17a–d in turn. In Figure 17a, violin plots show that the estimation deviations of PVE on MLPs are mainly distributed above and far from ED = 0, around $ED=1.21\times {10}^{-3}$, the estimation deviations of CVE on MLPs are mainly distributed below and far from ED = 0, around $ED=-1.13\times {10}^{-3}$, and the estimation deviations of PCCVE on MLPs are mainly distributed near ED = 0, around $ED=-3.05\times {10}^{-6}$. Importantly, it can be clearly seen that the distribution of estimation deviations of PCCVE is relatively closer to ED = 0 than that of PVE and CVE, which indicates that $E_{pc}$ is generally closer to $E_a$ than $E_p$ and $E_c$. That is, PCCVE is more accurate than PVE and CVE in giving the estimated errors of MLPs. Furthermore, in Figure 17b–d, the comparison between violin plots of estimation deviations of PVE, CVE, and PCCVE also indicate that PCCVE is more accurate than PVE and CVE in giving the estimated errors of ELMANs, LSTMs, and TCNs.

To verify the improvements of the PVE, CVE, and PCCVE to the model performance, the accuracy of the approaches based on different combinations of PVE, CVE, PCCVE, the dual fitness particle swarm optimization (D), and TCN-LSTM (T) are compared. The forecasted wind power series of different approaches are shown in Figure 18.

In Figure 18, it can be clearly seen that the forecasted wind power series of Ap T+D+PCCVE more approximate the actual wind power series than that of Ap T+D+PVE and Ap T+D+CVE, especially between time 90 and time 105. The forecasting errors of different approaches are shown in

Table 4. Forecasting errors of different approaches.

Approaches	Ap T+D+PVE	Ap T+D+CVE	Ap T+D+PCCVE
NRMSE $(\%)$	5.27	4.86	4.32
NMAE $(\%)$	4.32	4.02	3.78

.

In Table 4, it can be clearly seen that NRMSE and NMAE of Ap T+D+PCCVE are significantly reduced compared with Ap T+D+PVE and Ap T+D+CVE. The main reason is that the hyperparameters of T in Ap T+D+PCCVE are better configured. In the training result of Ap T+D+PCCVE, T is configured with $\left\{FS=32,CK=3,DF=2,HN=48,IL=0.1\right\}$, and the NRMSE of T estimated by PCCVE is 4.12. In the training result of Ap T+D+CVE, T is configured with $\left\{FS=32,CK=3,DF=2,HN=32,IL=0.15\right\}$, and the NRMSE of T estimated by CVE is 3.79. By further observing the training process of Ap T+D+CVE, it is found that T is also configured with $\left\{FS=32,CK=3,DF=2,HN=48,IL=0.1\right\}$ in an iteration, but the NRMSE of T estimated by CVE is 3.98. Obviously, 3.79 is smaller than 3.98, which makes $\left\{FS=32,CK=3,DF=2,HN=48,IL=0.1\right\}$ finally replaced by $\left\{FS=32,CK=3,DF=2,HN=32,IL=0.15\right\}$. However, 3.79 is far less than 4.86 in Table 4, which indicates the accuracy of CVE is insufficient. For the same reason, the accuracy of PVE is also insufficient. On the contrary, in Ap T+D+PCCVE, the NRMSE of T estimated by PCCVE is 4.12 and is very close to 4.32 in Table 4, which indicates the accuracy of PCCVE is sufficient. This proves that PCCVE is more effective than PVE and CVE in guiding the optimization algorithm to finally configure hyperparameters with optimal values instead of suboptimal values, thus promoting ANN to provide better wind power forecasting.

To verify the significance of the forecasting improvements of PCCVE relative to PVE and CVE, bootstrap is applied for significance testing. Based on NRMSE, 95 % confidence intervals of the forecasting improvements of PCCVE relative to PVE and CVE are calculated, which are [−1.06, −0.64] and [−0.69, −0.27], respectively. They prove that the forecasting improvements of PCCVE relative to PVE and CVE are significant and stable.

4.4. Ablation Analysis of Proposed Approach

To confirm the contribution of the box-plot local detection–correction (BLDC) and prequential-cross cooperative validation estimation (PCCVE) to the performance of our approach, the accuracy and complexity of the approaches based on different combinations of BLDC, PCCVE, the dual fitness particle swarm optimization (D), and CNN-BiLSTM-MHA (C) are compared. The forecasted wind power series of different approaches are shown in Figure 19.

In Figure 19, viewing from the distance of each approach from the forecasted wind power series to the actual wind power series, it can be seen that the distance of Ap D+C is largest, the distance of Ap BLDC+D+C is moderate, and the distance of our approach is smallest, especially between time 170 and time 200. The forecasting errors and runtimes of different approaches are shown in

Table 5. Forecasting errors and runtimes of different approaches.

Approaches	Ap D+C	Ap BLDC+D+C	Our Approach (BLDC+PCCVE+D+C)
NRMSE $(\%)$	6.16	5.65	3.74
NMAE $(\%)$	5.44	5.10	3.25
Runtime $\left(s\right)$	1223	1355	7467

.

In Table 5, it can be clearly seen that both BLDC and PCCVE effectively reduce forecasting error but also increase runtime. Viewing the forecasting error of Ap D+C and Ap BLDC+D+C, it can be seen that BLDC reduces NRMSE by 0.51% and NMAE by 0.34%. Viewing the forecasting error of Ap BLDC+D+C and our approach, it can be seen that PCCVE reduces NRMSE by 1.91% and NMAE by 1.85%. Compared with BLDC, PCCVE reduces more forecasting error. Viewing the runtime of these approaches, it can be seen that the runtimes of Ap BLDC+D+C and Ap D+C are close, while the runtime of our approach is nearly six times the runtime of Ap BLDC+D+C. Compared with BLDC, PCCVE increases more runtime. Obviously, PCCVE provides the main contribution to the accuracy and complexity of our approach.

4.5. Performance Comparison of Different Approaches

To comprehensively demonstrate the performance of our approach, the accuracy and complexity of our approach are compared with that of benchmark approaches. Generally, the forecasting approach has three main components: (a) data preprocessing, (b) model forecasting, and (c) model configuration optimization. The greater the difference between different forecasting approaches in three main components, the lower the comparability between them. To ensure the comparability between benchmark approaches and our approach, recent advanced approaches Ap [38], Ap [39], and Ap [40] that have similar main components to our approach are selected as benchmark approaches. The detailed implementations of benchmark approaches and our approach are shown in

Table 6. Detailed implementations of benchmark approaches and our approach.

Approaches	Ap [38] (17 Main Hyperparameters)	Ap [39] (9 Main Hyperparameters)	Ap [40] (6 Main Hyperparameters)	Our Approach (12 Main Hyperparameters)
Data preprocessing	Traditional box-plot elimination a	Traditional box-plot detection–correction	Traditional box-plot elimination a	Traditional box-plot detection–correction, BLDC (2 main hyperparameters K and D) b
Model forecasting	6 LSTMs (12 main hyperparameters include 6 pairs of HN and IL) c	TCN-LSTM-SA (5 main hyperparameters including FS, CK, DF, HN, and IL) c,d,e	CNN-LSTM (4 main hyperparameters including FS, CK, HN, and IL) c, d	CNN-BiLSTM-MHA (6 main hyperparameters including FS, CK, HN, IL, AH, and KC) c,d,f
Model configuration optimization	Monarch butterfly optimization (4 main hyperparameters including PS, MG, MR, and MP) g,h, CVE (1 main hyperparameter N) k	Improved coot optimization (4 main hyperparameters including PS, MG, LD, and CC) g,i	Coati optimization (2 main hyperparameters including PS and MG) g	Dual fitness particle swarm optimization (3 main hyperparameters including PS, MG, and IW) g,j, PCCVE (1 main hyperparameter N) k

^a The outlier points elimination was mentioned but not detailed, so the traditional box-plot is selected here. ^b K is the number of clusters, D is the number of decimal places. ^c HN is the number of hidden neurons, IL is the initial learn rates. ^d FS is the number of filters, CK is the size of convolution kernel. ^e DF is the dilation factor. ^f AH is the number of attention heads, KC is the number of key and query channels. ^g PS is the population size, MG is the maximum generations. ^h MR is the migration ratio, MP is the migration period. ⁱ LD is the ratio of leaders to population, CC is the curve control value. ^j IW is the inertia weight. ^k N is the number of subsets.

.

In Table 6, it can be seen that Ap [38], Ap [39], Ap [40], and our approach all use box-plot in data preprocessing to deal with outlier wind speed points, all select the deep neural network in model forecasting to forecast wind power, and all use the heuristic optimization in model configuration optimization to optimize the values of main hyperparameters. The number of main hyperparameters of Ap [38], Ap [39], Ap [40], and our approach is 17, 9, 6, and 12, respectively. Compared with Ap [38], Ap [39], and Ap [40], the number of main hyperparameters of our approach is moderate. To further ensure the fairness of the comparison, during the implementation of Ap [38], Ap [39], Ap [40], and our approach, the common main hyperparameters that need to be optimized are set to belong to the same value ranges. Specifically, $F{S}_{1\times 4}$ = $16,32,48,64$, $C{K}_{1\times 4}$ = $3,5,7,9$, $H{N}_{1\times 6}$ = $8,16,32,48,64,128$, and $I{L}_{1\times 7}$ = $0.005,0.02,0.035,0.05,0.065,0.08,0.095$. The common main hyperparameters that need to be given directly are set to the same value. Specifically, PS = 20 and MG = 100. The remaining main hyperparameters of these approaches are set through the trial–error method. In addition, the persistence model, as the popular and simple forecasting approach, is often used to demonstrate the performance of other approaches. Therefore, the persistence model is also used as a benchmark approach in the comparison. The forecasted wind power series of different approaches under forecast horizon h = 1 are shown in Figure 20.

In Figure 20, it can be clearly seen that in most situations, actual wind power moderately fluctuates, and the forecasted wind power series of all approaches well approximate the actual wind power series. Furthermore, in some situations, actual wind power severely fluctuates, and the forecasted wind power series of our approach better approximate the actual wind power series than that of other approaches, especially between time 90 and time 105. The reason is that PCCVE adapts to the non-stationarity degree of historical wind speed series and more accurately gives the estimated error of CNN-BiLSTM-MHA, which guides the optimization algorithm to better configure the values of CNN-BiLSTM-MHA’s hyperparameters. It ensures CNN-BiLSTM-MHA more effectively constructs the complex wind speed–power mapping relationship to better forecast the wind power under severe wind power fluctuations. However, in few situations, actual wind power has no fluctuation and remains around 0, and the forecasted wind power series of the persistence model better approximates the actual wind power series than that of our approach, especially between time 170 and time 200. This is because in these few situations, the fluctuation of actual wind power does not follow the fluctuation of actual wind speed, which makes it difficult for our approach, as well as Ap [38], Ap [39], and Ap [40] to effectively construct the wind speed–power mapping relationship to forecast the wind power. The forecasting way of the persistence model is to keep actual wind power at the last time as forecasted wind power at the next time, which shows its superiority in these few situations. The forecasting errors and runtimes of different approaches under forecast horizon h = 1 are shown in

Table 7. Forecasting errors and runtimes of different approaches (forecast horizon h = 1).

Approaches	Persistence Model	Ap [38]	Ap [39]	Ap [40]	Our Approach
NRMSE $(\%)$	7.14	6.07	4.36	5.27	3.74
NMAE $(\%)$	4.91	4.65	3.73	4.60	3.25
Runtime $\left(s\right)$	-	16,147	1055	821	7467

.

In Table 7, it can be seen that NRMSE and NMAE of our approach are smaller than those of other approaches. In addition, the runtime of our approach is moderate compared with that of other approaches. Specifically, due to our approach and Ap [38] adopting PCCVE and CVE, respectively, the runtimes of our approach and Ap [38] are significantly greater than that of Ap [39] and Ap [40]. Because our approach trains one CNN-LSTM-MHA, while Ap [38] trains multiple LSTMs, the runtime of our approach is lower than that of Ap [38].

Besides, the values of main hyperparameters of Ap [38], Ap [39], Ap [40], and our approach under forecast horizon h = 1 are detailed in Appendix A.

To better demonstrate the adaptability of our approach, the accuracy of different approaches under different forecast horizons are compared. The forecasted wind power series of different approaches under forecast horizon h = 3 are shown in Figure 21.

The forecasting errors of different approaches under forecast horizon h = 3 are shown in

Table 8. Forecasting errors of different approaches (forecast horizon h = 3).

Approaches	Persistence Model	Ap [38]	Ap [39]	Ap [40]	Our Approach
NRMSE $(\%)$	10.55	8.21	5.98	7.31	5.07
NMAE $(\%)$	7.20	6.82	5.31	6.63	4.64

.

The forecasted wind power series of different approaches under forecast horizon h = 5 are shown in Figure 22.

The forecasting errors of different approaches under forecast horizon h = 5 are calculated and shown in

Table 9. Forecasting errors of different approaches (forecast horizon h = 5).

Approaches	Persistence Model	Ap [38]	Ap [39]	Ap [40]	Our Approach
NRMSE $(\%)$	15.85	11.61	8.44	10.28	7.19
NMAE $(\%)$	10.76	9.65	7.83	9.04	6.75

.

By comparing Figure 20, Figure 21 and Figure 22, it can be seen that the performance of all approaches decreases with the increase in forecast horizon, without exception. However, under different forecasting horizons, the forecasted wind speed series of our approach all better approximate the actual wind speed series than that of other approaches, and the forecasting errors of our approach are all smaller, which proves the availability of our approach under different forecasting horizons.

By comprehensively comparing different approaches, it is found that our approach can more accurately forecast wind power with only moderate runtime.

5. Conclusions

In this paper, an approach based on model optimal configuration via prequential-cross cooperative validation estimation is proposed for short-term wind power forecasting.

The main contributions are as follows: (i) to enable CNN-BiLSTM-MHA to efficiently construct the wind speed–power mapping relationship, the prequential-cross cooperative validation estimation based on PVE and CVE is proposed to accurately give the co-estimated error of CNN-BiLSTM-MHA and better assist the optimization of the values of CNN-BiLSTM-MHA’s hyperparameters; (ii) to prevent CNN-BiLSTM-MHA from being interfered by local outlier wind speed points, the box-plot local detection–correction combining the spatial–temporal optimal-weighted fuzzy clustering and the sliding window connected box-plot is proposed for the first time to reasonably detect and correct local outlier wind speed points; and (iii) to ensure the accuracy of the sliding window connected box-plot, the spatial–temporal optimal-weighted fuzzy clustering, which adopts the traditional similarity objective function, novel balance objective function, and non-dominated sorting, is proposed for the first time to effectively mine and provide similar wind speed segments as the detection contexts of multiple sliding window connected box-plots. Furthermore, sufficient experiments demonstrated that our approach accurately and stably forecasts wind power, which is beneficial to the power dispatch of power systems.

The main limitations are as follows: (i) prequential-cross cooperative validation estimation and box-plot local detection–correction are relatively time-consuming; and (ii) prequential-cross cooperative validation estimation is specially designed for forecasting time series tasks, and it is difficult to be extended to other types of tasks. Therefore, how to reduce the time consumption of prequential-cross cooperative validation estimation and box-plot local detection–correction and expand the former to other types of tasks has become the focus of our future research.