Short-Term Wind Speed Prediction for Bridge Site Area Based on Wavelet Denoising OOA-Transformer

Abstract

Predicting wind speed in advance at bridge sites is essential for ensuring bridge construction safety under high wind conditions. This study proposes a short-term speed prediction model based on outlier correction, Wavelet Denoising, the Osprey Optimization Algorithm (OOA), and the Transformer model. The outliers caused by data entry and measurement errors are processed by the interquartile range (IQR) method. By comparing the performance of four different wavelets, the best-performing wavelet (Bior2.2) was selected to filter out sharp noise from the data processed by the IQR method. The OOA-Transformer model was utilized to forecast short-term wind speeds based on the filtered time series data. With OOA-Transformer, the seven hyperparameters of the Transformer model were optimized by the Osprey Optimization Algorithm to achieve better performance. Given the outstanding performance of LSTM and its variants in wind speed prediction, the OOA-Transformer model was compared with six other models using the actual wind speed data from the Xuefeng Lake Bridge dataset to validate our proposed model. The experimental results show that the mean absolute percentage error (MAPE), root mean square error (RMSE), and coefficient of determination ($R^2$) of this paper’s method on the test set were 4.16%, 0.0152, and 0.9955, respectively, which are superior to the other six models. The prediction accuracy was found to be high enough to meet the short-term wind speed prediction needs of practical projects.

1. Introduction

During construction, large-span bridges have yet to form their final system and have relatively low structural rigidity, making them highly sensitive to wind loads. Ensuring wind resistance safety in engineering construction is particularly important. Currently, measures such as wind-resistant cables and wind-resistant bearings are commonly used during the construction period of large-span bridges [1,2,3,4,5], and it is mandated that aerial construction work is prohibited under severe weather conditions with strong winds of level 6 and above [6]. However, daily weather forecasts cannot account for local effects at the bridge site, they fail to provide accurate wind speed information specific to the bridge location, and nor can they predict future wind speed trends. Therefore, large-span bridges such as arch, cable-stayed, and suspension bridges still face severe risks of wind-induced disasters. To achieve the goal of “early warning and early judgment”, it is necessary to precisely predict the short-term wind speed changes in the area of cable-stayed bridges, thus providing a basis for wind disaster auxiliary decision making, disaster prevention, and mitigation during the construction period of large-span bridges.

The wind speed data in the bridge site area have many complex characteristics, such as multiple outliers, high noise, strong non-stationarity, strong nonlinearity, and apparent multi-scale features. To address the complex characteristics of wind speed data in bridge site areas, we propose a hybrid short-term wind speed forecasting model: Wavelet Denoising OOA-Transformer. This hybrid model includes IQR outlier correction, Wavelet Denoising (WD) [7,8], the Osprey Optimization Algorithm (OOA) [9], and the Transformer model. Initially, the complex terrain at bridge sites may influence wind speed data due to turbulence effects, leading to numerous outliers or anomalies within the collected data. We have implemented an outlier correction method utilizing Interquartile Range (IQR) to tackle these challenges. By calculating the upper and lower quartiles and the IQR of the wind speed series, data points that exceed these bounds are identified as outliers and are subsequently corrected or removed, thus purging the data series of these anomalies and enhancing the overall data quality. Furthermore, various types of noise might affect the wind speed data at the bridge site, such as those emanating from the measuring equipment or environmental electromagnetic interferences. Wavelet Denoising (WD) is applied to the wind speed series to mitigate the noise characteristics and reduce noise. After comparing the signal-to-noise ratio, mean square error, and waveform similarity among four commonly utilized wavelets, the optimal wavelet is selected. The WD process then splits the wind speed series into low-frequency approximate components and high-frequency detail components. The detail components undergo soft thresholding to eliminate high-frequency noise disturbances, and this is followed by a reconstruction process to yield a denoised wind speed series. Additionally, WD aids in extracting the multi-scale features within the wind speed data, thereby furnishing more comprehensive information for subsequent modeling efforts. During the development phase of the Transformer model, the Osprey Optimization Algorithm (OOA) is deployed to optimize critical parameters such as the number of attention heads, the dimensions of feed-forward layers, and the number of layers in both the Encoder and Decoder. The OOA [9] combines the fast convergence of the fish swarm algorithm and the global search ability of the eagle swarm algorithm. The process of parameter optimization balances worldwide exploration and local exploitation, which helps find the optimal model structure of the Transformer and improves the generalization performance of the model. Ultimately, the refined Transformer model is used to predict the actual wind speed series. Compared with traditional machine learning models and shallow neural networks, the Transformer, through its self-attention mechanism and parallel computing framework, excels at capturing both the short-term and long-term dependencies, as well as the nonlinear windspeed characteristics within the wind speed series. By constructing hidden representations of the wind speed sequence in the Encoder and generating future predictions in the Decoder, the Transformer model delivers precise multi-step wind speed forecasts. Through methods like IQR outlier correction, WD noise reduction, OOA optimization, and Transformer modeling, the precision and robustness of short-term wind speed prediction at bridge sites are markedly enhanced, thus providing substantial support for the safety of bridge construction.

The main contributions of this paper are outlined as follows:

• A novel outlier processing method using quartile analysis and a Wavelet Denoising strategy is proposed. Outliers are identified by calculating the upper and lower quartiles of the data; the parts of the wavelet coefficients that fall below a certain threshold are either reduced or set to zero. By combining outlier processing and noise reduction, the quality of the wind speed data series is effectively enhanced.
• A short-term wind speed prediction model for the bridge site area with the OOA-transformer model is proposed. The model adopts the Transformer to capture the short-term and long-term dependencies and nonlinear windspeed characteristics in the windspeed series. The OOA is used to optimize the key parameters of the Transformer model to improve the prediction accuracy and stability.
• Experiments conducted on the actual wind speed dataset of the Xuefeng Lake Bridge demonstrate that the proposed model outperforms other models in all key evaluation metrics (MAPE, RMSE, and $R^2$).

The rest of this paper is organized as follows: In Section 2, the existing literature is organized and analyzed. This section provides the reader with a comprehensive overview of the progress made by experimentalists in wind speed prediction. In Section 3, a wind speed dataset based on the the Xuefeng Lake Bridge is presented, and the experimental methods used are described. Section 4 summarizes the main findings of this paper. This section describes a step-by-step process and analyzes the results, emphasizing the reliability and validity of the findings. Finally, Section 5 summarizes this paper, and it includes a summary of the key findings, broader implications, and limitations of our research work.

2. Related Works

The methods of wind speed prediction can be categorized into two main types: physics-based models and data-driven models [10]. Physics-based models of wind speed prediction are achieved by understanding and simulating the physical processes of atmospheric flow. These models typically rely on the fundamental principles of meteorology. Given that wind is a multifaceted physical phenomenon, direct wind speed forecasts with physical advantages can be obtained by numerical weather prediction (NWP) models [11], and spatially correlated flow dynamics models can also be used to predict wind speed [12].

However, considering the complexity of physical model construction, forecasting accuracy, and applicability issues, data-driven models such as statistical models, machine learning, and hybrid forecasting models have emerged. Statistical models based on the back-and-forth correlation of wind speed time series, according to the characteristics of the series and using the classical statistical model [13,14,15] to improve, are based on historical data to predict the trend change of wind speed. Statistical models have achieved good prediction results when dealing with smooth sequences, but wind speed sequences are more volatile and have obvious nonlinear features, so it is difficult to accurately predict complex changes using only statistical models. The autoregressive integrated moving average (ARIMA) model [16] is a classic model for wind speed prediction, and Yu et al. [17] utilized an ARIMA model to predict the bounded non-periodic data in coastal bridges. Unlike statistical methods, machine learning methods can better fit the nonlinear characteristics of wind speed and can recognize prediction patterns in the presence of relationship uncertainty in historical data [18]. Therefore, researchers have conducted numerous studies on the application of machine learning methods in the field of wind speed prediction. For instance, Alexiadis et al. [19] proposed an artificial neural network model that can predict the wind speed of a turbine in the next 10 min. Shao et al. [20] built a wind speed prediction model based on gated recurrent unit (GRU) neural networks using sensor-monitored wind speed data sets. The results of Décio Alves et al. [21] highlight the new efficacy of machine learning in predicting wind conditions when using high-resolution temporal data, and they also demonstrate that deep learning models outperform traditional methods in improving the accuracy of wind speed predictions. These methods frequently depend on single neural network models, which may lead to problems like overfitting in short-term wind speed predictions or local optimization. Hybrid models, which integrate the best features of different models, have been proposed as a way to increase forecast accuracy.

Composite neural networks, optimization algorithms, and data pretreatment are all combined in the hybrid forecasting model [22,23,24]. The data preprocessing strategy proposed by Qian et al. [25], consisting of outlier correction [26] and data decomposition, can significantly enhance the predictive performance of the entire model. After outlier correction, considering the inherent nonlinearity and noise characteristics of wind speed data, applying data decomposition techniques can effectively reduce the instability of wind speed time series and eliminate redundant information. Recent studies have shown that combining data decomposition methods with advanced machine learning models can significantly improve the accuracy of wind speed predictions [27,28]. For example, Liu et al. [29] utilized wavelet decomposition to reduce the volatility of time series and LSTM to extract time series features, thus constructing a hybrid model combining Wavelet Decomposition (WD) and LSTM for forecasting. Experiments have shown that this model effectively improves prediction accuracy when compared to use of the LSTM model alone [30]. Chen et al. [31] established a short-term wind speed prediction model using Wavelet Packet Decomposition (WPD), ARIMA, and LSTM, where the ARIMA model is utilized for predicting low-frequency time series and the LSTM model for high-frequency time series. By combining the advantages of both, the effectiveness is significantly enhanced. Liu et al. [32] employed the Empirical Mode Decomposition (EMD) algorithm to process the original wind speed series, utilized Long Short-Term Memory (LSTM) networks to predict high-frequency subsequences, and Autoregressive Integrated Moving Average (ARIMA) models to forecast the remaining low-frequency subsequences, thus establishing a wind speed prediction model that integrates Empirical Mode Decomposition (EMD) with a novel Recurrent Neural Network (RNN) and Autoregressive Integrated Moving Average (ARIMA).

Furthermore, the settings of parameters may affect the model’s predictive accuracy and training efficiency [33]. Researchers have developed adaptive short-term wind speed prediction models based on optimization algorithms to address these issues. For instance, Wang et al. [34] proposed a short-term wind speed prediction model for bridge sites based on Sparrow Search Algorithm (SSA), and Bidirectional Long Short-Term Memory Networks (BiLSTMs). Suo et al. [35] proposed a hybrid wind speed prediction model incorporating Time-Varying Filtering Empirical Mode Decomposition (TVFEMD), Partial Autocorrelation Function (PACF), Improved Chimpanzee Optimization Algorithm (IChOA), and Bidirectional Gated Recurrent Unit (BiGRU). This model uses TVFEMD to handle nonlinear sequences to capture temporal information at different time scales. It optimizes bidirectional gated recurrent networks with the Improved Chimpanzee Optimization Algorithm, significantly enhancing the model’s predictive accuracy and robustness. In summary, utilizing optimization algorithms enables the model to maintain optimal parameters [36,37,38,39,40], thereby potentially enhancing the accuracy of wind speed predictions.

Due to gradient vanishing and exploding issues in LSTM and BiLSTM models, which hinder their ability to capture long-distance dependencies, Vaswani et al. [41] proposed the Transformer model. Researchers are exploring applying the Transformer model to time series forecasting tasks. For instance, Zhang et al. [42] utilized the Transformer model to predict the power consumption of satellites, while Weng et al. [43] applied it to predict the deformation of dams. Additionally, some scholars [44,45,46,47] adopted decomposition methods to capture the global patterns of time series and used the Transformer model to capture more detailed structural features. This approach has achieved significant results in univariate time series forecasting tasks.

3. Materials and Methods

This section provides a detailed account of the dataset’s source, pre-training data preprocessing steps, and the construction and training process of the OOA-Transformer wind speed prediction model based on Wavelet Denoising. Firstly, we elaborate on the data acquisition specifics of the Xuefeng Lake Bridge, including timing, frequency, and the fundamental structure of the data. Subsequently, we delve into the pre-training data preprocessing steps in detail, encompassing the identification and handling of outliers, denoising procedures, and normalization techniques. These steps are aimed at optimizing data quality and aligning with the input requirements of the model. Finally, we introduce the integration of Wavelet Denoising technology with OOA algorithms and Transformer models, thus providing a comprehensive overview of model configuration, the training process, and strategies for effectively predicting wind speed data.

3.1. Data Sources and Pre-processing

Xuefeng Lake Bridge, located in Anhua County, Yiyang City, Hunan Province, is a key control project for the entire route of the Hunan Guanxin Expressway, which is designed as a double-tower, double-cable–surface steel–concrete composite girder cable-stayed bridge, with main structural spans of (60 + 160 + 500 + 160 + 60) meters, and the tallest No. 6 central tower is 202.4 m high. The topography of the reservoir at the bridge site is a U-shaped trough with a maximum construction water depth of 68 m. The bridge site is located above the reservoir in a mountainous area, with mountainous terrain on both sides and open water in the center. There are fewer studies on wind field prediction in similar terrain. Therefore, this paper conducted a wind speed prediction test at the Xuefeng Lake Bridge site and collected the data on the wind field by installing the ultrasonic anemometer on the construction crane for the experiment. The results of this study can provide a reference for wind field prediction experiments under similar geomorphologic conditions. In this paper, field measurement was carried out at the bridge site. Figure 1 shows the geographic location of the Xuefeng Lake Bridge. The wind measurement equipment was installed on the bridge construction crane, and the arrangement of measurement points is shown in Figure 2.

This paper presents a comprehensive wind speed dataset that was obtained from on-site measurements at the Xuefeng Lake Bridge, covering the period from 1 March 2018, at 00:00, to 1 June 2018, at 00:00. Figure 3 illustrates the actual wind speed series variation curve.

Considering that instrumental measurements often encounter issues such as instrumental failure, incorrect data entry, and measurement errors, it is crucial to apply robust methods for data cleaning. In this study, the interquartile range (IQR) rule was employed to correct outliers. The IQR is defined as the difference between the third quartile (Q3) and the first quartile (Q1), which helps to mitigate extreme values.

The raw wind speed data sequence, as shown in

Table 1. Wind Speed Data Sequence Statistical Information.

Dataset	Min	Max	Mean	Std	Skewness	Kurtosis
Raw	0.0000	25.2060	8.0777	5.1068	0.6619	−0.3347
IQR	0.2052	22.8112	8.0336	5.0457	0.6317	−0.4290

, initially ranged from 0.0000 m/s to 25.2060 m/s, with a mean of 8.0777 m/s and a standard deviation of 5.1068 m/s. The data display a positive skewness (0.6619) and a slightly negative kurtosis (−0.3347), indicating a distribution with a long tail on the higher value side and a peak that is less pronounced than a normal distribution.

From Table 1, several changes in the dataset characteristics can be observed before and after the processing of the IQR method. Firstly, the minimum value increases, suggesting that the corrective manipulation removes some shallow values, thereby raising the lower boundary of the data. Secondly, the maximum value decreases, indicating that the correction process effectively adjusts some extremely high values, thus narrowing the overall range of the data. Furthermore, the average value shows a slight decrease, with the change being relatively minor, which implies that the center of the data distribution shifts slightly to the left. However, the overall alteration is not significant. Additionally, the standard deviation decreases slightly, indicating a narrower and more concentrated data distribution. The skewness also decreases slightly, suggesting a reduction in the influence of extreme values and a shift toward a more symmetrical distribution. Finally, the kurtosis decreases slightly, reflecting a less peaked distribution. In conclusion, the processing of IQR anomalies leads to a reduction in the influence of extreme values, a narrowing in the distribution range, an increase in concentration, and an enhancement in both the symmetry and smoothness of the distribution.

3.2. Methods

In this paper, the IQR method was first used to handle outliers in the data, and this was followed by the application of wavelet transform to denoise the processed data, thus retaining key information while removing noise. Subsequently, the denoised data were normalized using min-max normalization to scale the values to the [0, 1] range in order to meet the input requirements of the Transformer model. During the training of the Transformer model, the OOA was employed to optimize the hyperparameters of the model to achieve optimal performance. Finally, the trained Transformer model was used to predict new input data, and the prediction results were rescaled to the original scale through inverse normalization, ultimately outputting the predicted values.

3.2.1. Wavelet Denoising

The principle of Wavelet Denoising involves using wavelet transform to perform multi-level decomposition on wind speed series at different scales. The wavelet coefficients are then processed, typically through methods like thresholding, and the denoised wind speed series is reconstructed through the inverse wavelet transform. Let the noisy wind speed series be denoted as $f\left(t\right)$. After applying the wavelet transform, the resulting wavelet coefficients, $W_f(a,b)$, are given by Equation (1):

$$W_f(a,b)=\frac{1}{\sqrt{a}}{\int }_{-\infty }^{+\infty }f\left(t\right){\psi }^{*}\left(\frac{t-b}{a}\right)dt,$$

(1)

where a is the level of decomposition, b is the translation parameter, and $\psi \left(t\right)$ is the mother wavelet function. The wavelet decomposition splits the wind speed series $f\left(t\right)$ into low-frequency and high-frequency components. Then, the thresholding processing is applied to the high-frequency coefficients obtained from the wavelet decomposition to remove noise. The modified wavelet coefficients ${\widehat{W}}_f(a,b)$ are defined as follows:

$${\widehat{W}}_f(a,b)=\left\{\begin{array}{cc}sgn\left(W_f(a,b)\right)\left(\right|W_f(a,b)|-\lambda ),\hfill & \mathrm{if}|W_f(a,b)|\ge \lambda \hfill \\ 0,\hfill & \mathrm{if}|W_f(a,b)|<\lambda ,\hfill \end{array}\right.$$

(2)

where $\lambda$ is a predefined threshold. If the absolute value of a coefficient is less than $\lambda$, then the coefficient is set to zero; otherwise, the coefficient is shrunk by the amount $\lambda$, meaning its absolute value is reduced by $\lambda$.

After thresholding, the processed wavelet coefficients ${\widehat{W}}_f(a,b)$ are used to reconstruct the denoised wind speed series through the inverse wavelet transform:

$$\widehat{f}\left(t\right)=\frac{1}{C_{\psi }}{\int }_0^{+\infty }{\int }_{-\infty }^{+\infty }\frac{1}{a^2}{\widehat{W}}_f(a,b)\psi \left(\frac{t-b}{a}\right)dadb.$$

(3)

Here, $C_{\psi }$ is the normalization constant of the wavelet function. The original function $f\left(t\right)$ is reconstructed by performing a double integral over the wavelet coefficients ${\widehat{W}}_f(a,b)$ across different levels of decomposition a and translations b.

This study selects four wavelet functions (Db4, Sym4, Coif1, and Bior2.2) for Wavelet Denoising, using a four-level decomposition to perform wavelet decomposition of the wind speed series data. The soft thresholding method is applied to threshold the wavelet coefficients, and the processed wavelet coefficients are used for data reconstruction.

Three metrics are employed to assess the effectiveness of noise reduction techniques. The Signal-to-Noise Ratio (SNR) is the most intuitive measure of denoising effectiveness. A higher SNR indicates less noise in the signal and better noise reduction. The SNR is calculated using the power of the pure signal, $P_{\mathrm{signal}}$, and the power of the pure noise, $P_{\mathrm{noise}}$, as follows:

$$\mathrm{SNR}\left(\mathrm{dB}\right)=10{log}_{10}\left(\frac{P_{\mathrm{signal}}}{P_{\mathrm{noise}}}\right).$$

(4)

Waveform similarity, measured by the Normalized Cross-Correlation (NCC), reflects the overall similarity of the signal waveforms before and after denoising. It does not characterize the detailed changes in waveform oscillations. Let $A_s$ denote the pure signal and $A_d$ the signal after filtering. The NCC is defined as follows:

$$\mathrm{NCC}=\frac{{\sum }_{n=1}^N{A}_s\left(n\right)A_d\left(n\right)}{\sqrt{\left({\sum }_{n=1}^N{A}_s^2\left(n\right)\right)\left({\sum }_{n=1}^N{A}_d^2\left(n\right)\right)}}.$$

(5)

The Mean Square Error (MSE) is a metric that reflects the discrepancy between the estimated and the true values. Let $A_{\mathrm{signal}}$ be the pure signal and $A_{\mathrm{denoised}}$ the denoised signal. The MSE is computed as follows:

$$\mathrm{MSE}=\frac{{\sum }_{i=1}^N{\left|A_{\mathrm{signal}}-A_{\mathrm{denoised}}\right|}^2}{N}.$$

(6)

3.2.2. Transformer

The Google team introduced the Transformer model [41] in 2017, which is structured into two components: an Encoder and a Decoder. The Encoder features three key modules: Multi-Head Self-Attention, a Position-Wise Feed-Forward Network, and the Add and Normalize layer for normalization. The Decoder builds upon the Encoder’s existing modules by adding a cross-attention module and introducing a masking mechanism in the Multi-Head Self-Attention. The specific structure of the Transformer model is shown in Figure 4.

• Positional Encoding: Since the Transformer model does not include recurrent or convolutional operations, positional encoding retains the temporal attributes associated with each data point, thus enabling the model to utilize this information to understand the sequential relationships in the series. Positional encoding is generated by alternating sine and cosine functions, with each dimension corresponding to a frequency:

  $${\mathrm{PE}}_{(\mathrm{pos},2i)}=sin\left(\frac{\mathrm{pos}}{{10000}^{2i/d_{\mathrm{model}}}}\right),$$

  (7)

  $${\mathrm{PE}}_{(\mathrm{pos},2i+1)}=cos\left(\frac{\mathrm{pos}}{{10000}^{2i/d_{\mathrm{model}}}}\right),$$

  (8)

  where pos is the position index, 2i and 2i + 1 are dimension indices, and $d_{\mathrm{model}}$ represents the embedding dimension.
• Multi-Head Attention allows the model to learn information simultaneously from different representational subspaces. This mechanism enhances the model’s focus and learning capabilities, thus enabling it to capture information at different levels of the sequence. Scaled dot-product attention is defined as follows:

  $$A(Q,K,V)=\mathrm{softmax}\left(\frac{Q{K}^T}{\sqrt{d_k}}\right)V,$$

  (9)

  where $Q,K$, and V are the query, key, and value matrices, respectively, and $d_k$ is the dimension of the critical vectors. In the multi-head attention mechanism, the scaled dot-product operations for different heads are processed through different linear transformations. The outputs are then merged:

  $$\mathrm{MultiHead}(Q,K,V)=\mathrm{Concat}({\mathrm{head}}_1,\dots ,{\mathrm{head}}_h)W^O.$$

  (10)

  This design not only reduces the vector dimension of the scaled dot-product calculation, but also captures the intrinsic associations of the sequence from multiple perspectives through the parallel processing of multiple heads. In the Decoder part, the multi-head attention mechanism uses masking to hide information after the current position, thus ensuring that predictions rely solely on known sequence information.
• Feed-Forward Network (FFN) provides additional nonlinear processing capabilities for each layer of the Transformer model. The feed-forward network consists of two linear transformations separated by a nonlinear activation function. The FFN formula is as follows:

  $$\mathrm{FFN}\left(x\right)=max(0,x{W}_1+b_1)W_2+b_2,$$

  (11)

  where x represents the input to the layer; $W_1$ and $b_1$ are the weights and biases of the first linear layer; $W_2$ and $b_2$ are the weights and biases of the second linear layer; and $max(0,·)$ is the ReLU activation function, which operates on each element.

3.2.3. OOA-Transformer

Before constructing the model, the data are processed using the min-max normalization method. This step helps eliminate the impact of differing scales among variables and prevents complications during model training due to differences in the numerical ranges of variables.

Manual selection of Transformer parameters can lead to errors. In this paper, we employed the Osprey Optimization Algorithm (OOA) [9] to optimize seven hyperparameters of the Transformer model, thereby enhancing the network’s predictive performance. The specific steps for optimizing the Transformer model using the OOA are as follows:

• Initialization: Identify the hyperparameters of the Transformer model to be optimized, i.e., the size of the osprey population (number of Transformer models), the dimension of the population (set to 1 for single-variable wind speed prediction), and the maximum number of iterations. First, form an initial solution set according to Equations (12) and (13):

  $${\theta }_i=\left\{{\theta }_{i,j}\right\},$$

  (12)

  $${\theta }_{i,j}=l{b}_j+r\times (u{b}_j-l{b}_j),$$

  (13)

  where ${\theta }_i$ is the set of parameters for the i-th Transformer model, ${\theta }_{i,j}$ is the j-th parameter of the Transformer model, j ranges from 0 to 7, $l{b}_j$ is the lower boundary of the optimization range, $u{b}_j$ is the upper boundary, and r is a random number between 0 and 1.
• Global Exploration (First Phase): For each Transformer model ${\theta }_i$, first define the fitness function $L\left({\theta }_i\right)$:

  $$L\left({\theta }_i\right)=\frac{1}{n}\sum _{j=1}^n|y_j-{\widehat{y}}_j|,$$

  (14)

  where n is the number of training samples, $y_j$ is the actual label of the j-th sample, and ${\widehat{y}}_j$ is the prediction of the model for that sample. Then, determine the set of models in the population with a smaller loss function value (better performance) $F{P}_i$:

  $$F{P}_i=\left\{{\theta }_k\right|k\in \{1,2,\dots ,N\},L\left({\theta }_k\right)<L\left({\theta }_i\right)\}\cup \left\{{\theta }_{\mathrm{best}}\right\},$$

  (15)

  where ${\theta }_{\mathrm{best}}$ is the parameter set of the model with the most minor loss in the current population. Next, randomly select a model ${\theta }_{SF}$ from the set $F{P}_i$ as the target for the osprey’s predation. Update the direction of ${\theta }_i$ toward ${\theta }_{SF}$ using the osprey algorithm’s two strategies (i.e., chasing fish and avoiding strong opponents) as follows:

  $${\theta }_{i,j}^{P1}={\theta }_{i,j}+r_{i,j}\times (2I_{i,j}-1)\times ({\theta }_{S{F}_i,j}-{\theta }_{i,j}),$$

  (16)

  where ${\theta }_{i,j}$ and ${\theta }_{S{F}_i,j}$ represent the values of the current and target models at the j-th parameter, respectively; $r_{i,j}$ is a random number within an interval used to update the step size; and $I_{i,j}$ is a randomly chosen integer value of 0 or 1, where $I_{i,j}=1$ brings the model closer to the target parameter and $I_{i,j}=0$ moves it away, thus avoiding local optima. Boundary Checks and Adjustments: Each parameter ${\theta }_{i,j}^{P1}$ is then checked and adjusted for boundaries as follows:

  $${\theta }_{i,j}^{P1}=\left\{\begin{array}{cc}{\theta }_{i,j}^{P1}\hfill & \mathrm{if}l{b}_j\le {\theta }_{i,j}^{P1}\le u{b}_j\hfill \\ l{b}_j\hfill & \mathrm{if}{\theta }_{i,j}^{P1}<l{b}_j\hfill \\ u{b}_j\hfill & \mathrm{if}{\theta }_{i,j}^{P1}>u{b}_j.\hfill \end{array}\right.$$

  (17)

  If the final $L\left({\theta }_{i,j}^{P1}\right)<L\left({\theta }_i\right)$, then replace ${\theta }_i$ with ${\theta }_{i,j}^{P1}$.
• Local Exploitation (Second Phase): For each iteration $t=1,2,\dots ,T$, and for each Transformer model ${\theta }_i$, generate a new candidate solution ${\theta }_{i,j}^{P2}$ using the following equation:

  $${\theta }_{i,j}^{P2}={\theta }_{i,j}+\frac{l{b}_j+r\times (u{b}_j-l{b}_j)}{t}.$$

  (18)

  Then, check and adjust each parameter of ${\theta }_{i,j}^{P2}$ for boundary constraints as follows:

  $${\theta }_{i,j}^{P2}=\left\{\begin{array}{cc}{\theta }_{i,j}^{P2}\hfill & \mathrm{if}l{b}_j\le {\theta }_{i,j}^{P2}\le u{b}_j\hfill \\ l{b}_j\hfill & \mathrm{if}{\theta }_{i,j}^{P2}<l{b}_j\hfill \\ u{b}_j\hfill & \mathrm{if}{\theta }_{i,j}^{P2}>u{b}_j.\hfill \end{array}\right.$$

  (19)

  If the final $L\left({\theta }_i^{P2}\right)<L\left({\theta }_i\right)$, replace ${\theta }_i$ with ${\theta }_i^{P2}$.
• Repeat: Repeat Steps 2 and 3 until the termination condition (maximum number of iterations) is met.

Figure 5 illustrates the parameter selection and training process of the Transformer model when it is optimized using the Osprey Algorithm.

3.2.4. The Overall Framework of Our Proposed Model

The diagram in Figure 6 illustrates a comprehensive framework for processing and predicting wind speed data. The process is divided into three main steps:

• Step 1: The process begins with the Interquartile Range (IQR) method applied to the initial wind speed data to remove potential errors and outliers. This method calculates the IQR and excludes the data points lying outside $Q1-1.5\times \mathrm{IQR}$ and $Q3+1.5\times \mathrm{IQR}$, thus aiming to improve data reliability. Following this preprocessing, the data are denoised using a Bior2.2 wavelet, which addresses non-stationary features in the data through wavelet decomposition, soft thresholding, and reconstruction.
• Step 2: After the Wavelet Denoising, min-max normalization is employed to scale the values to a [0, 1] range. This normalization facilitates the efficient processing of data. The Transformer model’s parameters are optimized by the Osprey Optimization Algorithm, thus enhancing model performance.
• Step 3: In the final step, the processed data are fed into the OOA-Transformer model to predict future wind speeds. The predictions undergo inverse normalization to revert them to their original scale, thus allowing for comparison with the actual data over different time intervals, as displayed in the diagram.

4. Results and Discussion

This subsection is structured into two different parts, the first part analyzes and discusses the noise reduction results of different wavelets, and the second part analyzes and discusses the prediction results of different models on the same test data.

4.1. The Results of Wavelet Denoising

Figure 7 displays a comparative plot of the data after Wavelet Denoising versus the data processed with the IQR method for outlier removal (a random selection is shown).

Figure 7 shows that the Bior2.2 wavelet effectively manages sudden sharp peaks, such as drastic changes in wind speed, while concurrently preserving a substantial amount of valuable information in the wavelet.

Table 2. Evaluation metrics for different wavelets.

Wavelet	SNR	MSE	NCC
Db4	22.2305	0.5384	0.9917
Sym4	22.1369	0.5502	0.9918
Coif1	21.7374	0.6032	0.9910
Bior2.2	22.9762	0.4535	0.9931

presents the values of three metrics—the Signal-to-Noise Ratio (SNR), Mean Squared Error (MSE), and Normalized Cross-Correlation (NCC)—under four different wavelets. The SNR is typically expressed in decibels (dB).

As shown in Table 2, the Bior2.2 wavelet outperformed the others in all evaluated metrics. It had the highest SNR, indicating better noise suppression while retaining useful signals; the lowest MSE, demonstrating minimal error; and the highest NCC, which shows that the denoised data maintain the best correlation with the original data. Based on these findings, the Bior2.2 wavelet was selected for denoising in this study.

4.2. Compared with Other Prediction Models

LSTM and its variants have outstanding performance in wind speed prediction. As such, in order to verify the effectiveness of the proposed model in this study, the model was compared with six models: LSTM, LSTM-AT, BiLSTM-AT, CNN-BiLSTM-AT, OOA-CNN-BiLSTM-AT, and Transformer.

4.2.1. Parameter Settings

In this study, the proposed model was experimented with using Python 3.9.18, Keras 2.6.0, and Tensorflow 2.6.0 on a GeForce RTX 3090.

The parameter settings for the LSTM, LSTM-AT, BiLSTM-AT, and CNN-BiLSTM-AT models [48] regarding the time step lookback value, regularization rate dropout, and learning rate lr are presented in

Table 3. Model parameter settings.

Model	Dropout	lr	Lookback
LSTM	0.2	0.001	100
LSTM-AT	0.2	0.001	100
BiLSTM-AT	0.2	0.001	100
CNN-BiLSTM-AT	0.2	0.001	100

.

Parameter Settings in OOA-CNN-BiLSTM-AT: This model utilizes the lookback value, which represents the number of time steps of the input features and the target sequence, along with the regularization rate dropout and the learning rate as its hyperparameters. The ranges for these values are set as follows: the regularization rate ranges from 0.1 to 0.5, the lookback value ranges from 1 to 200, and the learning rate ranges from 0.0001 to 0.01. Additionally, the population size in the Osprey Optimization Algorithm is set to 10, and the maximum number of iterations is 20.

Parameter Settings for the Transformer Model without Optimization Algorithms:

• window_len: This parameter defines the length of the input sequence for the model. In time series analysis (such as wind speed prediction), a window_len of 1 means using data from the previous hour to predict the wind speed at the next time point (or points);
• target_len: This specifies the output sequence length from the model. A target_len of 1 indicates that the model predicts wind speed for a single future time point;
• num_encoder_layers: The number of encoder layers in the Transformer model is 2. Encoder layers are responsible for processing the input sequence, and each additional layer can increase the model’s ability to capture complex relationships;
• num_decoder_layers: The number of decoder layers in the Transformer model is also 2. Decoder layers are used to generate the output sequence;
• d_model: The dimensionality of the output for all layers within the model is 64. For both encoder and decoder layers, d_model is the dimension size of their output vectors;
• num_heads: The model uses a multi-head attention mechanism with two heads. This allows the model to learn information from different representational subspaces at the same time;
• feedforward_dim: The dimension of the feedforward network in both encoder and decoder layers is 64. This represents the size of the hidden layer, which performs a separate, fully connected transformation on each position following the self-attention layer;
• Dropout: The dropout rate for regularization in the model is 0.1. Dropout is a common regularization technique that prevents overfitting by randomly dropping (setting to zero) a portion of the feature values;
• positional_encoding: “sinusoidal”. Since the Transformer model lacks recurrent or convolutional structures, it uses positional encoding to incorporate sequence order information, and “sinusoidal” refers to using fixed sinusoidal and cosinusoidal functions for positional encoding.

In this study, the parameters window_len, num_encoder_layers, num_decoder_layers, d_model, num_heads, feedforward_dim, and dropout were chosen as the hyperparameters for optimization in the OOA-Transformer model.

Table 4. Model parameter settings.

Parameter	Min	Max
window_len	1	50
num_encoder_layers	1	10
num_decoder_layers	1	10
d_model	16	512
num_heads	1	16
feedforward_dim	16	512
dropout	$1\times {10}^{-6}$	0.5

presents the range that was used for optimizing these seven parameters.

Additionally, the population size for the Osprey Optimization Algorithm was set to 10, with a maximum of 20 iterations.

4.2.2. Evaluation Criteria

To objectively assess the wind speed prediction performance of the proposed model, three metrics were used: Mean Absolute Percentage Error (MAPE), Root Mean Square Error (RMSE), and the Coefficient of Determination ($R^2$). The expressions for these metrics are given as follows:

$$\mathrm{MAPE}=\left(\frac{100\%}{n}\sum _{i=1}^n\left|\frac{y_i-h_i}{h_i}\right|\right),$$

(20)

$$\mathrm{RMSE}=\sqrt{\frac{{\sum }_{i=1}^n{(y_i-h_i)}^2}{n}},$$

(21)

$$R^2=1-\frac{{\sum }_{i=1}^n{(y_i-h_i)}^2}{{\sum }_{i=1}^n{(y_i-\overline{y})}^2},$$

(22)

where $y_i$ denotes the actual values, $\overline{y}$ is the mean of these actual values, $h_i$ represents the predicted values, and n is the length of the series.

4.2.3. The Experiment Results

The performance accuracy of the Transformer, CNN-BiLSTM-AT model significantly hinges on the selection of its parameters. The choice of hyperparameter values is based on empirical experience and often requires multiple iterations of trial and error to attain notably high prediction accuracy. In this paper, we introduce the Osprey Optimization Algorithm (OOA) to optimize the hyperparameters of the Transformer, CNN-BiLSTM-AT model. The range of parameters subject to optimization, along with the specific parameters to be optimized and the other initial parameters of the network, are detailed in Section 3.2.1. The $L\left({\theta }_i\right)$ is the fitness function to ascertain the optimal parameters. Figure 8 depicts the fitness variation curves derived from the Osprey Optimization Algorithm for the optimization computation of the CNN-BiLSTM-AT model, Transformer.

Figure 8 (left) displays the optimized adaptation curve for the CNN-BiLSTM-AT model, while Figure 8 (right) illustrates the same for the Transformer model. As shown in Figure 8, both models exhibited a continuous decrease in the fitness function ($L\left({\theta }_i\right)$) following optimization with the OOA, thus indicating a significant improvement over their pre-optimization performance. Moreover, the figure reveals that the fitness value of the Transformer model was lower than that of the CNN-BiLSTM-AT model when both were optimized with the same OOA. This lower fitness value suggests the superior performance of the Transformer model compared to the CNN-BiLSTM-AT model.

Table 5. Optimized hyperparameters for machine learning models.

Model	Parameter	Optimum Value
OOA-CNN-BiLSTM-AT	lookback	148
	dropout	0.0105
	learning rate (lr)	0.0031
OOA-Transformer	window_len	1
	number_encoder_layers	1
	number_decoder_layers	1
	d_model	16
	number_heads	1
	feedforward_dim	16
	Dropout	$1\times {10}^{-5}$

show the hyperparameters of the OOA-optimized CNN-BiLSTM-AT and Transformer models.

In this study, the mainstream models Transformer, OOA-CNN-BiLSTM-AT, CNN-BiLSTM-AT, BILSTM-AT, LSTM-AT, and LSTM were taken to be compared with the OOA-Transformer model proposed in this study, and the data from some randomly selected test sets will be shown. Figure 9 demonstrates the six model prediction effects.

Figure 9 shows that the OOA-Transformer model proposed in this paper is closer to the actual value. Judging the prediction results by graphs is subjective and prone to errors. In order to further confirm the prediction accuracy of the models, all the prediction models were compared with each other by three performance indexes.

Table 6. Predictive effectiveness of the different models.

Model	MAPE (%)	RMSE	${\mathit{R}}^2$
LSTM	10.36	0.0406	0.9679
LSTM-AT	9.35	0.0383	0.9716
BiLSTM-AT	9.21	0.0375	0.9727
CNN-BiLSTM-AT	8.93	0.0350	0.9762
OOA-CNN-BiLSTM-AT	4.57	0.0188	0.9930
Transformer	6.53	0.0296	0.9830
OOA-Transformer	4.16	0.0152	0.9955

demonstrates the results of different models on these three indicators.

Figure 10, Figure 11 and Figure 12 show the results of different models for three different metrics in the test set.

As shown in Figure 10, the OOA-Transformer model displayed the lowest MAPE value, indicating the highest prediction accuracy among all the models tested. The traditional LSTM and its variants (LSTM-AT and BiLSTM-AT, which incorporate the attention mechanism) showed higher MAPE values, suggesting lower prediction accuracy. The CNN-BiLSTM-AT model, combining a Convolutional Neural Network (CNN) with a Bidirectional Long Short-Term Memory Network (BiLSTM) and integrating an attention mechanism, outperformed the standalone LSTM model, yet it remained inferior to the OOA-Transformer model. The Transformer model also performed well but slightly less so than the OOA-Transformer.

Figure 11 clearly demonstrates that, in the comparison of predictive model performance, the OOA-Transformer model is better with its lowest RMSE value, thus proving its superior prediction accuracy among all the models compared. Additionally, the CNN-BiLSTM-AT model, which integrates various technologies, surpasses the basic LSTM model, but still shows certain disadvantages when compared to the Transformer model.

As depicted in Figure 12, the chart demonstrates the $R^2$ values of different predictive models. The OOA-Transformer model notably reaches the highest $R^2$ value, almost perfect at 1.00, due to its integration of optimization algorithms with transformer techniques. In contrast, the basic LSTM model records the lowest $R^2$ value, reflecting its limited ability to handle complex data. The successive enhancements in $R^2$ from LSTM-AT to BiLSTM-AT, and then CNN-BiLSTM-AT, show the advantages of adding attention mechanisms and convolutional layers. Despite the standard Transformer’s superiority over LSTM-based models, it falls behind the OOA-CNN-BiLSTM-AT model.

As Table 6 illustrates, although the LSTM, a fundamental model for sequence prediction tasks in deep learning, demonstrates excellent predictive performance, it is not the best compared to other models. This is primarily because the standard LSTM model can only capture information sequentially, thus ignoring potential backward dependencies that may lead to an incomplete utilization of information in some complex sequence prediction tasks. The introduction of the attention mechanism improved the predictive accuracy of the LSTM model, reducing the MAPE to 9.35%, the RMSE to 0.0383, and increasing the $R^2$ to 0.9716. The addition of the attention mechanism allows the model to focus more on significant time steps within the sequence, but its core still relies on unidirectional information processing, which is limited in handling data scenarios with bidirectional dependencies. The introduction of Bidirectional LSTM (BiLSTM), capturing both forward and backward information in the sequence, further optimizes the results. However, BiLSTM may overfit in certain situations, and its high computational complexity limits its application in resource-constrained environments. The integration of Convolutional Neural Networks (CNN) improves the performance of the model due to its ability to capture local features. Although CNNs can effectively handle spatial dependencies in time series data, they overlook long-term dependencies. The introduction of the Osprey Optimization Algorithm (OOA) significantly enhanced the accuracy of the model predictions, thus overcoming the limitations of traditional optimization algorithms in high-dimensional and complex adaptive landscapes. The Transformer model, with its unique self-attention mechanism, can consider all elements in the time series simultaneously, providing a global view of time dependencies. Unlike sequence processing models like LSTM, which rely on recursive structures to gradually transmit information, Transformers can directly learn dependencies between any two points in the sequence. Additionally, the multi-head attention mechanism of the Transformer allows the model to handle multiple different representational subspaces at the same time, enabling it to capture complex patterns and relationships in the data more meticulously. The optimized Transformer model exhibits superior performance, further reducing the MAPE to 4.16%, the RMSE to 0.0152, and achieving an astonishing $R^2$ of 0.9955. The proposed OOA-Transformer model can be applied to wind speed prediction and has significantly improved forecast accuracy, thus yielding highly satisfactory results.

5. Conclusions

In this study, a short-term wind speed prediction model for the bridge site area, based on the OOA-Transformer, was proposed. This model effectively reduces the impact of random fluctuations in wind speed data and accurately predicts wind speed levels every ten minutes for the Xuefeng Lake Bridge. The predictive results indicate that, in terms of the MAPE, RMSE, and $R^2$ evaluation metrics, the proposed model outperforms the six models that were compared: Transformer, OOA-CNN-BiLSTM-AT, CNN-BiLSTM-AT, BiLSTM-AT, LSTM-AT, and LSTM.

In this research, the IQR method was successfully utilized to eliminate outliers caused by human errors or machine failures; among the four wavelets compared, the Bior2.2 wavelet with soft thresholding showed the best filtering performance with SNR, MSE, and NCC values at 22.9762, 0.4535, and 0.9931, respectively, thus effectively preserving significant features in the wind speed series. The Osprey Optimization Algorithm optimized parameters for the Transformer and CNN-BiLSTM-AT networks using the fitness function value to assess parameter quality.

Visually, the proposed wind speed prediction model (OOA-Transformer) achieves high accuracy, with MAPE scores at approximately 4%, RMSE at 0.0152, and $R^2$ at 0.9955, thus surpassing several state-of-the-art models. These findings are significant for accurately predicting wind speeds in the bridge site area of similar terrain cable-stayed bridges.

Our model was mainly validated using the wind speed data from the Xuefeng Lake Bridge. The Bior2.2 wavelet effectively captured the significant features of these wind speed data. However, its effectiveness needs to be further validated using different datasets to confirm its general applicability. In the future, we intend to expand our dataset by incorporating additional wind speed data, relevant meteorological variables, and seasonal factors. We will also evaluate the performance of the Bior2.2 wavelet on this extended dataset to assess its broader utility.