4.1. Accuracy Analysis
The WF-AI model is used to forecast Hs over the following 15 days in the eastern coastal waters of the United States. In addition, the forecasting performance of various deep learning methods, including ANN, RNN, LSTM, and Transformer, is compared.
For the current model, analysis shows that for forecasts of 1, 3, 6, and 10 days, the model achieves the highest accuracy only when the ECMWF data corresponding to the forecast time is used as input. For forecasts extending to 12 and 15 days, the inclusion of buoy data for the current time significantly enhances accuracy. Finally, the optimal model inputs for different forecast periods are summarised in
Table 3.
The
Hs forecast results of the WF-AI model for periods of 1, 3, 6, 10, 12, and 15 days at Buoy 41040 are analysed. The ECMWF dataset is compared [
42,
43]. The specific error calculations are presented in
Table 4 and
Figure 11.
In
Figure 11a–f illustrate the variation of model errors at different forecast lead times. For each forecast duration, the MAE, RMSE, MAPE, and r are displayed on the same scale to facilitate the comparison of error variations over time. The comparison indicates a gradual decline in forecasting accuracy for both the ECMWF and WF-AI models as the forecast lead time increases.
For a 1-day forecast, the model demonstrates high accuracy in wave prediction, with MAE/RMSE/MAPE values below 0.13 m/0.16 m/5.89%, respectively, and r exceeding 0.93. When the forecast lead time reaches 6 days, the MAE/RMSE/MAPE is less than 0.18 m/0.22 m/8.10%, respectively, while r exceeds 0.88. At this stage, the forecasting errors remain within 10%, indicating a high accuracy in predicting Hs.
A decrease in model accuracy occurs when the forecast lead time reaches 10 days. The MAPE of the ECMWF model is 13.08%. The accuracy of the WF-AI model is slightly better than that of the ECMWF, with minimum values of MAE/RMSE/MAPE reaching 0.26 m/0.33 m/11.88%, respectively, and an r value of 0.72.
A noticeable decline in forecasting accuracy occurs when the forecast lead time reaches 12 days. The MAPE for the ECMWF and WF-AI models is 17.13% and 14.75%, respectively. At a 15-day forecast, the ECMWF model exhibits poor accuracy for Hs, with MAE/RMSE/MAPE/r values of 0.49 m/0.61 m/22.52%/0.33.
In contrast, the WF-AI model shows improved accuracy compared to ECMWF, with MAE/RMSE/MAPE/r values of 0.36 m/0.44 m/16.32%/0.35. It can be observed that as the forecasting time increases, the prediction accuracy of both the ECMWF and WF-AI models declines.
Both models exhibit relatively good accuracy for lead times of 1–6 days. In the 10–15-day forecasts, the accuracy of both models decreases; however, the WF-AI model demonstrates better performance than the ECMWF.
Furthermore, a comparison of the wave forecasting results between the WF-AI model and ECMWF is presented in
Figure 12. The forecasting error analysis indicates that both models exhibit high forecasting accuracy for lead times of 1–6 days, with the WF-AI model demonstrating slightly higher accuracy than the ECMWF. In this period, the MAPE for both models remains below 10%.
The differences in forecast accuracy between the two models become gradually apparent when the forecast time exceeds 10 days. The forecast errors for both models increase with the length of the forecast period. However, the WF-AI model accurately predicts Hs more accurately than ECMWF. When the forecast lead time reaches 15 days, the WF-AI model achieves a MAPE of 16.32%, better than the ECMWF’s 22.52%.
Additionally, the differences in wave forecasting results of the WF-AI model under various deep learning methods are presented, as shown in
Figure 13. The comparison reveals that the RNN, LSTM, and Transformer methods achieve comparable accuracy in wave forecasting, as measured by MAE, RMSE, and MAPE evaluation metrics.
The ANN method demonstrates slightly better forecasting accuracy in the early stages than other deep learning approaches. It was noted that the ANN model ignores the temporal correlations among data, which influences the long-term wave forecasts. It can be seen that the overall accuracy of the ANN method remains better than the ECMWF method, and the r of all models gradually decreases with increasing forecast time, indicating a weakening correlation between wave forecast results and the observed buoy values over time.
Overall, the WF-AI model demonstrates better accuracy in predicting Hs than the ECMWF forecast, particularly in medium- to long-term forecasts, with lower errors and suggesting improved stability and accuracy in Hs prediction tasks.
Further analysis of the correlation between the WF-AI model and ECMWF wave forecasting results, using buoy measurements, is conducted, as illustrated in
Figure 14. The comparison indicates that the correlation between the wave prediction results and the true values gradually weakens as the forecasting time increases. However, the correlation for the WF-AI model predictions in the right column remains superior to the ECMWF forecasting results on the left, particularly evident in the 10–15 day forecast range.
The
Hs forecasting results of the WF-AI model are compared with buoy data in
Figure 15 for different forecasting durations. The variations in model-predicted
Hs for 1, 3, 6, 10, 12, and 15 days are illustrated in
Figure 15.
It can be observed that the WF-AI model effectively forecasts wave conditions for the next 15 days. The accuracy is notably high for the 1–6 day forecasts, with the model predictions aligning closely with buoy measurements. The forecast accuracy for the 10–15 day range is slightly lower than that of the earlier period, with MAPE values of 11.88%, 14.75%, and 16.32% for the 10-, 12-, and 15-day forecasts, respectively. However, these values are more accurate compared to ECMWF predictions.
Finally, the forecasting performance of the model under extreme sea states is analysed. The prediction skill of ECMWF and the WF-AI model for sea states 6 and 7 over the 1–15 day forecast period is presented in
Table 5. In addition, the results are evaluated using the mean, standard deviation, and 95% confidence intervals of these differences. The specific calculation methods are as follows:
In Equations (20)–(23), δHs represents the absolute difference between the model calculations and the buoy observations. represents the sample mean, s represents the sample standard deviation, and CI represents the 95% confidence interval. Xi represents the sample data, and n represents the sample size.
A comparison reveals that under extreme sea conditions, the model outperforms the ECMWF in terms of accuracy. For forecasts of the next 10, 12, and 15 days, the WF-AI model achieves a MAPE not exceeding 13.16% for sea state 6. Specifically, the ECMWF model shows δHs = 0.30 ± 0.22 (±s), CI = (0.271, 0.326); δHs = 0.34 ± 0.25, CI = (0.311, 0.373); and δHs = 0.42 ± 0.33, CI = (0.380, 0.462) for the 10-, 12-, and 15-day forecasts, respectively. In comparison, the WF-AI model demonstrates δHs = 0.29 ± 0.19, CI = (0.261, 0.310); δHs = 0.29 ± 0.21, CI = (0.260, 0.313); and δHs = 0.32 ± 0.23, CI = (0.300, 0.353) for the same forecast periods, respectively. However, for sea state 7, the model’s forecasting accuracy is relatively poorer. Specifically, the ECMWF model shows δHs = 0.29 ± 0.26, CI = (0.253, 0.319); δHs = 0.50 ± 0.34, CI = (0.458, 0.543); and δHs = 0.49 ± 0.33, CI = (0.451, 0.534) for the 10-, 12-, and 15-day forecasts, respectively. In comparison, the WF-AI model shows δHs = 0.49 ± 0.24, CI = (0.454, 0.515); δHs = 0.58 ± 0.32, CI = (0.535, 0.616); and δHs = 0.55 ± 0.33, CI = (0.510, 0.592) for the same forecast periods, respectively. This discrepancy is primarily due to the significantly reduced number of sea state 7 occurrences, leading to fewer training samples. Therefore, there is still room for improvement in the model’s forecasting accuracy for extreme weather conditions.
The WF-AI model effectively enhances the forecasting accuracy of waves over long periods, showing improvements compared to ECMWF. Nevertheless, the current model still faces certain limitations in its application. While the model improves the accuracy of existing wave forecasting methods for long periods, discrepancies remain when compared to actual wave values. As illustrated above in
Figure 15f, there is still potential for improvement, particularly in forecasting extreme wave conditions. Additionally, the WF-AI model relies on existing wave forecast data and observational results for forecasting. As a result, the forecast accuracy is influenced by outcomes such as those from ECMWF IFS. Additionally, when observational data is missing, it can also affect the model’s forecasting performance to some extent.