Assessment of the ZJWARMS Forecast Model’s Adaptability and AI-Based Bias Correction over Complex Terrain
Abstract
1. Introduction
2. Data and Methodology
2.1. Data Sources
2.2. Correction Method Overview
2.2.1. XGBoost
2.2.2. Correction Model
- 1.
- Data Preprocessing:To correct the model outputs, the ZJWARMS model data is first spatiotemporally matched with observational data from meteorological stations, thereby generating a dataset in which each model prediction corresponds to an observed value.
- 2.
- Feature Selection:Based on the spatiotemporally matched dataset, the correlation between each observed meteorological variable (e.g., air temperature, 10 m wind speed, wind direction, daily maximum and minimum temperatures, etc.) and the target variables for correction (2 m air temperature, 10 m wind speed) is statistically analyzed. Observed variables with strong correlations are selected as input features for the correction model. Additionally, the uncorrected model outputs (2 m air temperature and 10 m wind speed) are also included as input features. The target labels of the correction model are the observed 2 m air temperature and 10 m wind speed.
- 3.
- Dataset Partitioning:The first 80% of the dataset is used for model training and parameter tuning, while the remaining 20% is reserved for evaluating the correction performance. To respect temporal dependence and avoid information leakage, no shuffling was performed. Within the 80% training portion, hyperparameters were tuned using a chronologically ordered validation split (i.e., the last segment of the training period), rather than random K-fold cross-validation.
- 4.
- Model Construction and Parameter Optimization:The XGBoost algorithm is implemented using the xgb library, with the objective function defined as the root mean square error (RMSE). Bayesian optimization is employed to determine the optimal values for key hyperparameters, including maximum tree depth (max_depth), number of trees (n_estimators), learning rate (learning_rate), subsample ratio (subsample), and feature subsample ratio (colsample_bytree), in order to obtain the best-performing correction model. Overfitting control relies on XGBoost’s built-in regularization and stochasticity (e.g., lambda/alpha, max_depth, min_child_weight, subsample, colsample_bytree), and all tuning uses the above time-ordered validation split.
- 5.
- Meteorological Element Correction:The optimized correction model is applied to the test set inputs to produce corrected (standardized) values of 2-m air temperature and 10-m wind speed, which are subsequently transformed back to the original scale through inverse standardization.
- 6.
- Error Evaluation:The correction results are assessed using various evaluation metrics, including error (E), RMSE, MAE, and accuracy. Here, “accuracy” denotes a tolerance-based hit rate: a forecast is counted as accurate when the absolute error for 2 m temperature is within 2 °C and the absolute error for 10 m wind speed is within 2 m/s. These thresholds were fixed in advance based on (i) approximate observational/representativeness uncertainty at the stations, (ii) the resolution and decision needs of user-facing products, and (iii) common practice in station-level post-processing. To facilitate international comparisons, we additionally report RMSE, MAE, and skill scores relative to the raw ZJWARMS forecasts.
2.2.3. Forecast Error Analysis and Evaluation Methods of the Model
3. Results and Analysis
3.1. Evaluation of ZJWARMS Forecasts and AI-Corrected Products
3.1.1. Temporal Evolution of Temperature Forecast Accuracy
3.1.2. Temporal Evolution of Wind Speed Forecast Accuracy
3.2. Error Distribution and Correction Effect Under Forecast Accuracy Extremes
3.3. Evolution of Extreme Error Samples and Evaluation of Correction Effectiveness
4. Conclusions and Discussion
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
MAE | Mean Absolute Error |
NWP | Numerical Weather Prediction |
AI | Artificial Intelligence |
XGBoost | Extreme Gradient Boosting |
RMSE | Root Mean Square Error |
References
- Xue, W.; Yu, H.; Tang, S.; Huang, W. Relationships between terrain features and forecasting errors of surface wind speeds in a mesoscale numerical weather prediction model. Adv. Atmos. Sci. 2024, 41, 1161–1170. [Google Scholar] [CrossRef]
- Zhou, S.; Gao, C.Y.; Duan, Z.; Xi, X.; Li, Y. A robust error correction method for numerical weather prediction wind speed based on Bayesian optimization, variational mode decomposition, principal component analysis, and random forest: VMD-PCA-RF (version 1.0.0). Geosci. Model Dev. 2023, 16, 6247–6266. [Google Scholar] [CrossRef]
- Liu, C.; Sun, J.; Yang, X.; Jin, S.; Fu, S. Evaluation of ECMWF precipitation predictions in China during 2015–18. Weather. Forecast. 2021, 36, 1043–1060. [Google Scholar] [CrossRef]
- Barthlott, C.; Zarboo, A.; Matsunobu, T.; Keil, C. Impacts of combined microphysical and land-surface uncertainties on convective clouds and precipitation in different weather regimes. Atmos. Chem. Phys. 2022, 22, 10841–10860. [Google Scholar] [CrossRef]
- Rotach, M.W.; Adams, K.; Adler, B.; Cermak, J.; Gohm, A.; Serafin, S. A collaborative effort to better understand, measure, and model atmospheric exchange processes over mountains (TEAMx). Bull. Am. Meteorol. Soc. 2022, 103, E1282–E1295. [Google Scholar] [CrossRef]
- Lagerquist, R.; McGovern, A.; Gagne II, D.J. Deep learning for spatially explicit prediction of synoptic-scale fronts. Weather. Forecast. 2019, 34, 1137–1160. [Google Scholar] [CrossRef]
- Houtekamer, P.L.; Zhang, F. Review of the ensemble Kalman filter for atmospheric data assimilation. Mon. Weather. Rev. 2016, 144, 4489–4532. [Google Scholar] [CrossRef]
- Bannister, R.N. A review of operational methods of variational and ensemble-variational data assimilation. Q. J. R. Meteorol. Soc. 2017, 143, 607–633. [Google Scholar] [CrossRef]
- Carrassi, A.; Bocquet, M.; Bertino, L.; Evensen, G. Data assimilation in the geosciences: An overview of methods, issues, and perspectives. Wiley Interdiscip. Rev. Clim. Change 2018, 9, e535. [Google Scholar] [CrossRef]
- Di, Z.; Duan, Q.; Shen, C.; Xie, Z. Improving WRF typhoon precipitation and intensity simulation using a surrogate-based automatic parameter optimization method. Atmosphere 2020, 11, 89. [Google Scholar] [CrossRef]
- Vannitsem, S.; Bremnes, J.B.; Demaeyer, J.; Evans, G.R.; Flowerdew, J.; Hemri, S.; Lerch, S.; Roberts, N.; Theis, S.; Atencia, A.; et al. Statistical postprocessing for weather forecasts: Review, challenges, and avenues in a big data world. Bull. Am. Meteorol. Soc. 2021, 102, E681–E699. [Google Scholar] [CrossRef]
- Glahn, H.R.; Lowry, D.A. The use of model output statistics (MOS) in objective weather forecasting. J. Appl. Meteorol. 1972, 11, 1203–1211. [Google Scholar] [CrossRef]
- Dueben, P.D.; Bauer, P. Challenges and design choices for global weather and climate models based on machine learning. Geosci. Model Dev. 2018, 11, 3999–4009. [Google Scholar] [CrossRef]
- Boukabara, S.A.; Krasnopolsky, V.; Stewart, J.Q.; Maddy, E.S.; Shahroudi, N.; Hoffman, R.N. Leveraging modern artificial intelligence for remote sensing and NWP: Benefits and challenges. Bull. Am. Meteorol. Soc. 2019, 100, ES473–ES491. [Google Scholar] [CrossRef]
- Cho, D.; Yoo, C.; Im, J.; Cha, D.H. Comparative assessment of various machine learning-based bias correction methods for numerical weather prediction model forecasts of extreme air temperatures in urban areas. Earth Space Sci. 2020, 7, e2019EA000740. [Google Scholar] [CrossRef]
- Li, H.; Yu, C.; Xia, J.; Wang, Y.; Zhu, J.; Zhang, P. A model output machine learning method for grid temperature forecasts in the Beijing area. Adv. Atmos. Sci. 2019, 36, 1156–1170. [Google Scholar] [CrossRef]
- Rasp, S.; Lerch, S. Neural networks for postprocessing ensemble weather forecasts. Mon. Weather. Rev. 2018, 146, 3885–3900. [Google Scholar] [CrossRef]
- Pang, C.; Song, T.; Sun, H.; Li, X.; Xu, D. A deep learning method for bias correction of wind field in the South China Sea. Front. Mar. Sci. 2024, 11, 1429057. [Google Scholar] [CrossRef]
- Han, L.; Chen, M.; Chen, K.; Chen, H.; Zhang, Y.; Lu, B.; Song, L.; Qin, R. A deep learning method for bias correction of ECMWF 24–240 h forecasts. Adv. Atmos. Sci. 2021, 38, 1444–1459. [Google Scholar] [CrossRef]
- Chang, J.; Peng, X.; Fan, G.; Che, Y. Error correction of numerical weather prediction with historical data. Acta Meteorol. Sin. 2015, 73, 341–354. (In Chinese) [Google Scholar]
- Xia, J.; Li, H.; Kang, Y.; Yu, C.; Ji, L.; Wu, L.; Lou, X.; Zhu, G.; Wang, Z.; Yan, Z.; et al. Machine learning-based weather support for the 2022 Winter Olympics. Adv. Atmos. Sci. 2020, 37, 927–932. [Google Scholar] [CrossRef]
Station Name | Longitude | Latitude | Elevation (m) | Model Data Period |
---|---|---|---|---|
Octagonal Palace | 119.13° | 28.79° | 273 | 2021.12.10–2022.12.26 |
Zheyuanli | 119.08° | 28.79° | 608 | 2021.12.06–2022.12.26 |
Mountainside | 119.08° | 28.78° | 903 | 2022.06.08–2022.12.25 |
Mountaintop | 119.07° | 28.75° | 1327 | 2022.02.23–2022.12.26 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhang, Q.; Shi, Y.; Wang, Y.; Mou, S.; Zhu, Z.; Qian, T.; Mao, Z.; Yuan, S.; Han, L.; Lao, X. Assessment of the ZJWARMS Forecast Model’s Adaptability and AI-Based Bias Correction over Complex Terrain. Atmosphere 2025, 16, 1151. https://doi.org/10.3390/atmos16101151
Zhang Q, Shi Y, Wang Y, Mou S, Zhu Z, Qian T, Mao Z, Yuan S, Han L, Lao X. Assessment of the ZJWARMS Forecast Model’s Adaptability and AI-Based Bias Correction over Complex Terrain. Atmosphere. 2025; 16(10):1151. https://doi.org/10.3390/atmos16101151
Chicago/Turabian StyleZhang, Qi, Yiwen Shi, Yifan Wang, Shiyun Mou, Zhidan Zhu, Tu Qian, Zhijun Mao, Shujie Yuan, Lin Han, and Xiaocan Lao. 2025. "Assessment of the ZJWARMS Forecast Model’s Adaptability and AI-Based Bias Correction over Complex Terrain" Atmosphere 16, no. 10: 1151. https://doi.org/10.3390/atmos16101151
APA StyleZhang, Q., Shi, Y., Wang, Y., Mou, S., Zhu, Z., Qian, T., Mao, Z., Yuan, S., Han, L., & Lao, X. (2025). Assessment of the ZJWARMS Forecast Model’s Adaptability and AI-Based Bias Correction over Complex Terrain. Atmosphere, 16(10), 1151. https://doi.org/10.3390/atmos16101151