Evaluation of Rainfall Forecasts by Three Mesoscale Models during the Mei-yu Season of 2008 in Taiwan. Part II: Development of an Object-Oriented Method
Abstract
:1. Introduction
2. Data
2.1. Observation Data
2.2. Model Data
3. Development and Methodology
3.1. Identification of Mesoscale Rainfall Objects
3.2. Attribute Parameters of Rainfall Objects
3.3. Matching Procedure for Rainfall Objects
3.3.1. Matching between Observed and Modeled Objects Occurring at the Same Time
3.3.2. Matching of Objects between Successive Times in the Same Dataset
3.3.3. Matching between Observed and Modeled Mesoscale Rainfall Systems
4. Assessment of the Developed Verification Method
4.1. Evaluation on Object Properties without Matching
4.2. Evaluation on the Properties of Matched Object Pairs
4.3. Evaluation on the Properties of MRSs in the Observation and Model Forecasts
5. Discussion and Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Fritsch, J.M.; Hauze, R.A., Jr.; Adler, R.; Bluestein, H. Quantitative precipitation forecasting: Report of the Eighth Prospectus Development Team, U.S. Weather Research Program. Bull. Am. Meteorol. Soc. 1998, 79, 285–299. [Google Scholar] [CrossRef]
- Golding, B.W. Quantitative precipitation forecasting in the UK. J. Hydrol. 2000, 239, 286–305. [Google Scholar] [CrossRef]
- Schaefer, J.T. The critical success index as an indicator of warning skill. Weather Forecast 1990, 5, 570–575. [Google Scholar] [CrossRef] [Green Version]
- Doswell, C.A.; Davies-Jones, R.; Keller, D.K. On summary measures of skills in rare event forecasting based on contingency tables. Weather Forecast 1990, 5, 576–585. [Google Scholar] [CrossRef] [Green Version]
- Mesinger, F.; Black, T.L. On the impact on forecast accuracy of the step-mountain (eta) vs. sigma coordinate. Meteorol. Atmos. Phys. 1992, 50, 47–60. [Google Scholar] [CrossRef]
- Wilks, D.S. Statistical Methods in the Atmospheric Sciences; Academic Press: Cambridge, MA, USA, 1995; p. 704. [Google Scholar]
- Chien, F.C.; Kuo, Y.H.; Yang, M.J. Precipitation forecast of MM5 in the Taiwan area during the 1998 Mei-yu season. Weather Forecast 2002, 17, 739–754. [Google Scholar] [CrossRef]
- Chien, F.C.; Jou, B.J.D. MM5 ensemble mean precipitation forecast in the Taiwan area for three early summer convective (Mei-Yu) seasons. Weather Forecast 2004, 19, 735–750. [Google Scholar] [CrossRef]
- Murphy, A.H. Forecast verification: Its complexity and dimensionality. Mon. Weather Rev. 1991, 119, 1590–1601. [Google Scholar] [CrossRef]
- Olson, D.A.; Junker, N.W.; Korty, B. Evaluation of 33 years of quantitative precipitation forecasting at the NMC. Weather Forecast 1995, 10, 498–511. [Google Scholar] [CrossRef] [Green Version]
- Chen, G.T.J. On the forecast skill of heavy rainfall in Taiwan. Atmos. Sci. 1991, 19, 188. [Google Scholar]
- Hong, J.S. Evaluation of the high-resolution model forecasts over the Taiwan area during GIMEX. Weather Forecast 2003, 18, 836–846. [Google Scholar] [CrossRef]
- Chien, F.C.; Liu, Y.C.; Jou, B.J.D. MM5 ensemble mean forecasts in the Taiwan area for the 2003 Mei-Yu season. Weather Forecast 2006, 21, 1006–1023. [Google Scholar] [CrossRef] [Green Version]
- Brooks, H.E.; Doswell, C.A., III. A comparison of measures-oriented and distributions-oriented approaches to forecast verification. Weather Forecast 1996, 11, 288–303. [Google Scholar] [CrossRef]
- Ebert, E.E.; McBride, J.L. Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol. 2000, 239, 179–202. [Google Scholar] [CrossRef]
- Roberts, N.M.; Lean, H.W. Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Weather Rev. 2008, 136, 78–97. [Google Scholar] [CrossRef]
- Mass, C.F.; Ovens, D.; Westrick, K.; Colle, B.A. Does increasing horizontal resolution produce more skillful forecasts? Bull. Am. Meteorol. Soc. 2002, 83, 407–430. [Google Scholar] [CrossRef]
- Brown, B.G.; Bullock, R.; Davis, C.; Manning, K.; Morss, R.; Mueller, C. An object-based diagnostic approach for QPF and convective forecast verification. In Proceedings of the Workshop on Making Verification More Meaningful, Boulder, CO, USA, 30 July–1 August 2002. [Google Scholar]
- Davis, C.; Brown, B.; Bullock, R. Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Weather Rev. 2006, 134, 1772–1784. [Google Scholar] [CrossRef] [Green Version]
- Ebert, E.E. Verifying satellite precipitation estimates for weather and hydrological applications. In Proceedings of the 1st International Precipitation Working Group (IPWG) Workshop, Madrid, Spain, 23–27 September 2002. [Google Scholar]
- Kain, J.S.; Baldwin, M.E.; Janish, P.R.; Weiss, S.J.; Kay, M.P.; Carbin, G.W. Subjective verification of numerical models as a component of a broader interaction between research and operations. Weather Forecast 2003, 18, 847–860. [Google Scholar] [CrossRef] [Green Version]
- Knievel, J.C.; Ahijevych, D.A.; Manning, K.W. Using temporal modes of rainfall to evaluate the performance of a numerical weather prediction model. Mon. Weather Rev. 2004, 132, 2995–3009. [Google Scholar] [CrossRef] [Green Version]
- Wang, C.C.; Chien, F.C.; Paul, S.; Lee, D.I.; Chuang, P.Y. An evaluation of WRF rainfall forecasts in Taiwan during three Mei-yu seasons of 2008–2010. Weather Forecast 2017, 32, 1329–1351. [Google Scholar] [CrossRef]
- Paul, S.; Wang, C.C.; Tseng, L.S.; Lee, D.I. Evaluation of rainfall forecasts by three mesoscale models during the Mei-yu season of 2008 in Taiwan. Part I: Subjective comparison. Asia Pac. J. Atmos. Sci. under review.
- Davis, C.; Brown, B.; Rullock, R. Object-based verification of precipitation forecasts. Part II: Application to convective rain systems. Mon. Weather Rev. 2006, 134, 1785–1795. [Google Scholar] [CrossRef] [Green Version]
- Ebert, E.E. Fuzzy verification in high resolution gridded forecasts: A review and proposed framework. Meteorol. Appl. 2008, 15, 51–64. [Google Scholar] [CrossRef]
- Gilleland, E.; Lindström, J.; Lindgren, F. Analyzing the image warp forecast verification method on precipitation fields from the ICP. Weather Forecast 2010, 25, 1249–1262. [Google Scholar] [CrossRef]
- Wang, C.C. On the calculation and correction of equitable threat score for model quantitative precipitation forecasts for small verification areas: The example of Taiwan. Weather Forecast 2014, 29, 788–798. [Google Scholar] [CrossRef]
- Cortesi, N.; Gonzalez-Hidalgo, J.C.; Trigo, R.M.; Ramos, A.M. Weather types and spatial variability of precipitation in the Iberian Peninsula. Int. J. Clim. 2014, 34, 2661–2677. [Google Scholar] [CrossRef]
- Foufoula-Georgiou, E.; Vuruputur, V. Patterns and organization in precipitation. In Spatial Patterns in Catchment Hydrology-Observations and Modelling; Grayson, R., Blöschl, G., Eds.; Cambridge University Press: Cambridge, UK, 2001; pp. 82–104. [Google Scholar]
- Jolliffe, I.T.; Stephenson, D.B. Forecast. Verification: A Practitioner’s Guide in Atmospheric Science; Wiley and Sons: Hoboken, NJ, USA, 2003; p. 240. [Google Scholar]
- Wang, C.C. The more rain, the better the model performs—The dependency of quantitative precipitation forecast skill on rainfall amount for typhoons in Taiwan. Mon. Weather Rev. 2015, 143, 1732–1748. [Google Scholar] [CrossRef]
- Shapiro, M.A.; Thorpe, A.J. The observing system research and predictability experiment (THORpex). In Proceedings of the International Conference on Mesoscale Convective Systems and Heavy Rainfall/Snowfall in East Asia, Tokyo, Japan, 29–31 October 2002; pp. 1–12. [Google Scholar]
- Fritsch, M.J.; Carbone, R.E. Improving quantitative precipitation forecasts in the warm season: A USWRP research and development strategy. Bull. Am. Meteorol. Soc. 2004, 85, 955–965. [Google Scholar] [CrossRef] [Green Version]
- Ebert, E.E.; Gallus, W.A., Jr. Toward better understanding of the Contiguous Rain Area (CRA) method for spatial forecast verification. Weather Forecast 2009, 24, 1401–1415. [Google Scholar] [CrossRef] [Green Version]
- Bytheway, J.L.; Kummerow, C.D. Toward an object-based assessment of high-resolution forecasts of long-lived convective precipitation in the central U.S. JAMES 2015, 7, 1248–1264. [Google Scholar] [CrossRef]
- Giannakaki, P.; Martius, O. An object-based forecast verification tool for synoptic-scale Rossby waveguides. Weather Forecast 2016, 31, 937–946. [Google Scholar] [CrossRef] [Green Version]
- Jou, B.J.D.; Lee, W.C.; Johnson, R.H. An overview of SoWMEX/TiMREX. In The Global Monsoon System: Research and Forecasts, 2nd ed.; Chang, C.-P., Ed.; World Scientific: Singapore, 2011; pp. 303–318. [Google Scholar] [CrossRef] [Green Version]
- Hsu, J. ARMTS up and running in Taiwan. Väisälä News 1998, 146, 24–26. [Google Scholar]
- Chen, T.C.; Yen, M.C.; Hsieh, J.C.; Arritt, R.W. Diurnal and seasonal variations of the rainfall measured by the Automatic Rainfall and Meteorological Telemetry System in Taiwan. Bull. Am. Meteorol. Soc. 1999, 80, 2299–2312. [Google Scholar] [CrossRef] [Green Version]
- Huffman, G.J.; Bolvin, D.T.; Nelkin, E.J.; Wolff, D.B.; Adler, R.F.; Gu, G.; Hong, Y.; Bowman, K.P.; Stocker, E.F. The TRMM Multisatellite Precipitation Analysis (TMPA): Quasi-global, multiyear, combined-sensor precipitation estimates at fine scales. J. Hydrometerol. 2007, 8, 38–55. [Google Scholar] [CrossRef]
- Cressman, G.P. An operational objective analysis system. Mon. Weather Rev. 1959, 87, 367–374. [Google Scholar] [CrossRef]
- Tsuboki, K.; Sakakibara, A. Large-scale parallel computing of cloud resolving storm simulator. In High Performance Computing; Zima, H.P., Joe, K., Sato, M., Seo, Y., Shimasaki, M., Eds.; Springer: Berlin, Germany, 2002; pp. 243–259. [Google Scholar]
- Tsuboki, K.; Sakakibara, A. Numerical prediction of high-impact weather systems. In The Textbook for Seventeenth IHP Training Course; HyARC, Nagoya University and UNESCO: Nagoya, Japan, 2007; p. 273. [Google Scholar]
- Skok, G.; Tribbia, J.; Rakovec, J.; Brown, B. Object-based analysis of satellite-derived precipitation systems over the Low- and midlatitude Pacific Ocean. Mon. Weather Rev. 2009, 137, 3196–3218. [Google Scholar] [CrossRef]
- Davis, C.A.; Brown, B.G.; Bullock, R.; Halley-Gotway, J. The method for object-based diagnostic evaluation (MODE) applied to numerical forecasts from the 2005 NSSL/SPC spring program. Weather Forecast 2009, 24, 1252–1267. [Google Scholar] [CrossRef] [Green Version]
- Gallus, W.A., Jr. Application of object-based verification techniques to ensemble precipitation forecasts. Weather Forecast 2010, 25, 144–158. [Google Scholar] [CrossRef] [Green Version]
- Ritter, G.X.; Wilson, J.N. Computer Vision Algorithms in Image Algebra; CRC Press: Boca Raton, FL, USA, 2001; p. 417. [Google Scholar]
- Wang, C.C.; Paul, S.; Lee, D.I. Evaluation of rainfall forecasts by three mesoscale models during the Mei-yu Season of 2008 in Taiwan. Part III: Application of an object-oriented verification method. Atmosphere 2020, 11, 705. [Google Scholar] [CrossRef]
- Ruppert, J.H., Jr.; Johnson, R.H.; Rowe, A.K. Diurnal circulations and rainfall in Taiwan during SoWMEX/TiMREX (2008). Mon. Weather Rev. 2013, 141, 3851–3872. [Google Scholar] [CrossRef] [Green Version]
- Hsiao, L.F.; Peng, M.S.; Chen, D.S.; Huang, K.N.; Yeh, T.C. Sensitivity of typhoon track predictions in a regional prediction system to initial and lateral boundary conditions. J. Appl. Meteorol. Clim. 2009, 48, 1913–1928. [Google Scholar] [CrossRef]
- Wang, C.C.; Huang, W.M. High-resolution simulation of a nocturnal narrow convective line off the southeastern coast of Taiwan in the Mei-yu season. Geophys. Res. Lett. 2009, 36, 1–5. [Google Scholar] [CrossRef] [Green Version]
Model | CReSS (version 2.2) |
---|---|
Grid dimension (x, y, z) | 330 × 280 × 40 |
Grid spacing (km) | 3.5 × 3.5 × 0.5 (stretched over 0.1–0.663 km in vertical) |
Domain size (km) | 1155 × 980 × 20 (centered at 23.5° N, 119.0° E) |
Projection | Lambert conformal (120° E, secant at 10° and 40° N) |
IC/BCs | NCEP GFS analyses/forecasts (1° × 1°, 26 levels, every 3 h) |
Topography and SST | Real at (1/120)° and NCEP analyses at 1° × 1° |
Initial times (daily) | 0000, 1200, and 1800 UTC (0800, 2000, and 0200 LST) |
Forecast length | 48 h |
Output frequency | Every 15 min. |
Cloud microphysics | Bulk cold-rain scheme (six species) |
Planetary boundary layer | 1.5-order closure with turbulent kinetic energy prediction |
parameterization | |
Surface processes | Energy/momentum fluxes and shortwave/longwave radiation |
Substrate soil model | 41 levels, every 5 cm to 2-m deep |
Type of Matching | Type 1 | Type 2 | Type 3 | Equation | |
---|---|---|---|---|---|
Round in Procedure | Rounds 1/2 | Round 3 | |||
Attribute/parameter | |||||
Centroid distance | 4 * | 4 *,@ | 4 * | 4 * | (1), (2) or (5) $ |
Area size | 2 | 2 | 2 | 2 | (4) or (6) $ |
Total water production | 1 | 1 | 1 | 1 | (4) or (6) $ |
Mean rainfall | 1 | 1 | 1 | (4) | |
Maximum rainfall | 1 | 1 | 1 | (4) | |
Aspect ratio | 1 | 1 | 1 | (4) | |
Long-axis orientation | 1 | 1 | 1 | (3) | |
Curvature | 1 | 1 | 1 | (4) | |
Mid-point difference in lifespan | 4 # | (7) | |||
Lifetime duration | 4 | (4) | |||
Maximum total score | 12 | 12 | 7 | 20 | |
Criteria for Matching | |||||
Round 1 | ≥9 ^ | ≥10 ^ | ≥16 ^ | ||
Round 2 | ≥6 | ≥7 | ≥12 | ||
Round 3 | ≥3.5 |
Parameter | Observation | CReSS | ||
---|---|---|---|---|
Mean | SD | Mean | SD | |
Total number of objects | 6081 | 8642 | ||
Total water production (106 ton) | 157.2 | 383.7 | 174.5 | 380.5 |
Area size (km2) | 7494 | 13,041.1 | 5875.5 | 11,708.2 |
Mean rainfall (mm) | 3.9 | 2.6 | 23.8 | 9 |
Maximum rainfall (mm) | 29 | 21.3 | 84.1 | 75.6 |
90-percentile of rainfall (mm) | 4.6 | 2.7 | 49.6 | 29.9 |
75-percentile of rainfall (mm) | 4.2 | 2.7 | 32.4 | 14.2 |
50-percentile of rainfall (mm) | 3.6 | 2.4 | 18.7 | 4.6 |
25-percentile of rainfall (mm) | 2.3 | 2.2 | 11.6 | 3.2 |
10-percentile of rainfall (mm) | 2 | 3 | 8.1 | 4 |
Centroid longitude (° E) | 118.24 | 3.45 | 118.37 | 2.95 |
Centroid latitude (° N) | 23.94 | 2.33 | 24.68 | 1.94 |
Long-axis orientation (°) | 13.6 | 49.2 | 17.3 | 42.9 |
Long-axis length (km) | 118.1 | 133.3 | 114.8 | 128.4 |
Short-axis length (km) | 44.1 | 38.2 | 34.4 | 29.6 |
Aspect ratio | 2.5 | 1.1 | 3 | 1.71 |
Curvature (10−2) | 0.93 | 1.87 | 0.6 | 1.43 |
Number of sub-centers | 0.24 | 0.65 | 0.81 | 1.16 |
Initial Time | 0000 UTC | 1200 UTC | ||
---|---|---|---|---|
Parameter | Observation | CReSS | Observation | CReSS |
Total number of matched objects | 309 | 309 | 295 | 295 |
Total water production (106 ton) | 411.3 | 531.6 | 412.9 | 475.8 |
Area size (km2) | 21,573.2 | 16,873.6 | 22,033.3 | 15,255.2 |
Mean rainfall (mm) | 16.3 | 27.8 | 16.2 | 28.1 |
Maximum rainfall (mm) | 37.8 | 131.2 | 36.9 | 130.7 |
90-percentile of rainfall (mm) | 24 | 60.1 | 23.9 | 61.2 |
75-percentile of rainfall (mm) | 19.5 | 36.9 | 19.4 | 37.3 |
50-percentile of rainfall (mm) | 15.2 | 20.2 | 15.1 | 20.2 |
25-percentile of rainfall (mm) | 12.2 | 11.9 | 12.2 | 11.9 |
10-percentile of rainfall (mm) | 10.5 | 8 | 10.4 | 8.1 |
Centroid longitude (° E) | 117.94 | 118.11 | 117.64 | 117.77 |
Centroid latitude (° N) | 24.18 | 24.36 | 24.35 | 24.53 |
Long-axis orientation (°) | 16.5 | 25 | 14.9 | 23.5 |
Long-axis length (km) | 216.7 | 250.5 | 223.4 | 214.7 |
Short-axis length (km) | 73.1 | 61 | 72.4 | 58.9 |
Aspect ratio | 2.8 | 4.1 | 2.9 | 3.7 |
Curvature (10−2) | 0.38 | 0.25 | 0.37 | 0.2 |
Number of sub-centers | 0.58 | 1.73 | 0.57 | 1.49 |
Initial Time | 0000 UTC | 1200 UTC | ||
---|---|---|---|---|
Parameter | Observation | CReSS | Observation | CReSS |
Total number of MRSs | 607 | 868 | 631 | 756 |
Total water production (106 ton) | 103.3 | 115.5 | 102.9 | 99.1 |
Area size (km2) | 6008.5 | 4046.8 | 5894.6 | 3470.6 |
Mean rainfall (mm) | 13.9 | 21.9 | 13.9 | 21.6 |
Maximum rainfall (mm) | 25.2 | 69.1 | 25.5 | 65 |
90-percentile of rainfall (mm) | 19 | 43.5 | 19.2 | 42.5 |
75-percentile of rainfall (mm) | 16.2 | 29.3 | 16.3 | 29.1 |
50-percentile of rainfall (mm) | 13.3 | 17.9 | 13.4 | 17.9 |
25-percentile of rainfall (mm) | 11.4 | 11.8 | 11.4 | 11.9 |
10-percentile of rainfall (mm) | 10.2 | 8.7 | 10.1 | 8.8 |
Centroid longitude (° E) | 118.19 | 118.42 | 118.16 | 118.41 |
Centroid latitude (° N) | 24.06 | 24.6 | 24.12 | 24.64 |
Long-axis orientation (°) | 12.7 | 17.8 | 13.4 | 18.3 |
Long-axis length (km) | 94.5 | 92.5 | 93.5 | 82.8 |
Short-axis length (km) | 36.3 | 28.2 | 35.5 | 26.8 |
Aspect ratio | 2.4 | 2.9 | 2.4 | 2.8 |
Curvature (10−2) | 1.14 | 0.77 | 1.15 | 0.9 |
Number of sub-centers | 0.16 | 0.6 | 0.16 | 0.52 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, C.-C.; Paul, S.; Lee, D.-I. Evaluation of Rainfall Forecasts by Three Mesoscale Models during the Mei-yu Season of 2008 in Taiwan. Part II: Development of an Object-Oriented Method. Atmosphere 2020, 11, 939. https://doi.org/10.3390/atmos11090939
Wang C-C, Paul S, Lee D-I. Evaluation of Rainfall Forecasts by Three Mesoscale Models during the Mei-yu Season of 2008 in Taiwan. Part II: Development of an Object-Oriented Method. Atmosphere. 2020; 11(9):939. https://doi.org/10.3390/atmos11090939
Chicago/Turabian StyleWang, Chung-Chieh, Sahana Paul, and Dong-In Lee. 2020. "Evaluation of Rainfall Forecasts by Three Mesoscale Models during the Mei-yu Season of 2008 in Taiwan. Part II: Development of an Object-Oriented Method" Atmosphere 11, no. 9: 939. https://doi.org/10.3390/atmos11090939
APA StyleWang, C. -C., Paul, S., & Lee, D. -I. (2020). Evaluation of Rainfall Forecasts by Three Mesoscale Models during the Mei-yu Season of 2008 in Taiwan. Part II: Development of an Object-Oriented Method. Atmosphere, 11(9), 939. https://doi.org/10.3390/atmos11090939