Evaluation of Version-7 Trmm Multi-satellite Precipitation Analysis Product during the Beijing Extreme Heavy Rainfall Event of 21 July 2012

The latest Version-7 (V7) Tropical Rainfall Measuring Mission (TRMM) Multi-satellite Precipitation Analysis (TMPA) products were released by the National 33 Aeronautics and Space Administration (NASA) in December of 2012. Their performance on different climatology, locations, and precipitation types is of great interest to the satellite-based precipitation community. This paper presents a study of TMPA precipitation products (3B42RT and 3B42V7) for an extreme precipitation event in Beijing and its adjacent regions (Measurements from a dense rain gauge network were used as the ground truth to evaluate the latest TMPA products. Results are summarized as follows. Compared to rain gauge measurements, both 3B42RT and 3B42V7 generally captured the rainfall spatial and temporal pattern, having a moderate spatial correlation coefficient (CC, 0.6) and high CC values (0.88) over the broader Hebei, Beijing and Tianjin (HBT) regions, but the rainfall peak is 6 h ahead of gauge observations. Overall, 3B42RT showed higher estimation than 3B42V7 over both HBT and Beijing. At the storm center, both 3B42RT and 3B42V7 presented a relatively large deviation from the temporal variation of rainfall and underestimated the storm by 29.02% and 36.07%, respectively. The current study suggests that the latest TMPA products still have limitations in terms of resolution and accuracy, especially for this type of extreme event within a latitude area on the edge of coverage of TRMM precipitation radar and microwave imager. Therefore, TMPA users should be cautious when 3B42RT and 3B42V7 are used to model, monitor, and forecast both flooding hazards in the Beijing urban area and landslides in the mountainous west and north of Beijing.


Introduction
Typhoon Morakot in Taiwan in August 2009 [1][2][3], severe floods in Pakistan in 2010 [4] and in Beijing in July 2012 [5,6], and Hurricane Sandy [7] in northeast U.S. in October 2012 have shown that the precipitation system tends to develop into heavier storms with a longer dry period between two precipitation events [8].On 21 July 2012, a 60-year return period extreme storm hit Beijing and adjacent regions, triggering flash floods in the urban area and landslides in the mountainous region.The storm led to 79 deaths and direct damages worth US $1.86 billion.The storm-triggered flood affected areas of 16,000 km 2 and concerned more than 1.6 million people, inundated many infrastructures like roads with trapped cars and buses, bridges, and collapsed buildings in the deep water [9,10].This extreme event was attributed to a frontal system and mid-tropospheric disturbance [6], and it was generated by a linear-shaped mesoscale convective system (MCS) [9].Sang and Liu [5] pointed out that the topographic effect and the natural climate factors in a changing climate system could also be responsible for the intensive precipitation in the storm.In addition, a low standard had been applied to control the mountain torrents for small and medium rivers in the flooding region.This was another factor for the occurrence of the flooding disaster [5].
Satellite-based precipitation estimation has witnessed significant developments in recent years.High resolution satellite rainfall products emerged in succession, e.g., hourly 0.04° spatial resolution infrared (IR)-dominated Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks [11] and later PERSIANN-Cloud Classification System (PERSIANN-CSS) [12], passive-microwave (PWM)-dominated 30-min 8 km National Oceanic and Atmospheric Administration's (NOAA) Climate Prediction Center (CPC) morphing technique (CMORPH) [13], 3-hourly 0.25° Tropical Rainfall Measuring Mission (TRMM) Multi-satellite Precipitation Analysis (TMPA) based on the incorporation of IR-and PWM-based techniques [14].The latest V7 TMPA products were released in May 2012.The new version of a gauge-adjusted post-real-time TRMM product (covering the period of 1998-present), 3B42V7, became available in late July 2012.The real time product 3B42RT of V7 (covering the period of 1998-present) was also available in October 2012.The accurate measurement of precipitation would greatly benefit the detection and warning of severe storms, flash flooding and landslides [1,15].Therefore, it is a key task for NASA's missions to improve the satellite's performance in measuring precipitation in order to advance our understanding of Earth's water and energy cycle and improve the forecast of extreme events that cause natural hazards and disasters [16][17][18].The comprehensive evaluation of satellite-based precipitation products is much desired by the satellite community.As for the latest TMPA products, related studies are still few, especially during extreme events.Chen, et al. [19] conducted a systematic study on the spatial error distribution of the V6 and V7 TMPA products over Continental United States (CONUS).In addition, Chen, et al. [20] conducted a systematic study on the similarity and difference of the two successive V6 and V7 TMPA products over China.Since TMPA provides rainfall estimation in a quasi-global coverage, it is helpful for NASA's algorithm developers to know TMPA's performance in other places of the world.Particularly, to the best of our knowledge, few efforts have been reported in evaluating V7 TMPA products for any extreme precipitation event.
The primary objective of this study is to evaluate the latest TMPA products with a relatively dense rain gauge network in Beijing and adjacent regions for the extreme precipitation event on 21 July 2012 (from 00:00 UTC 21 July 2012 to 00:00 UTC 22 July 2012).The findings reveal the performance of the latest TMPA products in an extreme event and will guide the application of TMPA products for a hydrological model.The uncertainties quantified for TMPA products can help algorithm developers to further improve the algorithm.It is worth noting that there is a big difference between the sampling volumes of rain gauge and TRMM sensors.The inhomogeneity of the precipitation may cause a non-uniform beam filling (NUBF) problem for remote sensing instruments, which is a common problem to the remote sensing community and has been reported in many previous studies (e.g., Aonashi and Liu [21], Bohren and Battan [22], Lehning, et al. [23] and Smith, et al. [24]).Other studies have also shown that the NUBF may lead to uncertainty in the evaluation of remote sensing measurements through in-situ (e.g., rain gauge) measurements [25,26].In this study, we averaged rain gauge measurements within an individual pixel of TRMM data in order to mitigate the evaluation uncertainty related to the NUBF effect.The area-averaged rain gauge measurements can reduce the intrinsic difference of sampling volume between rain gauge and satellite.We assumed that rain gauge measurements represent the precipitation truth within the corresponding pixel of satellite data.This kind of assumption is a common way to evaluate remote sensing products and has been widely applied by the remote sensing community [1,[27][28][29][30][31]. Considering that the TRMM products to be evaluated have the same spatio-temporal resolution, to better address the differences for these TRMM products, the uncertainty analysis associated with the NUBF effect has not been included in the current study.This paper is organized as follows.Section 2 introduces the study domain and the data used in the analysis.Section 3 analyzes the spatial characteristics and error quantification for the rainfall in terms of total rainfall accumulation, temporal variation, and rainfall contingency as a function of rainfall intensity.A summary of analysis results and the conclusion of the study are given in Section 4.

Study Region
The study region is composed of Hebei province, Beijing City and Tianjin City (HBT), where the latitude spans from 36.0° N to 42.5° N and the longitude spans from 113.5° E to 119.7° E (Figure 1).The northwestern region is a mountainous area where the Taihang and Yanshan mountain ridges are located.To the east is the Bohai Sea and to the southeast is part of the flat North China (Huabei) Plain.The study region is the largest industrial and the most densely populated area in northern China.Particularly, this region, including the mega-city Beijing-the capital of China-is an important core area of the current economic development in China.

Data
Measurements from a dense gauge network, which consists of 2401 gauges in the study domain (Figure 1), were used as the reference for this study.The average rainfall error of gauge observations is 6.52% [32], which is much smaller than the remote-sensing-based estimation.The satellite-based quantitative precipitation estimation (QPE) products to be evaluated include the V7 real time TMPA product 3B42RT and the gauge-adjusted post-real-time research product 3B42V7.The V7 TMPA products provide a number of scientific and data processing enhancements over the previous Version 6 products.Both versions have high temporal-spatial resolution of 3-hourly/0.25° with coverage of 60° N-S for real time product and 50° N-S for the gauge-adjusted post-real-time research product.The V7 TMPA products integrate multiple sources of passive microwave satellite products, which include TRMM Microwave Imager (TMI), Special Sensor Microwave Imager (SSMI), Special Sensor Microwave Imager/Sounder (SSMIS, 3B42V7 only), Advanced Microwave Scanning Radiometer-EOS (AMSR-E), Advanced Microwave Sounding Unit-B (AMSU-B), and Microwave Humidity Sounder (MHS).The IR sensors applied by TMPA come from all available geostationary satellites (GOES-8/10, METEOSAT-7/5 & GMS).The precipitation product estimated from MW data is named 3B40RT and is used to calibrate IR-based product 3B41RT (variable rain rate, or VAR).VAR data are used to fill in the field where 3B40RT data does not exist.The combination of 3B40RT and VAR data yields the real time QPE product 3B42RT.In addition, the new Global Precipitation Climatology Centre (GPCC) "full" gauge analysis, whenever available, and the GPCC "monitoring" gauge analysis since 2010 were incorporated in 3B42V7; whereas the previous monitoring product covered up to April 2005 and used the Climate Assessment and Monitoring System (CAMS) analysis thereafter [33].Compared to its previous version-6 TMPA, V7 TMPA applied the enhanced V7 TMPA Level-2 PR product that gained small to moderate improvements over the V6 Level-2 PR product (i.e., 2A25 or PRV6 [34,35].In addition, V7 TMPA makes use of the GPCC data with improved climatology and anomaly analysis, especially in the complex terrain [33].Thirdly, more satellite data were blended in V7 TMPA, including SSMIS and 0.07° Grisat-B1 infrared data, which is a superset of the International Satellite Cloud Climatology Program B1 data [36] and represent an improvement in resolution and areal coverage over the infrared data (1°/24 classes) used in the V6 algorithm.More details can be seen in [33,37].With these improvement input sources, we ask the question: Is the V7 TMPA ready for extreme events?With this question in mind, we conducted the evaluation of V7 TMPA during the 60-year return period extreme storm that hit Beijing and its adjacent regions with the latest V7 TMPA released in December of 2012.

Statistics Metrics
Relative bias (RB), root-mean-squared error (RMSE), correlation coefficient (CC), probability of detection (POD), false alarm ratio (FAR) and critical success index (CSI) are used in this study to evaluate the performance of TMPA products.RB and CC are dimensionless, while RMSE is in mm.All the above statistics are computed pixel by pixel.Compared with rain gauge observations, the number of hits (A), false alarms (B), and misses (C) can be computed, and then statistic metrics POD, FAR and CSI can be computed with A, B, and C.More detail is available in [1].

Accumulated Rainfall
Figure 2 shows the daily-accumulated rainfall (00:00 UTC on 21 July 2012 to 00:00 UTC on 22 July 2012).The 3-hourly rainfall rates at 0000 UTC and 2400 UTC were accumulated for 1.5 h while rainfall rates at other times (i.e., 0300, 0600, 0900, 1200, 1500, 1800, and 2100 UTC) were accumulated for 3 h to compute the daily rainfall from the TMPA rainfall rate.The gauge accumulation based on Kriging interpolation indicates that the intensive rainfall area extends from southwest of Beijing northeastward to the adjacent areas of Beijing and Hebei province and eastward to the center of Tianjin (Figure 2a).Kriging interpolation embedded in Interactive Data Language (IDL, Version 8.2) was used in this study.The 3B42RT captures the intensive rainfall areas well in southwestern and northeastern Beijing as shown in Figure 2b.The 3B42V7 (Figure 2c) demonstrates the most intensive rainfall area in northwestern Tianjin and northeastern Hebei province excluding the rainfall in the center of the region (indicated by the red "A" in Figure 2a).For quantitative comparison, only the grids that cover two and more rain gauges have been used to compute the statistics metrics.Such a grid-based comparison technique has been reported in many previous studies [1,[27][28][29][30].A total of 5086 gauge abnormal measurements were removed during the quality control.The total 345 grids were available for the direct comparison in the scale of spatial resolution 0.25° × 0.25° for the entire domain with 30 grids for.Figure 2d-e  that 3B42RT (3B42V7) overestimates the daily-accumulated rainfall by 21.95%( 11.92%) in the HBT region and by 56.04% (38.08%) in the Beijing Region.Figure 3 shows the time series cloud top brightness temperature maps derived from the MTSAT-2 satellite.It is noted that the spatial pattern of 3B42RT is highly correlated to the cloud top temperature patterns, especially the patterns shown in Figure 2e.The research product 3B42V7 still showed the effect of the IR sensor that was shown in 3B42RT, even though 3B42V7 was corrected by rescaling the 3B42RT to the monthly rain gauge data based on a limited number of gauges.This is likely due to the inherent shortfall of the IR technique: the IR brightness temperature to estimate precipitation produces large errors [38,39].

Time Series Precipitation
The capability of catching the temporal variation of rainfall is essential for the monitoring of flash floods and landslides.Figure 4 shows the time series plots of hourly-accumulated rainfall from gauge and satellite data in the HBT domain, Bejing and the rainfall center indicated by the red letter A in Figure 2a.The gauge-based rainfall generally increases steadily from the first hour to the peak occurring at the 20th hour and then decreases dramatically to less than 1mm/h in the 24th hour in the HBT and Beijing regions.The 3B42RT and 3B42V7 have high CC values (0.87 and 0.88, respectively) in HBT but negative CC values (−0.17) in Beijing.This figure shows that both 3B42RT and 3B42V7 have generally captured the temporal variation of rainfall in HBT but witnessed the rainfall peak 6.5 h ahead of gauge observations (Figure 4a).It is also evident that 3B42RT and 3B42V7 failed to capture the temporal trend of rainfall in Beijing (Figure 4b).This can be explained as follows.First, there were limited numbers of microwave overpasses over this area.Secondly, the cold center of cloud top temperature moved southeastward to Tianjin after 21:12 UTC July 2010 (see Figure 3d-g), and the IR-based product 3B41RT, which is used to fill in the field where 3B40RT does not exist, yields a large error when the convective cloud has a higher brightness temperature than the commonly used temperature threshold [38,39].For the rainfall center, although 3B42RT and 3B42V7 have a similar trend of increase in rainfall amount, both of them miss the rainfall peak with great underestimates by 29.02% and 36.07%,respectively (Figure 4c). Figure 3h implies that the cloud top brightness temperature at the rainfall center jumped quickly from the minimum value of about 208 K at the 11th hour to 255 K at the 18th hour, dropped dramatically to about 216 K at the 19th hour, and then rapidly went up to about 288 K at the 24th hour.TMPA showed their peak rainfall from 13:30 UTC to 16:30 UTC when the cloud top brightness temperature was high above 230 K.The poor performance, i.e., underestimation, of 3B42RT and 3B42V7 could be explained as follows.The rainfall center is located in the mountain areas, where the regional topographical lifting of moisture spawned the convective cells for the formation of the MCS [9].The IR-based 3B41RT, which is used to fill the gaps where 3B40RT is not available [37], has limitations in estimating the precipitation in the complex terrain [1,38,39,41].Additionally, the errors of the individual algorithms also have an important contribution to the poor performance of 3B42RT and 3B42V7 even when the microwave does contribute.

Probability Distribution
Probability distribution functions (PDFs), such as precipitation rate distribution and precipitation volume distribution as a function of precipitation rate, provide an effective way to study the precipitation characteristics.PDFs offer an insight into the error dependence on the precipitation rate and the potential error impact on hydrological applications [42].Figure 5 shows the 3-hourly mean precipitation PDFs of occurrence (PDFc) and the cumulative distribution of rainfall rates in terms of volume (CDFv), which are applied to characterize TMPA's ability in detecting the rainfall of different intensities.The 3B42V7 gives a better agreement with the gauge observations than 3B42RT regarding the occurrence of moderate rainfall rates (1~4 mm/h).Both 3B42RT and 3B42V7 underestimate the occurrence of light rainfall rates (<1 mm/h) and overestimate for high rainfall rates, especially for 12-24 mm/h (Figure 5a).The rain rates (<24 mm/h) account for almost 98% of the total volume for 3B42RT and 3B42V7 while just about 85% of the volume for rain gauge measurements.This fact suggests that 3B42RT and 3B42V7 have a poor detection of very intensive rainfall (>24 mm/h).Considering the data samples are limited, this result might not represent the general performance of 3B42RT and 3B42V7.However, this conclusion is suitable at least in this case.

Contingency Statistics
Contingency statistics uncover the performance of the probability of detection, the rate of successful hits and the false alarm rate.Figure 6 shows that 3B42RT (3B42V7) has a zero POD and CSI when the rainfall rate is greater than 25 mm/h (23 mm/h).Similarly, this result implies that 3B42RT and 3B42V7 have poor performances in detecting intensive rainfall in this case.Consistent results are also revealed by the probability distributions shown in Figure 5.

Summary and Conclusions
This study evaluates the performance of TMPA QPE products 3B42RT and 3B42V7 for an extreme precipitation event in Beijing and adjacent areas.The evaluation has used a dense gauge network to provide the ground truth.The evaluation metrics, including RB, RMSE, and CC, are applied for grid-based accumulated rainfall and time series rainfall with further analysis of probability distributions and contingency.Findings are summarized as follows:
shows that both 3B42RT and 3B42V7 have moderate CC (~0.6) with gauge-based observation, indicating 3B42RT and 3B42V7 cannot capture the spatial pattern of rainfall very well in the entire HBT region.The RB result shows

Figure 3 .
Figure 3. Cloud top time series brightness temperature map over the entire HBT domain (a-h) time series area-average brightness temperature at the rainfall center shown in Figure 2a.The brightness temperature data are provided by the Climate Prediction Center, National Oceanic and Atmospheric Administration [40].

Figure 5 .
Figure 5. (a) Occurrence probability distribution of rain rate; and (b) volume probability distribution of rain rate with bin interval of 0.5 mm/h.

Figure 6 .
Figure 6.Contingency statistics for (a) probability of detection (POD); (b) false alarm ratio (FAR); and (c) critical success index (CSI) with bin interval of 1 mm/h.