Comparative Evaluation of the GPM IMERG Early, Late, and Final Hourly Precipitation Products Using the CMPA Data over Sichuan Basin of China

: The Global Precipitation Measurement (GPM) mission has generated global precipitation products of improved accuracy and coverage that are promising for advanced hydrological and meteorological studies. This study evaluates three Integrated Multi-satellitE Retrievals for GPM (IMERG) Hourly products, including the Early-, Late-, and Final-run products (IMERG-HE, IMERG-HL, and IMERG-HF, respectively), over Sichuan Basin of China. This highly complex terrain of the steep mountainous region o ﬀ ers further scrutiny on the quality and applicability of the data. The China Meteorological Precipitation Analysis (CMPA) data from January 2016 to December 2018 are used as the reference for the evaluation. Results show that: (1) At grid scale, IMERG-HL and IMERG-HF outperform IMERG-HE in terms of correlation coe ﬃ cient (CC) and root-mean-square error (RMSE), but IMERG-HL has smaller relative bias (RB) than that of the IMERG-HF (by 21.16%). IMERG-HF presents the highest probability of detection (POD = 0.52) and critical success index (CSI = 0.32), except for high false alarm ratio (FAR) for light precipitation. (2) At regional scale, IMERG-HF outperforms IMERG-HE and IMERG-HL in annual evaluation in all the metrics except for the serious overestimation as shown in RB (20.18%, 3.84%, and 4.97%, respectively). Its accumulative precipitation deviation mainly comes from moderate precipitation events (1–10 mm / h), while better detection capability is seen in light precipitation ( < 1 mm / h). Seasonally, IMERG-HF performs the best in winter, while IMERG-HL performs the best in the other seasons. (3) IMERG-HF captures the peak precipitation more accurately in all seasons. In reproducing the diurnal cycle, IMERG-HF performs better in winter, while IMERG-HL performs better in summer and autumn, and IMERG-HE in spring. However, all three products overestimate the early morning precipitation (01:00–08:00 local standard time) of the diurnal cycle in spring, summer, and autumn. the IMERG-HL product shows the best performance in precipitation observation in spring, summer, and autumn. As for winter precipitation, the IMERG-HF product shows the best precipitation detection capability and accuracy. that both of the IMERG-HE and IMERG-HL products show the insufficient detection capability for the light precipitation, but the missed detection rate of the IMERG-HE product is significantly greater than that of the IMERG-HL product. More specifically, for the precipitation from 0.1 to 0.3 mm/h, the frequency values of the CMPA data and the three IMERG products are 3.38%, 2.48%, 3.01%, and 3.62%, respectively. These results indicate that the IMERG-HF product slightly exaggerate the detection ability of light precipitation. As the precipitation intensity increases (>1.0 mm/h), the IMERG-HE and IMERG-HL products show more consistent detection capability with the CMPA data, while the IMERG-HF product still has a certain degree of FAR.


Introduction
Precipitation plays a critical role in the global hydrological cycle and energy exchange of the atmosphere [1][2][3][4]. The uneven distribution of precipitation over large areas may directly cause the hydrologic system to be unstable [5][6][7] and easily lead to urban flooding, mountain mudslides, and flash floods [8][9][10]. Thus, observing the precipitation with high accuracy on a global scale is helpful in flood early warning, water resources management, and weather and geological disasters monitoring [11][12][13][14][15][16]. However, many more challenges remain in obtaining accurate precipitation over products over the Beijiang River, China. Sungmin et al. (2017) [44] compared the IMERG version 3 Early, Late, and Final precipitation products in southeastern Austria. Tan et al. (2017) [45] assessed the IMERG precipitation products in Malaysia. However, there is little research on the quantitative assessment and comparisons among the IMERG-E, IMERG-L, and IMERG-F hourly products with gauge-based products that have been proven to be very accurate, especially in the complex terrain regions. Meanwhile, the observation data used in many IMERG product evaluation studies span less than one year, thus the results are not as robust.
It is recognized that evaluation on the accuracy and reliability of the NRT IMERG hourly products is conducive to improvement of data quality. In particular, high quality precipitation data with high spatiotemporal resolution used in weather forecast and hydrological models can optimize the forecast accuracy of weather and geological disasters in complex terrain regions [48,49]. Regrettably, these studies have not been demonstrated in Sichuan Basin, which is one of the most complex terrain regions in the world and an ideal model for studying the influence of topography on precipitation.
Therefore, in this study, the three IMERG hourly products (IMERG-HE, IMERG-HL, and IMERG-HF, whereas "H" signifies "hourly") during the period of 2016 to 2018 are quantitatively assessed at high spatial resolution (0.1 • × 0.1 • ) over Sichuan Basin. Moreover, a high quality gauge network data with stringent quality control are used along with the three IMERG products in the present study to: (1) assess and analyze the differences in the measurement accuracy of the IMERG hourly products over Sichuan Basin; (2) compare the spatial distribution characteristics among the three IMERG hourly products; and (3) evaluate the capability of reproducing the diurnal cycle structure of precipitation for the three IMERG hourly products. This study reveals the errors, spatial distribution, and diurnal cycle characteristics among the three IMERG products at grid and regional scales, and provide valuable insights for the algorithm developers to improve the IMERG product quality and for users to select high-quality IMERG products in many relevant applications.
The remaining sections of the paper are structured as follows: Section 2 introduces the study area, precipitation datasets, the data quality control methods, data preprocessing, and statistical metrics. Section 3 evaluates the quality of all three IMERG products at grid scale. Section 4 presents comprehensive evaluations of all three IMERG products with multi-time scales at regional scale, and compares the accuracy and detection capability of various precipitation intensity. Section 5 focuses on the analysis of the capability in reconstructing the diurnal cycle of precipitation among the IMERG-HE, IMERG-HL, and IMERG-HF products at seasonal scales. Finally, discussion and summary are provided in Section 6.

Study Area
The study area is Sichuan Basin in China, located between 103 • 03 to 109 • 15 E and 28 • 15 to 32 • 03 N, which is the transition zone between the Qinghai-Tibet Plateau and the Middle-to-Lower Yangtze Plain. Additionally, Sichuan Basin consists of mountains with elevations between 1000 to 3000 m above sea level and basin floors with elevations between 250 to 750 m above sea level, and its terrain contour is approximately in a diamond shape (Figure 1a). The basin floor can be divided into Chengdu plain, central Sichuan hill area, and east Sichuan parallel ridge-and-valley area. Studies show that, for Sichuan Basin of complex topography and geographic locations, the precipitation is unavoidably influenced by its landform, east Asian, south Asian, and plateau monsoons and has strong diurnal variations [50]. Precipitation statistics show that close to 70% of the total precipitation occurred at night, which is usually called "Bashan Yeyu" (nocturnal precipitation in Bashan Mountains) [51]. In addition, high probability of precipitation usually occurs at the edge of the basin during the night [52]. Spatial distribution of mean precipitation over Sichuan Basin during 2016-2018 is shown in Figure 1b.

Gridded Ground Gauge Dataset
The gridded ground gauge dataset with high temporal and spatial resolutions (hourly and 0.1° × 0.1°) and the China Meteorological Precipitation Analysis (CMPA) V1.0 product are provided by the National Meteorology Information Center (NMIC) of the China Meteorological Administration (CMA) (http://data.cma.cn) as the benchmark to evaluate the satellite-based precipitation products. The CMPA product is generated by integrating the observations from more than 30,000 automatic meteorological stations (National and regional stations) in China and CMORPH satellite precipitation product. Meanwhile, in order to generate the hourly gridded CMPA data and ensure the dataset consistency, validity, and reliability, the first step is to carry out the strict quality control work on rain gauge data, which includes checking abnormal values and spatiotemporal consistency, and inserting the refined values by using inverse distance weighting (IDW) interpolation method [53,54]. Then, the systematic error of CMORPH data has to be corrected using the probability density function matching method base on the hourly gauge observations [55]. Besides, the CMORPH data (halfhourly and 8 km resolutions) is resampled to generate the gridded precipitation products with hourly and 0.1° × 0.1° resolutions [32], which have been shown a high accuracy in East Asia [56]. Finally, the CMPA dataset with hourly and 0.1° × 0.1° resolutions over mainland China is produced by using the Optimal Interpolation (OI) method to integrate the processed CMORPH data and gridded gauge data. More details on quality control of the CMPA product are described in Shen et al. (2014) [57]. Therefore, CMPA dataset effectively integrates the advantages of ground observations and satellite precipitation products, and its precipitation value and spatial distribution are more reasonable. In addition, a large number of independent precipitation samples are utilized to test the quality of

Gridded Ground Gauge Dataset
The gridded ground gauge dataset with high temporal and spatial resolutions (hourly and 0.1 • × 0.1 • ) and the China Meteorological Precipitation Analysis (CMPA) V1.0 product are provided by the National Meteorology Information Center (NMIC) of the China Meteorological Administration (CMA) (http://data.cma.cn) as the benchmark to evaluate the satellite-based precipitation products. The CMPA product is generated by integrating the observations from more than 30,000 automatic meteorological stations (National and regional stations) in China and CMORPH satellite precipitation product. Meanwhile, in order to generate the hourly gridded CMPA data and ensure the dataset consistency, validity, and reliability, the first step is to carry out the strict quality control work on rain gauge data, which includes checking abnormal values and spatiotemporal consistency, and inserting the refined values by using inverse distance weighting (IDW) interpolation method [53,54]. Then, the systematic error of CMORPH data has to be corrected using the probability density function matching method base on the hourly gauge observations [55]. Besides, the CMORPH data (half-hourly and 8 km resolutions) is resampled to generate the gridded precipitation products with hourly and 0.1 • × 0.1 • resolutions [32], which have been shown a high accuracy in East Asia [56]. Finally, the CMPA dataset with hourly and 0.1 • × 0.1 • resolutions over mainland China is produced by using the Optimal Interpolation (OI) method to integrate the processed CMORPH data and gridded gauge data. More details on quality control of the CMPA product are described in Shen et al. (2014) [57]. Therefore, CMPA dataset effectively integrates the advantages of ground observations and satellite precipitation products, and its precipitation value and spatial distribution are more reasonable. In addition, a large number of independent precipitation samples are utilized to test the quality of CMPA dataset. The results demonstrate that the CC, average deviation, and relative error of CMPA dataset are significantly better than those of CMORPH data [57,58].

GPM IMERG Precipitation Products
IMERG is the level 3 products of the GPM mission with half hour temporal resolution and 0.1 • × 0.1 • spatial resolution, which combined all passive microwave (PMW) and infrared (IR) data of the GPM constellation satellites, and calibrated by monthly gauge analysis of the Global Precipitation Climatology Centre (GPCC) [39,59]. More detailed descriptions about the IMERG products and precipitation retrieval algorithm can be found in Huffman et al. (2015) [60]. According to the generation schedules of precipitation products, IMERG-HE is an NRT product and produced about 4 h after nominal observation time for users who need to preliminarily estimate the probability of flooding or geological disasters in time. IMERG-HL is also an NRT product and produced with approximately 12 h latency for weather forecasters, geological monitors, or other users. IMERG-HF is a PRT product and released about 3.5 months later. Additionally, the generation process of the IMERG-HE product is simpler than the IMERG-HL and IMERG-HF products. For instance, instantaneous PMW precipitation estimates are only propagated forward in time by the morphing scheme of IMERG-HE, whereas both forward and backward morphing schemes are employed in IMERG-HL and IMERG-HF [44]. Therefore, the IMERG-HL and IMERG-HF products are supposed to be better than IMERG-HE in describing the features of precipitation structure changes. Moreover, in terms of bias calibration, the IMERG-HE and IMERG-HL products adopt climatological gauge data, while the IMERG-HF product uses monthly GPCC gauge analysis, thus, the accuracy and reliability of the IMERG-HF product are supposed to be better than the NRT IMERG products.
In this study, the research period is from January 2016 to December 2018. Moreover, we chose the IMERG-HE, IMERG-HL, and IMERG-HF version 5 products after calibrated for systematically evaluating the observation accuracy and precipitation detectability over Sichuan Basin. The IMERG hourly products are generated by accumulating the IMERG half-hourly products over 1 h, and the unit is mm/h. The IMERG products can be downloaded from the Precipitation Measurement Missions (PMM) website (https://pmm.nasa.gov/data-access/downloads/gpm).

Data Preprocessing
Before evaluating the IMERG products, we need to preprocess the IMERG and gridded rain gauge data to ensure their consistency and accuracy. The data preprocessing includes the following steps: (1) checking the continuity of gauge and IMERG data, then removing the abnormal and missing data for keeping their consistency and symmetry; and (2) accumulating 2 half-hourly IMERG products to get the hourly precipitation products. Before evaluating and comparing the reconstruction capability of the diurnal cycle among the IMERG-HE, IMERG-HL, and IMERG-HF products, the time of those products should be converted from Universal Time Coordinated (UTC) to Local Standard Time (LST). Moreover, it is necessary to adjust the time inconsistency of rain gauge data. For instance, the observation at 00:00 actually represents the precipitation from 23:00 to 00:00 of the previous day. Thus, before evaluating the IMERG capability in reproducing the diurnal cycle, the gauge data from 01:00 am to 00:00 am the next day should be selected correctly and used for the diurnal cycle.

Methodology and Statistical Metrics
For comprehensively and objectively evaluating the performance of the IMERG products at hourly scale over Sichuan Basin, the comparison and analysis of some statistical metrics are carried out in this study. These metrics can be generally divided into continuous and contingency statistical metrics [40]. The continuous statistical metrics are used to describe the agreement and the bias between the IMERG products and gauge observations, include CC, RB, and root-mean-square error (RMSE). CC focus on describing the correlation between the IMERG products and gauge observations in the grid-and regional-scale evaluations. RB is often used to evaluate the degree of overestimation or underestimation of the IMERG products. Positive and negative values of RB represent the overestimation and underestimation, respectively. RMSE refers to the accuracy of the IMERG products compared with gauge observations [61]. In addition, the contingency statistical metrics, including probability of detection (POD), false alarm ratio (FAR), and critical success index (CSI), are used to evaluate the precipitation detection capability of the IMERG products. POD and FAR represent the fraction of precipitation occurrences correctly and falsely detected by the IMERG products among all the actual precipitation events, respectively [62]. Being a function of POD and FAR, CSI comprehensively indicates the definite precipitation detection capability of the IMERG products, and is a more balanced evaluation indicator [63]. Meanwhile, the perfect values of CC, RB, RMSE, POD, FAR, and CSI are 1, 0, 0, 1, 0, and 1, respectively. Additionally, considering the detection resolution of rain gauges (0.1 mm/h), the threshold value for determining whether the precipitation event occurs are set to 0.1 mm/h for hourly metrics. Table 1 shows the formulas and perfect values of the continuous and contingency statistical metrics.
Note: N represents number of samples; i represents the tpye of IMERG, which is E, F, or L; S in represents IMERG data; G in represents Gauge data; S in represents the mean value of IMERG data; G in represents the mean value of Gauge data; σ S in represents standard deviations of IMERG data; σ Gin represents standard deviations of Gauge data; N 11 represents the precipitation detected by IMERG and Gauge simultaneously; N 10 represents the precipitation detected only by IMERG; N 01 represents the precipitation detected only by Gauge.

IMERG Grid-Scale Evaluation
The IMERG hourly products are evaluated against the CMPA data via the continuous and contingency statistical metrics at each grid cell. All of the IMERG hourly products and CMPA data used for this comparison are grid data with 1505 cells (0.1 • × 0.1 • ) over the Sichuan Basin. For the grid-scale evaluation, the time series of each grid cell from January 2016 to December 2018 is analyzed by calculating the statistical metrics of CC, RB, RMSE, POD, FAR, and CSI, as well as their distributions over the Sichuan Basin.
The spatial distributions of the statistical metrics for the IMERG-HE, IMERG-HL, and IMERG-HF products at grid scale over the Sichuan Basin are shown in Figure 2. In general, all three IMERG products exhibit similar spatial patterns of continuous statistical metrics, while the IMERG-HL and IMERG-HF products present the higher accuracy and lower error, especially in the eastern Sichuan Basin (CC > 0.55 and RMSE < 0.7 mm), which means that the IMERG-HL and IMERG-HF products have made a significant improvement in the agreement with the CMPA data. However, CCs of the IMERG-HL and IMERG-HF products are lower in the northwestern Sichuan Basin (CCs < 0.45). In terms of RB, the NRT IMERG hourly products generally tend to slightly overestimate or underestimate precipitation in most interior and marginal areas, but severely overestimate precipitation in the northwestern and southwestern Sichuan Basin. Nevertheless, the PRT IMERG-HF product significantly expands the area of overestimation, exacerbates the extent of overestimation, but reduces the extent of underestimation. In brief, the IMERG-HF product shows a slight overestimation in most areas, but shows a severe overestimation in the northwest and southwest, as well as an underestimation in the marginal regions of Sichuan Basin. Compared to the NRT IMERG hourly products, the RMSE of the IMERG-HF product is generally lower in most areas, which implies that the precipitation estimation has been effectively improved, while this improvement is limited in the western and southwestern Sichuan Basin. For the Water 2020, 12, 554 7 of 20 capability of the precipitation detection, the IMERG-HF product apparently presents higher POD (>0.5) than the IMERG-HE and IMERG-HL products in most areas of the Sichuan Basin as a whole, especially in the western, northwestern, and eastern regions (POD > 0.6). However, the FAR spatial pattern of the IMERG-HF product shows no significant difference compared to the IMERG-HL product, and both of them present high FARs (>0.7) in the northwestern regions. As for CSI, comparing with the IMREG-HE and IMERG-HL products, the IMERG-HF product shows better precipitation detection capability, and has a higher CSI (>0.35) in the eastern and northeastern regions. This phenomenon indicates that the IMERG-HF product improves the accuracy of precipitation, but severely overestimates precipitation and exaggerates the FAR of light precipitation events, which commonly occur in the northwestern Sichuan Basin (also see Figure 1b). The box plots of the metrics for the IMERG-HE, IMERG-HL, and IMERG-HF products are shown in Figure 3, which represents dispersion characteristics of the statistical metrics. After being arranged in the ascending order, the statistical data is divided by quartiles into four equal parts, which include the first quartile (Q1, 25%), the second quartile (Q2, 50%), and the third quartile (Q3, 75%). More specifically, the bottom and top edges of the box are Q1 and Q3, and the band inside the box is Q2. Furthermore, the ends of the upper and lower whiskers indicate the maximum and minimum values, respectively. The distributions of the metrics for the three IMERG products show strong symmetry, but the ends of the whiskers for CC, RB, RMSE, and FAR are far from Q2, indicating that these metrics are not concentrated. In terms of CC and RMSE, the IMERG-HF product (Q2 of CC and RMSE are 0.54 and 0.77 mm, respectively) presents comparable performance with the IMERG-HL product (Q2 of CC and RMSE are 0.53 and 0.78 mm, respectively). By comparison, the IMERG-HE product presents a little poorer performance (Q2 of CC is 0.46 but higher Q2 of RMSE at 0.86 mm) than the IMERG-HL product. As for RB, all three IMERG products overestimate the precipitation in most areas (shown in Figure 2d-f), with Q2 of 6.18%, 7.99%, and 21.16%, respectively, which indicates that the degree of overestimation of the IMERG-HF product is greater than that of the IMERG-HE and IMERG-HL products. By analyzing the metrics above, the IMERG-HL product shows the best agreement with the CMPA data than the IMERG-HE and IMERG-HF products at the grid scale evaluation. For the capability of the precipitation detection, the IMERG-HF product apparently presents higher POD and CSI (Q2 are 0.52 and 0.32, respectively) than the IMERG-HL (Q2 are 0.46 and 0.30, respectively) and IMERG-HE products (Q2 are 0.40 and 0.26, respectively); but Q2 of the FAR (0.53) is slightly higher than the IMERG-HL product (0.52) and lower than the IMERG-HE product (0.54). This implies that after correcting the deviation using the GPCC data, the light precipitation detection capability of the IMERG-HF product is improved obviously, but the FAR of the IMERG-HF product has not been improved at all.

Annual Evaluation
The three IMERG hourly products are compared and evaluated at a regional scale in order to facilitate the quantitative evaluation of the precipitation estimation deviation during the three-year period. The statistical metrics of these products are calculated and shown in Table 2.
The performance of the three IMERG hourly products at regional scale is very similar to their

Annual Evaluation
The three IMERG hourly products are compared and evaluated at a regional scale in order to facilitate the quantitative evaluation of the precipitation estimation deviation during the three-year period. The statistical metrics of these products are calculated and shown in Table 2. The performance of the three IMERG hourly products at regional scale is very similar to their performance at grid scale in almost all of the statistical metrics, except for the RB. The RBs of the three IMERG hourly products at regional scale (3.84%, 4.97%, and 20.18% for HE, HL, and HF products, respectively) are slightly decreased compared to the grid scale evaluation (6.18%, 7.99%, and 21.16%, correspondingly and respectively), which indicates that the IMERG hourly products perform slightly better at regional scale than at grid scale. More specifically, as for the CC and RMSE, the IMERG-HL product presents comparable performance (CC of 0.51 and RMSE of 0.80 mm) as for the IMERG-HF products (CC of 0.53 and RMSE of 0.79 mm), and both of the products show better performance than the IMERG-HE product. However, the RB of the IMERG-HL product (4.97%) is significantly lower than that of the IMERG-HF product (20.18%). Moreover, the RBs of the three IMERG hourly products are positive numbers, which indicate that these products all overestimate precipitation at regional scale over Sichuan Basin during the period of 2016 to 2018, and the most serious overestimation comes from the IMERG-HF product. Moreover, the IMERG-HF product has the highest POD and CSI (0.51 and 0.32, respectively) compared to the other IMERG products, and has the same FAR (0.53) as the IMERG-HL product. Overall, the IMEG-HL product performs better in accurately estimating precipitation, while the IMERG-HF product shows a better precipitation detection capability in regional evaluation. Note that the gaps of all the statistical metrics for the IMERG-HL and IMERG-HF products are narrow in regional scale evaluation, except for the RB. This implies that the performance of the IMERG-HF product has not dramatically improved, thus, the precipitation estimation method used by the IMERG-HF product should be further improved.

Seasonal Evaluation
The statistical metrics from the evaluation of seasonal precipitation from the three IMERG hourly products are shown in Figure 4. In general, the seasonal variation of the statistical metric profiles of the three IMERG hourly products are remarkably similar, and the gaps among the three IMERG hourly products for all the metric profiles are very close except for the RB. More specifically, the IMERG-HL and IMERG-HF products have a similar precipitation detection capability and present better performance than the IMERG-HE product in all seasons except for the RB. As for the RB, the IMERG-HF product exhibits more severe overestimation (21.49% in spring, 22.49% in summer, and 17.77% in autumn) than the IMERG-HE and IMERG-HL products, but only a slight overestimation in winter (3.87%). In contrast, the IMERG-HE and IMERG-HL products severely underestimate the precipitation in winter (−28.25% and −27.33%, respectively). By synthesizing all the statistical metrics in every season, compared to the other IMERG hourly products, the IMERG-HL product shows the best performance in precipitation observation in spring, summer, and autumn. As for winter precipitation, the IMERG-HF product shows the best precipitation detection capability and accuracy.

Monthly Evaluation
The monthly quantitative evaluation of the three IMERG hourly products at regional scale during January 2016 to December 2018 is also conducted, and the results are shown in Figure 5. It is evident that all three IMERG products have the weakest correlation with the CMPA data in January and December during 2016 to 2018. As for the RB, the IMERG-HF product overestimates the precipitation in almost all months, but only underestimates the precipitation in December. However, the IMERG-HE and IMERG-HL products tend to underestimate the precipitation in winter months, and overestimates the precipitation in other months. In addition, the RMSE profiles of the three IMERG hourly products have the obvious characteristics of seasonal variation, with the peak in June or July and the valley in January or December. As for the POD and FAR, all three IMERG hourly products with high PODs and low FARs can generally capture the precipitation during March to October, during which both of the IMERG-HL and IMERG-HF products show the similar precipitation detection capability and have better performance than the IMERG-HE product. However, the IMERG-HF product presents a little better precipitation detection capability than the other IMERG products in the winter months. This indicates that the IMERG-HL and IMERG-HF products with more calibrations are indeed able to improve the precipitation detection capability over Sichuan Basin, but are still too sensitive (with high FAR) in winter, which implies that the IMERG-HF does not exhibit the obvious improvements in terms of the FAR compared with other IMERG products over Sichuan Basin.
Considering what has been presented above, the IMERG-HF product exhibits a better performance in January and December, but for the rest of the months, the IMERG-HL product shows better agreement with the CMPA data and has a better precipitation detection capability than the other IMERG hourly products. From the above analysis, the quality of the IMERG-HF product has not been improved much by incorporating the monthly observation of GPCC; in fact, it is even worse than the IMERG-HE and IMERG-HL products in terms of RB for spring, summer, and autumn precipitation. Several factors may explain this phenomenon: (1) more convective precipitation events occurred in summer, which may affect the detection capability of the microwave and infrared sensors, and consequently, reduce the detection accuracy; (2) the scale of convective precipitation may be smaller than the resolution of GPM, which may also result in the overestimation of the IMERG products in summer [64]; (3) few precipitation events or light precipitation occur in winter over Sichuan Basin, and the large size of cloud drop increased by higher concentration of aerosols may intercept the ability of precipitation estimation by IMERG products [65]; and (4) monthly observations of GPCC may be more accurate than climatological gauge data only in winter over Sichuan Basin, which may result in the IMERG-HF product to performs better than the NRT IMERG hourly products in winter.

Monthly Evaluation
The monthly quantitative evaluation of the three IMERG hourly products at regional scale during January 2016 to December 2018 is also conducted, and the results are shown in Figure 5. It is evident that all three IMERG products have the weakest correlation with the CMPA data in January and December during 2016 to 2018. As for the RB, the IMERG-HF product overestimates the precipitation in almost all months, but only underestimates the precipitation in December. However, the IMERG-HE and IMERG-HL products tend to underestimate the precipitation in winter months, and overestimates the precipitation in other months. In addition, the RMSE profiles of the three IMERG hourly products have the obvious characteristics of seasonal variation, with the peak in June or July and the valley in January or December. As for the POD and FAR, all three IMERG hourly products with high PODs and low FARs can generally capture the precipitation during March to October, during which both of the IMERG-HL and IMERG-HF products show the similar precipitation detection capability and have better performance than the IMERG-HE product. However, the IMERG-HF product presents a little better precipitation detection capability than the other IMERG products in the winter months.
This indicates that the IMERG-HL and IMERG-HF products with more calibrations are indeed able to improve the precipitation detection capability over Sichuan Basin, but are still too sensitive (with high FAR) in winter, which implies that the IMERG-HF does not exhibit the obvious improvements in terms of the FAR compared with other IMERG products over Sichuan Basin.

Precipitation Dectection Capability
The accuracy and detection capability of the IMERG hourly products is closely related to the intensity of precipitation. Therefore, evaluation of the IMERG hourly products for different precipitation intensities may provide favorable guidance for data selection in early warning of drought and extreme precipitation events. According to the distribution characteristics of annual precipitation over Sichuan Basin and the precipitation threshold (0.1 mm/h) set forth earlier, we divided the intensity of precipitation into nine intervals, and calculated the cumulative precipitation and frequency of the precipitation events at all nine intervals from the IMERG grid-scale products and the CMPA data during 2016 to 2018. The statistical results are shown in Figure 6.
The distribution of the cumulative precipitation shows that the IMERG-HF product severely overestimates the precipitation compared to the CMPA data in the intensity range from 1.0 to 10 mm/h, while the IMERG-HE and IMERG-HL products only slightly overestimates the precipitation in the intensity ranges from 1.0 to 5.0 mm/h and greater than 10 mm/h, and underestimates the precipitation in the rest of the precipitation intensity ranges.
The distribution of precipitation frequency indicates that as for the light precipitation less than 1.0 mm/h, the IMERG-HF product is closer to the CMPA data than the IMERG-HE and IMERG-HL products, which suggest that both of the IMERG-HE and IMERG-HL products show the insufficient detection capability for the light precipitation, but the missed detection rate of the IMERG-HE product is significantly greater than that of the IMERG-HL product. More specifically, for the precipitation from 0.1 to 0.3 mm/h, the frequency values of the CMPA data and the three IMERG products are 3.38%, 2.48%, 3.01%, and 3.62%, respectively. These results indicate that the IMERG-HF product slightly exaggerate the detection ability of light precipitation. As the precipitation intensity increases (>1.0 mm/h), the IMERG-HE and IMERG-HL products show more consistent detection capability with the CMPA data, while the IMERG-HF product still has a certain degree of FAR. Considering what has been presented above, the IMERG-HF product exhibits a better performance in January and December, but for the rest of the months, the IMERG-HL product shows better agreement with the CMPA data and has a better precipitation detection capability than the other IMERG hourly products.

Precipitation Dectection Capability
The accuracy and detection capability of the IMERG hourly products is closely related to the intensity of precipitation. Therefore, evaluation of the IMERG hourly products for different precipitation intensities may provide favorable guidance for data selection in early warning of drought and extreme precipitation events. According to the distribution characteristics of annual precipitation over Sichuan Basin and the precipitation threshold (0.1 mm/h) set forth earlier, we divided the intensity of precipitation into nine intervals, and calculated the cumulative precipitation and frequency of the precipitation events at all nine intervals from the IMERG grid-scale products and the CMPA data during 2016 to 2018. The statistical results are shown in Figure 6. data are predominantly contributed by the precipitation with intensity from 1.0 to 10 mm/h. In this range, the overestimation of the IMERG-HE and IMERG-HL products are comparable, but less than that of the IMERG-HF product. In addition, the IMERG-HF product can capture more light precipitation events than the IMERG-HE and IMERG-HL products for the precipitation less than 1.0 mm/h, while the IMERG-HE and IMERG-HL products show a better detection capability for the precipitation more than 1.0 mm/h.

Evaluation on Reconstruction Ability of the Diurnal Precipitation Cycle
The diurnal cycle of precipitation is an important element in the evolution of weather systems, and it is greatly affected by the season and terrain [66]. For the Sichuan Basin, obtaining an accurate diurnal cycle of precipitation will help further study the influencing mechanisms of terrain on atmospheric dynamics and thermodynamics. Therefore, for the three IMERG hourly products, their reconstruction ability of the diurnal precipitation cycle during January 2016 to December 2018 (1096 days in total) over Sichuan Basin was also evaluated.
Because the structure of the diurnal precipitation cycle is strongly influenced by the seasons, we analyzed the IMERG hourly products and CMPA data in each season. The specific processes are as follows: (1) convert from UTC of the IMERG products and CMPA data to LST at the center latitude for Sichuan Basin; (2) adjust the time to get the correct observations from 01:00 to 24:00 LST in one day, because the CMPA data at 00:00 LST represents the accumulated precipitation in the previous hour; (3) gather the accumulation time of effective data and calculate the mean precipitation intensity. The diurnal cycle and the continuous statistical metrics of the IMERG-HE, IMERG-HL, and IMERG-HF products for each season during 2016 to 2018 are shown in Figure 7 and Table 3. 1-0.3 0.3-0.5 0.5-1.0 1.0-3.0 3.0-5.0 5.0-10 10-15  The distribution of the cumulative precipitation shows that the IMERG-HF product severely overestimates the precipitation compared to the CMPA data in the intensity range from 1.0 to 10 mm/h, while the IMERG-HE and IMERG-HL products only slightly overestimates the precipitation in the intensity ranges from 1.0 to 5.0 mm/h and greater than 10 mm/h, and underestimates the precipitation in the rest of the precipitation intensity ranges.
The distribution of precipitation frequency indicates that as for the light precipitation less than 1.0 mm/h, the IMERG-HF product is closer to the CMPA data than the IMERG-HE and IMERG-HL products, which suggest that both of the IMERG-HE and IMERG-HL products show the insufficient detection capability for the light precipitation, but the missed detection rate of the IMERG-HE product is significantly greater than that of the IMERG-HL product. More specifically, for the precipitation from 0.1 to 0.3 mm/h, the frequency values of the CMPA data and the three IMERG products are 3.38%, 2.48%, 3.01%, and 3.62%, respectively. These results indicate that the IMERG-HF product slightly exaggerate the detection ability of light precipitation. As the precipitation intensity increases (>1.0 mm/h), the IMERG-HE and IMERG-HL products show more consistent detection capability with the CMPA data, while the IMERG-HF product still has a certain degree of FAR.
In general, the accumulative deviations between the IMERG hourly products and the CMPA data are predominantly contributed by the precipitation with intensity from 1.0 to 10 mm/h. In this range, the overestimation of the IMERG-HE and IMERG-HL products are comparable, but less than that of the IMERG-HF product. In addition, the IMERG-HF product can capture more light precipitation events than the IMERG-HE and IMERG-HL products for the precipitation less than 1.0 mm/h, while the IMERG-HE and IMERG-HL products show a better detection capability for the precipitation more than 1.0 mm/h.

Evaluation on Reconstruction Ability of the Diurnal Precipitation Cycle
The diurnal cycle of precipitation is an important element in the evolution of weather systems, and it is greatly affected by the season and terrain [66]. For the Sichuan Basin, obtaining an accurate diurnal cycle of precipitation will help further study the influencing mechanisms of terrain on atmospheric dynamics and thermodynamics. Therefore, for the three IMERG hourly products, their reconstruction ability of the diurnal precipitation cycle during January 2016 to December 2018 (1096 days in total) over Sichuan Basin was also evaluated.
Because the structure of the diurnal precipitation cycle is strongly influenced by the seasons, we analyzed the IMERG hourly products and CMPA data in each season. The specific processes are as follows: (1) convert from UTC of the IMERG products and CMPA data to LST at the center latitude for Sichuan Basin; (2) adjust the time to get the correct observations from 01:00 to 24:00 LST in one day, because the CMPA data at 00:00 LST represents the accumulated precipitation in the previous hour; (3) gather the accumulation time of effective data and calculate the mean precipitation intensity. The diurnal cycle and the continuous statistical metrics of the IMERG-HE, IMERG-HL, and IMERG-HF products for each season during 2016 to 2018 are shown in Figure 7 and Table 3.  As for the variation characteristics of the diurnal precipitation cycle, the mean precipitation intensity of the CMPA data and the three IMERG hourly products show the same trend with time. However, mean precipitation intensity in spring, summer, and autumn is significantly higher than that in winter. In addition, the IMERG-HF product overestimates precipitation in all the seasons, which also results in a large RB (20.18%) in regional evaluation (Table 2). For the time of peak precipitation in the diurnal cycle, the times of the peak precipitation observed by the CMPA data in the four seasons are at 03:00, 04:00, 06:00, and 02:00 LST, respectively. From Figure 7, it is evident that the three IMERG hourly products can accurately capture the peak precipitation in spring and autumn. However, for the summer and winter peak precipitation, only the IMERG-HF product could accurately reproduce the characteristics of the peak precipitation, while the IMERG-HE and IMERG-HL products show a slight time lag and advance of diurnal peaks, respectively. More specifically, the  As for the variation characteristics of the diurnal precipitation cycle, the mean precipitation intensity of the CMPA data and the three IMERG hourly products show the same trend with time. However, mean precipitation intensity in spring, summer, and autumn is significantly higher than that in winter. In addition, the IMERG-HF product overestimates precipitation in all the seasons, which also results in a large RB (20.18%) in regional evaluation (Table 2). For the time of peak precipitation in the diurnal cycle, the times of the peak precipitation observed by the CMPA data in the four seasons are at 03:00, 04:00, 06:00, and 02:00 LST, respectively. From Figure 7, it is evident that the three IMERG hourly products can accurately capture the peak precipitation in spring and autumn. However, for the summer and winter peak precipitation, only the IMERG-HF product could accurately reproduce the characteristics of the peak precipitation, while the IMERG-HE and IMERG-HL products show a slight time lag and advance of diurnal peaks, respectively. More specifically, the IMERG-HE and IMERG-HL products show a time lag of 2 h and 1 h, respectively, for the peak precipitation in summer, while both of them advance the peak precipitation by 2 h in winter.
In addition, CC, RB, and RMSE are calculated and shown in Table 3. In spring, the CCs of the IMERG-HE, IMERG-HL, and IMERG-HF products are very close (0.95, 0.96, and 0.94, respectively), but the RB and RMSE of the IMERG-HE product are the smallest among the three IMERG hourly products. Additionally, although the CC of the IMERG-HL product is slightly less than that of the IMERG-HF product in summer and autumn, the RB and RMSE are much better than that of the IMERG-HF product. However, for the diurnal precipitation cycle in winter, CC, RB, and RMSE of the IMERG-HF product are better than those of the IMERG-HE and IMERG-HL products.
In general, by synthesizing all the statistical metrics and the ability of accurately capturing peak precipitation, the IMERG-HE and IMERG-HF products show the better capability of reproducing the diurnal cycle in spring and winter, respectively. Additionally, for daily peak precipitation in summer, despite the fact that the capture time of the IMERG-HL product is 1 h later than that of the CMPA data, its comprehensive performance is better than the other IMERG hourly products. With regard to autumn precipitation, the IMERG-HL product performs better than the other IMERG products. Nonetheless, there is still a relatively large deviation in the reconstruction of the diurnal cycle between the IMERG hourly products and the CMPA data, which means that the IMERG hourly products have a long way to go for the optimization of precipitation estimation algorithm. Specifically, the improvement of precipitation estimation algorithm should focus on reducing the overestimation of the daily precipitation in spring, summer, and autumn, and improving the correlation coefficient in winter over Sichuan Basin.
In order to more carefully evaluate the ability of the IMERG hourly products in reproducing the diurnal cycle, we divided the time range from 01:00 to 24:00 into three periods with an interval of 8 h, i.e., 01:00-08:00, 09:00-16:00, and 17:00-24:00. After that, for all precipitation events (intensity > 0.1 mm/h), the mean intensity and frequency of the CMPA data and the three IMERG hourly products are calculated for the four seasons, and their statistical results are shown in Figure 8.
As for the precipitation frequency, in the period of 01:00-08:00 LST in all four seasons, the precipitation frequency observed by the CMPA data is more than 40%, which is much higher than that of other time periods. This phenomenon fully reflected the characteristics of "nocturnal rain" in the Sichuan Basin. Comparing the precipitation frequency of the diurnal cycle observed by the IMERG hourly products to the CMPA data, it can be seen that the two data are very consistent in summer, but some differences exist in other seasons. More specifically, the difference between them mainly occurs at 09:00-16:00 in spring, 01:00-08:00 in autumn, and 09:00-24:00 in winter. In addition, for the precipitation frequency of the diurnal cycle, the IMERG-HE product is apparently closer to the CMPA data than the IMERG-HL and the IMERG-HF products in spring and autumn, but the IMERG-HL product and the IMERG-HF product perform slightly better in summer and winter, respectively. (c) (d) Figure 8. The precipitation intensity and frequency averaged for each season within different time ranges of a day, as derived from the CMPA data and three IMERG hourly products during 2016 to 2018: (a-d) represents spring, summer, autumn, and winter precipitation, respectively; all subgraphs have dual Y-axis: the left Y-axis represents the frequency (bars) of precipitation events, and the right Y-axis represents the mean precipitation intensity (dotted lines).
In conclusion, the statistical results of precipitation intensity and frequency within a diurnal cycle at each time period show that the IMERG-HE product reflect the variation characteristics of diurnal precipitation more accurately in spring, while the IMERG-HL product performs better than the other IMERG hourly products in summer and autumn. In addition, the IMERG-HF product has a better capability of diurnal precipitation reconstruction in winter.

Summary and Conclusions
This study is the first to evaluate the precipitation estimates from the IMERG-HE, IMERG-HL, and IMERG-HF products using the CMPA data over Sichuan Basin across three years (from January 2016 to December 2018). The evaluation results provide useful references for other similar regions, especially for complex terrain regions. Moreover, the evaluation work can also provide valuable insights, not only for the algorithm developers to improve the retrieval processes of the IMERG hourly products, and to achieve a better data quality, especially for the NRT IMERG hourly products, but also for the meteorology related users to select high quality IMERG hourly products in many  Figure 8. The precipitation intensity and frequency averaged for each season within different time ranges of a day, as derived from the CMPA data and three IMERG hourly products during 2016 to 2018: (a-d) represents spring, summer, autumn, and winter precipitation, respectively; all subgraphs have dual Y-axis: the left Y-axis represents the frequency (bars) of precipitation events, and the right Y-axis represents the mean precipitation intensity (dotted lines).
As for the mean precipitation intensity within the diurnal cycle, compared to the IMERG-HE and IMERG-HL products, the IMERG-HF product overestimates the precipitation more severely throughout the day in spring, summer, and autumn. Specifically, all the IMERG hourly products overestimate the precipitation from 01:00 to 08:00 and 17:00 to 24:00 LST in spring, but accurately estimate the precipitation from 09:00 to 16:00 LST in spring. In summer, all the IMERG hourly products also overestimate the precipitation from 01:00 to 08:00 LST, but slightly overestimate the precipitation at other time periods. Moreover, for all time periods of a day in autumn, the IMERG-HF product overestimates the precipitation, while the IMERG-HE and IMERG-HL products slightly underestimate the precipitation. In winter, fewer precipitation events occurred in Sichuan Basin, thus, for the CMPA data and IMERG hourly products, the mean precipitation intensity of the diurnal cycle is relatively small throughout the day, resulting in little difference between them. Overall, all the IMERG hourly products overestimate the early morning (01:00-08:00 LST) precipitation, which may be linked to the increased cloud drop size caused by a higher concentration of aerosols over Sichuan Basin [65].
In conclusion, the statistical results of precipitation intensity and frequency within a diurnal cycle at each time period show that the IMERG-HE product reflect the variation characteristics of diurnal precipitation more accurately in spring, while the IMERG-HL product performs better than the other IMERG hourly products in summer and autumn. In addition, the IMERG-HF product has a better capability of diurnal precipitation reconstruction in winter.

Summary and Conclusions
This study is the first to evaluate the precipitation estimates from the IMERG-HE, IMERG-HL, and IMERG-HF products using the CMPA data over Sichuan Basin across three years (from January 2016 to December 2018). The evaluation results provide useful references for other similar regions, especially for complex terrain regions. Moreover, the evaluation work can also provide valuable insights, not only for the algorithm developers to improve the retrieval processes of the IMERG hourly products, and to achieve a better data quality, especially for the NRT IMERG hourly products, but also for the meteorology related users to select high quality IMERG hourly products in many relevant applications, such as flood early warning, weather disaster forecasting, or hydrological model development. The main conclusions are summarized as follows: (1) For the grid-scale evaluation of the IMERG hourly products, the IMERG-HL product shows comparable performance (high CC and low RMSE at 0.53 and 0.78 mm, respectively) to the PRT IMERG-HF product (CC and RMSE are 0.54 and 0.77 mm, respectively), but the IMERG-HL product reduces RB from 21.16% to 7.99%. By comparison, both of them have a much better performance than the IMERG-HE product (CC and RMSE are 0.46 and 0.86 mm, respectively). In terms of spatial distribution, both of the IMERG-HL and IMERG-HF products have high CC (>0.55) and low FAR (<0.4) in the eastern Sichuan Basin, where heavy precipitation events occur frequently. However, they show low CC (<0.45) and high FAR (>0.65) in the northwestern Sichuan Basin, where light precipitation events occur frequently. For the capability of detecting the precipitation events, the IMERG-HF product demonstrates better performance with highest POD (0.52) and CSI (0.32). However, the IMERG-HF product presents slightly poorer performance (FAR is 0.53) than the IMERG-HL product (FAR is 0.52). Thus, after correcting the deviation using the GPCC data, the precipitation detection capability of the IMERG-HF product is improved obviously, but the FAR has not been improved at all, implying that precipitation retrieval algorithms used in the IMERG-HF product should be revised.
(2) Compared to the grid-scale evaluation, the performance of the IMERG hourly products on regional evaluation is improved a little in regard to almost all statistical metrics, except for the RB. However, the IMERG-HF product still overestimates the precipitation more seriously (RB is 20.18%) than the IMERG-HE and IMERG-HL products (RBs are 3.84% and 4.97%, respectively). In addition, the IMERG-HL product gets a better ability to accurately estimate precipitation than the other IMERG hourly products. Nonetheless, the IMERG-HF product, with the highest POD and lowest FAR, shows a better capability than the other IMERG hourly products in detecting precipitation events over larger areas. For the seasonal evaluation, the performance of the IMERG-HF product is close to the IMERG-HL product with similar CCs, RMSEs, PODs, FARs, and CSIs for all four seasons. Additionally, in terms of RB, the IMERG-HL product shows better performance than the IMERG-HF product in spring, summer, and autumn. Conversely, in winter, the IMERG-HF product only slightly overestimates the precipitation (RB is 3.87%), but the IMERG-HE and IMERG-HL products severely underestimates the precipitation (RB are −28.25% and −27.33%, respectively). The largest RB of the NRT IMERG hourly products suggests that the IMERG algorithm developers should take immediate actions to correct data when the NRT products are used in the study of hydrological forecasting and drought monitoring. For the monthly evaluation, the CC, RMSE, POD, FAR, and CSI profiles of the three IMERG hourly products have the obvious characteristics of seasonal variations, with the peak in June or July and the valley in January or December. In addition, for the rest of the months, the IMERG-HL product is slightly better than the other IMERG hourly products. Compared to the NRT IMERG hourly products, the IMERG-HF product shows better capability of light precipitation (<1.0 mm/h) detection. However, for precipitation intensity more than 1.0 mm/h, the precipitation detection capability of the NRT IMERG hourly products is closer to that of the CMPA data. Furthermore, in the aspect of the cumulative precipitation, the difference between the IMERG-HF product and the CMPA data is the largest than the NRT IMERG hourly products, and the difference mainly comes from moderate precipitation events (1.0-10 mm/h).
(3) Regarding the peak precipitation reconstruction ability of the diurnal cycle, the IMERG-HF product can accurately capture the precipitation in all four seasons, while the IMERG-HE and IMERG-HL products show slight time lag (2 h and 1 h, respectively) of diurnal peaks of intensity for summer precipitation, and show the same time advance (2 h) for winter precipitation. By synthesizing the analysis of the precipitation intensity and frequency, it can be found that the IMERG-HE product shows better capability in reproducing the diurnal cycle for the spring precipitation, while the IMERG-HL product performs better in summer and autumn, and the IMERG-HF product can better reflect the characteristics of the diurnal cycle in winter. In addition, we found that the time of overestimation in the three IMERG hourly products for spring, summer, and autumn precipitation mainly happens in the early morning (01:00-08:00 LST). Therefore, the capability of all the IMERG hourly products should be improved in the future, especially for the early morning precipitation.
In summary, the performance of all the IMERG hourly products is not very satisfactory in most of the above evaluations. Moreover, the IMERG-HF product is not always superior to that of the NRT IMERG hourly products in some evaluations. For instance, its RB and FAR is significantly higher than the NRT IMERG hourly products in grid-scale and region-scale evaluations, which indicates that the monthly gauge correction aggravates the overestimation of the IMERG-HF product over Sichuan Basin. Therefore, the quality of the IMERG hourly products needs to be improved further to get more accurate precipitation estimates, especially for light precipitation. Our future studies on the evaluation of the IMERG hourly products will focus on exploring the source of error and uncertainty over complex mountainous regions, in order to better understand the variations of the precipitation estimation error with different data sources or thresholds and to prompt the development and application of the IMERG algorithms and products.