Accuracy Evaluation of Two High-Resolution Satellite-Based Rainfall Products : TRMM 3 B 42 V 7 and CMORPH in Shanghai

Satellite-based rainfall products have extensive applications in global change studies, but they are known to contain deviations that require comprehensive verification at different scales. In this paper, we evaluated the accuracies of two high-resolution satellite-based rainfall products: the Tropical Rainfall Measurement Mission (TRMM) rainfall product 3B42V7 and the Climate Prediction Center morphing (CMORPH) technique from January 2010 to December 2011 in Shanghai, by using categorical metrics (Probability of Detection, False Alarm Ratio, and Critical Success Index) and statistical indicators (Mean Absolute Error, Root Mean Square Error, Relative Bias, and Correlation Coefficient). Our findings show that, firstly, CMORPH data has a higher accuracy than 3B42V7 at the daily scale, but it underestimates the occurrence frequency of daily rainfall for some intensity ranges. Most errors of the two products are distributed between −10 and 10 mm/day. Second, at the monthly scale, the total accuracy of 3B42V7 is higher than CMORPH in terms of the value of the Correlation Coefficient (CC) and Relative Bias (RB). Finally, CMORPH brings about daily rainfall detection results from categorical metrics computation better than 3B42V7. Generally, the two satellite-based rainfall products show a high correlation with rain gauge data in Shanghai, particularly in spring and winter. Unfortunately, in summer, both of them do not perform well in detecting the short-duration heavy rainfall events. Overall, the relatively poor data accuracy has limited their further applications in Shanghai and similar urban areas.


Introduction
Spatio-temporal patterns of global and regional precipitation are the highlighted focus of climate change in the globally warming world [1][2][3][4].IPCC (Intergovernmental Panel on Climate Change) reports show that the precipitation or extreme precipitation frequency has been increasing with global climate warming that results from elevated atmospheric CO 2 concentrations [5].Human activities and the underlying surface change greatly affect urban atmospheric boundary layers and the local climatic environments and result in more rainfall occurrences and even rainfall extremes [6,7].Rainfall diverges greatly across different regions [8].Different rainfall trends have been observed from many discrete ground meteorological stations [7].However, those studies are limited in terms of both their sparse spatial scope and data accuracy [9,10].
In recent years, satellite-based precipitation estimation has witnessed significant developments.High quality satellite-based rainfall techniques have been developed, e.g., the Tropical Rainfall Measurement Mission (TRMM) Multi-Satellite Precipitation Analysis (TMPA) [11], the Climate Prediction Center morphing (CMORPH) technique [12], the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PRESIANN) [13], and Fengyun meteorological satellite data [14].Some studies have validated the accuracies of these products in different regions all over the world [15][16][17].These products provide an important input for some hydrological models and natural hazard researches [18][19][20], and are used for simulating the rainfall spatio-temporal distribution at global and regional scales [21,22].Previous research indicates that the spatial and temporal accuracies of both the 3B42V7 and CMORPH products are obviously higher in China [23,24], and the three-hourly 0.25 • TRMM 3B42V7 and CMORPH data have a higher correlation with rain gauge data and can simulate precipitation spatio-temporal processes in urban areas [9].Zhang et al. used the TRMM 3B42V6 products to simulate the rainfall spatio-temporal distribution in the Beijing metropolitan region and found that there were fewer rain days and a higher rainfall intensity in urban areas than in other surrounding regions under the impact of urbanization [25].Chen et al. found that short-duration heavy rain may have a downward trend over the metropolitan regions of the Pearl River Delta based on CMORPH data [26].Additionally, Meng et al. used the downscaled TRMM rainfall time series to analyze the response from land surface temperature in a heavy rainfall event in Beijing [27].The above studies show that the two high satellite-based products are widely used for precipitation studies in megacities or cluster cities, but almost no careful validations have been conducted for these two rainfall products in urban areas.In addition, complicated urban environments and underlying surfaces affect rainfall processes [28].Some studies have proved that the underlying surface greatly affects urban atmospheric boundary layers and the local climatic environments, resulting in some rainfall occurrences or rainfall extremes [29].
Because of rapid urbanization in the last two decades, Shanghai has become one of the most crowded megacities in the world.It cannot be ignored that long-term observation shows that rainfall in Shanghai has a distinct characteristic of localization and urban-rural divergence caused by the urbanization process [30].Due to this differentiation, local scale rainfall events may sometimes be difficult to sample by using sparse rain gauge data [30].Only two ground calibration meteorological stations of the Global Precipitation Climate Center (GPCC) are located in the west and north of Shanghai, i.e., Shanghai Hongqiao (CHM00058367) and Shanghai (CHM00058362) [31].The limited number of calibration meteorological stations may bring about some spatial errors in the satellite-based rainfall products in Shanghai or similar megacities.Therefore, it is of great necessity to make a more systematic evaluation of satellite-based rainfall products in the megacity of Shanghai.
In this paper, TRMM 3B42V7 and CMORPH products have been used for accuracy evaluation from two aspects (detection capability and error statistics).The major objective of this study is to understand the quality and usability of the satellite-based rainfall products in Shanghai or similar large-scale urban areas.The rest of this paper is organized as follows.Section 2 describes the study area, and introduces two satellite-based rainfall data and rain gauge data sets.Meanwhile, the evaluation methods and statistic metrics are also presented in Section 2. Section 3 shows the results of accuracies at the daily and monthly scale.Section 4 presents the discussions on errors distribution and extreme rainfall detection.Section 5 summarizes our paper.

Study Area
Shanghai is located in the Yangtze River estuary, within a latitude of 30 • 40 N-31 • 53 N and longitude of 120 • 52 E-122 • 12 E.The south of Shanghai megacity is Hangzhou Bay, and the east is East Sea (Figure 1).Shanghai has a population of 24.15 million, covered an area of 6340.50 km 2 in 2015, and belongs to the northern subtropical monsoon climate.The average annual rainfall is about 1200 mm with obviously seasonal variations, with about 60% of annual rainfall amounting from 1 May to 30 September.

Study Area
Shanghai is located in the Yangtze River estuary, within a latitude of 30°40′ N-31°53′ N and longitude of 120°52′ E-122°12′ E. The south of Shanghai megacity is Hangzhou Bay, and the east is East Sea (Figure 1).Shanghai has a population of 24.15 million, covered an area of 6340.50 km 2 in 2015, and belongs to the northern subtropical monsoon climate.The average annual rainfall is about 1200 mm with obviously seasonal variations, with about 60% of annual rainfall amounting from 1 May to 30 September.

TRMM product
The Tropical Rainfall Measurement Mission (TRMM), launched on 27 November 1997, was jointly developed by the National Aeronautics and Space Administration (NASA) and the National Space Development Agency (NSDA).The Visible and Infrared Radiometer System (VIRS), the TRMM Microwave Imagery (TMI), and the Precipitation Radar (PR) are primary rainfall measurement equipment in the TRMM satellite [32].The Multi-Satellite Precipitation Analysis

• TRMM product
The Tropical Rainfall Measurement Mission (TRMM), launched on 27 November 1997, was jointly developed by the National Aeronautics and Space Administration (NASA) and the National Space Development Agency (NSDA).The Visible and Infrared Radiometer System (VIRS), the TRMM Microwave Imagery (TMI), and the Precipitation Radar (PR) are primary rainfall measurement equipment in the TRMM satellite [32].The Multi-Satellite Precipitation Analysis (TMPA) algorithm was proposed by Huffman et al. [11] and it was combined with several other high-quality rainfall estimation algorithms, e.g., merged Active/Passive Microwave and Infrared-based rainfall estimates, as well as other multi-source data fusion of rainfall products [33].The TMPA rainfall products are available both in 3B42RT (three-hourly) and 3B42 (three-hourly and daily), as well as the monthly rainfall dataset (3B43), covering the range of latitude 50 • S-50 • N and longitude 180 • W-180 • E. Compared to 3B42RT, 3B42V7 was corrected by the monthly rainfall data deviation from the Global Precipitation Climate Center (GPCC) calibration meteorological stations and the product has several computation improvements and a better data accuracy [34].The most recent TRMM 3B42 Version 7 is a gauge-adjusted post-real-time rainfall product covering the 1998-present period from various satellite systems [11].The 3B42V7 data is available from Universal Time Coordinated (UTC) 1 January 1998 to the present.We selected TRMM 3B42V7 rainfall products with a Hierarchical Data File (HDF) format from 2010 to 2011.

• CMORPH product
The CMORPH satellite rainfall product provides global rainfall data generated by the Climate Prediction Center (CPC) under the National Centers for Environmental Prediction (NCEP), and the data is available from UTC 3 December 2002 to present.The CMORPH retrieval algorithm was presented by Joyce et al. [12].It uses motion vectors derived from IR imagery to propagate the relatively high-quality precipitation estimates (QPE) derived from passive microwave data.The spatial distribution of CMORPH rainfall data covers the range of latitude 60 • S-60 • N and longitude 180 • W-180 • E.
The spatio-temporal resolution of CMORPH is three-hourly, 0.25 • , and the same is true for that of 3B42V7.The format of data is '*.comb' from a standard UNIX file, including merged microwave rainfall and 'CMORPH' rainfall estimates.

Rain Gauge Data
Figure 1 shows 11 rain gauges from the Shanghai Meteorological Bureau (SMB) and the rain gauges are distributed in different districts of Shanghai.The detailed information of rain gauges and satellite-based rainfall products is showed in Table 1, including ID, gauge number, name, and row/column number.ID indicates the satellite grid with the row and column number and corresponds to the location of rain gauges.If two or more rain gauges are located in the same satellite grid (e.g., Gauge 58361 and 58367 have the same ID a; Gauge 58461 and 58462 have the same ID h), the average rainfall data from these rain gauges is used for evaluation.The time benchmark of rain gauge data is GMT+8.Given the comparison of satellite-based products and rain gauge data, the following assumptions are made: (1) the rain gauge data is regarded as 'ground truth' to estimate the gridded satellite-based rainfall products [35]; (2) All uncertainty of satellite-based products is due to either random errors or systematic errors [36].The satellite-based products are considered, including systematic and random errors.

Evaluation Methods
Evaluation methods are selected from two aspects: detection capability analysis and error statistics.Detection capability includes three categorical metrics: Probability of Detection (POD) [37], False Alarm Ratio (FAR) [38], and Critical Success Index (CSI) [35], which reflect the capability of satellite-based rainfall products to estimate the possibility of rainfall events.The value of these metrics ranges from 0 to 1.The POD and CSI are positively correlated with the detection capability, whereas FAR is negatively correlated with it.On the basis of determining the threshold of daily rainfall events, these metrics are calculated based on the equations described in Table 2, where H is the number of rainfall events detected by rain gauges and satellites, M is the number just detected by rain gauges, and F is the number detected by satellites [13].

Categorical Statistics
Calculation Formula Optimal Value [0,1] Another aspect is error statistics including four indicators, i.e., Mean Absolute Error (MAE) [39], Root Mean Square Error (RMSE) [40], Relative Bias (RB) [41], and Correlation Coefficient (CC) [42].The equations of these indicators are described in Table 3, reflecting the quantitative consistency of satellite-based rainfall products with rain gauge data.MAE represents the averaged magnitude of the absolute error; RMSE indicates the averaged error magnitude; RB is used to measure the probability of overestimation (RB < 0) or underestimation (RB > 0) from satellite-based products; and CC reflects the synchronicity of precipitation variation between satellite precipitation products and meteorological stations.Compared with rain gauge data, the higher accuracy of satellite-based products has a higher CC, lower MAE, and RMSE [34].

Degree of Relevance Unitless
Notes: n is number of sample.G i is rainfall from rain gauges (mm).S i is estimated rainfall from satellites (mm).G and S are mean values of gauge rainfall and satellite-based rainfall, respectively.

Detection Capability Analysis on Rainfall Events
Based on the equations in Table 2 (see Section 2.3), three metrics of satellite detection capability were computed.In general, when we chose 0.1 mm/day as the threshold of the daily rainfall event, CMORPH had a better detection capability than 3B42V7, with an average POD of 0.68 (0.65), FAR of 0.25 (0.35), and CSI of 0.56 (0.48).We categorized the rainfalls into five classes according to different rainfall thresholds: I. Light rain (0.1-9.9 mm/day), II.Moderate rain (10-24.9mm/day), III.Heavy rain (25-49.9mm/day), IV.Torrential rain (50-99.9mm/day), and V. Extra heavy rain (≥100 mm/day) [43].These rainfall thresholds are also used to calculate categorical metrics, and when the rain gauge detects light rain and the satellite also detects light rain, it is considered a successful detection (H).With the increasing daily rainfall thresholds, POD and CSI of both 3B42V7 and CMORPH show a downward trend.The FAR in Figure 2 has a tendency to increase at first and then decrease at class III.Satellite-based products significantly underestimate heavy rain.Additionally, the higher ratio of rainfall false alarmed is distributed in class II. Figure 3 compares the total detection numbers of satellite-based rainfall and gauge rainfall under different rainfall thresholds.Satellites have lower detection numbers than rain gauges.CMORPH has a higher rainfall detection capability than 3B42V7, especially in light rain events.

Detection Capability Analysis on Rainfall Events
Based on the equations in Table 2 (see Section 2.3), three metrics of satellite detection capability were computed.In general, when we chose 0.1 mm/day as the threshold of the daily rainfall event, CMORPH had a better detection capability than 3B42V7, with an average POD of 0.68 (0.65), FAR of 0.25 (0.35), and CSI of 0.56 (0.48).We categorized the rainfalls into five classes according to different rainfall thresholds: I. Light rain (0.1-9.9 mm/day), II.Moderate rain (10-24.9mm/day), III.Heavy rain (25-49.9mm/day), IV.Torrential rain (50-99.9mm/day), and V. Extra heavy rain (≥100 mm/day) [43].These rainfall thresholds are also used to calculate categorical metrics, and when the rain gauge detects light rain and the satellite also detects light rain, it is considered a successful detection (H).With the increasing daily rainfall thresholds, POD and CSI of both 3B42V7 and CMORPH show a downward trend.The FAR in Figure 2 has a tendency to increase at first and then decrease at class III.Satellite-based products significantly underestimate heavy rain.Additionally, the higher ratio of rainfall false alarmed is distributed in class II. Figure 3 compares the total detection numbers of satellite-based rainfall and gauge rainfall under different rainfall thresholds.Satellites have lower detection numbers than rain gauges.CMORPH has a higher rainfall detection capability than 3B42V7, especially in light rain events.1).Water 2018, 10, 40 6 of 15

Detection Capability Analysis on Rainfall Events
Based on the equations in Table 2 (see Section 2.3), three metrics of satellite detection capability were computed.In general, when we chose 0.1 mm/day as the threshold of the daily rainfall event, CMORPH had a better detection capability than 3B42V7, with an average POD of 0.68 (0.65), FAR of 0.25 (0.35), and CSI of 0.56 (0.48).We categorized the rainfalls into five classes according to different rainfall thresholds: I. Light rain (0.1-9.9 mm/day), II.Moderate rain (10-24.9mm/day), III.Heavy rain (25-49.9mm/day), IV.Torrential rain (50-99.9mm/day), and V. Extra heavy rain (≥100 mm/day) [43].These rainfall thresholds are also used to calculate categorical metrics, and when the rain gauge detects light rain and the satellite also detects light rain, it is considered a successful detection (H).With the increasing daily rainfall thresholds, POD and CSI of both 3B42V7 and CMORPH show a downward trend.The FAR in Figure 2 has a tendency to increase at first and then decrease at class III.Satellite-based products significantly underestimate heavy rain.Additionally, the higher ratio of rainfall false alarmed is distributed in class II. Figure 3 compares the total detection numbers of satellite-based rainfall and gauge rainfall under different rainfall thresholds.Satellites have lower detection numbers than rain gauges.CMORPH has a higher rainfall detection capability than 3B42V7, especially in light rain events.1).1).

Error Statistics Analysis
Another aspect of satellite-based precipitation estimation is error statistics.We calculated four indicators of error statistics in daily and monthly scales.The four indicators (MAE, RMSE, RB, and CC) of error statistics show that the accuracies of CMORPH are higher than those of 3B42V7 at the daily scale.However, the accuracies diverge greatly among different seasons.Figure 4 shows that the MAEs of two satellite-based daily rainfall events vary from Sunday to Saturday.Compared with rain gauge data, the MAEs of satellite daily rainfall are less than 5 mm/day.In ID a, c, e, g, and h, the MAEs are less than 4 mm/day.The MAEs from 3B42V7 are almost larger than those of CMORPH.At the monthly rainfall scale of Figure 5, the two satellite-based rainfall products have higher MAEs in the rainy seasons (from 1 June to 30 September).The accuracies in most rain gauges show that CMORPH has larger MAEs than 3B42V7 in non-rainy seasons.
Water 2018, 10, 40 7 of 15 CC) of error statistics show that the accuracies of CMORPH are higher than those of 3B42V7 at the daily scale.However, the accuracies diverge greatly among different seasons.Figure 4 shows that the MAEs of two satellite-based daily rainfall events vary from Sunday to Saturday.Compared with rain gauge data, the MAEs of satellite daily rainfall are less than 5 mm/day.In ID a, c, e, g, and h, the MAEs are less than 4 mm/day.The MAEs from 3B42V7 are almost larger than those of CMORPH.At the monthly rainfall scale of Figure 5, the two satellite-based rainfall products have higher MAEs in the rainy seasons (from 1 June to 30 September).The accuracies in most rain gauges show that CMORPH has larger MAEs than 3B42V7 in non-rainy seasons.Further cross validations indicate that CMORPH exhibits a smaller MAE (averaged: 2.26 mm) and RMSE (averaged: 7.40 mm) than 3B42V7.Indicated by RB values, 3B42V7 has the most positive bias, whereas CMORPH shows a negative deviation.At last, CC of 3B42V7 (CMORPH) ranges from 0.52 (0.57) to 0.69 (0.66) at the daily scale (see Figure 6).The scatter plots explain that the rainfall data of CMORPH is more obviously underestimated than that of TRMM.Sometimes, rainfall data from 3B42V7 is overestimated.Figure 7 illustrates the distribution of daily rainfall between rain gauge    CC) of error statistics show that the accuracies of CMORPH are higher than those of 3B42V7 at the daily scale.However, the accuracies diverge greatly among different seasons.Figure 4 shows that the MAEs of two satellite-based daily rainfall events vary from Sunday to Saturday.Compared with rain gauge data, the MAEs of satellite daily rainfall are less than 5 mm/day.In ID a, c, e, g, and h, the MAEs are less than 4 mm/day.The MAEs from 3B42V7 are almost larger than those of CMORPH.At the monthly rainfall scale of Figure 5, the two satellite-based rainfall products have higher MAEs in the rainy seasons (from 1 June to 30 September).The accuracies in most rain gauges show that CMORPH has larger MAEs than 3B42V7 in non-rainy seasons.Further cross validations indicate that CMORPH exhibits a smaller MAE (averaged: 2.26 mm) and RMSE (averaged: 7.40 mm) than 3B42V7.Indicated by RB values, 3B42V7 has the most positive bias, whereas CMORPH shows a negative deviation.At last, CC of 3B42V7 (CMORPH) ranges from 0.52 (0.57) to 0.69 (0.66) at the daily scale (see Figure 6).The scatter plots explain that the rainfall data of CMORPH is more obviously underestimated than that of TRMM.Sometimes, rainfall data from 3B42V7 is overestimated.Figure 7 illustrates the distribution of daily rainfall between rain gauge  Further cross validations indicate that CMORPH exhibits a smaller MAE (averaged: 2.26 mm) and RMSE (averaged: 7.40 mm) than 3B42V7.Indicated by RB values, 3B42V7 has the most positive bias, whereas CMORPH shows a negative deviation.At last, CC of 3B42V7 (CMORPH) ranges from 0.52 (0.57) to 0.69 (0.66) at the daily scale (see Figure 6).The scatter plots explain that the rainfall data of CMORPH is more obviously underestimated than that of TRMM.Sometimes, rainfall data from 3B42V7 is overestimated.Figure 7    At the monthly scale, 3B42V7 and CMORPH products have a higher correlation with rain gauge data (see Figures 8 and 9).Similarly, the monthly 3B42V7 data overestimates rainfall at some rain gauges and has a larger deviation in ID d (RB = −26.34%).In contrast, the monthly CMORPH data underestimates rainfall in all of the selected rain gauges, especially in ID a (RB = 33.33%).Generally, in terms of CC and RB, 3B42V7 has higher accuracies than CMORPH.The CC value of 3B42V7 (CMORPH) ranges from 0.90 (0.76) to 0.98 (0.94), and the average CC is 0.94 (0.85).At the monthly scale, 3B42V7 and CMORPH products have a higher correlation with rain gauge data (see Figures 8 and 9).Similarly, the monthly 3B42V7 data overestimates rainfall at some rain gauges and has a larger deviation in ID d (RB = −26.34%).In contrast, the monthly CMORPH data underestimates rainfall in all of the selected rain gauges, especially in ID a (RB = 33.33%).Generally, in terms of CC and RB, 3B42V7 has higher accuracies than CMORPH.The CC value of 3B42V7 (CMORPH) ranges from 0.90 (0.76) to 0.98 (0.94), and the average CC is 0.94 (0.85).At the monthly scale, 3B42V7 and CMORPH products have a higher correlation with rain gauge data (see Figures 8 and 9).Similarly, the monthly 3B42V7 data overestimates rainfall at some rain gauges and has a larger deviation in ID d (RB = −26.34%).In contrast, the monthly CMORPH data underestimates rainfall in all of the selected rain gauges, especially in ID a (RB = 33.33%).Generally, in terms of CC and RB, 3B42V7 has higher accuracies than CMORPH.The CC value of 3B42V7 (CMORPH) ranges from 0.90 (0.76) to 0.98 (0.94), and the average CC is 0.94 (0.85).3B42V7 ((a-h) indicate that the gauge ID in Table 1).(a-h) indicate that the gauge ID in Table 1).

Discussion
In recent years, some studies showed that one important aspect of the observed intensification of rainfall or extreme rainfall events concerns urban areas [8,30].However, the complexity of driving mechanisms in urban areas makes it challenging for satellite-based rainfall products to be reliably simulated.Some studies have emphasized the evaluation of satellite-based rainfall products.In the Beijing-Tianjing-Hebei region, Zhang et al. [44] showed that TRMM satellite data provides a good judgment on rainstorm events and rainfall error control, but the errors vary in different rain levels.However, in this study, the error distribution characteristics of satellite-based rainfall products are not discussed further.Han et al. [9] also evaluated the accuracy of TRMM data in urban areas in different climatic regions (Houston, Atlanta, Las Vegas in the United States and one in Korea, Cheongju), as a function of climatic variables, and for different storm types.The results show that the relative magnitude of TRMM rain accumulation compared to rain gauge accumulation is smaller for higher rainfall intensity events.Tan and Zheng [45] evaluated the detection capability of two TRMM products (3B42 and 3B42RT) in Singapore, and found that satellite products do not capture  a-h) indicate that the gauge ID in Table 1).

Figure 8.
Scatter plots of monthly rainfall from rain gauge data and 3B42V7 ((a-h) indicate that the gauge ID in Table 1).
Figure 9. Scatter plots of monthly rainfall from rain gauge data and the CMORPH data ((a-h) indicate that the gauge ID in Table 1).

Discussion
In recent years, some studies showed that one important aspect of the observed intensification of rainfall or extreme rainfall events concerns urban areas [8,30].However, the complexity of driving mechanisms in urban areas makes it challenging for satellite-based rainfall products to be reliably simulated.Some studies have emphasized the evaluation of satellite-based rainfall products.In the Beijing-Tianjing-Hebei region, Zhang et al. [44] showed that TRMM satellite data provides a good judgment on rainstorm events and rainfall error control, but the errors vary in different rain levels.However, in this study, the error distribution characteristics of satellite-based rainfall products are not discussed further.Han et al. [9] also evaluated the accuracy of TRMM data in urban areas in different climatic regions (Houston, Atlanta, Las Vegas in the United States and one in Korea, Cheongju), as a function of climatic variables, and for different storm types.The results show that the relative magnitude of TRMM rain accumulation compared to rain gauge accumulation is smaller for higher rainfall intensity events.Tan and Zheng [45] evaluated the detection capability of two TRMM products (3B42 and 3B42RT) in Singapore, and found that satellite products do not capture  1).

Discussion
In recent years, some studies showed that one important aspect of the observed intensification of rainfall or extreme rainfall events concerns urban areas [8,30].However, the complexity of driving mechanisms in urban areas makes it challenging for satellite-based rainfall products to be reliably simulated.Some studies have emphasized the evaluation of satellite-based rainfall products.In the Beijing-Tianjing-Hebei region, Zhang et al. [44] showed that TRMM satellite data provides a good judgment on rainstorm events and rainfall error control, but the errors vary in different rain levels.However, in this study, the error distribution characteristics of satellite-based rainfall products are not discussed further.Han et al. [9] also evaluated the accuracy of TRMM data in urban areas in different climatic regions (Houston, Atlanta, Las Vegas in the United States and one in Korea, Cheongju), as a function of climatic variables, and for different storm types.The results show that the relative magnitude of TRMM rain accumulation compared to rain gauge accumulation is smaller for higher rainfall intensity events.Tan and Zheng [45] evaluated the detection capability of two TRMM products (3B42 and 3B42RT) in Singapore, and found that satellite products do not capture any extreme rainfall events (>50 mm/day) during the northeast monsoon period.In Shanghai, the amount of extreme rainfall accounted for 66.97% of the average annual rainfall from 2010 to 2011.It is necessary to evaluate the accuracy of extreme rainfall.Based on the results mentioned, error distribution characteristics and extreme rainfall detection should be explored further.

The Error Distribution Characteristics of 3B42V7 and CMORPH
The satellite-based rainfall products include some errors caused by a variety of factors, and they can be divided into two major categories: one is sampling errors and the other is retrieval errors [33].The sampling errors come from the grid data processing, for instance, the continuous spatial-temporal data is often made by integrating the sparse data [33].The sampling errors may be accumulated and propagated from the three-hourly scale to the daily or monthly scale.More detailed characteristics of sampling errors can be referred to in the literatures by Demaria et al. [46] and Steiner et al. [47].Retrieval errors are generated when deriving high quality precipitation estimates from multiple satellite rainfall observations.It correlates closely with satellite sensors, retrieval algorithms, rainfall types, and cloud physical characteristics [48].Retrieval error can be further decomposed into random errors and systematic deviations [16].Systematic deviations are caused by retrieval algorithms, which are characterized by underestimation or overestimation of satellite rainfall products.
Therefore, it is important to analyze the error distribution characteristics of 3B42V7 and CMORPH.The statistical principle we adopted is the absolute error of daily rainfall between satellite data and rain gauge data (more than 0.1 mm/day).If the error is greater than 0, satellite data is underestimated; otherwise, the satellite data is overestimated.Figures 10 and 11 illustrate the histograms of the frequency of daily rainfall errors from 3B42V7 and CMORPH, with an interval of 5 mm.The fitted curve shows the distribution of general errors.Skewness coefficient (S) is used to characterize the reciprocity of the data around the means of the sample [49]; if S is positive (negative), the peak of the distribution curve is on the left (right) side of the mean value, and the right (left) tail is thicker.The qualitative indicator illustrates that the histogram is more likely to be right (left) offset, and so the satellite data error is biased in terms of underestimating (overestimating) the measured rainfall.
The two figures show that the most errors of 3B42V7 and CMORPH are between −10 and 10 mm/day.According to the fitted distribution curve, CMORPH error is more concentrated in the range of −5 to 5 mm/day, and 3B42V7 error is relatively discrete.The results of the two satellites are different in terms of the S values.In ID a (S = −2.53)and h (S = −4.46), the lowest values of 3B42V7 approach −150, and that indicates that 3B42V7 has more overestimated results than the rain gauge data.
Water 2018, 10, 40 10 of 15 any extreme rainfall events (>50 mm/day) during the northeast monsoon period.In Shanghai, the amount of extreme rainfall accounted for 66.97% of the average annual rainfall from 2010 to 2011.It is necessary to evaluate the accuracy of extreme rainfall.Based on the results mentioned, error distribution characteristics and extreme rainfall detection should be explored further.

The Error Distribution Characteristics of 3B42V7 and CMORPH
The satellite-based rainfall products include some errors caused by a variety of factors, and they can be divided into two major categories: one is sampling errors and the other is retrieval errors [33].The sampling errors come from the grid data processing, for instance, the continuous spatial-temporal data is often made by integrating the sparse data [33].The sampling errors may be accumulated and propagated from the three-hourly scale to the daily or monthly scale.More detailed characteristics of sampling errors can be referred to in the literatures by Demaria et al. [46] and Steiner et al. [47].Retrieval errors are generated when deriving high quality precipitation estimates from multiple satellite rainfall observations.It correlates closely with satellite sensors, retrieval algorithms, rainfall types, and cloud physical characteristics [48].Retrieval error can be further decomposed into random errors and systematic deviations [16].Systematic deviations are caused by retrieval algorithms, which are characterized by underestimation or overestimation of satellite rainfall products.
Therefore, it is important to analyze the error distribution characteristics of 3B42V7 and CMORPH.The statistical principle we adopted is the absolute error of daily rainfall between satellite data and rain gauge data (more than 0.1 mm/day).If the error is greater than 0, satellite data is underestimated; otherwise, the satellite data is overestimated.Figures 10 and 11 illustrate the histograms of the frequency of daily rainfall errors from 3B42V7 and CMORPH, with an interval of 5 mm.The fitted curve shows the distribution of general errors.Skewness coefficient (S) is used to characterize the reciprocity of the data around the means of the sample [49]; if S is positive (negative), the peak of the distribution curve is on the left (right) side of the mean value, and the right (left) tail is thicker.The qualitative indicator illustrates that the histogram is more likely to be right (left) offset, and so the satellite data error is biased in terms of underestimating (overestimating) the measured rainfall.
The two figures show that the most errors of 3B42V7 and CMORPH are between −10 and 10 mm/day.According to the fitted distribution curve, CMORPH error is more concentrated in the range of −5 to 5 mm/day, and 3B42V7 error is relatively discrete.The results of the two satellites are different in terms of the S values.In ID a (S = −2.53)and h (S = −4.46), the lowest values of 3B42V7 approach −150, and that indicates that 3B42V7 has more overestimated results than the rain gauge data.

Extreme Rainfall Evaluation of 3B42V7 and CMORPH
From the above analysis in Sections 3.1 and 3.2, we found that the accuracies of satellite-based rainfall products decrease with increasing rainfall amounts.Particularly, in heavy or extreme rainfall events, satellite-based rainfall products have lower accuracies.In other words, satellite-based products are not suitable for a heavy rainfall evaluation.
For example, on the 4 July 2010, the daily rainfall of six rain gauges exceeded more than 50 mm/day, and the total extreme rainfall was 383.50 mm.But the number of satellites showed that only one gridded data set of 3B42V7 and four gridded data sets of CMORPH were detected, and the total extreme rainfall was 50.40 mm and 310.46 mm, respectively.An extreme rainfall event is defined as the daily rainfall over 50 mm in Shanghai [50].We selected 21 extreme rainfall events from rain gauge data and obtained the corresponding gridded data of the two satellite-based products from January 2010 to December 2011.It indicates that the daily rainfall of 3B42V7 (CMORPH) has 14 (eight) results that exceed 50 mm.3B42V7 has a higher detection capability of extreme rainfall than CMORPH, but the detection numbers from them both do not exceed half of the extreme rainfall events from rain gauge data.Table 4 lists detailed information of the extreme rainfall events.

Extreme Rainfall Evaluation of 3B42V7 and CMORPH
From the above analysis in Sections 3.1 and 3.2, we found that the accuracies of satellite-based rainfall products decrease with increasing rainfall amounts.Particularly, in heavy or extreme rainfall events, satellite-based rainfall products have lower accuracies.In other words, satellite-based products are not suitable for a heavy rainfall evaluation.
For example, on the 4 July 2010, the daily rainfall of six rain gauges exceeded more than 50 mm/day, and the total extreme rainfall was 383.50 mm.But the number of satellites showed that only one gridded data set of 3B42V7 and four gridded data sets of CMORPH were detected, and the total extreme rainfall was 50.40 mm and 310.46 mm, respectively.An extreme rainfall event is defined as the daily rainfall over 50 mm in Shanghai [50].We selected 21 extreme rainfall events from rain gauge data and obtained the corresponding gridded data of the two satellite-based products from January 2010 to December 2011.It indicates that the daily rainfall of 3B42V7 (CMORPH) has 14 (eight) results that exceed 50 mm.3B42V7 has a higher detection capability of extreme rainfall than CMORPH, but the detection numbers from them both do not exceed half of the extreme rainfall events from rain gauge data.Table 4 lists detailed information of the extreme rainfall events.

Conclusions
In this study, we used two evaluation methods and explored the accuracies of two satellite-based rainfall products: TRMM 3B42V7 and CMORPH, from January 2010 to December 2011 in Shanghai.The conclusions are summarized as follows.
During the study period, when we chose 0.1 mm/day as the threshold of the daily rainfall event, the results of POD, FAR, and CSI showed that CMORPH has a better detection capability than 3B42V7 in the daily rainfall detection, with a higher POD (0.68) and CSI (0.56) and a lower FAR (0.25).The capability of detecting rainfall from two satellites decreases as the rainfall amounts increase, and both satellite-based rainfall products have a lower detection capability for heavy rainfall events.
At the daily scale, CMORPH has lower MAE and RMSE values than those of 3B42V7.Some RBs of 3B42V7 overestimate the rain gauge rainfall, whereas the CMORPH underestimates the rainfall.In summer, both 3B42V7 and CMORPH have the highest RMSE of 15.53 mm/day and 13.13 mm/day, respectively.At the monthly scale, 3B42V7 shows a better performance than CMORPH, in terms of the value of CC and RB, and the two satellites have higher correlations with rain gauge data and higher MAEs in rainy seasons (from 1 June to 30 September).In non-rainy months, CMORPH has higher MAEs than 3B42V7.3B42V7 and CMORPH satellite rainfall products show more uncertainty at the daily scale than at the monthly scale.The errors of 3B42V7 and CMORPH are concentrated in the range from −10 to 10 mm/day.Both 3B42V7 and CMORPH have a worse detection capability for extreme rainfall events.
To summarize, 3B42V7 and CMORPH satellite-based products with a daily resolution have a poor performance in Shanghai, especially for a heavy rainfall evaluation.The accuracy of 3B42V7 and CMORPH at the monthly scale is higher, and the accuracy of 3B42V7 is higher than that of CMORPH.The monthly rainfall data from satellite-based rainfall products can be parameters that are input to the hydrological model and data assimilation [51][52][53][54].In the future, we will integrate the two evaluation methods and deeply analyze the calculation of different error statistics to elucidate insights into rainfall detection.Moreover, we will integrate the multi-source satellite-based rainfall products to improve the spatio-temporal resolution using data assimilation and statistical downscaling approaches.

Figure 1 .
Figure 1.The locations of the rain gauges in Shanghai.

Figure 1 .
Figure 1.The locations of the rain gauges in Shanghai.

Figure 2 .
Figure 2. The overall detection capacity of 3B42V7 and CMORPH in different rainfall thresholds.

Figure 3 .
Figure 3.The detection results of different rainfall thresholds ((a-h) indicate that the gauge ID in Table1).

Figure 2 .
Figure 2. The overall detection capacity of 3B42V7 and CMORPH in different rainfall thresholds.

Figure 2 .
Figure 2. The overall detection capacity of 3B42V7 and CMORPH in different rainfall thresholds.

Figure 3 .
Figure 3.The detection results of different rainfall thresholds ((a-h) indicate that the gauge ID in Table1).

Figure 3 .
Figure 3.The detection results of different rainfall thresholds ((a-h) indicate that the gauge ID in Table1).

Figure 4 .
Figure 4.The variations of MAEs from 3B42V7 and CMORPH at the daily scale from Sunday to Saturday ((a-h) indicate that the gauge ID in Table1).

Figure 4 .
Figure 4.The variations of MAEs from 3B42V7 and CMORPH at the daily scale from Sunday to Saturday ((a-h) indicate that the gauge ID in Table1).

Figure 4 .
Figure 4.The variations of MAEs from 3B42V7 and CMORPH at the daily scale from Sunday to Saturday ((a-h) indicate that the gauge ID in Table1).

Figure 5 .
Figure 5.The variations of MAEs from 3B42V7 and CMORPH at the monthly scale ((a-h) indicate that the gauge ID inTable1).
illustrates the distribution of daily rainfall between rain gauge data and 3B42V7 (CMORPH) in different seasons.CMORPH has a smaller RMSE (Spring: 4.03 mm; Summer: 13.13 mm; Autumn: 3.13 mm; Winter: 1.87 mm) than 3B42V7 (Spring: 7.29 mm; Summer: 15.53 mm; Autumn: 5.86 mm; Winter: 4.04 mm) in all seasons, and the largest RMSEs occur in Summer and the least in Winter.The RB result shows that CMORPH underestimates the daily rainfall by 42.36%, 4.59%, 47.03%, and 67.00% in the different seasons, respectively.Thus, 3B42V7 data only explains an underestimated trend in the winter (RB = 14.62%) and overestimates the rainfall in other seasons.As for CC, CMORPH has a stronger correlation with rain gauge observations, especially in spring (CC = 0.84).Water 2018, 10, 40 8 of 15 data and 3B42V7 (CMORPH) in different seasons.CMORPH has a smaller RMSE (Spring: 4.03 mm; Summer: 13.13 mm; Autumn: 3.13 mm; Winter: 1.87 mm) than 3B42V7 (Spring: 7.29 mm; Summer: 15.53 mm; Autumn: 5.86 mm; Winter: 4.04 mm) in all seasons, and the largest RMSEs occur in Summer and the least in Winter.The RB result shows that CMORPH underestimates the daily rainfall by 42.36%, 4.59%, 47.03%, and 67.00% in the different seasons, respectively.Thus, 3B42V7 data only explains an underestimated trend in the winter (RB = 14.62%) and overestimates the rainfall in other seasons.As for CC, CMORPH has a stronger correlation with rain gauge observations, especially in spring (CC = 0.84).

Figure 6 .
Figure 6.The correlations of different indicators between 3B42V7 and CMORPH at the daily scale.

Figure 7 .
Figure 7. Scatter plots of daily rainfall from rain gauge data and 3B42V7 (CMORPH) data in four seasons (Spring: 1 March to 31 May; Summer: 1 June to 31 August; Autumn: 1 September to 30 November; Winter: 1 December to 27 February in second year).

Figure 6 .
Figure 6.The correlations of different indicators between 3B42V7 and CMORPH at the daily scale.

Figure 6 .
Figure 6.The correlations of different indicators between 3B42V7 and CMORPH at the daily scale.

Figure 7 .
Figure 7. Scatter plots of daily rainfall from rain gauge data and 3B42V7 (CMORPH) data in four seasons (Spring: 1 March to 31 May; Summer: 1 June to 31 August; Autumn: 1 September to 30 November; Winter: 1 December to 27 February in second year).

Figure 7 .
Figure 7. Scatter plots of daily rainfall from rain gauge data and 3B42V7 (CMORPH) data in four seasons (Spring: 1 March to 31 May; Summer: 1 June to 31 August; Autumn: 1 September to 30 November; Winter: 1 December to 27 February in second year).

Figure 8 .
Figure 8. Scatter plots of monthly rainfall from rain gauge data and3B42V7 ((a-h) indicate that the gauge ID in Table1).

Figure 9 .
Figure 9. Scatter plots of monthly rainfall from rain gauge data and the CMORPH data ((a-h) indicate that the gauge ID in Table1).

Figure 8 .
Figure 8. Scatter plots of monthly rainfall from rain gauge data and3B42V7 ((a-h) indicate that the gauge ID in Table1).

Figure 9 .
Figure 9. Scatter plots of monthly rainfall from rain gauge data and the CMORPH data ((a-h) indicate that the gauge ID in Table1).

Figure 10 .
Figure 10.The histogram of daily rainfall error distribution for 3B42V7.

Figure 10 .
Figure 10.The histogram of daily rainfall error distribution for 3B42V7.

Figure 11 .
Figure 11.The histogram of daily rainfall error distribution for CMORPH.

Figure 11 .
Figure 11.The histogram of daily rainfall error distribution for CMORPH.

Table 1 .
Detailed information of rain gauges and satellite-based rainfall products.

Table 2 .
Categorical metrics of detection capability.

Table 3 .
Evaluation indicators of error statistics.

Table 4 .
The detection results of extreme rainfall events of the rain gauge, the 3B42V7, and CMORPH from 2010 to 2011.

Table 4 .
The detection results of extreme rainfall events of the rain gauge, the 3B42V7, and CMORPH from 2010 to 2011.