Comprehensive Evaluation of High-Resolution Satellite-Based Precipitation Products over China

Abstract

Characterizing the errors in satellite-based precipitation estimation products is crucial for understanding their effects in hydrological applications. Six precipitation products derived from three algorithms are comprehensively evaluated against gauge data over mainland China from December 2006 to November 2010. These products include three satellite-only estimates: the Global Satellite Mapping of Precipitation Microwave-IR Combined Product (GSMaP_MVK), the Climate Prediction Center (CPC) MORPHing (CMORPH), and Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN), as well as their gauge-corrected counterparts: the GSMaP Gauge-calibrated Product (GSMaP_Gauge), bias-corrected CMORPH (CMORPH_CRT), and PERSIANN Climate Data Record (PERSIANN-CDR). Overall, the bias-correction procedures largely reduce various errors for the three groups of satellite-based precipitation products. GSMaP_Gauge produces better fractional coverage with the highest correlation (0.95) and the lowest RMSE (0.53 mm/day) but also high RB (15.77%). In general, CMORPH_CRT amounts are closer to the gauge reference. CMORPH shows better performance than GSMaP_MVK and PERSIANN with the highest CC (0.82) and the lowest RMSE (0.93 mm/day), but also presents a relatively high RB (−19.60%). In winter, all six satellite precipitation estimates have comparatively poor capability, with the IR-based PERSIANN_CDR exhibiting the closest performance to the gauge reference. Both satellite-only and gauge-corrected satellite products show poor capability in detecting occurrence of precipitation with a low POD (<50%) and CSI (<35%) and a high FAR (>40%).

1. Introduction

Precipitation measurements provide useful information for evaluating the global water cycle and energy balance and also supply crucial input for hydrological applications [1]. However, obtaining reliable surface precipitation measurements is challenging due to the heterogeneity of precipitation at different spatiotemporal scales. Precipitation estimates obtained from rain gauges are generally considered the most accurate estimates since they directly measure the precipitation. Unfortunately, rain gauges tend to be unevenly distributed and gauge coverage is often quite sparse over remote/rural areas. This sparsity constitutes a significant challenge because gauge-based rainfall estimates are often not representative of nearby rainfall rates in precipitation events containing large gradients of precipitation rates [2]. With the development of remote sensing technology, it is possible to obtain Quantitative Precipitation Estimates (QPE) over regional and global scales, which consequently increase our understanding of the energy balance and water cycle. At present, a series of multi-sensor blended global precipitation products have been released that offer improvements over single sensor estimates. Some notable products of this type include the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) [3,4], the National Oceanic and Atmospheric Administration’s (NOAA) CPC morphing technique (CMORPH) [5], the Global Satellite Mapping of Precipitation (GSMaP) project [6], and the Tropical Rainfall Measuring Mission (TRMM) Multi-satellite Precipitation Analysis (TMPA) [7]. Many studies have investigated the uncertainties of satellite-based precipitation products over varying climatic, geographic and topographic regimes [8,9,10,11,12,13], and these products have been increasingly utilized in natural hazard (e.g., flood and landslide) monitoring, climate research, and hydrology-related fields [14,15,16,17,18,19,20]. Despite the proven utility of these techniques, the error sources associated with these products are generally not well understood. In addition, with the continuing evolution of retrieval algorithms and changes in data sources, the performance of these products continues to change [21,22]. With this variability in mind, the errors and performance of an algorithm must be characterized in order to provide useful estimations [23]. Furthermore, the quantification of error characteristics is necessary for data assimilation [24], development and improvement of the retrieval algorithms, climate analysis [25], hydrological modeling [26,27], as well as hazards’ monitoring and forecasting [1,28].

Much effort has been made to evaluate the TMPA, including comparisons using the near real time CMORPH and PERSIANN over different regions [4,29,30,31,32,33,34,35,36]. Similar comparisons have been made using GSMaP_MVK products over Japan, Nepal, Thailand, Vietnam and the United States [6,37,38,39,40,41,42,43]. However, to the best of our knowledge, there has not yet been a systematic evaluation of the gauge-corrected GSMaP product (GSMaP_Gauge, [44]) over China. Additionally, there have been few studies on the performance of the bias-corrected CMORPH and PERSIANN products (CMORPH_CRT and PERSIANN_CDR). The satellite-only datasets, which include GSMaP_MVK, CMORPH, and PERSIANN, have been extensively evaluated over China in previous studies [45,46,47,48,49,50].

The primary objective of this study is to identify the strengths and weaknesses of the currently popular satellite-based QPE products (i.e., GSMaP, CMORPH, and PERISANN) over China. Herein, GSMaP_MVK, CMORPH, and PERSIANN will refer to the satellite-only precipitation estimates, while GSMaP_Gauge, CMORPH_CRT, and PERSIANN_CDR will denote the corresponding bias-corrected estimates. Specifically, the error structures of each product are characterized as a function of time and space, and therefore, this study may assist in determining the optimal selection of QPE products for hydro-meteorological applications over China.

The remaining parts of this paper are organized as follows: Section 2 introduces the satellite-based and gauge precipitation datasets as well as the evaluation methods. Section 3 provides an analysis of the spatial characteristics and error quantification for different satellite-based QPE products. A summary of analysis results and notable conclusions are given in Section 4.

2. Study Area and Dataset

2.1. Study Area

Figure 1 shows the general topography of China via a digital elevation model (DEM). Generally, China can be partitioned into seven sub-regions classified by elevation, mountain ranges [51] and the annual mean precipitation distribution [52]. These sub-regions are shown in Figure 1 and are referred to as: (I) the Xinjiang (XJ) region in central Asia, which has arid and semi-arid climate characteristics; (II) Qinghai-Tibet Plateau (TP) which has an average elevation of about 4500 m; (III) Northwestern China (NW) bounded by the 400 mm annual precipitation isohyet; (IV) Northeastern China (NE) located in the north of the Yan mountain; (V) Northern China (NC) located to the north of Qinling Mountains-Huai River line; (VI) Yunnan-Guizhou Plateau in southwestern China (SW) which is bounded by the Ta-pa Mountains and Wulingshan mountains to the north and east; and (VII) Southern China (SC) south of the Nanling mountains and southeast of the Wuyi Mountains [53].

Figure 1. Topographic map of China. The black solid lines indicate the borders of the seven sub-regions: (i) Xinjiang (XJ); (ii) Qinghai-Tibetan plateau (TP); (iii) Northwest (NW); (iv) Northeast (NE); (v) North (NC); (vi) southwest Yungui Plateau (SW); and (vii) South (SC).

Figure 1. Topographic map of China. The black solid lines indicate the borders of the seven sub-regions: (i) Xinjiang (XJ); (ii) Qinghai-Tibetan plateau (TP); (iii) Northwest (NW); (iv) Northeast (NE); (v) North (NC); (vi) southwest Yungui Plateau (SW); and (vii) South (SC).

Precipitation in the XJ sub-region is primarily influenced by the mid-latitude westerlies and moisture contributions from the North Atlantic Ocean [54]. The subtropical monsoon, when coupled with orographic effects from complex terrain, results in large amounts of precipitation over southwestern, southeastern, and southern China. The landfall of typhoons [55] and the annual migration of the monsoon [45,46] result in the plum rain during June and July in the south and southeastern areas of China. The precipitation in southwestern China is primarily dominated by the Indian Ocean monsoon, and topographic influences resulting from the Qinghai-Tibet Plateau.

2.2. Satellite-Based Precipitation Dataset

The GSMaP project [6,56,57,58] was originally sponsored by the Japan Science Technology Agency (JST) and is now sponsored by the Japan Aerospace Exploration Agency (JAXA). It uses extensible input streams, including geostationary satellite data, Low Earth Orbit (LEO) satellite data, and some other ancillary data. The LEO passive microwave (PMW) data and infrared retrievals are merged using a Microwave-IR combined algorithm, a backward and forward morphing technique based on IR images [5] and a Kalman filter [6]. The Microwave radiometer algorithm applied in GSMaP uses various attributes from TRMM data to get the PMW retrievals [56,57,59]. Over land, the rain/no-rain classification (RNC) methods [60] have been improved by Radiative Transfer Model (RTM) calculations using hydrometeor profiles retrieved from the TRMM Precipitation Radar (PR), a melting layer model, and a scattering algorithm [61]. Without a bias-correction procedure, GSMaP_MVK is a satellite-only product, while the NOAA Climate Prediction Center (CPC) global rain gauge dataset and the global gauge data analysis by Xie [62] are introduced in GSMaP_Gauge dataset [44]. The version 5 products of GSMaP were selected for evaluation in this study, although the newer version products (version 6) with time period ranging from 2011–2014 have been recently provided by JAXA.

The CMORPH technique propagates rainfall estimates derived from PMW sensors using motion vectors derived from the half-hourly interval geostationary satellite IR imagery [5]. The previous CMORPH versions 0.x only supplied the satellite-only dataset (CMORPH, hereafter), while the latest CMORPH version 1.0 supplies both the satellite-only dataset and the bias-adjusted dataset (CMORPH_CRT, hereafter). The products are available in three spatial and temporal versions: 8 km-30 min, 0.25°-3 hourly and 0.25°-daily. The 0.25°/daily version is used in this paper to facilitate comparison with other products. In CMORPH_CRT, bias correction was conducted by adjusting the CMORPH satellite estimates with daily gauge analysis through PDF matching [63].

PERSIANN utilizes another kind of satellite-based estimation technique that uses local cloud textures from infrared images of the geostationary satellites to estimate surface rainfall rates based on an artificial neural network algorithm [3,4]. PERSIANN primarily uses infrared brightness temperature data from geostationary satellites to estimate rainfall rates, while updating parameters based on LEO satellite PMW observations [64]. To maintain consistency with Global Precipitation Climatology Project (GPCP) products, PERSIANN_CDR is adjusted using the GPCP monthly precipitation product at the 2.5°/monthly scale throughout the entire record [64].

Table 1 gives a brief summary of satellite-based precipitation products. Although the algorithms vary significantly, more or less similar input data are used in these products. GSMaP is similar to CMORPH in that the cloud motion vectors are derived from IR brightness temperature cloud top movement. Also, both GSMaP and CMORPH use a Kalman filter technique to assimilate propagated microwave and IR estimates where instantaneous microwave observations are not available [5]. Both satellite-only and bias-corrected satellite-based precipitation products are evaluated in this paper in order to provide a quantitative evaluation of the bias correction procedures for different applications. In order to eliminate the sampling biases in the evaluation, and to be consistent with the reference dataset, all satellite-based datasets were spatiotemporally aggregated to the 0.25°/daily resolution.

Table 1. Summary of satellite Quantitative Precipitation Estimates (QPE).

Name	Temporal Resolution	Spatial Resolution	Domain	Corrected by Gauges	References
GSMaP-MVK	1 h	0.1°	60°S–60°N	No	[57,58]
GSMaP-Gauge	1 h	0.1°	60°S–60°N	Yes	[44,57,58]
CMORPH	1 day	0.25°	60°S–60°N	No	[5]
CMORPH-CRT	1 day	0.25°	60°S–60°N	Yes	[65]
PERSIANN	3 h	0.25°	60°S–60°N	No	[3,4]
PRESIANN-CDR	1 day	0.25°	60°S–60°N	Yes	[3,4,64]

Table 1. Summary of satellite Quantitative Precipitation Estimates (QPE).

Name	Temporal Resolution	Spatial Resolution	Domain	Corrected by Gauges	References
GSMaP-MVK	1 h	0.1°	60°S–60°N	No	[57,58]
GSMaP-Gauge	1 h	0.1°	60°S–60°N	Yes	[44,57,58]
CMORPH	1 day	0.25°	60°S–60°N	No	[5]
CMORPH-CRT	1 day	0.25°	60°S–60°N	Yes	[65]
PERSIANN	3 h	0.25°	60°S–60°N	No	[3,4]
PRESIANN-CDR	1 day	0.25°	60°S–60°N	Yes	[3,4,64]

2.3. Ground Reference Data

The China daily Precipitation Analysis Product (CPAP), which is produced and routinely calibrated by the National Meteorological Information Center (NMIC) and the China Meteorological Administration (CMA), was used as a reference to evaluate the satellite-based precipitation products. Approximately 2400 rain gauge observations are gridded with the climatology-based Optimal Interpolation (OI) technique to yield the 0.25°/daily CPAP. The OI technique shows a higher correlation with individual gauge measurements than other techniques [66], and CPAP exhibits a relative bias of 3.21% at the 0.5° scale when compared to independent gauge observations in the validation [67]. It is important to keep in mind that the gauges used in CPAP may not be completely independent from those used in GSMaP_Gauge, CMORPH_CRT and PERSIANN_CDR. According to the China Meteorological Administration (CMA) [68], approximately 500 of the total gauges (~2400) used in CPAP are also potentially adopted in the GPCP data set which is used in PERSIANN_CDR. In addition, the NOAA Climate Prediction Center (CPC) analysis which includes some GTS gauges from CPAP is used in both GSMaP_Gauge [44] and CMORPH_CRT [69]. Furthermore, it should be noted that both CMORPH_CRT and GSMaP_Gauge are calibrated by daily rain gauge analysis while PERSIANN_CDR is calibrated by monthly rain gauge analysis.

2.4. Statistical Evaluation Metrics

GSMaP, CMORPH and PERSIANN products are systematically and quantitatively assessed by using a series of standard statistical evaluation metrics which include: Bias (the difference between the satellite-based products and CPAP), Relative Bias (RB, the bias divided by CPAP), Root-Mean-Square-Error (RMSE) and the Pearson linear correlation coefficient (CC) between satellite precipitation products and CPAP. RB and CC are dimensionless, whereas RMSE is in units of mm/day. When multiplied by 100, RB denotes the percentage of overestimation or underestimation. In order to evaluate the detection capability of these satellite-based precipitation products, three categorical statistical indices are calculated. Namely, the probability of detection (POD), false alarm ratio (FAR), and critical success index (CSI) are used to evaluate the capability in detecting rainfall events. Details of the definitions of these metrics can be found in [70]. Taylor diagrams are also used to visualize and quantify the overall performance of various products. The definition of the Taylor diagram can be found in [71].

3. Results and Discussion

3.1. Four Year Daily Mean Precipitation

Figure 2a shows the distribution of rain gauges over China that form the basis for CPAP. The gauge network is fairly dense in eastern China, while being uneven and sparse in northwestern China, particularly in the areas of XJ, TP, and NW. All satellite-based datasets can generally capture the spatial pattern of four-year mean daily precipitation (Figure 2b–h), but there are pronounced differences between the satellite-only precipitation products and bias-corrected products. GSMaP_Gauge, CMORPH_CRT, and PERSIANN_CDR, capture the spatial pattern of precipitation over China fairly well, while the satellite-only products perform much worse than the bias-corrected products. The satellite-only products exhibit significant underestimation over southwestern, southern, and southeastern China. Additionally, GSMaP_MVK shows a strip of intense precipitation along the northern hillside of the Tibetan Plateau. It is notable that CMORPH has many isolated grid boxes of abnormal overestimation, particularly in the TP and northern Loess Plateau in NW, which cannot be completely removed by the gauge-correction procedure. Consequently, those abnormalities are still evident in the CMORPH_CRT product. These features indicate that the bias-correction procedure in CMORPH_CRT fails to remove such overestimation “speckles”, some of which may be related to the complex mountainous terrain. PERSIANN overestimates precipitation in the southeastern TP and underestimates over the Tianshan Mountains in western XJ. After bias-correction, the PERSIANN_CDR shows a significant improvement over TP, but little improvement over the Tianshan Mountains. Generally, all bias-corrected satellite-based precipitation products agree much better with CPAP than their corresponding satellite-only counterparts according to the pattern of spatial precipitation distribution, and GSMaP_Gauge produces the closest spatial precipitation distribution pattern with CPAP among the bias-corrected products.

Figure 3a–f shows density-colored scatter plots and provide a quantitative comparison of all satellite-based precipitation products versus CPAP in terms of four-year mean precipitation over China. The high CPAP values in each scatterplots are caused by the heavy precipitation events over regions with complicated terrain (e.g., southwestern China) where satellite-based precipitation products struggle to estimate precipitation accurately. In addition, there seems to be odd gaps below the 1-to-1 line for CPAP in all scatterplots at 1–3 mm/day. This odd phenomenon may be related to the quality of CPAP. An additional validation based on four-year-mean precipitation (not shown here) shows that CPAP tends to observe high precipitation frequency from 0.8 mm/day to 2 mm/day over mainland China which may be a potential issue with CPAP. Based on four-year mean daily precipitation, CMORPH and PERSIANN tend to underestimate the precipitation rates by 4.9 mm/day and 1.8 mm/day, while GSMaP_MVK overestimates the precipitation by 1.4 mm/day. GSMaP_MVK shows intermediate performance among the satellite-only products with a moderate CC (0.67), RMSE (1.07 mm/day), and RB (5.65%), while CMORPH significantly underestimates the precipitation by 19.60%. PERSIANN displays a low CC (0.59), highlighting its poor spatial correlation with the CPAP. With bias-correction, CMORPH_CRT and PERSIANN_CDR show much improvement over their satellite-only counterparts with much higher CCs (0.89) and relatively lower RMSEs (<0.7 mm/day). However, GSMaP_Gauge shows increased RB from 5.65%–15.77%, though the CC is increased from 0.67–0.95 and the RMSE decreased from 1.07 mm/day to 0.53 mm/day. This may be because the positive and negative RBs cancel each other out, which leads to relatively lower RB of GSMaP_MVK. The unexpectedly increased RB value from GSMaP_MVK to GSMaP_Gauge indicates that the bias-correction procedure in GSMaP_Gauge may be not beneficial in a mean sense. Normally, when the satellite-only estimate is adjusted to gauges, the bias should become smaller with most points lying along the 1-to-1 line in scatterplots, as in the case of CMORPH and PERSIANN. In particular, GSMaP_Gauge has the highest CC (0.95) and lowest RMSE (0.53 mm/day) among all the bias-corrected products. More detailed comparisons for the seven sub-regions are shown in

Table A1. RB, RMSE, and CC between CPAP and the selected satellite-based products for Annual and Seasonal Precipitation in seven sub-regions (XJ, TP, NW, NE, NC, SW and SC). The best satellite-only product in each season over different sub-regions is highlighted with regular font in gray background and the best gauge-corrected product is highlighted with bold font in gray background.

Index	Time	Type	XJ	TP	NW	NE	NC	SW	SC
RB (%)	4 Years	GSMaP_MVK	108.20	67.70	56.89	9.36	2.47	−30.87	−29.03
		GSMaP_Gauge	5.29	27.07	40.91	26.21	29.15	7.58	5.26
		CMORPH	3.15	−4.26	13.34	−13.69	−6.37	−35.64	−33.81
		CMORPH_CRT	−12.85	−2.62	6.47	2.77	8.83	−8.13	−3.07
		PERSIANN	96.01	68.72	25.03	0.48	−11.59	−47.17	−43.71
		PERSIANN_CDR	−13.16	19.90	3.39	16.73	8.93	−6.32	1.73
	Spring	GSMaP_MVK	101.82	169.02	106.82	20.28	21.22	−29.12	−33.00
		GSMaP_Gauge	−1.24	53.84	59.97	23.28	37.16	7.82	3.41
		CMORPH	−13.32	24.99	35.17	−27.82	0.38	−26.75	−36.12
		CMORPH_CRT	−10.75	0.29	9.15	−2.24	12.37	−4.52	−1.79
		PERSIANN	83.05	205.60	79.08	−6.55	26.49	−37.56	−46.57
		PERSIANN_CDR	−22.11	30.12	7.29	22.10	10.94	−6.69	1.46
	Summer	GSMaP_MVK	77.97	−1.26	25.44	1.02	0.23	−22.21	−11.79
		GSMaP_Gauge	51.72	30.71	50.47	34.75	37.02	18.94	24.32
		CMORPH	40.65	−34.16	16.06	−5.76	−0.27	−34.92	−19.81
		CMORPH_CRT	−14.13	−5.71	7.96	4.33	4.77	−7.88	1.08
		PERSIANN	27.40	−0.85	−23.24	−25.13	−31.06	−42.94	−28.08
		PERSIANN_CDR	−4.39	21.35	3.53	10.33	5.77	−2.27	6.61
	Autumn	GSMaP_MVK	74.95	58.70	27.50	36.27	−16.21	−49.94	−40.69
		GSMaP_Gauge	−28.51	0.52	18.95	20.97	14.57	−6.35	1.42
		CMORPH	−15.13	20.12	−10.29	−5.73	−27.83	−48.42	−41.95
		CMORPH_CRT	−14.31	0.49	5.52	22.71	6.08	−13.90	−4.87
		PERSIANN	12.16	−1.91	0.82	32.39	−19.07	−70.86	−59.89
		PERSIANN_CDR	−22.92	5.25	4.29	30.01	7.99	−15.47	−3.85
	Winter	GSMaP_MVK	315.48	1228.02	552.85	−20.10	20.21	−49.54	−72.91
		GSMaP_Gauge	−66.16	2.01	−66.26	−68.28	−51.06	−69.72	−65.58
		CMORPH	−50.69	332.97	22.97	−87.49	−32.69	−20.63	−74.32
		CMORPH_CRT	−9.47	27.54	−26.91	−66.78	55.74	1.88	−22.07
		PERSIANN	583.45	1582.36	753.58	308.40	99.73	−21.53	−76.66
		PERSIANN_CDR	−0.36	51.13	−27.01	36.68	44.09	−7.17	−9.60
RMSE (mm/day)	4 Years	GSMaP_MVK	0.95	1.17	0.59	0.39	0.30	1.58	1.28
		GSMaP_Gauge	0.26	0.49	0.37	0.42	0.58	0.79	0.55
		CMORPH	0.48	0.61	0.36	0.46	0.35	1.63	1.47
		CMORPH_CRT	0.48	0.57	0.22	0.34	0.33	1.17	0.60
		PERSIANN	0.57	0.93	0.38	0.43	0.38	1.99	1.87
		PERSIANN_CDR	0.34	0.68	0.14	0.32	0.28	1.22	0.57
	Spring	GSMaP_MVK	0.87	1.69	0.69	0.43	0.43	1.61	1.98
		GSMaP_Gauge	0.37	0.56	0.39	0.35	0.59	1.05	0.63
		CMORPH	0.65	0.61	0.49	0.62	0.42	1.51	2.08
		CMORPH_CRT	0.72	0.52	0.28	0.53	0.39	1.26	0.89
		PERSIANN	0.70	1.64	0.69	0.30	0.65	1.86	2.67
		PERSIANN_CDR	0.46	0.53	0.21	0.35	0.29	1.40	0.81
	Summer	GSMaP_MVK	0.73	1.12	0.77	0.78	0.72	3.03	1.39
		GSMaP_Gauge	0.59	1.24	1.03	1.28	1.71	1.92	1.95
		CMORPH	0.81	1.51	0.74	0.91	0.85	3.33	1.82
		CMORPH_CRT	0.62	1.43	0.48	0.65	0.75	2.31	1.31
		PERSIANN	0.51	1.20	0.84	1.27	1.55	4.02	2.45
		PERSIANN_CDR	0.49	1.68	0.34	0.62	0.64	2.54	1.27
	Autumn	GSMaP_MVK	1.27	1.32	0.49	0.42	0.35	1.74	1.19
		GSMaP_Gauge	0.41	0.51	0.20	0.25	0.27	0.71	0.57
		CMORPH	0.67	0.89	0.53	0.36	0.47	1.73	1.26
		CMORPH_CRT	0.63	0.67	0.33	0.40	0.34	1.15	0.59
		PERSIANN	0.43	0.58	0.46	0.45	0.44	2.25	1.68
		PERSIANN_CDR	0.45	0.66	0.20	0.31	0.27	1.11	0.59
	Winter	GSMaP_MVK	1.71	2.02	1.01	0.35	0.37	0.44	1.31
		GSMaP_Gauge	0.29	0.09	0.09	0.19	0.28	0.41	1.16
		CMORPH	0.37	0.66	0.22	0.27	0.34	0.76	1.32
		CMORPH_CRT	0.44	0.21	0.13	0.29	0.37	0.76	0.65
		PERSIANN	1.29	1.84	0.99	0.87	0.49	0.44	1.45
		PERSIANN_CDR	0.24	0.17	0.07	0.17	0.21	0.26	0.44
CC	4 Years	GSMaP_MVK	0.37	0.43	0.60	0.56	0.85	0.23	0.74
		GSMaP_Gauge	0.84	0.92	0.96	0.89	0.94	0.85	0.82
		CMORPH	0.39	0.67	0.69	0.38	0.79	0.35	0.69
		CMORPH_CRT	0.37	0.72	0.88	0.71	0.89	0.47	0.76
		PERSIANN	0.68	0.70	0.73	0.29	0.81	−0.01	0.52
		PERSIANN_CDR	0.70	0.75	0.95	0.88	0.92	0.37	0.78
	Spring	GSMaP_MVK	0.54	0.30	0.70	0.57	0.90	0.42	0.70
		GSMaP_Gauge	0.79	0.87	0.94	0.84	0.95	0.76	0.94
		CMORPH	0.20	0.56	0.43	0.13	0.76	0.51	0.74
		CMORPH_CRT	0.23	0.71	0.72	0.36	0.89	0.60	0.85
		PERSIANN	0.25	0.49	0.61	0.64	0.66	0.21	0.42
		PERSIANN_CDR	0.62	0.79	0.87	0.82	0.93	0.46	0.88
	Summer	GSMaP_MVK	0.80	0.82	0.81	0.65	0.78	0.30	0.75
		GSMaP_Gauge	0.80	0.92	0.94	0.88	0.91	0.85	0.82
		CMORPH	0.55	0.70	0.80	0.51	0.70	0.44	0.67
		CMORPH_CRT	0.55	0.69	0.90	0.81	0.82	0.56	0.67
		PERSIANN	0.73	0.77	0.73	0.47	0.76	0.10	0.56
		PERSIANN_CDR	0.73	0.74	0.95	0.87	0.88	0.36	0.73
	Autumn	GSMaP_MVK	0.23	0.21	0.48	0.62	0.53	0.33	0.90
		GSMaP_Gauge	0.80	0.73	0.96	0.88	0.81	0.84	0.94
		CMORPH	0.15	0.39	0.36	0.40	0.50	0.28	0.88
		CMORPH_CRT	0.26	0.54	0.77	0.64	0.59	0.36	0.90
		PERSIANN	0.63	0.55	0.44	0.34	0.16	0.13	0.86
		PERSIANN_CDR	0.62	0.57	0.91	0.87	0.68	0.43	0.90
	Winter	GSMaP_MVK	−0.13	−0.06	−0.31	−0.28	0.10	0.14	0.54
		GSMaP_Gauge	0.50	0.49	0.53	0.67	0.92	0.45	0.81
		CMORPH	−0.26	0.04	−0.06	−0.29	0.33	0.08	0.59
		CMORPH_CRT	−0.19	0.13	0.21	−0.23	0.66	0.13	0.69
		PERSIANN	0.53	0.04	0.38	0.06	0.30	−0.16	−0.20
		PERSIANN_CDR	0.56	0.40	0.65	0.76	0.94	0.34	0.80

. It is evident that both GSMaP_MVK and PERSIANN greatly overestimate the precipitation over the XJ (108.20% and 96.01%, respectively) and NW (56.89% and 25.03%, respectively) regions. However, CMORPH shows much smaller RB (3.15% and −4.26%). Again, this result may stem from a “canceling out” effect between overestimation and underestimation.

Figure 2. (a) A 0.5° × 0.5° density map of gauges over China with 2419 gauges used in CPAP; (b–h) Spatial distribution of four year daily mean precipitation (mm/day) derived from (b) GSMaP_MVK; (c) CMORPH; (d) PERSIANN; (e) CPAP; (f) GSMaP_Gauge; (g) CMORPH_CRT; and (h) PERSIANN_CDR over China.

Figure 2. (a) A 0.5° × 0.5° density map of gauges over China with 2419 gauges used in CPAP; (b–h) Spatial distribution of four year daily mean precipitation (mm/day) derived from (b) GSMaP_MVK; (c) CMORPH; (d) PERSIANN; (e) CPAP; (f) GSMaP_Gauge; (g) CMORPH_CRT; and (h) PERSIANN_CDR over China.

In general, strong evaporation over the arid and semi-arid regions leads to overestimation because hydrometeors that are detected by the space-borne sensors (which generally sample the ice-phase or cloud tops) may partially or totally evaporate before they reach the surface during the warm seasons [72,73]. Additionally, these satellite precipitation estimates tend to underestimate precipitation over mountainous regions, especially for CMORPH [74]. Similarly, SSMI sensors have difficulty in identifying precipitation over arid areas due to the land surface emissivity characteristics which impacts the upwelling microwave radiation [75]. Finally, the relatively sparse and unevenly distributed rain gauges may result in the OI technique averaging out nonzero precipitation estimates. Above all, the ratios between positive RB and negative RB for GSMaP and PERSIANN are relatively larger than CMORPH (Figure 10b,j,r).

The performance of GSMaP, CMORPH, and PERSIANN products are also evaluated on a sub-regional scale. Table A1 and Figure 3 show that PERSIANN and GSMaP_MVK exhibit substantial overestimations of precipitation over TP (67.70% and 68.72%, respectively). Such overestimations are likely the result of ice or snow cover, which tends to complicate PMW-based overland retrievals [76]. In addition, the sparse and uneven distribution of rain gauges and the errors resulting from the aforementioned OI technique may also contribute to the substantial overestimation. Finally, due to factors such as a near-surface freezing level, ground clutter contamination, and bright band contamination, the TRMM Precipitation Radar (PR) has been known to have difficulties in accurately estimating precipitation in high altitude regions [77]. These factors may be substantial considering that TRMM PR products contribute significantly to the GSMaP retrieval algorithm. In general, the CMORPH outperforms the other two satellite-only estimates, and displays a small underestimation (RB, −4.26%). This is primarily because it uses another different retrieval algorithm that results in a more conservative estimate of rainfall area in the presence of snow cover [42]. This pattern is consistent with the results of several previous studies [45,47,48].

Figure 3. Density-colored scatterplots of various products (a) GSMaP_MVK; (b) CMORPH; (c) PERSIANN; (d) GSMaP_Gauge; (e) CMORPH_CRT and (f) PERSIANN_CDR against CPAP rainfall rate shown in Figure 2. The color represents the occurrence frequency. The dark oblique solid line represents 1:1 line.

Figure 3. Density-colored scatterplots of various products (a) GSMaP_MVK; (b) CMORPH; (c) PERSIANN; (d) GSMaP_Gauge; (e) CMORPH_CRT and (f) PERSIANN_CDR against CPAP rainfall rate shown in Figure 2. The color represents the occurrence frequency. The dark oblique solid line represents 1:1 line.

In order to determine which product more precisely estimates precipitation over mainland China and the seven sub-regions, Taylor diagrams [71] are plotted to visualize the concise statistical summary of how well each single satellite-based product matches the reference data (CPAP) in terms of their correlation coefficient, the ratio of their standard deviation, and their RMSE values (Figure 4). The position of each symbol appearing on the Taylor diagrams quantifies how closely the satellite-based precipitation product matches the reference data. Satellite-based precipitation products that agree with reference data will lie nearest to the solid pentagon denoted with CPAP on the x-axis. The superiority of gauge-corrected products is clearly evident by their closer distance to CPAP, especially for XJ, TP and NW. When compared to the other two satellite-only products, GSMaP_MVK demonstrates similar performance over NE, NC, SW and SC, but has poor performance and relatively lower CCs over XJ, TP, and NW, likely due to the complicated terrain in arid and semi-arid environments. Compared to CMORPH_CRT and PERSIANN_CDR, GSMaP_Gauge shows the best overall performance over contiguous China and the seven sub-regions.

Figure 4. Taylor diagrams showing the correlation coefficients, the normalized standard deviation, and the RMSE of four year daily mean precipitation between six satellite-based precipitation products and the reference data over (a) mainland China and seven sub-regions: (b) XJ; (c) TP; (d) NW; (e) NE; (f) NC; (g) SW; and (h) SC. The radial coordinate is the magnitude of the standard deviation, which is normalized by the standard deviation from CPAP and denoted by black dotted lines. The concentric green semi-circles denote RMSE values. The angular coordinate shows the correlation coefficient (denoted by blue dotted lines). In these Taylor diagrams, the closer the points representing the satellite-based precipitation products are to the CPAP point (marked by the black pentagon), the better the product is with relatively high correlation, low standard deviation and RMSE values. The products from the same algorithm are marked by the same marker with same color and denoted by different characters with the same color. CPAP is marked by a black solid pentagon, while GSMaP_MVK and GSMaP_Gauge are marked by red diamonds and denoted by red A and B, respectively. CMORPH and CMORPH_CRT are marked by orange solid circles and denoted by orange C and D, respectively, whereas PERSIANN and PERSIANN_CDR are marked by purple squares and denoted by purple E and F, respectively.

Figure 4. Taylor diagrams showing the correlation coefficients, the normalized standard deviation, and the RMSE of four year daily mean precipitation between six satellite-based precipitation products and the reference data over (a) mainland China and seven sub-regions: (b) XJ; (c) TP; (d) NW; (e) NE; (f) NC; (g) SW; and (h) SC. The radial coordinate is the magnitude of the standard deviation, which is normalized by the standard deviation from CPAP and denoted by black dotted lines. The concentric green semi-circles denote RMSE values. The angular coordinate shows the correlation coefficient (denoted by blue dotted lines). In these Taylor diagrams, the closer the points representing the satellite-based precipitation products are to the CPAP point (marked by the black pentagon), the better the product is with relatively high correlation, low standard deviation and RMSE values. The products from the same algorithm are marked by the same marker with same color and denoted by different characters with the same color. CPAP is marked by a black solid pentagon, while GSMaP_MVK and GSMaP_Gauge are marked by red diamonds and denoted by red A and B, respectively. CMORPH and CMORPH_CRT are marked by orange solid circles and denoted by orange C and D, respectively, whereas PERSIANN and PERSIANN_CDR are marked by purple squares and denoted by purple E and F, respectively.

3.2. Seasonal Daily Mean Precipitation

Precipitation over China varies dramatically geographically and seasonally due to the influences of the Asian monsoon, the Tibetan plateau, and the baroclinic westerlies initiated from North Atlantic Ocean. Figure 5 shows the seasonal pattern of four-year daily mean precipitation, while the corresponding density-colored scatter plots are given in Figure 6. The corresponding statistics of seasonal precipitation for the seven sub-regions are listed in Table A1. As shown by CPAP, the average precipitation decreases from the coastal regions in the southeast towards the semi-arid and arid regions in the northwest. Relative to CPAP, all three bias-corrected products demonstrate much better performance than the satellite-only estimates in every season, with increased CCs, and reduced RBs and RMSEs. GSMaP_Gauge outperforms the other two bias-corrected estimates in each season according to CCs (0.94, 0.94, 0.91 and 0.90), but it also displays a higher RBs (16.44%, 29.83%, 1.47% and −61.83%) and RMSEs (0.64 mm/day, 1.41 mm/day, 0.50 mm/day and 0.49 mm/day). CMORPH_CRT and PERSIANN_CDR have comparable performances in terms of CCs, RBs and RMSEs. Among the satellite-only products, PERSIANN shows the poorest performance in every season with the lowest CC, highest RMSE and high magnitude of RB. This result may primarily be attributed to the substantial overestimation (the highest RB in winter, 1582.36%) in TP. This overestimation may be the result of the poor performance of the IR-based rainfall estimation algorithm over the complex terrain, the limited gauge observations in the region and the corresponding errors resulting from the OI algorithm [36,78,79,80]. GSMaP_MVK overestimates large amounts of precipitation in the mountainous areas over XJ, TP and NW. Expectedly, the bias-correction procedure significantly improves the quality of GSMaP_Gauge over mountainous areas. As previously stated, CMORPH and CMORPH_CRT still suffer from isolated areas of abnormal overestimation in all four seasons. It should also be noted that all products perform worse in winter.

The satellite-only precipitation products without bias-correction (i.e., GSMaP_MVK, CMORPH and PERSIANN) significantly overestimate or underestimate precipitation (by 78.47%, −31.01%, and 160.87%, respectively). The bias-corrected products display much improvement but still show large biases in winter. The causes of this bias may be explained by the following: (1) The accuracy of PMW-based algorithms is influenced by snow and ice on the land surface [76], and specifically the PMW-based retrievals (e.g., AMSR-E, AMSU-B and SSM/I) are degraded by the presence of snow cover [81,82]. Therefore, the PMW-based algorithms may have difficulties in accurately estimating winter precipitation; (2) The PERSIANN overestimation of precipitation in winter may be associated with imperfect screening of cold surfaces from the IR images [45]; (3) The observed low performance from PERSIANN, especially the large overestimation over TP and the underestimation over SW and SC, may be linked to the flaws in the IR-based algorithm, which relies on indirect statistical relationships between IR observations of the cloud-top temperature and precipitation rate. Furthermore, it should be noted that the PERSIANN_CDR exhibits the best performance in winter relative to CPAP and displays the highest CC (0.91), the smallest RB (−0.01%) and RMSE (0.25 mm/day). This is probably due to its use of infrared brightness temperature data, which comes from geostationary satellites and are less influenced by cold climate and snow/ice covered land surfaces in winter [64].

A large region of abnormal zero precipitation estimates is produced by CMORPH and CMORPH_CRT over NE and northern XJ, and GSMaP_MVK displays a relatively smaller area of zero precipitation estimates over northwestern NE. These underestimations are likely due to reasons as follows: (1) The PMW-based algorithms used in GSMaP and CMORPH are good at detecting intense, convective precipitation events, but tend to miss shallow precipitation events [83]; (2) Ice and snow-covered surfaces tend to influence PMW-based accurate retrievals over the land surface [76]; (3) The CMORPH algorithm uses a more conservative estimate of raining areas in the presence of snow cover [42]. CPAP also detects three zero precipitation speckles in XJ and TP. This can be, at least partially, attributed to the sparse and uneven rain gauges in these areas and the flaws in the OI technique as mentioned previously.

Figure 5. Seasonal four-year daily mean precipitation distribution over China for (a–d) CPAP; (e–h) GSMaP_MVK; (i–l) GSMaP_Gauge; (m–p) CMORPH; (q–t) CMORPH_CRT; (u–x) PERSIANN; and (y–B) PERSIANN_CDR.

Figure 5. Seasonal four-year daily mean precipitation distribution over China for (a–d) CPAP; (e–h) GSMaP_MVK; (i–l) GSMaP_Gauge; (m–p) CMORPH; (q–t) CMORPH_CRT; (u–x) PERSIANN; and (y–B) PERSIANN_CDR.

Figure 6. Density-colored scatterplots of (a–d) GSMaP_MVK; (e–h) GSMaP_Gauge; (i–l) CMORPH; (m–p) CMORPH_CRT; (q–t) PERSIANN; (u–x) PERSIANN_CDR against CPAP rainfall rate shown in Figure 5. The color represents the occurrence frequency.

Figure 6. Density-colored scatterplots of (a–d) GSMaP_MVK; (e–h) GSMaP_Gauge; (i–l) CMORPH; (m–p) CMORPH_CRT; (q–t) PERSIANN; (u–x) PERSIANN_CDR against CPAP rainfall rate shown in Figure 5. The color represents the occurrence frequency.

3.3. Time Series Monthly Precipitation

In order to investigate the temporal behavior of Bias, CC, and RMSE of the satellite-based products over mainland China and the seven sub-regions, comparison statistics are calculated for each month during the four-year period and are displayed in Figure 7. It has been demonstrated that a sparse gauge network degrades the quality of gauge-based analysis [62], and specifically, a substantial magnitude of both bias and random error may occur in the gauge-based analysis over grid boxes with no gauge reports available nearby [66]. Therefore, in this study statistics are computed using monthly-mean data and grid cells only where/when there is at least one reporting gauge, and both CPAP and the satellite-based precipitation estimates are above zero.

Figure 7. Monthly series of Bias in the left column panels, Correlation Coefficient (CC) in the middle column panels, root-mean-square error (RMSE) in the right column panels between various products and CPAP over (a–c) continuous China and seven sub-regions for (d–f) XJ; (g–i) TP; (j–l) NW; (m–o) NE; (p–r) NC; (s–u) SW; (v–x) SC on 0.25° lat/lon grid box. Only the cells where there is at least one reporting gauge in CPAP and both CPAP and the satellite-based precipitation estimates are above zero have been selected for computation.

Figure 7. Monthly series of Bias in the left column panels, Correlation Coefficient (CC) in the middle column panels, root-mean-square error (RMSE) in the right column panels between various products and CPAP over (a–c) continuous China and seven sub-regions for (d–f) XJ; (g–i) TP; (j–l) NW; (m–o) NE; (p–r) NC; (s–u) SW; (v–x) SC on 0.25° lat/lon grid box. Only the cells where there is at least one reporting gauge in CPAP and both CPAP and the satellite-based precipitation estimates are above zero have been selected for computation.

The RMSE and CC (Figure 7c) generally show a season-dependent variation for all satellite-based precipitation products. The RMSE is larger between the satellite-based estimates and CPAP during the monsoon season from April to October. The bias-corrected estimates show fewer season-dependent variations in RMSE (Figure 7b). This implies that the bias-correction processes in the three bias-correction products have the potential to reduce the season-dependent error. GSMaP_Gauge consistently shows the highest CC (>0.7), but also exhibits a relatively higher season-dependent overestimation (underestimation) bias in warm seasons (cold seasons). GSMaP_MVK shows underestimation over China for the whole study period. This may imply that the gauge-adjustment can improve the CC and remove the negative bias [44], but also may generate some season-dependent positive bias. Overall, the gauge adjusted algorithm of GSMaP_Gauge still appears to have some room for improvement. CMORPH also shows a stable underestimation which has been observed in other studies [29,84,85]. Compared to the GSMaP products, CMORPH_CRT and PERSIANN_CDR have more stable (i.e., less fluctuating) seasonal biases. This pattern is consistent with the results shown in Figure 5 and Figure 6.

In the sub-regions, GSMaP_Gauge displays the highest CC for all seasons while CMORPH_CRT displays the worst performance with the lowest and highly fluctuating CCs. Both CMORPH and CMORPH_CRT exhibit inferior performance over XJ and TP with larger RMSE and lower CC. Additionally, these trends show significant seasonal fluctuations with the higher value in warm season and lower value in cold season. In a similar study on CMORPH and the microwave-only product (MWCOMB), Shen [45] found that MWCOMB shows the same bias sign and is close in magnitude with other PMW/IR blended products (CMORPH and PERSIANN) over China. Thus, these biases over XJ and TP regions may result from the deficiencies of the PMW sensors in estimating precipitation. PERSIANN demonstrates the poorest performance with the largest fluctuating bias and RMSE, as well as the lowest fluctuating CC over China and the sub-regions. As expected, the bias-correction procedure in PERSIANN_CDR improves the performance of PERSIANN significantly.

PERSIANN shows abnormally high biases and RMSEs over XJ, TP, NW, and NE during December 2006 and June 2007. After independent verification (not detailed here), we found that PERSIANN tends to significantly overestimate precipitation over semi-arid and arid regions, especially for TP, from October 2006 to July 2007. This may be, at least partly, attributed to the limitation of the PERSIANN algorithm, which is mainly based on IR data. In addition, hydrometeors detected by the microwave and infrared sensors (as well as the TRMM PR) may partially or totally evaporate before reaching the surface in semi-arid areas [53,86]. Therefore, PERSIANN without gauge-correction tends to overestimate precipitation. This can also be explained by the lack of training of the artificial neural network parameters over China, since PERSIANN is only adequately trained over the United States [48].

3.4. Probability Distribution by Occurrence

The precipitation rate occurrence distribution of intensities can provide an insight into the characteristics of precipitation rate dependent error and the potential error effect on hydrological applications. Figure 8 shows the Probability distribution functions (PDFs) by precipitation occurrence with different precipitation intensities. The thresholds of precipitation intensity are shown in Table 2. All of the satellite-only precipitation datasets tend to significantly overdetect the occurrence of light precipitation (0 < R ≤ 1 mm/day) over China, especially for the dry climate of western China (i.e., XJ, TP, NW), while tending to underdetect the occurrence of heavy precipitation (more than 20 mm/day) over the wet environment of eastern China (i.e., NE, NC, SW, and SC). This may be related to the fact that the PMW-based algorithms used in the estimates are good at detecting heavy, convective precipitation events, but fail to accurately detect shallow, warm precipitation [87,88]. The substantial evaporation in the dry environments over XJ, TP and NW, as well as the large amount of heavy precipitation over SW and SC, likely are additional contributions to these disparities. In comparison with CMORPH and PERSIANN, GSMaP_MVK shows slightly better performance in detecting light precipitation less than 1 mm/day in all regions and heavy precipitation over 20 mm/day in NE, NC, SW, and SC.

Figure 8. Probability Density Function by occurrence (PDFc) of daily precipitation (mm/day) for cases with different intensities over contiguous China and seven sub-regions for (a) China; (b) XJ; (c) TP; (d) NW; (e) NE; (f) NC; (g) SW; and (h) SC. Only grid cells where there is at least one reporting gauge and both the reference and the satellite-based precipitation estimates are nonzero can be selected for computing the occurrence.

Figure 8. Probability Density Function by occurrence (PDFc) of daily precipitation (mm/day) for cases with different intensities over contiguous China and seven sub-regions for (a) China; (b) XJ; (c) TP; (d) NW; (e) NE; (f) NC; (g) SW; and (h) SC. Only grid cells where there is at least one reporting gauge and both the reference and the satellite-based precipitation estimates are nonzero can be selected for computing the occurrence.

Table 2. Thresholds for precipitation intensity.

Index	R (mm/Day)
1	0 < R ≤ 1
2	1 < R ≤ 1.5
3	1.5 < R ≤ 3
4	3 < R ≤ 5
5	5 < R ≤ 10
6	10 < R ≤ 20
7	R > 20

Table 2. Thresholds for precipitation intensity.

Index	R (mm/Day)
1	0 < R ≤ 1
2	1 < R ≤ 1.5
3	1.5 < R ≤ 3
4	3 < R ≤ 5
5	5 < R ≤ 10
6	10 < R ≤ 20
7	R > 20

After gauge-correction, GSMaP_Gauge and PERSIANN_CDR tend to underestimate the light rain events, while CMORPH_CRT shows substantial overdetection for events of rain rate less than 1 mm/day. When rain rate is over 5 mm/day, GSMaP_Gauge detects many more events than CPAP, especially for NE, NC and SC. The bias-correction procedure in GSMaP_Gauge is helpful for reducing errors from both overdetection of light precipitation events (<1 mm/day) and underdetection of heavy precipitation (>20 mm/day) events. However, it also seems to yield an overcorrection for detecting heavy precipitation events. Both CMORPH and CMORPH_CRT slightly underdetect events with a rain rate of more than 1 mm/day.

3.5. Contingency Statistics

Probability of detection (POD), False Alarm Ratio (FAR) and Critical successful index (CSI) clarify the nature of the occurrence errors [35]. The contingency statistics (POD, FAR, and CSI) based on daily precipitation rate, with an interval of 5 mm/day from 1 mm/day to 250 mm/day for Mainland China and an interval of 2 mm/day from 1 mm/day to 60 mm/day for the seven sub-regions (Figure 9). All Satellite-based precipitation products display low PODs for rain rate less than 60 mm/day. By comparison, GSMaP_Gauge has a higher POD (up to 50%) and CSI (up to 35%), and lower FAR (down to 40%) for light precipitation. GSMaP_Gauge also has relatively higher POD and CSI, and a lower FAR than the other estimates for the precipitation rates less than 30 mm/day. Interestingly, CMORPH performs better than the other products, including its bias-corrected counterpart for precipitation accumulations ranging from 150 mm/day to 250 mm/day. Regionally, GSMaP_Gauge also shows the best scores among all products with a higher POD and CSI and lower FAR. When rain rates are greater than 30 mm/day, all products give low POD and CSI, and a high FAR in XJ, TP, and NW. Relatively high performances are found over the other regions (i.e., NE, NC, SW and SC). Interestingly, CMORPH exhibits a higher POD than its counterpart with bias-correction during 20 mm/day to 50 mm/day over XJ. CMORPH demonstrates relatively lower POD and CSI, and higher FAR than other satellite-only precipitation products over the semi-arid and arid area (i.e., XJ, TP, NW and NE). PERSIANN shows the lowest POD and CSI, and the highest FAR over SW and SC, which has a relatively humid climate. When the rainfall rate is over 30 mm/day, the percentage of POD and CSI (FAR) becomes higher (lower) from semi-arid and arid climate areas (e.g., XJ, TP and NW) to semi-humid and humid climates (e.g., NC, SW and SC). This may indicate a rainfall intensity dependency of contingency statistics (i.e., POD, FAR and CSI).

Figure 9. Contingency metrics of POD (left column panels), FAR (middle column panels) and CSI (right column panels) for different products over China and seven sub-regions for (a–c) China; (d–f) XJ; (g–i) TP; (j–l) NW; (m–o) NE; (p–r) NC; (s–u) SW; (v–x) SC, based on 0.25° lat/lon and daily mean precipitation rates (mm/day) day by day. Vertical axis scales were set according to the maximum of y-axis in different subpanels. The calculation of contingency metrics are derived using a 5 mm/day rain rate interval from 1 mm/day to 250 mm/day for mainland China and 2 mm/day interval from 1 mm/day to 60 mm/day for seven sub-regions. Only grid cells where there is at least one reporting gauge and both the reference and the satellite-based precipitation estimates are nonzero can be selected for computing these metrics.

Figure 9. Contingency metrics of POD (left column panels), FAR (middle column panels) and CSI (right column panels) for different products over China and seven sub-regions for (a–c) China; (d–f) XJ; (g–i) TP; (j–l) NW; (m–o) NE; (p–r) NC; (s–u) SW; (v–x) SC, based on 0.25° lat/lon and daily mean precipitation rates (mm/day) day by day. Vertical axis scales were set according to the maximum of y-axis in different subpanels. The calculation of contingency metrics are derived using a 5 mm/day rain rate interval from 1 mm/day to 250 mm/day for mainland China and 2 mm/day interval from 1 mm/day to 60 mm/day for seven sub-regions. Only grid cells where there is at least one reporting gauge and both the reference and the satellite-based precipitation estimates are nonzero can be selected for computing these metrics.

3.6. Spatial Analysis

The spatial distribution of the errors associated with satellite-based precipitation products can provide useful information, particularly for analyzing error propagation in regional hydrological and climatological applications. The spatial distribution maps of Bias, RB, RMSE, and CC of the satellite-based products versus CPAP are given in Figure 10. The Bias, RB, RMSE, and CC are computed grid by grid with four-year daily precipitation rates. Figure 11 shows the corresponding Probability Distribution Function (PDFc) and Cumulative Distribution Function (CDFc) of Bias, RB, RMSE, and CC.

Figure 10. Spatial distribution of Bias, RB, CC, and RMSE between different satellite-based products and CPAP over China on a 0.25° lat/lon grid for (a–d) GSMaP_MVK; (e–h) GSMaP_Gauge; (i–l) CMORPH; (m–p) CMORPH_CRT; (q–t) PERSIANN; and (u–x) PERSIANN_CDR.

Figure 10. Spatial distribution of Bias, RB, CC, and RMSE between different satellite-based products and CPAP over China on a 0.25° lat/lon grid for (a–d) GSMaP_MVK; (e–h) GSMaP_Gauge; (i–l) CMORPH; (m–p) CMORPH_CRT; (q–t) PERSIANN; and (u–x) PERSIANN_CDR.

Figure 11. Probability (Cumulative) Distribution Functions PDFc (CDFc) for spatial (a,e) Bias; (b,f) RB; (c,g) RMSE; and (d,h) CC values shown in Figure 10.

Figure 11. Probability (Cumulative) Distribution Functions PDFc (CDFc) for spatial (a,e) Bias; (b,f) RB; (c,g) RMSE; and (d,h) CC values shown in Figure 10.

All products tend to underestimate the four-year daily mean precipitation over the southwestern and southern China (i.e., SW and SC), while overestimating the precipitation over the rest of China. This tendency is clearly evident in Bias and RB metrics and is more pronounced for the non-bias-corrected products, especially GSMaP_MVK and CMORPH. The RMSE maps of satellite-based precipitation products present the systematic error amplitude distribution. The amplitude of RMSE declines from southeastern China to northwestern China, which suggests that satellite-based precipitation products tend to yield larger magnitude of errors over humid southeastern China than over the arid northwestern China. This is primarily a result of the greater volume of precipitation over the humid southeast relative to the arid/semi-arid northwest. Generally, the magnitude of errors for all products is higher (lower) over southern and southeastern (northern and northwestern) China with a wet (dry) climate; this result is consistent with previous findings from Shen [45]. The correlation between satellite-based products and CPAP reaches 0.5, and is higher for four-year daily precipitation. Overall, GSMaP_Gauge exhibits the best correlation over most of China (Figure 10, Table A1), suggesting that the bias-correction of GSMaP can effectively reduce errors resulting from complex terrain.

The occurrence Probability Distribution Functions (PDFc) and the occurrence Cumulative Distribution Functions (CDFc) can provide detailed information on the frequency of Bias, RB, RMSE, and CC with different thresholds. Figure 11 shows the statistics of Figure 10 in terms of PDFc and CDFc. CMORPH_CRT and PERSIANN_CDR show symmetrical quasi-Gaussian distributions of Bias and RB, while GSMaP_Gauge displays a right-skewed distribution (Figure 11b). Conversely, the satellite-only precipitation products show left-skewed distributions with much lower peak values. Furthermore, GSMaP_Gauge shows a higher percentage of positive bias for accumulations less than 1 mm/day, which indicates that the bias-correction procedure in GSMaP needs to be improved. All bias-corrected satellite precipitation products show slightly better performance than their satellite-only counterparts in terms of RMSE and CC. The GSMaP_Gauge has a higher percentage of high CC values than other products, which means more areas with higher CC than other products. Generally, the bias-correction procedures show good performance in reducing the magnitudes of Bias and RB, but do not show much improvement in reducing RMSE or improving CC for every grid box [83].

4. Summary and Conclusions

As alternative sources of precipitation information, future developments of satellite precipitation algorithms and utilization of satellite datasets in operational applications rely on a more in-depth understanding of errors and biases across different spatial and temporal scales. This paper provides a comprehensive and quantitative investigation of the error characteristics associated with three groups of high-resolution satellite precipitation products. The analysis is based on daily, seasonal and inter-annual rainfall comparisons with the gauge-based CPAP product for a four-year period over mainland China. In addition to the conventional statistical indices, frequency and spatial analyses have also been conducted in this study. The main findings of this paper can be summarized as follows:

(1)

Generally, the bias-correction procedures successfully reduce errors for all the three groups of satellite-based precipitation products. Specifically, regional overestimation or underestimation of precipitation, season-dependent errors and snow surface induced errors are reduced. However, the bias-correction procedure in GSMaP_Gauge may be not helpful in reducing errors in a mean sense.

(2)

For the four-year mean daily precipitation, all the six satellite products are capable of capturing the overall spatial pattern of precipitation. GSMaP_Gauge produces better fractional coverage with the highest CC (0.95) and lowest RMSE (0.53 mm/day), but also has a relatively high RB (15.77%) over China, while CMORPH_CRT exhibits amounts closer to CPAP. Among the satellite-based precipitation products without bias-adjustment, CMORPH demonstrates the best performance with the highest CC (0.82), and lowest RMSE (0.93 mm/day), but also a relatively higher RB (−19.60%). CMORPH_CRT fails to remove the highlighted overestimation speckles which appear in CMORPH without bias-correction. GSMaP_MVK displays relatively poor performance in XJ, TP and NW with relatively low CCs and shows large overestimation along the northern hillside of Tibetan Plateau. Therefore, it should be used cautiously in mountainous regions. CMORPH and GSMaP perform poorly over ice/snow covered surfaces, while PERSIANN displays a distinct overestimation over TP.

(3)

All the bias-corrected products provide improvements over their corresponding satellite-only counterparts. Among the bias-corrected products, GSMaP_Gauge shows the highest CC (0.90–0.94) but also a relatively high RB (−61.38%–16.44%) and RMSE (0.49–1.41 mm/day) for the seasonal four-year daily mean precipitation (Figure 4 and Figure 5, Table A1). Similar to the other precipitation products, it exhibits the worst performance in winter. PERSIANN_CDR shows the closest performance to CPAP in winter among all satellite-based products.

(4)

Season-dependent variations of error are present, and in particular, larger error values are observed during the monsoon season (April to October) for satellite-only precipitation products. These season-dependent variations are reduced by bias-adjustment procedures (Figure 7). GSMaP_MVK and CMORPH show underestimations for the entire study period. After bias-correction, GSMaP_Gauge consistently displays the highest CC (>0.7), but displays a relatively higher overestimation in warm seasons and a slight underestimation in cold seasons. This may be evidence for over-correction in the gauge-adjustment algorithm which could be improved in the future.

(5)

All the satellite-only precipitation products greatly overdetect the occurrence of rain rates less than 1 mm/day over dry regions (e.g., XJ, TP and NW) (Figure 8) and underdetect heavy rain events (>20 mm/day) over eastern China in wet climates. The bias-corrected products show decreased overdetection of light precipitation events. However, the bias-correction in PERSIANN and GSMaP tend to overcorrect the overdetection of light rain events and underdetection of heavy rain events. Moreover, gauge adjustment in CMORPH_CRT has little effect in reducing the light rain overdetection, but manifests detection patterns close to those of CPAP when the rain rate is over 1 mm/day.

(6)

Satellite-based precipitation products display low PODs for rain rates less than 60 mm/day. GSMaP_Gauge outperforms other two bias-corrected precipitation products with a maximum POD (~50%), maximum CSI (about 35%), and minimum FAR (~40%) for the rainfall rates less than 100 mm/day.

(7)

The underestimation by all satellite-based products is primarily distributed over southern and southeastern China while the overestimation is dominant over northern and western China. These satellite-based products exhibit better performance over the south and southeastern parts of China than over the north and northwestern parts of China in terms of Bias and RB (Figure 10 and Figure 11).

The error characteristics of the six high-resolution satellite-based precipitation estimates (GSMaP_MVK and GSMaP_Gauge, CMORPH and CMORPH_CRT, PERSIANN and PERSIANN_CDR) are identified and quantified over contiguous China in this paper. Knowledge of these characteristics will have significant implications for hydrometeorological applications and algorithm development. With the launch of the Global Precipitation Measurement Mission in late February 2014, the new product IMERG that combines the algorithms of CMORPH, PERSIANN, and TMPA has been released after much anticipation. The new version of GSMaP products has also been released recently. The systematic evaluation of CMORPH, PERSIANN, and GSMaP products will provide early and helpful references for algorithms developers and the end users of these future products.