Evaluation of Three High-Resolution Remote Sensing Precipitation Products on the Tibetan Plateau

Abstract

Remote sensing precipitation products provide rich data for ungauged basins. Evaluating the accuracy and detection capability of remote sensing precipitation products is crucial before application. In this study, an index system in terms of quantitative differences, capturing capacity and precipitation distribution was constructed to evaluate three precipitation products, TRMM 3B42 V7, GPM IMERGE Final and CMORPH V1.0, at various temporal and spatial scales on the Tibetan Plateau from 2001 to 2016. The results show that the correlations among the three products were larger at the monthly scale than at the annual scale. The lowest correlations between the products and observation data were found in December. GPM performed the best at the monthly and annual scales. Particularly, the GPM product presented the best capability of detection of both precipitation and non-precipitation events among the three products. All three precipitation products overestimated 0.1~1 mm/day precipitation, which occurred most frequently. An underestimation of precipitation at 10~20 mm/day was observed, and this intensity accounted for the majority of the precipitation. All three precipitation products showed an underestimation in terms of the annual maximum daily precipitation. The accuracy of the same product varied in different regions of the Tibetan Plateau, such as the south, the southeast, eastern–central region and the northeast, and there was a certain clustering of the accuracies of neighboring stations. GPM was superior to TRMM and CMORPH in the southern Tibetan Plateau, making it recommended for applications.

1. Introduction

Precipitation plays a crucial role in the water cycle [1,2]. As the output of weather systems and the main source of runoff, precipitation not only acts as an important bridge and link between the atmosphere and land, but also controls the availability of water [3,4,5]. The quantitative observation of precipitation is the basis for hydrological research. There are three main ways to obtain precipitation information: rain gauges, weather radar and meteorological satellites. Rain gauges can obtain precipitation directly. However, due to the limitation of observation-station planning and construction, it is still difficult to obtain precipitation information in key regions of climate change, such as polar regions and high mountains [6]. Remote sensing technology represented by weather radar and meteorological satellites has progressed in recent decades, and it has been gradually applied to various fields, such as meteorology and water resources. Weather radar can capture the spatial and temporal precipitation distribution in a short period by detecting instantaneous precipitation. However, it is difficult to carry out a large-scale regionalization of radar construction due to its limited monitoring range and high cost [7]. Satellites can continuously obtain precipitation information with a certain spatial and temporal resolution under various geographical conditions, especially at high altitudes. A large number of satellite-based remote sensing precipitation products have been developed in recent years. At present, satellite precipitation products include Tropical Rainfall Measuring Mission (TRMM) [8], Integrated Multi-satellite Retrievals for the Global Precipitation Measurement (GPM IMERGE) [9], Climate Prediction Center morphing technique (CMORPH) [10], Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks-Climate Data Record (PESIANN-CDR) [11], Global Satellite Mapping of Precipitation (GSMaP) [12], etc. However, various satellite products still make estimates of precipitation based on inversion algorithms. The applicability of products is limited by the selection of the study area [13,14,15], and the accuracy is influenced by the temporal scale [16,17,18], precipitation intensity [19,20,21], topographic conditions [4,22,23], etc. Therefore, there is a significant need to conduct regionalized precipitation-product evaluation studies.

The Tibetan Plateau (TP) is the most unique geological–geographical–ecological unit on Earth and is a natural laboratory for research on the evolution of the Earth and life, the interaction of circles and human–earth relations [24]. However, due to the wide area, high altitude and sparse rain gauges of the TP, knowledge about precipitation characteristics is still insufficient. In order to apply remote sensing products to precipitation research on the TP, many scholars have carried out studies to evaluate the quality of satellite precipitation products. In the Yellow River source area, the deviation of TRMM was positively correlated with rainfall intensity [25]. In the Yarlung Tsangpo River basin, the accuracy of TRMM at the monthly scale was high (correlation coefficient of 0.902), but the product overestimated weak precipitation and underestimated strong precipitation [26,27]. In addition, TRMM was also considered to have better accuracy in summer than in winter [28]. As an improvement to the previous generation of TRMM, GPM had less deviation and a stronger correlation with measured precipitation [29]. The detection capability of weak precipitation events of GPM was significantly better than that of TRMM, but both had shortcomings in the detection performance of strong precipitation events [30]. CMORPH could describe the spatial pattern and temporal variation of precipitation in China [31]. The product of CMORPH performed better in summer than in winter, but it overestimated the frequency of rainfall events with low intensity [32].

Evaluation studies for satellite products on the TP have mainly focused on sub-basins within the TP in the past, such as the Yellow River source and Yarlung Zangpo River, and less work has been performed to evaluate the entire plateau. The evaluation methods used were also limited. For example, the commonly used evaluation indexes were mainly correlation coefficient, relative deviation, root-mean-square error, probability of detection, false alarm ratio, critical success index, etc. [33,34,35,36,37]. Previous studies have mainly focused on the evaluation of the detection ability of precipitation events with little consideration of non-precipitation events. The contribution of different intensities of precipitation to total precipitation has also been rarely evaluated. Therefore, this study constructed a precipitation-product evaluation system, including the three aspects of quantitative differences, capturing capacity and precipitation distribution. In this evaluation system, we took into account the detection accuracy of no-rain events and the contribution of different intensities of precipitation to the total precipitation, which can comprehensively reflect the regional applicability of different precipitation products.

The constructed index system was used to compare the accuracy of three precipitation products (TRMM, GPM and CMORPH) considering multiple time scales. We also analyzed the ability to capture precipitation events and non-precipitation events by using the refined detection indexes. The evaluation in terms of frequency of occurrence, precipitation contribution and extreme precipitation was carried out, and the inversion accuracy of each station was also spatially explored. It is hoped that this study can provide a reference for the application of precipitation products on the TP.

2. Materials and Methods

2.1. Study Area

Most of the TP is located in southwest China, with a total area of about 2.5 million km² and an average altitude of more than 4000 m. It is the largest plateau in China and is located at the highest altitude in the world. The TP is known as the “Water Tower of Asia”, the “Roof of the World” and the “third Pole”. Many major rivers in Asia, such as Yangtze, Yellow, Lancang, Nu-Salween and Yarlung Zangpo, originate here. As shown in Figure 1, the boundary of the TP in China extends from the Pamir Plateau in the west to the Hengduan Mountains in the east and from the southern edge of the Himalayas in the south to the northern side of the Kunlun–Qilian Mountains in the north [38,39,40]. The precipitation on the TP decreases from southeast to northwest, and it is mainly concentrated in summer, which accounts for about 60% of annual precipitation. The annual precipitation of the TP shows an increasing trend of 3.8 mm/10a. The increasing rate of the mean annual temperature is 0.30 °C/10a [41,42].

2.2. Data

Although precipitation information can be obtained through satellites and atmospheric model simulations, the most direct and accurate way to measure precipitation is still the use of ground-based gauge measurements [43,44,45]. In this study, the daily precipitation data of 87 rain gauges (Figure 1) from 2001 to 2016 were selected for product validation and were provided by China Meteorological Administration. TRMM 3B42 V7 and GPM IMERG Final products were obtained from the National Aeronautics and Space Administration with a spatial resolution of 0.25° × 0.25° and 0.1° × 0.1°, respectively. CMORPH V1.0 was acquired from the Climate Prediction Center with a spatial resolution of 0.25° ×0.25°. The temporal resolution of all three products was daily resolution.

Figure 2 shows the annual and monthly precipitation variations in precipitation products and gauge data from the past 16 years. In general, the annual precipitation for each product showed fluctuating variations consistent with observation precipitation. TRMM and GPM had different levels of overestimations at annual scale throughout the study period, while that of the GPM was relatively small. CMORPH underestimated annual precipitation from 2001 to 2006. The overestimation of monthly precipitation for the three products tended to occur in the summer when the intensity of precipitation was high on the TP. The accumulation of overestimated precipitation during the flood season largely led to the deviation of annual precipitation between precipitation products and rain gauges.

3. Methods

3.1. Data Processing

A common pre-processing approach to evaluate remote sensing precipitation products is to use the precipitation at the grid of the precipitation product where the station is located to represent the inversion precipitation for that station [46,47]. However, the inversion precipitation obtained by this method has large uncertainties because of the strong spatial heterogeneity of precipitation. In this study, we used bilinear interpolation to interpolate the precipitation products with different resolutions to specific spatial locations where the rain gauges were located. This interpolation method takes into account the influence of geographical location on precipitation extraction and is common in meteorological studies [48,49]. The calculation principle can be seen in Figure 3, in which G(x,y) denotes the location of the gauge to be interpolated, and R₁(x₁,y₁), R₂(x₂,y₁), R₃(x₂,y₂) and R₄(x₁,y₂) denote the four vertices within the raster of remote sensing products where gauge G was located. The relevant calculation equations are as follows [50]:

$$P_G=(1-w_x)(1-w_y)P_1+w_x(1-w_y)P_2+w_x{w}_y{P}_3+(1-w_x)w_y{P}_4$$

(1)

$$w_x=\frac{x-x_1}{x_2-x_1}$$

(2)

$$w_y=\frac{y-y_1}{y_2-y_1}$$

(3)

where w_x and w_y denote the weight used for interpolation ($0\le w_x\le 1,0\le w_y\le 1$); P_G denotes the remote sensing precipitation corresponding to the gauge; P₁, P₂, P₃ and P₄ denote the remote sensing precipitation corresponding to points R₁, R₂, R₃ and R₄, respectively.

3.2. Evaluation Index System

To comprehensively compare the inversion capability of each product at different time scales, an evaluation system in terms of quantitative differences, capturing capability and precipitation distribution was constructed. Four common statistical indexes, correlation coefficient (CC), relative bias (BIAS), root-mean-square error (RMSE) and standard deviation (SD), were included. We also used seven capturing-capability indexes: probability of detection (POD), false alarm ratio (FAR), probability of false detection (POFD), missing alarm ratio (MAR) and Heidke skill score (HSS). In addition, two distribution indexes, probability distribution function by occurrence (PDFc) and probability distribution function by volume (PDFv), were involved. The equation of each index is shown in

Table 1. Evaluation indexes for remote sensing precipitation products.

Evaluation Index	Equation	Number
Correlation coefficient (CC)	$\mathrm{CC}=\frac{{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{n}}(P_{si}-\overline{P_{si}})(P_{oi}—\overline{P_{oi}})}}{\sqrt{{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{n}}{{\displaystyle (P_{si}-\overline{P_{si}})}}^2}}\sqrt{{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{n}}{{\displaystyle (P_{oi}—\overline{P_{oi}})}}^2}}}$	(1)
Relative bias (BIAS)	$\mathrm{BIAS}=\frac{{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{n}}(P_{si}-P_{oi})}}{{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{n}}{P}_{oi}}}$	(2)
Root-mean-square error (RMSE)	$\mathrm{RMSE}=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n{{\displaystyle (P_{si}-P_{oi})}}^2}}$	(3)
Standard deviation (SD)	$\mathrm{SD}=\frac{{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{n}}(P_i-\overline{P_i})}}{n}$	(4)
Probability of detection (POD)	$\mathrm{PODrain}=\frac{a}{a+c}$	(5)
	$\mathrm{PODnorain}=\frac{d}{b+d}$	(6)
False alarm ratio (FAR)	$\mathrm{FARrain}=\frac{b}{a+b}$	(7)
	$\mathrm{FARnorain}=\frac{c}{c+d}$	(8)
Probability of false detection (POFD)	$\mathrm{POFD}=\frac{b}{b+d}$	(9)
Missing alarm ratio (MAR)	$\mathrm{MAR}=\frac{c}{a+c}$	(10)
Heidke skill score (HSS)	$\mathrm{HSS}=\frac{2(ad-bc)}{(a+b)(b+d)+(a+c)(c+d)}$	(11)
Probability distribution function by occurrence (PDFc)	$\frac{n_p}{{\displaystyle \sum _{j=1}^k{n}_p}}$	(12)
Probability distribution function by volume (PDFv)	$\frac{v_p}{{\displaystyle \sum _{j=1}^k{v}_p}}$	(13)

.

CC indicates the degree of linear correlation between the precipitation products and gauge data. Its value varies between 1 and −1. The positive and negative values indicate positive and negative correlations, respectively. When the absolute value of CC is 1, the correlation is optimal. BIAS is used to verify the degree of systematic deviation between the precipitation products and gauge data, and the perfect value is 0. RMSE describes the overall deviation between the precipitation products and gauge data, and the optimal value is 0. SD reflects the degree of dispersion of the data, and the closer its value is to 0, the closer the data are to the mean of the sample.

Referring to the relevant literature [15,50], this study classified the capturing events of remote sensing products into precipitation events and non-precipitation events, and the critical threshold for both types of events was set to 1 mm/day. PODrain describes the ratio of the detected precipitation events to the number of precipitation events observed by the gauges, while PODnorain indicates the ratio of the detected non-precipitation events to the number of non-precipitation events observed by the gauges. The optimal values of PODrain and PODnorain are both 1. FARrain indicates the ratio of the incorrectly detected precipitation events to the total number of the detected precipitation events, while FARnorain indicates the ratio of the incorrectly detected non-precipitation events to the total number of the detected non-precipitation events. FARrain and FARnorain values close to 0 indicate the good detection capability of the satellite product. POFD and MAR further reflect the proportion of misreporting by the precipitation products for actual non-precipitation events and precipitation events, respectively, and their optimal value is 0. HSS can reflect the comprehensive estimation level of a precipitation product, and the range of its value is [−1, 1], where its largest value indicates the best detection capability. An HSS value of 1 represents that the satellite perfectly detects gauge precipitation. A positive value of HSS indicates that the detection capability is good, while a negative value means that the detection capability is not good enough.

In terms of the indexes of precipitation distribution, PDFc and PDFv reflect the differences in the frequency of precipitation occurrence and precipitation contribution between the products and gauge data in different intensity ranges, respectively. The former describes the distribution of the frequency of daily precipitation occurrence within different precipitation intensities, while the latter describes the distribution of the relative contribution of daily precipitation to the total precipitation within each precipitation intensity range [51,52,53].

Figure 4 depicts the framework of this study. After the work of remote sensing precipitation data extraction was completed, we analyzed the accuracy of precipitation products at different time scales and spatially explored their accuracy differences at the gauge scale under the constructed evaluation system.

4. Results

4.1. Evaluation of Precipitation Products at Annual and Monthly Scales

Figure 5 shows the correlation (with CC as the evaluation index) between the three precipitation products and gauge data at the annual and monthly scales. The fitted lines of all three products at the annual and monthly scales were biased to the right of the 1:1 straight line, indicating that in general, the remote sensing products underestimated the precipitation observed at the gauges. The correlations at the monthly scale were better than those at the annual scale for all three precipitation products. The correlations of TRMM and GPM at both time scales (CC > 0.8) were higher than that of CMORPH, but GPM had a higher correlation than TRMM at the annual (CC = 0.87) and monthly scales (CC = 0.91).

Based on the monthly precipitation series composed of all ground stations on the TP, the correlation differences among different products each month were further analyzed (as shown in Figure 6). Overall, GPM showed the best inversion performance. Except for CMORPH, the correlation between precipitation products and gauge data for most months was above 0.6, except for that of TRMM in December. There were five months (June, July, August, September and October) with CC values above 0.6 for CMORPH. The maximum values of monthly precipitation correlations for TRMM, GPM and CMORPH occurred in September (CC = 0.85), April (CC = 0.91) and August/September (CC = 0.78), respectively. The minimum values of CC all occurred in December, indicating that the improvement of the inversion capability of each product needs to be focused on December. The improvement of GPM over TRMM was particularly evident in the monthly precipitation assessment. Except for July and August when the CC values decreased, all other months showed different degrees of improvement, with the most significant improvements in December.

Figure 7 shows the precipitation Taylor plots at the annual and monthly scales and the bar plot that could simultaneously represent the differences in the relative deviation of the three products. The smaller the distance between the precipitation product and the gauge data in the Taylor plot was, the better the performance of the precipitation product was [15]. The results show that the best-performing product at both time scales was GPM, and the inversion differences at the monthly scale among the three precipitation products were smaller than those at the annual scale. The relative deviation showed that CMORPH had the smallest error with gauge precipitation, followed by GPM and TRMM.

4.2. Daily Precipitation Capturing Capacity and Distribution

Based on the daily precipitation data from all rain gauges during the study period on the TP, the capturing capabilities of the three precipitation products were evaluated according to the described indexes in this study. The box plots of each detection capability index are shown in Figure 8. The detection capabilities of both precipitation and non-precipitation events of GPM were better than those of the other products, as shown by the median, upper and lower quartiles of PODrain and PODnorain being greater than those of the other products. Compared with PODrain and PODnorain, FARrain and FARnorain of GPM showed the opposite variation, indicating that GPM could reduce the maximum degree of error for detected events. POFD and MAR represented the misreporting ratios for actual non-precipitation and actual precipitation events, and the box plots of POFD and MAR indicated that GPM had fewer detection errors than the other two products. The high score and the low dispersion of data distribution range for HSS indicated the performance of GPM was the best one, while TRMM and CMORPH had approximately equal detection capabilities.

To evaluate the inversion capability of the precipitation products for different levels of precipitation, this study divided the precipitation intensity into eight interval ranges [37]. The ranges included 0.1~1 mm/day, 1~2 mm/day, 2~5 mm/day, 5~10 mm/day, 10~20 mm/day, 20~50 mm/day, 50~100 mm/day and >100 mm/day. Figure 9 shows the probability distribution of the precipitation products and gauge data under the evaluation modes of PDFc and PDFv using gauge data as the reference standard. The precipitation on the TP was more frequent in the intensity intervals of 0.1~1 mm/day and 2~5 mm/day than in the other intervals, and the probability of occurrence in the range of 0.1~1 mm/day was the largest. All three products overestimated the number of precipitation occurrences at 0.1~1 mm/day and underestimated the number of occurrences at 2~20 mm/day. This estimation error was the largest for CMORPH among all products, while those of TRMM and GPM were not significantly different. Although the number of low-intensity precipitation occurrences in the 0.1~1 mm/day range was high, the overall quantity was small. On the other hand, the frequency of high-intensity precipitation above 50 mm/day on the TP was small, causing the PDFv curve to exhibit a low end and a high middle and to peak in the 10~20 mm/day precipitation range. It indicates that precipitation at 10~20 mm/day produced the maximum contribution.

The study of the inversion capability of extreme precipitation was useful for analyzing the application potential of the different precipitation products in disaster prevention and mitigation. Using the annual maximum daily precipitation as a characterization indicator of extreme precipitation, we also analyzed the inversion status of the three products on the actual precipitation values when the annual maximum daily precipitation was observed at each ground station. The results in Figure 10 show that the correlation of annual maximum daily precipitation between precipitation products and gauge data was poor, with CC values ranging from 0.22 to 0.41. The best correlation was found for GPM, with a CC value of 0.41. The data points of all products were biased to the right side of the 1:1 straight line, indicating that the precipitation products underestimated the annual maximum daily precipitation. The maximum underestimation value could reach more than 100 mm (partial point data of TRMM and CMORPH), and GPM performed relatively well in this respect.

4.3. Regional Differences and Proportion of Evaluation Points with Different Precisions

The inversion effect of regional precipitation at the annual and monthly scales ultimately depended on the estimated status of daily precipitation at each station. We analyzed the differences in the spatial distribution of the daily precipitation inversion performance of each product and counted the percentage of the number of evaluation points within the different index intervals described in

Table 2. The indicted range for each evaluation index interval.

Index	Division of Index Intervals
	I	II	III	IV
CC	<0.15	0.15~0.30	0.30~0.45	>0.45
BIAS	<0	0~1.5	1.5~3	>3
RMSE	<2.5	2.5~5	5~7.5	>7.5
PODrain	<0.25	0.25~0.50	0.50~0.75	>0.75
PODnorain	<0.7	0.7~0.8	0.8~0.9	>0.9
FARrain	<0.4	0.4~0.6	0.6~0.8	>0.8
FARnorain	<0.1	0.1~0.2	0.2~0.3	>0.3
POFD	<0.1	0.1~0.2	0.2~0.3	>0.3
MAR	<0.25	0.25~0.50	0.50~0.75	>0.75
HSS	<0.08	0.08~0.16	0.16~0.24	>0.24

. The indicated ranges of the different intervals for each index correspond to the color scale shown on the right side of Figure 11 and Figure 12.

Figure 11 shows the differences in the spatial distribution of CC, BIAS and RMSE for the precipitation products and the proportion of evaluation points within the different intervals of the three indexes. The correlation between precipitation products and gauge data was generally better in the southeastern region than in the other areas, with most CC values lying between 0.15 and 0.45. GPM showed more stations with high CC values (>0.3) than the other two products, with an average CC value of 0.36. The BIAS values of the three products mainly lay below 1.5. All products had negative deviation values for about 30% of evaluation points, with TRMM having the smallest number of negative deviation values among all products. For the three precipitation products, RMSE values showed the distribution characteristics of being larger in the southeast and smaller in the northeast than in the other areas. Their RMSE range mainly lay between 2.5 and 7.5 mm/day, accounting for about 90% of all evaluation points. On average, CMORPH had the smallest RMSE value among all products, but its average RMSE value was not significantly different from that of GPM.

Figure 12 shows the spatial distribution of PODrain, PODnorain, FARrain, FARnorain, POFD, MAR and HSS for the three precipitation products and the percentage of evaluation points in the different intervals of each index. For the three products, PODrain and PODnorain values of more than half of the stations ranged from 0.50 to 0.75 and from 0.80 to 0.90, respectively. At some stations in the northeastern part of the TP, the PODnorain and FARrain values were higher for all three products than in other areas, while the opposite was true for the FARnorain and POFD values. The spatial distribution of PODrain, MAR and HSS differed significantly among all products. In the south, GPM had higher PODrain and lower MAR than the other products. In the northeast, TRMM had higher MAR than the other products. Stations with high HSS scores of TRMM and CMORPH were mainly located in the southeast, while HSS values of GPM were higher in the south and southeast than in the other areas. The mean HSS score of GPM was 0.2, while those of TRMM and CMORPH were 0.16 and 0.15, respectively. In general, GPM had the best detection capability among the three products, while TRMM and CMORPH were not very different, which was confirmed by the box plots of detectability indexes shown in Figure 6.

5. Conclusions and Discussion

Based on daily precipitation data from 87 rain gauges on the TP, we comprehensively evaluated the accuracy of three precipitation products (TRMM, GPM and CMORPH) at multiple time scales. The index system was constructed from three aspects of quantitative differences, capturing capacity and precipitation distribution. The regional differences between each product and the proportion of stations with different accuracy were also analyzed. The main conclusions are as follows:

• For all three products, the correlation was always higher at the monthly scale than at the annual scale, with the correlation in December being the lowest among all months. The correlations at the annual scale, at the monthly scale and in December of GPM were higher than those of the other products. In general, GPM performed best at both the annual and monthly scales, followed by TRMM and CMORPH;
• GPM had the best detection accuracy for both precipitation and non-precipitation events, while TRMM and CMORPH had comparable capturing capabilities. The three products overestimated 0.1~1 mm/day precipitation, which occurred most frequently; underestimated precipitation at 10~20 mm/day, which contributed the largest proportion for precipitation of the TP; and underestimate the annual maximum daily precipitation;
• Compared with TRMM and CMORPH, GPM had higher CC in the southeast, higher PODrain in the south, higher HSS in the southern and central–eastern regions, and lower MAR in the southern part of the TP. The mean values of CC, BIAS, PODrain, PODnorain, FARrain, FARnorain, POFD, MAR and HSS of all stations showed that GPM had the best performance among the three products.

In this study, we evaluated the accuracy of three precipitation products on the TP using a variety of indexes. The timely evaluation of newly released products is important for the practical application of the products and the improvement of their inversion algorithms. However, due to the large topographic gradient and sparse gauges in the western part of the TP and the drastic changes in precipitation in the southeastern part of the plateau, it is necessary to further strengthen the precipitation monitoring network in the above regions. In addition, the remote sensing precipitation products have the advantages of wide coverage and long time series, and we aim to continue to apply remote sensing precipitation products in the TP in the future studies. There are two aspects to be considered in our future studies. One is to further analyze the trend of precipitation changes in the TP in recent years. The other one is to combine the inversion characteristics of different products for multi-source precipitation data fusion to provide a set of precipitation data with high accuracy for the TP.