Assessment of IMERG-V06, TRMM-3B42V7, SM2RAIN-ASCAT, and PERSIANN-CDR Precipitation Products over the Hindu Kush Mountains of Pakistan, South Asia

Abstract

In this study, the performances of four satellite-based precipitation products (IMERG-V06 Final-Run, TRMM-3B42V7, SM2Rain-ASCAT, and PERSIANN-CDR) were assessed with reference to the measurements of in-situ gauges at daily, monthly, seasonal, and annual scales from 2010 to 2017, over the Hindu Kush Mountains of Pakistan. The products were evaluated over the entire domain and at point-to-pixel scales. Different evaluation indices (Correlation Coefficient (CC), Root Mean Square Error (RMSE), Bias, and relative Bias (rBias)) and categorical indices (False Alarm Ration (FAR), Critical Success Index (CSI), Success Ratio (SR), and Probability of Detection (POD)) were used to assess the performances of the products considered in this study. Our results indicated the following. (1) IMERG-V06 and PERSIANN capably tracked the spatio-temporal variation of precipitation over the studied region. (2) All satellite-based products were in better agreement with the reference data on the monthly scales than on daily time scales. (3) On seasonal scale, the precipitation detection skills of IMERG-V06 and PERSIANN-CDR were better than those of SM2Rain-ASCAT and TRMM-3B42V7. In all seasons, overall performance of IMERG-V06 and PERSIANN-CDR was better than TRMM-3B42V7 and SM2Rain-ASCAT. (4) However, all products were uncertain in detecting light and moderate precipitation events. Consequently, we recommend the use of IMERG-V06 and PERSIANN-CDR products for subsequent hydro-meteorological studies in the Hindu Kush range.

1. Introduction

Reliable precipitation information is crucial for hydro-climatological studies, hydrological infrastructure planning, crop irrigation scheduling, and prediction of floods and droughts. Traditionally, ground-based weather radars and gauging-stations are considered as reliable sources of precipitation information [1]. However, scarcity and uneven distribution of these ground-based stations in mountainous and remote areas, hamper the access to frequent measurements for many applications [2]. Furthermore, uncertainties in ground-based measurements due to wind-induced errors, missing values, and heterogeneity, also restrict their use [3,4,5]. Similar limitations of ground-based measurements also exist in the Hindu Kush Mountains. Therefore, the exploration of other reliable sources of uninterrupted precipitation data for such areas is indispensable.

Over the past few decades, several satellite-based precipitation products (SPPs) were developed to provide uninterrupted data of global precipitation at fine spatial and temporal scales [6,7]. Generally, algorithms of SPPs retrieve the precipitation information from microwave (WM) [6] or infrared (IR) sensors [7,8]. Some of the recently developed algorithms of SPPs are capable of providing precipitation information, by combining the information of both MW and IR sensors [9]. Such algorithms were used in Tropical Rainfall Measuring Mission (TRMM) Multi-satellite Precipitation Analysis (TMPA) [10,11], Integrated Multi-Satellite Retrievals for Global Precipitation Measurement (IMERG) [12], and Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) [13]. These SPPs provide the information of precipitation occurrence at the finest temporal (≤3 h) and spatial (≤0.25°) resolutions available from current satellite instruments [14].

TRMM was launched in 1997 as a joint mission of National Aeronautics and Space Administration (NASA) and the Japanese Aerospace Exploration Agency (JAXA). TRMM provides precipitation estimates at a 3-hourly time-step with a spatial resolution of 0.25°, over a spatial domain of 50° N-S. Considering the success of the TRMM project, NASA, in a joint venture with JAXA, introduced IMERG in 2014, with the aim of providing uninterrupted global precipitation at high spatial and temporal resolutions. Recently, they launched Version 6 of IMERG products (known as IMERG-V06). This version of IMERG provides global precipitation estimates at a half-hour time-step with a spatial resolution of 0.1°. The algorithm of SM2Rain-ASCAT is based on the “bottom-up” approach and uses the information of the Advanced SCATterometer (ASCAT), Soil Moisture and Ocean Salinity (SMOS), and Advanced Microwave Scanning Radiometer for Earth Observing (AMSR-E) to estimate the global soil moisture [12,15,16]. SM2Rain-ASCAT (hereafter SM2rain) product is available since 2007. PERSIANN-CDR (hereafter PERSIANN) provides daily global precipitation information at a 0.25° grid scale. It was introduced by the Centre for Hydrometeorology and Remote Sensing, University of California, and is available since March 2000.

Although SPPs provide uninterrupted information of precipitation, their accuracy varies from region to region [17,18]. Therefore, it is important to assess their accuracy before direct application. In previous years, several researchers assessed the accuracy of SPPs in different regions of the world, such as in Africa [17,18,19], Europe [20,21,22], America [23,24,25], Austria [26], Australia [27], China [28,29], Iran [30], and Pakistan [9,31,32,33,34,35]. For instance, Anjum et al. [36] assessed the performance of IMERG-V06 and TRMM-3B42V7 (hereafter IMERG and TRMM, respectively) over the Tianshan Mountains of China and concluded that the latest post real-time (final ̶ run) IMERG product outperformed the TRMM. Mosaffa et al. [30] recommended the use of PERSIANN to understand the spatiotemporal variability of precipitation in Iran. Shraifi et al. [26] concluded that the performance of IMERG was significantly better than SM2Rain in Austria.

Although few studies evaluated the performances of IMERG-V05, TRMM, SM2Rain, and PERSIANN in Pakistan [15,32,37], the error characteristics and advantages of recently developed products (IMERG-V06 and SM2Rain-ASCAT) are not documented yet, particularly in the Hindu Kush Mountains. This mountainous range is situated on the northwestern region of Pakistan, which is about 800 km long and about 240 km wide. In the north, this mountainous system separates the Amu Darya Valley from the Indus River Basin. In the east, the Hindu Kush defends Pamir range, at the border of China and Pakistan. Most of the eastern part of the range lies in the Pakistani territory. This range splits key climate zones of Asia. It is situated at the junction of monsoon and westerlies circulation systems. This range is a source region of the Swat, Panjkora, Chitral, Gilgit, Kunhar, and Kabul Rivers. These rivers play a very important role in the agriculture, hydropower production, and socio-economic development of Pakistan. Recently, the Government of Pakistan started the construction of a multipurpose dam in this region. The literature review revealed that the scarcity of in situ meteorological stations hampered the application of their data for several hydro-meteorological applications in the Hindu Kush range [9,33]. Therefore, this study was conducted to assess and inter-compare the error characteristics of four SPPs (IMERG, TRMM, SM2Rain, and PERSIANN) against the measurements of available meteorological stations in the Hindu Kush range of Pakistan. This is the first rigorous assessment of IMERG and SM2Rain products over the Hindu Kush Mountainous range and uses all available in situ gauging stations data. Findings of this study would be beneficial for algorithm developers of considered satellite products, as well as for the data users of SPPs.

2. Materials and Methods

2.1. Study Area

The Hindu Kush Mountainous range (Figure 1), one of the largest mountainous ranges in South Asia, was selected for error characterization of four SPPs (IMERG, TRMM, SM2Rain, and PERSIANN). The eastern side of the range mainly receives precipitation from the monsoon circulation system, however, the western parts of this range are under the influence of the westerlies circulation system [9]. In this range, the snow/rainy spell starts in July and ends in September, and the winter season is mostly dry. Figure 2 demonstrates the spatial variation of mean annual precipitation. The precipitation magnitude varies significantly over the entire domain. Generally, the southern part of the range receives more precipitation, while the eastern part receives lesser precipitation. This range has a very rugged terrain; the altitude varies between 339 m and 7609 m above mean sea level (a.m.s.l.), with a north to south gradient.

2.2. Datasets

The daily precipitation datasets of 24 in situ meteorological stations were obtained from Water and Power Development Authority (WAPDA) and Pakistan Meteorological Department (PMD) for the period of January 2010 to December 2017. The locations of considered meteorological stations are presented in Figure 1. The quality of precipitation records was ensured by two data handling organizations (WAPDA and PMD). Previously, several researchers used the daily records of considered meteorological stations for various hydro-climatological applications [9,29,34,38].

Table 1. Salient features of the meteorological stations.

Sr. No.	Station	Longitude	Latitude	Altitude	Average Annual
		(°)	(°)	(m)	Precipitation (mm)
1	Amandara	71.98	34.63	664	753.1
2	Astore	74.90	35.37	2394	359.7
3	Balakot	73.35	34.38	995	1301.6
4	Bunji	74.63	35.67	1372	179.0
5	Chillas	74.10	35.42	1251	171.2
6	Chitral	71.83	35.85	1498	432.2
7	Dir	71.85	35.20	1425	1303.1
8	Drosh	71.78	35.57	1464	509.9
9	Gilgit	74.33	35.92	1460	168.1
10	Gupis	73.40	36.17	2156	176.2
11	Kakul	73.25	34.18	1308	1273.6
12	Kalam	72.60	35.47	2744	904.1
13	Khot	72.58	36.52	3505	541.2
14	Kohistan	73.19	35.32	841	924.8
15	Lower Dir	71.82	34.83	786	876.3
16	Naltar	74.27	36.22	2810	675.7
17	Naran	73.65	34.90	2363	1824.8
18	Pattan	73.03	35.10	752	1091.3
19	Peshawar	71.51	33.99	362	488.0
20	Saidu Sharif	72.35	34.82	961	985.8
21	Ushkore	73.36	36.02	3350	286.7
22	Yasin	73.30	36.63	3353	296.2
23	Zani Post	72.15	36.28	3000	194.1
24	Ziarat	74.28	36.83	3669	773.9

elaborates the salient features of in situ meteorological stations.

The PERSIANN product is capable of providing information on global daily precipitation at a 0.25° grid-scale over a spatial domain of 50° S to 50° N [39]. In the present study, daily precipitation estimates of PERSIANN at 0.25° grid scale for eight years (from 2010 to 2017) were downloaded from the website CHRS (https://chrsdata.eng.uci.edu/). The estimates of daily precipitation of SM2Rain-ASCAT product at the 12.5 km grid scale were obtained from http://hydrology.irpi.cnr.it/download-area/sm2rain-data-sets/. The three-hourly precipitation estimates of the TRMM product at a 0.25° grid scale were obtained from https://gpm.nasa.gov/data/directory, while half-hourly precipitation estimates of the IMERG Final-Run product at a 0.1° grid-scale were downloaded from https://giovanni.gsfc.nasa.gov/giovanni/. The SPPs with half-hourly and three-hourly temporal resolutions (IMERG and TRMM, respectively) were first adjusted by considering the local data acquisition time (from UTC 00:00 to UTC 05:00) and were then accumulated to acquire daily precipitation estimates. Monthly, seasonal, and annual time series data of all SPPs were obtained by accumulating daily data. Previously, several researchers followed the same methodology to obtain the monthly, seasonal, and annual data from the daily or sub-daily precipitation estimates of SPPs [9,40].

2.3. Methods

The error characteristics of four SPPs (IMERG, TRMM, PERSIANN, and SM2Rain) were assessed against the measurements of in situ meteorological stations. In this study, only grids of SPPs that contained at least one reference meteorological station were considered, by following the methodology of Wang et al. [40]. The performances of SPPs were assessed at different temporal (daily, monthly, seasonal, and annual) and spatial (entire domain and point-to-pixel) scales, against measurements of reference gauging stations, for the period of January 2010 to December 2017. The spatial distribution map of annual average precipitation over the study domain was developed by using geostatistical spatial interpolation techniques (Kriging with external drift (KED)). KED is recommended for interpolation in mountainous regions [37,38].

The skills of all SPPs to reproduce the spatial variability of precipitation over the Hindu Kush Mountains were assessed. Several widely used evaluation indices (Correlation Coefficients (CC), Bias, relative Bias (rBias), and Root Mean Square Error (RMSE), categorical indices (Probability of Detection (POD), False Alarm Ratio (FAR), and Critical Success Index (CSI) were used to assess the performance of SPPs. Previously, many researchers used these indices to evaluate the error characteristics of SPPs in different climatic and topographic conditions [9,36,41,42,43]. The skill of SPPs to characterize the temporal variation of precipitation over the mountains was also examined by comparing their daily temporal variation with variation of the reference data.

The dimensionless CC was estimated to measure the linear agreement between reference measurements and estimates of SPPs. BIAS (mm/time) was estimated to assess the overestimation or underestimation of the precipitation magnitude. The rBias (in percentage) was determined to evaluate the relative difference between the observations of in situ stations and SPPs. RMSE was computed to quantify the average magnitude of error (mm/time) in the satellite-based data, with reference to the gauge-based data.

$$CC=\frac{{\displaystyle {\sum }_{i=1}^n(G_i}-\overline{G})(S_i-\overline{S})}{\sqrt{{\displaystyle {\sum }_{i=1}^n}{(G_i-\overline{G})}^2}\times \sqrt{{\displaystyle {\sum }_{i=1}^n{(S_i-\overline{S})}^2}}}$$

(1)

$$Bias=\frac{{\displaystyle {\sum }_{i=1}^n(S_i-G_i)}}{n}$$

(2)

$$rBias=\frac{{\displaystyle {\sum }_{i=1}^n(S_i-G_i)}}{{\displaystyle {\sum }_{i=1}^n{G}_i}}\times 100$$

(3)

$$RMSE=\sqrt{\frac{1}{n}{{\displaystyle {\sum }_{i=1}^n(S_i-G_i)}}^2}$$

(4)

where $G_i$ denotes the gauge-based (reference) data, $\overline{G}$ is mean of reference data, Si means the satellite-based estimates, $\overline{S}$ indicates the mean of satellite-based estimates, and n indicates the total number of observations. If the estimated values of BIAS and RMSE for a satellite product are zero, and the value of CC is unity, then that product is considered as the perfect proxy of gauge-based data. Some recent performance assessments of SPPs [9,43,44] documented the thresholds of CC and rBias to accept their accuracy for hydro-climatological applications. The documented thresholds for rBias and CC are ±10 and ≥0.70, respectively. Negative/positive values of BIAS denote underestimation/overestimation of the precipitation amount.

The Probability of Detection (POD), Critical Success Index (CSI), and False Alarm Ratio (FAR) were calculated to evaluate the ability of SPPs to represent the occurrence of precipitation events [45]. The POD indicates the proportion of events that were correctly detected by the product, out of the total reported events. Higher values of POD indicate that majority of the precipitation events were detected by the SPPs. FAR is the proportion of events that were incorrectly recorded by the precipitation product out of the total reported events of the product. Higher values of FAR show a higher error in the precipitation detection skill of the SPPs. CSI is the ratio between the events that were correctly detected by the product to the total number of precipitation events recorded by the product. Higher values of CSI indicate a high ratio of the events that were correctly detected by the product. In the present study, the threshold for the computation of CSI, FAR, and POD was 1 mm/day, by adopting the methodology of previous studies [9,40]. The categorical indices were computed using the following equations:

$$POD=\frac{H}{H+M}$$

(5)

$$CSI=\frac{H}{H+M+F}$$

(6)

$$FAR=\frac{F}{H+F}$$

(7)

where H indicates the events that were correctly detected by the SPPs, F represents the events that were falsely reported by the product, and M denotes the events that were missed by the product. The perfect value of CSI and POD is 1 and that of the FAR is zero.

Probability Density Function (PDF) of the daily estimates of four SPPs and reference gauges were also calculated at different precipitation intensities. To estimate the PDF of SPPs, different precipitation intensities were selected by following the recommendations of World Meteorological Organization (WMO). Additionally, the Taylor diagrams (Taylor (2001)) were drawn to present the brief summary of the agreements of satellite products, with the gauging stations. In Taylor diagrams, the values of standard deviation of the gauge-based measurements and the estimates of SPPs were plotted on the vertical axis, values of RMSE were displayed on the angular distance from the x-axis, and values of CC were drawn along the azimuthal angle.

3. Results

3.1. Skill of SPPs to Track the Spatio-Temporal Variability of Precipitation

Figure 3 illustrates the spatial variation of average daily precipitation acquired from the reference meteorological stations and four SPPs over the study area. Significant spatial variability in the precipitation magnitude was found over the study domain, which might be due to the complex topography. Overall, all data sources (reference gauges and SPPs) indicated higher precipitation amounts over the low elevation areas (southern part of the mountain). Daily station-based precipitation showed considerable spatial variation, as shown in Figure 3. The records of reference gauges indicated a higher amount of precipitation over the south-western parts of the Hindu Kush range, whereas the meteorological stations installed in the north-western parts of the study area exhibited a lesser amount of precipitation. Results showed that TRMM was unable to track the spatial variability of observed precipitation and it significantly underestimated the precipitation magnitude over all reference stations. PERSIANN slightly overestimated and underestimated the precipitation over the north-eastern and north-western parts, respectively. It showed better performance over the meteorological stations installed in the southern parts of the study domain. Although the ability of SM2Rain to track the spatial variability of observed precipitation was better over the low elevation areas, it was uncertain to accurately report the precipitation magnitude over the higher altitudes. Despite the fact that IMERG showed considerable over and underestimations at some of the considered meteorological stations, the overall performance of IMERG in terms of ability to track the spatial variability of precipitation magnitude was better than the other considered SPPs.

Figure 4 shows the comparison of the temporal variability of average daily precipitation magnitude obtained from the reference stations and four SPPs. The time-series of precipitation were obtained by taking the moving average of daily data of reference stations and the considered SPPs for the period of January 2010 to December 2017. Previously, [8,36] applied the same methodology to compare the variability of daily precipitation obtained from the reference stations and SPPs over the Tianshan Mountains of China and the northwestern part of South America, respectively. The Hindu Kush range receives high precipitation in two seasons, winter and summer, due to westerlies and monsoon circulation systems, respectively [18]. The reference (station-based) data showed two peaks in a year, as shown in Figure 4. Comparison showed that the PERSIANN and IMERG products were capable of tracking the temporal variability of observed precipitation over the study domain. The average of daily data obtained from the reference stations was 1.9 mm/day, with a variation between 0.0 and 10.2 mm/day. In comparison, the satellite-based average daily precipitation magnitude varied between 0.0 and 11.1 mm/day (with an average of 1.7 mm/day), between 0.0 and 6.5 mm/day (with an average of 1.8 mm/day), between 0.0 and 5.8 mm/day (with average 1.5 mm/day), and between 0.0 and 2.7 mm/day (with average 0.5 mm/day) for IMERG, PERSIANN, SM2Rain, and TRMM, respectively. SM2Rain underestimated the precipitation magnitude in the winter season and overestimated it in summer season, whereas TRMM underestimated the precipitation magnitude throughout the year. Comparatively, IMERG and PERSIANN showed better performances than the TRMM and SM2Rain products in terms of ability to track the temporal variability of daily precipitation.

3.2. Performance of SPPs at Monthly Scale

Figure 5 (Taylor diagram) summarizes the performances of the four SPPs (PERSIANN, SM2Rain, TRMM, and IMERG) at a monthly scale. In order to develop the Taylor diagram, we used the normalized areal average data of meteorological stations and SPPs, following [31]. Generally, all SPPs indicated a good linear relationship with the reference monthly data, except SM2Rain. The correlations between the monthly station-based data and the estimates of IMERG, PERSIANN, and TRMM were higher than 0.80, however, the value of CC for SM2Rain was less than 0.50. The estimated values of RMSE for IMERG and PERSIANN were lower than 0.50. Conversely, the values of RMSE for TRMM and SM2Rain were higher than 0.5 (Figure 5), which showed a higher error in the estimates of SM2Rain and TRMM products over the Hindu Kush range. The values of SD of all SPPs were comparable with the SD value of the reference data.

Figure 6 displays the variation in the estimated values of CC, BIAS, and RMSE for four SPPs (PERSIANN, SM2Rain, TRMM, and IMERG) at each meteorological station. Box length of CC for TRMM showed significant variation, and box length of CC for the PERSIANN showed minimum variation. The variations in the BIAS and rBIAS values of IMERG were minimum, as compared to other SPPs in the Hindu Kush range, as indicated by the box lengths of BIAS and rBIAS for this product. The box lengths of RMSE showed the highest variation in the values of RMSE for TRMM.

3.3. Performance of the Satellite-Based Products at Daily Scale

Figure 7 summaries the performances of SPPs (PERSIANN, SM2Rain, TRMM, and IMERG) against daily gauge-based precipitation data over the study domain. It was found that the CC values of IMERG, PERSIANN, SM2Rain, and TRMM were 0.41, 0.42, 0.20, and 0.30, respectively. This showed poor agreements of the daily gauge-based data, with the estimates of SM2Rain and TRMM products. However, all SPPs exhibited poor performance in terms of RMSE, as indicated by its higher values (>0.5) in Figure 7.

Figure 8 displays the box plots of CC, BIAS, and RMSE for four SPPs (PERSIANN, SM2Rain, TRMM, and IMERG). Box length of CC for IMERG indicates the lowest variation in the values of CC for this product (Figure 8a). Generally, all SPPs showed high inconsistency and low linear relationship with the daily gauge data. TRMM showed a higher variation in the BIAS values, whereas IMERG indicated the lowest variation in the BIAS values (Figure 8b). In comparison, the box length of RMSE for IMERG (Figure 8c) showed the highest variation in the RMSE values of this product. Overall, all products showed significant inconsistencies in the values of evaluation indices on a daily scale.

Figure 9 shows the effect of elevation on the performance of all used satellite products in terms of CC, BIAS, rBIAS, and RMSE. It was found that the RMSE values of all SPPs decreased with the increase of elevation. With the increase of elevation, the values of CC and rBIAS of SM2Rain and IMERG-V06 decreased, whereas the values of PERSIANN and TRMM increased with an increase in elevation.

Figure 10 represents the influence of the average daily precipitation on the evaluation indices (CC, BIAS, rBIAS, and RMSE) of SPPs. A decrease in the BIAS (rBIAS) and an increase in the RMSE was found with the increases of the precipitation rate. All products underestimated the precipitation at higher precipitation rates, compared to the station data on daily time scales. The values of CC of SM2Rain showed an increasing trend with an increase of the precipitation rate. However, the influence of precipitation rate on the CC of PERSIANN, TRMM, and IMERG with the station data was not considerable, as shown in Figure 10. Overall, CC did not seem too dependent on the precipitation rate. Thus, it was deduced that the products were uncertain to capture the correct magnitude of precipitation and tended to underestimate it with an increasing precipitation rate.

Figure 11 displays the spatial variation of rBIAS, CC, and RMSE estimated for the PERSIANN, SM2Rain, TRMM, and IMERG products at a daily scale. Considerable spatial variations in the values of evaluation indices were found in the study area. The spatial pattern of rBIAS showed that all TRMM and SM2Rain products were uncertain to accurately estimate the precipitation amount at most meteorological stations. TRMM showed significant underestimation of precipitation amount over most stations, while IMERG was the least biased product over most stations, followed by the PERSIANN. SM2Rain showed a good performance over the lower altitudes. The spatial pattern of CC indicated that all agreements between the SPPs and reference gauges were inconsistent over the study domain. Comparatively, the IMERG and PERSIANN products were in good agreement with the reference data at the higher altitudes. The spatial pattern of RMSE showed significant variations in the error characteristics of SPPs. Generally, all SPPs indicated higher values of RMSE over the southern parts of the range. It is worth pointing out that the best agreement in terms of RMSE was in the northeast region of the domain, where all SPPs had their lowest RMSE values.

3.4. Performance of Satellite Products at Seasonal Scale

Figure 12 exhibits the variations in the estimated values of CC, BIAS, and RMSE for SPPs, at a seasonal scale. The box lengths of CC showed high variations in the linear agreements between the reference data and the estimates of PERSIANN, SM2Rain, and IMERG products in the autumn and winter seasons. The box lengths of BIAS of all SPPs indicated less variation in the over- and underestimation of precipitation magnitude in autumn, while all products showed considerable variations in the Bias in other seasons, as indicated by the box lengths of Bias of all products. The values of RMSE for IMERG showed the highest variations, as compared to the other SPPs.

Figure 13 shows the relative Biases (%) calculated for all precipitation products against the station-based precipitation data, at annual and seasonal scales over the entire study domain. SM2Rain and TRMM products showed high inconsistency in all seasons. TRMM indicated significant underestimation of precipitation in all seasons, whereas SM2Rain overestimated the precipitation in the summer season and underestimated it in all other seasons. On annual scales, both IMERG and PERSIANN showed an acceptable underestimation of precipitation, whereas TRMM and SM2Rain exhibited significant underestimation of precipitation. Overall, the performances of IMERG and PERSIANN products were acceptable in all seasons, albeit IMERG slightly underestimated the values in the spring season and PERSIANN slightly overestimated the precipitation magnitude in the autumn season.

3.5. Ability of SPPs to Detect Occurrence of Precipitation

Figure 14 shows the performance diagram of precipitation estimates obtained from four SPPs over the Hindu Kush Mountains of Pakistan. Roebber [46] introduced a performance diagram to graphically compare the results of the categorical indices (POD, SR, CSI, and Bias) of the products, where SR is equal to 1-FAR. It is known as the Roebber Performance diagram (or simply performance diagram) and shows the geometric relation between the reference gauges and satellite products. Previously, some researchers used the performance diagram to illustrate the performances of different precipitation products in different countries. For instance, [9] used this diagram in the northern mountainous terrain of Pakistan, [2] used this diagram in Egypt, and [47] used in Brazil. In the performance diagram, if the values of all estimated indices for a product are close to one, then that product is regarded as a suitable proxy for reference data. The values of the POD for the IMERG, PERSIANN, SM2Rain, and TRMM are 0.76, 0.70, 0.55, and 0.19, respectively, as shown in Figure 14. The higher values of POD of IMERG and PERSIANN products showed that the detection skills of these products were good and most events were detected by these SPPs. This finding is also supported by the high values of SR for both of these products. The TRMM was uncertain in detecting the occurrence of precipitation events, as exhibited by the low value of POD for this product.

The ability of SPPs to accurately report the occurrence of precipitation events is summarized in terms of performance diagrams on the seasonal scale. Figure 15 shows that the detection skills of IMERG and PERSIANN products were generally better than SM2Rain and TRMM products, particularly in the spring and winter seasons. SM2Rain showed good detection ability in the summer season, while in the other three seasons, it showed poor detection ability. Overall, SM2Rain was relatively better than TRMM product, which displayed very poor performance in all seasons.

Figure 16 shows the Probability Density Function (PDF) calculated for the daily precipitation events reported by the reference stations and SPPs in the Hindu Kush Mountains. Analysis of the daily measurements of the reference gauges for the entire study period showed that the light precipitation events (<2 mm/day) occurred most frequently (72.0%). TRMM showed overestimation of light events (>95%), IMERG and SM2Rain slightly overestimated the light precipitation events (74.7% and 74.3%, respectively), whereas PERSIANN slightly underestimated the occurrence of light precipitation events (69.9%). Generally, all SPPs showed underestimation of the occurrence of moderate precipitation events (2–10 mm/day), except PERSIANN, which showed slight overestimation of moderate precipitation events. On a seasonal scale, the overall performances of all SPPs in terms of PDF of light and heavy precipitation events were generally consistent with their performances at the daily scale. Generally, TRMM and SM2Rain showed larger deviations from the PDFs for winter, spring, and summer. IMERG consistently showed the best agreement. It is worth pointing out that summer showed the worst agreement for all SPPs.

4. Discussion

In this study, different statistical indices were computed to analyze the error characteristics and accuracies of four satellite-based precipitation products over the Hindu Kush Mountains of Pakistan. Previously, several researchers assessed the performances of the SPPs (including, PERSIANN, SM2Rain, TRMM, and IMERG) in different climatic and topographic conditions of the world [14,26,28,35,36,45,47,48,49,50,51,52]. It is well-documented that the performances of SPPs products are strongly dependent on the in-situ topographic and climatic conditions. For instance, Tan et al. [50] compared six SPPs (including, PERSIANN-CDR and TRMM-3B42V7) over Malaysia. They concluded that the regional climatology and precipitation retrieval algorithm of satellites significantly affected the accuracy of the precipitation products. Xu et al. [29] compared GPM and TRMM over the Southern Tibetan Plateau. They suggested that different indices (BIAS and RMSE) were significantly correlated with topographic variations.

In the current study, we compared the accuracy and error characteristics of the latest product of GPM (IMERG-V06) with the TRMM, PERSIANN, and SM2Rain, over the Hindu Kush Mountains of Pakistan. The SPPs were assessed against the measurements of 24 reference meteorological stations, at different spatial and temporal scales from January 2010 to December 2017. Results of the present study advocated that the performances of considered satellite products were considerably affected by the topography and climatic conditions of the study area, which was consistent with the results of previous studies [9,31,36]. SM2Rain and TRMM products were unable to represent the spatial variability of the precipitation, which was consistent with the finding of [30,51]. Among the considered SPPs, the skills of IMERG and PERSIANN to characterize the spatial variability of the observed precipitation were better. In a previous study, Alijanian et al. [52] confirmed that the skill of PERSIANN–CDR to characterize the spatial variation of the precipitation was better than other satellite-products. Correlations of all SPPs with the reference data at a daily scale were not good, as witnessed by the estimated values of CC (<0.50) of products. The correlations of IMERG, PERSAINN, and TRMM with the reference monthly data were >0.80, which advocated for better agreements of monthly estimates of SPPs, with the reference data, as compared to the daily data. It was found that the linear agreements between the reference data and estimates of SM2Rain and TRMM decreased with an increase in the precipitation magnitude, while there was an opposite trend in the agreements between the station-based data and the estimates of IMERG and PERSIANN. Previously, Wang et al. [36] also reported a similar pattern of correlation between IMERG and gauge-based data, in a mountainous region of China. The probability of detection (POD) of IMERG and PERSIANN was quite better than SM2rain and TRMM products, with POD > 0.60. Similar findings of POD for the considered SPPs were also reported by [8]. However, TRMM showed very poor performance in terms of POD than the other three products at a daily scale, which was consistent with the finding of [9]. TRMM and SM2Rain showed significant underestimation of the precipitation magnitude at daily and monthly scales. Previous studies confirmed that TRMM showed underestimation at the Himalayan region [53,54,55]. The overall performances of all SPPs were better at the monthly scale than the daily scale, which was consistent with a previous study [50]. On seasonal scales, the correlations of SM2Rain and TRMM with the gauge data were also very poor (<0.40), whereas CC of IMERG and PERSIANN with the reference data were moderate (>0.40) in all seasons. TRMM and SM2Rain showed significant underestimation of precipitation magnitude in all seasons, except summer, when Sm2Rain significantly overestimated the precipitation magnitude. PERSIANN and IMERG showed the lowest Bias with the gauge data in all seasons.

5. Conclusions

In the present study, the performances of four satellite-based precipitation products (PERSIANN, SM2Rain, TRMM, and IMERG) were assessed on different temporal (daily, monthly, and seasonal) and spatial (station and regional) scales, with reference to the observations of 24 in situ gauges in the Hindu Kush region of Pakistan. All precipitation products were evaluated for the period of January 2010 to December 2017. The main conclusions of the present study are as following:

• Two of the considered SPPs (IMERG and PERSIANN) were capable of characterizing the spatial variability of precipitation over the Hindu Kush Mountains of Pakistan. However, SM2Rain and TRMM products were unsuitable for understanding the spatial variation of precipitation over the said spatial domain.
• The temporal variation of average daily precipitation was captured well by the IMERG and PERSIANN products, while SM2Rain and TRMM products were uncertain to characterize the temporal variability of precipitation.
• The overall performances of all considered SPPs were better at the monthly scale than the daily scale.
• TRMM and SM2Rain showed a significant underestimation (73.95% and 20.89%, respectively) of precipitation magnitude, while IMERG and PERSIANN exhibited a slight underestimation of the precipitation amount by −8.85% and −1.24%, respectively, over the Hindu Kush region.
• The precipitation detection capabilities of PERSIANN and IMERG products were better than the TRMM and SM2Rain products. The IMERG showed the best performance in terms of probability of detection (0.76), followed by PERSIANN (0.70). The performance of TRMM in terms of POD was very poor (<0.30).
• Detection skills of IMERG and PERSIANN in all seasons were good (>0.70). In this area, the overall performance of TRMM was very poor in all seasons.

The results of this study advocated that the IMERG-V06 and PERSIANN-CDR products outperformed the TRMM and SM2Rain products for the Hindu Kush Mountains in Pakistan. Both of these products (IMERG-V06 and PERSIANN) were capable of characterizing the spatio-temporal variability of the observed precipitation over the study region, which confirmed their application in understanding the spatial variability of precipitation over the Hindu Kush Mountains of Pakistan. Moreover, the value of CC for IMERG-V06 and PERSIANN products was >0.70 and rBias was within the acceptable limit (±10), at monthly scales. Thus, we recommend the use of their monthly estimates as a complimentary substitute for hydro-meteorological studies in the Hindu Kush range.