Comparing Multiple Precipitation Products against In-Situ Observations over Different Climate Regions of Pakistan

Abstract

Various state-of-the-art gridded satellite precipitation products (GPPs) have been derived from remote sensing and reanalysis data and are widely used in hydrological studies. An assessment of these GPPs against in-situ observations is necessary to determine their respective strengths and uncertainties. GPPs developed from satellite observations as a primary source were compared to in-situ observations, namely the Climate Hazard group Infrared Precipitation with Stations (CHIRPS), Multi-Source Weighted-Ensemble Precipitation (MSWEP), Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks-Climate Data Record (PERSIANN-CDR) and Tropical Rainfall Measuring Mission (TRMM) multi-satellite precipitation analysis (TMPA). These products were compared to in-situ data from 51 stations, spanning 1998–2016, across Pakistan on daily, monthly, annual and interannual time scales. Spatiotemporal climatology was well captured by all products, with more precipitation in the north eastern parts during the monsoon months and vice-versa. Daily precipitation with amount larger than 10 mm showed significant (95%, Kolmogorov-Smirnov test) agreement with the in-situ data, especially TMPA, followed by CHIRPS and MSWEP. At monthly scales, there were significant correlations (R) between the GPPs and in-situ records, suggesting similar dynamics; however, statistical metrics suggested that the performance of these products varies from north towards south. Temporal agreement on an interannual scale was higher in the central and southern parts which followed precipitation seasonality. TMPA performed the best, followed in order by CHIRPS, MSWEP and PERSIANN-CDR.

1. Introduction

Precipitation is a key factor in water and energy cycles and a significant number of studies have attempted to produce high resolution precipitation products for use in climate studies [1,2,3,4]. The applications of precipitation products extend into hydrological and energy cycle characterization [5,6] agricultural studies [7] and land-atmosphere interactions and feedback studies [8,9]. These applications make use of precipitation data to improve precipitation forecasts and the related geophysical fluxes [10,11], long term historical trends [12] and future projections based on historical records [13,14]. In terms of natural hazards, the excess and shortage of precipitation has significant impacts on every aspect of society, including human casualties, agriculture, livestock, displacement and damage to civil and hydrologic structures [15,16].

The primary land-based instrument for precipitation measurement is the rain gauge. The density of rain gauges is not uniform globally, with few rain gauges in remote regions and areas with limited access. In such cases, studies rely upon data from nearby rain gauges that is extended through statistical interpolation and extrapolation techniques. Although in-situ data have several limitations and uncertainties, they have been used for decades as a key source of precipitation information in decision making and planning and data assimilation [17].

In recent decades, remote sensing has become a promising tool for measuring precipitation. Radiances measured in the visible, infrared or microwave bands are transformed into precipitation with the help of statistically, quantitatively and physically based algorithms [18]. Remote sensing has proven a reliable and cost-effective way of retrieving precipitation at various scales. Among the most commonly used remotely sensed precipitation products include the Tropical Rainfall Measuring Mission (TRMM), the Multi-satellite Precipitation Analysis (TMPA), [19], the Climate Hazard group Infrared Precipitation (CHIRPS) [4], the Climate Prediction Center MORPHing product (CMORPH) [20], the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Network (PERSIANN) [21], the Multi-Source Weighted-Ensemble Precipitation (MSWEP) [22,23] and the Asian Precipitation-Highly-Resolved Observational Data Integration Towards Evaluation (APHRODITE) [24]. The recent development in the precipitation retrieval is the launch of Global Precipitation Measurement (GPM) mission. GPM is a successor of TRMM, launched by NASA and JAXA with partnering agencies and countries during 2014. GPM consists of one core and eight constellation satellites, the core satellite is equipped with dual-frequency precipitation radar and a microwave radiometer and the constellation satellites have micro radiometers onboard [25,26].

The precipitation products listed above have been developed for different purposes [27,28]. Their applications are complicated and the selection of one product over another is always debatable. Therefore, it is important to evaluate such products against in-situ observations, which are considered to be the standard [27,28]. The strengths and uncertainties of gridded satellite precipitation products (hereinafter to be referred to as GPPs) are thoroughly reported through comparisons with in-situ data from various locations. When in-situ data is used as reference data, GPPs perform well during warm and wet seasons over certain regions but their performance is less effective during cold seasons and in dry regions with less precipitation [29,30]. The major uncertainties reported are an underestimation of precipitation and relatively high error metrics over oceans and across surfaces where the topography and land cover varies. The sources of uncertainty also vary and depend on the quality of in-situ data, which also contain deviations and uncertainties [27,31]. Efforts have been made to reduce the uncertainties of existing precipitation products by developing GPPs with a higher spatial and temporal resolution [32,33]. However, the validation and evaluation of these products is based on thresholds and scores or is limited to certain regions with higher station densities. The conclusions from such studies might not be realistic for representing regional and larger scale precipitation behavior.

Few studies have been conducted to compare the performance of GPPs with in-situ data that are limited to small regions, areas with a low density of station records or over short time periods. In the Swat river watershed in northern Pakistan TRMM products have been used in hydrological modelling but it has been reported that TMPA did not capture the precipitation magnitude and spatial patterns at annual and seasonal scales well enough for hydrological applications [34]. TMPA and APHRODITE products were compared with station data for 1998–2013. It was found that TMPA products slightly overestimated daily, monthly and annual precipitation, with variable error metrics according to the terrain elevation and the climatic region [35]. TRMM 3B43 monthly and reanalysis products were used in the Indus Basin alongside in-situ and river discharge data, significant uncertainties were observed in all GPPs mainly due to the lower number of in-situ records for validation in the complex topography [36]. TMPA and TMPA-RT products for 2005–2008 were compared over the monsoon regions of Pakistan and it was found that these datasets were reliable for monsoon season forecasts and applications [37]. TMPA (3B42 V-6 &V-7) and reanalysis precipitation for 1998–2006 were compared over India and it was found that the improved TMPA V-7 could capture monsoon and post-monsoon precipitation [38]. Monthly TRMM products were used in the Indus Basin for calibration purposes in runoff and water balance studies. It was reported that TRMM products underestimated precipitation over north western and south western coastal regions and overestimated precipitation in mountainous regions [39].

In this study, four GPPs with different resolutions, namely MSWEP, CHIRPS, PERSIANN-CDR and TMPA-3B42 V-7 were compared over a range of climatic zones in Pakistan, with varying land cover and elevation for the time period of 1998–2016. Daily precipitation records from 51 rain gauges were used, which enabled a more detailed analysis in terms of temporal and spatial coverage compared to previous studies in the same study area. The remainder of the paper consists of sections covering the study area, data and methods and results, while Section 5 and Section 6 provides a discussion and conclusion.

2. Study Area

Pakistan is located in south Asia, with a latitudinal range of about 23°N to 37°N and longitudinal range of about 60°E to 78°E (Figure 1). The elevation of the study area varies from an altitude of a few meters in the south to more than 5000 m in the north [14]. The land cover and cropping pattern of the study area differs with the fluctuating topography and climate [36].

Precipitation in the country is largely dependent on the monsoon during summer, which accounts for >60% of annual precipitation in the eastern parts and westerlies during the pre—monsoon season in the western parts. There is also great variation in temperature from north to south, with the maximum average temperature ranging from 15 to 35 °C, while the minimum temperature ranges from less than zero to 14 °C [14,40]. Climate change has had a significant impact on Pakistan in terms of heavy precipitation/drought and temperature extreme events, which have caused damage on a vast scale [41,42]. An assessment of the performance of GPPs is therefore extremely important, enabling it to be further used for different applications including hydrological and agricultural studies and in planning and decision making.

3. Data and Methods

3.1. Data

3.1.1. In-Situ Observations

To record synoptic scale precipitation data, the Pakistan Meteorological Department (PMD) has installed a network of meteorological observatories across the country as shown in Figure 1b. The density of these stations differs from region to region within the country but the large-scale dominant precipitation features can be determined. The daily observations collected from these stations are already quality controlled and are used for PMD operational purposes. To ensure that the data is homogenous and consistent, a suite of visual and statistical techniques, including a standard normal homogeneity test (SNHT), are adopted here [43]. SNHT is a statistical technique most often used for homogeneity estimation of climate data records, where the purpose of SNHT and other homogenization tests is primarily to detect outliers or spikes in a dataset that could be attributed to non-climatic factors. Such changes might be induced through changes in location, measurement techniques and physical features of the surrounding environment, whereas climate signals are preserved [43,44,45]. Only a small amount of data (<4.5%) was missing at a few stations. To make the comparison more accurate, the corresponding values from GPPs were also removed. This procedure did not substantially affect the results of the study.

3.1.2. MSWEP

MSWEP is a recently developed product, specifically designed for global scale hydrological applications. The product is developed from a number of high quality input datasets that include gauge-based interpolated products (WorldClim, GHCN-D, GSOD, Climate Prediction Center (CPC) Unified and Global Precipitation Climatology Centre (GPCC)), remotely sensed products (Climate Prediction Center Morphing method (CMORPH), Global Satellite Mapping of Precipitation using Moving Vector with Kalman filter (GSMaP-MVK), TMPA 3B42 RT and GridSat and reanalysis products (European Centre for Medium-Range Weather Forecasts (ECMWF) Re-Analysis (ERA-Interim) and Japanese 55-year Reanalysis (JRA-55). The main features of MSWEP include a bias correction of orographic effects and gauge undercatch inferred from globally available stream flow records. The MSWEP has been continuously updated to improve its performance at global and regional scales [3]. In this study we used the MSWEP version 2.01 daily product, with a grid size resolution of 0.1 degrees [22,23].

3.1.3. CHIRPS

The Climate Hazard Group Infrared Precipitation with Stations Data (CHIRPS) is a combined product of remotely sensed and in-situ observations developed for monitoring droughts and extremes. The main input data sources used in CHIRPS are monthly precipitation climatology from Climate Hazard group (CHPClim), quasi global geostationary thermal infrared observations from National Oceanic and Atmospheric Administration, Climate Prediction Center (NOAA-CPC), the National Climatic Data Center (NCDC), TRMM 3B42, NOAA Climate Forecast System version 2 (CFSv2) and gauge observations from various sources. The algorithm is based on the cold cloud duration (CCD) notion, that is, the amount of time a given pixel is covered by high cold clouds infrared brightness temperature (IR BT < 235 K). Precipitation estimates are obtained through a CCD’s calibration procedure exploiting the TMPA 3B42 data set from 2000 to 2013. These IR-based precipitation estimates are expressed as a percentage of the 1981–2013 mean precipitation estimates and then multiplied by the corresponding CHPclim values, so obtaining the CHIRP product. This last step is carried out to reduce the systematic bias in the IR based precipitation estimates. Finally, IR estimates are merged to the rain gauge measurements using the inverse distance weighting algorithm to produce the CHIRPS product. For the current study we used daily observations with a grid size resolution of 0.05 degrees. Further details can be seen at [4].

3.1.4. PERSIANN-CDR

PERSIANN-CDR is a long-term gridded precipitation product developed for hydro-climatic studies. PESIANN-CDR is developed from infrared and passive microwave observations that are processed with PERSIANN algorithm for generating the rain rates. Rain rate information are retrieved from low earth and geostationary satellites, however the passive microwave observations used are only available from 1997 and prior period. To overcome the limitation of passive microwave observations, National Centers for Environmental Prediction (NCEP) stage IV radar data is used to train the nonlinear regression parameters of the artificial neural network model. The artificial neural network estimates precipitation from cold cloud pixel information and nearby features, using the gridded satellite brightness temperature information of international satellite cold climatology project. Finally, PERSIANN-CDR is checked and corrected for bias correction against GPCP monthly product. The final PERSIANN-CDR product provide daily scale precipitation which covers the period from 1983 to the present day, with a grid size of 0.25° between 60°S and 60°N. Further details are provided in References [2,21].

3.1.5. TMPA

The TRMM was launched in 1997 by JAXA and NASA for monitoring tropical and sub-tropical precipitation conditions. TRMM operated in low earth orbit, with an inclination of 35° and altitude of 400 km and orbits the Earth 16 times per day. TRMM satellite was equipped with a number of instruments, including the Precipitation Radar (PR), Microwave Imager (TMI), Visible Infrared Scanner (VIRS), Cloud and Earth Radiant Energy System (CERES) and Lightning Imaging Sensor. TRMM Multi-satellite Precipitation Analysis (TMPA) uses Passive Microwave (PM) precipitation estimates from low orbiting satellites and in case PM observations are not available, then PM calibrated Infrared observations are used for gap filling [19]. In the next step, the monthly gauge-based observations are used to produce TMPA from TMPA real time precipitation estimates. The input data used for TMPA is derived from the Special Sensor Microwave Imager (SSM/I) and Sounder (SSMI/S), Microwave Imager (TMI), the Advanced Microwave Sounding Unit (AMSU), the Microwave Humidity Sounder (MHS) and Advanced Microwave Scanning Radiometer for Earth Observing system (AMSR-E). In this study, daily TMPA version 7 with a grid size resolution of 0.25 degrees was used for comparison with station data. Further details are provided in References [1,46].

3.2. Methods

In-situ data represents point-scale observations, which are not truly representative of the area averaged precipitation of the GPPs; thus, the interpretation of a direct comparison should consider such a problem. To overcome this, numerous studies have limited the comparisons to the pixels where an appropriate number of stations were present [12,47], with a minimum of one to more than five stations in each pixel, according to the purpose of the study. In this study, we used minimum of one station to maximum of four in-situ stations in one degree-pixel. Furthermore, for GPPs, the nearest GPP precipitation value to the location of each station were retrieved for comparison (Figure 1b). The density of in-situ stations is not uniform and thus remapping the GPPs to a common grid size or area averaged precipitation for both in-situ and GPPs might introduce more noise into the analysis. Moreover, the in-situ data in the study region and usually in most of the developing countries is mostly recorded through traditional rain gauges, which does not accurately measure trace precipitation events (<0.25 mm/day) which were thoroughly checked for and treated.

Four locations were selected for a regional scale comparison as shown in Figure 1b. These regions were selected based on variations in climate, elevation and land cover (

Table 1. Description of the regional elevation, climate class and land cover of the four regions selected for comparison of GPPs and in-situ observations as shown in Figure 1b.

Region	Elevation (m)	Climate	Land Cover
1	3001–6000	Semi-arid	Grassland
2	0–600	Humid	Cropland
3	601–1600	Arid	Shrubland
4	0–600	Extremely Arid	Cropland

), following the classification of [48] for the study region. The climate descriptions in Table 1 correspond to the climate zones ’Wet Semi-Arid,’ ’Sub Humid,’ ’Arid’ and ’Extremely Arid’ as described in Reference [48].The dominant features of the selected regions such as, the land cover, climate and precipitation pattern typically remain the same. The main seasons in the study region were classified as winter (December, January, February and March), pre—monsoon (April, May and June), monsoon (July, August and September) and autumn (October and November) following [35,37]. For regional comparison, the precipitation products are compared at daily, monthly, annual and different seasons at interannual time scales. For a daily scale comparison, frequency versus intensity plots and Empirical Cumulative Distribution Function (ECDF) are used for comparison of in-situ and GPPs. Both in-situ and GPPs estimate precipitation with different procedures thus variability at daily scale is high and the information needed are obtained with using the above mentioned methods at daily scale thus scatterplot is not used for daily scale comparison. For frequency versus intensity plots, the regional precipitation observations were arranged in ascending order and grouped into bins of 1.85 mm per day following [49,50]. The grouped precipitation observations were then normalized and plotted against station observations, where the frequency count (precipitating days) was placed on the y-axis and magnitude is shown on the x-axis. For daily scale ECDF, the cumulative probability derived for regional precipitation from in-situ and GPPs were compared using extreme precipitation thresholds of 2.5, 5 and 10 mm/day following [47]. To compare the similarity of precipitation between in-situ data and GPPs, a nonparametric Kolmogorov-Smirnov (K-S) significance test with 95% confidence level was applied, assuming that both in-situ data and GPPs have similar distributions.

For monthly, annual, seasonal and interannual scale comparison at regional scales, the scatterplots were constructed against in-situ precipitation as independent and GPPs as dependent variables. The scatterplots express linear relationship between two or more than two variables. A best fit line (regression line) can easily infer the linear relationship between independent and dependent variable, whereas 1:1 fitting line can further simplify the nature of relationship as over- or underestimation of dependent variables to aid in the interpretation of results. The station scale comparison further elaborated the seasonal variation of correlation (Equation (1)) and RMSD (Equation (3)) at interannual scales.

For station-scale comparison, the GPPs were compared with in-situ precipitation data at the locations of each station, for this purpose the nearest pixel to the station was derived and compared. For descriptive statistics, commonly used error metrics were applied: correlation (R), bias, root mean-square difference (RMSD) and the ratio of standard deviation (STD). These statistical metrics were determined as follows:

$$R=\frac{{{\displaystyle \sum }}_{i=1}^n(Statio{n}_{\left(i\right)}-\overline{Statio{n}_{\left(i\right)})} (Gridde{d}_{\left(i\right)}-\overline{Gridded\left(i\right))}}{\sqrt{{{\displaystyle \sum }}_{i=1}^n(Statio{n}_{\left(i\right)}-{\overline{Statio{n}_{\left(i\right)})}}^2{{\displaystyle \sum }}_{i=1}^n(Gridde{d}_{\left(i\right)}-{\overline{Gridded\left(i\right))}}^2}}$$

(1)

$$Bias=\overline{(Statio{n}_{\left(l\right)}}-\overline{Gridde{d}_{\left(l\right)})}$$

(2)

$$RMSD=\sqrt{\frac{1}{N}{{\displaystyle \sum }}_{i=1}^N{\left(Gridde{d}_{\left(i\right)}-Statio{n}_{\left(i\right)}\right)}^2}$$

(3)

$$Ratio of Standard Deviation \left(STD\right)=\frac{{\sigma }_{gridded}}{{\sigma }_{Station}}$$

(4)

In the above equations, ‘gridded’ and ‘station’ represent the precipitation products from the different GPPs and in-situ data, respectively, while ‘n’ is the number of observations. The correlation (R) (Equation (1)) was used to quantify the scale of linear relationship between station and GPPs. Bias (Equation (2)) was used to show the average direction of the centered deviation of GPPs from in-situ observations; thus, a positive value implied that the GPPs underestimated the in-situ precipitation and a negative value implies an overestimation of precipitation. RMSD (Equation (3)) was used to estimate the mean magnitude of the differences between the GPPs and in-situ precipitation products, while STD (Equation (4)) was used to assess the relative variation and differences in the amplitude of GPPs and in-situ observed precipitation distribution.

4. Results

4.1. Climatology

Figure 2 shows the seasonal mean precipitation climatology of different seasons for the GPPs employed over the 1998–2016 study period. We see that across all GPPS, the northwestern part of the country exhibited more precipitation than the southeast, with mean values being observed at >50 mm in the winter. From this maximum, all GPPs displayed a pattern of precipitation gradually decreasing towards the east and south. Likewise, during the pre-monsoon season, precipitation was also relatively consistent in all GPPs with relatively higher precipitation in the northern parts of the country (80 mm), while the central and southern regions received approximately 40 mm and <20 mm, respectively. A similar decrease in precipitation from north (maxima) to south (minima) was observed.

In all GPPs, a similar monsoonal precipitation pattern was obvious with relatively large amounts of precipitation in the northeast and southeast of the study region. A precipitation maximum of >120 mm was recorded in the north, while a minimum of ≥ 80 mm occurred in southern coastal regions. During autumn, the precipitation distribution across the whole country ranged from <10 mm in the south, to >40 mm in the north. The GPPs showed a consistent precipitation pattern, with minor deviations in the mean observed in the northwest.

Two precipitation patterns are observed in Pakistan, the first is due to disturbances of westerlies in the winter and pre—monsoon season and the South Asian monsoon, which is the primary driver of the hydrological cycle in the region [37,48]. The GPPs clearly showed these two precipitation patterns, with a slight deviation from the mean; however, during the winter and autumn seasons, MSWEP and CHIRPS captured higher precipitation values in the northwest which could be associated with relatively higher grid size resolution of these two GPPs.

Figure 3a–d shows the mean annual precipitation cycle for the four selected regions as shown in Table 1. From Figure 3a it can be seen that precipitation maxima occurred in April (≥50 mm) and the minima occurred in October (<20 mm). With in-situ as reference, all GPPs were able to show the seasonality but overestimated the [monthly average] precipitation (Figure 3a). MSWEP, PERSIANN-CDR and CHIRPS captured larger values than in-situ throughout the seasonal cycle; however, TMPA performed relatively well.

In Figure 3b, we see that the hydrological cycle of region 2 is dependent on westerlies during the winter (>30 mm) and the monsoon (>180 mm) during the warm summer. GPPs closely followed in-situ precipitation pattern, with similar maxima and minima captured for the monsoon and winter seasons. For the winter, the precipitation observed was around 60 mm, GPPs exhibited similar variation in precipitation to that of in-situ data. However, for monsoonal precipitation, there were obvious deviations in magnitude, which are more obvious for PERSIANN-CDR.

In Figure 3c, peak precipitation occurred during winter (February >45 mm) followed by the monsoon season. MSWEP, CHIRPS and TMPA displayed similar seasonal precipitation dynamics to in-situ data, with the peak precipitation occurring in February, with estimated precipitation values of 38, 40 and 50 mm, respectively. PERSIANN-CDR performed well during the winter but during the summer, it overestimated the observed precipitation.

In region 4 (Figure 3d), most of the precipitation occurred during the monsoon season (with August values reaching >70 mm), GPPs captured precipitation dynamics quite well but there were slight deviations in magnitude. MSWEP, CHIRPS and TMPA displayed peak precipitation during the monsoon season, with an abrupt decrease in autumn. PERSIANN-CDR also captured this seasonal cycle but overestimated precipitation.

Finally, from the mean annual precipitation cycle (Figure 3), it can be seen that precipitation seasonality was captured quite well by GPPs, which agreed with in-situ in terms of identifying the maximum and minimum precipitation months. Regional scale deviations and differences, especially in relatively higher precipitation seasons were more prominent in PERSIANN-CDR, followed by MSWEP.

4.2. Comparison on a Daily Scale

For a daily scale comparison, frequency versus intensity histograms were plotted in Figure 4 for the precipitation in each region. In-situ stations (traditional rain gauges) might not capture precipitation events with magnitudes of less than 0.25 mm/day very well, which is generally treated as traces, thus precipitating days considered were ≥0.25 mm per day [49]. Figure 4a–d shows the frequency and relative intensity of in-situ and GPPs at each region, while the arrows indicate 95th percentiles. Figure 4a indicates that in-situ data had a lower frequency of days with relatively low precipitation amounts than the GPPs. Both in-situ and GPPs had a similar frequency of precipitation events of <25 mm and excluding MSWEP, rest of the GPPs showed good agreement at the 95th percentile moreover PERSIANN-CDR does not identify days with precipitation amount > 25 mm/day. Figure 4b shows that both in-situ and the GPPs had an alike frequency and intensity for events with precipitation of <45 mm, although relatively lower magnitude events had a higher frequency in CHIRPS and PERSIANN-CDR than in TMPA and MSWEP.

In Figure 4c, we see that in region 3, GPPs reported a relatively similar pattern to in-situ data in terms of the frequencies of precipitation events of ≤15 mm, while above 15 mm, GPPs appeared to show deviations in magnitude. For precipitation events with an intensity of ≥20 mm, CHIRPS and TMPA captured relatively similar frequencies. In Figure 4d, both in-situ and GPPs captured the highest frequency of the days with low precipitation amount, while PERSIANN-CDR captured more precipitation events with smaller intensities as compared to in-situ and the GPPs. The 95th percentile shows closer agreements with that of in-situ data, while the frequency of high precipitation events was also well captured by TMPA and MSWEP.

From Figure 4 it can be deduced that GPPs generally captured the frequency and intensities of precipitation events quite well. GPPs produced closer distribution of higher percentile with those of in-situ as a reference, however deviations and differences were obvious for MSWEP and PERSIANN-CDR. The capability of GPPs to locate higher percentile precipitation estimates closely resemble in-situ data, especially TMPA and CHIRPS [28,47].

The performance of GPPs and in-situ precipitation products was further assessed based on the ECDFs of daily scale precipitation at three thresholds (2.5, 5 and 10 mm/day). Based on results of a K-S test, the GPPs that significantly (K-S test) matched the in-situ data distribution were identified inside their respective threshold boxes in Figure 5. In region 1, Figure 5a–c, for precipitation events characterized by ≥2.5 mm/day, only TMPA had a significant similarity with in-situ, although observed precipitation was nonetheless overestimated. For precipitation events of ≥5 mm/day, PERSIANN-CDR underestimated the observed precipitation, while CHIRPS significantly represented a similar pattern to that of in-situ. For precipitation events of ≥10 mm/day, all precipitation products except PERSIANN-CDR overestimated in-situ but MSWEP and TMPA were significantly closer to in-situ precipitation.

In Figure 5d–f, we see that in region 2 GPPs underestimated precipitation events of ≥2.5 mm/day. For precipitation events of ≥5 mm/day, all GPPs consistently underestimated in-situ precipitation, while CHIRPS and TMPA had a significant resemblance to in-situ precipitation. For precipitation events of ≥10 mm/day, MSWEP, CHIRPS and TMPA had a significant similarity to the in-situ. In region 3, Figure 5g–i, MSWEP produced a significant ECDF for precipitation events of ≥2.5 mm/day, however, precipitation was overestimated. For precipitation events of ≥5 mm/day CHIRPS and TMPA had a significant similarity to in-situ data. PERSIANN-CDR underestimated in-situ precipitation. The performance of gridded data improved when capturing precipitation events of ≥10 mm/day, for which the ECDFs of CHIRPS, PERSIANN-CDR and TMPA were significant, although they underestimated the in-situ precipitation at daily scale.

From Figure 5j–l, in region 4, MSWEP and TMPA produced significant ECDFs for all precipitation events ranging from ≥2.5 to ≥10 mm/day. For precipitation events of ≥10 mm/day, PERSIANN-CDR showed significant distribution with respect to in-situ. The GPPs consistently underestimated these precipitation events, except MSWEP and TMPA, which followed the precipitation distribution of in-situ very closely.

From Figure 5, we see that on daily scales, the ECDF of GPPs and in-situ precipitation estimates for different thresholds over different regions are subjected to deviations and differences. The similarity of GPPs and in-situ precipitation increases with increase in the magnitude of precipitation, except PERSIANN-CDR. Excluding region 2, we observe that precipitation events (≥2.5 and ≥5 mm/day) of MSWEP, TMPA showed relatively significant similarity with those of in-situ precipitation. The varying degree of similarity can possibly be attributed to different factors which may include the climate, land cover and altitude. A detailed assessment is however required to identify the source of variability. We note that these results are in agreement with [47,51], who also reported similar performances for TMPA, CHIRPS and PERSIANN-CDR products.

4.3. Comparison on a Monthly Scale

The scatter plots (Figure 6) and descriptive statistics of

Table 2. Descriptive statistics of GPPs versus in-situ precipitation at monthly time scale.

Region	Name	Regression Coefficient	RMSD (mm)	R	STD	Bias (mm)
1	MSWEP	1.81	34.56	0.72	0.40	–36.0
	CHIRPS	0.73	21.12	0.56	0.78	–23.0
	PERSIANN-CDR	0.83	24.82	0.55	0.67	–32.0
	TMPA	0.86	20.43	0.65	0.75	−9.0
2	MSWEP	0.73	19.63	0.92	1.27	12.0
	CHIRPS	0.74	15.11	0.95	1.28	9.2
	PERSIANN-CDR	1.18	28.57	0.93	0.79	–31.0
	TMPA	0.96	12.43	0.98	1.01	−3.0
3	MSWEP	0.75	13.01	0.80	1.05	1.8
	CHIRPS	0.58	12.03	0.75	1.28	1.0
	PERSIANN-CDR	0.65	17.63	0.65	0.99	–10.0
	TMPA	0.93	13.91	0.84	0.89	−2.4
4	MSWEP	0.98	11.18	0.96	0.92	−1.0
	CHIRPS	0.76	15.37	0.90	1.17	3.0
	PERSIANN-CDR	1.29	21.66	0.93	0.71	–20.0
	TMPA	1.14	12	0.97	0.85	–3.0

display the comparison of GPPs on monthly scale for each selected regions. In region 1 (Figure 6a), MSWEP substantially overestimated precipitation, with a linear regression coefficient of >1. CHIRPS, PERSIANN-CDR and TMPA overestimated monthly precipitation lower than 50 mm/month but underestimated larger precipitation events of >50 mm/month. TMPA was relatively the best performing product followed by CHIRPS and PERSIANN-CDR. In region 2 (Figure 6b), GPPs had a good linear relationship with in-situ, of which TMPA had the closest relationship followed by PERSIANN-CDR, CHIRPS and MSWEP. In region 3 (Figure 6c), TMPA results were consistent with in-situ, with a regression coefficient of 0.93, MSWEP, CHIRPS and PERSIANN-CDR overestimated precipitation events of <40 mm, while precipitation events of >70 mm were underestimated by all GPPs. At region 4 (Figure 6d), TMPA and PERSIANN-CDR overestimated and CHIRPS underestimated in-situ precipitation.

The descriptive statistics in Table 2, suggest that at monthly time scale TMPA had relatively improved linear regression coefficients among all GPPs. The RMSD values varied from region to region, with a relatively larger range of values at region 1 and more prominent large values for PERSIANN-CDR and MSWEP. STD suggests that the standard deviation of GPPs is relatively smaller at region 1, however a slight increase can be seen at region 2 and PERSIANN-CDR systematically overestimated monthly precipitation. The performance of GPPs is relatively improved on monthly time scale, though deviations in precipitation magnitude do exist which could possibly be attributed to regional scale precipitation variability, climate and land surface features.

4.4. Comparison on an Annual Scale

For annual scale comparison, regional scatter plots and statistics are shown in Figure 7a–d and

Table 3. Descriptive statistics of GPPs versus in-situ precipitation at annual time scale.

Region	Name	Regression Coefficient	RMSD (mm)	R	STD	Bias (mm)
1	MSWEP	1.45	17.38	0.42	0.29	–36.48
	CHIRPS	0.65	5.51	0.55	0.85	–24
	PERSIANN-CDR	0.83	9.41	0.44	0.53	–32.5
	TMPA	0.62	6.73	0.46	0.75	−9.0
2	MSWEP	1.32	10.06	0.83	0.62	12.0
	CHIRPS	0.67	9.34	0.77	1.15	9.0
	PERSIANN-CDR	0.97	6.21	0.87	0.89	–30.0
	TMPA	1.01	3.22	0.96	0.95	−3.0
3	MSWEP	0.68	4.42	0.75	1.11	2.0
	CHIRPS	0.44	2.39	0.81	1.83	1.0
	PERSIANN-CDR	0.59	6.28	0.57	0.97	–10.0
	TMPA	0.78	4.03	0.82	1.04	–2.0
4	MSWEP	0.88	3.2	0.95	1.07	−1.0
	CHIRPS	0.67	2.72	0.94	1.45	3.0
	PERSIANN-CDR	1.24	5.18	0.94	0.75	–21.0
	TMPA	1.05	3.71	0.96	0.96	–3.2

, respectively. At region 1 (Figure 7a), GPPs had shown overall positive linear relationship with regression coefficient >0.65; deviations are obvious and mostly the in-situ precipitation is overestimated. Figure 7b shows that at region 2, TMPA outperformed the other GPPs in terms of linear relationship with in-situ, having a regression coefficient of 1.01. PERSIANN-CDR overestimated observed precipitation, while CHIPRS and MSWEP underestimated in-situ precipitation.

In region 3, GPPs consistently overestimated in-situ precipitation products, as can be seen from Figure 7c, where TMPA outperformed rest of the GPPs. Figure 7d shows that in region 4, MSWEP and TMPA performed relatively well, while CHIRPS underestimated and PERSIANN-CDR overestimated in-situ precipitation.

Table 3 suggest that at annual scale the performance of all GPPs is relatively weaker in region 1, with lower regression values except PERSIANN-CDR, differences and biases in mean precipitation are also obvious for PERSIANN-CDR (RMSD = 9.41 mm, Bias = –32.5 mm) and MSWEP (RMSD = 17.38 mm, Bias = –36.48) respectively. Overall, TMPA relatively outperformed all the other GPPs, followed by CHIRPS and MSWEP. The performance of GPPs on an annual scale suggests an improvement towards semi-arid regions with decrease in mean differences, biases and improved correlation for all GPPS; but a larger number of regions require study before such a trend can be accepted as definitive.

4.5. Comparison on an Interannual Scale for Different Seasons

To assess long term variation, GPPs and in-situ precipitation datasets were further compared on an interannual scale for different seasons. In Figure 8, during winter, MSWEP produced R values of ≤0.20 in the northern parts of the study area, while in the central and southern parts of the study area relatively higher R values of ≥0.40 and ≥0.80 can be seen, respectively. CHIRPS, PERSIANN-CDR and TMPA have shown relatively similar precipitation variations as those of in-situ precipitation. The R values for CHIRPS, PERSIANN-CDR and TMPA were in the range of ≥0.80 in the north and ≥0.60 in central and southern regions, suggesting that these three products had similar dynamics for producing winter precipitation over the different parts of the study area.

During the pre—monsoon season, MSWEP performance in depicting precipitation was similar to its winter performance, with low R values in the north (≤0.60) and high values in central (≥0.60) and southern (≤0.80) regions. CHIRPS and PERSIANN-CDR were less effective at capturing precipitation variability. For TMPA there were also low R values in the north, while in the central and southern regions R values were observed to be ≥0.80 for most stations.

In the monsoon season, MSWEP reached a higher level of agreement ≥ 0.80 in the southern and eastern parts of the study region. The CHIRPS precipitation product clearly indicated a monsoonal precipitation pattern, with large R values of ≥0.60 to 0.80 in the monsoon regions. The interannual monsoon precipitation variability was well captured by PERSIANN-CDR, with high R (0.80) values. For monsoonal precipitation, TMPA outperformed all GPPs with R values of ≥ 0.80 in all parts of the study region under monsoon influence.

In autumn, there was an obvious pattern in the R values, with higher values in the north and south, while central regions had relatively low R values. Both MSWEP and TMPA had similar R values of ≥0.80 in these regions, while CHIRPS and PERSIANN-CDR also produced a similar pattern; however, their R value of ≥0.60 were lower than that of MSWEP and TMPA.

The precipitation seasons of the study regions are determined by the westerly disturbances that are mostly active during the winter/cold season and the pre—monsoon and monsoonal precipitation during warm summer season [14]. There were stronger correlations between the GPPs and in-situ in the cold season, while in the monsoon season MSWEP and TMPA produced higher R values than CHIRPS and PERSIANN-CDR. The strength of the correlation varied with magnitude in precipitation during different seasons, however orographic influence is also obvious in the boundary regions of low lands and high terrain and thus need to be further investigated.

The RMSD values (Figure 9) shows a decreasing tendency from the north of the country towards the south. In winter, MSWEP had the highest RMSD values of >16 to <32 mm in the northwest, which persisted at all stations located in the northern region of the study area. In the central and southern parts of the country, the RMSD was less than 10 mm. CHIRPS and PERSIANN-CDR had slightly higher RMSD values in the northern parts of the region, ranging from 10 mm to < 30 mm among the different stations. In central and southern areas, RMSD values were consistent with those of MSWEP, with a magnitude of around 10 mm. TMPA followed the same pattern of RMSD values as observed for MSWEP, with higher values in northern parts but these values were smaller than those of MSWEP, CHIRPS and PERSIANN-CDR. In the pre—monsoon season, MSWEP had a high RMSD of >20 mm for stations located in northern regions, CHIRPS and PERSIANN—CDR produced the same pattern as MSWEP, with relatively low values in southern parts of the study region, while TMPA produced smaller RMSD values than rest of the GPPs.

In the monsoon season, with peak precipitation, the differences were also higher for all GPPs. MSWEP, CHIRPS and PERSIANN-CDR produced higher differences (>28 mm) in the upper parts, with a decrease toward the southern plain regions. TMPA produced low RMSD values for stations located in both northern (<32 mm) and southern (<16 mm) regions. In autumn, the performance of GPPs improved, MSWEP still had higher RMSD values (>24 mm) for stations located in northern parts of the country.

From Figure 9 it can be seen that the RMSD was higher during peak precipitation seasons, as indicated in the winter, pre—monsoon season and especially the monsoon season. Stations located in different climate regions and at different elevations reported dissimilar results. The higher RMSD values in the north and lower values in central and southwestern regions could be possibly due to differences in land cover, precipitation magnitude, station geography towards precipitation bearing winds and altitude that influence the rain rates [35,37].

Figure 10 shows the linear relationship of westerlies (Figure 10a,b) and the monsoonal (Figure 10c,d) precipitation system, which together are the key drivers of the local hydrological cycle in the country. In the northern region (Figure 10a), MSWEP and PERSIANN-CDR substantially overestimated in-situ precipitation. In the southwest (Figure 10b), gridded products tended to produce a positive linear relationship with respect to in-situ precipitation. MSWEP slightly underestimated precipitation events, while, except for TMPA, the other products overestimated precipitation events.

Figure 10c shows that during the monsoon season, MSWEP underestimated in-situ precipitation, CHIRPS produced a very similar pattern to in-situ, while PERSIANN-CDR overestimated in-situ records. TMPA produced the best results among all GPPs, with a very close linear relationship and thus exhibited similar dynamics as of in-situ.

In terms of a close positive relationship with in-situ data, the performance of all GPPs was best in the southern parts of the study region (Figure 10d). MSWEP was closely related to in-situ precipitation and thus produced a similar pattern, whereas PERSIANN-CDR overestimated precipitation. The descriptive statistics (

Table 4. Descriptive statistics of the interannual precipitation variability in selected regions. Regions 1 and 3 experience pre—monsoon precipitation, while regions 2 and 4 experience monsoonal precipitation.

Region	Name	Regression Coefficient	RMSD (mm)	R	STD	Bias (mm)
1	MSWEP	1.14	26.22	0.46	0.40	–55.0
	CHIRPS	0.44	10.74	0.45	1.06	–18.0
	PERSIANN-CDR	0.73	16.87	0.47	0.63	–35.0
	TMPA	0.42	10.8	0.43	1.01	−8.0
2	MSWEP	0.87	25.47	0.76	0.80	38.0
	CHIRPS	0.66	17.68	0.78	1.18	23.0
	PERSIANN-CDR	0.77	16.91	0.84	1.08	–63.0
	TMPA	0.92	11.53	0.93	1.01	2.1
3	MSWEP	0.67	10.04	0.74	1.10	2.0
	CHIRPS	0.49	6.55	0.78	1.58	–3.0
	PERSIANN-CDR	0.57	10.35	0.67	1.18	–5.0
	TMPA	0.82	9.9	0.81	0.97	–7.0
4	MSWEP	0.95	10.71	0.97	1.01	−3.2
	CHIRPS	0.69	11.56	0.94	1.30	3.2
	PERSIANN-CDR	1.245	14.95	0.97	0.77	–47.0
	TMPA	1.13	10.46	0.98	0.87	–12.0

) further demonstrate the performance of GPPs in both the westerly and monsoon seasons in the study region. For westerlies (regions 1 and 3), variations among the GPPs in depicting the in-situ precipitation pattern are obvious, however, in region 3 the correlation between GPPs and in-situ was improved. The regression coefficients for TMPA in both regions (0.92 and 1.13 in regions 1 and 4 respectively) indicated a close linear relationship with in-situ. Compared to MSWEP, CHIRPS and PERSIANN-CDR, TMPA had lower RMSD values for mean precipitation, a higher temporal agreement, relative amplitude agreement and a lower bias for monsoonal precipitation.

Figure 10 shows that for westerlies and monsoonal precipitation at interannual scale, TMPA produced better results than the other GPPs for both westerlies and monsoonal precipitation, followed by CHIRPS and MSWEP. These findings concur with those of [36,37,38,52] for TMPA, [47,53] for PERSIANN-CDR, [51,54,55] for CHIRPS and [56,57] for MSWEP. MWSEP is a relatively new product; thus, very few studies have reported similar findings in such a diverse range of climate.

5. Discussion

The application of remotely sensed precipitation data is important in assimilation, estimation and forecast and for improving model simulation skill to better understand land-atmosphere interactions in regions where in-situ data is sparse. Comparisons of GPPs with available precipitation records are crucial to determine their strengths and limitations in such regions. The current study used in-situ records to validate GPPs across a range of climate, land cover types and terrain in Pakistan. The in-situ records may also be subject to physical and non-physical differences and efforts were made to reduce their effects. The precipitation patterns for both westerlies and monsoonal precipitation were consistent among all GPPs and agreed with previous results [14,36,37]. The seasonal scale magnitudes of precipitation reported by the GPPs were mostly larger than in-situ in the northern region, while they performed relatively well in the other regions. PERSIANN-CDR consistently overestimated in-situ precipitation, while TMPA, CHIRPS and MSWEP performed adequately. These findings concurred with those of [35,47,54,58], with a possible reason being that infrared based retrievals have limitations in estimating precipitation over complex terrains [35,59,60]. The daily scale performance suggests that GPPs were consistent for daily scale thresholds, with PERSIANN-CDR and MSWEP overestimating in some regions. Daily scale performance is improved for precipitation events of ≥10 mm/day in all regions, which concurred with the results of [47,61].

At monthly time scale, GPPs usually overestimated precipitation events, with lower R values, high RMSD values and differences in mean for different regions. At southern parts of the region relatively good performance was observed in terms of the linear regression with in-situ and improved descriptive statistics. The differences in performance on monthly scale could possibly be due to the complex terrain induced deviations, precipitation magnitude, in-situ stations uncertainties and their low density, which did not accurately represent the precipitation pattern in such regions [18,62]. GPCC precipitation products may also be subjected to uncertainties, which are derived from in-situ stations, thus climatologies developed from limited number of in-situ stations in such regions might not be accurate estimation of the regional precipitation behaviors [53,63]. Calibration and validations of GPPs with GPCC products in such data scarce region might be subjected to noise and deviations. Infrared and microwave based algorithms also have limitations due to terrain and wet and dry regional climates [64,65,66,67]. At monthly time scale, the performance of GPPs is relatively improved from daily time scale, however the deviations in error metrics were also obvious and consistent at regions with more precipitation.

At interannual time scale, the results indicated that the temporal agreement of GPPs varied at both regional and seasonal scale. In the northern regions, the performance of GPPs was unsatisfactory both in terms of temporal agreements and RMSD values, with similar findings also reported by [28,53]. In the southwestern parts of the country, the overall temporal agreement of the GPPs agreed with that of in-situ. The statistical metrics had consistent values in terms of interannual variations and positive linear relationship with in-situ during the high precipitation seasons. The differences in precipitation magnitudes were higher in regions and during seasons with higher precipitation amounts. The RMSD values of stations located in the northern parts of the study area were usually higher than those stations located in central and southern regions [34,35,36,66]. TMPA produced the highest R values and lowest RMSD values, followed by MSWEP and CHIRPS. PERSIANN-CDR displayed similar dynamics regarding interannual scale precipitation variation, although the RMSD values during the peak precipitation season were much higher than for the other GPPs in all regions [47,53]. The differences in PERSIANN-CDR could be linked with GPCP precipitation product with a pixel scale resolution of 2.5 degrees [47,53], which is very coarse as compared to the resolution of PERSIANN-CDR. However, PERSIANN-CDR provides long term precipitation information which results have shown promising results in station rich regions [2,61].

The point source precipitation may not be able to capture exact complex behavior of precipitation variability with simple error metrics. The regional precipitation variation is subjected to latitudinal increase and after certain degrees of elevation it started decreasing while longitudinal increase/decrease is subjected to summer and winter seasons. The two precipitation patterns produce more precipitation in the southeast and southwest during monsoon and winter seasons respectively [14,36]. The differences reported in statistical metrics could possibly be associated with rain shadow regions, seasonal wind patterns and location of in-situ stations in the lee sides of the mountains and valleys. Thus, when making any comparison at regional scale about the performance of GPPs, it might be prudent to keep in mind the regional distribution of precipitation pattern, wind and topography of the region.

6. Conclusions

Gridded satellite precipitation products (GPPs), each with individual strengths and differences, were compared on daily, monthly, seasonal and interannual scales. The main conclusions of the study are listed as follows.

The GPPs captured a consistent climatological precipitation distribution over the diverse topography of Pakistan. The multiyear monthly mean precipitation had the same pattern as in-situ precipitation although differences are obvious at regional scale. With in-situ precipitation as a reference, we see that GPPs are able to capture extreme precipitation events (95th percentile) quite consistently, where TMPA and MSWEP were found to be most similar to in-situ, followed by CHIRPS. The performance of GPPs improves with increased (>5 mm/day) thresholds. The Correlation/RMSD values were higher during peak precipitation seasons.

Based on the performance of the GPPS, they can be used (with proper adjustments) to fill in the gaps between gauges for application to flood forecasting, droughts, land atmosphere interactions studies, extreme events, agricultural planning and many others [34,35,37]. The source of the deviations need to be further explored, with detailed studies of the possible reasons for these differences in both GPPs and in-situ data.