Evaluation of Gridded Precipitation Datasets in Malaysia

Abstract

This study compares five readily available gridded precipitation satellite products namely: Climate Hazards Group Infrared Precipitation with Station Data (CHIRPS) at 0.05° and 0.25° resolution, Tropical Rainfall Measuring Mission Multi-Satellite Precipitation Analysis (TMPA 3B42v7) and Princeton Global Forcings (PGFv3), both at 0.25°, and Global Satellite Mapping of Precipitation Reanalysis (GSMaP_RNL) at 0.1°, and evaluates their quality and reliability against 41 rain gauge stations in Malaysia. The evaluation was based on three numerical statistical scores (r, Root Mean Squared Error (RMSE) and Bias) and three categorical scores (Probability of Detection (POD), False Alarm Ratio (FAR) and Critical Success Index (CSI)) at temporal resolutions of daily, monthly and seasonal. The results showed that TMPA 3B42v7, PGFv3, CHIRPS25 and CHIRPS05 slightly overestimated the rain gauge data, while the GSMaP_RNL underestimated the value with the largest bias for monthly data. The CHIRPS25 showed the best POD score, while TMPA 3B42v7 scored highest for FAR and CSI. Overall, TMPA 3B42v7 was found to be the best-performing dataset, while PGFv3 registered the worst performance for both for numerical (monthly) and categorical (daily) scores. All products captured the intensity of heavy rainfall (20–50 mm/day) rather well, but tended to underestimate the intensity for categories of no or little rain (rain <1 mm/day) and extremely heavy rain (rain >50 mm/day). In addition, overestimation occurred for low moderate (2–5 mm/day) to low heavy rain and (10–20 mm/day). In the case study of the extreme flooding event of 2006/2007 in the southern area of Peninsular Malaysia, TMPA 3B42v7 and GSMaP_RNL performed well in capturing most heavy rainfall events but tended to overestimate light rainfalls, consistent with their performance for the occurrence intensity of rainfall at different intensity level.

1. Introduction

Generally, in most countries, especially the least developed and developing ones, the number of meteorological stations is limited and sparsely distributed. In mountainous and inaccessible areas, there is likely no meteorological station available. Hence, this can be a problem for various applications including validation of weather and climate simulations [1]. Most applications resort to gridded data derived from satellite products. Since the 1970s, due to their wide spatial and long temporal coverage, satellite-gridded precipitation datasets have been widely used in climate studies [2,3]. However, the readily available satellite-based precipitation datasets can have differences in quality, reliability, resolution and spatial-temporal coverage. The quality and reliability of these products can also be influenced by the onboard sensors and algorithms used for rainfall conversion [4]. For satellite and rain gauge blended datasets, the quality can also be dependent on rain gauge observations and interpolation techniques used to produce the gridded products. Hence, the pre-evaluation of these products prior to application, particularly for climate model evaluation, is necessary [5,6].

In Malaysia, despite the availability of various satellite or blended gridded-precipitation data, evaluation studies thus far are incomplete and limited to a few products. Six products have been evaluated including the Tropical Rainfall Measuring Mission (TRMM) [4,7], the Global Precipitation Measurement (GPM) [2], the Global Precipitation Climatology Project (GPCP) [8], the Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Network (PERSIANN) [9], the Climate Prediction Center Morphing Technique (CMORPH) [10] and the Asian Precipitation-Highly Resolved Observational Data Integration Towards Evaluation (APHRODITE) [11]. Among these products, TRMM has been shown to have the best quality where it has the smallest biases and highest correlation when compared with rain gauge observation [12,13,14]. Other products such APHRODITE, CMORPH and GPM have also been considered as good product in some studies [14,15]. However, for extreme rainfalls Soo et al. [16] found that at basin scale in northeast Peninsular Malaysia, both TRMM and CMORPH showed better quality. In addition, Tan et al. [15] also showed good performance of TRMM and GPM in approximating extreme rainfalls. Nevertheless, because of its finer resolution (0.1°) and data availability in 30 minutes, GPM could be a better choice for investigation of extreme rainfall during flood events in Malaysia. Overall, the findings of these studies suggested that TRMM and GPM were considered as the most reliable and accurate gridded precipitation data over Malaysia. However, due to its long temporal coverage, TRMM has the advantage over GPM for most applications especially in climate model evaluation.

The reliability of a dataset is also dependent on the temporal resolutions and seasons. Studies found that most products performed well in approximating the annual and monthly rainfall, but less well in the daily values [14,16]. In addition, the performance was also dependent on the amount of rainfall and tended to be better during seasons with high rainfall. In Malaysia, rainfall amount tends to peak during the winter monsoon (locally known as northeast monsoon) although spatial variation is also featured [17]. Hence, the performance of the products can be dependent on location, where better performance is indicated over the eastern and southern parts of Peninsular Malaysia during the northeast monsoon, because of the higher amount of rainfall received in these areas [12,14].

A number of other products including the Climate Hazards Group Infrared with station data (CHIRPS) [18], Princeton Global Forcings (PGF) [19] and Global Satellite Mapping of Precipitation (GSMaP) [20] have yet to be evaluated for their quality and reliability over Malaysia. In many places around the world these data products have been investigated thoroughly. CHIRPS, which has the highest spatial resolution of 5 km, has been evaluated in various countries including Ethiopia [21,22], China [23], Brazil [24], Venezuela [25], Italy [26], Vietnam [27] and Indonesia [28]. CHIRPS exhibited a good performance in southern China during the warm season [23], and the performance was less affected by elevation variation at the Upper Blue Nile Basin in Ethiopia [22]. CHIRPS also approximated well the rain gauge data and its spatial pattern in the south-southeast region of Minas Gerais State, Brazil, although TRMM was of relatively better quality and reliability [24]. CHIRPS has also been used for extreme indices including drought and flood monitoring in the Lower Mekong Basin [27], South Sulawesi, Indonesia [28] and the Haihe River Basin, China [29]. Both the GSMaP and PGF have also been evaluated in some areas around the globe. In the Haihe River Basin, GSMaP showed good agreement with rain gauge measurements [30]. In the Karpuz River Basin, Turkey, GSMaP was shown to have reasonable performance in approximating the observation although it has a tendency to underestimate extreme rainfall during flash floods [31]. However, a study in Singapore found GSMaP has higher uncertainty than TRMM for sub-daily precipitation [32]. In addition, a study carried out in Adige Basin, Italy found that PGFv3 performance was less reliable compared to CHIRPS and TRMM [26].

This paper focuses on evaluating the quality and reliability of three gridded precipitation products that have yet to be validated over Malaysia namely CHIRPS, GSMaP and PGFv3 by comparing the gridded data to rain gauge observation data. For comparison, TMPA 3B42v7 will also be included. The evaluation analyses consider spatial as well as temporal variations and possible factors that may contribute to the performances of the products.

2. Study Area and Datasets

2.1. Study Area and Its Climatology

Malaysia lies approximately in the center of Southeast Asia, bounded from 1°N to 8°N and 99°E to 120°E. Malaysia consists of two parts: Peninsular Malaysia, which is part of the Asian landmass protruding southward and located between Thailand in the north and Singapore in the south, and East Malaysia, which is part of northern Borneo consisting of Sabah and Sarawak states (Figure 1). Malaysia’s climate is characterized by a hot and humid tropical climate with a pronounced rainfall seasonality modulated by the Asian-Australian monsoon [17]. The wet season occurs during the northeast monsoon (December to February, DJF) while the dry season prevails during the southwest monsoon (June to August, JJA). In between these two pronounced seasons are the inter-monsoon periods of March–April–May (MAM) and September–October–November (SON) [33]. Influenced by various factors (e.g., topography, synoptic circulations and the Inter-Tropical Convergence Zone (ITCZ) migration), the rainfall distribution among these seasons can have pronounced differences [34]. For example, the annual rainfall cycles of stations on the west coast of Peninsular Malaysia have two maxima during MAM and SON that coincide with inter-monsoon periods [34]. The minimum rainfall during the JJA period and lower rainfall during DJF are associated with the blocking effects of the Barisan mountain over Sumatra and the Titiwangsa mountain over Peninsular Malaysia, respectively. On the east coast of Peninsular Malaysia, rainfall peaks during DJF to coincide with the cold surges and Borneo vortex [35]. Due to multiple interactions of various factors (cold surges, Borneo vortex and Madden-Julian Oscillations), extreme precipitation events that can lead to major floods can occur during DJF on the east coast of Peninsular Malaysia [33]. One example was the flood event that occurred in southern Peninsular Malaysia during the 2006/2007 rainy season, with reported economic losses of around USD 500 million, 16 deaths and 200,000 victims [33]. Equally, the flood that occurred in December 2014 in northeast Peninsular Malaysia was caused by strong cold surges [36]. Over northern Borneo, the annual rainfall cycle is partly influenced by the tails of typhoons passing the Philippines. Over Sarawak and Sabah, far fewer meteorological stations are available compared to those of Peninsular Malaysia; hence, the use of rainfall-gridded products to validate climate models over this region is necessary.

2.2. Rain Gauge Dataset

In this study, observed daily rainfall datasets were obtained from rain gauge stations of the Malaysia Meteorological Department (MMD). A period from 2008 to 2012 was chosen because the number of stations with complete data was maximum. According to the World Meteorological Organization (WMO)’s Guide to Climatological Practices, the period of five years is considered sufficient to carry out this kind of study [37]. This constitutes a total of 41 rain gauge stations of which 28 stations were in Peninsular Malaysia and 13 were in Borneo (Figure 1). The daily rainfall amount represents the 24 h accumulation for a period beginning at 0800 local time [38]. Peninsular Malaysia was sub-divided into five different sub-regions: northern Peninsular Malaysia (NPM), eastern Peninsular Malaysia (EPM), middle Peninsular Malaysia (MPM), western Peninsular Malaysia (WPM) and southern Peninsular Malaysia (SPM). Meanwhile, East Malaysia consists of two sub-regions: northern East Malaysia (NEM) and southern East Malaysia (SEM) (Figure 1) [12,13,14]. The latitude-longitude coordinates and elevations of these stations are provided in

Table A1. Location and elevation of 41 rain gauge stations across Malaysia.

Station ID	Station Name	Latitude (°)	Longitude (°)	Elevation (m)
Northern Peninsular Malaysia (NPM)
41529	Perai	5.35	100.40	1.5
48600	Pulau Langkawi	6.33	99.73	6.4
48601	Bayan Lepas	5.30	100.27	2.8
48602	Butterworth	5.47	100.38	2.8
48603	Alor Setar	6.20	100.40	3.9
48604	Chuping	6.48	100.27	21.7
Eastern Peninsular Malaysia (EPM)
48615	Kota Bharu	6.17	102.28	4.6
48616	Kuala Krai	5.53	102.20	68.3
48617	Gong Kedak	5.80	102.48	6.0
48618	Kuala Terengganu Airport	5.38	103.10	5.2
48619	K. K. Terengganu	5.33	103.13	35.1
48657	Kuantan	3.78	103.22	15.3
Middle Peninsular Malaysia (MPM)
48631	K. Tanah Rata	4.47	101.38	1471.6
48632	Cameron Highlands	4.47	101.37	1545.0
48642	Batu Embun	3.97	102.35	59.5
48649	Muadzam Shah	3.05	103.08	33.3
48653	Temerloh	3.47	102.38	39.1
Western Peninsular Malaysia (WPM)
48620	Sitiawan	4.22	100.70	7.0
48623	Lubok Merbau	4.80	100.90	77.5
48625	Ipoh	4.57	101.10	40.1
48647	Subang	3.12	101.55	16.5
48648	Petaling Jaya	3.10	101.65	60.8
48650	KLIA Sepang	2.73	101.70	16.3
Southern Peninsular Malaysia (SPM)
48665	Malacca	2.27	102.25	8.5
48670	Batu Pahat	1.87	102.98	6.3
48672	Kluang	2.02	103.32	88.1
48674	Mersing	2.45	103.83	43.6
48679	Senai	1.63	103.67	37.8
Northern East Malaysia (NEM)
96471	Kota Kinabalu	5.93	116.05	2.3
96477	Kudat	6.92	116.83	3.5
96481	Tawau	4.30	118.12	17.0
96491	Sandakan	5.90	118.07	10.3
96465	Labuan	5.30	115.25	29.3
Southern East Malaysia (SEM)
96412	Mulu	4.05	114.82	26.2
96413	Kuching	1.48	110.33	21.7
96418	Sri Aman	1.22	111.45	9.6
96421	Sibu	2.25	111.97	30.9
96441	Bintulu	3.20	113.03	23.1
96449	Miri	4.33	113.98	17.0
96450	Limbang	4.80	115.00	3.6
96420	Kapit	2.00	112.92	35.0

. Stations situated within a sub-region are taken as representative of rain gauge observations of that particular sub-region.

2.3. Gridded Rainfall Products

Five gridded products were evaluated in this study including CHIRPS at 0.25° and 0.05°, GSMaP_RNL at 0.1°, PGF version 3 and TRMM Multi-Satellite Precipitation Analysis (TMPA) 3B42v7 at 0.25° (

Table 1. Summary of five gridded precipitation products evaluated in this study including input data and related satellite sensors.

Datasets	Input Data & Sensors	Period	Spatial/Finest Temporal Resolution	Coverage
CHIRPS05	CHIRPS-CHPclim, NOAA/TMPA-IR(CCD), gauge	1981–Present	0.05°/Daily	50°N–50°S
CHIRPS25		1981–Present	0.25°/Daily	50°N–50°S
GSMaP_RNL	TRMM-TMI, Aqua-AMSR-E, DMSP-SSMI, CPC(NOAA)-IR, JRA-55	2000–2014	0.10°/1-Hourly	60°N–60°S
PGFv3	NCEP-NCAR reanalysis, CRU TS2.0, GPCP, TRMM RT, gauge	1948–2016	0.25°/3-Hourly	90°N–90°S
TMPA 3B42v7	TRMM-TCI, TRMM-TMI, DMSP-SSMI, DMSP-SSMIS, ASMR-E, NOAA-AMSU, NOAA/MetOp-A-MHS, IR, Gauge	1998–Present	0.25°/3-Hourly	50°N–50°S

). While the TMPA 3B42v7 has been evaluated previously and considered to be the best product [14], the other three products have yet to be analyzed. Hence, the performance of the three gridded products can be compared with TMPA 3B42v7.

CHIRPS is a dataset of daily, pentadal and monthly precipitation climatology (CHPclim) blended from multiple sources including rain gauge data of various sources, such as the Cold Cloud Duration (CCD)-based precipitation on thermal infrared (IR) data archived in the NOAA National Climate Data Center (NCDC), Version 7 TRMM 3B42 data and the Version 2 atmospheric model rainfall field of the NOAA Climate Forecast System (CFS) (Table 1) [18] (ftp://ftp.chg.ucsb.edu/pub/org/chg/products/CHIRPS-2.0). CHIRPS products are available in 0.05° and 0.25° horizontal resolutions covering a period spanning from 1981 until present. In this study, we denote CHIRPS05 and CHIRPS25 as products at 0.05° and 0.25° resolution, respectively.

GSMaP is a purely satellite-based rainfall dataset derived from the thermal IR and microwave radiometer (TIR-MWR) to produce hourly data products that cover the globe from 60°N to 60°S at 0.1° resolution [39]. The associated sensors include SSMI (Special Sensor Microwave/Imager) by the Defense Meteorological Satellite Program (DSMP), TRMM Microwave Imager (TMI) by TRMM, and Advanced Microwave Scanning Radiometer for EOS (ASMR-E) by Aqua. The IR sensor came from the CPC (Climate Prediction Center) of NOAA (Table 1) [40]. The development of GSMaP data was sponsored by the Core Research for Evolutional Science and Technology (CREST) of the Japan Science and Technology Agency (JST) from 2002–2007. From 2007, GSMaP activities are promoted by the JAXA Precipitation Measuring Mission (PMM) Science Team and the GSMaP products are distributed by the Earth Observation Research Center, Japan Aerospace Exploration Agency. GSMaP comprises of three variants: Rainfall Watch (GSMaP_RNL, GSMaP_MVK, GSMaP_NRT), GSMaP Realtime (GSMaP_NOW) and RIKEN Nowcast (GSMaP_RNC). The GSMaP_RNL uses the Japanese 55-year Reanalysis (JRA-55) data as ancillary data to produce a continuous and homogenous dataset. In this study, the GSMaP_RNL (Reanalysis Ver.), which has the same algorithm as GSMaP_MVK (moving vector with Kalman filter to estimate the rainfall rate) and with availability from March 2000 to February 2014 was used [40] (http://sharaku.eorc.jaxa.jp/GSMaP/index.htm).

PGF is a long-term dataset of meteorological forcings for the land surface hydrology modelling developed by Princeton University. The product is based on the merging of National Centers for Environmental Prediction-National Center for Atmospheric Research (NCEP-NCAR) reanalysis and globally available observation datasets. Although the original spatial resolution of PGF was 1.0° with temporal resolution of 3-hourly, the product was further downscaled to 0.5° and 0.25° resolutions with temporal resolutions of daily and monthly for a period from January 1948 to December 2016. This product provides various meteorological variables including precipitation, air temperature at 2 m above ground, downward long/shortwave radiation at surface, surface pressure, specific humidity and wind speed. The precipitation product was constructed from NCEP reanalysis, TRMM real time data, GPCP, Climatic Research Unit (CRU) and observation-based precipitation datasets (Table 1) through a process of downscaling and bias correction in both spatial and temporal resolution [19,41] (http://hydrology.princeton.edu/data/pgf/). In this study, PGF version 3 of daily precipitation at 0.25° resolution was used.

The TRMM product, introduced in 1997, was from a collaborative mission of United States National Aeronautics and Space Administration (NASA) and the Japan Aerospace Exploration Agency to study tropical and subtropical rainfall for weather and climate research [7,42]. It uses several instruments including Visible Infrared Radiometer (VIRS), TMI, Precipitation Radar (PR), Cloud and Earth Radiant Energy Sensor (CERES) and Lightning Imaging Sensor (LIS). The product datasets were available at 0.25 spatial resolutions and two temporal resolutions, i.e., 3-hourly and 7-day accumulated rainfall. The estimated precipitation is derived from two products, namely 3-hourly combined with microwave-IR estimation (with gauge adjustment) and monthly combined MWR-IR-gauge of about two months after the end of each month (satellite-gauge-gridded precipitation products). The onboard sensors include TRMM Combined Instrument algorithm (2B31) (TCI), TMI, SSMI, Special sensor microwave/Imager-Sounder (SSMIS), ASMR-E, Advanced Microwave Sounding Unit (AMSU) and Microwave Humidity Sounder (MHS) (Table 1) [42]. Different variants of TRMM products of various time resolutions can be found at https://pmm.nasa.gov/data-access/downloads/trmm. In this study, we used the daily precipitation totals derived from 3B42 Research Version using the Version 7 TRMM Multi-Satellite Precipitation Analysis released in June 2012. TMPA 3B42v7 provides precipitation data from January 1998 until present.

3. Methodology

3.1. Statistical Measures for Product Validation

For evaluation of performance of gridded precipitation datasets against the observed rain gauge dataset, we used pairwise comparison numerical statistics and categorical performance [43]. According to Tan et al. [44], the point-to-pixel and pixel-to-pixel comparison essentially produce similar results. Hence, the technique of point-to-pixel comparison was chosen due to insufficient station data for interpolation into gridded data [24,44]. In this technique, the values of gridded data were compared to the values of station data. The use of pairwise comparison based on three numerical statistical indicators, summarized in

Table 2. Statistical measures (G_i, rain gauge measurement; Ḡ_i, mean rain gauge measurement; S_i, gridded rainfall estimate; ${\overline{\mathrm{S}}}_{\mathrm{i}}$, mean gridded rainfall estimate; and n, number of data pairs).

Name	Formula	Perfect Score
Pearson Correlation Coefficient, r	$\frac{\sum \left({\mathrm{G}}_{\mathrm{i}}-{\overline{\mathrm{G}}}_{\mathrm{i}}\right)\left({\mathrm{S}}_{\mathrm{i}}-{\overline{\mathrm{S}}}_{\mathrm{i}}\right)}{\sqrt{\sum {\left({\mathrm{G}}_{\mathrm{i}}-{\overline{\mathrm{G}}}_{\mathrm{i}}\right)}^2}\sqrt{\sum {\left({\mathrm{S}}_{\mathrm{i}}-{\overline{\mathrm{S}}}_{\mathrm{i}}\right)}^2}}$	1
Root Mean Squared Error, RMSE	$\sqrt{\frac{1}{n}\sum {\left({\mathrm{S}}_{\mathrm{i}}-{\mathrm{G}}_{\mathrm{i}}\right)}^2}$	0
Relative Bias, Bias	$\frac{\sum {\mathrm{S}}_{\mathrm{i}}}{\sum {\mathrm{G}}_{\mathrm{i}}}-1$	0

, allows the estimation of accuracy and reliability of gridded products [45]. The Pearson Correlation Coefficient (r) measures the degree of association between the gridded dataset and the rain gauge data. The Root Mean Squared Error (RMSE) calculates the differences between two datasets and provides the average magnitude of error. The bias measures the tendency of the gridded data to overestimate (Bias > 0) or underestimate (Bias < 0) the rain gauge data.

In addition, three categorical measures were used to detect rainfall occurrence and non-occurrence between satellite data and rain gauge data (

Table 3. Categorical performance measures for comparing station and gridded precipitation data.

Name	Formula	Perfect Score
Probability of detection, POD	$\frac{A}{A+C}$	1
False Alarm Ratio, FAR	$\frac{B}{A+B}$	0
Critical Success Index, CSI	$\frac{A}{A+B+C}$	1

) [16]. These measures are based on a contingency table where A is defined as hits (i.e., both gridded and station observations occur), B represents a false alarm (i.e., station does not occur, gridded occurs), C means misses (i.e., station occurs but gridded does not occur), D represents a correct negative (i.e., both station and gridded do not occur) and n is the sum of A, B, C and D. The rain occurrence threshold was set at 1 mm per day to detect a rainy or not rainy day.

The Probability of Detection (POD) calculates the rain occurrence in gridded datasets by ignoring false alarms. On the other hand, False Alarm Ratio (FAR) shows the sensitivity towards the event of rain occurrence in satellite data that does not occur in station data without calculating the rain-missed event. Meanwhile, the Critical Success Index (CSI) shows the correct estimation of the rain occurrence while removing the negative occurrence. The values of POD, FAR, and CSI range from 0 to 1. A POD score of 1 implies perfect agreement of rainfall occurrences in both station and gridded data, whereas a score of zero corresponds to a complete disagreement between the two datasets. On the other hand, FAR measures the false alarm rates where a score of zero indicates no false alarm occurred. Meanwhile, the CSI index provides the measure of a critical success rate that blends POD and FAR, whereas a perfect score of 1 implies zero occurrences in both the false alarm and misses categories. The frequencies of defined rain intensities were also compared between rain gauge data and satellite products. Thresholds for different categories of daily rainfall intensity follow World Meteorological Organization (WMO) [46] definitions: rain <1 mm (no/very light rain), 1–2 mm (light rain), 2–5 mm (low moderate rain), 5–10 mm (high moderate rain), 10–20 mm (low heavy rain), 20–50 mm (high heavy rain) and rain >50 mm (violent rain) [14,45,47].

3.2. Extreme Precipitation Events

To evaluate the appropriateness of the gridded data precipitation during extreme precipitation events, three events were considered as examples: i) 17–20 December 2006, ii) 24–28 December 2006 and iii) 11–14 January 2007, all of which contributed to the 2006/2007 flood event in the southern parts of Peninsular Malaysia [33].

4. Results

4.1. Performances Based on Statistical Measures

Figure 2 shows the 2-D histogram of total monthly data from five precipitation datasets versus rain gauge data during the five-year period. The color bar values indicate counts with the scale shown in the right y-axis. All products show variations when compared with rain-gauge observation as not all points fall on the straight lines. Generally, the larger the scatter implies the greater disagreement. In terms of point scattering TMPA 3B42v7, CHIPRS05 and CHIRPS25 products show similar patterns with correlation values of >0.80. All three products also show comparable peaks of point concentration. However, GSMaP_RNL and PGFv3 products show large scattering indicating high degree of disagreement with the rain gauge observation with correlation values of 0.76 and 0.62, respectively.

The statistical performance indicators for monthly data are summarized in

Table 4. Summary of statistical measures for five monthly gridded precipitation datasets. The best values for each statistical indicator are shown in bold.

Dataset	r	RMSE (mm/mon)	Bias
CHIRPS05	0.83	98.01	0.06
CHIRPS25	0.82	100.70	0.08
GSMaP_RNL	0.76	117.56	−0.11
PGFv3	0.62	138.40	0.03
TMPA 3B42v7	0.87	84.91	0.02

. TMPA 342v7 slightly overestimated the station data with the smallest positive bias of 0.02, followed by PGFv3 (0.03), CHIRPS05 (0.06) and CHIRPS25 (0.08). The GSMaP_RNL was the only gridded precipitation dataset that underestimated the rain gauge with the highest bias magnitude of −0.11. TMPA 3B42v7 also recorded the lowest RMSE, whereas PGFv3 presented the highest RMSE value. Hence, TMPA 3B42v7 performed best for monthly total precipitation followed by the CHIRPS product. The CHIRPS05 performed slightly better than CHIRPS25, in agreement with a study in China [23], but differed from that in Adige Basin in Italy [26].

Figure 3 presents the monthly statistical performance indicators for all products based on seven sub-regions in Malaysia. In NPM, all products show monthly variation with highest values in March, June and July. In EPM, TMPA 3B42v7 and both CHIRPS products perform best during the wet season while performances decrease during dry season. Contrary to other products, the PGFv3 shows a negative correlation and high RMSE values during October, November and December, while GSMaP_RNL shows relatively high correlations, but underestimated the station rainfall by more than 50% in February. The MPM sub-region, which consists of two highlands, shows the highest statistical scores during February, March, April and October, but has low values during September and November. In WPM regions, the gridded precipitation datasets perform best during May–July and in October. Overall, GSMaP_RNL appears to perform the lowest compared to the other products. In the SPM, the gridded products perform the best during wet season. Meanwhile, in NEM, throughout the year, all gridded precipitation products except for GSMaP_RNL display relatively good statistical performance. Over the southern part of East Malaysia, gridded precipitation datasets recorded a better performance measure in January and May–September. Overall, in East Malaysia, the PGFv3 performs comparably with TMPA 3B42v7, while GSMaP_RNL is considered the worst.

Table 5. Seasonal values of statistical measures of CHIRPS05, CHIRPS25, GSMaP_RNL, PGFv3 and TMPA 3B42v7. The best values for each indicator are in bold.

		CHIRPS05	CHIRPS25	GSMaP_RNL	PGFv3	TMPA 3B42v7
DJF	r	0.64	0.62	0.58	0.57	0.72
	RMSE (mm/mon)	93.83	98.17	113.81	119.66	79.87
	Bias	0.18	0.25	−0.09	0.31	0.18
MAM	r	0.58	0.58	0.39	0.59	0.66
	RMSE (mm/mon)	86.21	88.97	109.95	91.81	76.53
	Bias	0.10	0.13	−0.10	0.15	0.09
JJA	r	0.59	0.59	0.49	0.64	0.69
	RMSE (mm/mon)	69.05	70.17	87.77	63.02	62.48
	Bias	0.06	0.08	−0.15	0.08	0.00
SON	r	0.53	0.52	0.52	0.46	0.66
	RMSE (mm/mon)	94.79	97.52	104.53	122.48	83.30
	Bias	0.10	0.13	0.05	0.01	0.06

compares the performance of five gridded products based on a seasonal basis for the whole of Malaysia. Overall, the performance of gridded precipitation appears higher during DJF (wet season) followed by JJA (dry season) and inter-monsoon periods, consistent with that found by Tan and Santo [15]. TMPA 3B42v7 performs the best among the other products for all seasons.

The performance of all gridded products according to each sub-region is shown in Figure 4. Over NPM and SPM, the r and RMSE values of these products are comparable for all products, although the biases can be different. Generally, most performances of all products tend to be lower in the MPM region. In EPM, most products perform well, except PGFv3 which has a high RMSE and underestimated precipitation. However, PGFv3 performs comparably with other products in East Malaysia. The GSMaP_RNL has the lowest performance at WPM and both locations in East Malaysia. Overall, TMPA 3B42v7 performs best in most sub-regions, followed by CHIRPS of both resolutions.

4.2. Categorical Measurement Ability

Table 6. Categorical performance scores of daily precipitations: POD (Probability of Detection), False Alarm Ratio (FAR), and Critical Success Index (CSI). The best values for each result statistical indicator are in bold.

Dataset	POD	FAR	CSI
CHIRPS05	0.78	0.40	0.51
CHIRPS25	0.86	0.42	0.53
GSMaP_RNL	0.78	0.32	0.57
PGFv3	0.72	0.45	0.45
TMPA 3B42v7	0.80	0.31	0.59

depicts the categorical performances of daily precipitation estimates of all gridded data products. Overall, TMPA 3B42v7 recorded its best scores in FAR and CSI, followed by GSMaP_RNL. On the other hand, CHIRPS25 recorded the highest score in POD, but has the lowest score in FAR. Despite having high resolution CHIRPS05 has lower POD compared to CHIRPS25. However, CHIRPS05 recorded lower FAR when compared to CHIRP25, which is consistent with the statistical score in Table 4. Meanwhile, PGFv3 recorded the worst score in almost all categorical performances.

Figure 5 shows the categorical scores for each month in all sub-regions. Consistent with Varikoden et al. [12] and Tan et al. [14], generally, all sub-regions in Peninsular Malaysia and East Malaysia tended to record the best (worst) values of POD, FAR and CSI scores coinciding with maximum (minimum) rainfall. Hence, all scores tend to have strong seasonality. PGFv3 product shows consistently lowest POD and highest FAR, indicating its inferiority. The CSI values also tended to be lower. However, despite identified as the most reliable product in previous analysis, TMPA 3B42v7 does not show the highest POD. It is CHIRPS25 that tended to produce highest POD values. However, in both FAR and CSI, TMPA 3B42v7 shows its superiority. GSMaP_RNL’s scores of POD, FAR and CSI indicate its performances to be somewhere in between TMPA 3B42v7 and CHIRPS.

The overall categorical performance scores for all products are shown in Figure 6. Consistent with the monthly values, TMPA 3B42v7 and GSMaP_RNL show better performances in FAR and CSI, while CHIRPS25 recorded slightly good performance in POD. However, both CHIRPS05 and CHIRPS25 tended to have much higher values in FAR compared to TMPA 3B42v7 and GSMaP_RNL. Hence, these scores suggest that TMPA 3B42v7 is the overall best performing product followed by GSMaP_RNL and CHIRSPS. The PGFv3, on the other hand, appears to have the lowest scores in almost all categories in most sub-regions, and hence can be considered the worst performing product. Furthermore, the POD and CSI (FAR) scores of all products tended to be the highest (lowest) in SEM compared with other sub-regions. Hence, these scores indicated that the products approximated the stations data much better in SEM.

4.3. Evaluation of Frequency on Different Rain Intensity

The frequency (or occurrence percentage) according to rainfall intensity in Malaysia is shown in Figure 7. The TMPA3B42v7 and GSMaP_RNL show small differences throughout all rain intensity categories. However, based on comparison with station data, the performance of all gridded products was considered inferior, with a tendency for underestimation in the no rain category (<1 mm/day), especially CHIRPS25 which underestimated it by as much as 22%. For light rain (1–2 mm/day), the differences are small except for CHIRPS05 and PGFv3, which underestimated the intensity by more than 5%, while in low moderate rain (2–5 mm/day), PGFv3 underestimated the rainfall frequency by more than 5%. From high moderate rain (5–10 mm/day) to low heavy rain (10–20 mm/day), all products tended to overestimate the frequency, especially CHIRPS05, CHIRPS25 and PGFv3. Noticeably, the PGFv3 shows different performances in different rainfall categories where it tended to underestimate the frequency of low moderate rain (5.24%), but overestimated the moderate rain (9.56%) and low heavy rain (20.76%). Interestingly, for high heavy rain (20–50 mm/day), all products appear to have reasonably estimated the frequency correctly. Meanwhile, for the frequency of rainfall above 50 mm per day, all products tended to underestimate, although the differences can be minimal.

4.4. Evaluation During the 2006/2007 Flood Event

The 2006/2007 flood event in the southern part of Peninsular Malaysia produced three episodes of extreme precipitation events [33]. The stations affected by the heavy rainfall events included Senai, Kluang, Batu Pahat, Mersing, Muadzam Shah and Kuantan, mostly situated in SPM and also in MPM and EPM (indicated with blue triangles in Figure 1). The time series of daily precipitation of affected areas for all gridded products are presented together with station data in Figure 8. Both TMPA 3B42v7 and GSMaP_RNL showed good agreement with rain gauge data in Senai, Kluang and Mersing, but indicated weaknesses in other stations for overestimating light rain and unable to completely capture all episodes of extreme rainfall. Generally, CHIRPS and PGFv3 can be considered having poor performances in detecting extreme events since their maximum intensity of heavy rainfall were lower compared to observations. The performance measures of gridded products during the three extreme events for the six rain gauge stations are given in

Table 7. Performance measures of CHIRPS05, CHIRPS25, GSMaP_RNL, PGFv3 and TMPA 3B42v7 in affected stations during flood event 2006/2007. The best values are in bold.

STATION ID/NAME		CHIRPS05	CHIRPS25	GSMaP_RNL	PGFv3	TMPA 3B42v7
48679/Senai	r	0.71	0.68	0.67	0.40	0.86
	RMSE (mm/mon)	91.33	91.48	68.64	101.78	47.66
	Bias	−0.48	−0.47	−0.13	−0.61	−0.01
48672/Kluang	r	0.53	0.59	0.86	0.38	0.93
	RMSE (mm/mon)	86.89	83.42	43.93	95.45	38.66
	Bias	−0.49	−0.44	−0.13	−0.68	0.04
48670/Batu Pahat	r	0.78	0.80	0.91	0.34	0.95
	RMSE (mm/mon)	33.90	33.83	38.44	42.58	29.59
	Bias	−0.15	−0.18	0.75	−0.42	0.56
48674/Mersing	r	0.74	0.81	0.84	0.05	0.71
	RMSE (mm/mon)	63.02	62.96	65.40	90.74	61.60
	Bias	−0.36	-0.38	−0.57	−0.62	−0.22
48649/Muadzam Shah	r	0.61	0.69	0.81	0.43	0.87
	RMSE (mm/mon)	59.38	57.55	47.68	73.36	34.55
	Bias	−0.29	−0.29	−0.37	−0.66	−0.15
48657/Kuantan	r	0.66	0.80	0.89	0.87	0.90
	RMSE (mm/mon)	45.53	38.22	44.49	40.63	26.79
	Bias	−0.16	−0.05	−0.62	−0.51	−0.20

. The ability of gridded products to detect extreme events varies according to stations. In general, the correction values are high, but both RMSE and the bias are higher as well. Generally, both TMPA 3B42v7 and GSMaP_RNL have good scores in some stations, while other products tended to underestimate, consistent with the findings of Soo et al. [16].

5. Discussion

This analysis explored the reliability and quality of five readily available gridded precipitation datasets compared to observed rain gauge data. Consistent with Tan et al. [14,15] the result indicated that the TMPA 3B42v7 is the overall best performing gridded precipitation dataset in Malaysia followed by CHIRPS, GSMaP_RNL and PGFv3. These relative performances could be due various factors related to input data, onboard sensors, algorithm, and interpolation techniques (Table 1). However, it is difficult to attribute and quantify the effects of each factor. The blending of rain gauge data does not guarantee the highest quality of the products. For examples, both TMPA 3B42v7 and CHIRPS incorporated rain gauges (although number of stations used may be different) but the former has been consistently proven to be the best gridded precipitation data in this study and previous investigations [12,13,14,16]. Hence, other factors such as onboard sensors and algorithm could be more important. However, the performance could vary spatially due the characteristics of the sub-regions, especially in terms of elevation and topography [1,48]. In East Malaysia, PGFv3 appears to be a better product than GSMaP_RNL.

The results of the analyses could be least influenced by the number of rain gauges in each sub-region. In particular, with roughly equal number of stations: NPM–6 stations, EPM–6 stations, MPM–5 stations, WPM–6 stations, SPM–5 stations, NEM–5 stations and SEM–8 stations, the total number of stations may not be the dominant factor in determining the performance of gridded products. However, the MPM is the only sub-region with two rain gauges located in mountainous locations: K. Tanah Rata and Cameron Highlands (indicated with green triangles in Figure 1). The results showed that the gridded product performed the worst in MPM, possibly due to the existence of these two mountainous rain gauges. Figure 9 depicts the scatter plot of five locations in MPM for TMPA 3B42v7, which the two stations in the mountainous region showed lowest correlation. This relatively poor performance could be due the shortcoming in the satellite algorithm when applied in the highly complex terrain of mountainous areas [49]. In addition, in mountainous regions, radar signals that hit the surface could return false echoes [50].

Higher spatial-temporal gridded precipitation data was suggested to be of better quality compared to the lower-resolution product [4]. However, in this study, the higher resolution of the CHIRPS05 product does not improve much over TMPA 3B42v7. This could be a disadvantage of using the CCD technique which may classify some clouds from satellite signals as cold clouds, when in fact most of them are high clouds with no rainfall activity [51]. In addition, perhaps remote sensing algorithms used in gridded precipitation datasets play an important role in its performance.

The accuracy of gridded precipitation datasets is also influenced by the seasonality of rainfall in each sub-region (Table 5). The ability of gridded precipitation datasets in approximating the observed rain gauge data is better during the wet season than the dry season, probably due to the capability of the satellite sensor to detect a convective precipitation system [15]. However, the PGFv3 does not seem to have the same characteristic in EPM for the period from October through December where the rainfall is considered high (Figure 3). The scatter plot in Figure 10 shows that PGFv3 was unable to correlate with station data in November, unlike the TMPA 3B42v7 product in Figure 11. This could be contributed by the downscaling application of reanalysis of global datasets [52].

Interpolation techniques within each product may also influence the quality. For example, some products may have better characteristics of precipitation gradient at the edge of the mountains compared with others. Figure 12 shows the spatial maps of annual rainfall of all products. It appears that the spatial rainfall distribution of TMPA 3B42v7 correlates well with topography with indication of precipitation gradient, both in Peninsular Malaysia and East Malaysia. While similar features are also indicated in CHIRPS and GSMaP_RNL in Peninsular Malaysia, much weak gradient can be seen in PGFv3. Nevertheless, the annual rainfall of GSMaP_RL in WPM and MPM is lower compared with TMPA 3B42v7. In SEM of East Malaysia, the precipitation gradient of GSMaP_RNL appears to be reversed to that of TMPA 3B42v7. In fact, the area of maximum rainfall of TMPA 3B42v7 in SEM was not replicated in GSMaP_RNL. In addition, similar to WPM and MPM, GSMaP_RNL underestimated TMPA 3B42v7 rainfall in northeastern SEM and southern NEM. These differences may be caused by the input data in the products. Using TMPA 3B42v7 as a reference dataset, a Taylor Diagram can be used to provide measures of similarity (in terms of both spatial RMSE and correlation values) among other products (Figure 13). In terms of correlations, both CHRIPS products have the highest values (0.9) followed by PGFv3 (0.84) and GSMaP (0.73). However, based on RMSE, GSMaP_RNL shows the smallest value although GSMaP_RNL tended to underestimate the rain gauge data (Table 4) as well as TMPA 3B42v7.

6. Conclusions

In this study, evaluation of the performance and reliability of five gridded datasets, namely CHIRPS05, CHIRPS25, PGFv3, GSMaP_RNL and TMPA 3B42v7, in approximating rain gauge station observations were conducted. The comparison was assessed for categorical score on daily data and statistical measures for monthly and seasonal timescales from 2008 to 2012. The findings of this study can be summarized as follows:

• Overall, TMPA 3B42v7 was considered the best product for daily, monthly and seasonal precipitation comparison with rain gauge data based on both the categorical and statistical scores. CHIRPS05 performed slightly better than CHIRPS25 in monthly data, while CHIRPS25 was better at daily precipitation. The GSMaP_RNL had slightly better daily performance compared to TMPA 3B42v7, despite its tendency to underestimate monthly rainfall. Meanwhile, the PGFv3 had the lowest performance in most temporal assessment.
• Overall, the wet season (DJF) had higher correlations than the dry season (JJA). However, the dry season recorded lower RMSE and has a better bias than the wet season. Therefore, gridded precipitation performs better over sub-regions that receive higher annual precipitation such as NPM and EPM, as well as East Malaysia. Meanwhile, gridded products perform the worst at MPM due to its highly complex terrain. The TMPA 3B42v7 performed consistently well in all regions. The PGFv3 performed rather poorly over EPM but showed high correlation in East Malaysia regions. Meanwhile, GSMaP_RNL performed worst with high underestimation in most regions.
• For rain detection ability, CHIRPS25 recorded highest Probability of Detection (POD) value followed by TMPA 3B42v7 that recorded low False Alarm Ratio (FAR) and high Critical Success Index (CSI). Meanwhile, GSMaP_RNL showed reasonably good FAR and CSI scores behind TMPA 3B42v7 as well as fair score in POD. PGFv3 showed the poorest score in rain detection ability.
• Most of the gridded precipitation datasets had a tendency to underestimate no rain (< 1mm/day). For light rain (1–2 mm/day), different gridded datasets show different frequencies with CHIRPS05, and PGFv3 had more than 5% underestimation. In general, all products tended to overestimate from low moderate rain to low heavy rain (2–5 mm/day; 20–50 mm/day). However, all products performed reasonably well for high heavy rain (20–50 mm/day) and slight underestimation for rain above 50 mm/day. Interestingly, the PGFv3 recorded underestimation by more than 5% for 1–2 mm/day and 2–5 mm/day, while having overestimation by 9.5% for 5–10 mm/day and 21% for 10–20 mm/day rainfall categories. Overall, TMPA 3B42v7 and GSMaP_RNL were able to detect extreme events, while other products tended to underestimate.
• Based on overall performance, TMPA 3B42v7 can be considered as the most appropriate gridded product for climate and meteorological study in Malaysia.