Evaluation of Sixteen Gridded Precipitation Datasets over the Caribbean Region Using Gauge Observations

Abstract

The existence of several gridded precipitation products (GPP) has facilitated studies related to climate change, climate modeling, as well as a better understanding of the physical processes underpinning this key variable. Due to complexities in estimating rainfall, gridded datasets exhibit different levels of accuracy across regions, even when they are developed at relatively high resolution or using sophisticated procedures. The performance of 16 GPP are evaluated over the Caribbean region, which includes the Caribbean Islands, and portions of Central South America. Monthly data for sixty weather stations are used as a reference for the period 1983–2010. The 16 GPP include six products based on station data only, two that combine ground station and satellite information, two merging station and reanalysis information, four based on reanalysis, and two using multisource information. The temporal resolution of the GPP ranged between daily and monthly and spatial resolution from 0.033° to 0.5°. The methodological approach employed combined a comparison of regional and sub-regional precipitation annual cycles, the Kling–Gupta efficiency (KGE) index, as well as several metrics derived from the standardized precipitation index (SPI). Overall, the best performances were obtained from GPCC025 and MSWEP2, likely reflecting the positive impact of the large number of station data utilized in their development. It is also demonstrated that a higher spatial resolution does not always mean better accuracy. There is a need for this kind of assessment when undertaking climate studies in regions like the Caribbean where resolution is a significant consideration. ERA5 performed best among the reanalyses analyzed and has the potential to be used to develop regionally based GPP by applying bias correction or downscaling techniques. The methodological approach employed provides a comprehensive and robust evaluation of the relative strengths and weaknesses of GPP in the Caribbean region.

1. Introduction

Precipitation is one of the most important climatic variables and it is included in a large number of studies of weather and climate. Precipitation datasets in the form of a regular grid (herein, gridded precipitation products (GPP)) have particular importance for various applications related to climate change detection [1], climate model evaluation [2], and the parameterization of stochastic generators [3,4]. GPP are also useful for understanding the interactions between climate and terrain, as well as hydrological and biogeochemical processes [5,6,7].

The adequate representation of the spatial and temporal variations of precipitation is difficult, mainly because it is significantly affected by the availability and quality of observed data. Although the rain gauges are the most suitable ways to estimate precipitation [8], stations are often sparsely located, and present errors that arise from instrumental and non-instrumental problems (e.g., wind influences) which limit the quality of the observed data [9,10]. Also, despite the notable advances in estimating precipitation from satellite data, this option is limited by temporal sampling and algorithm errors [11]. Additionally, the spatial distribution of precipitation is also affected by complex topography [12].

In recent years, many GPP have been developed, covering different periods, and temporal and spatial scales. These datasets were created, not only using ground-based measurements [13,14] or satellite data [15] but also by combining different sources of precipitation data [16,17,18]. The accuracy of those GPP has been assessed at different spatial and temporal scales using disparate methodologies, showing the advantages and limitations of each product. Most of these assessments were conducted on global or regional scales [18,19], such as in Africa [20,21], Europe [22], Asia [23], and North America [24]. In Latin America, most evaluation studies are concentrated in portions of South America [25,26,27]. In general, these studies suggest different pictures of the best GPP datasets at global and/or regional scales and reinforce the need to evaluate the accuracy of a particular GPP in a region. This problem is especially critical for insular regions such as the Caribbean Islands.

For the Caribbean region, where small scattered countries are dominant and a sparse land-based observational precipitation network exists, there are very few studies evaluating the accuracy of GPP. An intercomparison study using several GPP, including various models from the Coupled Model Intercomparison Project phase 3 (CMIP3) [28], highlights differences in representing the spatial pattern of rainfall in the eastern half of the Caribbean Islands. Ref. [28] concluded that no single GPP was without its discrepancy and that the high-resolution satellite products examined were reasonably representative notwithstanding their short time span (five years at the time of the study). Many of the GPP examined in that study have since been updated to incorporate more observations, improved interpolation techniques, and/or a finer resolution, suggesting the need for re-evaluation. Other studies have compared different GPP when evaluating the spatial and temporal variability of rainfall across the Caribbean [29] and the performance of climate models [30,31]. However, these studies start with the premise that the GPP offer either the best available or the most reasonable representation of the regional climate and thereafter use the GPP as the reference for analysis being undertaken. Recently, ref. [32] produced a very high-resolution monthly dataset (0.033° × 0.033°) that, along with other climate variables, includes precipitation. The authors prove a better accuracy of their downscaled precipitation product, as compared with the original low-resolution (1° × 1°) Global Precipitation Climatology Centre (GPCC) version 7 [14].

As suggested by Jones et al. (2016) [29], many studies have investigated the dynamical drivers of precipitation variability in the Caribbean, aiming to improve the skill of weather and climate forecasting. Most of these studies used different GPP without a comprehensive evaluation of their abilities to represent the main features of precipitation in the Caribbean, and/or conclude that their results were limited to the large-scale because the datasets employed do not have a horizontal resolution appropriate for the Caribbean islands. An accurate description of the temporal and spatial variations of precipitation in the Caribbean requires products that are capable of reproducing regional features at the appropriate spatial resolution. Theoretically, higher resolution GPP could best fulfill this requirement, as compared with lower resolution ones. However, since higher resolution does not necessarily mean a better representation of the observed climate features, it is critical to always evaluate the GPP performance.

This work aims to: (1) assess the ability of 16 GPP in representing the characteristics of precipitation in the Caribbean as compared to surface stations, and (2) determine which products perform best over the region using common statistical metrics. To do so, we conduct a comprehensive evaluation and intercomparison of several GPP across the Caribbean, including the latest versions of some previously evaluated for the region and others that have not yet been evaluated for the Caribbean. Results from this work might be used as a reference to select the most appropriate GPP (e.g., for model evaluation and climate change studies), based on their advantages and limitations. This paper is organized as follows. Section 2 presents the study domain, the data, and methods used in this work. In Section 3, we describe the results, while Section 4 provides the conclusions of the study.

2. Data and Methods

2.1. Study Area

As in [29,33,34], our study domain is located between 10–24° N, and 90–59° W, which includes the Caribbean Islands, and portions of Central and South America (Figure 1). Precipitation seasonality in the Caribbean generally has two main seasons: a dry season from November to April, and a wet season from May to October [33]. An exception is the Eastern Caribbean, where the rainy season extends until November. An important feature of the rainfall annual cycle is a relatively drier period that occurs within the wet season (more commonly in July), called the mid-summer drought (MSD) [35], which is more prominent in the Western Caribbean than in the Eastern Caribbean. Because of the differences in precipitation between the Western and Eastern Caribbean, the study area was subdivided into two sub-regions (see Figure 1b,c).

2.2. Datasets

We use monthly precipitation data from 60 rain-gauges spatially distributed across the study area (Figure 1 and

Table A1. Rain-gauges used in the study. The stations are organized from the western to the eastern Caribbean.

No.	Station Name	Country	Latitude	Longitude	No.	Station Name	Country	Latitude	Longitude
1	Central Farm (CAYO)	Belize	17.1	−89.0	31	Nuevitas	Cuba	21.6	−77.2
2	Melinda (STANN Creek)	Belize	17.0	−88.3	32	Manzanillo	Cuba	20.2	−77.2
3	Phillip Goldson International Airport	Belize	17.5	−88.3	33	Las Tunas	Cuba	20.9	−76.9
4	Cabo San Antonio	Cuba	21.9	−85.0	34	Jucarito	Cuba	20.7	−76.9
5	Paso Real de San Diego	Cuba	22.6	−83.3	35	Orange	Jamaica
6	Bahía Honda	Cuba	22.9	−83.2	36	WorthyPark2	Jamaica
7	La Fe	Cuba	21.7	−82.8	37	La Jíquima	Cuba	20.9	−76.5
8	Punta del Este	Cuba	21.6	−82.5	38	Universidad	Cuba	20.0	−75.8
9	Casablanca	Cuba	23.1	−82.3	39	Punta Lucrecia	Cuba	21.1	−75.6
10	Owen Robert A. Georgetown	Cayman	19.3	−81.4	40	Maisí	Cuba	20.2	−74.1
11	Jovellanos	Cuba	22.8	−81.2	41	Jimaní	Dominican Republic	18.3	−71.5
12	Aguada de Pasajeros	Cuba	22.4	−80.8	42	Barahona	Dominican Republic	18.2	−71.1
13	Cienfuegos	Cuba	22.2	−80.4	43	Padre Las Casas	Dominican Republic	18.7	−70.9
14	Santo Domingo	Cuba	22.6	−80.2	44	Santo Domingo	Dominican Republic	−69.5
15	Sagua la Grande	Cuba	22.8	−80.1	45	Las Americas	Dominican Republic	18.3	−69.4
16	Topes de Collante	Cuba	21.9	−80.0	46	San Juan	Puerto Rico	18.5	−66.1
17	Yabú	Cuba	22.5	−80.0	47	St. Thomas	Virgin Islands	18.3	−65.0
18	Trinidad	Cuba	21.8	−80.0	48	St. Croix	Virgin Islands	17.7	−64.8
19	Sancti Spíritus	Cuba	22.0	−79.4	49	National Agricultural Station	St. Kitts	17.3	−62.7
20	Jibaro	Cuba	21.7	−79.2	50	V.C. Bird International Airport	Antigua	17.1	−61.5
21	Júcaro	Cuba	21.6	−78.9	51	St. Augustine	Trinidad & Tobago	10.6	−61.4
22	Venezuela	Cuba	21.8	−78.8	52	Canefield Airport	Dominica	15.2	−61.3
23	Camilo Cienfuegos	Cuba	22.2	−78.8	53	Melville Hall Airport	Dominica	15.3	−61.3
24	Florida	Cuba	21.5	−78.3	54	Piarco	Trinidad & Tobago	10.6	−61.3
25	Esmeralda	Cuba	21.8	−78.1	55	Arnosvale/E.T. Joshua Airport	St. Vincent	13.1	−61.2
26	Santa Cruz del Sur	Cuba	20.7	−78.0	56	Lamentin	Martinique	14.6	−61.0
27	Camagüey	Cuba	21.4	−77.8	57	Hewanorra	St. Lucia	13.7	−60.9
28	Bsavannah	Jamaica			58	Crown Point	Trinidad & Tobago	11.2	−60.8
29	Cabo Cruz	Cuba	19.8	−77.7	59	Caribbean Institute for Meteorology and Hidrology	Barbados	13.2	−59.6
30	Palo Seco	Cuba	21.1	−77.3	60	Grantley Adams Airport	Barbados	13.1	−59.5

). Data from Belize and Cuba were supplied by the respective Meteorological Services from each country, while data from the Dominican Republic are homogenized series from [36]. Station data for the rest of the study domain were those used by [34].

We use fifteen global and one regional monthly GPP (

Table 1. Information on GPP used in this study. The abbreviations in column 3 are defined as follow: G, gauges; S, satellite; and R, reanalysis. The term land in the column 5 means the dataset is limited to land areas only. A detailed description of each GPP is provided as Supplementary Information.

No	Abbreviation	Sources	Full Name	Spatial Resolution	Temporal Coverage	References
1	CRU	G	Climatic Research Unit (CRU) Time-Series (CRUTS4.01)	0.5° (land)	1901–2016	[13]
2	CPCGlobal	G	Climate Prediction Center (CPC) Unified	0.5° (land)	1979–present	[38]
3	PRECL	G	PRECipitation REConstruction over Land (PREC/L)	0.5° (land)	1948–present	[39]
4	GPCC05	G	Global Precipitation Climatology Centre (GPCC) Full Data Reanalysis	0.5° (land)	1891–2016	[40]
5	GPCC025	G	Global Precipitation Climatology Centre (GPCC) Full Data Reanalysis	0.25° (land)	1891–2016	[40]
6	HERREAULT	G	High resolution Regional statistical downscaled GPCC v7 for the Caribbean	0.033° (land)	1950–2017	[32]
7	CHIRPS	G,S	Climate Hazards group Infrared Precipitation with Stations (CHIRPS)	0.05° (land)	1981–present	[17]
8	PERSIANNCDR	G,S	Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) Climate Data Record (CDR) PERSIANN-CDR	0.25°	1983–present	[16]
9	TERRACLIMATE	G,R	High-resolution global dataset of monthly climate and climatic water balance from 1958–2015	0.0416° (land)	1958–2015	[41]
10	CHELSA	G,R	Climatologies at high resolution for the earth’s land surface areas	0.0083° (land)	1979–2013	[42]
11	CFSR	R	National Centers for Environmental Prediction (NCEP) Climate Forecast System Reanalysis (CFSR)	0.3125°	1979–2010	[43]
12	ERAi	R	European Center for Medium-range Forecast Reanalysis Interim (ERA-Interim)	0.25°	1979–2017	[44]
13	ERA5	R	European Center for Medium-range Forecast Reanalysis	0.25°	1979–present	[45]
14	JRA55	R	Japanese 55-year Reanalysis (JRA-55)	0.56°	1959–present	[46]
15	MSWEP	G,S,R	Multi-Source Weighted-Ensemble Precipitation (MSWEP) version 1	0.25° (land)	1979–2015	[18]
16	MSWEP2	G,S,R	Multi-Source Weighted-Ensemble Precipitation (MSWEP) version 2	0.1° (land)	1979–2015	[24]

). These datasets vary in their spatial and temporal resolutions, as well as in the methods used to develop them. The 16 GPP includes 6 products based on station data only, 2 that combine ground station information and satellite, 2 merging station and reanalysis, 4 based on reanalysis, and 2 using multisource (i.e., a combination of data sources) information. The GPP were selected based on their prior use in the region as well as on their availability and accessibility for the period 1983–2010 (28 years) which overlapped with the available station data. It is difficult to consider a longer time-interval due to, for example, limitations such as surface station data availability, completeness and inhomogeneities. However, as suggested by previous studies, this is a common constraint for most countries in the Caribbean [34,37].

2.3. Methodology

We followed a point-to-grid approach by extracting the GPP grid cell over the underlying weather stations, which produces similar results as the neighborhood interpolation method. Features of precipitation at both point and spatial scale are analyzed over the subregions shown in Figure 1. We evaluate the GPP following 2 steps. First, we assess the ability of each GPP to represent the main features of the rainfall annual cycle in our study domain, including the presence of the MSD. Secondly, we use indicators of performance such as the modified efficiency index of Kling-Gupta (KGE) [47] and the standardized precipitation index (SPI) at 3, 6, and 12 months [48,49,50]. Since the KGE index integrates other statistical metrics such as the bias and correlation coefficients, we also use these metrics in our evaluation. The KGE is calculated for every location (station/grid point) using monthly precipitation during the dry (November–April) and wet (May–October) periods and annually with the following equation:

$$KGE = 1-\sqrt{{\left(R-1\right)}^2+{\left(B-1\right)}^2+{\left(V-1\right)}^2}$$

(1)

$$R = \frac{{\sum }_{i=1}^n\left(O_i-\overline{O}\right)\left(S_i-\overline{S}\right)}{\sqrt{{\sum }_{i=1}^n\left(O_i-\overline{O}\right)}\sqrt{{\sum }_{i=1}^n\left(S_i-\overline{S}\right)}}$$

(2)

$$B = \frac{\overline{S}}{\overline{O}}$$

(3)

$$V = \frac{C{V}_s}{C{V}_o}$$

(4)

where n is the number of observations, O_i and S_i are the observed and estimated data, respectively, in month i; O and S are the average values of the observations and the estimates, respectively.

The optimal value of the KGE index and its components (R, B, and V) is equal to one. R is the linear correlation between observations and precipitation estimates; B is the bias, which reflects when the estimated values are greater or lower than observations, while V is similar to the bias but for variability.

We calculate SPIs for 3, 6, and 12 months to explore how each GPP represents the observed statistics of drought and pluvial conditions (SPI ≤ −1.0 and SPI ≥ 1.0, respectively) [48] in the Caribbean. The evaluation considers both the drought statistics as well as the number of grid points within the region with different SPI categories. We evaluate drought statistics such as the frequency of dry months (DF) and the number of drought events (DE). We define a drought considering the experiences from the Meteorological Service of Cuba, in which a drought occurs when the SPI value of a single or sequence of months falls below the −0.5 threshold (herein SPI-0.5). A dry interval ends when the SPI becomes greater than −0.5. Similarly, an SPI value of −1.5 (SPI-1.5) is also used to examine severe droughts. The number of the grid points either under pluvial or drought conditions is determined by creating a time series of the number of stations or GPP grid cells affected by each category. Since this time series is an indicator of the pluvial/drought area, we called it the SPI Areal Assessment (SPIAA). We use the CARiDRO system [51] to derive most of the hydroclimate statistics and calculations.

3. Results and Discussion

3.1. The Mean Annual Cycle of Precipitation

The mean precipitation annual cycle from the GPP (Figure 2a–c) reveals a high correspondence with observations for the subregions defined in Figure 1. For the western sub-region and the Caribbean as a whole, the GPP represent well the bimodal pattern of the rainfall, including the MSD (Figure 2a,b). There is strong coincidence between the GPP that overestimate or underestimate observations in the western Caribbean and the Caribbean as a whole. This may reflect the fact that the larger proportion of stations fall in the western Caribbean (45 out of the 60 stations). In the Eastern Caribbean (Figure 2c), where the MSD is absent and the land masses are smaller, the GPP similarly capture the main features of the observed annual variation pattern including the occurrence of the maximum in November. However, differences from the observations tend to be greater than for the western Caribbean and some GPP a such as PERSIANNCDR, CFSR, and CHELSA are even displaying a displaying the opposite sign in total precipitation particularly in the wet season months (not show). The implication is that a GPP that does well at capturing mean climate in the western Caribbean or the Caribbean as a whole may not necessarily be equally good in the eastern Caribbean.

For the eastern sub-region, four gauge only/gauge-based GPP (CHIRPS, PRECL, GPCC025, and GPCC05) are among the best performers, having the lowest absolute differences. In comparison, the reanalysis products tend to underestimate the monthly mean precipitation, with ERAi producing the best representation of observations (smallest monthly deviations). GPP with largest deviations are CPCGLOBAL, ERA5, PERSIANNCDR and MSWEP2 (underestimates) and CRU, MSWEP and HERREAULT (overestimates). These GPP use different methodologies in their development and different sources of data; for example, whereas CPCGLOBAL and CRU use only rain-gauges, PERSIANNCDR and CHIRPS merge rain-gauges and satellite information, while ERA5 is a reanalysis product. We suggest that the relatively poorer performance of most of the GPP over the eastern Caribbean may result from: (i) the scarcity of rain-gauges in this subregion, and (ii) the poor representation by the reanalysis-based GPP of the precipitation processes that occur in the small islands (e.g., with areas ≤ 1500 km²).

The GPP’ performance is significantly better in the western Caribbean, though substantial departures exist between observations and some GPP e.g., CPCGLOBAL and ERA5. We note that the better performance by some GPP with respect to the entire domain reflect their ability to minimize differences with ground stations in both subregions. Over the entire domain, GPCC025, GPCC05, CHIRPS, and CHELSA stand out as the better performing GPP with respect to the annual precipitation cycle features in the Caribbean.

3.2. Evaluation of GPP Based on the Modified Efficiency Index of Kling-Gupta

Figure 3 shows the results from the modified KGE analysis and its components (Bias, Correlation and Variability). The five GPP with highest KGE median values are the two versions of GPCC (GPCC025 and GPCC05), the two MSWEP products (MSWEP and MSWEP2) (all greater than 0.7) and the CHELSA product (median score of 0.68). The GPP with the lowest median KGE was CPCGLOBAL (<0.4) which also had large deviations from observations in the climatological analysis. The same GPP emerge as the best performers with respect to median KGE for the wet and dry seasons (Figure 4), though their relative rankings shift depending on season. Across the two seasons, CPCGLOBAL remains the worst performer. The five best GPP generally perform well across the entire Caribbean, with high values of KGE distributed across the region (Figure 5). The inclusion of a higher number of rain gauges and other improvements in the 8th version of GPCC [40] might explain the good skill of GPCC025 and GPCC05. The MSWEP and MSWEP2 products likely also perform well because of the use of a high-resolution dataset based on surface stations. Furthermore, since CHELSA uses a similar downscaling procedure as GPCC025, it shares similarities with GPCC in our study region.

Figure 3 also shows the annual KGE component scores for each GPP. Relative performance with respect to correlation generally matches that for the overall KGE score suggesting the strong influence of the latter in the calculation of the former. It is noted that the five best GPP and the worst GPP with respect to median KGE also emerge as the GPP with highest correlation values (ranging from 0.752 to 0.841) and lowest value (0.582) respectively. A similar result was found by [24] when evaluating various daily precipitation datasets over the conterminous US. As in [24], we confirm that correlation has a dominant influence on KGE scores. For GPP to have a better representation of observations, then, improving this statistic, without degrading the information content, should be a priority. When the analysis is done by season (not shown), the median values of temporal correlations are comparable, ranging from 0.5 to 0.8 (0.45 to 0.75) for the dry (wet) season. Almost all GPP underestimate the rainfall variability in the region (median values from 0.7 to 0.95), except for CPCGLOBAL and MSWEP2, which are the only products that overestimate it. Across individual seasons MSWEP2 also shows the best results for variability (not shown).

Table 2. Mean median KGE components scores. Bias, Correlation and Variability are expressed as |B − 1|, | R− 1| and |V − 1|, respectively (see Equation (1)).

	Annual	Dry	Wet
Bias	0.083	0.109	0.096
Correlation	0.254	0.350	0.358
Variability	0.131	0.168	0.227

presents the mean scores across all GPP for the median KGE’s components. Table 2 highlights the poor performance of correlation in general, which greatly influences the final KGE scores. Table 2 also suggests that the variability component should also be taken into account to improve the GPP performance in the wet season.

We also note the following based on the calculated scores for KGE and its components:

• Based on the KGE index, GPCC025 and GPCC05 (gauge only), MSWEP and MSWEP2 (gauge, satellite, and reanalysis), and CHELSA (gauge and reanalysis) display an overall good performance, including for each season and the whole year. Of the two datasets that merge rain-gauge and satellite information, we find that, at least in our study domain, PERSIANNCDR performs worse than CHIRPS. We suggest that the use of a coarse precipitation dataset (2.5°) such as GPCP [16], possibly reduces the skill of PERSIANNCDR to adequately represent precipitation features in the Caribbean.
• CRU and PRECL datasets are among the station-based products with the lowest performance in representing precipitation variability across the Caribbean. This is likely due to the lower number of surface station taken into account in their development, as compared with other station based GPP such as GPCC025, GPCC05, and HERREAULT.
• Although we expected similar performance for HERREAULT as GPCC, considering the former was built upon a GPCC product [32], this was not observed. In this regard, we found significant differences between the GPCC version 7 (used in HERREAULT) and version 8 (evaluated here), as shown by their low correlations over our study domain (Figure 6). This could in part explain the differences in the performance between HERREAULT and the GPCC products used in this paper. Nevertheless, further investigation should be carried out due to the potential to generate a more accurate regional high-resolution product (such as HERREAULT) based on the GPCC full data version 8.

• The lowest KGE scores, suggesting a relatively poor performance, correspond to most reanalyses (except for ERA5) and consistently CPCGLOBAL. The reanalysis products are among the GPP with the lowest performance, which reflects the limitations of the current atmospheric models to represent the fine-scale precipitation processes. This might be due to limitations in representing sub-grid convection and land-surface parameterization schemes. This result also reveals a constraint related to the spatial resolution needed to adequately represent the small islands of the Lesser Antilles. A possible solution to this limitation could be the incorporation of high-resolution, non-hydrostatic, regional models [52].
• Although ERA5 significantly underestimates precipitation across the Caribbean, it is the reanalysis product with the best performance based on the KGE scores. This is seen in the better correlation and variability scores with ground stations. This suggests its potential for use in the development of new GPP products for the Caribbean especially with bias correction techniques employed—which may prove easier than trying to improve the linear correlations [27]. For example, it is hypothesized that the CHELSA product, which was developed based on ERA-Interim [44], may perform even better over the region if using ERA5 instead, which is currently the primary reanalysis model from the ECMWF.

3.3. GPP Evaluation Based on SPI

The results for selected drought statistics determined using SPI-0.5 and SPI-1.5 are presented in Figure 7 for SPI 3, 6, and 12. Overall, most of the GPP fail in representing accurately the SPI-0.5 statistics (Figure 7a,c), having different ratings for different statistics. The GPP with best performance were MSWEP, MSWEP2, ERA5, and CHELSA. For SPI-1.5 (Figure 7b,d), the GPP better resemble the observations with a discernible distinction between the worst and best performers, and a tendency to overestimate the maximum consecutive months with drought and the number of drought events. In general, products such as MSWEP2, ERA5, CHELSA, and MSWEP were the most accurate. The best performing products across both SPI statistics are the same as those with highest correlations in the KGE analysis (see Figure 3). We note, however, that care must be exercised when assessing GPP performance on the basis of an individual drought statistic, as it only offers a limited perspective while not capturing the overall ability of the GPP across SPI’s of different intensity and length. For instance, following the methodology used by [23], who only consider DF in the evaluation of GPP, we would conclude that CRU or TERRACLIMATE are among the best GPP for SPI-0.5. Conversely, if we take into account the DE statistics, the performance of those GPP is not as good. Similarly, we find that PRECL is among the GPP with best skill in representing DE statistics for SPI-0.5, but not DF for SPI-0.5.

In conducting the SPIAA analysis, we compared the SPIAA time series from observations against the GPP for SPI 3, 6, and 12. To do so, we selected −0.5 and 0.5 as SPI thresholds to identify droughts and pluvial events. Figure 8 presents the results of SPIAA using SPI12. (Though SPI3 and SPI6 are not shown they indicate similar results). For drought conditions, the GPP have different performances across the drought categories, with correlations ranging from 0.3 to 0.8, and temporal variability greater than observations. The results are better for areas with SPI lower than −0.5 (Figure 8a) and worse when moderate and severe drought categories are considered (Figure 8b,c). MSWEP, CHIRPS, and GPCC025 show the best skill overall across SPI categories, although their accuracy for severe dry category is very low. On the other hand, it is also noted that some GPP show noticeable better skill for pluvial conditions, as reflected by a lower dispersion within each category, higher correlation scores, and standard deviation values closer to the observations (Figure 8d–f). In this regard, the GPCC025, GPCC05, and MSWEP2 products display the best performance.

The aforementioned results suggest the following about the use of SPI as a performance measure. Firstly, the use of drought statistics offers only a general idea about the GPP performance, without reflecting possible differences in the temporal occurrence of drought events between observations and GPP. Secondly, GPP skills are seemingly better for pluvial conditions than for drought ones, at least for the time period considered in this study. Thirdly, the assessment based on SPI suggests that MSWEP2, MSWEP, and GPCC025 are among the best overall performers, even when other GPP products show a better ability for individual SPI indicators. This suggests, once again, that GPP products based on a larger number of station data have a better ability to reproduce the behavior of rainfall in the Caribbean. Finally, the combination of drought statistics with other indicators that represent temporal variations of drought (such our SPIAA) offers a better approach to evaluate the GPP by avoiding possible bias in the interpretation of the results.

4. Conclusions

To our knowledge, this is the first study that objectively evaluates the performance of GPP across the entire Caribbean. We evaluated the strengths and weaknesses of sixteen gridded monthly precipitation products for the period 1983–2010, by comparing them to 60 weather stations across the Caribbean. The evaluation was limited to 28-year period due to data availability and data quality. The evaluation was carried out by analyzing the differences in the representation of the rainfall annual cycle and by employing different statistical metrics such as the KGE as well as others derived from SPIs which characterize various features for drought and pluvial conditions. We summarize the main findings of this work as follows:

• With respect to the applied methodological approach, we conclude that:
  • The combined analysis of the annual precipitation cycle and KGE and SPI metrics, provide a robust and comprehensive understanding of the performance of the considered precipitation products for the Caribbean. While the comparison of the annual precipitation cycles between GPP and observations provided useful preliminary information about the best and worst GPP, the KGE index added a global and objective picture of the GPP abilities by integrating correlation, bias, and variability. At the same time, SPIs gave insights about the GPP skill with respect to other important features of precipitation. For instance, ERA5 was the worst reanalysis product in representing the annual precipitation cycle for the Caribbean, underestimating the total monthly precipitation. However, it performed better than other GPP in the SPI analysis, mainly because of its higher correlation with observations. Similarly, GPP, such as CHELSA or TERRACLIMATE, were among the best for some individual performance indicators (e.g., DF and DE statistics) but did not show similar ability in other ones (e.g., SPIAA moderate and severe (Figure 8b,c)).
• With respect to the annual cycle analysis, we conclude that:
  • The analysis of the annual precipitation cycle revealed a high correspondence between GPP and observations across the Caribbean as whole, as well as for the two sub-regions analysed. Features such as the MSD and the maximum of rainfall during November were well represented for the Western and Eastern sub-regions respectively. There was a better performance for the majority of the GPP over the Western Caribbean when compared with the eastern side. This might either be due to the low station density used in the development of these GPP, especially over the Eastern Caribbean, or the inability of the reanalysis products to properly represent the small territories and the precipitation processes that happen on these small land areas. Considering this analysis, we suggest that the best performing products for the whole Caribbean and the sub-regions were the gauge-based GPCC025 and GPCC05 and the gauge and satellite-based CHIRPS.
• With respect to the evaluation based on the KGE index we highlight the following:
  • Although correlation has a strong influence in the KGE scoring, it was the worst component captured by the GPP. We suggest that for further improvement or development of GPP for use in the Caribbean, improving correlation should be a primary goal.
  • Generally speaking, the reanalysis products were among the lowest-performing GPP group, although some of them performed better than PERSIANCDR (gauge and satellite) and CPCGLOBAL (gauge, satellite, and reanalysis). The lower representativeness of reanalysis suggests deficiencies of atmospheric models to reproduce the small-scale precipitation processes. In this group, ERA5 had the best performance, despite its significant precipitation underestimation. This result calls the attention of the potential users of ERA5 to develop new regional precipitation products by using bias correction or downscaling techniques.
  • On the basis of the KGE analysis, GPCC025, GPCC05, MSWEP, and MSWEP2 yielded better results for both annual and seasonal time scales. We suggest that the use of higher surface station density in these GPP highly influenced this outcome, notwithstanding that their development methodologies are different.
• With respect to the evaluation based on SPI, we observe the following:
  • Overall, the GPP fail in representing accurately the SPI-0.5 statistics, having different ratings for different statistics. The GPP with best performance were MSWEP, MSWEP2, ERA5, and CHELSA. For SPI-1.5, the GPP better resemble the observations with a discernible distinction between the worst and best performers, and a tendency to overestimate the maximum consecutive months with drought and the number of drought events. In general, products such as MSWEP2, ERA5, CHELSA, and MSWEP were the most accurate.
  • Considering the areas affected by different SPI categories, all GPP perform better for pluvial conditions than for drought for all SPI lengths. For wet periods, the correlations were higher, and the amplitudes were greater but closer to the observations. The GPCC025, MSWEP, MSWEP2, CHIRPS, and HERREAULT products had the best performance.
  • The strategy to explore GPP performance across varying SPI statistics as well as using the temporal variations of the areas affected by pluvial and drought conditions are complementary. This provides a more robust evaluation of the GPP and avoids potential bias in the interpretation of results.

In general, we conclude that GPCC025 and MSWEP2 appear to be the best performers for the whole study domain and each sub-region as they were always among the better performing GPP across the range of metrics and evaluation methods examined.

It is not a surprise that the GPP that use a greater number of ground stations in their development have a better representation of precipitation as compared to others GPP that uses much less observations. For example, in the development of GPP such GPCC, the number of stations used within our study region is relatively high, even though the number of stations has decreased in recent decades. In addition, GPCC uses many of the 60 stations we used to evaluate all GPP in this work. This fact might explain the better performance of GPCC over other station-based products.

GPCC025 offers the opportunity to develop climate studies over longer time intervals because of its temporal span (1891–2017) and relatively high spatial resolution. MSWEP2 offers the possibility for studies that requires a very high spatial resolution (0.1°) over the last three decades. The use of other products will generally depend on the aim and particularities of their application. For example, multivariate datasets, such as TERRACLIMATE or CRU, can be considered when different climate variables are needed to calculate derived parameters, especially for very long-term assessments. At the same time, caution is recommended when employing a GPP with a relatively low performance, even if it has a high spatial resolution or shows good accuracy at global scale. As indicated earlier, whereas higher resolution is a key consideration for the Caribbean region with its small islands, it does not necessarily mean better performance, and it is still necessary to evaluate the products.

A similar strategy to the methodology applied in this study can be used when assessing climate model outputs against observations, i.e., extending the analysis beyond just bias or annual cycles. Our results also indicate that revisiting previous evaluations of model performance over the region could produce new interpretations about accuracy, at least in the case of precipitation. For example, the conclusions reached by [30,31,46] on the accuracy of regional climate models to reproduce the observed precipitation may potentially be modified if GPCC025 is used as a reference observational dataset, instead of CRU, particularly for the Eastern Caribbean.