Performance Evaluation of the ERA5, MERRA-2, and PERSIANN-CDR Gridded Products in the Tambo Basin ^†

Abstract

Gridded meteorological data help to address the scarcity of data in sparse hydrometeorological networks, but their validation is crucial. This study evaluated the performance of ERA5, MERRA-2, and PERSIANN-CDR gridded products in the Tambo basin, comparing their data with meteorological stations and basin-wide averages using the Pearson correlation coefficient (CC), percent bias (PBIAS), and root mean square error (RMSE). PERSIANN-CDR showed the best performance (CC: 0.84–0.94, PBIAS: 6.90–83.10%, RMSE: 21.97–38.78 mm/month). MERRA-2 underestimated precipitation, while ERA5, despite its high correlation (CC: 0.83–0.94), overestimated it. PERSIANN-CDR is the recommended product for the region, providing a better representation of precipitation for hydrological studies and water resource management.

1. Introduction

Precipitation is a fundamental climatic variable whose variations affect numerous human activities. Understanding its spatial and temporal variability is essential for planning and decision-making, especially to minimize the impacts of extreme events [1]. It has a significant impact on people’s lives and the hydrological cycle, as well as fluctuations that affect water resource management, environmental planning, and disaster mitigation [2]. Its application is crucial for improving the accuracy of hydrological models and predictions [3], thus contributing to a deeper understanding of hydrological balance [4]. However, the insufficient network of rain gauges poses a challenge for accurately estimating the spatial variability of precipitation [5].

Satellite and grid-based data estimates offer an alternative to address data scarcity, although their usefulness depends on product accuracy [6]. With high spatiotemporal resolution, they can serve as substitutes for ground-based measurements in hydrological and atmospheric models [7]. Their main advantage is broad coverage, but their accuracy varies depending on the sensor, atmospheric conditions, and other technical factors [8].

In this context, gridded precipitation products have been widely applied in various studies. Examples include multisatellite precipitation analysis in China [9], hydroclimatological analysis in the Great Lakes basin [10], comparisons of the GPM IMERG, TMPA 3B42, and PERSIANN-CDR satellite precipitation products in Malaysia [11], and evaluation of CHIRPS, TRMM 3B42 V7, and PERSIANN-CDR in different climate regimes in Arabia [12]. MERRA-2 has been used for hydrological estimation alongside MERRA, MERRA-Land, and ERA-Interim/Land [13]. In the Sinu River Basin, Colombia, precipitation estimates from MERRA-2 and ERA5 have been compared with rain gauge measurements [14]. Furthermore, precipitation products such as GSMaP-G-NRT, PERSIANN-CCS, PERSIANN-CDR, and PERSIANN have been evaluated in the Titicaca Basin [15].

However, no evaluation of the ERA5, MERRA-2, and PERSIANN-CDR gridded products has been conducted in the Tambo Basin. Therefore, this study aims to evaluate the performance of ERA5, MERRA-2, and PERSIANN-CDR gridded products in the Tambo Basin. This evaluation is essential for improving the understanding of precipitation variability in the region and enhancing the accuracy of hydroclimatic studies.

2. Materials and Methods

2.1. Study Area

The Tambo Basin (TB) is located in southern Peru, in the Moquegua department (Figure 1). It covers an area of approximately 11,826.63 km², with elevations ranging from 212 to 5608 m above sea level (m a.s.l.) and an average elevation of 3741.52 m a.s.l. The basin has an average slope of 15.92% and an annual mean precipitation of 427.70 mm.

2.2. Terrain Data

The Digital Elevation Model (DEM) was obtained through the Google Earth Engine (GEE) platform, with an approximate spatial resolution of 30 m (https://earthengine.google.com/, accessed on 20 April 2024).

2.3. Meteorological Data

Precipitation data were provided by the Servicio Nacional de Meteorología e Hidrología of Peru and the Sistema Nacional de Inforación de Recursos Hídricos (SNIRH) of the Autoridad Nacional del Agua (ANA). Nine meteorological stations were selected based on the influence of Thiessen polygons in the study basin (Figure 1 and

Table 1. Geographical location and elevation of weather stations.

N°	Station	Latitude (°)	Longitude (°)	Altitude (m a.s.l.)
1	Mazocruz	−16.74	−69.72	4003
2	Vilacota	−17.08	−70.04	4465
3	Ubinas	−16.37	−70.85	3380
4	Ichuña	−16.14	−70.54	3756
5	Suches	−16.92	−70.38	4452
6	Laraqueri	−16.15	−70.07	3970
7	Las Salinas	−16.32	−71.16	4378
8	Lagunillas	−15.77	−70.67	4200
9	Imata	−15.84	−71.09	4475

). The daily total precipitation data cover the period from 1 January 1984 to 31 December 2019.

2.4. Gridded Meteorological Data

ERA5 is a global climate dataset developed by the Copernicus Climate Change Service of the ECMWF, covering more than 30 years. It has a spatial resolution of 24 km (0.25° × 0.25°) and spans the period from 2 January 1979 to the present. ERA5 has proven its ability to reproduce the spatial and temporal distribution of precipitation [16], as well as to represent spatial patterns and temporal trends. However, its performance varies depending on the climatic region, precipitation intensity, and topography [17].

PERSIANN-CDR was developed by the Center for Hydrometeorology and Remote Sensing at the University of California, Irvine (UC-IRVINE/CHRS) and utilizes satellite infrared data to estimate precipitation with high accuracy. It has a spatial resolution of 24 km, enabling detailed analysis on a high-precision grid. This dataset has been available since 1 January 1983.

MERRA-2 is a climate dataset produced by the Global Modeling and Assimilation Office (GMAO). It has a spatial resolution of approximately 50 km (0.5° × 0.625°) and covers the period from 31 December 1980 onward. Like its predecessor, MERRA, it is designed for historical climate analysis, providing estimates based on satellite observations, global climate models, and ground-based station data worldwide to enhance the representation of initial conditions [18].

The daily total precipitation products from ERA5, PERSIANN-CDR, and MERRA-2 were obtained through the online server Climate Engine (https://www.climateengine.org/, accessed on 20 June 2024), covering the same period as the observed precipitation data. This cloud-based web platform facilitates access, processing, and downloading of climate and remote sensing data through Google Earth Engine, offering more than 600 datasets without requiring programming [19].

2.5. Methodology

The homogenization and filling of missing data were carried out using ANDREA (Análisis de Datos Hídricos y Recursos Estadísticos) (https://snirh.ana.gob.pe/ANDREA/Inicio.aspx, accessed on 23 July 2024). To group the stations into homogeneous regions, the Ward method (1963) was used, which minimizes differences within each group while balancing their size and variability [20]. Subsequently, based on these groupings, the Regional Vector Model (RVM) was applied, comparing each station with the regional average to identify hydrological patterns [21].

To evaluate the performance of ERA5, MERRA-2, and PERSIANN-CDR, daily data were extracted from the Climate Engine platform using the point pixel sampling method. This method directly assigns the pixel value corresponding to the location of each meteorological station without additional interpolations, preserving consistency with the original dataset. The extracted data and observed records were aggregated to a monthly scale to ensure the same temporal resolution. The evaluation was conducted without filling in missing values in the observed data. The basin-wide average precipitation was estimated using the arithmetic mean for both datasets.

The evaluation metrics (

Table 2. Performance metrics.

Metric	Equation	Unit	Optimal Value
PBIAS	$PBIAS=100{\displaystyle \frac{{\sum }_{i=1}^n(S_i-O_i)}{{\sum }_{i=1}^n{O}_i}}$	$\%$	0
RMSE	$RMSE=\sqrt{{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^n}{(S_i-O_i)}^2}$	$\mathrm{mm}$	0
CC	$r={\displaystyle \frac{{\sum }_{i=1}^n\left(O_i-\overline{O}\right)\left(S_i-\overline{S}\right)}{\sqrt{{\sum }_{i=1}^n{\left(O_i-\overline{O}\right)}^2{\sum }_{i=1}^n{(S_i-\overline{S})}^2}}}$	$-$	1

) included the Pearson correlation coefficient (CC), percentage bias (PBIAS), and root mean square error (RMSE), which are widely used for validating climate products.

Where $i$ represents the location index, $S_i$ corresponds to the simulated precipitation value, $O_i$ are the observed values, $\overline{S}$ and $\overline{O}$ refer to the average values of $S$ and $O$, respectively, and $n$ indicates the number of data pairs.

3. Results

3.1. Evaluation by Meteorological Station

Figure 2 shows the spatial distribution of CC, PBIAS, and RMSE at a monthly scale in the Tambo basin. ERA5 exhibits a high correlation with observations (CC between 0.83 and 0.91) (Figure 2a), indicating a good alignment of its precipitation temporal patterns. However, it significantly overestimates precipitation (PBIAS between 71.98% and 217.47%) (Figure 2d) and shows high RMSE values (51.67 mm/month to 80.58 mm/month) (Figure 2g), suggesting considerable dispersion from the observed data.

PERSIANN-CDR shows the highest correlation among the three products (CC between 0.84 and 0.91) (Figure 2b), reflecting a strong agreement with observations. Its bias (PBIAS) ranges from 6.90% to 83.10% (Figure 2e) and presents moderate RMSE values (21.97 mm/month to 38.78 mm/month) (Figure 2h), indicating greater accuracy compared to ERA5.

On the other hand, MERRA-2 exhibits a lower correlation (CC between 0.72 and 0.82) (Figure 2c) and underestimates precipitation (PBIAS between −77.86% and −30.80%) (Figure 2f). However, it records RMSE values between 26.58 mm/month and 60.55 mm/month) (Figure 2i).

3.2. Areal Average Evaluation

The comparison of basin-averaged precipitation between observed data and the ERA5, PERSIANN-CDR, and MERRA-2 products revealed significant differences in their performance. ERA5 (Figure 3a) exhibits a high correlation (CC = 0.94) but considerably overestimates precipitation, with a bias of 106.8% and an RMSE of 49.2 mm/month. In contrast, PERSIANN-CDR (Figure 3b) demonstrates the best performance, providing the highest accuracy in precipitation estimation, the lowest RMSE (18.0 mm/month), and a moderate bias of 24.5%, while maintaining a high correlation (CC = 0.94). MERRA-2 (Figure 3c), although showing a lower bias (−53.5%) and an RMSE of 34.9 mm/month, has a slightly lower correlation (CC = 0.88).

4. Discussion

Validation of precipitation products is crucial for climate and hydrological studies [22]. The results indicate that PERSIANN-CDR is the most accurate, with a lower percentage bias (PBIAS) and root mean square error (RMSE), as well as a high correlation coefficient (CC) across most stations, confirming its strong agreement with observed data. This performance aligns with studies in the Lake Titicaca basin [15] and South Asia [23].

ERA5, while showing a high temporal correlation with observed data, outperforms MERRA-2 [24,25] but significantly overestimates precipitation [26]. This overestimation may be linked to the irregular distribution of meteorological stations, introducing uncertainty in regional precipitation estimates.

In contrast, MERRA-2 underestimates precipitation but maintains a moderate correlation with observed data [18,27]. Despite this bias, its lower error dispersion compared to ERA5 suggests a better fit under certain conditions.

5. Conclusions

The evaluation of precipitation products in the Tambo basin reveals significant differences in their accuracy and reliability. PERSIANN-CDR exhibits the best overall performance, with the highest correlation, lower bias, and moderate RMSE values, indicating a strong agreement with observed data. ERA5, despite its high correlation, considerably overestimates precipitation, limiting its applicability for accurate hydrological assessments. MERRA-2, while showing lower bias and RMSE compared to ERA5, underestimates precipitation and has a slightly lower correlation.