Evaluation and Reliability of CHIRPS Satellite Rainfall Products Against Station Rainfall Data in the Siwaluh Basin ^†

Abstract

The characteristics of rainfall in the Siwaluh river basin are poorly understood due to a lack of data. Many rainfall data are missing and many rain gauge posts are damaged. The availability of rainfall data is necessary for the planning and management of water resources. Technological advances can utilize satellite-derived rainfall products. These offer an alternative for areas where data are limited. However, the use of satellite data must be validated with field data. Statistical analyses used for calibration and validation involve linear regression. Validation with evaluation statistics measures how good the model is. The required data is rainfall for 20 years. This study evaluates data from the Climate Hazard Group InfraRed precipitation with Station Data (CHIRPS) satellite with rainfall station data. Satellite rainfall data were compared with three field rainfall stations. This coding was used for one Siwaluh watershed. The results show that the calibration of CHIPRS satellite data produced a correction factor in the form of a linear regression equation. The validation stage corrected the values of NSE = 0.8, RSR = 0.5, and KR = 3.95% and gave an interpretation of a very good value. This research shows that the CHRIPS data are consistent with the rain station data. These findings provide the potential for satellite data to be used to assist decision-making in areas where rainfall data are limited.

1. Introduction

Indonesia has two seasons: the rainy season and the dry season. The dry season causes drought, and the rainy season causes inundation and flooding [1]. Rainfall is one of the important meteorological parameters from several perspectives, such as hydrology, agriculture, and disaster prevention [2]. Rainfall data are important data for hydrological calculations in the planning of water buildings as a part of a conservation effort [3]. Flood discharge calculations in waterworks planning require complete and up-to-date rainfall data for optimal planning. Information on rainfall data is indispensable in water resources management [1,4].

However, complete and comprehensive rainfall data are still quite difficult to obtain [5,6]. Many rain data were lost and the equipment was damaged. Limited data must be modified to make it feasible to use [7,8]. Some rain station posts are located in very remote areas so they are not optimized and many areas do not have rain station posts. The level of risk for hydrological analysis is still not optimal [9]. The problems regarding rainfall stations cause the rainfall data at these stations to be unable to accurately represent the actual rainfall in field conditions [10]. Our study area is located in the Karanganyar Regency. The river passes through the Siwaluh River. The study location is in the Siwaluh basin, which has an area of 50.75 km² and a river length of 33.3 km². Some rain gauges are damaged, and data are lost. The data used are thus less effective. These problems can be overcome by using satellite rain data [11,12].

Some rainfall satellites that are often used as an alternative means of determining rainfall due to the availability of rain data include the CHIRPS [13], GLDAS 2.1 [14], CML [2], TRMM [15], RCM [16], JST [17], and other rainfall satellites. This research uses CHIRPS (Climate Hazard Group Infrared Precipitation with Station Data) satellite data. The reliable performance CHIRPS model has been validated. CHIRPS represents actual rainfall data and addresses the limited availability of rainfall data in basin management [18,19]. Research on the performance of the CHIRPS satellite in several regions in Indonesia and various other countries has produced quite-good values [12,20].

CHIRPS has been used to estimate rainfall data at the pixel level [4,17]. Some studies state that the CHIRPS satellite can be used as an alternative to the availability of rainfall data for water resources [4,5,21,22,23]. The larger size of the data over a wide area affects the accuracy of CHIRPS. Although many studies have been conducted with this satellite data, this study identifies the limitations of rainfall data in the Siwaluh Basin. Many rainfall stations have damaged equipment. The resulting data are incomplete. Limited data causes difficulties in hydrological analysis. Water resources management is less than optimal. Given these conditions, it is important to assess the reliability of satellite rainfall data. However, the reliability of CHIRPS in the Siwaluh Basin is still very limited. The contribution of this research is the validation of CHIRPS data with rainfall stations in the field. CHIRPS data can provide data accuracy despite the limited station data. The purpose of this research is to validate CHIRPS rainfall data with rain station post data in the Siwaluh basin. This validation is important in determining the level of accuracy of CHIRPS data and its potential use in various applications in the Siwaluh basin. High spatial and temporal differences in rainfall in Karanganyar are factors that will affect the accuracy of rainfall data. This research is expected to show that CHIRPS satellite rainfall data have a significant and reliable correlation. These data can serve as an important source of information in water resources management in areas with minimal measurements.

2. Materials and Methods

2.1. Study Area

This research was conducted in Karanganyar Regency, which is located in Central Java, Indonesia. The research location can be seen in Figure 1.

The location of this research is in Karanganyar Regency. One of the rivers that passes through is the Siwaluh River. The topography of the Siwaluh basin varies from upstream Mount Lawu and the downstream lowlands. A picture of the station location is shown in Figure 2.

The Siwaluh River has an area of 50.75 km² and a river length of 33.3 km² and empties into the Bengawan Solo River. Elevation The catchment area of the Siwaluh basin in the downstream is 500 m above sea level in the upstream is 1500 m above sea level. This difference in elevation results in variations in temperature and rainfall in the Siwaluh Basin area. The Siwaluh Basin has several soil types: latosol, regosol, andosol, and alluvial. The designation of this basin is as follows: forests (as a protected area) (40%); agriculture with various types of crops, such as rice and corn (30%); settlements in the lowlands (20%) passing through the Karanganyar sub-district, which is the economic center in Karanganyar Regency; and tourism (10%) because in the mountains, there are many tourist attractions.

The analysis method used in this study is the validation of CHIRPS satellite rainfall data via the data on 20-year rainfall stations (2004–2023). The data used from Karanganyar, Tasikmadu, and Tawangmangu rainfall stations were sorted based on data availability and distance. The data consistency test used for rainfall data comprises the double mass curve test and RAPS (rescaled adjusted partial sums), while the consistency test for CHIRPS satellite rainfall is RAPS [5]. We test the Rainfall data validation by considering the largest rainfall in one year via several stations.

This research concerns the reliability of satellite data, employing rain data on a regional basis and the observed rainfall from each rainfall station using the Thiessen polygon method. Field rainfall data and CHIRPS satellite rainfall data that have been tested for data quality and calculated regional rainfall can then be used for the next stage.

2.2. Thissen Polygon Method

The Thiessen polygon method is obtained through the creation and analysis of Arc MAP. The Thissen polygon method is used to determine rainfall stations that can represent a watershed. The coordinates of the rainfall station are entered in UTM form, as in

Table 1. Coordinates of rain stations.

No	Rain Station Name	X	Y
1	Karanganyar	495,367.20	9,160,306.8
2	Tasikmadu	491,870.47	9,162,907.4
3	Tawangmangu	513,914.83	9,152,661.46

below.

This Thiessen analysis, by entering points at the train station, will be processed to obtain results from the rain station. The Thissen polygon calculation formula is as follows [24]:

$$R={\displaystyle \frac{R1\ast A1+R2\ast A2+Rn\ast An}{A1+A2+An}}$$

(1)

where:

• A1: Area representing the observation station (km²);
• R1: Amount of rainfall for each station: 1, 2, 3 …, n (mm);
• n: Number of stations.

2.3. Calibration

Rainfall data calibration is the process of adjusting measured rainfall data to the actual conditions in the field. Calibration has the aim of increasing the accuracy of rainfall data, reducing bias and error in rainfall measurements, and improving the spatial representation of rainfall in the watershed. Rainfall data calibration methods include the correlation method, the frequency distribution method, and hydrology. This research selected the correlation method of comparing rainfall station data with CHIRPS satellite rainfall data. The selected equation is determined from the highest coefficient of determination (R²) value produced in the following equation:

$$R^2={\displaystyle \frac{{\sum }_{i=1}^N(X_i-\widehat{X})(Y_i-\widehat{Y})}{{\left[\left\{{{\sum }_{i=1}^N(X_i-\widehat{X})}^2\right\}\left\{{\sum }_{i=1}^N{(Y_i-\widehat{Y})}^2\right\}\right]}^{\frac{1}{2}}}}$$

(2)

where:

• R²: Coefficient of determination;
• X: Observation data;
• Y: Modeled data;
• $\widehat{X}$: Average of observation data;
• $\widehat{Y}$: Average of modeled data;
• i: Mean of observed data;
• N: A lot of data.

From the above equation, the selected regression equation will be used in the evaluation process at the validation stage of corrected satellite rainfall data with the validation data range. The process of correcting satellite data is conducted by entering satellite rainfall data into the regression equation that was selected during the calibration stage.

2.4. Validation

Validation is a process carried out to obtain the level of uncertainty of a model in estimating or predicting a hydrological process carried out via evaluation. The validation process is carried out to determine the reliability of the CHIRPS satellite in estimating the rain that occurs via three statistical indicators, namely, Nash–Sutcliffe efficiency (NSE), RMSE–standard deviation ratio (RSR), and relative error (KR) [21].

2.4.1. Nash–Sutcliffe Efficiency (NSE)

Nash–Sutcliffe efficiency (which can be referred to as NSE) is a value that shows the accuracy of an observation value plotting result via comparisons using simulated or satellite values corresponding to the 45° line, with a range of values ∞ up to 1 [18]. The NSE value can be determined by the equation below:

$$\mathrm{N}\mathrm{S}\mathrm{E}=1-{\displaystyle \frac{{{\sum }_{i=1}^N(Y_i-X_i)}^2}{{{\sum }_{i=1}^N(Y_i-{\widehat{Y}}_i)}^2}}$$

(3)

where:

• X: Observation data;
• Y: Modeled data;
• i: Mean of observed data;
• N: Total data.

The criteria for the value of the Nash–Sutcliffe efficiency (NSE) parameter are shown in

Table 2. Criteria for the value of the Nash–Sutcliffe efficiency (NSE) parameter.

NSE Value	Interpretation
0.75 < NSE ≤ 1	Very Good
0.65 < NSE ≤ 0.75	Good
0.50 < NSE ≤ 0.65	Meets
NSE ≤ 0.50	Does not meet

below.

2.4.2. RMSE–Standard Deviation Ratio (RSR)

The RMSE–standard deviation ratio (commonly referred to as RSR) is a stage involving the standardizing of RMSE using the standard deviation of the observation data and combining error indices. Calculating the RSR value can be undertaken by using the ratio of RMSE and the standard deviation of the observation data such that the RSR value will be obtained with the provision that the smaller the RSR value, the better the performance of the simulation model. The RSR value can be determined by the equation below:

$$\mathrm{R}\mathrm{S}\mathrm{R}={\displaystyle \frac{\sqrt{{{\sum }_{i=1}^N(Y_i-X_i)}^2}}{\sqrt{{{\sum }_{i=1}^N(Y_i-{\widehat{Y}}_i)}^2}}}$$

(4)

where:

• X: Observation data;
• Y: Modeled data;
• $\widehat{Y}$: Average of Modeled data;
• i: Mean of observed data;
• N: All data.

The value criteria of the RMSE–standard deviation ratio (RSR) parameter are shown in

Table 3. Criteria for the value of the RMSE–standard deviation ratio (RSR) parameter.

RSR	Interpretation
0.0 < RSR ≤ 0.5	Very Good
0.5 < RSR ≤ 0.6	Good
0.6 < RSR ≤ 0.7	Meets
NSE ≤ 0.7	Does not meet

below.

2.4.3. Relative Error (KR)

Relative error value (or KR) is a value calculated with the aim of knowing the percentage of relative error of modeled satellite rain data. The relative error can be known by comparing it with existing observation data. The relative error value (KR) can be determined by the equation below:

$$\mathrm{K}\mathrm{R}=\left|{\displaystyle \frac{{{\sum }_{i=1}^N(Y_i-X_i)}^2}{{\sum }_{i=1}^N{Y}_i}}\right|$$

(5)

where:

• X: Observation data;
• Y: Modeled data;
• i: Mean of observed data;
• N: All data.

3. Results and Discussion

3.1. Regional Rainfall

The rainfall data used for calculations using the Thiessen polygon method are the regional rainfall data obtained from the Siwaluh Watershed, Karanganyar Regency, Central Java. Calculations using the Thiessen polygon method are carried out by calculating the weight of each area that represents a rain station. The application of the Thiessen polygon method to the Siwaluh basin, Karanganyar Regency, Central Java, is shown in Figure 3 below.

After drawing the Thiessen polygon of the Siwaluh basin, Karanganyar Regency, Central Java, the next step is to calculate the influence factor (Kr) of each rainfall station and its area of influence. The results of the calculation can be seen in

Table 4. Rain station post influence area factor value (Kr).

Station	Area (km2)	Kr	Area of Influence (%)
Karanganyar	14.57	0.29	29%
Tasikmadu	24.55	0.48	48%
Tawangmangu	11.63	0.23	23%
Total	50.75	1.00	100%

below.

3.2. Validation of Uncorrected Satellite Rainfall Data

In this study, the validation of the uncorrected satellite rainfall data is carried out to determine the accuracy of the CHIRPS satellite before it is corrected, which will be used to estimate the rainfall that occurs on the Earth’s surface. At this stage, the Nash–Sutcliffe efficiency (NSE), RMSE–standard deviation ratio (RSR), and relative error (KR) values are necessary, using the rainfall station data for a period of 20 years (2004–2023). The validation of the uncorrected data is addressed in

Table 5. Results of uncorrected CHIRPS data validation analysis.

No	Year	p Ground (mm)	p Satellite (mm)
1	2003	72	47.5
2	2004	79	58.3
3	2005	97	44.1
4	2006	90	66.7
5	2007	217	55.2
6	2008	189	41.5
7	2009	153	61.7
8	2010	131	57.5
9	2011	114	45.7
10	2012	175	39.8
11	2013	104	78.1
12	2014	71	52.7
13	2015	100	51.1
14	2016	147	72.5
15	2017	115	65.3
16	2018	116	60.4
17	2019	107	40.7
18	2020	89	60.3
19	2021	71	38.1
20	2022	113	83.5

.

Table 5 presents the comparison between rainfall data and CHIRPS satellite data without correction. Data are presented for 20 years. There are differences in rainfall values obtained from measurements with satellite data. Satellite data are not fully accurate in representing the rainfall conditions in the field. Rainfall fluctuates from year to year. Differences in ground p-values are not always followed by similar changes in satellite p-values. The calculation results obtained from the validation stage of the uncorrected satellite rainfall data are, respectively, for NSE, RSR, and KR values of 0.912, 5.756, and 6.15%. Based on the values obtained, the validation of the uncorrected satellite rainfall data resulted in the interpretation being “Not Met”. The values obtained from the validation of the uncorrected satellite rainfall data will thus be further analyzed. CHIRPS satellite rainfall data without correction still require improvement before they can be treated as accurate data. Accurate data estimation requires the calibration and validation of satellite data via field data.

3.3. Calibration of Satellite Rainfall Data

The calibration of satellite rainfall data is carried out with the aim that the modeled data, with respect to the observational data, have a high level of accuracy [25]. In analyzing the data, five regression equation models are needed, namely, linear, exponential, polynomial, logarithmic, and power. The resulting values are presented in

Table 6. Data analysis.

Period	Satellite	R2					R2 Max
		Linear	Exponential	Polynomials	Logarithms	Ranked
Annual	CHIRPS	0.9971	0.8175	0.9933	0.8897	0.9787	0.9971

, as follows.

Table 6 shows the comparison of five analysis methods, namely, linear, exponential, polynomial, logarithm, and power. The linear method has the highest R² value of 0.9971, indicating a good agreement between ground and satellite rainfall data. The linear method is used for CHIRPS data analysis with good accuracy. We need to consider further bias correction and validation before making decisions based on these data. The graphical representation of the linear method is shown in Figure 4.

This graph shows the relationship between satellite p and ground p, with a positive correlation between the two. The straight line passing through the data points indicates a linear relationship between the rain station data and the satellite data. The coefficient of determination is 0.9971, indicating a very strong relationship. The graph shows that the rainfall data measured by the satellite have a close relationship with the rainfall data at the rain station. Satellite data can be used to predict rainfall based on satellite data.

3.4. Validation of Corrected Satellite Rainfall Data

The last stage of this research is the validation process for corrected satellite rainfall data, in which we compare the observation data with the corrected satellite data. The corrected satellite data are obtained from the regression equation, which was obtained at the satellite rainfall data calibration stage. At this stage, the calculation of the Nash–Sutcliffe efficiency (NSE), the RMSE–standard deviation ratio (RSR), and the relative error (KR) values is carried out using 20-year data from rainfall stations (2004–2023) that have been corrected using the regression equation selected at the calibration stage. The regression synchronization results are shown in

Table 7. Linear regression method.

No	Year	p Ground (mm)	p Satellite (mm)	p Satellite Calibrated (mm)
1	2003	72	47.5	105.3
2	2004	79	58.3	129.1
3	2005	97	44.1	97.7
4	2006	90	66.7	147.7
5	2007	217	55.2	122.2
6	2008	189	41.5	91.9
7	2009	153	61.7	136.6
8	2010	131	57.5	127.3
9	2011	114	45.7	101.2
10	2012	175	39.8	88.2
11	2013	104	78.1	173.0
12	2014	71	52.7	116.8
13	2015	100	51.1	113.2
14	2016	147	72.5	160.6
15	2017	115	65.3	144.6
16	2018	116	60.4	133.9
17	2019	107	40.7	90.1
18	2020	89	60.3	133.6
19	2021	71	38.1	84.4
20	2022	113	83.5	184.9

.

In Table 7, rainfall data have been measured over the period 2003 to 2022. In 2003, the rainfall was 72 mm, while the uncalibrated satellite rainfall data gave a rainfall value of 47.6 mm, and after calibration it became 105.3 mm, so it changed. The satellite rainfall data are calibrated to correct discrepancies with rain station data. Satellite data calibration improves the accuracy and relevance of rainfall data. The results obtained from the validation stage for corrected satellite rainfall data are, respectively, for NSE, RSR, and KR, values of 0.80, 0.51, and 3.95%. Based on the values obtained, the validation of corrected satellite rainfall data produces a very good interpretation.

The analysis results show that CHIRPS rainfall data have an excellent performance, with an R² value of 0.99. This value indicates that 99% of these satellite data can reliably be used for hydrological analyses. The use of satellite data provides a solution in areas where data are limited. Calibration in this study strengthens the validity of the data, as well as the effective management of water resources. The NSE value shows a very good model. The value of 0.92 changed to 0.8 after calibration. This increases the accuracy of the model describing the real field conditions. The RSR value also needs to strengthen the satellite data. The significant contribution of satellite data to the Siwaluh basin confirms the importance of satellite data in water resources management strategies.

These data can be used in hydrological modeling for flood mitigation. Rainfall data research needs to consider the suitability of the data for further analysis.

4. Conclusions

The characteristics of rainfall in the Siwaluh basin are not well known because many rainfall stations are damaged, and data are limited. The limitations in rainfall data can be overcome by using globally estimated rainfall data. In this study, the spatial variability of rainfall has been analyzed using CHIRPS data. The CHIRPS satellite data to be evaluated requires 20 years of rainfall data. From the results of the validation, the CHIRPS estimated rainfall is consistent with the station rainfall data. CHIRPS data, after going through the calibration process, show a strong correlation. The evaluation parameters were NSE, RSR, and KR. CHIRPS rainfall performs well in identifying rain.

The CHIRPS rainfall data values are as follows: NSE = 0.8, RSR = 0.51, and KR 3.95%, with a very good interception. The CHRIPS satellite can be recommended for estimating the rainfall that occurs in the Siwaluh basin area. This research shows the potential use of satellite data in water resource planning. In areas where station data are limited, satellite rainfall can serve as an effective alternative in understanding rainfall and decision-making. This research opens up possibilities for further research into the use of satellite technology in climate change adaptation, and contributes to the management of water resources in various regions.