Assessing the Water Budget Closure Accuracy of Satellite/Reanalysis-Based Hydrological Data Products over Mainland China

Abstract

A good water budget involving four variables, including precipitation (P), evapotranspiration (ET), streamflow (R), and terrestrial water storage change (TWSC), is reflected in two aspects: a high accuracy against observations for each budget component and the low water budget closure residual error (ΔRes). Due to the lack of consideration of observations of budget components in existing water budget closure assessment methods (BCMs), when the ΔRes of budget components is low, their error against respective observations may still be high. In this study, we assess the water budget closure accuracy of satellite/reanalysis-based hydrological data products over mainland China based on six popular P products and multiple datasets of additional budget components (ET, R, and TWSC). The results indicated that the ΔRes changes between ±15 mm over mainland China. Satellite P products such as GPM IMERG showed better performance by comparing them with rain gauge-based observations. However, reanalysis P products such as GLDAS and FLDAS showed a better water budget closure since the selected datasets of additional budget components (ET and R) are also derived from reanalysis datasets. This indicates that these same data sources for budget components make it easier to close the water budget. The further development of satellite P products should consider the closure of the water budget with other water cycle variables.

1. Introduction

Precipitation (P) is the main driver of the terrestrial hydrological cycle [1,2,3]. A great number of satellite and reanalysis precipitation (SRP) products, characterized by global coverage and high temporal and spatial resolution, have been produced and widely applied in large-scale meteorological and hydrological analyses [4,5,6,7]. This is because, despite the high accuracy of in situ rainfall observations, they are sparse, and the record length in many basins around the world is insufficient, limiting the accuracy of hydrological research in these basins greatly [8]. Despite the advantages of SRP products over rain gauge-based observations in terms of spatial coverage and resolution, they have large uncertainties and errors arising from sensor deficiencies, retrieval algorithms, and discordant data resolution [9,10,11]. More importantly, the water budget closure accuracy of SRP products with respect to products of additional budget components (ET, R, and TWSC), which is critical for accurate hydrological research, may be low. Therefore, when studying hydrological changes in regional and global basins, the products of hydrological variables should meet high accuracy targets in two aspects, i.e., the high accuracy against observations for each budget component and the low water budget closure residual error.

In the literature, the assessment of water budget closures has become a basic condition for verifying the reliability of hydrological products [12,13,14,15]. Water budget closure-based assessment methods (BCMs) help users understand the error sources and uncertainties of budget component datasets by quantifying water budget imbalances [16,17] The residual error in water budget closure ΔRes = P − ET − R − TWSC has been commonly used to assess the water budget imbalance caused by errors in budget component datasets [13,18,19]. Based on ΔRes, a series of derived methods (BCMs) have been developed; they include the proportion of ΔRes to mean precipitation and the method for closing water and energy budgets, simultaneously developed by Hobeichi et al. [20]. Sheffield et al. [9] assess the ability of remote sensing products of P, ET, and TWSC to close water budget by comparing their residual error represented by the estimated R (R = P − ET − TWSC) with streamflow observations; Lehmann et al. [21] assess water budget closure of budget component datasets by using the estimated TWSC as the residual error, and then the estimated TWSC was compared with the measured TWSC using the Gravity Recovery And Climate Experiment (GRACE) satellite mission. However, these BCMs lack consideration of observations of budget components when assessing the water budget accuracy of budget component products and are subject to mutual cancellation of errors in budget component products from different sources, which was defined as the high accuracy of water budget closure but the low accuracy of individual budget components in this study. As a result, the accuracy of the selected budget component products (against observations) based on BCMs may be low, although the ΔRes of these selected datasets is small. The combination of existing BCMs and observations of budget components provides an opportunity to solve this problem.

Many studies have been conducted to assess how existing hydrological products can close water budgets in different global basins since different products of budget components show various accuracies in different regions [22,23]. Abolafia-Rosenzweig et al. [24] evaluated three techniques: proportional redistribution, constrained Kalman filter, and multiple collocations for enforcing water budget closure in 24 global basins. Luo et al. [12] proposed a novel error-based method for assessing the water budget closure of satellite-based hydrological products in the Tarim River Basin, and Soltani et al. [18] proposed a probabilistic framework for estimating water budgets in low runoff regions of the central Basin of Iran. Sahoo et al. [25] use the ΔRes/P to assess the water budget closure accuracy of satellite remote sensing data of budget components in ten global basins and found that the ΔRes/P is changed between 5 and 25% in most basins. Different studies in different basins showed various water budget closure accuracies due to the diverse inversion accuracy of hydrological products in these basins. There is currently no relevant report on the question of how satellite/reanalysis-based hydrological products can close the terrestrial water budget over mainland China.

Based on the above analysis, BCMs have the advantage of quantifying the water budget closure of SRP products with respect to datasets of additional budget components but are subject to the mutual cancellation of errors in budget components. Maintaining a high precision in budget component datasets in both aspects of against observations and water budget closure is critical for accurate hydrological research. Statistical metrics, such as root mean square error, correlation coefficient, and false alarm ratio, were commonly used to assess SRP products by comparing them with rain gauge-based observations [26]. If high-precision datasets can be pre-selected using statistical metrics and then the selected datasets are applied to BCMs, it is possible that SRP products based on existing BCMs have high accuracy in the two aspects mentioned above. Therefore, this study focuses on assessing and intercomparing the water budget closure accuracy of six popular SRP products relative to datasets of additional budget components in mainland China by combining statistical methods and existing BCMs. The assessment helps understand the water budget closure state of existing hydrological products in mainland China. In detail, the statistical methods were first used to quantify the accuracy of SRP products by comparing them with rain gauge-based observations to ensure that the accuracy of SRP products applied to water budget closure assessment is high. The existing BCMs were then used to quantify the ΔRes of the selected SRP products with respect to datasets of additional budget components.

The specific objectives of this study were: (1) to assess the water budget closure accuracy of six popular SRP products relative to the selected datasets of additional budget components in mainland China. The difference in closing water budget between satellite and reanalysis P products is further investigated due to their different inversion methods; (2) to compare the difference in SRP assessment between statistical methods and BCMs with mainland China as a case study; and (3) to investigate the uncertainties that affect SRP assessment.

This study was organized as follows. Study area and data are described in Section 2.1 and Section 2.2; Methods used in this study are represented in Section 2.3; Results for assessing SRP products and intercomparing their accuracies for closing water budget are presented in Section 3; The main results and uncertainties affecting water budget closure assessment are discussed in Section 4; Finally, the conclusions are summarized in Section 5.

2. Study Area, Datasets, and Methodology

2.1. Study Area

Figure 1a, b shows the study area of mainland China, with an area of approximately 9.6 million km². It covers diverse topography and climatic regions, indicating that the performance of SRP products in different topographies and climatic conditions can be well investigated with mainland China as a case study. The mean annual P across mainland China ranges from 7 to 2754 mm [27]. It decreases significantly from the southeast to northwest of mainland China and is affected by the prevailing monsoon climate from the Indian and Pacific Oceans [28]. As shown in Figure 1a, rain gauge stations within mainland China are mainly distributed in eastern China and less distributed in western China, covered mainly by desert and alpine landscapes.

2.2. Datasets

2.2.1. Rain Gauge Observations

A monthly ground-station dataset of precipitation derived from the China Meteorological Data Service Center (http://data.cma.cn/en (accessed on 31 October 2023)) was used as the reference for assessing the accuracy of SRP products. Locations of rain gauge stations used in this study are shown in Figure 1a. These data were collected from 2000 to 2020. In order to ensure the quality of data, many rain gauge stations with a large number of missing values were removed. In detail, we use 5% as a threshold to filter rain gauge stations. That is, rain gauge stations with the amount of missing data accounting for less than 5% of these total data, which means that the overall evaluation performance will not be affected by missing data, were selected; for the selected rain gauge stations, months with no data during the study period were interpolated using a sample linear interpolation method.

2.2.2. SRP Products

Table 1. Features of the selected six popular precipitation products.

Precipitation Product	Resolution	Period	Provider	Reference
Monthly Global Precipitation Measurement (GPM) v6 (GPM IMERG)	0.1 degree 3 h	2000-present	NASA GES DISC at NASA Goddard Space Flight Center, Maryland, United States	Huffman et al. 2019 [31]
TRMM (TMPA/3B43) Rainfall Estimate L3 (TRMM 3B43)	0.25 degree 1 month	1998–2019	NASA GES DISC at NASA Goddard Space Flight Center, Maryland, United States	Huffman et al 2010 [32]
NOAA Climate Data Record (CDR) of Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN-CDR), Version 1 Revision 1 (PERSIANN-CDR)	0.25 degree 1 day	1983-present	NOAA NCDC, North Carolina, United States	Sorooshian et al. 2014 [33]
The fifth generation ECMWF atmospheric reanalysis of the global climate (ERA5)	0.25 degree 1 month	1979-present	ECMWF/Copernicus Climate Change Service, Reading, United Kingdom	Copernicus Climate Change Service, 2017 [34]
GLDAS Noah Land Surface Model L4 (GLDAS)	0.25 degree 3 h	2000-present	NASA GES DISC at NASA Goddard Space Flight Center, Maryland, United States	Rodell et al. 2004 [35]
FLDAS Noah Land Surface Model L4 (FLDAS)	0.10 degree 1 month	1982-present	NASA GES DISC at NASA Goddard Space Flight Center, Maryland, United States	McNally et al. 2017 [36]

shows the SRP products used in this study: GPM IMERG, TRMM 3B43, PERSIANN-CDR, ERA5, GLDAS, and FLDAS. They include different data sources and inversion algorithms and have been widely used for hydrological analysis in previous studies [29,30]. Figure 2 shows the spatial distribution of these six SRP products.

The GPM IMERG is a new-generation satellite-based P dataset produced by combining passive microwave data, microwave-calibrated IR satellite data, and P gauge observations [31]. This dataset can be obtained from https://search.earthdata.nasa.gov/search?q=GPM_3IMERGM_06 (accessed on 31 October 2023). TRMM 3B43 is a satellite-based P dataset combining GPCC rain gauge analysis [32]. It can be downloaded from https://search.earthdata.nasa.gov/search?q=TRMM_3B43 (accessed on 31 October 2023). PERSIANN-CDR is also a satellite-based P product, which can be obtained from https://chrsdata.eng.uci.edu/ (accessed on 31 October 2023). ERA5 is the reanalysis dataset, which estimates P by combining model simulation and rain gauge-based observations. Monthly reanalysis data of ERA5 can be downloaded from https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=form (accessed on 31 October 2023). GLDAS is the global land data assimilation system. It estimates almost all budget fluxes, including P, ET, and R, based on land surface models such as the variable infiltration capacity (VIC). The GLDAS dataset can be downloaded from https://search.earthdata.nasa.gov/search?q=GLDAS_NOAH025_M_2.1 (accessed on 31 October 2023). Like GLDAS, FLDAS also estimates almost all budget components, including P, ET, and R, but not the state variable of TWSC. The product can be downloaded from https://search.earthdata.nasa.gov/search?q=FLDAS_NOAH01_C_GL_M (accessed on 31 October 2023).

2.2.3. Datasets of Additional Budget Components

Together with P, the additional budget components of R, ET, and TWSC jointly constitute a closed hydrological system. Both products, GLDAS and FLDAS, provide global coverage of ET and runoff estimates. The R used in this study is the sum of the surface runoff and subsurface runoff simulated using hydrological models in GLDAS and FLDAS. Since there are no sufficient TWSC observations in most global basins, GRACE TWSC has been widely used for hydrological analysis in previous studies [19,37,38,39]. GRACE satellite monitors TWSC by mapping variations in the Earth’s gravity field [40]. In this study, three GRACE TWSC products were considered: the CSR produced by U. Texas/Center for Space Research (Austin, TX, USA), GFZ produced by GeoForschungsZentrum Potsdam (Potsdam, Germany), and JPL produced by NASA Jet Propulsion Laboratory (Pasadena, CA, USA) [41]. These datasets cover the time period from 2002 to 2017 since the GRACE mission was launched in March 2002 and finished in October 2017. The coarse spatial resolution of GRACE TWSC data (1 degree) determines that they are more suitable for TWSC research in large-scale watersheds. Since these three GRACE TWSC products are produced by different centers, producing TWSC data individually, their values may differ slightly. In most previous studies, they are commonly merged into one single TWSC product to reduce uncertainties [25,42]. In this study, the data merging technology in Equation (8) below was used to merge these three GRACE TWSC products.

2.3. Methodology

Figure 3 shows the flowchart for assessing the water budget closure accuracy of SRP products relative to datasets of additional budget components. It mainly consists of three parts: (1) data preparation, (2) assessing the accuracy of SRP products by comparing them with rain gauge-based observations using statistical methods for selecting high-precision datasets, and (3) assessing the water budget closure accuracy of the selected SRP products in combination with datasets of additional budget components using BCMs. The selection of high-precision datasets in step 2 ensures that the accuracy of SRP products applied to step 3 is high. In this study, we showed all results of the ΔRes in the Section 3 instead of only the ΔRes for the selected high-precision SRP products to intercompare the difference in closing water budget between satellite and reanalysis precipitation products.

2.3.1. Data Preparation and Statistical Metrics

The datasets used in this study were pre-processed to form a unified format. In detail, all SRP products were mapped to a 1 degree spatial resolution and aggregated to a 1 month temporal resolution. We assess SRP products on a monthly timescale for the following reasons. The assessment of water budget closure requires datasets of four budget components (P, ET, R, and TWSC). However, GRACE TWSC datasets used in this study are monthly-based satellite observation data, constraining the study of water budget at finer scales. This is one of the main reasons why most current studies assess water budget only at a monthly time scale [12,13,16,25]. In addition, assessing the water budget at a time of less than a month will inevitably introduce more uncertainties because the water budget quantifies all the water that flows into and out of the study area. It is difficult to estimate them at daily and hourly timescales accurately.

Three commonly used statistical metrics were employed to assess the accuracy of SRP products, including the root mean square error (RMSE), Pearson correlation coefficient (CC), and relative error (RE) [43,44]. The metrics CC, RMSE, and RE ranges are $[-1,1]$, $[0,+\infty )$, $(-\infty ,+\infty )$, respectively. The values of 1, 0, and 0, respectively, mean the best performance.

$$RMSE=\sqrt{\frac{{\displaystyle {\sum }_{i=1}^n{(Ob{s}_i-Si{m}_i)}^2}}{n}}$$

(1)

$$CC=\frac{{\displaystyle {\sum }_{i=1}^n(Ob{s}_i}-\overline{Obs})(Si{m}_i-\overline{Sim})}{\sqrt{{\displaystyle {\sum }_{i=1}^n(Ob{s}_i}-\overline{Obs}{)}^2}\sqrt{{\displaystyle {\sum }_{i=1}^n(Si{m}_i}-\overline{Sim}{)}^2}}$$

(2)

$$RE=\frac{{\displaystyle {\sum }_{i=1}^n(Si{m}_i-Ob{s}_i)}}{{\displaystyle {\sum }_{i=1}^{n}Ob{s}_i}}$$

(3)

where n is the number of observed p values; Obs_i and Sim_i are the observed and estimated p values at time i, respectively; $\overline{O}$ and $\overline{S}$ are the averages of the observed and estimated p values, respectively.

2.3.2. Water Budget Closure Assessment

The general expression of the water budget equation is shown in Equation (4) [16,25]. Note that the monthly terrestrial water storage anomalies (TWSA) provided by GRACE are relative values (relative to the average in 2004–2009). Therefore, TWSA data in GRACE needs to be further processed to obtain TWSC data in Equation (4). In this study, the backward difference equation (Equation (5) was used to deal with the gravity anomalies of the TWSA caused by GRACE sensors [17,18]. Accordingly, the budget components, except for the TWSC, were adjusted according to Equation (6) owing to the pre-processing of the GRACE TWSC products.

$$TWSC=P-ET-R$$

(4)

$$TWSC=\frac{TWSA(t+1)-TWSA(t)}{\Delta t}$$

(5)

$$\Delta W(t)=\left(\frac{P(t+1)+P(t)}{2}\right)-\left(\frac{ET(t+1)+ET(t)}{2}\right)-\left(\frac{R(t+1)+R(t)}{2}\right)$$

(6)

where $\Delta W(t)$ represents changes in water storage represented by the water budget flux terms P, ET, and R.

If all budget components in Equations (5) and (6) take “true values,” TWSC is equal to $\Delta W(t)$, that is, Equation (7) strictly holds. However, in practice, the water budget is rarely closed due to errors within the budget-component products. That is, the ΔRes error always exists in practice. According to previous studies, the ΔRes (due to errors in budget components) has been commonly used to assess the imbalance in the water budget [13,18]. It is calculated based on Equations (5) and (6). A small ΔRes represents high water budget closure accuracy, i.e., the high data consistency in the budget component products.

$$\Delta Res=TWSC-\Delta W(t)$$

(7)

The specific steps for assessing the water budget closure accuracy of SRP products relative to datasets of additional budget components are as follows. First, different water budget ensembles constituted by SRP products and the same products of additional budget components (ET, R, and TWSC) were formed (see Figure 3). Six SRP products were considered in this study. Assuming three were selected according to the first step: $P1\_Selected$, $P2\_Selected$, and $P3\_Selected$. Therefore, three budget ensembles were formed in Figure 3: $\Delta Res\_P1$, $\Delta Res\_P2$, and $\Delta Res\_P3$; Second, we calculate and compare the ΔRes of budget ensembles for selecting high-precision SRP products with good water budget closure relative to additional budget component products. Finally, SRP products with high accuracies in terms of both aspects of univariate (compared with rain gauge-based observations) and water budget closure (small ΔRes error) were determined.

There are two main error sources affecting water budget closure assessment using ΔRes, i.e., the error from products of additional budget components and the error from different P products. The ideal situation is that the accuracy of SRP products and additional budget component products, which are inputs for the ΔRes calculation, is high. For the former, there are no sufficient observations for each additional budget component at the global grid-scale due to the limited spatial distribution of ground observation sites. Therefore, observations of additional budget components cannot be directly applied to the calculation of the ΔRes. In addition, no single product for each budget component can capture the spatial pattern of the budget component for all regions of the globe [45]. To reduce the uncertainty of SRP assessment caused by products of additional budget components, the data merging technology in Equation (8) was used to derive hybrid estimates for each additional budget component since the merged values from multiple products have been shown to be close to observations and minimize data uncertainty [25,46].

$$Pro{d}_x={\displaystyle \sum _{i=1}^{n}Pro{d}_{x,i}}•{\omega }_i\begin{array}{l}\hfill \end{array}and\begin{array}{l}\hfill \end{array}{\omega }_i=\frac{\frac{1}{{{\displaystyle \sigma }}_i^2}}{{\displaystyle \sum _{j=1}^n\frac{1}{{{\displaystyle \sigma }}_j^2}}}$$

(8)

where Prod_x denotes the merged value of budget component x, Prod_x,i denotes the i-th budget component product, ω_i denotes the weight of the i-th product, ${\sigma }_i^2$ denotes the error variance of the i-th product, and n denotes the number of products for the budget component.

For the error from different P datasets, SRP products contain various sources of error due to the different P inversion algorithms and sensor sampling errors [11,47,48]. For example, P estimates from microwave sensors are based on microwave emission and scattering of raindrops and ice in low-frequency and high-frequency channels, respectively [4,11]. Reanalysis-based approaches are different from satellite inversion algorithms. They estimate P by combining rain gauge-based observations and climate model simulated data. Therefore, different data sources produce different errors in the estimation of SRP products. Moreover, the accuracy of SRP products varies between different regions [10]. In this study, we reduce the error caused by different data sources of P by selecting high-precision SRP products using statistical metrics.

3. Results

3.1. Assessment of SRP Products Using Statistic Metrics

This section focuses on assessing the accuracy of the selected six SRP products in mainland China. Figure 4 shows the spatial distribution of the statistical metrics CC, RMSE, and RE. For the metric CC (Figure 4a–f), all SRP products showed a better performance of P estimates in most eastern and southeastern regions of mainland China (CC > 0.7 for most stations) than the western regions (CC < 0.7 for most stations). RE (Figure 4m–r) showed the same spatial distribution as CC. According to Figure 1a, rain gauge stations were mainly distributed in China’s eastern and southeastern regions. This may be one of the main reasons for the better estimates of SRP products in these regions. However, the metric RMSE in Figure 4g–l showed an opposite result compared with CC and RE. That is, RMSE with small values (≤30 mm), showing a better performance of P estimation, was mainly distributed in the western regions of mainland China. We speculate that this is mainly attributed to the smaller P magnitudes in western China (Figure 2), leading to a smaller difference in magnitude between SRP products and rain gauge-based observations.

According to Figure 5, GPM IMERG showed a better performance of P estimation than GLDAS, FLDAS, and TRMM 3B43 in mainland China, followed by PERSIANN-CDR and ERA5. Although the results of statistical metrics based on observations were reliable, the misestimates of SRP products caused by the information reuse of rain gauge-based observations (the overlap in gauge-based observations used for producing and verifying SRP products) affected the accuracy of the results. Most SRP products, such as GPM IMERG, were combined with rain gauge-based observations during their production [31,49]. Therefore, it is very likely that the information of rain gauge stations used for bias correction or data fusion when producing SRP products was the same as that used for the performance assessment of SRP products using statistical metrics, resulting in information reuse errors. Some studies have shown that satellite-gauge products have higher P detection capabilities than satellite-only products [28]. Budget closure-based assessment methods can reduce the problem of data reuse for gauge-based observations to some extent, as they consider all budget components of P, ET, R, and TWSC simultaneously rather than comparing them with observations. In addition, water budget closure has rarely been considered in the production of SRP products because of the lack of sensors capable of monitoring all budget components simultaneously [25].

We compared the results with previous studies in mainland China. We reached the same conclusion as previous studies [44,47,50,51,52,53,54]. That is, GPM IMERG outperforms reanalysis products in mainland China. According to Jiang et al. [53], the performance of GPM IMERG is better than ERA5 over mainland China on the daily, monthly, and seasonal timescales. The results of Wei et al. [47] on the monthly timescale and Tang et al. [54] on the hourly timescale also showed the same results. All these studies indicated that reanalysis datasets (mainly referring to ERA5) tend to overestimate P against ground observations in many regions of China. A possible reason for GPM IMERG overperforming ERA5 in mainland China is that the former has been calibrated based on the monthly P products from GPCC (1.0°, monthly; Huffman et al. [55]) and daily P data from the Climate Prediction Center (CPC) (0.5°, daily; Mega et al. [56]), respectively (Xu et al. [47]; Jiang et al. [53]). However, ERA5 only assimilates the P estimates from the National Centers for Environment Prediction (NCEP) in the USA [57]. ERA5 products have not been calibrated based on P observations in mainland China [44]. It is worth mentioning that each SRP has its own strengths and weaknesses. The performance of SRP products varies with different regions (climatic zones), time scales, and precipitation phases [44]. It is difficult to find a product that performs well in all regions of the world [45].

3.2. Water Budget Closure Assessment of SRP Products Relative to Additional Budget Component Products

This section focuses on assessing the water budget closure accuracy of SRP products relative to datasets of additional budget components on monthly and seasonal scales.

Table 2. Monthly and seasonal ΔRes of P products relative to additional budget components in mainland China.

Month	GPM-IMERG	TRMM-3B43	PERSIANN-CDR	ERA5	GLDAS	FLDAS
1	2.64	3.44	3.22	7.51	3.35	1.33
2	0.6	1.65	1.95	6.57	1.07	−0.2
3	−2.79	−1.29	−1.46	6.42	−1.42	−2.86
4	−2.26	−0.81	−0.91	10.49	0.32	−1.56
5	−4.25	−3.27	−2.11	11.29	−0.65	−2.71
6	−2.58	−2.17	0.88	13.95	2.2	−0.1
7	−14.69	−14.64	−8.18	3.26	−7.6	−8.89
8	−6.25	−6.76	1.54	14.01	2.54	2.44
9	−4.72	−5.23	−1.08	12.49	0.36	2.46
10	1.07	1.72	2.57	13.91	3.45	4.5
11	2.54	3.81	3.12	10.79	3.61	2.43
12	7.44	8.47	6.79	12.17	7.52	5.9
Spring	−9.52	−5.42	−4.89	27.72	−1.99	−6.85
Summer	−24.23	−23.06	−6.99	31.27	−4.76	−6.26
Autumn	−0.92	0.64	4.03	37.28	6.85	9.89
Winter	4.08	6.83	−0.53	22.23	5.35	0.85

shows the monthly and seasonal average ΔRes. Note that the ΔRes in Table 2 is calculated based on different P products in Table 2 and the same merged ET, R, and TWSC products in Section 2.2.3.

According to Table 2, the monthly average ΔRes for SRP products over mainland China varied between ±15 mm, which is acceptable based on previous studies that concluded that the value varies between ±20 mm in most global basins [12,25,58]. Among the SRP products, GLDAS and FLDAS exhibited the best water budget closure relative to the selected datasets of additional budget components in Table 1, followed by GPM IMERG and TRMM 3B43, whereas PERSIANN-CDR and ERA5 exhibited the worse water budget closure. Therefore, BCMs showed a different result from that of statistical metrics, which indicates that GPM IMERG performed the best by comparing with rain gauge-based observations. This fully illustrates the importance of selecting high-precision SRP products using statistical metrics before assessing their water budget closure accuracy relative to datasets of additional budget components. Figure 6 shows the results of the ΔRes in nine major basins of mainland China. It shows that basins with large ΔRes values are HRB, PRB, SEB, and SWB. These basins are mainly distributed in the coastal areas of eastern and southern China, which are greatly affected by monsoon. The ΔRes in basins of YB, CB, and SLRB located in northern China is relatively small.

The different results in the performance assessment of SRP products based on statistical methods and BCMs are mainly due to the different error sources of the two methods. For the accuracy of SRP products, it was assessed by comparing them with rain gauge-based observations using statistical metrics. Therefore, the results of SRP assessment are affected by the quality of rain gauge-based observations (e.g., the spatiotemporal coverage, data integrity, and record length of observed p values). However, the water budget closure assessment is affected by the data accuracy of SRP products and additional budget component products. Therefore, when we assess the water budget closure accuracy of SRP products, it is highly dependent on the selected datasets of additional budget components.

3.3. Monthly and Seasonal Variations of the ΔRes

Figure 7 shows the monthly variation of the ΔRes in mainland China, which is calculated based on SRP products and datasets of additional budget components. It is obvious that ERA5 exhibited a clear overestimation of P compared with the remaining SRP products, as the ΔRes of ERA5 was notably greater than zero in almost all the months (Figure 7). The high ΔRes for ERA5 is caused by its high magnitude compared with the remaining SRP products. For the remaining ΔRes, they exhibited fluctuation changes. The Mann–Kendall (MK) method was used to test the significance of ΔRes changes. It is a non-parametric trend testing method. For the method, the sign of the statistic “Z” is used to determine whether the time series of ΔRes has a trend. When Z takes a positive value, it indicates an upward trend; on the contrary, it indicates a downward trend. When the absolute value of Z is greater than or equal to 1.64, 1.96, and 2.58, respectively, it means 90%, 95%, and 99% significance [59]. We perform the MK test using the Python software package. The Z-values for compositions GPM IMERG_ΔRes, TRMM_ΔRes, PERSIANN_ΔRes, ERA5_ΔRes, GLDAS_ΔRes, and FLDAS_ΔRes are −0.24, −0.25, 0.56, −1.22, 0.24, and 0.27, respectively. Therefore, the ΔRes for some compositions (GPM IMERG, TRMM 3B43, and ERA5) showed a decreasing trend over time in mainland China (mainly satellite products) and a slightly increasing trend for some other compositions (GLDAS and FLDAS, mainly reanalysis products). However, the decreasing and increasing trends are not significant since the absolute Z-values for all compositions are less than 1.64 (failed the 90% significance test). Many previous studies have also shown similar results. For example, Kinouchi et al. [22] reported a 0.03 mm/month increasing trend for residual errors according to their study in the Upper Chao Phraya River Basin.

The reduced ΔRes over time for GPM IMERG may be caused by the improved accuracy of GPM IMERG data. According to Tang et al. [54], the inversion accuracy of GPM IMERG significantly improved by comparing it with rain gauge-based observations because of the increasing number of passive microwave samples. According to Chen et al. (2020c), the IMERG Late Run performed slightly better in the estimation of global P than the IMERG Early Run. This shows that the improved accuracy of budget component products contributes to water budget closure. However, the decreasing trend of the ΔRes is not significant, which indicates that the development of satellite sensors and inversion algorithms pays more attention to the improvement of SRP inversion against rain gauge-based observations but lacks consideration of the water budget closure. Currently, no satellite sensor can monitor all budget components of hydrological systems simultaneously [25]. How to reduce the ΔRes error is the focus of further development of satellite hydrological data products. According to previous studies, some water budget closure correction methods, which focus on enforcing terrestrial water budget closure, have been proposed, such as the Proportional Redistribution, the Constrained Kalman Filter, the Multiple Collocation and the Minimized Series Deviation methods [13,24,41,54]. In addition, Luo et al. [6] proposed a method for balancing the terrestrial water budget and improving the estimation of individual budget components by combining existing water budget closure correction methods and observations of budget components. These methods can be further combined with P inversion/simulation algorithms for improving the water budget closure accuracy of P inversion in the future.

Figure 8 shows the seasonal variation of the ΔRes in mainland China. According to Figure 8a, the monthly ΔRes varied between ±20 mm. The ΔRes calculated based on reanalysis of SRP products (PERSIANN, GLDAS, and FLDAS) showed a smaller value than that calculated based on satellite SRP products (GPM IMERG and TRMM 3B43). The seasonal ΔRes (the sum of ΔRes values for three months corresponding to each season) in mainland China changed between ±30 mm, except for the ΔRes calculated based on ERA5. The ΔRes for PERSIANN, GLDAS, and FLDAS also showed similar seasonal changes with a value smaller than that of other SRP products (absolute value). In the summer, the ΔRes calculated based on satellite SRP products showed high negative values. It showed positive values in autumn and winter. The ΔRes calculated based on reanalysis of SRP products showed high values in autumn.

4. Discussion

Statistical methods and water budget closure-based methods assess the accuracy of SRP products from different perspectives, i.e., comparison with observations and the data consistency with datasets of additional budget components, respectively. The high accuracy of SRP products against observations based on statistical methods does not mean that their water budget closure accuracy is also high. The results of SRP evaluation in this study showed that water budget closure-based methods yielded a different result compared with statistical metrics. For the former, it showed a better performance for GLDAS and FLDAS than GPM IMERG and TRMM 3B43, followed by PERSIANN-CDR and ERA5. However, the latter showed a better performance for GPM IMERG than the remaining SRP products.

Regarding SRP assessment using statistical metrics, the improvement in the accuracy of SRP products is mainly caused by improved satellite sensors and inversion algorithms, and more importantly, many globally available P observations have been merged into SRP products using data assimilation technologies [60,61,62]. The performance of water budget closure is determined from the errors in all budget components, not just the error in the P variable. If the inversion accuracy of all budget components (meaning that all budget components are close to their “true” values) is improved, it can obtain a perfect water budget closure rather than a “false closure.” Therefore, previous studies on SRP inversion have focused mainly on improving the performance of SRP data against observations. Reducing the ΔRes error has received less attention. Developing integrated satellite sensors with abilities to monitor all budget components is critical for improving the accuracy of water budget closures, thereby laying the foundation for more accurate hydrological research [25].

According to this study, a combination of statistical methods and water budget closure-based methods provides an opportunity for selecting high-precision hydrological datasets. Combined with previous studies, we focus on a system summary of uncertainties that affect SRP assessment to provide insights into the further development of SRP assessment methods. The main sources of uncertainty include the inconsistent spatial and temporal resolution of budget component products, the quality of rain gauge-based observations, the selection of statistical metrics, and the errors in products of additional budget components [16,63].

Inconsistent spatial and temporal resolution of budget component datasets is a widely concerned issue in water budget closure assessment. The water budget involves four variables (P, ET, R, and TWSC). Their data are usually derived from different sources (observation sites, remoting sensing, land surface models, and hydrological models), determining that they have different spatial and temporal resolutions [18,46]. Regardless of which method (downscaling or upscaling) is used to deal with inconsistent data resolution, it inevitably introduces uncertainties, although the error introduced by upscaling is commonly less than that of downscaling. For statistical metrics, they represent different statistical significance and should be selected according to the research purpose and data availability of budget component observations [26,64]. Dense gauge networks facilitate the performance evaluation of SRP products [65]. However, observations often suffer from data integrity, length, and information reuse. The information reuse means that rain gauge-based observations merged into SRP products during their production are used again by users as a reference to assess the performance of these SRP products. Most SRP products, such as GPM IMERG, have merged with rain gauge-based observations during their production [31,49].

According to this study, the accuracy of water budget closure is highly related to errors in additional budget components. Methods that reduce uncertainties of additional budget components should be integrated with water budget closure assessment methods. For example, merged data from multiple products for each budget component performs better than one single product. This study assesses the water budget closure of SRP products relative to additional budget components by using the merged data of GLDAS and FLDAS for ET and R variables and merged data of three GRACE TWSC datasets for the TWSC variable.

5. Conclusions

Closing the water budget is critical for accurate hydrological research. This study focuses on assessing the accuracy of six popular SRP products in closing the water budget with respect to the selected datasets of additional budget components in mainland China by combining statistical methods and water budget closure-based methods. The integration of these two methods helps to obtain budget component datasets with high accuracy against observations and the small ΔRes. The main conclusions are as follows:

(1)

The ΔRes for reanalysis products of budget components is smaller than that of satellite products, although the accuracy of the latter is higher than the former when compared with ground-based observations. Combinations based on GLDAS and FLDAS showed a smaller ΔRes than GPM IMERG and TRMM 3B43. However, when compared with rain gauge-based observations, the accuracy of GPM IMERG and TRMM 3B43 is higher than GLDAS and FLDAS.

(2)

In contrast to the results of statistical metrics, which showed an increasing accuracy of SRP products, the ΔRes for all combinations did not show a significant decreasing trend in mainland China. This implies that there was a lack of attention on water budget closure in the production of budget component products in previous studies, although it is critical for more accurate hydrological research and has attracted widespread attention recently.

(3)

The main error sources affecting the SRP assessment include the inconsistent spatial and temporal resolution of budget component datasets, the quality of rain gauge-based observations, the selection of statistical metrics, and the error in datasets of additional budget components. Methods that reduce uncertainties of budget components should be integrated with existing water budget closure assessment methods for reducing the uncertainties during water budget closure assessment.