Ensemble Learning for Blending Gridded Satellite and Gauge-Measured Precipitation Data

Abstract

Regression algorithms are regularly used for improving the accuracy of satellite precipitation products. In this context, satellite precipitation and topography data are the predictor variables, and gauged-measured precipitation data are the dependent variables. Alongside this, it is increasingly recognised in many fields that combinations of algorithms through ensemble learning can lead to substantial predictive performance improvements. Still, a sufficient number of ensemble learners for improving the accuracy of satellite precipitation products and their large-scale comparison are currently missing from the literature. In this study, we work towards filling in this specific gap by proposing 11 new ensemble learners in the field and by extensively comparing them. We apply the ensemble learners to monthly data from the PERSIANN (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks) and IMERG (Integrated Multi-satellitE Retrievals for GPM) gridded datasets that span over a 15-year period and over the entire contiguous United States (CONUS). We also use gauge-measured precipitation data from the Global Historical Climatology Network monthly database, version 2 (GHCNm). The ensemble learners combine the predictions of six machine learning regression algorithms (base learners), namely the multivariate adaptive regression splines (MARS), multivariate adaptive polynomial splines (poly-MARS), random forests (RF), gradient boosting machines (GBM), extreme gradient boosting (XGBoost) and Bayesian regularized neural networks (BRNN), and each of them is based on a different combiner. The combiners include the equal-weight combiner, the median combiner, two best learners and seven variants of a sophisticated stacking method. The latter stacks a regression algorithm on top of the base learners to combine their independent predictions. Its seven variants are defined by seven different regression algorithms, specifically the linear regression (LR) algorithm and the six algorithms also used as base learners. The results suggest that sophisticated stacking performs significantly better than the base learners, especially when applied using the LR algorithm. It also beats the simpler combination methods.

1. Introduction

Precipitation data are needed for solving a large variety of water resource engineering problems (e.g., those investigated in [1,2,3,4,5,6,7]) and can be obtained through either ground-based gauge or satellite networks [8]. The former networks are known to offer more accurate data, while the latter are in general more spatially dense because of their lower cost ([9,10,11,12]). In this view, the widely adopted strategy of blending satellite and gauge-measured precipitation data for forming new precipitation products, with higher accuracy than the purely satellite ones and higher spatial density than the gauge-measured ones, is reasonable. Besides, the importance of the more general problem of merging gridded satellite and gauge-measured data for earth observation is well recognised, as proven by both the number and the diversity of the relevant studies (see, e.g., [13,14,15]). Moreover, as noted in [16], even products that already rely on both gridded satellite and gauge-measured precipitation data could be further improved in terms of their accuracy by using gauge-measured precipitation data in post-processing frameworks. Therefore, in what follows, data from such products and data from purely satellite products will not be explicitly distinguished, as the focus herein will be on their merging with gauge-measured data.

Machine learning regression algorithms ([17,18,19]) are regularly used for blending multiple precipitation datasets with differences in terms of spatial density and accuracy (see the relevant reviews by [20,21], as well as examples of such studies in [16,22,23,24,25,26,27,28,29,30]). In such spatial downscaling (and, more generally, spatial interpolation) settings, satellite precipitation data and topography factors are the predictor variables, and gauged-measured precipitation data are the dependent variables. Alongside these, it is increasingly recognized in many hydrological disciplines that combining individual algorithms through ensemble learning, a concept from the machine learning field that is alternatively referred to as “forecast combinations” in the forecasting field, can lead to considerable improvements in terms of predictive performance (see, e.g., the relevant discussions in the review by [31]). Detailed reviews on ensemble learning methods can be found in [32,33].

Aside from the applications of a few individual (ready-made) algorithms that rely on the construction of ensemble learning, such as the random forecast (RF; [34]) and boosting ([35,36]) ones, this concept has not spread in the literature that fuses gridded satellite products and ground-based precipitation datasets. Indeed, ensemble learning methods, such as the simple ones appearing in [37,38,39,40], and the more sophisticated ones appearing in [41,42,43,44,45,46,47,48], have not been proposed and extensively compared with each other in this literature. An exception to this can be found in the study by [49], who introduced a sophisticated ensemble learning algorithm. This algorithm adopts a stacking strategy [41] and the least absolute shrinkage and selection operator (LASSO; [50]) algorithm to combine the independent predictions of three individual algorithms. The latter are the multilayer perceptron neural networks, support vector machines ([51]) and RF. Still, many more ensemble learners and large-scale comparisons of them using gauge-measured data and multiple satellite products for an extensive time period and a large geographical region are needed in the field for reaching higher accuracy levels. In this study, we work towards filling in this specific gap. More precisely, our objectives are to propose a large number of new ensemble learners in the field, as well as to extensively compare them.

The remainder of this paper is structured as follows: Section 2 lists and describes the ensemble learning methods introduced and compared in the field by this work. Section 3 presents the data and the framework that facilitated the comparison. Section 4 presents the results. Section 5 discusses the importance of the findings in light of the literature and provides ideas for exploiting the proposed framework for future research. The manuscript concludes with Section 6. For the sake of completeness, Appendix A lists and briefly describes the machine and statistical learning regression algorithms on which the comparison relied, and Appendix B provides statistical software information, ensuring the reproducibility of the work and its methods.

2. Ensemble Learners and Combiners

2.1. Ensemble Learners

In this work, 11 ensemble learners (see Figure 1) were compared for improving the accuracy of satellite precipitation datasets. Each of these learners combined predictions that were delivered independently by six regression algorithms. The latter are referred to as “base learners” throughout the work and include the multivariate adaptive regression splines (MARS), multivariate adaptive polynomial splines (poly-MARS), RF, gradient boosting machines (GBM), extreme gradient boosting (XGBoost) and Bayesian regularized neural networks (BRNN), which are briefly described in Appendix A. These learners are regularly applied in hydrology (see, e.g., the applications by [52,53,54,55]). Also notably, they are largely diverse with each other and exhibit considerably good performances for the task of interest according to [30]. Therefore, their use as base learners is reasonable. While the base learners were the same for all the 11 ensemble learners, the procedures followed for the combination of the independent predictions were different. These procedures are known as “combiners” in the literature of ensemble learning and, in this work, they included the mean combiner (else referred to as the “equal-weight combiner” in the literature and throughout this work; see Section 2.2), the median combiner (see Section 2.3), two best learners (see Section 2.4) and seven variants of a sophisticated stacked generalization method (see Section 2.5).

2.2. Mean Combiner

The mean combiner performs simple averaging of the independent predictions delivered by the base learners. Thus, it does not require training.

2.3. Median Combiner

The median combiner computes the median of the independent predictions delivered by the base learners. Similar to the mean combiner, it does not require training.

2.4. Best Learners

The best learners are ensemble learners that select one base learner, specifically the one that scores the best in a predictive performance comparison on a training dataset, and then adopt the predictions of the selected base learner as their own for all the new problems that are set. In this work, we applied two best learners, one that bases the identification of the best base learner on the mean squared error (MSE) and another that bases this identification on the median squared error (MdSE).

2.5. Stacking of Regression Algorithms

The strategy of stacking a regression algorithm on top of others facilitates more sophisticated combinations than the mean combiner, median combiner and best learners. In this ensemble learning strategy, the base learners produce predictions independently of each other for a dataset. This dataset plays the role of a training dataset for a selected regression algorithm, which is hereafter referred to as the “meta-learner”. More precisely, the predictions of the base learners for this dataset are used together with their corresponding true values for training the meta-learner. The trained meta-learner is then given predictions of the base learners for new problems and delivers its own predictions based on them. For facilitating an extensive comparison between ensemble learners, as well as between combiners, seven variants of this stacking methodology were applied in this work, with each of these variants relying on a different meta-learner. The meta-learners were the linear regression (LR) algorithm and the six algorithms also used as base learners (see their brief descriptions in Appendix A).

3. Data and Application

3.1. Data

We applied the methods to open data. The gauge-measured precipitation data are described in Section 3.1.1, the satellite precipitation data are described in Section 3.1.2 and the elevation data are described in Section 3.1.3.

3.1.1. Gauge-Measured Precipitation Data

We used data from the Global Historical Climatology Network monthly database, version 2 (GHCNm; [56]). From this database, we extracted total monthly precipitation data in the time period 2001–2015 from 1 421 stations in the contiguous United States (CONUS). The locations of these stations are depicted in Figure 2. The data were sourced from the National Oceanic and Atmospheric Administration (NOAA) data repository (https://www.ncei.noaa.gov/pub/data/ghcn/v2; accessed on 24 September 2022).

3.1.2. Satellite Precipitation Data

Satellite precipitation data from the CONUS and for the same time period (i.e., 2001−2015) were also used. We sourced these data from two repositories. The first of them is the current operational PERSIANN (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks) database ([57,58,59]), which was created by the Centre for Hydrometeorology and Remote Sensing (CHRS) at the University of California, Irvine (UCI). The grid extracted from this database has a spatial resolution of 0.25 degree x 0.25 degree and covers the region of interest entirely (see Figure 3a). In particular, the PERSIANN precipitation data are daily and can be extracted from the Center for Hydrometeorology and Remote Sensing (CHRS) repository (https://chrsdata.eng.uci.edu; accessed on 7 March 2022). We transformed the daily data into total monthly precipitation data to be consistent with the gauge-measured data.

The second gridded satellite precipitation dataset used in this work is the GPM IMERG (Integrated Multi-satellitE Retrievals) late Precipitation L3 1 day 0.1 degree x 0.1 degree V06 dataset, which was developed by the NASA (National Aeronautics and Space Administration) Goddard Earth Sciences (GES) Data and Information Services Center (DISC) [60] and is available at the repository of NASA Earth Data (https://doi.org/10.5067/GPM/IMERGDL/DAY/06; assessed on 10 December 2022). After its extraction, the original GPM IMERG daily data were transformed to the CMORPH0.25 grid with a spatial resolution of 0.25 degree x 0.25 degree (see Figure 3b) with a bilinear interpolation procedure. The IMERG data covers the region of interest entirely. Similar to the PERSIANN data, we transformed the daily data into total monthly precipitation data to be consistent with the gauge-based measurements.

Notably, both the PERSIANN and IMERG families of gridded satellite precipitation products are extensively used and examined in the literature (see the relevant examples of studies by [58,61,62,63,64,65,66,67]). Therefore, we believe that the above-outlined data can effectively support the comparison of this study.

3.1.3. Elevation Data

As it is highlighted by [68], elevation is a useful predictor variable for many hydrological processes. Thus, its value was estimated at all the gauged geographical locations presented in Figure 2 by extracting point elevation data from the Amazon Web Services (AWS) Terrain Tiles (https://registry.opendata.aws/terrain-tiles; accessed on 25 September 2022).

3.2. Regression Settings and Validation Procedure

The dependent variable is the gauge-measured total monthly precipitation at a point of interest. According to procedures proposed in [16,30,69], we formed the regression settings by finding, separately for the PERSIANN and IMERG grids (see Figure 3a and Figure 3b, respectively), the four closest grid points to each of the geographical locations of the precipitation ground-based stations (see Figure 2) and by computing the respective distances d_i, i = 1, 2, 3 and 4 (in meters). We also indexed these four grid points S_i, i = 1, 2, 3 and 4, according to their distance from the stations, where d₁ < d₂ < d₃ < d₄ (see Figure 4). Hereinafter, the distances d_i, i = 1, 2, 3 and 4 will be called “PERSIANN distances 1, 2, 3 and 4” or “IMERG distances 1, 2, 3 and 4” and the total monthly precipitation values at the grid points 1, 2, 3 and 4 will be called “PERSIANN values 1, 2, 3 and 4” or “IMERG values 1, 2, 3 and 4”. We formulated and used three sets of predictor variables, which will be simply called “predictor sets 1, 2 and 3” throughout this work and correspond to three different regression settings. These settings facilitated three different comparisons between the learners. Predictor sets 1, 2 and 3 are, respectively, defined as follows: {PERSIANN values 1, 2, 3 and 4, PERSIANN distances 1, 2, 3 and 4, station elevation}, {IMERG values 1, 2, 3 and 4, IMERG distances 1, 2, 3 and 4, station elevation}, and {PERSIANN values 1, 2, 3 and 4, IMERG values 1, 2, 3 and 4, PERSIANN distances 1, 2, 3 and 4, IMERG distances 1, 2, 3 and 4, station elevation}.

Each regression setting consisted of 91 623 samples and was divided into three datasets of equal length (hereafter referred to as “datasets 1, 2 and 3”). The division was made randomly; therefore, the various different hydroclimatic and topographic conditions are represented equally well in all the three datasets of each regression setting. The same holds for possible biases in the gridded satellite products. We first trained the six base learners on dataset 1 and applied them to obtain predictions for dataset 2. These predictions were then used by the best learners and the stacking variants (as described in Section 2.4 and Section 2.5, respectively) for selecting a best base learner and training the meta-learner, respectively. Then, we trained the base learners on {dataset 1, dataset 2} and applied them to obtain predictions for dataset 3. These latter predictions were used by the mean combiner, the median combiner, the best learners and the stacking variants (as described in Section 2.2, Section 2.3, Section 2.4 and Section 2.5, respectively) for forming their predictions for dataset 3. They were additionally used for assessing the base learners, as these are also used as benchmarks for the ensemble learners.

3.3. Predictive Performance Comparison

To compare the 17 learners (i.e., the six base learners and the 11 ensemble learners), we first computed the squared error scoring function, which is defined as

S(x, y) := (x − y)²,

(1)

where y is the materialization of the spatial process and x is the respective prediction. The squared error scoring function is consistent for the mean functional of the predictive probability distribution [70]. The performance criterion takes the form of MSE, which is defined as

$$\mathrm{MSE} :=(1/n) {\sum }_{i=1}^{n}S(x_i,y_i),$$

(2)

where $x_i$ and $y_i$, i ∈ {1, …, n} are the predictions and materialization of the process, respectively. MSE is computed separately for each set {learner, predictor set}.

Subsequently, skill scores were computed for each set {learner, predictor set}, according to Equation (3). More precisely, type-1 skill scores were computed by using as the reference case (benchmark) the MARS algorithm when run with the same predictor set as the modelling approach to which the relative score referred. On the other hand, type-2 skill scores were computed by considering the set {MARS, predictor set 1} as the reference case (benchmark) for all the sets {learner, predictor set}. Type-1 and type-2 skill scores were used to rank the learners from the best-performing to the worst-performing. The results of the predictive performance comparison are presented in Section 4.1.

RS_{{learner, predictor set}} := 100 (1 − MSE_{{learner, predictor set}}/MSE_benchmark)

(3)

3.4. Additional Investigations

Aside from comparing the 11 ensemble learners of this work in terms of their predictive performance in the context of interest, we also recorded and compared the computational time required for their application. The results of this comparison are available in Section 4.2. Moreover, we applied explainable machine learning (see the relevant reviews by [71,72,73]) to rank the base learners based on their contribution in ensemble learning using stacking. More precisely, we implemented procedures of the RF algorithm to compute the permutation importance score for the predictions produced by the base learners for dataset 2 (on which the training of the regression algorithm that combines the base learners is made; see Section 3.2) and ranked them based on this score. We additionally implemented procedures of the XGBoost algorithm to compute the gain score for the predictions produced by the base learners for dataset 2 and then determined the corresponding rankings. The results of these two comparisons of the base learners are available in Section 4.3. Lastly, we implemented the above-outlined explainable machine learning procedures to compare the total of the predictor variables (see the predictor set 3 in Section 3.2) in terms of their relevance in solving the regression problem investigated. The results of the comparisons of the predictor variables are available in Section 4.4.

4. Results

4.1. Predictive Performance

Figure 5 presents a comparison based on MSE between the learners. According to this comparison, the six base learners (i.e., MARS, polyMARS, RF, GBM, XGBoost and BRNN) exhibit notable differences in their performance, with RF and XGBoost being the most accurate and MARS being the least accurate for all three predictor sets investigated. Moreover, ensemble learning using the mean combiner gave more accurate predictions than ensemble learning using the median combiner. The latter ensemble learning approach produced less accurate predictions than both RF and XGBoost (with a single exception concerning predictor set 2). Ensemble learning using the mean combiner was also more accurate than all the base learners for predictor sets 2 and 3. The same, however, was not the case for predictor set 1. Indeed, for this predictor set, both the simple combination approaches investigated in this work were found to be less accurate than both RF and XGBoost.

Furthermore, the best learners performed exactly the same with RF, XGBoost or BRNN (as these three base learners were identified as the best on dataset 2) and, thus, better than the median combiner (with one single exception; see above) and worse than the mean combiner for predictor sets 2 and 3, but not for predictor set 1. The best among all the ensemble learning methods that use regression algorithms as meta-learners were stacking with LR and stacking with polyMARS, while the worst was stacking with XGBoost (probably due to the importance of parametrization in the implementation of this latter regression algorithm) for predictor sets 1 and 2, and stacking with BRNN for predictor set 3 due to outliers. The remaining ensemble learning methods using regression algorithms as meta-learners were more accurate than or at least as accurate as the base learners, the simple combination methods and the best learners, with a few exceptions concerning stacking with MARS and stacking with GBM for predictor set 1, and stacking with RF for predictor set 2.

Figure 6 additionally allows us to compare the three predictor sets investigated in this work in terms of their usefulness, as they are based on the type-2 skill score. Overall, we can state that predictor set 2 (for which the type-2 skill scores ranged from 17.27% to 30.73% and the respective rankings ranged from 12 to 33) is more useful than predictor set 1 (for which the type-2 skill scores ranged from 0 to 19.87% and the respective rankings ranged from 29 to 45). We can also state that combining predictor sets 1 and 2 into predictor set 3 leads to substantial performance improvements for both the base learners and the ensemble learning methods. Indeed, for predictor set 3, the type-2 skill scores ranged between 27.32% and 39.77%, and the respective rankings ranged between 1 and 21, with a single exception concerning stacking with BRNN (for which the type-2 skill score was 6.77 and the respective ranking was 40).

4.2. Computational Time

Figure 7 facilitates comparisons of the learners in terms of computational time. Notably, BRNN is by far the most computationally expensive algorithm used as both a base learner and a meta-learner in this work and, probably due to its Bayesian regularization procedure, its computational requirements can differ a lot from dataset to dataset even when the number of predictors is the same (see, e.g., the measured time for predictor sets 1 and 2) and especially when this number differs considerably (see, e.g., the measured time for predictor set 3 compared to the measured time for predictor sets 1 and 2). Simple combination methods simply require the sum of the computational time required by each of the base learners increased by a small amount, which is practically the time required for computing the mean or the median of the predictions for each data sample. The same, however, does not apply to the remaining ensemble learning methods, which are more computationally expensive, because they require two or three different trainings. As the time required to train the algorithms on the predictions of the base learners is much less than the time required for training the base learners, these methods have comparable computational requirements, with the ensemble learning method using BRNN for combining the predictions of the base learners being the most computationally expensive overall.

4.3. Contribution of Base Learners

Figure 8 presents the comparison of the base learners in terms of their contribution to ensemble learning using RF as the meta-learner. The rankings of the base learners based on permutation importance are not exactly the same across the predictor sets. Still, there are notable similarities, with XGBoost and RF being the most important and the second most important base learners, respectively, for all three predictor sets investigated in this study. This latter finding seems reasonable in light of the better predictive performance of XGBoost and RF compared to the remaining base learners (see Figure 5). In addition, Figure 9 presents the comparison of the base learners in terms of their contribution to ensemble learning using XGBoost as the meta-learner. Based on the gain scores (which are computed by taking each feature’s contribution to each tree in the model), the three most important base learners for generating a prediction are RF, BRNN and XGBoost. This result is again reasonable, as BRNN also exhibits a better predictive performance than MARS, polyMARS and GBM (see Figure 5), with a single exception concerning predictor set 1, for which polyMARS performed better.

4.4. Importance of Predictor Variables

In brief, Figure 6 indicates that for the investigated problem: (a) both the PERSIANN and IMERG datasets offered useful predictors; and (b) the IMERG dataset offered more useful predictors than the PERSIANN dataset, to some extent. Additionally, Figure 10 and Figure 11 present more detailed investigations for predictor variable importance and, more precisely, the permutation importance score computed for the predictor variables using RF, the gain score computed for the same variables using XGBoost and the respective rankings. The comparison based on XGBoost indicates that the IMERG values are more important predictor variables than the PERSIANN values. It also indicates that elevation is more important than the PERSIANN values, and that the PERSIANN and IMERG distances are the least important. This latter finding is also supported by the comparison based on RF, which additionally supports the relatively large importance of elevation as a predictor variable for the problem of interest. On the other hand, the order of the PERSIANN and IMERG values varies a lot in the two comparisons, thereby reinforcing the outcome from Figure 6 that both satellite products offer useful predictors.

5. Discussion

This large-scale work is the first one that compares a large number of ensemble learners and combiners in the field of gridded satellite precipitation data correction. In summary, it suggests that stacking a regression algorithm on top of others can lead to large improvements in predictive performance in the context of interest and at the monthly temporal scale, beating the mean and other simple combiners. Such combiners can be “hard to beat in practice” ([74,75,76,77,78]); therefore, this important finding could not be assumed in advance. In fact, it could only stem from large-scale comparison studies involving both a large number of methods and large datasets.

The improvements in performance were found to be the largest when the combination was made using LR for two of the three predictor sets investigated and polyMARS for the third predictor set. Still, other regression algorithms also performed well as meta-learners, and RF and XGBoost additionally allowed the identification of the most useful base learners (i.e., RF, XGBoost and BRNN) when utilized in explainable machine learning settings. Also notably, the RF and XGBoost algorithms also allowed us to compare the relevance of five categories of predictor variables (i.e., the PERSIANN values, PERSIANN distances, IMERG values, IMERG distances and ground-based station elevation) in solving the problem of interest. Contrary to what applies to the daily time scale (see [69]), in the monthly temporal resolution, the utilization of both the PERSIANN and IMERG datasets leads to substantial predictive performance improvements with respect to using the IMERG dataset only.

Although this work already produced a large amount of large-scale results on the use of ensemble learning in the context of improving the accuracy of gridded satellite precipitation products, additional benefits could stem from applying extensions of its methodological framework in the future. Perhaps the most notable among these extensions are the ones referring to the daily time scale (or even to finer time scales), which could additionally benefit from a much larger number of ground-based stations with sufficient record lengths (see, e.g., the number of stations in [16,69]) and, thus, could lead to comparisons on an even larger scale. Other notable extensions are those referring to probabilistic predictions, and they could comprise machine and statistical learning algorithms, such as those summarized in the reviews by [31,79]. Indeed, it would also be useful to know whether stacking a regression algorithm on top of others can lead to notable performance improvements in the context of assessing the predictive uncertainty in gridded satellite precipitation product correction.

The methodological framework proposed in this work could additionally be extended for investigating whether and how the relative predictive performance of various ensemble learning methods depends on the features of the time series datasets (see the relevant study by [80]). Such investigations would be relevant at various temporal scales, such as the monthly, daily and finer ones, and could eventually lead to the proposal of feature-based ensemble learning methods (see, e.g., those by [81,82]) for the task of interest. Lastly, it should be noted that, although this study focused on the precipitation domain of earth observation, the framework that it proposed is also applicable to other domains of earth observation. Indeed, the problem of merging gridded satellite and gauge-measured datasets holds a prominent position for earth observation in general.

6. Conclusions

To improve the accuracy of gridded satellite precipitation products (i.e., to perform the so-called “bias correction” of these products), machine learning regression algorithms and ground-based measurements are often used in spatial interpolation settings. In such settings, the ground-based measurements are the dependent variables and the gridded satellite data are the predictor variables. In previous studies, individual machine learning algorithms were compared in terms of their skill in performing bias correction of satellite precipitation products. The respective comparisons were conducted either within regional cases or at large spatial scales (i.e., at spatial scales of continental size). While there is value in both these methodological approaches, findings from studies that apply the latter can be considered more useful in the important endeavour of understanding the properties of the algorithms compared. In spite of the accuracy benefits that can stem from ensemble learners (i.e., methods that combine two or more individual machine and statistical learning algorithms to obtain predictive performance improvements), a large number of such learners and their comparison at a large spatial scale were absent from the literature of satellite precipitation product bias correction prior to this work. According to our results, the strategy of stacking a regression algorithm on top of others (which are referred to as “base learners” in the literature) for combining their independent predictions can offer significantly better predictions than the base learners in the context investigated. It also beats the equal-weight combiner, the median combiner and two best learners, one based on the mean squared error and one based on the median squared error.