2.3. Method
The computational experiment used a common leakage-controlled framework to compare simple baselines, direct ERA5-EQM replacement, deterministic spatial interpolation, and three machine-learning scenarios. Fifteen percent of complete station-day observations were reserved as an independent final-test set, and all calibration, predictor generation, SHAP analysis, cross-validation, and parameter selection were restricted to the remaining 85% development pool.
The development pool was evaluated by five-fold shuffled stratified cross-validation at the individual station-day level. Validation targets were masked before precipitation-dependent predictors were generated; dates and stations were not used as grouping units. The precipitation-intensity strata and model-selection rule are defined in
Section 2.3.14.
The predictor groups and their scientific roles are described in
Section 2.3.5. SHAP was used to interpret the fitted full-hybrid LightGBM model rather than to eliminate predictors; all 17 predefined predictors were retained for the full-hybrid analysis.
Hybridization was implemented at the predictor level. For each target station-day, the target observation was masked and a same-day leave-one-out ordinary Kriging estimate was generated from the remaining stations and added as kriging_feature. The final estimate was produced by the two-stage machine-learning model; Kriging and machine-learning outputs were not averaged. Basic machine learning used temporal, neighbouring-gauge, and station-context predictors; the Kriging-supported scenario added kriging_feature; and the full-hybrid scenario additionally included EQM-corrected ERA5-Land precipitation and meteorological covariates. All scenarios used the same data partitions, algorithms, and evaluation metrics.
Data analysis was performed using Python version 3.13.7.
2.3.1. Data Control
For the proper planning and management of water resources, precipitation data collected by meteorological observation stations must be consistent and complete [
8,
25]. Precipitation data must be 0 or positive values; they cannot be negative [
26]. Before imputing missing precipitation data, it is crucial to verify the data to select the right methods and ensure better model performance. In this study, the location, status, number, and any missing or incorrect values of the 14 stations’ data were checked manually and via software. The checks revealed that the distance between stations was variable, with some exceeding 100 km, indicating a sparse station network. There were no negatively signed precipitation values, and since the stations are in a semi-arid region, zero-value precipitation events were frequent. The leakage-controlled experimental workflow and the compared precipitation-imputation scenarios are summarized in
Figure 2.
2.3.2. Calculating Relative Humidity from ERA5-Land Data
Relative humidity represents atmospheric moisture conditions and may provide useful auxiliary information for precipitation estimation. However, ERA5 reanalysis climate data do not directly contain relative humidity. It must be generated through calculation. Since ERA5 reanalysis relative humidity (RH) data are not directly provided, they were calculated using temperature (T) and dew point temperature (Td) variables based on the updated Magnus formulation by Alduchov & Eskridge (1996) [
27]. This method is considered one of the most accurate and widely accepted approaches in the literature. This method, which is routinely employed by the NOAA and the ECMWF, is widely recognized in the literature as one of the most common and reliable approaches [
5,
28]. Since ERA5 does not directly provide relative humidity in the downloaded variables, daily mean relative humidity was derived from 2 m air temperature and 2 m dew point temperature using the Magnus-type saturation vapor pressure formulation. ERA5 temperature variables were first converted from Kelvin to degrees Celsius using Equations (1) and (2):
The saturation vapor pressure at temperature
was computed as Equation (3):
Relative humidity was then calculated as Equation (4):
where
is air temperature in °C,
is dew point temperature in °C,
and
are the saturation vapor pressures in hPa, and
is the relative humidity expressed as a percentage. Values were constrained to the physically meaningful interval of 0–100%.
2.3.3. Simple Temporal and Statistical Baselines
To test whether the target-station time series alone was sufficient, two low-complexity reference methods were evaluated on exactly the same 22,878 independent final-test records as the spatial and machine-learning approaches: temporal linear interpolation and the station-specific monthly climatological mean. Before either baseline was calculated, all final-test precipitation values were masked simultaneously and the 876 naturally missing records remained unavailable. Thus, no withheld precipitation value contributed to another prediction or to the climatological statistics. Neither baseline required hyperparameter tuning [
7,
8].
For a target record at station s and date t, temporal linear interpolation used the nearest earlier and later non-test precipitation observations at the same station, occurring at t1 and t2, respectively. The estimate was calculated as follows:
Both bracketing observations were available for 22,869 of the 22,878 final-test records. For the nine boundary records lacking a valid observation on one side of the target date, the station-specific monthly climatological mean defined below was used as a fallback. No test record was removed from metric calculation.
The station-specific monthly climatological mean assigned each target record the arithmetic mean of all development-pool precipitation observations from the same station s and calendar month m:
where Ddev denotes the development pool, and N(s,m) is the number of available development observations for station s in month m. This baseline represents broad station-specific seasonality but contains no event-specific neighbouring-station, reanalysis, or geostatistical information.
2.3.4. Inverse Distance Weighting for ERA5-Land Transfer and Station-Based Gap Filling
Inverse Distance Weighting (IDW) was used in two distinct operations, both following the distance-decaying weighting expression in Equation (7) [
9]. First, the daily ERA5-Land variables were transferred from the regular grid to each meteorological station using the four nearest grid-cell centres. Euclidean distances were calculated in WGS84/UTM Zone 37N, the distance exponent was fixed at p = 2, and no fixed search radius was imposed because the four nearest grid cells were always available on the regular ERA5-Land grid.
Mathematically, the unknown value of Z(x_0) is shown in Equation (7):
where denotes the interpolated value at the target location, represents the observed values at neighbouring points, is the distance between the unknown and known points, is the distance weighting exponent, and is the number of observation points.
For the independent station-to-station IDW benchmark, the same equation was applied to available same-day MGM precipitation observations rather than ERA5-Land grid-cell values. The candidate distance exponents (1.5, 2.0, and 2.5), maximum neighbourhood sizes (4, 6, and 8 stations), and maximum search radii (120, 150, and 200 km) were selected entirely within the 85% development pool by five-fold cross-validation. The final configuration used p = 1.5, a maximum of eight neighbouring stations, and a 150 km search radius. If fewer than three stations were available within that radius, the three nearest available stations were used as a fallback so that an estimate could still be generated. Thus, the two IDW applications shared the same mathematical form but differed in their source points and parameter-selection rules.
2.3.5. Candidate Predictors and Their Scientific Rationale
Seventeen candidate predictors were defined a priori and grouped according to their temporal, neighbouring-station, geographic, topographic, reanalysis-based, and geostatistical roles. The temporal group consisted of the sine and cosine transformations of day of year (sin_doy and cos_doy), which represented the annual cycle, and the one- and two-day target-station precipitation lags (lag1 and lag2), which represented short-term temporal persistence. The lag variables corresponded strictly to precipitation observed at the target station at times t − 1 and t − 2; no future precipitation information was used. Within each cross-validation fold, the lag variables were regenerated after masking the validation observations so that the withheld target value could not enter its own predictor set.
The neighbouring-station group comprised same-day precipitation from the first and second nearest gauges (n1_pr and n2_pr). UTM easting and northing (x and y) were included as static station-location covariates so that the nonlinear learners could represent broad east-west and north-south geographic gradients across the station network. Geographic coordinates are one possible class of spatial covariates in machine-learning prediction, but proximity and autocorrelation are more directly represented by distance-based and geostatistical features [
29]. Accordingly, the predictor set also included distance to the nearest station (dist_n1), elevation difference from the nearest station (alt_diff_n1), same-day neighbouring rainfall (n1_pr and n2_pr), and the leakage-controlled leave-one-out Kriging estimate (kriging_feature). Distances from each station to the ERA5-Land grid-cell centres were calculated separately in UTM Zone 37N and were used only to construct the four-neighbour IDW weights described in
Section 2.3.4. Thus, x and y identify the station position within the regional domain; they do not encode ERA5-Land grid distance.
The ERA5-Land predictor group consisted of EQM-corrected precipitation (era_tp_cal), 2 m air temperature (era_t2m), 2 m dew-point temperature (era_d2m), derived relative humidity (era_rh), and surface pressure (era_sp). These variables were intended to represent the large-scale precipitation signal, atmospheric thermal state, absolute moisture content, proximity to saturation, and surface-pressure environment, respectively. The combination of environmental information from the target location and precipitation from surrounding gauges is consistent with machine-learning precipitation-recovery frameworks reported in the literature [
12]. All 17 predictors were retained; SHAP was subsequently used to quantify their relative attribution within the fitted full-hybrid model rather than to eliminate lower-ranked variables.
2.3.6. XGBoost Method
XGBoost (eXtreme Gradient Boosting) is a scalable implementation of gradient boosting decision trees introduced by Tianqi Chen and Carlos Guestrin (2016) [
15]. This method provides a powerful machine-learning approach for imputing missing daily precipitation data. XGBoost sequentially combines weak learners (typically decision trees) to minimize prediction errors while preventing overfitting through regularization techniques. The core principle of the method is to optimize the loss function using a second-order Taylor expansion. The objective function is shown in Equation (8):
where
is the loss function measuring prediction error,
denotes the
-th tree,
is the true value,
is the prediction from the previous iteration, and
is the regularization term. Here,
represents the number of leaves,
are leaf weights, and
and
are hyperparameters. For missing precipitation imputation, XGBoost is trained using available data, and missing values are replaced with model predictions. Based on the gradient boosting framework proposed by Friedman (2001) [
30], XGBoost can inherently handle missing values by automatically assigning them to optimal split directions.
2.3.7. LightGBM Method
LightGBM (Light Gradient Boosting Machine), developed by Microsoft and introduced by Ke et al. (2017) [
16], is a gradient boosting framework that offers faster training and lower memory usage compared to XGBoost, making it suitable for large-scale daily precipitation datasets.
LightGBM employs a histogram-based algorithm and leaf-wise tree growth. The gain is as shown in Equation (9):
where
are the gradient sums of left and right nodes,
are the corresponding Hessian sums, and
are regularization parameters. For missing precipitation data, LightGBM constructs a regression model using available data. Missing values are handled via a “default direction,” and efficiency is improved using Gradient-based One-Side Sampling (GOSS) and Exclusive Feature Bundling (EFB) [
16].
2.3.8. Random Forest (RF) Method
Random Forest (RF) is an ensemble learning method developed by Leo Breiman (2001) [
17]. It operates by combining multiple decision trees through the bootstrap aggregating (bagging) technique and provides a robust approach for filling missing daily precipitation data. Each tree is trained on a randomly sampled subset of the data, while features are selected randomly at each split. The prediction is obtained as the average of all trees, as given in Equation (10):
where
denotes the number of trees and
represents the
-th tree. In this study, Random Forest was used as a supervised learning model within the proposed precipitation imputation framework. It was employed both as a candidate regression model and, in the two-stage architecture, as the classifier for estimating wet-day probability. The RF models were trained and evaluated using the same leakage-controlled development and independent final-test framework applied to the other machine-learning methods.
Recent precipitation-imputation studies have demonstrated the applicability of Random Forest under different missing-data mechanisms and missing-data rates [
31]. In a separate sub-hourly rainfall study, RF was evaluated within both direct and two-step imputation frameworks; in the two-step approach, an RF classifier was used for rain/no-rain detection, followed by rainfall-depth estimation using alternative regression methods, including RF, under different neighbouring-station configurations [
32]. Together, these studies support the use of RF as a flexible nonlinear model for precipitation gap filling and provide methodological support for its evaluation within the two-stage framework adopted in the present study.
2.3.9. Neural Network (MLP) Method
The Multilayer Perceptron (MLP) is a fundamental form of artificial neural networks, popularized by David E. Rumelhart et al. (1986) [
33]. It is a regression-based approach capable of modelling non-linear relationships to estimate missing daily precipitation values.
An MLP consists of an input layer, one or more hidden layers, and an output layer. The forward propagation process is defined as in Equation (11):
where
denotes the activation function (e.g., ReLU), while
and
represent the weights and biases, respectively. Training is performed using backpropagation by minimizing the Mean Squared Error (MSE) loss function. For imputation, the MLP is trained using available data, and missing values are subsequently filled using model predictions.
2.3.10. SVR Method
Support Vector Regression (SVR) is the regression variant of Support Vector Machines developed by Vladimir Vapnik (1995) [
19,
20]. It provides robust predictions using the
-insensitive loss function and is resistant to outliers in precipitation data.
SVR solves the optimization problem given in Equation (12):
Constraint:
The kernel trick (e.g., Radial Basis Function, RBF: ) enables modelling of non-linear relationships. For imputation, the SVR model is trained using available data, and missing values are estimated through predictions.
2.3.11. ERA5-EQM Method
Quantile Mapping (QM), widely used in the statistical calibration of climate data, is an effective technique for correcting distributional biases in model outputs [
34,
35,
36]. In this study, the Empirical Quantile Mapping (EQM) approach was specifically adopted. Due to the highly skewed nature of daily precipitation and its large proportion of zero values (dry days), EQM was preferred over parametric methods that force the data into theoretical distributions (e.g., Gamma or Weibull). The EQM calibration functions were derived exclusively from the 85% development dataset, while the independent 15% test dataset remained completely excluded from the calibration process. This procedure was adopted to reduce potential information leakage and to ensure an unbiased evaluation of model performance. EQM directly matches the empirical cumulative distribution functions (eCDF) of station observations and ERA5 data via linear interpolation, allowing the empirical distribution of ERA5 precipitation to be adjusted toward the station-observed distribution without assuming a parametric form. To achieve this, EQM aligns these cumulative-distribution functions as expressed in Equation (13):
where
is the bias-corrected ERA5 precipitation amount,
is the raw ERA5 precipitation amount downscaled to the station location,
is the empirical cumulative distribution function of ERA5 precipitation, and
is the inverse empirical cumulative distribution function of observed station precipitation. The EQM functions were fitted only on the development/training data, and the independent final-test observations were not used in calibration.
2.3.12. Kriging Method
Kriging is a geostatistical interpolation method formalized by Matheron [
11]. It was used both as an independent deterministic method and as a spatial predictor in the hybrid framework. Station coordinates were transformed to UTM Zone 37N so that distances were expressed in metric units. The variogram family, maximum neighbourhood size, and search radius were selected within the 85% development pool by five-fold cross-validation. Candidate variogram families were spherical, exponential, and Gaussian; candidate search radii were 120, 150, and 200 km; and candidate maximum neighbourhood sizes were 4, 6, and 8 stations.
The Kriging procedure was purely spatial and was repeated independently for each target station-day; no temporal lag or spatiotemporal covariance term was included. For the standalone Kriging benchmark, the exponential variogram family, a maximum neighbourhood of eight stations, and a 120 km search radius were selected by five-fold cross-validation within the 85% development dataset. A single regional nugget, sill, or range was not imposed across all predictions. Because variogram parameters were not supplied explicitly to PyKrige, they were estimated adaptively from the available same-day local neighbouring observations for each Kriging fit. Zero precipitation values were retained as valid observations. If the selected values were effectively constant, including an all-zero neighbourhood, their mean was returned; if fewer than three usable neighbours were available or the Kriging fit failed, the corresponding IDW estimate was used. Negative Kriging estimates were constrained to zero. Thus, the procedure represents target-specific local spatial Kriging, whereas wet-day occurrence and positive rainfall amount were handled separately by the two-stage machine-learning framework.
The Ordinary Kriging estimator is expressed in Equation (14):
The weights are determined using the semivariogram, defined in Equation (15):
where is the estimated value at the unknown location, are observed values at known locations, are weighting coefficients, is the number of neighbouring observations, is the distance between two points, is the semivariogram (a measure of spatial dependence), and denotes the expectation operator. Missing values are interpolated using neighbouring observations.
A pooled same-day empirical semivariogram was calculated by grouping semivariances from station pairs observed on the same date into 12 distance classes across the study period. The fitted curve was used only as a descriptive summary; local Kriging predictions used the cross-validated exponential family with adaptively fitted parameters.
2.3.13. Two-Stage Wet-Day Probability and Rainfall Amount Modelling
Because daily precipitation data are zero-inflated and highly right-skewed, the machine-learning framework was implemented as a two-stage architecture. In the first stage, a Random Forest classifier estimated the probability of a wet day using the retained predictors. In the second stage, the positive rainfall amount was estimated using the optimized regression model. The regression target was transformed using log1p, defined as log1p(y) = ln(1 + y), where y is precipitation in millimetres. This transformation is defined at y = 0 and compresses large positive values, thereby reducing right skewness and stabilizing model fitting. Predicted values were returned to the original precipitation scale using the inverse function expm1(z) = exp(z) − 1. The final precipitation estimate was obtained by multiplying the predicted wet-day probability by the inverse-transformed positive rainfall amount. This structure reduced the tendency of regression models to convert dry days into artificial low-intensity rainfall and improved the physical consistency of the imputed series.
The final two-stage estimate can be written as in Equation (16):
where
is the final estimated precipitation amount in mm,
is the predicted wet-day probability for the predictor vector
, and
is the model prediction in the log-transformed precipitation space.
2.3.14. Testing Method
An artificial missing-data experiment was used for controlled evaluation. Complete station-day records were assigned to five precipitation-intensity strata: dry days (≤0.1 mm) and four wet-day classes defined by quartiles of positive precipitation. The strata contained 116,667 dry records and 9587, 8379, 8968, and 8921 records in the four successive wet-day classes. With random seed 42, 15% of each stratum was assigned to the 22,878-record final-test set and the remaining 129,644 records formed the development pool.
The development pool was partitioned into five shuffled stratified folds at the individual station-day level. Dates were not blocked and stations were not held out as groups. In each fold, validation precipitation values were masked before lag, neighbouring-station, ERA5-EQM, and Kriging predictors were regenerated, preventing a target value from entering its own predictors while preserving same-day observations at other stations.
All IDW and Kriging grid configurations and the predefined machine-learning searches were evaluated using the same five folds. The configuration with the lowest mean validation RMSE was refitted on the complete development pool and evaluated once on the untouched final-test set.
The evaluation represents isolated or irregular station-day gaps for which contemporaneous observations at other stations may remain available. Long continuous gaps, complete-station holdouts, and network-wide outages require separate blocked validation designs.
2.3.15. Accuracy Metrics
To evaluate daily precipitation gap-filling performance objectively, four complementary metrics were used: RMSE, MAE, Pearson correlation coefficient (r), and Nash-Sutcliffe Efficiency (NSE). RMSE and MAE quantified error magnitude, Pearson r described linear covariation, and NSE assessed predictive skill relative to the observed mean.
The Root Mean Square Error (RMSE) represents the square root of the mean of squared differences between observed and estimated values. Due to the squaring mechanism, RMSE penalizes larger errors more heavily and is therefore particularly useful for identifying methods that produce substantial estimation errors in precipitation datasets. Lower RMSE values indicate better model performance, with a value of zero representing a perfect match between observed and estimated values.
The Root Mean Square Error (RMSE) was calculated as Equation (17):
The Mean Absolute Error (MAE) is defined as the average of the absolute differences between observed and predicted values. Unlike RMSE, it treats all errors equally and is less sensitive to outliers, providing a robust measure of the average magnitude of prediction errors. Lower MAE values indicate higher prediction accuracy, and a value of zero corresponds to perfect agreement between observations and estimates.
The Mean Absolute Error (MAE) was calculated as Equation (18):
The Pearson correlation coefficient (r) measures the strength and direction of the linear relationship between observed and predicted precipitation. It ranges from −1 to 1, with values closer to 1 indicating stronger positive linear agreement.
The Pearson correlation coefficient (r) was calculated as Equation (19):
The Nash–Sutcliffe Efficiency (NSE) assesses the relative magnitude of residual variance compared to the variance of the observed data. It ranges from −∞ to 1, where a value of 1 indicates perfect agreement between observed and predicted values. Values greater than 0 indicate that the model performs better than the observed mean, whereas values below 0 suggest that the observed mean provides a better predictor than the model. Therefore, NSE is widely used to evaluate the overall predictive skill of precipitation gap-filling methods, with values approaching 1 indicating superior predictive performance.
The Nash-Sutcliffe Efficiency (NSE) was calculated as Equation (20):