Sensitivity Analysis to Investigate the Reliability of the Grid-Based Rainfall-Runoff Model

Abstract

This study aimed to assess the suitability of the parameters of a physically based, distributed, grid-based rainfall-runoff model. We analyzed parameter sensitivity with a dataset of eight rainfall events that occurred in two catchments of South Korea, using the Sobol’ method. Parameters identified as sensitive responded adequately to the scale of the rainfall events and the objective functions employed. Parameter sensitivity varied depending on rainfall scale, even in the same catchment. Interestingly, for a rainfall event causing considerable runoff, parameters related to initial soil saturation and soil water movement played a significant role in low flow calculation and high flow calculation, respectively. The larger and steeper catchment exhibited a greater difference in parameter sensitivity between rainfall events. Finally, we found that setting an incorrect parameter range that is physically impossible can have a large impact on runoff simulation, leading to substantial uncertainty in the simulation results. The proposed analysis method and the results from our study can help researchers using a distributed rainfall-runoff model produce more reliable analysis results.

1. Introduction

The distributed rainfall-runoff model simulates runoff on a grid, taking into account the spatial variability in physical properties of a catchment using geographic information system (GIS) techniques and the spatiotemporal variability of rainfall. Because of these advantages, this model is suitable for simulating rainfall-runoff from the common view [1] and aim to improve the simulation of hydrological process in the catchment [2]. For this reason, the distributed rainfall-runoff model is becoming increasingly popular in water resource planning and management [3,4], as well as in various other research fields [5,6,7,8,9,10,11,12,13]. In particular, Soil and Water Assessment Tool (SWAT) [14] and Topography based hydrological Model (TOPMODEL) [15] are widely used, because the former has the advantage of simulating hydrology, plant growth, sediments, nutrients, and pesticides in watersheds, and the latter has the advantage of simulating the catchment’s observed flow more appropriately, by simulating the dynamically variable slope of the catchment.

The grid-based rainfall-runoff model (GRM) [16] is a physically based, distributed rainfall-runoff model for simulating short-term flood events. This model uses GIS techniques such as a digital extraction model (DEM), soil map, and land use cover. This model has been used for various studies [16,17,18], and is used as an important model for flood simulation in Korean catchments.

The parameters of the distributed rainfall-runoff model simulate catchment runoff through calibration. However, estimated parameter values may not be true values that completely represent the hydrological process in the catchment, and are inevitably uncertain [19]. It is noteworthy that the parameters of the physically based distributed rainfall-runoff model can theoretically be estimated using observation data; thus, it does not require manual or automatic parameter calibration. However, parameters are generally calibrated because of errors in the observation data, missing data required for parameter estimation, and problems arising from the difference in grid scales between observation data and the rainfall-runoff model [20,21,22,23,24,25,26]. Uncertainty in parameters can be caused by extreme interaction between parameters or parameter non-uniqueness [27]; such uncertainty can cause an equifinality problem [28,29]. Parameter uncertainty is one of the most important uncertainty factors in hydrologic modeling; therefore, it is hard to construct accurate hydrologic predictions without parameter uncertainty assessment [30].

Parameter sensitivity analysis (SA) is a valuable tool for assessing important parameters [31], and is particularly important in distributed rainfall-runoff model assessment [32] and hydrological research [19]. Parameter sensitivity analysis has the following goals: (1) understand parameter characteristics that simulate the hydrologic system of catchments; (2) estimate appropriate parameter ranges and values; (3) improve the rainfall-runoff model; and (4) reduce uncertainties in the model and simulation results [33,34,35]. Sobol’ [36], Morris [37], and Regional Sensitivity Analysis [38] methods are widely used SA methods.

This study aimed to analyze sensitivity of parameters of the GRM, in order to investigate the reliability of simulation results. Parameter sensitivity was analyzed using the Sobol’ method, which is a global sensitivity analysis method [36], and data of eight rainfall events that occurred in Danseong and Seonsan catchments in South Korea. In the Sobol’ method, the parameter sensitivity index considers the interaction between parameters and allows direct interpretation. These benefits have made this method increasingly popular in recent hydrological studies [39]. The purpose of this study was: (1) to analyze the characteristics of GRM parameters and assess their suitability for different rainfall event scales and objective functions; (2) to analyze the effect of catchment characteristics on the sensitivity of parameters; and (3) to analyze the effect of GRM parameter range on parameter sensitivity, where it may affect the uncertainty of the simulation results.

2. Catchment and Data

The study sites were the Danseong and Seonsan catchments in South Korea (Figure 1). These catchments are based in major tributaries of the Nakdong river, which flows through southeast South Korea. The Danseong catchment is the first catchment in Korea to have a sudden flood forecasting system, due to frequent floods. Therefore, this catchment is an area where flood modeling is important. The Seonsan catchment is a small catchment about half the size of the Danseong catchment, but flood defenses are important because it includes urban areas. Therefore, the Danseong and Seonsan catchments are areas where various studies are needed. The Danseong and Seonsan catchment have different catchment characteristics. The areas of Danseong and Seonsan catchments are 1709 km² and 977 km², respectively. Approximately 73% of the Danseong catchment comprises steep mountains (mean slope 16.2%, maximum elevation 1834 m, minimum elevation 47 m), and 21% of the area near the river is farmland. The Seonsan catchment is gentler than Danseong catchment (mean slope 12.8 %, maximum elevation 1292 m, minimum elevation 38 m). Approximately 65% of the Seonsan catchment area is low mountains, 28% is farmland, and 4% is urban.

The rainfall data used in this study are eight rainfall events that occurred in Danseong and Seonsan catchments (

Table 1. Storm events for each catchment.

Catchment	Event Number	Periods	Rainfall (mm)	Peak Flow (m3/s)
Danseong	1	2011.07.03, 20:00– 2011.07.05, 12:00	43	687
	2	2012.07.14, 15:00– 2012.07.16, 15:00	63	1232
	3	2012.09.16, 17:00– 2012.09.18, 07:00	225	11,970
	4	2015.07.12, 01:00– 2015.07.14, 01:00	123	2173
Seonsan	1	2010.08.11, 00:00– 2010.08.12, 20:00	66	1361
	2	2010.09.01, 20:00– 2010.09.03, 12:00	43	491
	3	2011.07.09, 11:00– 2011.07.12, 20:00	154	1814
	4	2012.09.16, 15:00– 2012.09.19, 00:00	198	3542

). The average rainfall data for each catchment were generated by the Thiessen Polygon method, using the observed rainfall of the rainfall stations (Figure 1). The rainfall stations are operated by the Ministry of Land, Infrastructure and Transport (http://www.molit.go.kr). The time interval of the data was 10 min, and the precipitation amounts varied from 43 mm to 225 mm. The precipitation time distribution of the rainfall events, and resulting hydrographs, show substantial variation (Figure 2).

3. Material and Sensitivity Analysis Method

3.1. Grid-Based Rainfall-Runoff Model

The GRM is an event based rainfall-runoff model. It simulates overland flow, channel flow, infiltration, subsurface flow, percolation, and baseflow (Figure 3). Runoff is calculated using the kinematic wave equation, and infiltration is calculated using the Green-Ampt model. The GRM can properly simulate short-term flood events using a short simulation time interval, such as a 10 min time interval (e.g., in References [17,18]). The governing equations of GRM are as follows.

$$\frac{\partial h}{\partial t}+\frac{\partial q}{\partial x}=r-f+\frac{q_r}{\Delta y}$$

(1)

$$\frac{\partial A}{\partial t}+\frac{\partial Q}{\partial x}=r\Delta y+q_L+q_{ss}+q_b$$

(2)

$$S_0=S_f$$

(3)

where h is flow depth, q is flow rate per unit width, r is rainfall intensity, f is infiltration rate, q_r is return flow into the overland flow, ∆y is the width of control volume, A is channel cross-sectional area, Q is discharge in the channel, q_L is lateral flow from overland flow, q_ss is subsurface flow, q_b is baseflow, S₀ is surface slope, and S_f is friction slope. Input data for the GRM include hydrological topographic factors (slope of each grid and flow direction) extracted by a digital elevation model (DEM), and soil and land cover data. These data are inputted in raster file format. In this study, we converted the 30 m × 30 m DEM provided by the National Geographic Information Institute (http://www.ngii.go.kr) into 500 m × 500 m data to generate input topographic data of the target catchments. For land cover data, the upper-category land cover map of the Ministry of Environment (http://www.me.go.kr) was used. Soil texture and soil depth maps were produced using the high-precision soil map provided by the National Academy of Agricultural Science (http://www.naas.go.kr), in order to apply the Green-Ampt infiltration model.

Table 2. Parameters of the GRM model.

Number	Parameter	Lower	Upper	Unit	Description
1	ISSR	0	1	-	Initial soil saturation ratio
2	MSLS	0.0001	0.01	-	Minimum slope of land surface
3	MSCB	0.0001	0.01	-	Minimum slope of channel bed
4	CRC	0.008	0.2	-	Channel roughness coefficient
5	CLCRC	0.6	1.3	-	Correction factor for land cover roughness coefficient
6	CSD	0.8	1.2	-	Correction factor for soil depth
7	CSP	0.9	1.1	-	Correction factor for soil porosity
8	CSWS	0.25	4	-	Correction factor for soil wetting front suction head
9	CSHC	0.05	20	-	Correction factor for soil hydraulic conductivity

shows the parameters used in the GRM. Note that all grids had different attribute/parameter values extracted from GIS data. The parameters related to land cover and soil of the GRM collectively adjusted the attribute/parameter values of these grids using ratios.

3.2. The Sobol’ Method

The Sobol’ method is a variance-based global sensitivity analysis method, which is currently widely used across various research fields [2,41,42,43,44,45,46,47]. This approach quantifies the relative effect of a parameter and its interaction with other parameters on model output variance, using the decomposition of model output variance.

This method calculates the first-order sensitivity index (FSI) and total sensitivity index (TSI). The FSI indicates the sensitivity of each parameter in relation to the model output, and the TSI is the sum of a parameter’s FSI and the sensitivity of the parameter’s interaction with other parameters influencing the model output [48]. Therefore, if there is no interaction between parameters, the sum of all parameters’ FSIs becomes 1. The FSI and TSI of parameters range between 0 and 1. In this study, we analyzed GRM parameter sensitivity using the latter, because TSI produces more suitable results than FSI when analyzing the general effect of each parameter on the model output [48]. The TSI is defined below [49].

$$TSI=\frac{E_{X_{~i}}\left(V_{X_i}\left(Y|X_{~i}\right)\right)}{V\left(Y\right)}$$

(4)

where X_i is the ith parameter X_~i represents the vector of all parameters not including X_i, and Y is the scalar objective function value. The inner variance of the numerator means that the variance of Y is considered over all possible values of X_i while maintaining X_~i fixed. The outer expectation of the numerator is considered over all possible values of X_~i. Thus, the numerator of TSI represents the expected variance that would be left if all parameters were fixed, not including X_i [49]. The variance of Y in the denominator indicates total unconditioned variance.

Saltelli’s scheme [50] was used for parameter sampling. This scheme shortens the time required for sensitivity calculation by reducing the number of samples from n × (2k + 2) to n × (k + 2). Here, n represents the initial sample size (10,000 in this study), and k is the number of parameters in the model. Therefore, the total number of samples used for a single sensitivity analysis of the GRM with nine parameters is 110,000. Latin Hypercube sampling was employed for the initial sampling, further details of which can be found in Shin et al. [51]. The R ‘sensitivity’ package [52] was used for the sensitivity analysis.

Three objective functions were used to analyze parameter sensitivity for high flow, medium flow, and low flow (low flow is baseflow of the recession curve during an event). First, the Nash-Sutcliffe efficiency (NSE) [53] was modified to analyze parameter sensitivity for high flow. The NSE function is expressed as:

$$NSE=1-\frac{{\displaystyle {\sum }_{i=1}^n{\left(Q_{obs,i}-Q_{sim,i}\right)}^2}}{{\displaystyle {\sum }_{i=1}^n{\left(Q_{obs,i}-\overline{Q_{obs}}\right)}^2}}$$

(5)

where Q_obs,i is the observed flow with time interval i (10 min in this study), $\overline{Q_{obs}}$ is the mean value of observed flow, Q_sim,i is the simulated flow, and n is the number of time steps. NSE ranges from −∞ to 1, where 1 indicates a perfect match between observed flow and simulated flow. NSE squares the difference between observed flow and simulation flow values; thus, a relatively greater weight is given to high flow fitting on the hydrograph [54,55,56]. However, the TSI value can turn into a bias due to a large negative NSE value, because NSE values range from −∞ to 1. For this reason, we employed NSE* [57], modified from NSE. The NSE* objective function is expressed as:

$${NSE}^{\ast }=\left\{1-\frac{{\displaystyle {\sum }_{i=1}^n{\left(Q_{obs,i}-Q_{sim,i}\right)}^2}}{{\displaystyle {\sum }_{i=1}^n{\left(Q_{obs,i}-\overline{Q_{obs}}\right)}^2}}\right\}/\left\{1+\frac{{\displaystyle {\sum }_{i=1}^n{\left(Q_{obs,i}-Q_{sim,i}\right)}^2}}{{\displaystyle {\sum }_{i=1}^n{\left(Q_{obs,i}-\overline{Q_{obs}}\right)}^2}}\right\}$$

(6)

NSE* values range from −1 to 1. The NSE* objective function reduces the effect of a large negative NSE value on sensitivity without modifying the analysis of NSE values. When calculating NSE through a log transformation of observed and simulated data (NSElog), low flow fitting is given more weight [54,56]. Thus, NSElog*, modified from NSElog, was used for parameter sensitivity analysis of low flow. Finally, 0.5(NSE* + NSElog*), which gives equivalent weights to NSE* and NSElog*, was used to analyze parameter sensitivity for medium flow. These three objective functions were previously employed in the study of Shin et al. [51].

We first analyzed the effects of rainfall events on the sensitivity of parameters, using the various rainfall events and catchments mentioned above. We then analyzed whether the parameters reacted appropriately for various objective functions. Third, we analyzed the influence of the characteristics of the catchment on the sensitivity of the parameters. Finally, we analyzed the effect of the range of parameters on the sensitivity of parameters. The sensitivity analysis of the parameters was analyzed using the parameter values generated by the Latin Hypercube sampling method, so calibration and validation of the parameters for sensitivity analysis of the parameters was not performed.

4. Results and Discussion

4.1. Parameter Sensitivity Analysis Reflecting the Scale of Rainfall Events

Figure 4 and Figure 5 show parameter TSI values for four rainfall events in Danseong and Seonsan catchments for the three objective functions. Parameter sensitivity varied depending on the scale of rainfall events, even in the same catchment. The rainfall events of Danseong catchment showed greater parameter sensitivity differences than those of Seonsan catchment. Considering the fact that the precipitation amount during events was similar in both catchments (Table 1), the parameter sensitivity was likely affected by the runoff difference caused by different catchment characteristics (see Section 2).

In Danseong catchment, the TSIs of ISSR and CSHC (parameters related to soil water content, Table 2) of the fourth rainfall event (Danseong 4) were lower than those of the first (Danseong 1) and second (Danseong 2) rainfall events. Therefore, the fourth and largest rainfall event in Danseong catchment (Danseong 4) exhibited relatively low sensitivity of soil water content-related parameters. This means that the surface runoff of a river had a greater impact on total runoff than runoff from the soil in larger-scale rainfall events. This phenomenon is physically reasonable, and indicates that these parameters reacted appropriately to the different scales of rainfall events. This result is also supported by the fact that the TSIs of MSCB and CRC (river channel-related parameters, Table 2) in the Danseong 4 rainfall event were greater than those of Danseong 1 and Danseong 2. This phenomenon was also observed in Seonsan catchment; thus, the GRM effectively simulates the runoff of various catchments and rainfall events.

It is notable that Danseong 3 had a greater TSI of ISSR for initial soil saturation than Danseong 4, although the runoff volume of Danseong 3 was far greater than that of Danseong 4 (Figure 2). This phenomenon was observed with all objective functions, and the TSI of ISSR was higher using the low flow objective function (NSElog*) than when using that for high flow (NSE*). This suggests that the initial soil saturation is particularly important when calculating low flow during large rainfall events with very high runoff. In NSE*, Danseong 3 had a greater TSI of CSHC for soil water movement than Danseong 4, which indicates that this parameter plays an important role in high flow calculations of large rainfall events with very high runoff. As for MSCB and CRC, Danseong 3 had smaller TSI values than Danseong 4. This is probably because total runoff simulations consider the interaction between ISSR, MSCB, and CRC. For example, the TSI of river runoff-related MSCB and CRC decreases when the TSI of ISSR related to initial soil saturation increases.

4.2. Parameter Sensitivity Analysis Reflecting Different Objective Functions

Figure 6 and Figure 7 show the TSI of parameters for Danseong and Seonsan catchments, calculated for the three objective functions. The two soil-related parameters (ISSR and CSHC) and the two river-related parameters (MSCB and CRC) were highly sensitive in both catchments. Therefore, these four parameters must be estimated more carefully when estimating optimum parameter values. However, CSD was less sensitive and the remaining parameters (MSLS, CLCRC, CSP, and CSWS) exhibited minimal sensitivity. These insensitive parameters were not sensitive to all objective functions; thus, they can be fixed to specific values for Danseong and Seonsan catchments, as suggested by Shin et al. [51]. However, these parameters may be sensitive in other catchments with different characteristics and rainfall events. A preliminary sensitivity analysis is indispensable when applying the GRM to other catchments.

The soil-related ISSR and CSHC parameters were more sensitive to high flow than to low flow in Danseong and Seonsan catchments. This is reasonable because the range of change of the hydrograph due to the variation of soil water content is greater during high flow than low flow for normal rainfall events. However, for a large rainfall event with very high runoff (Danseong 3), the parameter for initial soil saturation (ISSR) had a greater effect on low flow simulation. Conversely, the river-related MSCB and CRC parameters were typically more sensitive to low flow than high flow. This is reasonable because low flow is more sensitive to friction due to river roughness than high flow. In summary, it was concluded that these parameters functioned appropriately according to the objective functions.

4.3. Parameter Sensitivity Analysis Reflecting Different Catchment Characteristics

The sensitivity of the parameters may vary depending on the catchment characteristics. In this section, we analyzed the effect of catchment characteristics on the sensitivity of parameters. Figure 8 shows the maximum and minimum TSI values of parameters among the four rainfall events in Danseong and Seonsan catchments. When the TSI values of parameters lie closer to the diagonal line, the maximum and minimum TSIs have similar values. TSI values of the parameters for Danseong catchment, calculated for all three objective functions, were farther from the reference line than those for Seonsan catchment. Danseong catchment is approximately twice as large as Seonsan catchment and has steeper mountains. Although there was a slight gap between the rainfall volumes in the two catchments (Table 1), the GRM parameter sensitivity changes were greater in the larger and steeper catchment than in the smaller, gentler catchment.

4.4. Effect of Parameter Range on Sensitivity

Figure 9 and Figure 10 show the sensitivity results of the parameters for NSE*, using the parameter range suggested in Table 2 (original range), except for the parameters of MSLS and MSCB, which used a modified range with both MSLS and MSCB upper limits changed to 1 (45°) (wide range) (i.e., parameter ranges of MSLS and MSCB were all 0.0001–1). In the kinematic wave model used in the GRM, surface slope was used for the hydraulic gradient. These two parameters were employed to solve calculation errors caused by no transmission of the flood wave, due to extremely low land surface and channel bed slopes in the catchments. The reason for modifying the range of these two parameters was to test the effect of an artificially fixed slope value within the parameter range through pre-treatment of the DEM (fill sinks and remove flat areas) used to establish GRM input data on the simulation results.

As a result, the TSI of MSLS was closer to 0 in the original range, but mostly over 0.2 in the modified range. This is because MSLS was a parameter for setting the minimum gradient for overland flow in most areas of the catchment. When this value becomes unrealistically large, it will have a considerable impact on runoff simulation in the catchment. Conversely, the TSI values of the other parameters typically decreased, except for ISSR, which is likely because the effect of the other parameters on runoff decreased as the sensitivity of MSLS increased.

Interestingly, the TSI value of MSCB decreased although the range of the MSCB parameter increased. The river grid influenced by MSCB accounted for 29% and 36% of the total grids of Seonsan and Danseong catchments, respectively. This was probably because it accounted for a smaller ratio than the catchment surface grid affected by MSLS; thus, the effect of MSCB on runoff decreased. However, setting 45° as the upper limit of MSLS and MSCB was not physically valid. Setting a physically invalid range for even one or two GRM parameters can not only substantially change the sensitivity of the parameter in question, but also that of other parameters, and can produce erroneous runoff simulation results. Therefore, the parameter range should be carefully determined by hydrology specialists to ensure physical validity.

5. Conclusions

In this study, we analyzed parameter sensitivity using the Sobol method in order to assess the suitability of the parameters of the GRM, which is a physically based, distributed rainfall-runoff model. Eight rainfall events in two different sized catchments in South Korea were used to analyze parameter sensitivity according to their different catchment and rainfall characteristics, and to determine parameter suitability.

As a result, parameter sensitivity varied depending on the scale of rainfall events, even in the same catchment. The size and characteristics of the catchment also had an effect on parameter sensitivity in different rainfall events. This result supports the results of van Griensven et al. [35], that the sensitivity of the parameters varies between catchments. Sensitive parameters adequately responded to the different scales of rainfall events and the objective functions in both catchments; thus, we concluded that the GRM model can produce reliable simulation results. The parameter ISSR, which is related to the initial soil saturation, was the most sensitive parameter for most catchments and rainfall events. River runoff-related parameters, MSCB and CRC, and the subsurface runoff-related parameter, CSHC, were identified as sensitive. These four sensitive parameters are important parameters for parameter estimation. The remaining parameters (CSD, MSLS, CLCRC, CSP, and CSWS) had less sensitivity for all objective functions; therefore, they can be fixed to specific values, at least for these study catchments, as suggested by Shin et al. [51]. However, the sensitivity of the parameters depends on catchment characteristics and rainfall events, so a preliminary sensitivity analysis is indispensable when applying the GRM to other catchments. Interestingly, parameters related to initial soil saturation and soil water movement played an important role in low flow simulation and high flow simulation respectively, in rainfall events with very high runoff. It was also found that the difference of parameter sensitivity for various rainfall events increased for the larger, steeper catchment. Finally, a physically impossible parameter ranges for the two parameters, related to catchment and river slopes, greatly influenced runoff simulation results. This implies that an incorrect parameter range can increase the probability of errors in runoff simulation results or uncertainties when analyzing runoff results. Therefore, it is important to set the parameter range based on physical validity.

In this study, it was important to evaluate the sensitivity of all the parameters used in the GRM, unlike previous studies (e.g., references [16,17]) that used only a few parameters of the GRM to calibrate the parameters. The results of this study can be used as an important guideline for researchers who use the GRM as to which parameters should be most carefully calibrated. In addition, the methods and results in this study can be an important precedent study for researchers who want to evaluate the sensitivity of parameters of distributed rainfall-runoff models other than the GRM. We analyzed parameter sensitivity for eight rainfall events in two South Korean catchments. In order to analyze parameters identified as not sensitive in this study, it is necessary to analyze the sensitivity of more catchments and rainfall events. However, the sensitivity analysis methods and results presented in this study can help researchers using the physically based distributed rainfall-runoff model to produce more reliable results.