# LENS-GRM Applicability Analysis and Evaluation

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Study Area

#### 2.2. Rainfall Events

#### 2.3. Limited Area ENsemble Prediction System (LENS)

#### 2.4. Grid Based on Rainfall-Runoff Model (GRM)

^{3}/s in September 2016.

#### 2.5. Likelihood for GLUE

^{2}= L

_{1}, NSE = L

_{2}, PBIAS = L

_{3}and log-normal = L

_{4}were used for readability.

^{2}was calculated as the square of the Pearson correlation coefficient r, representing the ratio of the total variance of the observed values to the simulated values. R

^{2}has a value from 0.0 to 1.0 and the closer R

^{2}= 1, the better the relationship [12]. R

^{2}is calculated as shown in Equation (1), where N represents the number of data points, O

_{t}and P

_{t}represent the actual rainfall and LENS rainfall at time t and the average value of the actual rainfall and LENS rainfall.

_{t}and P

_{t}represent the actual rainfall and LENS rainfall at time t, respectively, and is the average value of the actual rainfall.

_{t}

^{obs}and Y

_{t}

^{sim}represent the actual rainfall and LENS rainfall at time t.

^{2}is the variance of the simulated value, which is equal to Equation (5), ε

_{j}(θ) is the vector value of the residual, the difference between the simulated value and the observed value over time t, and is equal to Equation (6).

#### 2.6. Behavioral Model Selection and Uncertainty Assessment

_{2}, L

_{3}and L

_{4}. Behavioral models were selected in the range L

_{2}> 0.65 [17] L

_{3}< |25| [18] and the top 30% L

_{4}[19] by likelihood. To calculate the optimal parameter value, a sensitivity analysis was conducted using the random parameter value and the likelihood function and the parameter sensitivity analysis was conducted using the cumulative histogram with the parameter value selected as the behavior model. In addition, the reliability and confidence interval of the behavior model were calculated using a 95% prediction uncertainty (95PPU) analysis and the best simulation results for each likelihood are shown. The process used for uncertainty evaluation is illustrated in Figure 6.

## 3. Results

#### 3.1. LENS Point and Area Applicability Analysis

^{2}(L1), NSE(L2) and PBIAS(L3). Table 4 and Table 5 present numerical information on LENS point value prediction results and observation values for each event. The numerical results of LENS area value prediction results and observation values for each event are included in Table 6 and Table 7.

_{1}, L

_{2}and L

_{3}showed higher point values.

_{1}, L

_{2}and L

_{3}. The total rainfall difference, peak rainfall difference and peak time difference showed slight differences and a high degree of accuracy.

_{1}, L

_{2}and L

_{3}showed higher area values and the difference in rainfall prediction was insignificant, indicating a high accuracy.

_{1}, L

_{2}and L

_{3}and the difference in rainfall prediction is highly accurate at both points and areas. In the box plot, the quartile range tended to slightly overestimate the observed values.

_{1}, L

_{2}and L

_{3}showed a higher area value and, in the difference in rainfall prediction, the area value was highly accurate with respect to the difference between time and peak value.

_{1}, L

_{2}and L

_{3}suitability calculation results.

#### 3.2. Event.W Parameter Post-Distribution by Likelihood

_{2}-based behavioral model, L

_{2}> 0.65 was used as a threshold through calculation of parameters for each GRM likelihood. The number of L

_{2}-based behavioral models was 1121 (56.05%). Among the L

_{2}-based behavioral models, the sub-light coefficient and initial saturation parameters were the most sensitive. The ranges of the L

_{2}-based behavioral models for each parameter are presented in Table 8. In addition, to confirm the distribution of the L

_{2}-based behavioral models by parameter, it is expressed as shown in Figure 8, using the cumulative distribution function (CDF) and histogram.

_{3}-based behavioral model selection, |L

_{3}| < 25 was used as a threshold by calculating the parameters for each GRM likelihood. The number of L

_{3}-based behavioral models was 1151 (57.55%). Among the L

_{3}-based behavioral models, the most sensitive parameter was the initial saturation. The ranges of the L

_{3}-based behavioral models for each parameter are presented in Table 9. In addition, to confirm the distribution of the L

_{3}-based behavioral models by parameter, it is expressed as shown in Figure 8, using the cumulative distribution function (CDF) and histogram.

_{4}-based behavioral model was selected as the threshold for the top 30% by calculating the parameters for each GRM likelihood. The number of L

_{4}-based behavioral models was 600 (30%) [7]. The L

_{4}-based behavioral model was sensitive to the initial saturation, river roughness coefficient and soil depth. Table 10 shows the range of the behavioral model based on each parameter. In addition, to confirm the distribution of L

_{4}-based behavioral models by parameter, it is expressed as shown in Figure 8, using the cumulative distribution function (CDF) and a histogram.

_{2}= 1121, L

_{3}= 1151 and L

_{4}= 600. Accordingly, the cumulative distribution function (CDF) over time was calculated for the simulation discharge simulation value (Simulation Discharge) of the behavior model and 95% reliability analysis was conducted through the cumulative probability distribution.

_{2}-based behavior model was shown as a green range, the L

_{2}95 PPU optimum value “best simulation” was shown as a red dotted line and the observed flow rate was shown as a solid blue line. The L

_{2}-based behavioral model outflow simulation value was very similar to the observed flow rate. The L

_{2}value of Best Simulation was 0.9421, the highest value-based L

_{3}was |0.8081| and L

_{4}was −1924.25.

#### 3.3. Application and Evaluation of the GRM Model

_{1}, L

_{2}and L

_{3}were higher at the point value, whereas L

_{1}showed a higher fit of 0.8 L

_{2}of 0.6.

_{1}, L

_{2}and L

_{3}showed a higher fit with the area values.

_{1}, L

_{2}and L

_{3}showed a very high suitability for both the points and areas in Table 12 and Table 13.

_{2}was also found to show a lower suitability compared to all data.

## 4. Discussion

## 5. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Bauer, P.; Auligné, T.; Bell, W.; Geer, A.; Guidard, V.; Heilliette, S.; Kazumori, M.; Kim, M.-J.; Liu, E.H.-C.; McNally, A.P.; et al. Satellite cloud and precipitation assimilation at operational NWP centres. Q. J. R. Meteorol. Soc.
**2011**, 137, 1934–1951. [Google Scholar] [CrossRef][Green Version] - Korea Meteorological Administration Republic of Korea (KMA). Joint WMO Technical Progress Report on the Global Data Processing and Forecasting System and Numerical Weather Prediction Research Activities for 2016; Korea Meteorological Administration Republic of Korea (KMA): Seoul, Republic of Korea, 2016.
- Lee, M.; Kang, N.; Kim, J.; Kim, H.S. Estimation of optimal runoff hydrograph using radar rainfall ensemble and blending technique of rainfall-runoff models. J. Korea Water Resour Assoc.
**2018**, 51, 221–233. [Google Scholar] - Ministry of Construction and Transportation Gyeongsangbuk-do. The Basic Plan for River Maintenance in Wicheon; Ministry of Construction and Transportation: Sejong, Republic of Korea, 1998. [Google Scholar]
- Lee, S.-M.; Kang, H.-S.; Kim, Y.-H.; Byun, Y.-H.; Cho, C. Verification and Comparison of Forecast Skill between Global Seasonal Forecasting System Version 5 and Unified Model during 2014. Atmosphere
**2016**, 26, 59–72. [Google Scholar] [CrossRef][Green Version] - Lee, B.-J.; Bae, D.-H.; Jeong, C.-S. Runoff Curve Number Estimation for Cover and Treatment Classification of Satellite Image(II):—Application and Verification. J. Korea Water Resour. Assoc.
**2003**, 36, 999–1012. [Google Scholar] [CrossRef][Green Version] - Seong, Y.; Choi, C.-K.; Jung, Y. Assessment of Uncertainty in Grid-Based Rainfall-Runoff Model Based on Formal and Informal Likelihood Measures. Water
**2022**, 14, 2210. [Google Scholar] [CrossRef] - Horton, R.E. The Rôle of infiltration in the hydrologic cycle. Trans. Am. Geophys. Union
**1933**, 14, 446–460. [Google Scholar] [CrossRef] - Dunne, T.; Black, R.D. An Experimental Investigation of Runoff Production in Permeable Soils. Water Resour. Res.
**1970**, 6, 478–490. [Google Scholar] [CrossRef] - Bras, R.L. Hydrology: An Introduction to Hydrologic Science; Addison Wesley Publishing Company: Boston, MA, USA, 1990. [Google Scholar]
- Jin, X.; Xu, C.-Y.; Zhang, Q.; Singh, V. Parameter and modeling uncertainty simulated by GLUE and a formal Bayesian method for a conceptual hydrological model. J. Hydrol.
**2010**, 383, 147–155. [Google Scholar] [CrossRef] - Krause, P.; Boyle, D.P.; Bäse, F. Comparison of different efficiency criteria for hydrological model assessment. Adv. Geosci.
**2005**, 5, 89–97. [Google Scholar] [CrossRef][Green Version] - Nash, J.E.; Sutcliffe, J.V. River flow forecasting through conceptual models part I—A discussion of principles. J. Hydrol.
**1970**, 10, 282–290. [Google Scholar] [CrossRef] - Legates, D.R.; McCabe Jr, G.J. Evaluating the use of “goodness-of-fit” measures in hydrologic and hydroclimatic model validation. Water Resour. Res.
**1999**, 35, 233–241. [Google Scholar] [CrossRef] - Nourali, M.; Ghahraman, B.; Pourreza-Bilondi, M.; Davary, K. Effect of formal and informal likelihood functions on uncertainty assessment in a single event rainfall-runoff model. J. Hydrol.
**2016**, 540, 549–564. [Google Scholar] [CrossRef] - Beven, K.; Binley, A. The future of distributed models: Model calibration and uncertainty prediction. Hydrol. Process.
**1992**, 6, 279–298. [Google Scholar] [CrossRef] - Moriasi, D.N.; Gitau, M.W.; Pai, N.; Daggupati, P. Hydrologic and water quality models: Performance measures and evaluation criteria. Trans. ASABE
**2015**, 58, 1763–1785. [Google Scholar] - Beven, K.J.; Kirkby, M.J. A physically based, variable contributing area model of basin hydrology/Un modèle à base physique de zone d’appel variable de l’hydrologie du bassin versant. Hydrol. Sci. J.
**1979**, 24, 43–69. [Google Scholar] [CrossRef][Green Version] - Ajmal, M.; Kim, T.W. Quantifying excess stormwater using SCS-CN–based rainfall runoff models and different curve number determination methods. J. Irrig. Drain. Eng.
**2015**, 141, 04014058. [Google Scholar] [CrossRef] - Lorenz, E. The butterfly effect. World Sci. Ser. Nonlinear Sci. Ser. A
**2000**, 39, 91–94. [Google Scholar] - Leutbecher, M.; Palmer, T.N. Ensemble forecasting. J. Comput. Phys.
**2008**, 227, 3515–3539. [Google Scholar] [CrossRef] - Kang, N.; Joo, H.; Lee, M.; Kim, H.S. Generation of radar rainfall ensemble using probabilistic approach. J. Korea Water Resour. Assoc.
**2017**, 50, 155–167. [Google Scholar]

**Figure 1.**Main features of Wicheon watershed. (

**a**) DEM; (

**b**) Slope; (

**c**) Stream; (

**d**) Land cover; (

**e**) Soil depth; (

**f**) Soil texture.

**Figure 8.**Distribution of L3−based behavioral models by parameter (

**a1**–

**a8**) NSE results; (

**b1**–

**b8**) PBIAS results; (

**c1**–

**c8**) LOG results.

Wicheon | |
---|---|

Rainfall Station Name | Station Weight (%) |

Uiseong | 67.55% |

Daegu | 9.31% |

Gumi | 8.70% |

Yeongcheon | 7.51% |

Sangju | 6.93% |

Num | Parameters | Description | Range | |
---|---|---|---|---|

Lower | Upper | |||

1 | IniSaturation (ISSR) | Initial soil saturation ratio | 0 | 1 |

2 | MinSlopeOF (MSLS) | Minimum slope of land surface | 0.0001 | 0.01 |

3 | ChRoughness (CRC) | Channel roughness coefficient | 0.008 | 0.2 |

4 | CalCoefLCRoughness (CLCRC) | Channel roughness coefficient | 0.6 | 1.3 |

5 | CalCoefSoilDepth (CSD) | Correction factor for soil depth | 0.8 | 1.2 |

6 | CalCoefPorosity (CSP) | Correction factor for soil porosity | 0.9 | 1.1 |

7 | CalCoefWFSuctionHead (CSWS) | Correction factor for soil wetting front suction head | 0.25 | 4 |

8 | CalCoefHydraulicK (CSHC) | Correction factor for soil hydraulic conductivity | 0.05 | 20 |

Date | Point | Area | ||||
---|---|---|---|---|---|---|

L1 | L_{2} | L_{3} | L1 | L_{2} | L_{3} | |

2016/09/15 00UTC | 0.020 | −0.125 | 18.17 | 0.006 | −0.171 | 24.34 |

2016/09/15 12UTC | 0.721 | 0.648 | −42.95 | 0.816 | 0.796 | −26.68 |

2016/09/16 00UTC | 0.736 | 0.716 | −28.03 | 0.807 | 0.789 | −25.06 |

2016/09/16 12UTC | 0.806 | 0.654 | −17.95 | 0.847 | 0.709 | −13.97 |

2016/09/17 00UTC | 0.640 | 0.117 | −111.25 | 0.743 | 0.548 | −60.77 |

2016/09/17 12UTC | 0.092 | −0.175 | −5.13 | 0.324 | 0.302 | −12.10 |

Wicheon (Point) | 2016/09/15 00UTC | 2016/09/15 12UTC | 2016/09/16 00UTC | |||
---|---|---|---|---|---|---|

Obs | LENS | Obs | LENS | Obs | LENS | |

Total rainfall | 93 | 76.11 | 93.5 | 133.66 | 93.6 | 119.84 |

Total rainfall error | −16.89 | 40.16 | 26.24 | |||

Peak rainfall | 11 | 5.41 | 11 | 11.65 | 11 | 9.71 |

Peak rainfall error | −5.59 | 0.65 | −1.29 | |||

Peak time | 50 | 72 | 38 | 40 | 26 | 24 |

Peak time error | 22 | 2 | −2 |

Wicheon (Point) | 2016/09/16 12UTC | 2016/09/17 00UTC | 2016/09/17 12UTC | |||
---|---|---|---|---|---|---|

Obs | LENS | Obs | LENS | Obs | LENS | |

Total rainfall | 91 | 107.33 | 20.5 | 43.31 | 2.5 | 2.63 |

Total rainfall error | 16.33 | 22.81 | 0.13 | |||

Peak rainfall | 11 | 13.24 | 8 | 9.29 | 0.5 | 0.77 |

Peak rainfall error | 2.24 | 1.29 | 0.27 | |||

Peak time | 14 | 13 | 5 | 5 | 4 | 11 |

Peak time error | −1 | 0 | 7 |

Wicheon (Area) | 2016/09/15 00UTC | 2016/09/15 12UTC | 2016/09/16 00UTC | |||
---|---|---|---|---|---|---|

Obs | LENS | Obs | LENS | Obs | LENS | |

Total rainfall | 101.3 | 76.64 | 102.4 | 129.72 | 102.4 | 128.06 |

Total rainfall error | −24.66 | 27.32 | 25.66 | |||

Peak rainfall | 11 | 4.82 | 11 | 9.92 | 11 | 9.7 |

Peak rainfall error | −6.18 | −1.08 | −1.3 | |||

Peak time | 50 | 72 | 38 | 40 | 26 | 24 |

Peak time error | 22 | 2 | −2 |

Wicheon (Area) | 2016/09/16 12UTC | 2016/09/17 00UTC | 2016/09/17 12UTC | |||
---|---|---|---|---|---|---|

Obs | LENS | Obs | LENS | Obs | LENS | |

Total rainfall | 99.7 | 113.63 | 25.3 | 40.67 | 3.5 | 3.92 |

Total rainfall error | 13.93 | 15.37 | 0.42 | |||

Peak rainfall | 11 | 14.06 | 7.6 | 7.39 | 0.8 | 0.53 |

Peak rainfall error | 3.06 | −0.21 | −0.27 | |||

Peak time | 14 | 13 | 5 | 5 | 5 | 11 |

Peak time error | −1 | 0 | 6 |

Number | Parameters | Initial Range | Behavioral Range | ||||
---|---|---|---|---|---|---|---|

Lower | Median | Upper | Lower | Median | Upper | ||

1 | ISSR | 0.5 | 0.75 | 1 | 0.3015 | 0.5692 | 0.7677 |

2 | MSLS | 0.0001 | 0.00355 | 0.007 | 0.0001 | 0.0036 | 0.0070 |

3 | CRC | 0.008 | 0.074 | 0.14 | 0.0210 | 0.0301 | 0.0410 |

4 | CLCRC | 0.6 | 0.95 | 1.3 | 0.6004 | 0.9624 | 1.3000 |

5 | CSP | 0.9 | 1 | 1.1 | 0.9002 | 0.9972 | 1.0998 |

6 | CSWS | 0.25 | 2.125 | 4 | 0.2521 | 2.0665 | 3.9989 |

7 | CSHC | 1 | 1.25 | 1.5 | 1.0008 | 1.2356 | 1.4997 |

8 | CSD | 1 | 2 | 3 | 1.0004 | 1.8466 | 2.9972 |

Number | Parameters | Initial Range | Behavioral Range | ||||
---|---|---|---|---|---|---|---|

Lower | Median | Upper | Lower | Median | Upper | ||

1 | ISSR | 0.5 | 0.75 | 1 | 0.3436 | 0.6024 | 0.7692 |

2 | MSLS | 0.0001 | 0.00355 | 0.007 | 0.0001 | 0.0036 | 0.0070 |

3 | CRC | 0.008 | 0.074 | 0.14 | 0.0210 | 0.0304 | 0.0410 |

4 | CLCRC | 0.6 | 0.95 | 1.3 | 0.6004 | 0.9658 | 1.3000 |

5 | CSP | 0.9 | 1 | 1.1 | 0.9007 | 0.9958 | 1.0998 |

6 | CSWS | 0.25 | 2.125 | 4 | 0.2518 | 2.0273 | 3.9989 |

7 | CSHC | 1 | 1.25 | 1.5 | 1.0008 | 1.2469 | 1.4997 |

8 | CSD | 1 | 2 | 3 | 1.0013 | 1.8519 | 2.9953 |

Number | Parameters | Initial Range | Behavioral Range | ||||
---|---|---|---|---|---|---|---|

Lower | Median | Upper | Lower | Median | Upper | ||

1 | ISSR | 0.5 | 0.75 | 1 | 0.3488 | 0.5804 | 0.7648 |

2 | MSLS | 0.0001 | 0.00355 | 0.007 | 0.0001 | 0.0035 | 0.0070 |

3 | CRC | 0.008 | 0.074 | 0.14 | 0.0210 | 0.0304 | 0.0407 |

4 | CLCRC | 0.6 | 0.95 | 1.3 | 0.6004 | 0.9578 | 1.3000 |

5 | CSP | 0.9 | 1 | 1.1 | 0.9007 | 0.9953 | 1.0998 |

6 | CSWS | 0.25 | 2.125 | 4 | 0.2539 | 2.0278 | 3.9925 |

7 | CSHC | 1 | 1.25 | 1.5 | 1.0008 | 1.2419 | 1.4994 |

8 | CSD | 1 | 2 | 3 | 1.0013 | 1.8140 | 2.9891 |

Point | Area | |||||
---|---|---|---|---|---|---|

L1 | L_{2} | L_{3} | L1 | L_{2} | L_{3} | |

2016/09/15 00UTC | 0.876 | 0.644 | 35.12 | 0.780 | 0.496 | 42.13 |

2016/09/15 12UTC | 0.888 | 0.410 | −43.05 | 0.921 | 0.806 | −23.73 |

2016/09/16 00UTC | 0.980 | 0.701 | −33.27 | 0.971 | 0.774 | −30.31 |

2016/09/16 12UTC | 0.413 | −0.438 | −28.35 | 0.538 | −0.043 | −23.34 |

2016/09/17 00UTC | 0.592 | 0.059 | −41.53 | 0.864 | 0.704 | −22.62 |

2016/09/17 12UTC | 1.000 | 1.000 | 0.01 | 1.000 | 1.000 | 0.06 |

Wicheon (Point) | 2016/09/15 00UTC | 2016/09/15 12UTC | 2016/09/16 00UTC | |||
---|---|---|---|---|---|---|

Obs | LENS | Obs | LENS | Obs | LENS | |

Total discharge | 16,672.18 | 10,817.27 | 19,806.4 | 28,332.51 | 21,763.86 | 29,004.48 |

Total discharge error | −5854.91 | 8526.11 | 7240.62 | |||

Peak discharge | 757.81 | 553.8 | 807.12 | 1255.26 | 868.66 | 1110.27 |

Peak discharge error | −204.01 | 448.14 | 241.61 | |||

Peak discharge time | 65 | 72 | 53 | 48 | 41 | 38 |

Peak time error | 7 | −5 | −3 |

Wicheon (Area) | 2016/09/15 00UTC | 2016/09/15 12UTC | 2016/09/16 00UTC | |||
---|---|---|---|---|---|---|

Obs | LENS | Obs | LENS | Obs | LENS | |

Total discharge | 18,015.87 | 10,425.93 | 21,538.51 | 26,649.46 | 23,425.63 | 30,526.89 |

Total discharge error | −7589.94 | 5110.95 | 7101.26 | |||

Peak discharge | 858.89 | 589.8 | 921.41 | 1121.91 | 969.12 | 1175.69 |

Peak discharge error | −269.09 | 200.5 | 206.57 | |||

Peak discharge time | 65 | 72 | 53 | 49 | 40 | 37 |

Peak time error | 7 | −4 | −3 |

Wicheon (Point) | 2016/09/16 00UTC | 2016/08/17 12UTC | 2016/09/17 00UTC | |||
---|---|---|---|---|---|---|

Obs | LENS | Obs | LENS | Obs | LENS | |

Total discharge | 22,773.09 | 29,230.11 | 9641.61 | 13,645.62 | 7323.98 | 7323.61 |

Total discharge error | 6457.02 | 4004.01 | −0.37 | |||

Peak discharge | 850.58 | 1280.84 | 216.88 | 290.8 | 191.87 | 191.36 |

Peak discharge error | 430.26 | 73.92 | −0.51 | |||

Peak discharge time | 30 | 22 | 45 | 33 | 59 | 60 |

Peak time error | −8 | −12 | 1 |

Wicheon (Area) | 2016/09/16 00UTC | 2016/08/17 12UTC | 2016/09/17 00UTC | |||
---|---|---|---|---|---|---|

Obs | LENS | Obs | LENS | Obs | LENS | |

Total discharge | 24,813.82 | 30,605 | 10,196.42 | 12,502.88 | 7434.39 | 7429.74 |

Total discharge error | 5791.18 | 2306.46 | −4.65 | |||

Peak discharge | 989.25 | 1405.62 | 226.42 | 265.11 | 193.06 | 193.11 |

Peak discharge error | 416.37 | 38.69 | 0.05 | |||

Peak discharge time | 29 | 23 | 44 | 37 | 58 | 60 |

Peak time error | −6 | −7 | 2 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Lee, S.; Seong, Y.; Jung, Y. LENS-GRM Applicability Analysis and Evaluation. *Water* **2022**, *14*, 3897.
https://doi.org/10.3390/w14233897

**AMA Style**

Lee S, Seong Y, Jung Y. LENS-GRM Applicability Analysis and Evaluation. *Water*. 2022; 14(23):3897.
https://doi.org/10.3390/w14233897

**Chicago/Turabian Style**

Lee, Sanghyup, Yeonjeong Seong, and Younghun Jung. 2022. "LENS-GRM Applicability Analysis and Evaluation" *Water* 14, no. 23: 3897.
https://doi.org/10.3390/w14233897