# Predictive Uncertainty Estimation of Hydrological Multi-Model Ensembles Using Pair-Copula Construction

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

_{t}|ŷ

_{t}) of y conditional on a single or multiple model prediction(s) ŷ [2]. The predictive uncertainty is conditional on the model structure, its parameters and the forcing data. The hydrological model predictions ŷ can be a deterministic prediction by single or multiple models (multi-model ensemble) forced by a deterministic meteorological prediction or by meteorological ensemble predictions.

_{t}|y

_{t}) of the prediction ŷ conditional on the known observation y. This uncertainty is important to describe and improve the skill of hydrological models. What is more important to a flow forecaster is the uncertainty of the future observation given a particular decision than the uncertainty of the hydrological model prediction.

## 2. Materials and Methods

#### 2.1. Study Area

^{2}and a mean flow of 40 m

^{3}/s and gauge Trier at the Moselle with an area of 23,857 km

^{2}and a mean flow of 280 m

^{3}/s (see Figure 1 and Table 1). Both gauges show a rainfall dominated (pluvial) flow regime with maximum flow during winter and early spring and minimum flow in late summer.

#### 2.2. Hydrological and Statistical Models

#### 2.3. Copula Theory

_{1}, X

_{2}, X

_{3}, …, X

_{n}can be described through the copula function C. The copula function is a multivariate distribution function of an n-dimensional random vector on the unit cube, i.e., C: [0,1]

^{n}→ [0,1], and is strictly invariant under the condition of monotonously increasing transformations of X

_{1}, X

_{2}, X

_{3}, …, X

_{n}:

_{1}, u

_{2}, …, u

_{n}∊ [0,1] are uniformly distributed random realizations of the variates defined as ${u}_{1}={F}_{{X}_{1}}\left({x}_{1}\right)$, ${u}_{2}={F}_{{X}_{2}}\left({x}_{2}\right)$, …, ${u}_{n}={F}_{{X}_{n}}\left({x}_{n}\right)$.

#### 2.4. Pair-Copula Construction

_{j}has n + 1 − j nodes and n − j edges. In total, the decomposition is defined by n (n − 1)/2 edges and each edge is represented by a pair-copula density. The labels of the edges represent subscript of the copula density, e.g., 13|2 denotes the copula density c

_{13|2}(·) [39,45].

**υ**, υ

_{j}an arbitrary element of

**υ**and

**υ**

_{-j}the (m − 1)-dimensional vector

**υ**excluding element υ

_{j}.

_{4}in Figure 2) conditional on other variables can be solved analytically by means of Equation (4). In this case, the key variable has no direct connection to the remaining variables in the most important first tree, and the dependence is only modelled indirectly in the downstream trees. Hence, it is recommended to locate the key variable at the root (variable X

_{1}in Figure 2) of the C-Vine copula [45]. The conditional copula density of the root variable conditional on the other variables is:

_{1,2,…,n}numerically over the first variable between 0 and 1 while fixing the remaining ones.

_{1}, u

_{2}) are also compared. In this application, the C-Vine choice and parameter estimation was performed with the R-package CDVine [46].

#### 2.5. Marginal Distributions and Normal Quantile Transform

^{−1}the inverse cumulative distribution function of the standard Normal distribution.

_{1}and u

_{2}, the probability density function (pdf) and the cumulative distribution function (cdf) of the mixed probability distribution are defined as:

_{m}(y) with F

_{c}(u

_{1}) and the upper distribution f

_{M}(y) with [1 − F

_{c}(u

_{2})] is required to ensure the area under the probability density graph to equal unity.

_{1}to u

_{2}.

_{1}for negative k: y ≤ u

_{1}; k

_{1}< 0 and a lower and upper boundary for positive k: u

_{1}− a

_{1}/k

_{1}≤ y ≤ u

_{1}; k

_{1}> 0.

_{2}for negative k: u

_{2}≤ y; k

_{2}< 0 and a lower and upper boundary for positive k: u

_{2}≤ y ≤ u

_{2}+ a

_{2}/k

_{2}; k

_{2}> 0.

_{1}and u

_{2}. Using the conditions f

_{c}(y = u

_{1}) = f

_{m}(y = u

_{1}) F

_{c}(u

_{1}) and f

_{c}(y = u

_{2}) = f

_{M}(y = u

_{2}) [1 − F

_{c}(u

_{2})] the parameters a

_{1}and a

_{2}of the generalized Pareto distributions are obtained as:

_{l}and upper boundary b

_{u}of the mixture probability distribution as an additional condition, the shape parameters k

_{1}and k

_{2}of the GP distribution result equal to:

_{1}, u

_{2}. All parameters are estimated simultaneously using maximum likelihood estimation. Solari and Losada [58] used a two-parameter log-Normal distribution as central distribution function f

_{c}. In this application, different types of central distributions (generalized Extreme Value, Logistic, generalized Logistic, Pearson3, log-Pearson3, Frechet, Gamma, Normal, log-Normal, Weibull, log-Weibull, Gumbel, Exponential, and generalized Pareto distribution [59]) were applied and the type of distribution with the best fit based on the AIC was selected.

#### 2.6. Alternative Methods applied for PU Estimation

_{τ}, b

_{τ,ι}the parameters of the multiple linear regression relationship and M the number of models. The R-package quantreg [61] was used for parameter estimation.

_{i}is the posterior probability of prediction i being the best, and M the number of models. The weights w

_{i}are non-negative and add up to unity [15]. The conditional Normal distribution of the individual model prediction is centered on a linear dependency for the model prediction. Parameters a

_{i}and b

_{i}act as bias correction factors. The weights w

_{i}and standard deviations σ

_{i}are estimated from past model performance. We applied a two-step parameter estimation procedure: (1) maximize log-likelihood with the Expectation Maximum (EM) algorithm to estimate weights and standard deviations and (2) minimize the Continuous Ranked Probability Score (CRPS) by applying Equation (23) to optimize the estimation of the standard deviations. This procedure is implemented in the R-package ensemble BMA [62].

#### 2.7. Verification

_{i}, considering that observation o

_{i}is 1 if the event occurs and 0 in its absence [21]:

_{i}

^{f}and the distribution of the observation ${{F}_{i}}^{o}$ [21]:

_{FOR}/CRPS

_{REF}and BSS = 1 − BS

_{FOR}/BS

_{REF}are calculated from the CRPS

_{FOR}and BS

_{FOR}of the predictive distribution and the CRPS

_{REF}and BS

_{REF}of the reference forecast, for which the climatological forecast, i.e., the observed daily mean flow of the period from 1 October 2000 to 30 September 2015 is used. Due to the strong seasonality of the flow at the two gauges (see Figure 1) the CRPSS and BSS are calculated for each month separately and the climatological forecast is calculated only from the observed values of the corresponding month excluding the current month of verification. A perfect forecast has a skill score of one, negative skill score values indicate that the climatological forecast is superior to the predictive uncertainty derived from the model predictions.

## 3. Results and Discussion

#### 3.1. Model Predictions

#### 3.2. Estimation of the Marginal Distributions

^{3}/s was used as lower and three times the maximum rate as the upper boundary. For the distribution of the observed values, the maxima shown in Table 1 and for the distributions of the predictions the maximum values of the simulated time series 1 October 2000 to 30 September 2015 were used. The types of the central distributions have been selected based on the AIC, using the maximum likelihood method for parameter estimation.

_{1}and u

_{2}. At Bollendorf, the identified central distribution of the mixture distribution was the log-Pearson3 and, at Trier, the generalized Pareto distribution using all data of the 1 October 2000 to 30 September 2015 analysis period. The selection of the type of the central distribution is relatively stable. When using the LOOCV subsample sets at gauge Bollendorf, the same distribution (log-Pearson3) was identified 10 times out of 15 subsamples and, at gauge Trier, the same distribution (generalized Pareto distribution) was identified for all subsamples. The variability of the fitted distributions and the two thresholds u

_{1}and u

_{2}for the different subsamples in the LOOCV are shown in Figure 3 as dotted lines.

#### 3.3. C-Vine Copula Selection and Estimation

_{1}, u

_{2}) takes on values between 0.1 and 0.3. No theoretical copula function exists, which is able to capture the irregularity in the λ-function of the bivariate sample data present at both gauges.

#### 3.4. Verification

## 4. Conclusions

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Krzysztofowicz, R. Bayesian theory of probabilistic forecasting via deterministic hydrologic model. Water Resour. Res.
**1999**, 35, 2739–2750. [Google Scholar] [CrossRef] - Todini, E. A model conditional processor to assess predictive uncertainty in flood forecasting. Int. J. River Basin Manag.
**2008**, 6, 123–137. [Google Scholar] [CrossRef] - Todini, E. History and perspectives of hydrological catchment modelling. Hydrol. Res.
**2011**, 42, 73–85. [Google Scholar] [CrossRef] - Todini, E. The role of predictive uncertainty in the operational management of reservoirs. In Proceedings of the ICWRS2014—Evolving Water Resources Systems: Understanding, Predicting and Managing Water-Society Interactions, Bologna, Italy, 4–6 June 2014.
- Ramos, M.H.; van Andel, S.J.; Pappenberger, F. Do probabilistic forecasts lead to better decisions? Hydrol. Earth Syst. Sci.
**2013**, 17, 2219–2232. [Google Scholar] [CrossRef] - Verkade, J.S.; Werner, M.G.F. Estimating the benefits of single value and probability forecasting for flood warning. Hydrol. Earth Syst. Sci.
**2011**, 15, 3751–3765. [Google Scholar] [CrossRef] - Krzysztofowicz, R. Bayesian system for probabilistic river stage forecasting. J. Hydrol.
**2002**, 268, 16–40. [Google Scholar] [CrossRef] - Krzysztofowicz, R.; Kelly, K.S. Hydrologic uncertainty processor for probabilistic river stage forecasting. Water Resour. Res.
**2000**, 36, 3265–3277. [Google Scholar] [CrossRef] - Krzysztofwicz, R.; Maranzano, C.J. Bayesian system for probabilistic stage transition forecasting. J. Hydrol.
**2004**, 299, 15–44. [Google Scholar] [CrossRef] - Reggiani, P.; Renner, M.; Weerts, A.H.; van Gelder, P. Uncertainty assessment via Bayesian revision of ensemble streamflow predictions in the operational river Rhine forecasting system. Water Resour. Res.
**2009**, 45, 1–14. [Google Scholar] [CrossRef] - Coccia, G.; Todini, E. Recent developments in predictive uncertainty assessment based on the model conditional processor approach. Hydrol. Earth Syst. Sci.
**2011**, 15, 3253–3274. [Google Scholar] [CrossRef] - Todini, E.; Coccia, G.; Ortiz, E. On the proper use of ensembles for predictive uncertainty assessment. Geophysical Research Abstracts 2015. In Proceedings of the EGU2015, Vienna, Austria, 12–17 April 2015.
- Weerts, A.H.; Winsemius, H.C.; Verkade, J.S. Estimation of predictive hydrological uncertainty using quantile regression: Examples from the national flood forecasting system (England and Wales). Hydrol. Earth Syst. Sci.
**2011**, 15, 255–265. [Google Scholar] [CrossRef] - Lopez, P.L.; Verkade, J.S.; Weerts, A.H.; Solomatine, D.P. Alternative configurations of quantile regression for estimating predictive uncertainty in water level forecasts for the upper Severn river: A comparison. Hydrol. Earth Syst. Sci.
**2014**, 18, 3411–3428. [Google Scholar] [CrossRef] - Raftery, A.E.; Gneiting, T.; Balabdaoui, F.; Polakowski, M. Using Bayesian model averaging to calibrate forecast ensembles. Mon. Weather Rev.
**2005**, 133, 1155–1174. [Google Scholar] [CrossRef] - Duan, Q.Y.; Ajami, N.K.; Gao, X.G.; Sorooshian, S. Multi-model ensemble hydrologic prediction using Bayesian model averaging. Adv. Water Resour.
**2007**, 30, 1371–1386. [Google Scholar] [CrossRef] - Vrugt, J.A.; Robinson, B.A. Treatment of uncertainty using ensemble methods: Comparison of sequential data assimilation and Bayesian model averaging. Water Resour. Res.
**2007**, 43, 1–15. [Google Scholar] [CrossRef] - Madadgar, S.; Moradkhani, H. Improved Bayesian multimodeling: Integration of copulas and Bayesian model averaging. Water Resour. Res.
**2014**, 50, 9586–9603. [Google Scholar] [CrossRef] - Gneiting, T.; Raftery, A.E.; Westveld, A.H.; Goldman, T. Calibrated probabilistic forecasting using ensemble model output statistics and minimum crps estimation. Mon. Weather Rev.
**2005**, 133, 1098–1118. [Google Scholar] [CrossRef] - Hemri, S.; Lisniak, D.; Klein, B. Multivariate post-processing techniques for probabilistic hydrological forecasting. Water Resour. Res.
**2015**. [Google Scholar] [CrossRef] - Wilks, D.S. Statistical Methods in the Atmospheric Sciences, 3rd ed.; Academic Press: Oxford, UK, 2011. [Google Scholar]
- Madadgar, S.; Moradkhani, H.; Garen, D. Towards improved post-processing of hydrologic forecast ensembles. Hydrol. Process.
**2014**, 28, 104–122. [Google Scholar] [CrossRef] - Brown, J.D.; Seo, D.J. Evaluation of a nonparametric post-processor for bias correction and uncertainty estimation of hydrologic predictions. Hydrol. Process.
**2013**, 27, 83–105. [Google Scholar] [CrossRef] - R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2015. [Google Scholar]
- Bergström, S. The hbv model. In Computer Models of Watershed Hydrology; Singh, V.P., Ed.; Water Resources Publications: Highlands Ranch, CO, USA, 1995; pp. 443–476. [Google Scholar]
- Reggiani, P.; Rientjes, T.H.M. Flux parameterization in the representative elementary watershed approach: Application to a natural basin. Water Resour. Res.
**2005**, 41. [Google Scholar] [CrossRef] - Reggiani, P.; Rientjes, T.H.M. Closing horizontal groundwater fluxes with pipe network analysis: An application of the REW approach to an aquifer. Environ. Model. Softw.
**2010**, 25, 1702–1712. [Google Scholar] [CrossRef] - Jaeger, H.; Haas, H. Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science
**2004**, 304, 78–80. [Google Scholar] [CrossRef] [PubMed] - Lukoševičius, M.; Jaeger, H. Reservoir computing approaches to recurrent neural network training. Comput. Sci. Rev.
**2009**, 3, 127–149. [Google Scholar] [CrossRef] - Favre, A.C.; El Adlouni, S.; Perreault, L.; Thiemonge, N.; Bobee, B. Multivariate hydrological frequency analysis using copulas. Water Resour. Res.
**2004**, 40. [Google Scholar] [CrossRef] - Bardossy, A.; Pegram, G.G.S. Copula based multisite model for daily precipitation simulation. Hydrol. Earth Syst. Sci.
**2009**, 13, 2299–2314. [Google Scholar] [CrossRef] - Klein, B.; Pahlow, M.; Hundecha, Y.; Schumann, A. Probability analysis of hydrological loads for the design of flood control systems using copulas. J. Hydrol. Eng.
**2010**, 15, 360–369. [Google Scholar] [CrossRef] - Kao, S.C.; Govindaraju, R.S. A copula-based joint deficit index for droughts. J. Hydrol.
**2010**, 380, 121–134. [Google Scholar] [CrossRef] - Joe, H. Multivariate Models and Dependence Concepts; Chapman and Hall: New York, NY, USA, 1997. [Google Scholar]
- Nelsen, R.B. An Introduction to Copulas; Springer: New York, NY, USA, 1999. [Google Scholar]
- Salvadori, G.; De Michele, C.; Kottegoda, N.T.; Rosso, R. Extremes in Nature: An Approach Using Copulas; Springer: Dordrecht, The Netherlands, 2007; Volume 56. [Google Scholar]
- Sklar, A. Fonctions de Répartition à n Dimensions et Leurs marges; Publications de l’Institut de Statistique de L’Université de Paris: Paris, France, 1959; Volume 8, pp. 229–231. [Google Scholar]
- Joe, H. Families of m-variate distributions with given margins and m(m − 1)/2 bivariate dependence parameters. In Distributions with Fixed Marginals and Related Topics; Ruschendorf, L., Schweizer, B., Taylor, M.D., Eds.; Institute of Mathematical Statistics: Hayward, CA, USA, 1996; Volume 28, pp. 120–141. [Google Scholar]
- Bedford, T.; Cooke, R.M. Probability density decomposition for conditionally dependent random variables modeled by vines. Ann. Math. Artif. Intell.
**2001**, 32, 245–268. [Google Scholar] [CrossRef] - Bedford, T.; Cooke, R.M. Vines—A new graphical model for dependent random variables. Ann. Stat.
**2002**, 30, 1031–1068. [Google Scholar] [CrossRef] - Kurowicka, D.; Cooke, R.M. Uncertainty Analysis with High Dimensional Dependence Modelling; John Wiley & Sons: Chichester, UK, 2006. [Google Scholar]
- Graeler, B.; van den Berg, M.J.; Vandenberghe, S.; Petroselli, A.; Grimaldi, S.; De Baets, B.; Verhoest, N.E.C. Multivariate return periods in hydrology: A critical and practical review focusing on synthetic design hydrograph estimation. Hydrol. Earth Syst. Sci.
**2013**, 17, 1281–1296. [Google Scholar] [CrossRef] [Green Version] - Xiong, L.H.; Yu, K.X.; Gottschalk, L. Estimation of the distribution of annual runoff from climatic variables using copulas. Water Resour. Res.
**2014**, 50, 7134–7152. [Google Scholar] [CrossRef] - Vernieuwe, H.; Vandenberghe, S.; De Baets, B.; Verhoest, N.E.C. A continuous rainfall model based on vine copulas. Hydrol. Earth Syst. Sci.
**2015**, 19, 2685–2699. [Google Scholar] [CrossRef] [Green Version] - Aas, K.; Czado, C.; Frigessi, A.; Bakken, H. Pair-copula constructions of multiple dependence. Insur. Math. Econ.
**2009**, 44, 182–198. [Google Scholar] [CrossRef] [Green Version] - Brechmann, E.C.; Schepsmeier, U. Modeling dependence with C- and D-vine copulas: The R-package CDvine. J. Stat. Softw.
**2013**, 52, 1–27. [Google Scholar] [CrossRef] - Akaike, H. A new look at the statistical model identification. IEEE Trans. Autom. Control
**1974**, 19, 716–723. [Google Scholar] [CrossRef] - Cherubini, U.; Luciano, E.; Vecchiato, W. Copula Methods in Finance; Wiley: Chichester, UK, 2004. [Google Scholar]
- Genest, C.; Rivest, L.-P. Statistical inference procedures for bivariate archimedean copulas. J. Am. Stat. Assoc.
**1993**, 88, 1034–1043. [Google Scholar] [CrossRef] - Box, G.E.P.; Cox, D.R. An analysis of transformations. J. R. Stat. Soc. Ser. B
**1964**, 26, 211–243. [Google Scholar] - Wang, Q.J.; Shrestha, D.L.; Robertson, D.E.; Pokhrel, P. A log-sinh transformation for data normalization and variance stabilization. Water Resour. Res.
**2012**, 48, 1–7. [Google Scholar] [CrossRef] - Van der Waerden, B.L. Order tests for two-sample problem and their power I. Indag. Math.
**1952**, 14, 453–458. [Google Scholar] [CrossRef] - Van der Waerden, B.L. Order tests for two-sample problem and their power II. Indag. Math.
**1953**, 15, 303–310. [Google Scholar] [CrossRef] - Van der Waerden, B.L. Order tests for two-sample problem and their power III. Indag. Math.
**1953**, 15, 311–316. [Google Scholar] [CrossRef] - Bogner, K.; Pappenberger, F.; Cloke, H.L. Technical note: The normal quantile transformation and its application in a flood forecasting system. Hydrol. Earth Syst. Sci.
**2012**, 16, 1085–1094. [Google Scholar] [CrossRef] - Hundecha, Y.; Pahlow, M.; Schumann, A. Modeling of daily precipitation at multiple locations using a mixture of distributions to characterize the extremes. Water Resour. Res.
**2009**, 45, 1–15. [Google Scholar] [CrossRef] - Furrer, E.M.; Katz, R.W. Improving the simulation of extreme precipitation events by stochastic weather generators. Water Resour. Res.
**2008**, 44, 1–13. [Google Scholar] [CrossRef] - Solari, S.; Losada, M.A. A unified statistical model for hydrological variables including the selection of threshold for the peak over threshold method. Water Resour. Res.
**2012**, 48, 1–15. [Google Scholar] [CrossRef] - Rao, A.R.; Hamed, K.H. Flood Frequency Analysis; CRC Press: Boca Raton, FL, USA, 2010. [Google Scholar]
- Koenker, R. Quantile Regression; Cambridge University Press: Cambridge, UK, 2005. [Google Scholar]
- Koenker, R. Quantile Regression in R: A Vignette. Available online: http://cran.r-project.org/web/packages/quantreg/vignettes/rq.pdf (accessed on 21 December 2015).
- Fraley, C.; Raftery, A.; Gneiting, T.; Sloughter, M.; Berrocal, V. Probabilistic weather forecasting in R. R J.
**2011**, 3, 55–63. [Google Scholar] - Gneiting, T.; Balabdaoui, F.; Raftery, A.E. Probabilistic forecasts, calibration and sharpness. J. R. Stat. Soc. Ser. B Stat. Methodol.
**2007**, 69, 243–268. [Google Scholar] [CrossRef] - Kling, H.; Fuchs, M.; Paulin, M. Runoff conditions in the upper Danube basin under an ensemble of climate change scenarios. J. Hydrol.
**2012**, 424, 264–277. [Google Scholar] [CrossRef]

**Figure 1.**Study area Moselle river, elevation above sea level (digital elevation model DEM), sub-catchments gauge Trier and gauge Bollendorf, long-term mean monthly flow (1981–2010).

**Figure 2.**Graphical representation of a 4-dimensional C-Vine copula [45].

**Figure 3.**Cumulative distribution function (cdf) plots of the mixture distributions (

**red**) fitted to the observed daily mean flow at (

**a**) Gauge Bollendorf; and (

**b**) Gauge Trier and the two thresholds u

_{1}(

**blue**) and u

_{2}(

**green**) using all data (solid) and Leave-One-Out Cross Validation (LOOCV) subsample sets (dotted) for parameter estimation. The probabilities on the x-axis of the cdf are in a log-log scale.

**Figure 4.**Estimated bivariate copulas of the first tree of the C-Vine for the bivariate samples observation and HBV prediction (first row); observation and ESN prediction (second row); observation and REW prediction (third row) of (

**a**) gauge Bollendorf; and (

**b**) gauge Trier using all data of the analysis period. Shown are the scatter plots of the bivariate sample (

**red**) and a random sample (10 times the size of the bivariate sample) from the fitted copula (

**black**) in the copula space (first column) and back-transformed to the real space (second column). In the third column, the empirical λ-functions (

**black**) and the theoretical λ-functions (

**grey**) are shown. The dashed lines are bounds corresponding to independence and comonotonicity (λ = 0), respectively.

**Figure 5.**PIT-histograms (

**a**) gauge Bollendorf; (

**b**) gauge Trier.

**Upper left**COpula uncertainty Processor (COP),

**lower left**quantile regression (QR)

**upper right**Bayesian Model Averaging (BMA),

**lower right**Multivariate Truncated Normal (MTN) distribution.

**Figure 6.**Distributions of the 5%–95% quantile widths are shown as modified boxplots at (

**a**) gauge Bollendorf; (

**b**) gauge Trier. The box represents the 25% to 75% quantile, the median is the band inside the box and the whiskers represent the 5% and the 95% quantile. Outliers are not shown. In the upper part of the figure, bars represent the “coverage”, i.e., the percentage of observations falling in the 5%–95% quantile range.

**Figure 7.**Brier Skill Score for different flow rate thresholds (

**a**) gauge Bollendorf; (

**b**) gauge Trier. COpula uncertainty Processor (COP) blue line, Bayesian Model Averaging (BMA) green line, quantile regression (QR) orange line and Multivariate Truncated Normal (MTN) distribution brown line.

**Figure 8.**Predictive uncertainty of the flood event 15 December 2002–18 January 2003 at the gauge Trier estimated with the COpula uncertainty Processor (COP) (

**a**), Bayesian Model Averaging (BMA) (

**b**), quantile regression (QR) (

**c**) and Multivariate Truncated Normal (MTN) distribution (

**d**). Model predictions of HBV (continuous), ESN (dashed) and REW (dotted) as red lines.

Gauge | River | Area (km^{2}) | Mean Flow (m^{3}/s) | Max. Flow (m^{3}/s) |
---|---|---|---|---|

Trier | Moselle | 23,857 | 280 | 3840 |

Bollendorf | Sauer | 3213 | 40 | 823 |

**Table 2.**Goodness-of-Fit (GoF) measure KGE′ and its components for the two hydrological model predictions HBV, REW and the prediction with the echo state network model ESN for the period 1 October 2000 to 30 September 2015 at the gauges Bollendorf and Trier.

GoF Measure | Bollendorf | Trier | ||||
---|---|---|---|---|---|---|

HBV | ESN | REW | HBV | ESN | REW | |

KGE′ | 0.85 | 0.78 | 0.75 | 0.83 | 0.87 | 0.85 |

Correlation ρ | 0.94 | 0.89 | 0.89 | 0.95 | 0.95 | 0.89 |

Bias-Ratio β | 1.10 | 0.99 | 1.22 | 0.84 | 1.03 | 1.01 |

CV-Ratio γ | 0.91 | 0.80 | 1.05 | 0.95 | 0.88 | 1.10 |

**Table 3.**Pair-copula families of the C-Vine identified using all data and LOOCV (validation year as identifier) at gauge Trier.

LOOCV | Tree 1 | Tree 2 | Tree 3 | |||
---|---|---|---|---|---|---|

Sample | C_{Obs,HBV} | C_{Obs,ESN} | C_{Obs,REW} | C_{HBV,ESN|Obs} | C_{HBV,REW|Obs} | C_{ESN,REW|Obs,HBV} |

All Data | Gumbel | BB1 | BB8 | Student t | Student t | Student t |

2001, 2014 | Gumbel | BB1 | BB8 | Student t | Student t | Independence |

2002, 2004 | BB1 | Student t | BB8 | Student t | Student t | Independence |

2003, 2005, 2006, 2007, 2008, 2009, 2010, 2013, 2015 | Gumbel | BB1 | BB8 | Student t | Student t | Student t |

2011 | Gumbel | Student t | BB8 | Student t | Student t | Independence |

2012 | BB1 | BB1 | BB8 | Student t | Student t | Independence |

**Table 4.**Deterministic Goodness-of-Fit (Gof) measure KGE′ and its components of the expected values of the predictive distributions estimated with the COpula uncertainty Processor (COP), Bayesian Model Averaging (BMA), Quantile Regression (QR), Multivariate Truncated Normal (MTN) distribution and the probabilistic skill score CRPSS of the predictive distributions for the gauges Bollendorf and Trier.

GoF Measure | Bollendorf | Trier | ||||||
---|---|---|---|---|---|---|---|---|

COP | BMA | QR | MTN | COP | BMA | QR | MTN | |

KGF′ | 0.91 | 0.90 | 0.90 | 0.93 | 0.96 | 0.96 | 0.96 | 0.96 |

Correlation ρ | 0.94 | 0.94 | 0.94 | 0.94 | 0.97 | 0.97 | 0.97 | 0.97 |

Bias-Ratio β | 1.03 | 1.05 | 0.99 | 1.00 | 1.00 | 1.01 | 1.00 | 1.00 |

CV-Ratio γ | 0.95 | 0.94 | 0.92 | 0.95 | 0.99 | 0.98 | 0.97 | 0.98 |

CRPSS | 0.61 | 0.62 | 0.63 | 0.63 | 0.71 | 0.71 | 0.72 | 0.72 |

**Table 5.**Deterministic Goodness-of-Fit (GoF) measure KGE′ and its components of the expected values of the predictive distributions estimated with the copula uncertainty processor with different combinations of model predictions and the probabilistic skill score CRPSS of the predictive distributions for the gauge Trier.

GoF Measure | HBV REW ESN | HBV | ESN | REW | HBV ESN | HBV REW | ESN REW |
---|---|---|---|---|---|---|---|

KGE′ | 0.96 | 0.95 | 0.95 | 0.86 | 0.96 | 0.95 | 0.96 |

Correlation ρ | 0.97 | 0.95 | 0.96 | 0.90 | 0.96 | 0.96 | 0.96 |

Bias-Ratio β | 1.00 | 1.00 | 1.00 | 1.01 | 1.00 | 1.00 | 1.00 |

CV-Ratio γ | 0.99 | 0.98 | 0.97 | 0.90 | 0.99 | 0.98 | 0.97 |

CRPSS | 0.71 | 0.64 | 0.66 | 0.49 | 0.70 | 0.66 | 0.69 |

© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons by Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Klein, B.; Meissner, D.; Kobialka, H.-U.; Reggiani, P.
Predictive Uncertainty Estimation of Hydrological Multi-Model Ensembles Using Pair-Copula Construction. *Water* **2016**, *8*, 125.
https://doi.org/10.3390/w8040125

**AMA Style**

Klein B, Meissner D, Kobialka H-U, Reggiani P.
Predictive Uncertainty Estimation of Hydrological Multi-Model Ensembles Using Pair-Copula Construction. *Water*. 2016; 8(4):125.
https://doi.org/10.3390/w8040125

**Chicago/Turabian Style**

Klein, Bastian, Dennis Meissner, Hans-Ulrich Kobialka, and Paolo Reggiani.
2016. "Predictive Uncertainty Estimation of Hydrological Multi-Model Ensembles Using Pair-Copula Construction" *Water* 8, no. 4: 125.
https://doi.org/10.3390/w8040125