Next Article in Journal
The Impact of Financial and Macroeconomic Shocks on the Entropy of Financial Markets
Previous Article in Journal
Development of Postural Stability Index to Distinguish Different Stability States

Entropy 2019, 21(3), 315; https://doi.org/10.3390/e21030315

Article
Confidence Interval Estimation for Precipitation Quantiles Based on Principle of Maximum Entropy
College of Water Resources and Architectural Engineering, Northwest A&F University, Yangling 712100, China
*
Author to whom correspondence should be addressed.
Received: 15 January 2019 / Accepted: 18 March 2019 / Published: 22 March 2019

Abstract

:
The principle of maximum entropy (POME) has been used for a variety of applications in hydrology, however it has not been used in confidence interval estimation. Therefore, the POME was employed for confidence interval estimation for precipitation quantiles in this study. The gamma, Pearson type 3 (P3), and extreme value type 1 (EV1) distributions were used to fit the observation series. The asymptotic variances and confidence intervals of gamma, P3, and EV1 quantiles were then calculated based on POME. Monte Carlo simulation experiments were performed to evaluate the performance of the POME method and to compare with widely used methods of moments (MOM) and the maximum likelihood (ML) method. Finally, the confidence intervals T-year design precipitations were calculated using the POME for the three distributions and compared with those of MOM and ML. Results show that the POME is superior to MOM and ML in reducing the uncertainty of quantile estimators.
Keywords:
principle of maximum entropy; quantile estimation; confidence interval; Monte Carlo simulation; precipitation frequency analysis

1. Introduction

One of the objectives of hydrological frequency analysis is to estimate the magnitude of a hydrologic event with a given return period [1]. Due to limited data records, inappropriate assumption regarding the parent distribution, and errors associated with parameters estimation, there are inevitably uncertainties in this estimation [2,3]. Hence, a point estimate of quantile corresponding to a desired return period is usually not enough because it cannot adequately describe the reliability of the estimation. Confidence interval is a convenient approach to quantifying the uncertainty of the estimates and provides more information than just a point estimate or its standard error [4].
The calculation of confidence interval requires a standard error of quantile estimator, and several methods have been proposed for determining such standard error. Hoshi and Barges derived the expressions for calculating the sampling variances and covariances of log-Pearson type 3 (P3) distribution parameters as well as the sampling variance of T-year flood event using the method of moments (MOM) [5]. Condie gave the maximum likelihood estimators for the parameters of a log-Pearson type 3 distribution, derived the expressions for asymptotic standard error of a T-year event, and concluded that the maximum likelihood (ML) method is markedly superior to MOM in the estimation of asymptotic standard error of T-year event [6]. Lu and Stedinger derived the simple formulas for estimating the asymptotic variance of probability weighted moments (PWM) quantile estimators for generalized extreme value (GEV) distribution when the location and scale parameters were estimated with a fixed regional shape parameter or all three parameters were estimated [4]. Phien derived the explicit formulas for the variances and covariances of the parameter estimates of log-Pearson type 3 distribution when the method of direct and mixed moments was used for parameter estimation [7]. The confidence intervals of MOM and ML quantile estimators for log-Gumbel, Weibull, and generalized logistic distribution distributions have also been investigated [8,9,10].
Shannon defined the concept of entropy as a measure of uncertainty of a random variable or its probability distribution [11]. Jaynes later formulated the principle of maximum entropy (POME), which provides a rational approach to choosing the most unbiased probability distribution for hydrologic frequency analysis [12]. Sonuga developed a minimally biased probability distribution appropriate for hydrologic frequency analysis in the absence of a large amount of data [13]. Singh developed a procedure for derivation of a number of frequency distributions used in hydrology using POME [14]. Lu derived the generalized distribution for flood and extreme rainfall frequency analysis, and she concluded that the entropy-based generalized distributions are superior or comparable to other traditional distributions [15,16]. POME also provides a way to estimate parameters of a given distribution from the specified constraints. Singh summarized the entropy method for parameter estimation for the commonly used distributions and indicated that the entropy method is reasonable and efficient for parameter estimation [17]. The POME-based parameter estimations for some other distributions have also been derived [18,19,20]. In recent years, an integration of entropy and copula has been developed to construct joint distribution function capable of bivariate flood and drought analysis as well as streamflow simulation [21,22,23].
For the estimation of the POME-based variance, Phien provided the formulas for calculating the approximate variances of the parameter estimators and T-year event for the extreme value type-1 (EV1) distribution and P3 distributions [24,25]. Through applications of the formulas to simulated data, he concluded that the approximate variance of estimates of the T-year event are of sufficient accuracy. However, there are no follow-up studies on the POME-based confidence interval estimation of quantile estimators.
The objective of this study is therefore to apply POME further in the estimation of confidence intervals of quantile estimators. The Monte Carlo simulation was carried out to evaluate the performance of POME in the calculation of confidence intervals based on simulated data sets. Then, the hamma, P3, and EV1 distributions were used to fit the observed annual precipitation series. The distribution parameters and confidence intervals of annual precipitation quantiles for different return periods were estimated using POME, MOM, and ML. Finally, the confidence intervals based on different methods were compared.

2. Confidence Interval Estimation of the Quantile

2.1. Estimation of Quantile

A general form for calculating x ^ T of a given distribution can be written in terms of the distribution moments and the frequency factor K T [26]:
x ^ T = μ ^ 1 + K T μ ^ 2
where μ ^ 1 and μ ^ 2 are the mean and the standard error of the population, respectively, and they equal the sample moments only when the MOM is used for parameter estimation; K T is the frequency factor specific to the chosen distribution, which can be derived from the distribution parameters, sample size, and return period T or cumulative probability of exceedance of the design event. Expressions of K T for different distributions are commonly given in statistics texts [1].

2.2. Calculation of Confidence Interval

The standard error and confidence interval are two measures to describe the precision of a statistical quantity, such as the T-year quantile estimator x ^ T . The ( 1 α ) confidence interval for x ^ T is approximated by [27]:
x ^ L = x ^ T ± u 1 α 2 s T
where x ^ L is the confidence interval; u 1 α 2 is the quantile of the standard normal distribution for confidence levels equal to 1 α 2 ; x ^ T is the design value for the return period T; s T is the standard error of x ^ T , which can be expressed as [27]:
s T 2 = var ( x ^ T ) = ( x T θ 1 ) 2 var ( θ ^ 1 ) + ( x T θ 2 ) 2 var ( θ ^ 2 ) + ( x T θ 3 ) 2 var ( θ ^ 3 ) + 2 x T θ 1 x T θ 2 cov ( θ ^ 1 , θ ^ 2 ) + 2 x T θ 2 x T θ 3 cov ( θ ^ 2 , θ ^ 3 ) + 2 x T θ 1 x T θ 3 cov ( θ ^ 1 , θ ^ 3 )
where θ ^ i , i = 1 , 2 , 3 denotes the estimators of either moments or distribution parameters; var ( θ ^ i ) is the variance of θ i ; cov ( θ ^ i , θ ^ j ) is the covariance of θ ^ i and θ ^ j ; i , j = 1 , 2 , 3 .
In this paper, the MOM, ML, and POME were considered, and the asymptotic variances estimated by these methods are described below.

2.2.1. Method of Moments (MOM)

The MOM asymptotic variance of x ^ T for a three-parameter distribution is given by [27]:
s T 2 = μ 2 n { 1 + γ 1 K T + K T 2 4 ( γ 2 1 ) + K T γ 1 [ 2 γ 2 3 γ 1 2 6 + K T ( γ 3 3 2 γ 1 γ 2 5 2 γ 1 ) ] + ( K T γ 1 ) 2 ( γ 4 3 γ 1 γ 3 6 γ 2 + 9 4 γ 1 2 γ 2 + 35 4 γ 1 2 + 9 ) }
where γ j , j = 1 , 2 , 3 , 4 are the cumulants.
For a two-parameter distribution, the frequency factor K T does not depend on γ 1 , then K T / γ 1 = 0 in the above equation and the expression simplifies to:
s T 2 = μ 2 n [ 1 + γ 1 K T + K T 2 4 ( γ 2 1 ) ]

2.2.2. Maximum Likelihood (ML) Method

ML is a probability distribution-related method that requires the log-likelihood function of the probability density function (pdf) of a specific distribution. The ML parameters estimators of the commonly used distributions in hydrology are available in the literature [1].
The asymptotic variance and covariance terms for the ML parameter estimators are the elements of the inverse of the information matrix I [28]:
[ var ( θ ^ 1 ) cov ( θ ^ 1 , θ ^ 2 ) cov ( θ ^ 1 , θ ^ 3 ) var ( θ ^ 2 ) cov ( θ ^ 2 , θ ^ 3 ) var ( θ ^ 3 ) ] = [ E ( 2 log L θ 1 2 ) E ( 2 log L θ 1 θ 2 ) E ( 2 log L θ 1 θ 3 ) E ( 2 log L θ 2 2 ) E ( 2 log L θ 2 θ 3 ) E ( 2 log L θ 3 2 ) ] 1 = I 1
Differentiating Equation (1) with parameters θ 1 , θ 2 , and θ 3 , one obtains the derivatives of x T with respect to θ 1 , θ 2 , and θ 3 . Substituting the derivative terms and the asymptotic variances and covariances in Equation (6) into Equation (3) yields the asymptotic variance of the ML quantile estimators. Finally, the confidence interval of quantile estimators can be calculated by using Equation (2).

2.2.3. Principle of Maximum Entropy (POME) Method

POME involves essentially five steps in the estimation of the distribution parameters: (1) specification of constraints from the given information; (2) derivation of the probability density function of the maximum entropy distribution; (3) derivation of the relationship between Lagrange multipliers and constraints; (4) derivation of the relationship between Lagrange multipliers and distribution parameters; and (5) derivation of the relationship between distribution parameters and constraints [17,19].
The constraints in POME can be expressed in terms of moments, therefore, the variance and covariances of the parameters can be obtained from the relationship between the variance and covariances of the moments and that of the parameter estimates. Let P, Q, and R denote the three moments, thus one can approximately write the vector of variance and covariances of P, Q, and R of a three-parameter distribution as [24]:
V M = D V P
where V M and V P are the vectors of variance and covariances of the moments and parameter estimators, respectively:
V M = [ var ( P ) var ( Q ) var ( R ) cov ( P , Q ) cov ( Q , R ) cov ( P , R ) ] , V P = [ var ( θ ^ 1 ) var ( θ ^ 2 ) var ( θ ^ 3 ) cov ( θ ^ 1 , θ ^ 2 ) cov ( θ ^ 2 , θ ^ 3 ) cov ( θ ^ 1 , θ ^ 3 ) ]
and θ 1 , θ 2 , and θ 3 are the distribution parameters; D is the matrix with elements d i j ( 1 i , j 6 ), which are the partial derivatives of the moments with respect to the distribution parameters. For example:
d 11 = ( P / θ 1 ) 2 , d 12 = ( P / θ 2 ) 2 , d 13 = ( P / θ 3 ) 2 , d 14 = 2 ( P / θ 1 ) ( P / θ 2 ) , d 15 = ( P / θ 2 ) ( P / θ 3 ) , d 16 = 2 ( P / θ 1 ) ( P / θ 3 ) ,
Consequently, the V p can be calculated using Equation (10) as long as the elements of D and the V M have been calculated.
V p = D 1 V M
where D 1 is the inverse matrix of D.
Substituting the elements of V p and the partial derivatives of x T with respect to distribution parameters into Equation (3), one can obtain the variances of quantile estimators. The confidence interval of quantile estimators can then be calculated by using Equation (2).
The variances and covariances of MOM parameter estimates are calculated by using the relationship between the parameters and the population moments, which is relatively simple and understandable. However, the calculation of the second and higher order sample moments introduces sampling errors, which affects the accuracy of the estimation. The ML method is frequently applied owing to its large sample properties of yielding consistent estimates with minimum variance. Estimates for small samples have found general acceptance in practice as well [28]. However, this method involves some complicated calculations and approximations, which makes it inconvenient. The POME requires less artificial assumption due to insufficient data. Though it is comparable to the ML in parameter estimation, POME has the advantages of simple and fast calculation [17]. The calculation POME-based confidence interval also requires some approximations. Therefore, it is necessary to compare the performance of different methods to choose the most efficient one.

3. Asymptotic Variances of Quantile Estimators for Different Distributions

Three commonly used distributions—gamma distribution, P3 distribution, and EV1 were considered in this study.

3.1. Gamma Distribution

The pdf of the gamma distribution is given by:
f ( x ) = 1 α β Γ ( β ) x β 1 exp ( x / α )
where α and β are the scale and shape parameters, and Γ ( ) is the gamma function, and 0 < x < .
For the gamma distribution, the T-year quantile is given by:
x ^ T = α ^ β ^ + K T α ^ 2 β ^
Differentiation of Equation (12) with respect to α and β yields:
x T α = β + K T β α | α |
x T β = α + K T 2 α 2 / β α 2 β K T C S
where K T / C S can be calculated by using Wilson–Hilferty transformation [1].

3.1.1. Estimation of Asymptotic Variances by MOM and ML

Based on MOM, the standard error of x ^ T for the gamma distribution can be calculated directly by [29]:
s T 2 = μ 2 n [ ( 1 + K T C v ) 2 + 1 2 ( K T + 2 C v K T γ 1 ) 2 ( 1 + C v 2 ) ]
where C v is the coefficient of variation, and C v = μ 2 1 / 2 / μ 1 ; γ 1 = C s is the coefficient of skewness.
The asymptotic variance and covariances of ML parameter estimators are derived as [1]:
[ var ( α ) cov ( α , β ) var ( β ) ] = [ α 2 ψ D α D β D ]
where ψ = ψ ( β ) = d 2 log Γ ( β ) d β 2 is the tri-gamma function; D = 1 β ψ 1 .
Substituting Equations (13) and (14) and the variances and covariances terms in Equation (16) into Equation (3) yields the variance of the ML quantile estimator.

3.1.2. Estimation of Asymptotic Variances by POME

For the gamma distribution, the relation between parameters and constraints can be expressed as [17]:
{ E ( x ) = α β E [ ln ( x ) ] = ln ( α ) + ψ
where ψ = ψ ( β ) = d log Γ ( β ) d β is digamma function. The parameter estimators α ^ and β ^ can be obtained by solving the following equations:
{ α β = X ¯ ln ( α ) + ψ ( β ) = W ¯
where X ¯ is the sample mean of x, and W ¯ is the sample mean of the random variable W = ln ( x ) .
Then, V M and V p are written by:
V M = [ var ( X ¯ ) var ( W ¯ ) cov ( X ¯ , W ¯ ) ] , V p = [ var ( α ^ ) var ( β ^ ) cov ( α ^ , β ^ ) ]
For the gamma distribution,
var ( X ¯ ) = var ( x ) / n = α 2 β / n
Exact formulas for computing var ( W ¯ ) and cov ( X ¯ , W ¯ ) of V M are derived in Appendix A. Accordingly,
var ( W ¯ ) = var ( W ) / n = ψ / n
cov ( X ¯ , W ¯ ) = α / n
Consequently, one obtains:
V M = 1 n [ α 2 β ψ α ]
Additionally, taking the partial derivates of X ¯ and W ¯ with respect to α and β , one can obtain the matrix D:
D = [ β 2 α 2 2 α β 1 / α 2 ψ 2 2 ψ / α β / α α ψ 1 + β ψ ]
Thus, all the components of V M and D are obtained. Substituting V M and D [Equations (23) and (24)] into Equation (10) yields V P . The variance of the quantile estimator can then be obtained by substituting the terms of V P and Equations (13) and (14) into Equation (3).

3.2. Pearson Type 3 (P3) Distribution

The pdf of P3 distribution is given by:
f ( x | α , β , γ ) = 1 α Γ ( β ) ( x γ α ) β 1 e x γ α , γ < x <
where α , β and γ are the scale, shape, and location parameters, respectively, and γ < x < .
The T-year quantile of P3 distribution is given by:
x ^ T = α ^ β ^ + γ ^ + K T α ^ 2 β ^
Taking partial derivatives of Equation (34) with respect to α , β , γ yields:
x T α = β + K T β α | α |
x T β = α + K T 2 α 2 β α 2 β K T C S
x T γ = 1

3.2.1. Estimation of Asymptotic Variances by MOM and ML

For the P3 distribution, the asymptotic variance of MOM quantile estimator is given by:
s T 2 = μ 2 n [ 1 + γ 1 K T + K T 2 2 ( 3 4 γ 1 2 + 1 ) + 3 K T K T γ 1 ( γ 1 + γ 1 3 4 ) + 3 ( K T γ 1 ) 2 ( 2 + 3 γ 1 2 + 5 γ 1 4 8 ) ]
The asymptotic variance and covariances of ML parameter estimators are given by [1]:
[ var ( α ) cov ( α , β ) cov ( α , γ ) var ( β ) cov ( β , γ ) var ( γ ) ] = [ 1 n α 2 G [ ψ ( β ) ( β 2 ) 1 ( β 1 ) 2 ] 1 n α 3 G ( 1 β 2 1 β 1 ) 1 n α 2 G [ 1 β 1 ψ ( β ) ] 2 α 2 ( β 2 ) 1 n α 3 G [ β ( β 1 ) 1 ] 1 n α 2 G [ β ψ ( β ) 1 ] ]
where G = 1 ( β 2 ) α 4 [ 2 ψ 2 β 3 ( β 1 ) 2 ] .
Substituting Equations (27)–(29) and the variance and covariance terms in Equation (31) into Equation (3) yields the asymptotic variance of the quantile estimator.

3.2.2. Estimation of Asymptotic Variances by POME

On the basis of POME, the relation between parameters and constraints for P3 distribution is given by [17]:
{ E ( x ) = α β + γ E [ ln ( x γ ) ] = ln ( α ) + ψ var ( x ) = α 2 β
where ψ = ψ ( β ) is digamma function. The parameter estimators α ^ , β ^ , and γ ^ can be obtained by solving the following equations:
{ α β + γ = X ¯ ln ( α ) + ψ ( β ) = W ¯ 1 α 2 β = S 2
where X ¯ and S 2 are the sample mean and variance of x, and W ¯ 1 is the sample mean of the random variable W 1 defined as W 1 = ln ( x γ ) .
Therefore, V M and V p can be written by:
V M = [ var ( X ¯ ) var ( S 2 ) var ( W ¯ 1 ) cov ( X ¯ , S 2 ) cov ( S 2 , W ¯ 1 ) cov ( X ¯ , W ¯ 1 ) ] , V p = [ var ( α ^ ) var ( β ^ ) var ( γ ^ ) cov ( α ^ , β ^ ) cov ( β ^ , γ ^ ) cov ( α ^ , γ ^ ) ]
Following Phine [24], the V M is given by:
V M = 1 n [ α 2 β 2 α 4 β ( β + 3 ) ψ ( β ) 2 α 3 β α 2 α ]
Taking partial derivatives of X ¯ , W ¯ a n d S 2 with respect to α , β , and γ yields the matrix D:
D = [ β 2 α 2 1 2 α β 2 α 2 β 4 α 2 β 2 α 4 0 4 α 3 β 2 0 0 1 / α 2 ψ 2 0 2 ψ / α 0 0 2 α β α 3 0 3 α 2 β α 2 2 α β 2 β α 2 ψ 0 α ( 1 + 2 β ψ ) 0 0 β / α α ψ 0 1 + β ψ ψ 1 / α ]
where ψ = ψ ( β ) = d 2 log Γ ( β ) d β 2 is the tri-gamma function.
Substituting V M and D [Equations (35) and (36)] into Equation (10) yields V P . Accordingly, the asymptotic variance of the quantile estimator can then be obtained by substituting the terms of V P and Equations (27)–(29) into Equation (3).

3.3. Extreme Value Type 1 (EV1) Distribution

The pdf and the cumulative distribution function of EV1 distribution can be expressed respectively as:
f ( x ) = 1 α exp [ x u α exp ( x u α ) ]
F ( x ) = exp [ exp ( x u α ) ]
where α and u are the scale and shape parameters, respectively, and < x < .
The T-year quantile of EV1 distribution can be obtained from Equation (38) by substituting F ( x ) = 1 1 / T and solving for x:
x ^ T = u ^ α ^ log ( log ( 1 1 / T ) )
Differentiating Equation (39) with α and u yields the derivatives of x T with respect to α and u:
x T α = log ( log ( 1 1 / T ) )
x T u = 1

3.3.1. Estimation of asymptotic variance by MOM and ML

The asymptotic variance of MOM quantile estimator is given by:
s T 2 = μ 2 n ( 1 + 1.1396 K T + 1.1 K T 2 )
where K T is given by:
K T = 6 π [ log ( log ( 1 1 T ) ) + ε ]
where ε is Euler constant, ε = 0.5772157 .
The variance and covariances for the ML parameter estimators are given by [1]:
[ var ( α ) cov ( α , u ) var ( u ) ] = α 2 n [ 0.8046 0.2287 1.1128 ]
Substituting Equations (40) and (41) and the variance and covariance terms in Equation (44) into Equation (3) yields the asymptotic variance of the quantile estimator.

3.3.2. Estimation of Asymptotic Variances by POME

The relation between parameters and constraints for EV1 distribution can be expressed as [17]:
{ E [ ( x u ) α ] = ε E [ exp ( x u α ) ] = 1
The estimators α ^ and u ^ of the parameters can be obtained by solving the following equations:
{ Y ¯ = ε V ¯ = 1
where Y ¯ and V ¯ are the sample mean of variables defined by y = ( x u ) / α and V = exp ( y ) , respectively.
The variances and covariances of the moments and parameter estimators are written respectively as:
V M = [ var ( Y ¯ ) var ( V ¯ ) cov ( Y ¯ , V ¯ ) ] , V p = [ var ( α ^ ) var ( u ^ ) cov ( α ^ , u ^ ) ]
According the derivations in [17], the V M is given by:
V M = 1 n [ π 2 / 6 1 1 ]
Taking partial derivatives of Y ¯ and V ¯ with respect to α and u yields:
D = α 2 [ ε 2 1 2 ε W ¯ 2 1 2 W ¯ ε W ¯ 1 ( ε + W ¯ ) ]
where W = y exp ( y ) .
Substituting V M and D [Equations (48) and (49)] into Equation (10) yields V P . The asymptotic variance of quantile estimator can then be obtained by substituting the terms of V P and Equations (40) and (41) into Equation (3).

4. Simulation Experiments

In this section, the Monte Carlo simulation experiments were performed to evaluate the performance of POME in calculation of the asymptotic variances and confidence intervals of quantiles and to compare it with the MOM and ML methods. In this study, four kinds of data sets were generated from the Wakeby distribution with parameters as shown in Table 1 [16,30]. The quantile function of the Wakeby distribution is given by [31]:
x ( F ) = ξ + α β [ 1 ( 1 F ) β ] γ δ [ 1 ( 1 F ) δ ]
where F is the uniform (0,1) variate, and ξ , α , β , γ , δ are the parameters.
Ns = 1000 samples with size n (n = 20, 50, 100, 1000) were generated from each Wakeby distribution. Then, the quantiles x ^ T corresponding to different return periods (T = 10, 100, and 200) and their asymptotic variances and confidence intervals were calculated for EV1. Table 2 lists the median values of the estimated quantiles ( x ^ T ), standard errors (St), and confidence interval width (CI width).
From Table 2, generally for all methods and for all cases, it was observed that the standard errors and confidence interval widths of the quantiles increased with the return period T and decreased with the sample size. For all cases, the selected three methods exhibited very similar behaviors. Thus, we would take case III as an example in the latter discussion.
From case III, it was observed that the POME generally gave the smallest median of both standard errors and confidence interval widths of quantiles regardless of the sample size and return period. MOM was always the worst of the three competing methods and gave the largest results. The results of ML fell between MOM and POME. For example, when the sample size equaled 50, the median standard errors of MOM, ML, and POME quantile estimator for return period T = 100 were 69.2, 66.2, and 63.5, respectively. Correspondingly, the confidence interval widths were 271.4, 259.4, and 248.8, respectively, which indicated that the uncertainty of the POME estimator was less than that of the MOM and the ML estimators. Therefore, the performance of the POME was found to be superior to the MOM and the ML.
In addition, for each method considered, the median of both standard errors and confidence interval widths of quantiles decreased significantly when the sample size increased from 20 to 1000. For T = 100, when the sample size increased from 20 to 1000, the median of standard errors of MOM quantile estimators decreased from 109.5 to 8.2, the median of standard errors of ML quantile estimators decreased from 99.9 to 8.7, and the median of standard errors of POME quantile estimators decreased from 99 to 8.3. The median of confidence interval widths decreased from 429.3 to 32.2 for MOM quantile estimators, 391.8 to 34.2 for ML quantile estimators, and 388 to 32.6 for POME quantile estimators. This was an indication of the influence sample size had on the estimation accuracy.

5. Application

The annual precipitation data from four gauging stations at the Weihe River basin in China were considered as the case study. All data were obtained from the National Climate of China Meteorological Administration and were complete. The detailed information of these data is given in Table 3.
The gamma, P3, and EV1 distributions were used to fit the data set, and the MOM, ML, and POME were used to estimate the parameters of these distributions, as given in Table 4. It can be seen that the parameters of the gamma distribution estimated by MOM, ML, and POME were very close, as were the EV1 distribution, while those of the P3 distribution departed significantly.
To evaluate and compare the performances of the three methods and the distributions, the ordinary least square (OLS) criterion, Akaike information criterion (AIC), and quasi-optimal deterministic coefficient test (QD) were employed and can be defined as:
O L S = 1 n i = 1 n ( x i x ^ i ) 2
A I C = n ln ( 1 n i = 1 n ( x i x ^ i ) 2 ) + 2 m
Q D = 1 i = 1 n ( x i x ^ i ) 2 i = 1 n ( x i x ¯ ) 2
where x i and x ^ i are the observed data and the predicted values of a given (i-th) quantile, respectively, x ¯ is the mean value of observed data, m is the number of parameters of a given model, and n is the sample size.
The OLS criterion is recommended as a curve optimization rule for measuring the difference between empirical and theoretical values in hydrological frequency analysis in China. The smaller OLS values represent the better performance of the model. The AIC is more appropriate for the comparison of models have different number of parameters. Given a set of candidate models for the data, the best model is the one with the minimum AIC value. QD is used to describe the fitting degree of observed values and theoretical values and the best fit model is the one that gets the QD value closest to 1. The OLS, AIC, and QD were calculated as given in Table 5.
It is seen from Table 5 that the selected best parameter estimation method for each distribution by the three criterions is coincident and the result of the best fitted distribution for each station by the three criterions is the same as well. Take the Changwu station in Table 5 for example. According to the smallest OLS and AIC values and the largest QD values, the POME, MOM and POME are suggested to be the best methods for parameter estimation for Gamma, P3 and EV1 distributions, respectively. And the best fitted distribution for Changwu station recommended by the OLS, AIC and QD criteria is P3 distribution. Additionally, according to the results given in Table 5, the best fitted distributions for the gauging stations Meixian, Tongguan and Lintong recommended by the OLS, AIC and QD methods, is EV1 distribution with the parameters estimated by POME. Thus the best estimation method for each station is POME and this is coincident with the results of the simulation experiments in Section 4, which shows that the performance of POME is better than MOM and ML. The bold values in the table denote the smallest OLS and AIC values and the largest QD values.
The quantiles along with the standard errors and 95% confidence intervals for 10, 20, 50, 100, 200, and 500 years return periods of the best fitted distribution based on the parameters estimated by POME are given in Table 5. For the sake of comparison, the quantiles, standard errors, and 95% confidence interval widths based on MOM and ML are also given in Table 6. The results show that the standard errors and confidence interval widths of quantile estimators obtained by POME were smaller than those obtained by the MOM and the ML methods with the exception of the results of T = 10 at Changwu station, which indicated that the POME yielded more precise parameters and quantiles estimations.
To better understand the performance of the different methods, the differences in the uncertainty reductions for the standard errors and 95% confidence interval widths of the quantile estimators were given in terms of relative deviation, as shown in Table 7. For the relatively long return period ( T 50 ), there were significant reductions in the standard errors and 95% confidence interval widths obtained by POME compared to MOM. For example, for a return period of T = 500, the reductions in standard errors and the confidence interval widths were of about 32%, 17%, 17%, and 16% for Changwu, Lintong, Meixian, and Tongguan, respectively.
It can also be seen from Table 6 that, for Changwu station, the reduction in standard errors and 95% confidence interval widths obtained by POME were significant when compared to ML. For example, the reductions in the standard errors and confidence interval widths of a 500-year quantile was about 19%. For Lintong, Meixian, and Tongguan stations, the reductions were relatively smaller—about 6%, 6%, and 5%, respectively. Overall, the POME provided more accurate quantile estimators.

6. Conclusions

In this study, the POME method was applied for the estimation of the asymptotic variances and confidence intervals of quantiles, and the corresponding calculation formulas for gamma, P3, and EV1 distributions based on the POME method were deduced. The calculation procedures of the MOM and the ML methods were also reviewed briefly for comparison. The Monte Carlo simulation experiments were carried out to evaluate the performance of the POME method and to compare it with the MOM and the ML methods. In addition, annual precipitation data from four stations at the Weihe River basin in China were selected as the case study. The following conclusions were drawn from this study:
(1)
The calculation formulas of the asymptotic variances and confidence intervals of quantiles for three distributions based on POME are given. The results of simulation experiments and the case study show that the POME method can provide an effective way for reducing the uncertainty of quantile estimators.
(2)
Results of the simulation experiments demonstrate that the POME method yields the smallest standard errors and the narrowest confidence intervals of quantile estimators compared with the results of MOM and ML. This may benefit from fewer sampling errors and approximation in derivation. Thus, the POME can give more accurate estimates. Furthermore, the standard errors and confidence interval widths of the quantiles increased with the return period T and decreased with the sample size.
(3)
Results of the case study indicate that when using different criteria for distribution selection, the results are coincidental, and the POME is the optimal method for parameter estimation. Furthermore, the POME can give more reliable precipitation quantiles since the standard errors and 95% confidence interval widths of precipitation quantiles obtained by POME are smaller than those obtained by the MOM and the ML methods.
This study investigated the calculation of asymptotic variances and confidence intervals based on POME for three commonly used distributions and compared the performance of POME with that of MOM and ML. In addition, the POME-based asymptotic variances and confidence intervals of quantiles for more distributions deserve more thorough investigation.

Author Contributions

S.S. and Ting Wei designed the computations; S.S. and T.W. wrote the paper. All authors have read and approved the final manuscript.

Funding

The present study is financially supported by the National Natural Science Foundation of China (Grant Nos. 51479171, 41501022 and 51409222).

Acknowledgments

The authors would like to appreciate the editor and anonymous reviewers for their constructive comments which greatly improve the quality of this manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

The gamma distribution: Estimation of var ( W ¯ ) , cov ( X ¯ , W ¯ ) .

Estimation of var ( W ¯ )

For the gamma distribution, the mean and variance are given by:
E ( x ) = α β
var ( x ) = α 2 β
E ( W ) = ln ( α ) + ψ
Since var ( W ¯ ) = var ( W ) / n = ( E ( W 2 ) [ E ( W ) ] 2 ) / n , one needs to obtain the E ( W 2 ) .
E ( W 2 ) = E [ ( ln x ) 2 ] = 0 [ ln x ] 2 1 α β Γ ( β ) x β 1 exp ( x / α ) d x = ( ln α ) 2 Γ ( β ) 0 y β 1 e ( y ) d y + 2 ln α Γ ( β ) 0 ( ln y ) y β 1 e ( y ) d y + 1 Γ ( β ) 0 ( ln y ) 2 y β 1 e ( y ) d y
Let y = x / α , x = α y , d x = α d y . Substituting these quantities into Equation (A4) and changing the integral limits, we obtain:
E ( W 2 ) = ( ln α ) 2 Γ ( β ) 0 y β 1 e ( y ) d y + 2 ln α Γ ( β ) 0 ( ln y ) y β 1 e ( y ) d y + 1 Γ ( β ) 0 ( ln y ) 2 y β 1 e ( y ) d y
Using the property of the gamma function d k Γ ( β ) / d β k = 0 ( ln y ) k y β 1 e y d y , then Equation (A5) can be expressed as:
E ( W 2 ) = ( ln α ) 2 Γ ( β ) 0 y β 1 e ( y ) d y + 2 ln α Γ ( β ) 0 ( ln y ) y β 1 e ( y ) d y + 1 Γ ( β ) 0 ( ln y ) 2 y β 1 e ( y ) d y = ( ln α ) 2 + 2 ln α ψ + ψ + ψ 2
Then, we obtain var ( W ¯ ) :
var ( W ¯ ) = ψ / n

Estimation of cov ( X ¯ , W ¯ )

cov ( X ¯ , W ¯ ) = cov ( X , W ) / n = [ E ( x W ) E ( x ) E ( W ) ] / n
E ( x W ) is written by:
E ( x W ) = 0 x ln x 1 α β Γ ( β ) x β 1 exp ( x / α ) d x = α ln α Γ ( β ) 0 y β e ( y ) d y + α Γ ( β ) 0 ( ln y ) y β e ( y ) d y = α + α β [ ln α + ψ ( b ) ]
Substitution of Equations (A1), (A3), and (A9) into Equation (A8) yields:
cov ( X ¯ , W ¯ ) = α / n

References

  1. Hamed, K.; Rao, A.R. Flood Frequency Analysis; Taylor & Francis Inc.: Bosa Roca, FL, USA, 1999. [Google Scholar]
  2. Obeysekera, J.; Salas, J.D. Quantifying the uncertainty of design floods under nonstationary conditions. J. Hydrol. Eng. 2014, 19, 1438–1446. [Google Scholar] [CrossRef]
  3. Chowdhury, J.U.; Stedinger, J.R. Confidence interval for design floods with estimated skew coefficient. J. Hydraul. Eng. 1991, 117, 811–831. [Google Scholar] [CrossRef]
  4. Lu, L.H.; Stedinger, J.R. Sampling variance of normalized GEV/PWM quantile estimators and a regional homogeneity test. J. Hydrol. 1992, 138, 223–245. [Google Scholar] [CrossRef]
  5. Hoshi, K.; Burges, S.J. Sampling properties of parameter estimates for the log Pearson type 3 distribution, using moments in real space. J. Hydrol. 1981, 53, 305–316. [Google Scholar] [CrossRef]
  6. Condie, R. The Log Pearson Type 3 Distribution: The T-Year Event and Its Asymptotic Standard Error by Maximum Likelihood Theory. Water Resour. Res. 1977, 13, 987–991. [Google Scholar] [CrossRef]
  7. Phien, H.N. Variance of the T-year event in the log Pearson type-3 distribution. J. Hydrol. 1985, 77, 141–158. [Google Scholar] [CrossRef]
  8. Heo, J.H.; Salas, J.D. Estimation of quantiles and confidence intervals for the log-Gumbel distribution. Stoch. Hydrol. Hydraul. 1996, 10, 187–207. [Google Scholar] [CrossRef]
  9. Heo, J.H.; Salas, D.J.; Kim, K.D. Estimation of confidence intervals of quantiles for the Weibull distribution. Stoch. Environ. Res. Risk Assess. 2001, 15, 284–309. [Google Scholar] [CrossRef]
  10. Shin, H.; Kim, T.; Kim, S.; Heo, J.H. Estimation of asymptotic variances of quantiles for the generalized logistic distribution. Stoch. Environ. Res. Risk Assess. 2010, 24, 183–197. [Google Scholar] [CrossRef]
  11. Shannon, E.C. A mathematical theory of communication. Bell Labs Tech. J. 1948, 4, 379–423. [Google Scholar] [CrossRef]
  12. Jaynes, E.T. On the rationale of maximum-entropy methods. Proc. IEEE 1982, 70, 939–952. [Google Scholar] [CrossRef]
  13. Sonuga, J.O. Principle of maximum entropy in hydrologic frequency analysis. J. Hydrol. 1972, 17, 177–191. [Google Scholar] [CrossRef]
  14. Singh, V.P.; Rajagopal, A.K.; Singh, K. Derivation of some frequency distributions using the principle of maximum entropy (POME). Adv. Water Resour. 1986, 9, 91–106. [Google Scholar] [CrossRef]
  15. Lu, C.; Singh, V.P. Entropy-based derivation of generalized distributions for hydrometeorological frequency analysis. J. Hydrol. 2018, 557, 699–712. [Google Scholar]
  16. Chen, L.; Singh, V.; Xiong, F. An Entropy-Based Generalized Gamma Distribution for Flood Frequency Analysis. Entropy 2017, 19, 239. [Google Scholar] [CrossRef]
  17. Singh, V. Entropy-Based Parameter Estimation in Hydrology; Kluwer Academic Publishers: Boston, MA, USA, 1998. [Google Scholar]
  18. Singh, V.P.; Deng, Z.Q. Entropy-Based Parameter Estimation for Kappa Distribution. J. Hydrol. Eng. 2003, 8, 81–92. [Google Scholar] [CrossRef]
  19. Song, S.; Song, X.; Kang, Y. Entropy-Based Parameter Estimation for the Four-Parameter Exponential Gamma Distribution. Entropy 2017, 19, 189. [Google Scholar] [CrossRef]
  20. Fiorentino, M.; Arora, K.; Singh, V.P. The two-component extreme value distribution for flood frequency analysis: Derivation of a new estimation method. Stoch. Hydrol. Hydraul. 1987, 1, 199–208. [Google Scholar] [CrossRef]
  21. Hao, Z.; Singh, V.P. Modeling multisite streamflow dependence with maximum entropy copula. Water Resour. Res. 2013, 49, 7139–7143. [Google Scholar] [CrossRef][Green Version]
  22. Kong, X.M.; Huang, G.H.; Fan, Y.R.; Li, Y.P. Maximum entropy-Gumbel-Hougaard copula method for simulation of monthly streamflow in Xiangxi river, China. Stoch. Env. Res. Risk Assess. 2015, 29, 833–846. [Google Scholar] [CrossRef]
  23. Aghakouchak, A. Entropy—Copula in Hydrology and Climatology. J. Hydrometeorol. 2014, 15, 2176–2189. [Google Scholar] [CrossRef]
  24. Phien, H.N.; Nguyen, V.T.V. Variances and covariances of the maximum entropy estimators for the Pearson type-3 distribution. Can. J. Civ. Eng. 1990, 17, 590–596. [Google Scholar] [CrossRef]
  25. Phien, H.N. Sampling properties of the maximum entropy estimators for the extreme-value type-1 distribution. J. Hydrol. 1986, 86, 391–398. [Google Scholar] [CrossRef]
  26. Chow, V.T. A General Formula for Hydrologic Frequency Analysis. Eos Trans. Am. Geophys. Union 1951, 32, 231–237. [Google Scholar] [CrossRef]
  27. Kite, G.W. Confidence Limits for Design Events. Water Resour. Res. 1975, 11, 48–53. [Google Scholar] [CrossRef]
  28. Chowdhary, H.; Singh, V.P. Reducing uncertainty in estimates of frequency distribution parameters using composite likelihood approach and copula-based bivariate distributions. Water Resour. Res. 2010, 46. [Google Scholar] [CrossRef][Green Version]
  29. Bobee, B. Sample error of T-year events commuted by fitting a Pearson type 3 distribution. Water Resour. Res. 1973, 9, 1264–1270. [Google Scholar] [CrossRef]
  30. Strupczewski, W.G.; Kochanek, K.; Singh, V.P.; Węglarczyk, S. Are parsimonious FF models more reliable than true ones? II. Comparative assessment of the performance of simple models versus the parent distributions. Acta Geophys. Pol. 2005, 53, 437–457. [Google Scholar]
  31. Houghton, J.C. Birth of a Parent: The Wakeby Distribution for Modeling Flood Flows. Water Resour. Res. 1978, 14, 1105–1109. [Google Scholar] [CrossRef]
Table 1. The Monte Carlo simulation data sets generated from the Wakeby distribution.
Table 1. The Monte Carlo simulation data sets generated from the Wakeby distribution.
CaseξαβγδCvCs
I016160.40.040.381.10
I15.4308.810.2538.5−0.300.360.48
III273.69521.101.254.77−0.210.240.64
Table 2. Median of estimated quantiles ( x ^ T ), standard error (St), and 95% confidence interval (CI) width from generated data; MOM = methods of moments, ML = maximum likelihood, POME = principle of maximum entropy.
Table 2. Median of estimated quantiles ( x ^ T ), standard error (St), and 95% confidence interval (CI) width from generated data; MOM = methods of moments, ML = maximum likelihood, POME = principle of maximum entropy.
CaseSample Size nReturn Period TMOMMLPOME
x ^ T StCI Width x ^ T StCI Width x ^ T StCI Width
I20101.970.220.852.080.220.852.020.200.80
1002.830.411.603.060.381.492.970.351.37
2003.090.471.833.360.431.693.260.401.55
50102.000.140.562.130.150.582.060.130.52
1002.890.271.053.200.261.013.050.230.89
2003.140.311.203.520.291.143.350.261.00
100102.010.100.402.160.110.422.080.100.37
1002.930.190.763.240.190.733.090.160.64
2003.200.220.873.560.210.823.390.180.72
1000102.020.030.132.170.030.132.090.030.12
1002.950.060.243.270.060.233.120.050.20
2003.230.070.283.600.070.273.420.060.23
II2010106.912.147.3111.111.946.5109.711.444.7
100155.322.889.3164.720.781.3161.319.777.2
200169.826.0102.1180.423.491.9176.822.287.1
5010106.912.147.3112.37.730.2110.37.428.9
100155.322.889.3167.813.552.8163.712.850.0
200169.826.0102.1184.415.259.7179.414.456.5
10010107.15.521.5113.05.521.6110.85.320.6
100156.310.340.5169.09.637.7164.89.135.7
200170.911.846.4185.610.942.6180.810.340.3
100010107.51.76.8113.51.86.9111.21.76.6
100156.73.312.9169.93.112.0165.52.911.3
200171.33.814.7186.63.513.6181.63.312.8
III2010676.458.1227.6685.757.2224.1698.557.2224.4
100905.4109.5429.3944.999.9391.8958.199.0388.0
200974.0125.2490.91023.0113.0442.91036.1111.6437.6
5010675.036.7143.9700.537.8148.4700.236.8144.3
100905.569.2271.4971.766.2259.4965.663.5248.8
200974.379.2310.31053.774.8293.31044.371.7280.9
100101065.592.4362.01160.386.3338.21148.582.5323.5
100674.826.0102.0707.765.5106.9700.226.2102.7
200906.949.1192.4985.341.5186.9968.745.2177.4
100010975.756.1220.01067.247.7211.31048.451.1200.2
100674.78.232.2713.68.734.2701.48.332.6
200906.915.560.8994.115.359.8971.314.456.3
Table 3. Basic information on each of the monthly precipitation series used in this study.
Table 3. Basic information on each of the monthly precipitation series used in this study.
Station NameRecord Length (Year)Mean (mm)Standard DeviationCoefficient of VariationSkewnessFirst-Order Serial Correlation Coefficient
Changwu51580.6131.81770.22700.50703.2153
Lintong50579.5129.20140.22300.62993.7670
Meixian50578.0129.72140.22450.58283.4614
Tongguan52605.5143.46480.23690.57713.6438
Table 4. Parameter values of each distribution estimated by the three methods; P3 = Pearson type 3, EV1 = extreme value type 1.
Table 4. Parameter values of each distribution estimated by the three methods; P3 = Pearson type 3, EV1 = extreme value type 1.
Station NameMethodGammaP3EV1
αβαβγαu
ChangwuMOM29.928219.399315.562333.414760.5782102.7783521.2619
ML29.051019.985116.009732.587858.8660114.9989518.3959
POME29.345819.789311.101739.5620141.3794111.0142516.5091
LintongMOM28.806320.116910.080440.6938169.2817100.7383521.3449
ML27.687320.929917.148830.690853.1797113.0664519.0634
POME28.684920.219210.313940.2305164.5577108.5086516.8608
MeixianMOM29.116119.849811.776237.8015132.7913101.1438519.5689
ML28.087520.576817.727130.338340.1383113.7362517.1681
POME28.757520.108910.800239.4726151.6378109.1477514.9499
TongguanMOM33.992717.812212.009041.3991108.3228111.8595540.9202
ML32.953518.374015.034336.574155.6191125.0433537.9242
POME33.724317.965310.375644.5388143.3688120.6552535.8444
Table 5. Ordinary least square (OLS), Akaike information criterion (AIC) and quasi-optimal deterministic coefficient test (QD) values of three distributions calculated by MOM, ML, and POME.
Table 5. Ordinary least square (OLS), Akaike information criterion (AIC) and quasi-optimal deterministic coefficient test (QD) values of three distributions calculated by MOM, ML, and POME.
Station NameMethodGammaP3EV1
OLSAICQDOLSAICQDOLSAICQD
ChangwuMOM16.6696292.98610.983716.3741291.16130.984319.3927308.41970.9779
ML17.5392298.17290.981917.0194295.10410.98316.8854294.29780.9833
POME17.2128296.25650.982616.0997289.43750.984816.6415292.81350.9837
LintongMOM19.6949304.03610.976318.516297.86380.97918.8765299.7920.9782
ML20.6502308.77250.973920.1246306.19440.975216.3975285.71290.9836
POME19.7466304.29820.976218.5442298.01550.97916.0775283.7420.9842
MeixianMOM17.9613294.82190.980416.9804289.20610.982517.8405294.14690.9807
ML18.9359300.1060.978318.5121297.84260.979215.0757277.30830.9862
POME18.2446296.38690.979816.8553288.46640.982814.6984274.77360.9869
TongguanMOM20.419319.71260.979320.0141317.62940.980221.9677327.31560.9761
ML21.2629323.92420.977620.8105321.68750.978519.5845315.3730.981
POME20.5825320.5420.97919.9542317.31780.980319.2731313.70590.9816
Bold values indicate the smallest OLS and AIC values and the largest QD values.
Table 6. Quantile estimators, standard error, and 95% confidence interval widths based on MOM, ML, and POME for the annual precipitation (mm).
Table 6. Quantile estimators, standard error, and 95% confidence interval widths based on MOM, ML, and POME for the annual precipitation (mm).
Station NameBest fitted DistributionReturn Period (Year)MOMMLPOME
QuantileStandard ErrorConfidence Interval WidthQuantileStandard ErrorConfidence Interval widthQuantileStandard ErrorConfidence Interval Width
ChangwuP310755.031.2122.1753.130.5119.6755.731.7124.3
20814.740.7159.7811.938.9152.3817.538.4150.7
50885.656.1220.0881.951.7202.5891.647.3185.6
100935.369.1270.7930.862.2243.8943.854.0211.6
200982.382.8324.5977.173.3287.2993.560.6237.5
5001041.4101.8398.91035.388.5347.01056.369.2271.2
LintongEV110704.431.8124.7724.532.2126.1714.030.6120.0
20748.037.5146.9773.537.0144.9761.035.0137.1
50820.647.4185.7854.945.2177.3839.242.5166.7
100984.870.7277.11039.264.6253.31016.060.3236.2
2001054.880.8316.91117.873.1286.41091.568.0266.5
5001147.394.3369.71221.684.3330.31191.178.2306.6
MeixianEV110703.331.9125.2723.832.4126.9713.330.8120.7
20747.237.6147.5773.137.2145.8760.635.2137.9
50820.047.6186.4855.045.5178.3839.142.8167.6
100984.871.0278.31040.465.0254.81017.060.6237.6
2001055.281.2318.21119.573.5288.11093.068.4268.0
5001148.094.7371.21223.984.8332.31193.278.7308.4
TongguanEV110744.234.6135.7765.134.9136.8755.133.5131.5
20792.640.8160.0819.340.1157.1807.438.4150.4
50873.251.6202.2909.349.0192.2894.246.7183.1
1001055.577.0301.81113.170.1274.71090.966.3259.9
2001133.388.0345.11200.179.2310.61174.874.8293.3
5001236.0102.7402.51314.991.4358.21285.586.1337.6
Table 7. Change in the uncertainty in quantile estimators based on POME compared with the MOM and ML methods (%).
Table 7. Change in the uncertainty in quantile estimators based on POME compared with the MOM and ML methods (%).
Station NameReturn Period(Year)POME to MOMPOME to ML
QuantileStandard ErrorConfidence Interval WidthQuantileStandard ErrorConfidence Interval Width
Changwu100.091.741.740.343.783.78
200.34−5.63−5.630.68−1.02−1.02
500.67−15.66−15.661.10−7.68−7.68
1000.91−21.83−21.831.39−11.88−11.88
2001.14−26.82−26.821.67−15.33−15.33
5001.43−32.02−32.022.02−18.99−18.99
Lintong101.37−3.74−3.74−1.49−4.93−4.93
201.74−6.67−6.67−1.67−5.29−5.29
502.27−10.23−10.23−1.92−5.70−5.70
1003.17−14.76−14.76−2.35−6.18−6.18
2003.48−15.92−15.92−2.50−6.29−6.29
5003.82−17.05−17.05−2.66−6.41−6.41
Meixian101.41−3.58−3.58−1.50−4.95−4.95
201.79−6.51−6.51−1.68−5.32−5.32
502.34−10.08−10.08−1.93−5.73−5.73
1003.27−14.62−14.62−2.37−6.21−6.21
2003.58−15.78−15.78−2.51−6.33−6.33
5003.93−16.92−16.92−2.68−6.44−6.44
Tongguan101.47−3.14−3.14−1.35−3.92−3.92
201.86−5.98−5.98−1.51−4.20−4.20
502.41−9.45−9.45−1.73−4.53−4.53
1003.35−13.88−13.88−2.11−4.92−4.92
2003.66−15.00−15.00−2.23−5.02−5.02
5004.01−16.12−16.12−2.37−5.11−5.11

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Back to TopTop