Next Article in Journal
Disentangling Complexity from Randomness and Chaos
Previous Article in Journal
Control Parameters for Boundary-Layer Instabilities in Unsteady Shock Interactions
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An Estimation of the Entropy for a Double Exponential Distribution Based on Multiply Type-II Censored Samples

1
Department of Statistics, Yeungnam University, Dae-dong, Gyongsan 712-749, Korea
2
Department of Statistics, Pusan National University, Geumjeong-gu, Busan 609-735, Korea
3
Informatization Team, Ministry of Patriots and Veterans Affairs, Yeouido-dong, Seoul 150-874, Korea
4
Senior Research Fellow, The Korea Institute for Health and Social Affairs, Bulgwangdong, Eunpyeonggu, Seoul 122-705, Korea
*
Author to whom correspondence should be addressed.
Entropy 2012, 14(2), 161-173; https://doi.org/10.3390/e14020161
Submission received: 20 November 2011 / Revised: 14 January 2012 / Accepted: 18 January 2012 / Published: 31 January 2012

Abstract

:
In many life-testing and reliability studies, the experimenter might not always obtain complete information on failure times for all experimental units. Multiply Type-II censored sampling arises in a life-testing experiment whenever the experimenter does not observe the failure times of some units placed on a life-test. In this paper, we obtain estimators for the entropy function of a double exponential distribution under multiply Type-II censored samples using the maximum likelihood estimation and the approximate maximum likelihood estimation procedures. We compare the proposed estimators in the sense of the mean squared errors by using Monte Carlo simulation.

1. Introduction

Let X be a random variable with a continuous distribution function (cdf) F(x) and probability density function (pdf) f(x). The differential entropy H(X) of the random variable is defined by Cover and Thomas [1] to be
H ( X ) = H ( f ) = f ( x ) log f ( x ) d x
Consider a double exponential distribution with the pdf
f x ; θ , σ = 1 2 σ e | x θ | / σ , < x < , σ > 0
and the cdf
F x ; θ , σ = 1 2 exp [ θ x σ ] , x < θ 1 1 2 exp [ x θ σ ] , x θ
For the pdf (2), entropy simplifies to
H ( f ) = 1 + log ( 2 σ ) , σ > 0
The double exponential distribution is used to model symmetric data with long tails. This distribution also arises directly when a random variable occurs as the difference of two variables with exponential distributions with the same scale (see, Johnson, et al. [2]).
In most cases of censored and truncated samples, the maximum likelihood method does not provide explicit estimators. So we need another method for the purpose of providing the explicit estimators.
Govindarajulu [3] gave the coefficients of the best linear unbiased estimators for the location and the scale parameters in the double exponential distribution from complete and symmetric censored samples. Raghunandanan and Srinivasan [4] presented some simplified estimators of the location and the scale parameter of a double exponential distribution. Bain and Engelhardt [5] discussed the usefulness of the double exponential distribution as a model for statistical studies and obtained the confidence intervals based on the maximum likelihood estimators for the location and the scale parameters of a double exponential distribution. Kappenman [6] obtained conditional confidence intervals for the parameters of a double exponential distribution.
For some reason, suppose that we have to terminate the experiment before all items have failed. For example, individuals in a clinical trial may drop out of the study, or the study may have to be terminated for lack of funds. In an industrial experiment, units may break accidentally. There are, however, many situations in which the removal of units prior to failure is pre-planned. One of the main reasons for this is to save time and cost associated with testing. Data obtained from such experiments are called censored data.
Multiply Type-II censored sampling arises in a life-testing experiment whenever the experimenter does not observe the failure times of some units placed on a life-test. Another situation where multiply censored samples arise naturally is when some units failed between two points of observation with the exact times of failure of these units unobserved.
The approximated maximum likelihood estimating method was first developed by Balakrishnan [7] for the purpose of providing explicit estimators of the scale parameter in the Rayleigh distribution. It has been noted that in most cases, the maximum likelihood method does not provide explicit estimators based on censored samples (see [7]). When the sample is multiply censored, the maximum likelihood method does not admit explicit solutions. Therefore, it is desirable to develop which approximations to this maximum likelihood method would provide us with estimators that are explicit functions of order statistics.
Balakrishnan [8] presented a simple approximation to the likelihood equation and derived explicit estimators which are linear functions of order statistics of the location and scale parameters of an exponential distribution based on the multiply Type-II censored sample. Balasubramanian and Balakrishnan [9] derived explicit best linear unbiased estimators for one- and two-parameter exponential distributions when the available sample is multiply Type-II censored. Kang [10] obtained the approximate maximum likelihood estimator (AMLE) for the scale parameter of the double exponential distribution based on Type-II censored samples and showed that the proposed estimator is generally more efficient than the best linear unbiased estimator and the optimum unbiased absolute estimator. Childs and Balakrishnan [11] developed procedures for obtaining confidence intervals for the parameters of a double exponential distribution based on progressively Type-II censored samples. Balakrishnan, et al. [12] discussed point and interval estimation for the extreme value distribution under progressively Type-II censoring. Kang and Lee [13] proposed some estimators of the location and the scale parameters of the two-parameter exponential distribution based on multiply Type-II censored samples. They also obtained the moments for the proposed estimators.
In this paper, we derive the estimators for the entropy function of the double exponential distribution with unknown parameters under multiply Type-II censoring. We also compare the proposed estimators in the sense of the mean squared error (MSE) for various censored samples.

2. Estimation of the Entropy

2.1. Maximum Likelihood Estimation

Suppose n items are placed in a life-testing experiment and that the a1th, a2th, …, asth failure-times are only made available, where
1 a 1 < a 2 < < a s n
Let
X a 1 : n X a 2 : n X a s : n
be the available multiply Type-II censored sample from the double exponential distribution with pdf (2).
Let a0 = 0, a s + 1 = n + 1 , F ( x a 0 : n ) = 0 , and F ( x a s + 1 : n ) = 1 .
Then the likelihood function based on the multiply Type-II censored sample (5) is given by
L = n ! j = 1 s + 1 ( a j a j 1 1 ) ! j = 1 s + 1 [ F ( z a j : n ) F ( z a j 1 : n ) ] a j a j 1 1 1 σ s j = 1 s f ( z a j : n ) = 1 σ s n ! j = 1 s + 1 ( a j a j 1 1 ) ! [ F ( z a 1 : n ) ] a 1 1 [ 1 F ( z a s : n ) ] n a s × j = 1 s f ( z a j : n ) j = 2 s [ F ( z a j : n ) F ( z a j 1 : n ) ] a j a j 1 1
where Z i : n = ( X i : n θ ) / σ , and f(z) and F(z) are the pdf and the cdf of the standard double exponential distribution, respectively.
We propose the estimator of the parameter θ based on multiply Type-II censored samples. We consider the estimator of the parameter θ as follows
θ ^ = X a ( s + 1 ) / 2 : n , s is odd ( X a s / 2 : n + X a ( s / 2 ) + 1 : n ) / 2 , s is even
By realizing that f ( z ) f ( z ) = | z | z , z ≠ 0, we can find the MLE for σ by solving the following log-likelihood equation:
ln L σ = 1 σ [ s + ( a 1 1 ) f ( z a 1 : n ) F ( z a 1 : n ) z a 1 : n ( n a s ) f ( z a s : n ) 1 F ( z a s : n ) z a s : n j = 1 s | z a j : n | + j = 2 s ( a j a j 1 1 ) f ( z a j : n ) z a j : n f ( z a j 1 : n ) z a j 1 : n F ( z a j : n ) F ( z a j 1 : n ) ] = 0
Equation (7) can be solved numerically using the Newton-Raphson method and an estimate of the entropy function (4) is
H ˜ ( f ) = 1 + log ( 2 σ ˜ )

2.2. Approximate Maximum Likelihood Estimator

Since the log-likelihood equation does not admit explicit solutions, it will be desirable to consider an approximation to the likelihood equation which provides us with explicit estimator for the scale parameter.
Equation (7) does not admit an explicit solution for σ. But we can expand the functions
f ( z a 1 : n ) F ( z a 1 : n ) , f ( z a s : n ) 1 F ( z a s : n ) , f ( z a j : n ) F ( z a j : n ) F ( z a j 1 : n ) , and f ( z a j 1 : n ) F ( z a j : n ) F ( z a j 1 : n )
as taking first two term in an expansion of Taylor series around the points ξ a 1 , ξ a s , and ( ξ a j 1 , ξ a j ), respectively (for example, f ( z a j : n ) f ( ξ a j : n ) + f ( ξ a j : n ) ( z a j : n ξ a j ) ) , where
ξ a 1 = F 1 ( p a 1 ) = ln ( 2 p a 1 ) , p a 1 0.5 ln 2 ( 1 p a 1 ) , p a 1 > 0.5
ξ a s = F 1 ( p a s ) = ln ( 2 p a s ) , p a s 0.5 ln 2 ( 1 p a s ) , p a s > 0.5
ξ a j 1 = F 1 ( p a j 1 ) = ln ( 2 p a j 1 ) , p a j 0.5 o r p a j 1 0.5 p a j ln 2 ( 1 p a j 1 ) , p a j 1 > 0.5
ξ a j = F 1 ( p a j ) = ln ( 2 p a j ) , p a j 0.5 ln 2 ( 1 p a j ) , p a j 1 0.5 p a j o r p a j 1 > 0.5
and
p a j = a j n + 1
For Equation (7), we need to consider the three cases as z a 1 : n 0 , z a 1 : n < 0 < z a s : n , z a s : n 0 .

Case 1: z a 1 : n 0 .

Since F ( z a s : n ) = 1 f ( z a s : n ) , the expansion of the functions
f ( z a 1 : n ) F ( z a 1 : n ) , f ( z a j : n ) F ( z a j : n ) F ( z a j 1 : n ) , and f ( z a j 1 : n ) F ( z a j : n ) F ( z a j 1 : n )
are required. We approximate these functions by
f ( z a 1 : n ) F ( z a 1 : n ) α 1 + β 1 z a 1 : n
f ( z a j : n ) F ( z a j : n ) F ( z a j 1 : n ) α 1 j + β 1 j z a j : n + γ 1 j z a j 1 : n
and
f ( z a j 1 : n ) F ( z a j : n ) F ( z a j 1 : n ) α 2 j + β 2 j z a j : n + γ 2 j z a j 1 : n
where
α 1 = 1 , p a 1 0.5 f ( ξ a 1 ) p a 1 + f ( ξ a 1 ) ( p a 1 ) 2 ξ a 1 , p a 1 > 0.5 β 1 = 0 , p a 1 0.5 f ( ξ a 1 ) ( p a 1 ) 2 , p a 1 > 0.5 α 1 j = f ( ξ a j ) p a j p a j 1 1 ξ a j + K j , p a j 0.5 f ( ξ a j ) p a j p a j 1 1 + ξ a j + K j , p a j 1 0.5 < p a j o r p a j 1 > 0.5 β 1 j = f ( ξ a j ) p a j p a j 1 1 f ( ξ a j ) p a j p a j 1 , p a j 0.5 f ( ξ a j ) p a j p a j 1 1 + f ( ξ a j ) p a j p a j 1 , p a j 1 0.5 < p a j o r p a j 1 > 0.5 γ 1 j = f ( ξ a j ) f ( ξ a j 1 ) ( p a j p a j 1 ) 2 , α 2 j = f ( ξ a j 1 ) p a j p a j 1 1 ξ a j 1 + K j , p a j 0.5 o r p a j 1 0.5 < p a j f ( ξ a j 1 ) p a j p a j 1 1 + ξ a j 1 + K j , p a j 1 > 0.5 β 2 j = f ( ξ a j ) f ( ξ a j 1 ) ( p a j p a j 1 ) 2 = γ 1 j , γ 2 j = f ( ξ a j 1 ) p a j p a j 1 1 + f ( ξ a j 1 ) p a j p a j 1 , p a j 0.5 o r p a j 1 0.5 < p a j f ( ξ a j 1 ) p a j p a j 1 1 f ( ξ a j 1 ) p a j p a j 1 , p a j 1 > 0.5 K j = f ( ξ a j ) ξ a j f ( ξ a j 1 ) ξ a j 1 p a j p a j 1
By substituting the Equations (9)–(11) into the Equation (7), we may approximate the log-likelihood Equation (7) by
ln L Case 1 σ 1 σ [ s + ( a 1 1 ) ( α 1 + β 1 z a 1 : n ) z a 1 : n ( n a s ) z a s : n j = 1 s | z a j : n | + j = 2 s ( a j a j 1 1 ) ( ( α 1 j + β 1 j z a j : n + γ 1 j z a j 1 : n ) z a j : n ( α 2 j + β 2 j z a j : n + γ 2 j z a j 1 : n ) z a j 1 : n ) ] = 0
From solving Equation (12), we derive an AMLE of σ as
σ ^ = B + B 2 4 s C 2 s
where
B = ( a 1 1 ) α 1 X a 1 : n ( n a s ) X a s : n j = 1 s X a j : n θ ^ + j = 2 s ( a j a j 1 1 ) ( α 1 j X a j : n α 2 j X a j 1 : n ) ( a 1 1 ) α 1 ( n a s ) + j = 2 s ( a j a j 1 1 ) ( α 1 j α 2 j ) θ ^ C = j = 2 s ( a j a j 1 1 ) { β 1 j ( X a j : n θ ^ ) 2 + 2 γ 1 j ( X a j : n θ ^ ) ( X a j 1 : n θ ^ ) γ 2 j ( X a j 1 : n θ ^ ) 2 } + ( a 1 1 ) β 1 ( X a 1 : n θ ^ ) 2
Upon solving the Equation (12) for σ we get a quadratic equation in σ which has two roots; however, one of them drops out since β 1 0 and β 1 j ( X a j : n θ ^ ) 2 + 2 γ 1 j ( X a j : n θ ^ ) ( X a j 1 : n θ ^ ) γ 2 j ( X a j 1 : n θ ^ ) 2 0 for example p a j 0.5 and hence C 0 .

Case 2: z a 1 : n < 0 < z a s : n .

Since F ( z a 1 : n ) = f ( z a 1 : n ) and F ( z a s : n ) = 1 f ( z a s : n ) we need the expansion of the following two functions
f ( z a j : n ) F ( z a j : n ) F ( z a j 1 : n ) and f ( z a j 1 : n ) F ( z a j : n ) F ( z a j 1 : n )
By substituting the Equations (10) and (11) into Equation (7), we may approximate the log-likelihood Equation (7) by
ln L Case 2 σ 1 σ [ s + ( a 1 1 ) z a 1 : n ( n a s ) z a s : n j = 1 s | z a j : n | + j = 2 s ( a j a j 1 1 ) ( ( α 1 j + β 1 j z a j : n + γ 1 j z a j 1 : n ) z a j : n ( α 2 j + β 2 j z a j : n + γ 2 j z a j 1 : n ) z a j 1 : n ) ] = 0
From solving Equation (14), we obtain an AMLE of σ:
σ ^ = D + D 2 4 s E 2 s
where
D = ( a 1 1 ) X a 1 : n ( n a s ) X a s : n j = 1 s X a j : n θ ^ + j = 2 s ( a j a j 1 1 ) ( α 1 j X a j : n α 2 j X a j 1 : n ) ( a 1 1 ) ( n a s ) + j = 2 s ( a j a j 1 1 ) ( α 1 j α 2 j ) θ ^ , E = j = 2 s ( a j a j 1 1 ) { β 1 j ( X a j : n θ ^ ) 2 + 2 γ 1 j ( X a j : n θ ^ ) ( X a j 1 : n θ ^ ) γ 2 j ( X a j 1 : n θ ^ ) 2 }
Upon solving Equation (14) for σ we get a quadratic equation in σ which has two roots; however, one of them drops out since E 0 .

Case 3: z a s : n 0 .

Since F ( z a 1 : n ) = f ( z a 1 : n ) , the expansion of the functions
f ( z a s : n ) 1 F ( z a s : n ) , f ( z a j : n ) F ( z a j : n ) F ( z a j 1 : n ) , and f ( z a j 1 : n ) F ( z a j : n ) F ( z a j 1 : n )
are required. We approximate these functions by Equations (10), (11), and
f ( z a s : n ) 1 F ( z a s : n ) γ 1 + δ 1 z a s : n
where
γ 1 = 1 , p a s > 0.5 f ( ξ a s ) 1 p a s 1 ξ a s f ( ξ a s ) 1 p a s ξ a s , p a s 0.5 δ 1 = 0 , p a s > 0.5 f ( ξ a s ) ( 1 p a s ) 2 , p a s 0.5
By substituting Equations (10), (11), and (16) into Equation (7), we may approximate the log-likelihood Equation (7) by
ln L Case 3 σ 1 σ [ s + ( a 1 1 ) z a 1 : n ( n a s ) ( γ 1 + δ 1 z a s : n ) z a s : n j = 1 s | z a j : n | + j = 2 s ( a j a j 1 1 ) ( ( α 1 j + β 1 j z a j : n + γ 1 j z a j 1 : n ) z a j : n ( α 2 j + β 2 j z a j : n + γ 2 j z a j 1 : n ) z a j 1 : n ) ] = 0
From solving Equation (17), we derive an AMLE of σ as
σ ^ = F + F 2 4 s G 2 s
where
F = ( a 1 1 ) X a 1 : n ( n a s ) γ 1 X a s : n j = 1 s X a j : n θ ^ + j = 2 s ( a j a j 1 1 ) ( α 1 j X a j : n α 2 j X a j 1 : n ) ( a 1 1 ) ( n a s ) γ 1 + j = 2 s ( a j a j 1 1 ) ( α 1 j α 2 j ) θ ^ G = j = 2 s ( a j a j 1 1 ) { β 1 j ( X a j : n θ ^ ) 2 + 2 γ 1 j ( X a j : n θ ^ ) ( X a j 1 : n θ ^ ) γ 2 j ( X a j 1 : n θ ^ ) 2 } ( a 1 1 ) δ 1 ( X a 1 : n θ ^ ) 2
Upon solving the Equation (17) for σ we get a quadratic equation in σ which has two roots; however, one of them drops out since δ 1 0 and hence G ≤ 0.
We derive an estimator of the entropy function (4) as
H ^ ( f ) = 1 + log ( 2 σ ^ )

2.3. Nonparametric Entropy Estimates

The procedures of nonparametric estimation have no meaningful associated parameters. As nonparametric methods make fewer assumptions, their applicability is much wider than the corresponding parametric methods. In particular, they may be applied in situations where less is known about the application in question. Also, due to the reliance on fewer assumptions, nonparametric methods are more robust.
Another advantage for the use of nonparametric methods is simplicity. In certain cases, even when the use of parametric methods is justified, nonparametric methods may be easier to use.
The best known and widely used entropy estimator was proposed by Vasicek [14]. Vasicek’s estimator of entropy has the following form;
H m = 1 n i = 1 n ln n 2 m ( X i + m X i m )
where m is a positive integer smaller than n/2 and X i m = X 1 for i m < 1 and X i + m = X n for i + m > n .
van Es [15] suggested a new estimator, which has the following form
V m = 1 n m i = 1 n m ln n + 1 m ( X i + m X i ) + k = m n 1 k + ln ( m ) ln ( n + 1 )
These estimators of entropy cannot be used for the censored samples. So we propose the modified entropy estimators based on multiply Type-II censored samples.
First, we propose the modified Vasicek’s entropy estimator as follows
H m = 1 n s j = 1 s ln n 2 m ( X a j + m : n X a j m : n )
where m is a positive integer smaller than s/2 and X a j m : n = X a 1 : n for a j m < a 1 and X a j + m : n = X a s : n for a j + m > a s .
Secondly, we propose the modified van Es’s entropy estimator as follows
V m = 1 n m s j = 1 n m s ln n + 1 m ( X a j + m : n X a j : n ) + k = m n 1 k + ln ( m ) ln ( n + 1 )

3. Results and Discussion

In order to evaluate the performance of the proposed estimators, the MSEs of all proposed estimators were simulated by a Monte Carlo method for sample sizes n = 10, 20, 30, 50, the window sizes m = 2, 4, 6 and various choices of censoring (k = ns was the number of unobserved or missing data).
All computations were programmed in Microsoft Visual C++ 6.0 and random numbers for simulations were generated by IMSL subroutines.
The convergence of the Newton-Raphson method depended on the choice of the initial values. For this reason, the proposed AMLE was used as starting values for the iterations, and the MLE was obtained by solving the nonlinear Equation (7).
The simulation procedure was repeated 10,000 times. These values are given in Table 1 and Table 2, from which we can see that the estimators H ˜ ( f ) and H ^ ( f ) are more efficient than Hm and Vm in the sense of the MSE.
Table 1. The relative MSEs and biases for the proposed estimators ( H ˜ ( f ) , H ^ ( f ) , H2, V2).
Table 1. The relative MSEs and biases for the proposed estimators ( H ˜ ( f ) , H ^ ( f ) , H2, V2).
H ˜ ( f ) H ^ ( f ) H2V2
nkajMSE(bias)MSE(bias)MSE(bias)MSE(bias)
1001∼100.122(−0.111)0.122(−0.111)0.400(−0.518)0.200(−0.262)
21∼80.095(−0.079)0.157(−0.108)0.726(−0.749)0.315(−0.375)
2∼90.158(−0.252)0.162(−0.142)0.904(−0.863)0.387(−0.467)
2001∼200.054(−0.053)0.054(−0.053)0.157(−0.313)0.097(−0.187)
21∼180.043(−0.041)0.060(−0.051)0.288(−0.472)0.159(−0.301)
3∼200.117(−0.237)0.060(−0.052)0.290(−0.474)0.161(−0.302)
2∼190.068(−0.152)0.061(−0.060)0.339(−0.523)0.183(−0.337)
64∼170.105(−0.245)0.081(−0.078)0.731(−0.804)0.385(−0.542)
1 2 6∼9 12∼15 17∼200.058(0.071)0.063(0.073)0.076(0.108)0.118(0.226)
3001∼300.036(−0.036)0.036(−0.036)0.103(−0.252)0.065(−0.150)
21∼280.030(−0.030)0.039(−0.036)0.185(−0.379)0.107(−0.248)
3∼300.072(−0.182)0.039(−0.037)0.187(−0.381)0.108(−0.249)
2∼290.045(−0.114)0.039(−0.039)0.211(−0.412)0.119(−0.270)
64∼270.069(−0.197)0.046(−0.046)0.439(−0.625)0.248(−0.441)
1 2 6∼9 12∼15 17∼300.036(−0.009)0.044(0.066)0.042(−0.026)0.048(0.044)
1716∼280.058(−0.081)0.102(−0.111)0.477(−0.616)0.211(−0.321)
5001∼500.021(−0.021)0.021(−0.021)0.064(−0.202)0.037(−0.109)
21∼480.019(0.001)0.022(−0.012)0.074(−0.224)0.040(−0.119)
2∼490.025(−0.077)0.022(−0.022)0.123(−0.314)0.065(−0.197)
3∼500.038(−0.125)0.022(−0.021)0.112(−0.296)0.061(−0.185)
64∼470.038(−0.144)0.024(−0.024)0.246(−0.469)0.136(−0.327)
1 2 6∼9 12∼15 17∼500.029(−0.087)0.023(0.027)0.042(−0.136)0.031(−0.068)
2726∼480.033(−0.047)0.049(−0.059)0.251(−0.448)0.113(−0.239)
284∼15 31∼400.025(0.019)0.087(0.237)0.062(−0.130)0.053(0.023)
When m is a positive integer smaller than n/2, the estimator Hm is satisfactory. For this reason, the MSEs of H6 are empty when n = 10 and k = 0, 2 (s = 10, 8) in Table 2.
The MSEs of the estimators Hm and Vm generally increase as window size m increases.
As expected, the MSEs of all estimators decrease as sample size n increases. For fixed sample size, the MSE increases generally as the number of unobserved or missing data k = ns increases.
Table 2. The relative MSEs and biases for the proposed estimators (H4, V4, H6, V6).
Table 2. The relative MSEs and biases for the proposed estimators (H4, V4, H6, V6).
H4V4H6V6
nkajMSE(bias)MSE(bias)MSE(bias)MSE(bias)
1001∼100.486(−0.600)0.238(−0.343)0.217(−0.322)
21∼80.861(−0.838)0.331(−0.410)0.265(−0.324)
2∼91.097(−0.977)0.419(−0.512)0.353(−0.456)
2001∼200.143(−0.289)0.136(−0.281)0.162(−0.311)0.168(−0.334)
21∼180.283(−0.467)0.208(−0.379)0.327(−0.510)0.242(−0.422)
3∼200.285(−0.469)0.209(−0.380)0.330(−0.513)0.243(−0.423)
2∼190.340(−0.525)0.236(−0.413)0.399(−0.578)0.273(−0.457)
64∼170.782(−0.838)0.438(−0.593)0.924(−0.921)0.467(−0.617)
1 2 6∼9 12∼15 17∼200.076(0.080)0.079(0.141)0.073(−0.000)0.066(0.093)
3001∼300.082(−0.207)0.093(−0.234)0.081(−0.199)0.123(−0.290)
21∼280.159(−0.344)0.144(−0.320)0.166(−0.351)0.178(−0.370)
3∼300.161(−0.347)0.145(−0.321)0.168(−0.354)0.179(−0.371)
2∼290.186(−0.382)0.157(−0.340)0.197(−0.394)0.193(−0.389)
64∼270.418(−0.609)0.294(−0.494)0.460(−0.643)0.333(−0.532)
1 2 6∼9 12∼15 17∼300.042(0.004)0.044(−0.049)0.046(0.011)0.055(−0.113)
1716∼280.488(−0.628)0.219(−0.336)0.561(−0.683)0.221(−0.333)
5001∼500.043(−0.145)0.053(−0.175)0.039(−0.127)0.073(−0.224)
21∼480.052(−0.172)0.056(−0.180)0.049(−0.160)0.074(−0.226)
2∼490.094(−0.266)0.088(−0.253)0.090(−0.259)0.112(−0.297)
3∼500.083(−0.245)0.083(−0.242)0.078(−0.235)0.106(−0.287)
64∼470.207(−0.427)0.164(−0.372)0.208(−0.429)0.192(−0.408)
1 2 6∼9 12∼15 17∼500.030(−0.082)0.044(−0.143)0.026(−0.055)0.064(−0.201)
2726∼480.229(−0.426)0.120(−0.259)0.247(−0.446)0.127(−0.270)
284∼15 31∼400.043(−0.062)0.051(0.094)0.042(−0.083)0.061(0.159)
In order to illustrate the methods of inference developed in this paper, we will present one example in this section.
Let us consider the 33 years of flood data from two stations on Fox River in Wisconsin (see [5]). The following ordered differences, zi = yixi, were obtained, where yi denotes the flood stage downstream at Wrightstown and xi denotes the flood stage upstream at Berlin:
1.96 1.97 3.60 3.80 4.79 5.66 5.76 5.78 6.27 6.30 6.76 7.65 7.84 7.99 8.51 9.18 10.13 10.24
10.25 10.43 11.45 11.48 11.75 11.81 12.34 12.78 13.06 13.29 13.98 14.18 14.40 16.22 17.06.
This data had been utilized earlier by Kappenman [6]. The data are assumed to represent a random sample of observations of a double exponential random variable.
For complete data (n = 33, s = 33, k = 0, aj = 1 ∼ 33), we can obtain the MLE σ ˜ = 3.361 , and the AMLE σ ^ = 3.361 . For this example of n = 33, s = 23, k = 10 (aj = 1 ∼ 2, 6 ∼ 9, 13 ∼ 15, 20 ∼ 33), and the multiply Type-II censored samples are
1.96 1.97 - - - 5.66 5.76 5.78 6.27 - - - 7.84 7.99 8.51 - - - - 10.43 11.45 11.48 11.75
11.81 12.34 12.78 13.06 13.29 13.98 14.18 14.40 16.22 17.06.
We can obtain the MLE σ ˜ = 3.377 , and the AMLE σ ^ = 4.328 .
We also compute the estimators for the entropy function for complete data and the multiply Type-II censored sample. These values are presented in Table 3.
Table 3. Estimates of the entropy for complete data and the multiply Type-II censored sample in example.
Table 3. Estimates of the entropy for complete data and the multiply Type-II censored sample in example.
H ˜ ( f ) H ^ ( f ) H2V2H4V6H6V6
Complete data :2.9052.9052.6332.7832.6092.7422.5682.686
Multiply Type-II censored sample :2.9103.1582.9653.0642.9063.0442.8713.013
Application and estimation of the entropy for a double exponential distribution were studied in Johnson, et al. [2], Balakrishnan and Nevzorov [16]. In this study, we derived the estimators for the entropy function in the double exponential under multiply Type-II censoring. The scale parameter σ is estimated by the maximum likelihood estimation method and the approximate maximum likelihood estimation method.

4. Conclusions

In most cases of censored and truncated samples, the maximum likelihood method does not provide explicit estimators. So we discuss another method for the purpose of providing the explicit estimators.
We obtain estimators for the entropy function of the double exponential distribution under multiply Type-II censored samples using the maximum likelihood estimation, the approximate maximum likelihood estimation, and the nonparametric estimation procedures. Based on the results and discussions, the parametric procedures perform better than the nonparametric ones. But the nonparametric procedures are simplicity under multiply Type-II censored samples. The MSEs of the estimators Hm and Vm generally increase as window size m increases.
In future studies, we will consider estimation for the entropy function based on progressively Type-II censored samples.

Acknowledgements

We would like to express our thanks to three referees for their helpful comments and constructive suggestions. This research was supported by the Yeungnam University research grants in 2008.

References

  1. Cover, T.M.; Thomas, J.A. Elements of Information Theory; Wiley: Hoboken, NJ, USA, 2005. [Google Scholar]
  2. Johnson, N.L.; Kots, S.; Balakrishnan, N. Continuous Univariate Distributions; John Wiley & Sons: New York, NY, USA, 1994. [Google Scholar]
  3. Govindarajulu, Z. Best linear estimates under symmetric censoring of the parameters of a double exponential population. J. Am. Stat. Assoc. 1966, 61, 248–258. [Google Scholar]
  4. Raghunandanan, K.; Srinivasan, R. Simplified estimation of parameters in a double exponential distribution. Technometrics 1971, 13, 689–691. [Google Scholar] [CrossRef]
  5. Bain, L.J.; Engelhardt, M. Interval estimation for the two-parameter double exponential distribution. Technometrics 1973, 15, 875–887. [Google Scholar] [CrossRef]
  6. Kappenman, R.F. Conditional confidence intervals for double exponential distribution parameters. Technometrics 1975, 17, 233–235. [Google Scholar] [CrossRef]
  7. Balakrishnan, N. Approximate MLE of the scale parameter of the Rayleigh distribution with censoring. IEEE Trans. Reliab. 1989, 38, 355–357. [Google Scholar] [CrossRef]
  8. Balakrishnan, N. On the maximum likelihood estimation of the location and scale parameters of exponential distribution based on multiply Type-II censored samples. J. Appl. Stat. 1990, 17, 55–61. [Google Scholar] [CrossRef]
  9. Balasubramanian, K.; Balakrishnan, N. Estimation for one-parameter and two-parameter exponential distributions under multiple Type-II censoring. Stat. Paper. 1992, 33, 203–216. [Google Scholar] [CrossRef]
  10. Kang, S.B. Approximate mle for the scale parameter of the double exponential distribution based on Type-II censored samples. J. Kor. Math. Soc. 1996, 33, 69–79. [Google Scholar]
  11. Childs, A.; Balakrishnan, N. Conditional inference procedures for the Laplace distribution when the observed samples are progressively censored. Metrika 2000, 52, 253–265. [Google Scholar] [CrossRef]
  12. Balakrishnan, N.; Kannan, N.; Lin, C.T.; Wu, S.J.S. Inference for the extreme value distribution under progressive Type-II censoring. J. Stat. Comput. Simulat. 2004, 74, 25–45. [Google Scholar] [CrossRef]
  13. Kang, S.B.; Lee, S.K. AMLEs for the exponential distribution based on multiple Type-II censored samples. The Korean Communications in Statistics 2005, 12, 603–613. [Google Scholar]
  14. Vasicek, O. A test for normality based on sample entropy. J. Roy. Stat. Soc. B Stat. Meth. 1976, 38, 54–59. [Google Scholar]
  15. van Es, B. Estimating functionals related to a density by a class of statistics based on spacings. Scand. J. Stat. 1992, 19, 61–72. [Google Scholar]
  16. Balakrishnan, N.; Nevzorov, V.B. A Primer on Statistical Distribution; John Wiley & Sons: Hoboken, NJ, USA, 2003. [Google Scholar]

Share and Cite

MDPI and ACS Style

Kang, S.-B.; Cho, Y.-S.; Han, J.-T.; Kim, J. An Estimation of the Entropy for a Double Exponential Distribution Based on Multiply Type-II Censored Samples. Entropy 2012, 14, 161-173. https://doi.org/10.3390/e14020161

AMA Style

Kang S-B, Cho Y-S, Han J-T, Kim J. An Estimation of the Entropy for a Double Exponential Distribution Based on Multiply Type-II Censored Samples. Entropy. 2012; 14(2):161-173. https://doi.org/10.3390/e14020161

Chicago/Turabian Style

Kang, Suk-Bok, Young-Seuk Cho, Jun-Tae Han, and Jinsoo Kim. 2012. "An Estimation of the Entropy for a Double Exponential Distribution Based on Multiply Type-II Censored Samples" Entropy 14, no. 2: 161-173. https://doi.org/10.3390/e14020161

Article Metrics

Back to TopTop