An Estimation of the Entropy for a Double Exponential Distribution Based on Multiply Type-II Censored Samples

Abstract

In many life-testing and reliability studies, the experimenter might not always obtain complete information on failure times for all experimental units. Multiply Type-II censored sampling arises in a life-testing experiment whenever the experimenter does not observe the failure times of some units placed on a life-test. In this paper, we obtain estimators for the entropy function of a double exponential distribution under multiply Type-II censored samples using the maximum likelihood estimation and the approximate maximum likelihood estimation procedures. We compare the proposed estimators in the sense of the mean squared errors by using Monte Carlo simulation.

1. Introduction

Let X be a random variable with a continuous distribution function (cdf) F(x) and probability density function (pdf) f(x). The differential entropy H(X) of the random variable is defined by Cover and Thomas [1] to be

$$\begin{array}{c}\hfill H\left(X\right)=H\left(f\right)=-{\int }_{\infty }^{\infty }f\left(x\right)\log f\left(x\right)dx\end{array}$$

(1)

Consider a double exponential distribution with the pdf

$$\begin{array}{c}\hfill f\left(x;\theta ,\sigma \right)=\frac{1}{2\sigma }{e}^{-|x-\theta |/\sigma },-\infty <x<\infty ,\sigma >0\end{array}$$

(2)

and the cdf

$$\begin{array}{c}\hfill F\left(x;\theta ,\sigma \right)=\left\{\begin{array}{cc}\frac{1}{2}\exp [-\frac{\theta -x}{\sigma }],\hfill & x<\theta \hfill \\ 1-\frac{1}{2}\exp [-\frac{x-\theta }{\sigma }],\hfill & x\ge \theta \hfill \end{array}\right.\end{array}$$

(3)

For the pdf (2), entropy simplifies to

$$\begin{array}{c}\hfill H\left(f\right)=1+\log \left(2\sigma \right),\sigma >0\end{array}$$

(4)

The double exponential distribution is used to model symmetric data with long tails. This distribution also arises directly when a random variable occurs as the difference of two variables with exponential distributions with the same scale (see, Johnson, et al. [2]).

In most cases of censored and truncated samples, the maximum likelihood method does not provide explicit estimators. So we need another method for the purpose of providing the explicit estimators.

Govindarajulu [3] gave the coefficients of the best linear unbiased estimators for the location and the scale parameters in the double exponential distribution from complete and symmetric censored samples. Raghunandanan and Srinivasan [4] presented some simplified estimators of the location and the scale parameter of a double exponential distribution. Bain and Engelhardt [5] discussed the usefulness of the double exponential distribution as a model for statistical studies and obtained the confidence intervals based on the maximum likelihood estimators for the location and the scale parameters of a double exponential distribution. Kappenman [6] obtained conditional confidence intervals for the parameters of a double exponential distribution.

For some reason, suppose that we have to terminate the experiment before all items have failed. For example, individuals in a clinical trial may drop out of the study, or the study may have to be terminated for lack of funds. In an industrial experiment, units may break accidentally. There are, however, many situations in which the removal of units prior to failure is pre-planned. One of the main reasons for this is to save time and cost associated with testing. Data obtained from such experiments are called censored data.

Multiply Type-II censored sampling arises in a life-testing experiment whenever the experimenter does not observe the failure times of some units placed on a life-test. Another situation where multiply censored samples arise naturally is when some units failed between two points of observation with the exact times of failure of these units unobserved.

The approximated maximum likelihood estimating method was first developed by Balakrishnan [7] for the purpose of providing explicit estimators of the scale parameter in the Rayleigh distribution. It has been noted that in most cases, the maximum likelihood method does not provide explicit estimators based on censored samples (see [7]). When the sample is multiply censored, the maximum likelihood method does not admit explicit solutions. Therefore, it is desirable to develop which approximations to this maximum likelihood method would provide us with estimators that are explicit functions of order statistics.

Balakrishnan [8] presented a simple approximation to the likelihood equation and derived explicit estimators which are linear functions of order statistics of the location and scale parameters of an exponential distribution based on the multiply Type-II censored sample. Balasubramanian and Balakrishnan [9] derived explicit best linear unbiased estimators for one- and two-parameter exponential distributions when the available sample is multiply Type-II censored. Kang [10] obtained the approximate maximum likelihood estimator (AMLE) for the scale parameter of the double exponential distribution based on Type-II censored samples and showed that the proposed estimator is generally more efficient than the best linear unbiased estimator and the optimum unbiased absolute estimator. Childs and Balakrishnan [11] developed procedures for obtaining confidence intervals for the parameters of a double exponential distribution based on progressively Type-II censored samples. Balakrishnan, et al. [12] discussed point and interval estimation for the extreme value distribution under progressively Type-II censoring. Kang and Lee [13] proposed some estimators of the location and the scale parameters of the two-parameter exponential distribution based on multiply Type-II censored samples. They also obtained the moments for the proposed estimators.

In this paper, we derive the estimators for the entropy function of the double exponential distribution with unknown parameters under multiply Type-II censoring. We also compare the proposed estimators in the sense of the mean squared error (MSE) for various censored samples.

2. Estimation of the Entropy

2.1. Maximum Likelihood Estimation

Suppose n items are placed in a life-testing experiment and that the a₁th, a₂th, …, a_sth failure-times are only made available, where

$$\begin{array}{c}\hfill 1\le a_1<a_2<\cdots <a_s\le n\end{array}$$

Let

$$\begin{array}{c}\hfill X_{a_1:n}\le X_{a_2:n}\le \cdots \le X_{a_s:n}\end{array}$$

(5)

be the available multiply Type-II censored sample from the double exponential distribution with pdf (2).

Let a₀ = 0, $a_{s+1}=n+1$, $F\left(x_{a_0:n}\right)=0$, and $F\left(x_{a_{s+1}:n}\right)=1$.

Then the likelihood function based on the multiply Type-II censored sample (5) is given by

$$\begin{array}{ccc}\hfill L& =& \frac{n!}{{\prod }_{j=1}^{s+1}(a_j-a_{j-1}-1)!}\left[\prod _{j=1}^{s+1}{[F\left(z_{a_j:n}\right)-F\left(z_{a_{j-1}:n}\right)]}^{a_j-a_{j-1}-1}\right]\frac{1}{{\sigma }^s}\prod _{j=1}^{s}f\left(z_{a_j:n}\right)\hfill \\ & =& \frac{1}{{\sigma }^s}\frac{n!}{{\prod }_{j=1}^{s+1}(a_j-a_{j-1}-1)!}{\left[F\left(z_{a_1:n}\right)\right]}^{a_1-1}{[1-F\left(z_{a_s:n}\right)]}^{n-a_s}\hfill \\ & & \times \left[\prod _{j=1}^{s}f\left(z_{a_j:n}\right)\right]\left[\prod _{j=2}^s{[F\left(z_{a_j:n}\right)-F\left(z_{a_{j-1}:n}\right)]}^{a_j-a_{j-1}-1}\right]\hfill \end{array}$$

(6)

where $Z_{i:n}=\left(X_{i:n}-\theta \right)/\sigma$, and f(z) and F(z) are the pdf and the cdf of the standard double exponential distribution, respectively.

We propose the estimator of the parameter θ based on multiply Type-II censored samples. We consider the estimator of the parameter θ as follows

$$\begin{array}{c}\hfill \widehat{\theta }=\left\{\begin{array}{cc}{X}_{a_{(s+1)/2}:n},\hfill & s\mathrm{is}\mathrm{odd}\hfill \\ (X_{a_{s/2}:n}+X_{a_{(s/2)+1}:n})/2,\hfill & s\mathrm{is}\mathrm{even}\hfill \end{array}\right.\end{array}$$

By realizing that $\frac{f^{\prime }\left(z\right)}{f\left(z\right)}=-\frac{|z|}{z}$, z ≠ 0, we can find the MLE for σ by solving the following log-likelihood equation:

$$\begin{array}{ccc}\hfill \frac{\partial \ln \mathit{L}}{\partial \sigma }& =& -\frac{1}{\sigma }[s+(a_1-1)\frac{f\left(z_{a_1:n}\right)}{F\left(z_{a_1:n}\right)}{z}_{a_1:n}-(n-a_s)\frac{f\left(z_{a_s:n}\right)}{1-F\left(z_{a_s:n}\right)}{z}_{a_s:n}\hfill \\ & & -\sum _{j=1}^s|z_{a_j:n}|+\sum _{j=2}^s(a_j-a_{j-1}-1)\frac{f\left(z_{a_j:n}\right)z_{a_j:n}-f\left(z_{a_{j-1}:n}\right)z_{a_{j-1}:n}}{F\left(z_{a_j:n}\right)-F\left(z_{a_{j-1}:n}\right)}]\hfill \\ & =& 0\hfill \end{array}$$

(7)

Equation (7) can be solved numerically using the Newton-Raphson method and an estimate of the entropy function (4) is

$$\begin{array}{c}\hfill \tilde{H}\left(f\right)=1+\log \left(2\tilde{\sigma }\right)\end{array}$$

(8)

2.2. Approximate Maximum Likelihood Estimator

Since the log-likelihood equation does not admit explicit solutions, it will be desirable to consider an approximation to the likelihood equation which provides us with explicit estimator for the scale parameter.

Equation (7) does not admit an explicit solution for σ. But we can expand the functions

$$\frac{f\left(z_{a_1:n}\right)}{F\left(z_{a_1:n}\right)},\frac{f\left(z_{a_s:n}\right)}{1-F\left(z_{a_s:n}\right)},\frac{f\left(z_{a_j:n}\right)}{F\left(z_{a_j:n}\right)-F\left(z_{a_{j-1}:n}\right)},\mathrm{and}\frac{f\left(z_{a_{j-1}:n}\right)}{F\left(z_{a_j:n}\right)-F\left(z_{a_{j-1}:n}\right)}$$

as taking first two term in an expansion of Taylor series around the points ${\xi }_{a_1}$, ${\xi }_{a_s}$, and (${\xi }_{a_{j-1}}$, ${\xi }_{a_j}$), respectively (for example, $f\left(z_{a_j:n}\right)\simeq f\left({\xi }_{a_j:n}\right)+f^{\prime }\left({\xi }_{a_j:n}\right)(z_{a_j:n}-{\xi }_{a_j}))$, where

$$\begin{array}{ccc}\hfill {\xi }_{a_1}=F^{-1}\left(p_{a_1}\right)& =& \left\{\begin{array}{cc}\ln \left(2p_{a_1}\right),\hfill & p_{a_1}\le 0.5\hfill \\ -\ln \left\{2(1-p_{a_1})\right\},\hfill & p_{a_1}>0.5\hfill \end{array}\right.\hfill \end{array}$$

$$\begin{array}{ccc}\hfill {\xi }_{a_s}=F^{-1}\left(p_{a_s}\right)& =& \left\{\begin{array}{cc}\ln \left(2p_{a_s}\right),\hfill & p_{a_s}\le 0.5\hfill \\ -\ln \left\{2(1-p_{a_s})\right\},\hfill & p_{a_s}>0.5\hfill \end{array}\right.\hfill \end{array}$$

$$\begin{array}{ccc}\hfill {\xi }_{a_{j-1}}=F^{-1}\left(p_{a_{j-1}}\right)& =& \left\{\begin{array}{cc}\ln \left(2p_{a_{j-1}}\right),\hfill & p_{a_j}\le 0.5or{p}_{a_{j-1}}\le 0.5\le p_{a_j}\hfill \\ -\ln \left\{2(1-p_{a_{j-1}})\right\},\hfill & p_{a_{j-1}}>0.5\hfill \end{array}\right.\hfill \end{array}$$

$$\begin{array}{ccc}\hfill {\xi }_{a_j}=F^{-1}\left(p_{a_j}\right)& =& \left\{\begin{array}{cc}\ln \left(2p_{a_j}\right),\hfill & p_{a_j}\le 0.5\hfill \\ -\ln \left\{2(1-p_{a_j})\right\},\hfill & p_{a_{j-1}}\le 0.5\le p_{a_j}or{p}_{a_{j-1}}>0.5\hfill \end{array}\right.\hfill \end{array}$$

and

$$\begin{array}{c}\hfill p_{a_j}=\frac{a_j}{n+1}\end{array}$$

For Equation (7), we need to consider the three cases as $z_{a_1:n}\ge 0$, $z_{a_1:n}<0<z_{a_s:n}$, $z_{a_s:n}\le 0$.

Case 1: $z_{a_1:n}\ge 0$.

Since $F\left(z_{a_s:n}\right)=1-f\left(z_{a_s:n}\right)$, the expansion of the functions

$$\frac{f\left(z_{a_1:n}\right)}{F\left(z_{a_1:n}\right)},\frac{f\left(z_{a_j:n}\right)}{F\left(z_{a_j:n}\right)-F\left(z_{a_{j-1}:n}\right)},\mathrm{and}\frac{f\left(z_{a_{j-1}:n}\right)}{F\left(z_{a_j:n}\right)-F\left(z_{a_{j-1}:n}\right)}$$

are required. We approximate these functions by

$$\begin{array}{ccc}\hfill \frac{f\left(z_{a_1:n}\right)}{F\left(z_{a_1:n}\right)}& \simeq & {\alpha }_1+{\beta }_1{z}_{a_1:n}\hfill \end{array}$$

(9)

$$\begin{array}{ccc}\hfill \frac{f\left(z_{a_j:n}\right)}{F\left(z_{a_j:n}\right)-F\left(z_{a_{j-1}:n}\right)}& \simeq & {\alpha }_{1j}+{\beta }_{1j}{z}_{a_j:n}+{\gamma }_{1j}{z}_{a_{j-1}:n}\hfill \end{array}$$

(10)

and

$$\begin{array}{c}\hfill \frac{f\left(z_{a_{j-1}:n}\right)}{F\left(z_{a_j:n}\right)-F\left(z_{a_{j-1}:n}\right)}\simeq {\alpha }_{2j}+{\beta }_{2j}{z}_{a_j:n}+{\gamma }_{2j}{z}_{a_{j-1}:n}\end{array}$$

(11)

where

$$\begin{array}{ccc}\hfill {\alpha }_1& =& \left\{\begin{array}{cc}1,\hfill & p_{a_1}\le 0.5\hfill \\ \frac{f\left({\xi }_{a_1}\right)}{p_{a_1}}+\frac{f\left({\xi }_{a_1}\right)}{{\left(p_{a_1}\right)}^2}{\xi }_{a_1},\hfill & p_{a_1}>0.5\hfill \end{array}\right.\hfill \\ \hfill {\beta }_1& =& \left\{\begin{array}{cc}0,\hfill & p_{a_1}\le 0.5\hfill \\ -\frac{f\left({\xi }_{a_1}\right)}{{\left(p_{a_1}\right)}^2},\hfill & p_{a_1}>0.5\hfill \end{array}\right.\hfill \\ \hfill {\alpha }_{1j}& =& \left\{\begin{array}{cc}\frac{f\left({\xi }_{a_j}\right)}{p_{a_j}-p_{a_{j-1}}}\left(1-{\xi }_{a_j}+K_j\right),\hfill & p_{a_j}\le 0.5\hfill \\ \frac{f\left({\xi }_{a_j}\right)}{p_{a_j}-p_{a_{j-1}}}\left(1+{\xi }_{a_j}+K_j\right),\hfill & p_{a_{j-1}}\le 0.5<p_{a_j}or{p}_{a_{j-1}}>0.5\hfill \end{array}\right.\hfill \\ \hfill {\beta }_{1j}& =& \left\{\begin{array}{cc}\frac{f\left({\xi }_{a_j}\right)}{p_{a_j}-p_{a_{j-1}}}\left(1-\frac{f\left({\xi }_{a_j}\right)}{p_{a_j}-p_{a_{j-1}}}\right),\hfill & p_{a_j}\le 0.5\hfill \\ -\frac{f\left({\xi }_{a_j}\right)}{p_{a_j}-p_{a_{j-1}}}\left(1+\frac{f\left({\xi }_{a_j}\right)}{p_{a_j}-p_{a_{j-1}}}\right),\hfill & p_{a_{j-1}}\le 0.5<p_{a_j}or{p}_{a_{j-1}}>0.5\hfill \end{array}\right.\hfill \\ \hfill {\gamma }_{1j}& =& \frac{f\left({\xi }_{a_j}\right)f\left({\xi }_{a_{j-1}}\right)}{{(p_{a_j}-p_{a_{j-1}})}^2},\hfill \\ \hfill {\alpha }_{2j}& =& \left\{\begin{array}{cc}\frac{f\left({\xi }_{a_{j-1}}\right)}{p_{a_j}-p_{a_{j-1}}}\left(1-{\xi }_{a_{j-1}}+K_j\right),\hfill & p_{a_j}\le 0.5or{p}_{a_{j-1}}\le 0.5<p_{a_j}\hfill \\ \frac{f\left({\xi }_{a_{j-1}}\right)}{p_{a_j}-p_{a_{j-1}}}\left(1+{\xi }_{a_{j-1}}+K_j\right),\hfill & p_{a_{j-1}}>0.5\hfill \end{array}\right.\hfill \\ \hfill {\beta }_{2j}& =& -\frac{f\left({\xi }_{a_j}\right)f\left({\xi }_{a_{j-1}}\right)}{{(p_{a_j}-p_{a_{j-1}})}^2}=-{\gamma }_{1j},\hfill \\ \hfill {\gamma }_{2j}& =& \left\{\begin{array}{cc}\frac{f\left({\xi }_{a_{j-1}}\right)}{p_{a_j}-p_{a_{j-1}}}\left(1+\frac{f\left({\xi }_{a_{j-1}}\right)}{p_{a_j}-p_{a_{j-1}}}\right),\hfill & p_{a_j}\le 0.5or{p}_{a_{j-1}}\le 0.5<p_{a_j}\hfill \\ -\frac{f\left({\xi }_{a_{j-1}}\right)}{p_{a_j}-p_{a_{j-1}}}\left(1-\frac{f\left({\xi }_{a_{j-1}}\right)}{p_{a_j}-p_{a_{j-1}}}\right),\hfill & p_{a_{j-1}}>0.5\hfill \end{array}\right.\hfill \\ \hfill K_j& =& \frac{f\left({\xi }_{a_j}\right){\xi }_{a_j}-f\left({\xi }_{a_{j-1}}\right){\xi }_{a_{j-1}}}{p_{a_j}-p_{a_{j-1}}}\hfill \end{array}$$

By substituting the Equations (9)–(11) into the Equation (7), we may approximate the log-likelihood Equation (7) by

$$\begin{array}{ccc}\hfill \frac{\partial \ln {\mathit{L}}_{\mathit{Case}\mathit{1}}}{\partial \sigma }& \simeq & -\frac{1}{\sigma }[s+(a_1-1)({\alpha }_1+{\beta }_1{z}_{a_1:n})z_{a_1:n}-(n-a_s)z_{a_s:n}-\sum _{j=1}^s|z_{a_j:n}|\hfill \\ & & +\sum _{j=2}^s(a_j-a_{j-1}-1)\left(\right({\alpha }_{1j}+{\beta }_{1j}{z}_{a_j:n}+{\gamma }_{1j}{z}_{a_{j-1}:n})z_{a_j:n}\hfill \\ & & -({\alpha }_{2j}+{\beta }_{2j}{z}_{a_j:n}+{\gamma }_{2j}{z}_{a_{j-1}:n})z_{a_{j-1}:n}\left)\right]\hfill \\ & =& 0\hfill \end{array}$$

(12)

From solving Equation (12), we derive an AMLE of σ as

$$\begin{array}{c}\hfill \widehat{\sigma }=\frac{-B+\sqrt{B^2-4sC}}{2s}\end{array}$$

(13)

where

$$\begin{array}{ccc}\hfill B& =& (a_1-1){\alpha }_1{X}_{a_1:n}-(n-a_s)X_{a_s:n}-\sum _{j=1}^s\left|X_{a_j:n}-\widehat{\theta }\right|\hfill \\ & & +\sum _{j=2}^s(a_j-a_{j-1}-1)({\alpha }_{1j}{X}_{a_j:n}-{\alpha }_{2j}{X}_{a_{j-1}:n})\hfill \\ & & -\left[(a_1-1){\alpha }_1-(n-a_s)+\sum _{j=2}^s(a_j-a_{j-1}-1)({\alpha }_{1j}-{\alpha }_{2j})\right]\widehat{\theta }\hfill \\ \hfill C& =& \sum _{j=2}^s(a_j-a_{j-1}-1)\{{\beta }_{1j}{(X_{a_j:n}-\widehat{\theta })}^2+2{\gamma }_{1j}(X_{a_j:n}-\widehat{\theta })(X_{a_{j-1}:n}-\widehat{\theta })\hfill \\ & & -{\gamma }_{2j}{(X_{a_{j-1}:n}-\widehat{\theta })}^2\}+(a_1-1){\beta }_1{(X_{a_1:n}-\widehat{\theta })}^2\hfill \end{array}$$

Upon solving the Equation (12) for σ we get a quadratic equation in σ which has two roots; however, one of them drops out since ${\beta }_1\le 0$ and ${\beta }_{1j}{(X_{a_j:n}-\widehat{\theta })}^2+2{\gamma }_{1j}(X_{a_j:n}-\widehat{\theta })(X_{a_{j-1}:n}-\widehat{\theta })-{\gamma }_{2j}{(X_{a_{j-1}:n}-\widehat{\theta })}^2\le 0$ for example $p_{a_j}\le 0.5$ and hence $C\le 0$.

Case 2: $z_{a_1:n}<0<z_{a_s:n}$.

Since $F\left(z_{a_1:n}\right)=f\left(z_{a_1:n}\right)$ and $F\left(z_{a_s:n}\right)=1-f\left(z_{a_s:n}\right)$ we need the expansion of the following two functions

$$\begin{array}{c}\hfill \frac{f\left(z_{a_j:n}\right)}{F\left(z_{a_j:n}\right)-F\left(z_{a_{j-1}:n}\right)}\mathrm{and}\frac{f\left(z_{a_{j-1}:n}\right)}{F\left(z_{a_j:n}\right)-F\left(z_{a_{j-1}:n}\right)}\end{array}$$

By substituting the Equations (10) and (11) into Equation (7), we may approximate the log-likelihood Equation (7) by

$$\begin{array}{ccc}\hfill \frac{\partial \ln {\mathit{L}}_{\mathit{Case}\mathit{2}}}{\partial \sigma }& \simeq & -\frac{1}{\sigma }[s+(a_1-1)z_{a_1:n}-(n-a_s)z_{a_s:n}-\sum _{j=1}^s|z_{a_j:n}|\hfill \\ & & +\sum _{j=2}^s(a_j-a_{j-1}-1)\left(\right({\alpha }_{1j}+{\beta }_{1j}{z}_{a_j:n}+{\gamma }_{1j}{z}_{a_{j-1}:n})z_{a_j:n}\hfill \\ & & -({\alpha }_{2j}+{\beta }_{2j}{z}_{a_j:n}+{\gamma }_{2j}{z}_{a_{j-1}:n})z_{a_{j-1}:n}\left)\right]\hfill \\ & =& 0\hfill \end{array}$$

(14)

From solving Equation (14), we obtain an AMLE of σ:

$$\begin{array}{c}\hfill \widehat{\sigma }=\frac{-D+\sqrt{D^2-4sE}}{2s}\end{array}$$

(15)

where

$$\begin{array}{ccc}\hfill D& =& (a_1-1)X_{a_1:n}-(n-a_s)X_{a_s:n}-\sum _{j=1}^s\left|X_{a_j:n}-\widehat{\theta }\right|\hfill \\ & & +\sum _{j=2}^s(a_j-a_{j-1}-1)({\alpha }_{1j}{X}_{a_j:n}-{\alpha }_{2j}{X}_{a_{j-1}:n})\hfill \\ & & -\left[(a_1-1)-(n-a_s)+\sum _{j=2}^s(a_j-a_{j-1}-1)({\alpha }_{1j}-{\alpha }_{2j})\right]\widehat{\theta },\hfill \\ \hfill E& =& \sum _{j=2}^s(a_j-a_{j-1}-1)\{{\beta }_{1j}{(X_{a_j:n}-\widehat{\theta })}^2+2{\gamma }_{1j}(X_{a_j:n}-\widehat{\theta })(X_{a_{j-1}:n}-\widehat{\theta })\hfill \\ & & -{\gamma }_{2j}{(X_{a_{j-1}:n}-\widehat{\theta })}^2\}\hfill \end{array}$$

Upon solving Equation (14) for σ we get a quadratic equation in σ which has two roots; however, one of them drops out since $E\le 0$.

Case 3: $z_{a_s:n}\le 0$.

Since $F\left(z_{a_1:n}\right)=f\left(z_{a_1:n}\right)$, the expansion of the functions

$$\frac{f\left(z_{a_s:n}\right)}{1-F\left(z_{a_s:n}\right)},\frac{f\left(z_{a_j:n}\right)}{F\left(z_{a_j:n}\right)-F\left(z_{a_{j-1}:n}\right)},\mathrm{and}\frac{f\left(z_{a_{j-1}:n}\right)}{F\left(z_{a_j:n}\right)-F\left(z_{a_{j-1}:n}\right)}$$

are required. We approximate these functions by Equations (10), (11), and

$$\begin{array}{c}\hfill \frac{f\left(z_{a_s:n}\right)}{1-F\left(z_{a_s:n}\right)}\simeq {\gamma }_1+{\delta }_1{z}_{a_s:n}\end{array}$$

(16)

where

$$\begin{array}{ccc}\hfill {\gamma }_1& =& \left\{\begin{array}{cc}1,\hfill & p_{a_s}>0.5\hfill \\ \frac{f\left({\xi }_{a_s}\right)}{1-p_{a_s}}\left[1-{\xi }_{a_s}-\frac{f\left({\xi }_{a_s}\right)}{1-p_{a_s}}{\xi }_{a_s}\right],\hfill & p_{a_s}\le 0.5\hfill \end{array}\right.\hfill \\ \hfill {\delta }_1& =& \left\{\begin{array}{cc}0,\hfill & p_{a_s}>0.5\hfill \\ \frac{f\left({\xi }_{a_s}\right)}{{(1-p_{a_s})}^2},\hfill & p_{a_s}\le 0.5\hfill \end{array}\right.\hfill \end{array}$$

By substituting Equations (10), (11), and (16) into Equation (7), we may approximate the log-likelihood Equation (7) by

$$\begin{array}{ccc}\hfill \frac{\partial \ln {\mathit{L}}_{\mathit{Case}\mathit{3}}}{\partial \sigma }& \simeq & -\frac{1}{\sigma }[s+(a_1-1)z_{a_1:n}-(n-a_s)({\gamma }_1+{\delta }_1{z}_{a_s:n})z_{a_s:n}-\sum _{j=1}^s|z_{a_j:n}|\hfill \\ & & +\sum _{j=2}^s(a_j-a_{j-1}-1)\left(\right({\alpha }_{1j}+{\beta }_{1j}{z}_{a_j:n}+{\gamma }_{1j}{z}_{a_{j-1}:n})z_{a_j:n}\hfill \\ & & -({\alpha }_{2j}+{\beta }_{2j}{z}_{a_j:n}+{\gamma }_{2j}{z}_{a_{j-1}:n})z_{a_{j-1}:n}\left)\right]\hfill \\ & =& 0\hfill \end{array}$$

(17)

From solving Equation (17), we derive an AMLE of σ as

$$\begin{array}{c}\hfill \widehat{\sigma }=\frac{-F+\sqrt{F^2-4sG}}{2s}\end{array}$$

(18)

where

$$\begin{array}{ccc}\hfill F& =& (a_1-1)X_{a_1:n}-(n-a_s){\gamma }_1{X}_{a_s:n}-\sum _{j=1}^s\left|X_{a_j:n}-\widehat{\theta }\right|\hfill \\ & & +\sum _{j=2}^s(a_j-a_{j-1}-1)({\alpha }_{1j}{X}_{a_j:n}-{\alpha }_{2j}{X}_{a_{j-1}:n})\hfill \\ & & -\left[(a_1-1)-(n-a_s){\gamma }_1+\sum _{j=2}^s(a_j-a_{j-1}-1)({\alpha }_{1j}-{\alpha }_{2j})\right]\widehat{\theta }\hfill \\ \hfill G& =& \sum _{j=2}^s(a_j-a_{j-1}-1)\{{\beta }_{1j}{(X_{a_j:n}-\widehat{\theta })}^2+2{\gamma }_{1j}(X_{a_j:n}-\widehat{\theta })(X_{a_{j-1}:n}-\widehat{\theta })\hfill \\ & & -{\gamma }_{2j}{(X_{a_{j-1}:n}-\widehat{\theta })}^2\}-(a_1-1){\delta }_1{(X_{a_1:n}-\widehat{\theta })}^2\hfill \end{array}$$

Upon solving the Equation (17) for σ we get a quadratic equation in σ which has two roots; however, one of them drops out since ${\delta }_1\ge 0$ and hence G ≤ 0.

We derive an estimator of the entropy function (4) as

$$\begin{array}{c}\hfill \widehat{H}\left(f\right)=1+\log \left(2\widehat{\sigma }\right)\end{array}$$

(19)

2.3. Nonparametric Entropy Estimates

The procedures of nonparametric estimation have no meaningful associated parameters. As nonparametric methods make fewer assumptions, their applicability is much wider than the corresponding parametric methods. In particular, they may be applied in situations where less is known about the application in question. Also, due to the reliance on fewer assumptions, nonparametric methods are more robust.

Another advantage for the use of nonparametric methods is simplicity. In certain cases, even when the use of parametric methods is justified, nonparametric methods may be easier to use.

The best known and widely used entropy estimator was proposed by Vasicek [14]. Vasicek’s estimator of entropy has the following form;

$$\begin{array}{c}\hfill H_m=\frac{1}{n}\sum _{i=1}^n\ln \left\{\frac{n}{2m}(X_{i+m}-X_{i-m})\right\}\end{array}$$

(20)

where m is a positive integer smaller than n/2 and $X_{i-m}=X_1$ for $i-m<1$ and $X_{i+m}=X_n$ for $i+m>n$.

van Es [15] suggested a new estimator, which has the following form

$$\begin{array}{c}\hfill V_m=\frac{1}{n-m}\sum _{i=1}^{n-m}\ln \left\{\frac{n+1}{m}(X_{i+m}-X_i)\right\}+\sum _{k=m}^n\frac{1}{k}+\ln \left(m\right)-\ln (n+1)\end{array}$$

(21)

These estimators of entropy cannot be used for the censored samples. So we propose the modified entropy estimators based on multiply Type-II censored samples.

First, we propose the modified Vasicek’s entropy estimator as follows

$$\begin{array}{c}\hfill H_m=\frac{1}{n-s}\sum _{j=1}^s\ln \left\{\frac{n}{2m}(X_{a_{j+m}:n}-X_{a_{j-m}:n})\right\}\end{array}$$

(22)

where m is a positive integer smaller than s/2 and $X_{a_{j-m}:n}=X_{a_1:n}$ for $a_{j-m}<a_1$ and $X_{a_{j+m}:n}=X_{a_s:n}$ for $a_{j+m}>a_s$.

Secondly, we propose the modified van Es’s entropy estimator as follows

$$\begin{array}{c}\hfill V_m=\frac{1}{n-m-s}\sum _{j=1}^{n-m-s}\ln \left\{\frac{n+1}{m}(X_{a_{j+m}:n}-X_{a_j:n})\right\}+\sum _{k=m}^n\frac{1}{k}+\ln \left(m\right)-\ln (n+1)\end{array}$$

(23)

3. Results and Discussion

In order to evaluate the performance of the proposed estimators, the MSEs of all proposed estimators were simulated by a Monte Carlo method for sample sizes n = 10, 20, 30, 50, the window sizes m = 2, 4, 6 and various choices of censoring (k = n − s was the number of unobserved or missing data).

All computations were programmed in Microsoft Visual C₊₊ 6.0 and random numbers for simulations were generated by IMSL subroutines.

The convergence of the Newton-Raphson method depended on the choice of the initial values. For this reason, the proposed AMLE was used as starting values for the iterations, and the MLE was obtained by solving the nonlinear Equation (7).

The simulation procedure was repeated 10,000 times. These values are given in Table 1 and Table 2, from which we can see that the estimators $\tilde{H}\left(f\right)$ and $\widehat{H}\left(f\right)$ are more efficient than H_m and V_m in the sense of the MSE.

Table 1. The relative MSEs and biases for the proposed estimators ($\tilde{H}\left(f\right)$, $\widehat{H}\left(f\right)$, H₂, V₂).

			$\tilde{\mathit{H}}\mathbf{\left(}\mathit{f}\mathbf{\right)}$	$\widehat{\mathit{H}}\mathbf{\left(}\mathit{f}\mathbf{\right)}$	H2	V2
n	k	aj	MSE(bias)	MSE(bias)	MSE(bias)	MSE(bias)
10	0	1∼10	0.122(−0.111)	0.122(−0.111)	0.400(−0.518)	0.200(−0.262)
	2	1∼8	0.095(−0.079)	0.157(−0.108)	0.726(−0.749)	0.315(−0.375)
		2∼9	0.158(−0.252)	0.162(−0.142)	0.904(−0.863)	0.387(−0.467)
20	0	1∼20	0.054(−0.053)	0.054(−0.053)	0.157(−0.313)	0.097(−0.187)
	2	1∼18	0.043(−0.041)	0.060(−0.051)	0.288(−0.472)	0.159(−0.301)
		3∼20	0.117(−0.237)	0.060(−0.052)	0.290(−0.474)	0.161(−0.302)
		2∼19	0.068(−0.152)	0.061(−0.060)	0.339(−0.523)	0.183(−0.337)
	6	4∼17	0.105(−0.245)	0.081(−0.078)	0.731(−0.804)	0.385(−0.542)
		1 2 6∼9 12∼15 17∼20	0.058(0.071)	0.063(0.073)	0.076(0.108)	0.118(0.226)
30	0	1∼30	0.036(−0.036)	0.036(−0.036)	0.103(−0.252)	0.065(−0.150)
	2	1∼28	0.030(−0.030)	0.039(−0.036)	0.185(−0.379)	0.107(−0.248)
		3∼30	0.072(−0.182)	0.039(−0.037)	0.187(−0.381)	0.108(−0.249)
		2∼29	0.045(−0.114)	0.039(−0.039)	0.211(−0.412)	0.119(−0.270)
	6	4∼27	0.069(−0.197)	0.046(−0.046)	0.439(−0.625)	0.248(−0.441)
		1 2 6∼9 12∼15 17∼30	0.036(−0.009)	0.044(0.066)	0.042(−0.026)	0.048(0.044)
	17	16∼28	0.058(−0.081)	0.102(−0.111)	0.477(−0.616)	0.211(−0.321)
50	0	1∼50	0.021(−0.021)	0.021(−0.021)	0.064(−0.202)	0.037(−0.109)
	2	1∼48	0.019(0.001)	0.022(−0.012)	0.074(−0.224)	0.040(−0.119)
		2∼49	0.025(−0.077)	0.022(−0.022)	0.123(−0.314)	0.065(−0.197)
		3∼50	0.038(−0.125)	0.022(−0.021)	0.112(−0.296)	0.061(−0.185)
	6	4∼47	0.038(−0.144)	0.024(−0.024)	0.246(−0.469)	0.136(−0.327)
		1 2 6∼9 12∼15 17∼50	0.029(−0.087)	0.023(0.027)	0.042(−0.136)	0.031(−0.068)
	27	26∼48	0.033(−0.047)	0.049(−0.059)	0.251(−0.448)	0.113(−0.239)
	28	4∼15 31∼40	0.025(0.019)	0.087(0.237)	0.062(−0.130)	0.053(0.023)

Table 1. The relative MSEs and biases for the proposed estimators ($\tilde{H}\left(f\right)$, $\widehat{H}\left(f\right)$, H₂, V₂).

			$\tilde{\mathit{H}}\mathbf{\left(}\mathit{f}\mathbf{\right)}$	$\widehat{\mathit{H}}\mathbf{\left(}\mathit{f}\mathbf{\right)}$	H2	V2
n	k	aj	MSE(bias)	MSE(bias)	MSE(bias)	MSE(bias)
10	0	1∼10	0.122(−0.111)	0.122(−0.111)	0.400(−0.518)	0.200(−0.262)
	2	1∼8	0.095(−0.079)	0.157(−0.108)	0.726(−0.749)	0.315(−0.375)
		2∼9	0.158(−0.252)	0.162(−0.142)	0.904(−0.863)	0.387(−0.467)
20	0	1∼20	0.054(−0.053)	0.054(−0.053)	0.157(−0.313)	0.097(−0.187)
	2	1∼18	0.043(−0.041)	0.060(−0.051)	0.288(−0.472)	0.159(−0.301)
		3∼20	0.117(−0.237)	0.060(−0.052)	0.290(−0.474)	0.161(−0.302)
		2∼19	0.068(−0.152)	0.061(−0.060)	0.339(−0.523)	0.183(−0.337)
	6	4∼17	0.105(−0.245)	0.081(−0.078)	0.731(−0.804)	0.385(−0.542)
		1 2 6∼9 12∼15 17∼20	0.058(0.071)	0.063(0.073)	0.076(0.108)	0.118(0.226)
30	0	1∼30	0.036(−0.036)	0.036(−0.036)	0.103(−0.252)	0.065(−0.150)
	2	1∼28	0.030(−0.030)	0.039(−0.036)	0.185(−0.379)	0.107(−0.248)
		3∼30	0.072(−0.182)	0.039(−0.037)	0.187(−0.381)	0.108(−0.249)
		2∼29	0.045(−0.114)	0.039(−0.039)	0.211(−0.412)	0.119(−0.270)
	6	4∼27	0.069(−0.197)	0.046(−0.046)	0.439(−0.625)	0.248(−0.441)
		1 2 6∼9 12∼15 17∼30	0.036(−0.009)	0.044(0.066)	0.042(−0.026)	0.048(0.044)
	17	16∼28	0.058(−0.081)	0.102(−0.111)	0.477(−0.616)	0.211(−0.321)
50	0	1∼50	0.021(−0.021)	0.021(−0.021)	0.064(−0.202)	0.037(−0.109)
	2	1∼48	0.019(0.001)	0.022(−0.012)	0.074(−0.224)	0.040(−0.119)
		2∼49	0.025(−0.077)	0.022(−0.022)	0.123(−0.314)	0.065(−0.197)
		3∼50	0.038(−0.125)	0.022(−0.021)	0.112(−0.296)	0.061(−0.185)
	6	4∼47	0.038(−0.144)	0.024(−0.024)	0.246(−0.469)	0.136(−0.327)
		1 2 6∼9 12∼15 17∼50	0.029(−0.087)	0.023(0.027)	0.042(−0.136)	0.031(−0.068)
	27	26∼48	0.033(−0.047)	0.049(−0.059)	0.251(−0.448)	0.113(−0.239)
	28	4∼15 31∼40	0.025(0.019)	0.087(0.237)	0.062(−0.130)	0.053(0.023)

When m is a positive integer smaller than n/2, the estimator H_m is satisfactory. For this reason, the MSEs of H₆ are empty when n = 10 and k = 0, 2 (s = 10, 8) in Table 2.

The MSEs of the estimators H_m and V_m generally increase as window size m increases.

As expected, the MSEs of all estimators decrease as sample size n increases. For fixed sample size, the MSE increases generally as the number of unobserved or missing data k = n − s increases.

Table 2. The relative MSEs and biases for the proposed estimators (H₄, V₄, H₆, V₆).

			H4	V4	H6	V6
n	k	aj	MSE(bias)	MSE(bias)	MSE(bias)	MSE(bias)
10	0	1∼10	0.486(−0.600)	0.238(−0.343)	−	0.217(−0.322)
	2	1∼8	0.861(−0.838)	0.331(−0.410)	−	0.265(−0.324)
		2∼9	1.097(−0.977)	0.419(−0.512)	−	0.353(−0.456)
20	0	1∼20	0.143(−0.289)	0.136(−0.281)	0.162(−0.311)	0.168(−0.334)
	2	1∼18	0.283(−0.467)	0.208(−0.379)	0.327(−0.510)	0.242(−0.422)
		3∼20	0.285(−0.469)	0.209(−0.380)	0.330(−0.513)	0.243(−0.423)
		2∼19	0.340(−0.525)	0.236(−0.413)	0.399(−0.578)	0.273(−0.457)
	6	4∼17	0.782(−0.838)	0.438(−0.593)	0.924(−0.921)	0.467(−0.617)
		1 2 6∼9 12∼15 17∼20	0.076(0.080)	0.079(0.141)	0.073(−0.000)	0.066(0.093)
30	0	1∼30	0.082(−0.207)	0.093(−0.234)	0.081(−0.199)	0.123(−0.290)
	2	1∼28	0.159(−0.344)	0.144(−0.320)	0.166(−0.351)	0.178(−0.370)
		3∼30	0.161(−0.347)	0.145(−0.321)	0.168(−0.354)	0.179(−0.371)
		2∼29	0.186(−0.382)	0.157(−0.340)	0.197(−0.394)	0.193(−0.389)
	6	4∼27	0.418(−0.609)	0.294(−0.494)	0.460(−0.643)	0.333(−0.532)
		1 2 6∼9 12∼15 17∼30	0.042(0.004)	0.044(−0.049)	0.046(0.011)	0.055(−0.113)
	17	16∼28	0.488(−0.628)	0.219(−0.336)	0.561(−0.683)	0.221(−0.333)
50	0	1∼50	0.043(−0.145)	0.053(−0.175)	0.039(−0.127)	0.073(−0.224)
	2	1∼48	0.052(−0.172)	0.056(−0.180)	0.049(−0.160)	0.074(−0.226)
		2∼49	0.094(−0.266)	0.088(−0.253)	0.090(−0.259)	0.112(−0.297)
		3∼50	0.083(−0.245)	0.083(−0.242)	0.078(−0.235)	0.106(−0.287)
	6	4∼47	0.207(−0.427)	0.164(−0.372)	0.208(−0.429)	0.192(−0.408)
		1 2 6∼9 12∼15 17∼50	0.030(−0.082)	0.044(−0.143)	0.026(−0.055)	0.064(−0.201)
	27	26∼48	0.229(−0.426)	0.120(−0.259)	0.247(−0.446)	0.127(−0.270)
	28	4∼15 31∼40	0.043(−0.062)	0.051(0.094)	0.042(−0.083)	0.061(0.159)

Table 2. The relative MSEs and biases for the proposed estimators (H₄, V₄, H₆, V₆).

			H4	V4	H6	V6
n	k	aj	MSE(bias)	MSE(bias)	MSE(bias)	MSE(bias)
10	0	1∼10	0.486(−0.600)	0.238(−0.343)	−	0.217(−0.322)
	2	1∼8	0.861(−0.838)	0.331(−0.410)	−	0.265(−0.324)
		2∼9	1.097(−0.977)	0.419(−0.512)	−	0.353(−0.456)
20	0	1∼20	0.143(−0.289)	0.136(−0.281)	0.162(−0.311)	0.168(−0.334)
	2	1∼18	0.283(−0.467)	0.208(−0.379)	0.327(−0.510)	0.242(−0.422)
		3∼20	0.285(−0.469)	0.209(−0.380)	0.330(−0.513)	0.243(−0.423)
		2∼19	0.340(−0.525)	0.236(−0.413)	0.399(−0.578)	0.273(−0.457)
	6	4∼17	0.782(−0.838)	0.438(−0.593)	0.924(−0.921)	0.467(−0.617)
		1 2 6∼9 12∼15 17∼20	0.076(0.080)	0.079(0.141)	0.073(−0.000)	0.066(0.093)
30	0	1∼30	0.082(−0.207)	0.093(−0.234)	0.081(−0.199)	0.123(−0.290)
	2	1∼28	0.159(−0.344)	0.144(−0.320)	0.166(−0.351)	0.178(−0.370)
		3∼30	0.161(−0.347)	0.145(−0.321)	0.168(−0.354)	0.179(−0.371)
		2∼29	0.186(−0.382)	0.157(−0.340)	0.197(−0.394)	0.193(−0.389)
	6	4∼27	0.418(−0.609)	0.294(−0.494)	0.460(−0.643)	0.333(−0.532)
		1 2 6∼9 12∼15 17∼30	0.042(0.004)	0.044(−0.049)	0.046(0.011)	0.055(−0.113)
	17	16∼28	0.488(−0.628)	0.219(−0.336)	0.561(−0.683)	0.221(−0.333)
50	0	1∼50	0.043(−0.145)	0.053(−0.175)	0.039(−0.127)	0.073(−0.224)
	2	1∼48	0.052(−0.172)	0.056(−0.180)	0.049(−0.160)	0.074(−0.226)
		2∼49	0.094(−0.266)	0.088(−0.253)	0.090(−0.259)	0.112(−0.297)
		3∼50	0.083(−0.245)	0.083(−0.242)	0.078(−0.235)	0.106(−0.287)
	6	4∼47	0.207(−0.427)	0.164(−0.372)	0.208(−0.429)	0.192(−0.408)
		1 2 6∼9 12∼15 17∼50	0.030(−0.082)	0.044(−0.143)	0.026(−0.055)	0.064(−0.201)
	27	26∼48	0.229(−0.426)	0.120(−0.259)	0.247(−0.446)	0.127(−0.270)
	28	4∼15 31∼40	0.043(−0.062)	0.051(0.094)	0.042(−0.083)	0.061(0.159)

In order to illustrate the methods of inference developed in this paper, we will present one example in this section.

Let us consider the 33 years of flood data from two stations on Fox River in Wisconsin (see [5]). The following ordered differences, z_i = y_i − x_i, were obtained, where y_i denotes the flood stage downstream at Wrightstown and x_i denotes the flood stage upstream at Berlin:

1.96 1.97 3.60 3.80 4.79 5.66 5.76 5.78 6.27 6.30 6.76 7.65 7.84 7.99 8.51 9.18 10.13 10.24

10.25 10.43 11.45 11.48 11.75 11.81 12.34 12.78 13.06 13.29 13.98 14.18 14.40 16.22 17.06.

This data had been utilized earlier by Kappenman [6]. The data are assumed to represent a random sample of observations of a double exponential random variable.

For complete data (n = 33, s = 33, k = 0, a_j = 1 ∼ 33), we can obtain the MLE $\tilde{\sigma }=3.361$, and the AMLE $\widehat{\sigma }=3.361$. For this example of n = 33, s = 23, k = 10 (a_j = 1 ∼ 2, 6 ∼ 9, 13 ∼ 15, 20 ∼ 33), and the multiply Type-II censored samples are

1.96 1.97 - - - 5.66 5.76 5.78 6.27 - - - 7.84 7.99 8.51 - - - - 10.43 11.45 11.48 11.75

11.81 12.34 12.78 13.06 13.29 13.98 14.18 14.40 16.22 17.06.

We can obtain the MLE $\tilde{\sigma }=3.377$, and the AMLE $\widehat{\sigma }=4.328$.

We also compute the estimators for the entropy function for complete data and the multiply Type-II censored sample. These values are presented in Table 3.

Table 3. Estimates of the entropy for complete data and the multiply Type-II censored sample in example.

	$\tilde{\mathit{H}}\mathbf{\left(}\mathit{f}\mathbf{\right)}$	$\widehat{\mathit{H}}\mathbf{\left(}\mathit{f}\mathbf{\right)}$	H2	V2	H4	V6	H6	V6
Complete data :	2.905	2.905	2.633	2.783	2.609	2.742	2.568	2.686
Multiply Type-II censored sample :	2.910	3.158	2.965	3.064	2.906	3.044	2.871	3.013

Table 3. Estimates of the entropy for complete data and the multiply Type-II censored sample in example.

	$\tilde{\mathit{H}}\mathbf{\left(}\mathit{f}\mathbf{\right)}$	$\widehat{\mathit{H}}\mathbf{\left(}\mathit{f}\mathbf{\right)}$	H2	V2	H4	V6	H6	V6
Complete data :	2.905	2.905	2.633	2.783	2.609	2.742	2.568	2.686
Multiply Type-II censored sample :	2.910	3.158	2.965	3.064	2.906	3.044	2.871	3.013

Application and estimation of the entropy for a double exponential distribution were studied in Johnson, et al. [2], Balakrishnan and Nevzorov [16]. In this study, we derived the estimators for the entropy function in the double exponential under multiply Type-II censoring. The scale parameter σ is estimated by the maximum likelihood estimation method and the approximate maximum likelihood estimation method.

4. Conclusions

In most cases of censored and truncated samples, the maximum likelihood method does not provide explicit estimators. So we discuss another method for the purpose of providing the explicit estimators.

We obtain estimators for the entropy function of the double exponential distribution under multiply Type-II censored samples using the maximum likelihood estimation, the approximate maximum likelihood estimation, and the nonparametric estimation procedures. Based on the results and discussions, the parametric procedures perform better than the nonparametric ones. But the nonparametric procedures are simplicity under multiply Type-II censored samples. The MSEs of the estimators H_m and V_m generally increase as window size m increases.

In future studies, we will consider estimation for the entropy function based on progressively Type-II censored samples.