The Entropy of Progressively Censored Samples

Abstract

In many life-testing and reliability studies, the experimenter might not always obtain complete information on failure times for all experimental units. Among the different censoring schemes, the progressive censoring scheme has received a considerable attention in the last few years. The aim of this paper is simplifying the entropy of progressively Type II censored samples. We propose an indirect approach using a decomposition of the entropy in progressively Type II censored samples to simplify the calculation. Some recurrence relations for the entropy in progressively Type II censored samples are derived to facilitate this calculation. An efficient computational method is derived that simplifies computation of the entropy in progressively Type II censored samples to a sum; entropy in collections order statistics. We compute the entropy in a collection of progressively Type II censored samples for some known distributions.

1. Introduction

Information theory provides an intuitive tool to measure the uncertainty of random variables and the information shared by them, in which the entropy and the mutual information are two critical concepts.

Let X be a random variable with a cumulative distribution function (cdf) $F\left(x\right)$ and probability density function (pdf) $f\left(x\right)$. The differential entropy $H\left(X\right)$ of the random variable is defined by Cover and Thomas [1] to be

$$H\left(X\right)=-{\int }_{-\infty }^{\infty }f\left(x\right)logf\left(x\right)dx$$

(1)

Let us consider a life-testing experiment where n units is kept under observation until failure. These units could be some system, components, or computer chips in reliability study experiments, or they could be patients put under certain drug or clinical conditions. Suppose the life lengths of these n units are independent identical random variables with a common cdf $F\left(x\right)$ and pdf $f\left(x\right)$. Data collected from such experiments called the order statistics sample

$$X_{1:n}<X_{2:n}<\cdots <X_{n:n}$$

where $X_{r:n}$ is called the rth-order statistics (OS).

For some reason, suppose that we have to terminate the experiment before all items have failed. For example, individuals in a clinical trial may drop out of the study, or the study may have to be terminated for lack of funds. In an industrial experiment, units may break accidentally. There are, however, many situations in which the removal of units prior to failure is pre-planned. One of the main reasons for this is to save time and cost associated with testing. Data obtained from such experiments are called censored data.

The most common censoring schemes are Type I and Type II censoring. In conventional Type I censoring, the experiment continues up to a prespecified time T. Any failures that occur after T are not observed. The termination point T of the experiment is assumed to be independent of the failure times. In conventional Type II censoring, the experimenter decides to terminate the experiment after a prespecified number of items $r\le n$ fail. In this scenario, only the smallest lifetimes are observed. In Type I censoring, the number of failures observed is random and the endpoint of the experiment is fixed, whereas in Type II censoring the endpoint is random, while the number of failures is fixed.

Park [2] studied the entropy of Type II censored sample. Park [3] considered testing exponentiality based on the Kullback-Leibler information with the Type II censored data. The entropy of a single $X_{r:n}$, and a complete order statistic sample has been studied in Wong and Chen [4] and Ebrahimi et al. [5].

Here we considers progressive Type II censored schemes. Among the different censoring schemes, the progressive censoring scheme has received a considerable attention in the last few years, particularly in reliability analysis. It is a more general censoring mechanism than the traditional Type I and Type II censoring [6]. The recent review article by Balakrishnan [7] provide details on progressive censoring schemes and on its different applications. This paper is concerned with simplifying calculation of the entropy in progressively Type II censored data from the i.i.d. random sample of size n. However, the extension to progressively Type II censored data is not so straightforward, because the joint entropy of progressively Type II censored data is an n-dimensional integral. Besides, removals cause additional complications.

Following Balakrishnan and Aggarwala [8], progressively Type II censored samples can be described as follows. Let n units be placed in test at time zero.

• The m, and the $R_i$ and $i=1\cdots ,m-1$ are fixed prior to the test.
• At the first failure, $R_1$ units are randomly removed from the remaining $n-1$ surviving units.
• At the second failure, $R_2$ units are randomly removed from the remaining $n-R_1-2$ units.
• The test continues until the mth failure, when all remaining $R_m=n-R_1-R_2-\cdots -R_{m-1}-m$ are removed from experiment, so the life testing stops at the mth failure..
• The observed failure times $\mathbf{X}=(X_{1:m:n},X_{2:m:n}\cdots X_{m:m:n})$ constitute Type II progressive censored OS.
• If $R_1=R_2=\cdots =R_{m-1}=0,$ then $R_m=n-m$ which corresponds to the Type II censoring.
• If $R_1=R_2=\cdots =R_m=0,$then $m=n$ which corresponds to the usual order statistics.

Thus, usual OS and the Type II censoring become a special cases of progressively Type II censored samples. So any result established for progressively Type II censoring becomes a generalization of the corresponding result for OS and the Type II censoring.

The likelihood function may be written as [8]

$$f_{1:m:n,....,m:m:n}{(}_{1:m:n},x_{2:m:n}\cdots x_{m:m:n})=c\prod _{i=1}^{m}f(x_{i:m:n};\theta ){[1-F(x_{i:m:n};\theta )]}^{R_i}$$

(2)

where $c=n(n-R_1-1)(n-R_1-R_2-2)\cdots (n-R_1-R_2-R_3\cdots -R_{m-1}-m+1)$. The joint entropy contained in $(X_{1:m:n},X_{2:m:n}\cdots X_{i:m:n})$, i.e., a collection of first i of progressively Type II censored OS, is defined to be

$$H_{1....i:m:n}=E\{log{f}_{1:m:n,....,i:m:n}(X_{1:m:n},X_{2:m:n}\cdots X_{i:m:n})\}$$

(3)

where $f_{1:m:n,....,i:m:n}(x_{1:m:n},x_{2:m:n}\cdots ,x_{i:m:n})$, is the density function of $(X_{1:m:n},X_{2:m:n}\cdots X_{i:m:n})$. To our knowledge, Balakrishnan et al. [9] generalized the result of Park [3] testing exponentiality based on the Kullback-Leibler information with the type II censored data to a progressively Type II censored data and obtained an approximate to the joint entropy in progressively Type II censored samples based on nonparametric estimation. Hence, the exact values of the joint entropy in progressively Type II censored samples has not been obtained. Several applications for entropy such as characterization, tests for goodness-of-fit based on censored data, parameter estimation and quantization theory are known, for example see [3,9].

In the case of $H_{1....i:m:n}$, difficulty arise from the removal as well as the expression of $H_{1....i:m:n}$, which involves integration over i random variables, so simplifying the calculation of $H_{1....i:m:n}$ is more attractive. In this article we focus on the study of the properties of the joint entropy in progressively Type II censored OS. In Section 2 we developed the idea of Park [2] about the decomposition of entropy in OS to introduce an indirect approach for decomposition of entropy in progressive Type II censored OS. In Section 3 we derive a recurrence relations for the entropy in progressively Type II censored samples, which will prove helpful in calculating the entropy. In Section 4 we derive an efficient computational method to reduce r-dimensional integrals in the calculation of $H_{1\cdots r:m:n}$ to no integral where the computation of the entropy in progressively Type II censored samples simplifies to a sum; entropy of the smallest OS of varying sample size. In Section 5 we apply our results for computing the entropy in collections of a progressively Type II censored samples from normal and exponential distributions.

2. Decomposition of the Joint Entropy

Park [2] and Wong and Chen [4] have shown that the total entropy of i.i.d. random sample of size n is decreased if the sample is ordered. Park [2] showed how much the entropy of i.i.d. random sample of size n is decreased if the sample is ordered through, the following identity about the entropy of the ordered data $h_{1....:n:n}$

$$h_{1....:n:n}=n{h}_{1:1}-logn!$$

(4)

In view of Equation (4) and noting that progressive Type II censored sample can be seen as an ordered sample

$$(X_{1:m:n},X_{2:m:n}\cdots X_{m:m:n})$$

with the removals $(R_1,R_2,\cdots R_m)$, we have the following result for the entropy of the progressive Type II censored OS sample.

Lemma 2.1.

$$H_{1....m:m:n}=n{H}_{1:1:1}-logc$$

(5)

where $c=n(n-R_1-1)(n-R_1-R_2-2)\cdots (n-R_1-R_2-R_3\cdots -R_{m-1}-m+1)$, and $H_{1:1:1}=h_{1:n}=1-{\int }_{-\infty }^{\infty }f\left(x\right)logh\left(x\right)dx$.

Since the progressively Type II censored sample form a Markov chain [8], we have the following results.

Lemma 2.2.

• $H_{r+1....m:m:n|i...r:m:n}=H_{r+1....m:m:n|r:m:n},i=1,\cdots r$
• $H_{r+1....i:m:n}-H_{r+1....i:m:n|r:m:n}=H_{r+1:m:n}-H_{r+1:m:n|r:m:n},i=r+1,\cdots m.$

PROOF. From the Markov chain property of progressive Type II censored OS, the first part follows directly. The second part can be shown by using the first part and the symmetry of the mutual information Csisz$\acute{a}r$ [10],

$$\begin{array}{ccc}\hfill H_{r+1....i:m:n}-H_{r+1....i:m:n|r:m:n}& =& H_{r+1:m:n}-H_{r+1:m:n|r:m:n}\hfill \\ & =& H_{r:m:n}-H_{r:m:n|r+1:m:n}\hfill \end{array}$$

Next we show the following decomposition of the entropy of progressive Type II censored OS.

Lemma 2.3.

$$H_{1....m:m:n}=H_{1....r:m:n}+H_{r+1....m:m:n|r:m:n}$$

(6)

PROOF. By the additive property of the entropy measure and Lemma 2.1. we have the result.

We see from Equations (5) and (6) that the entropy of r progressive censored data $H_{1....r:m:n}$ can be obtained from $H_{r+1....m:m:n|r:m:n}$. So we consider $H_{r+1....m:m:n|r:m:n}$ to study $H_{1....r:m:n}$. Let $(X_{1:m:n},X_{2:m:n}\cdots X_{m:m:n})$ be a progressively Type II censored sample with censoring scheme $(R_1,R_2,\cdots R_m)$. The entropy in a collection of first i progressively Type II censored $(X_{1:m:n},X_{2:m:n}\cdots X_{i:m:n})$ is defined by Equation (3), and can be written as

$$\begin{array}{ccc}\hfill H_{1....i:m:n}& =& -{\int }_{-\infty }^{\infty }\cdots {\int }_{-\infty }^{x_{2:m:n}}{f}_{1..i:m:n}(x_{1:m:n},\cdots ,x_{i:m:n})\hfill \\ & \times & log{f}_{1..i:m:n}(x_{1:m:n},\cdots ,x_{i:m:n})d{x}_{1:m:n}\cdots d{x}_{i:m:n}\hfill \end{array}$$

(7)

where $f_{1....i:m:n}(x_{1:m:n},\cdots ,x_{i:m:n})$ is the joint pdf of the first i order statistics of the progressively Type II censored sample.

Using the Markov chain property of the order statistics from progressive Type II censored samples, we have the following decomposition for the score function:

$$log{f}_{1\cdots m:m:n}=log{f}_{1\cdots i:m:n}+log{f}_{i+1\cdots m:m:n|i:m:n}$$

where $f_{i+1\cdots m:m:n|i:m:n}$ is the pdf of $(X_{i+1\cdots m:m:n},\cdots X_{m:m:n})$ given $X_{i:m:n}=x_i$. The following decomposition follows from the strong additivity of the entropy

$$H_{1\cdots m:m:n}=H_{1\cdots i:m:n}+H_{i+1\cdots m:m:n|i:m:n},$$

where $H_{i+1\cdots m:m:n|i:m:n}$ is the average of the conditional information in $(X_{i+1\cdots m:m:n},\cdots ,X_{m:m:n})$ given $X_{i:m:n}=x_i$.

On the other hand, in view of the result of Balakrishnan and Aggarwala [8], the $f_{i+1\cdots m:n|i:m:n}$ is the joint density of the progressively Type II censored sample of size $(m-i)$, with censoring scheme $(R_{i+1},,\cdots R_m)$, from $(n-{\sum }_{j=1}^{j=i}{R}_j-i)$ drawn from the parent distribution $f\left(x\right)$ truncated from the left at $x_i$ with density $\frac{f\left(x\right)}{1-F\left(x_i\right)},x>x_i$. Therefore $H_{i+1\cdots m:m:n|i:m:n}$ can be written as the double integral

$$\begin{array}{ccc}\hfill H_{i+1\cdots m:m:n|i:m:n}& =& (n-\sum _{j=1}^{j=i}{R}_j-i){\int }_{-\infty }^{\infty }g\left(w\right)f_{i:m:n}\left(w\right)dw\hfill \\ & & -log(n-\sum _{j=1}^{j=i}{R}_j-i)!\hfill \end{array}$$

(8)

where

$$g\left(w\right)=-{\int }_w^{\infty }\frac{f(x;\theta )}{1-F(w;\theta )}log\frac{f\left(x\right)}{1-F\left(w\right)}dx$$

and $f_{i:m:n}\left(x\right)$ is defined by

$$f_{i:m:n}\left(x\right)=c_{i-1}\sum _{j=1}^i{a}_j\left(i\right){(1-F\left(x\right))}^{{\gamma }_j-1}f\left(x\right),-\infty <x<\infty ,1\le i\le m$$

where

$${\gamma }_i=n-i+\sum _{j=i}^m{R}_i,c_{i-1}=\prod _{j=1}^i{\gamma }_j,1\le i\le m$$

(9)

and

$$a_j\left(i\right)=\prod _{r=1,r\ne j}^i\frac{1}{{\gamma }_r-{\gamma }_j},1\le j\le i\le m$$

(10)

Since we already know about the entropy of the complete sample $H_{1\cdots m:m:n}$, the entropy $H_{1\cdots i:m:n}$ can be now easily derived from Equations (6) and (8).

EXAMPLE 2.1. For the exponential density $exp(-x)$, we can show that $g\left(w\right)=1$ so that

$$\begin{array}{ccc}\hfill H_{i+1\cdots m:m:n|i:m:n}& =& (n-\sum _{j=1}^i{R}_j-i){\int }_{-\infty }^{\infty }{f}_{i:m:n}\left(w\right)dw\hfill \\ & & -log(n-\sum _{j=1}^i{R}_j-i)!\hfill \end{array}$$

(11)

$$H_{i+1\cdots m:m:n|i:m:n}=(n-\sum _{j=1}^i{R}_j-i)-log(n-\sum _{j=1}^i{R}_j-i)!$$

Thus in that case

$$H_{1\cdots i:m:n}=i+\sum _{j=1}^{j=i}{R}_j+log(n-\sum _{j=1}^i{R}_j-i)!-log\left(n\right)-\sum _{k=1}^{m}log(n-\sum _{j=1}^k{R}_j-k)$$

where $H_{1:1:1}=1$ is the entropy in a single observation from the exponential density $exp(-x)$.

REMARK 2.1. We note that all of Park’s results concerning the entropy for the minimum order statistics $X_{1:n}$ works for the case of progressively Type II censored sample, since $f_{1:n}=f_{1:m:n}$.

3. Recurrence Relations

Recurrence relations between the cdf (pdf) of OS and progressive Type II censored OS have been studied by many authors for the purpose of simplifying the calculation of moments of OS and progressive Type II censored OS.

The standard recurrence relation for the moments of OS was obtained by Cole [11], and can be written as

$$n{\mu }_{r:n-1}^k=(n-r){\mu }_{r:n}^k+r{\mu }_{r+1:n}^k$$

(12)

where ${\mu }_{i:j}^k$ is the moments of the usual OS $X_{i:j}$.

This result can be directly derived from the corresponding recurrence relation between the cdf’s of OS. Kamps and Cramr Lemma 4 [12] obtained the corresponding recurrence relation for generalized OS as

$$\begin{array}{ccc}\hfill (k+n-r-1& +\hfill & \sum _{j=1}^{n-1}{m}_j)f_{X(r:n-1:(m_1,·,m_{n-1}))}\left(x\right)\hfill \\ & =& (k+n-r-1+\sum _{j=r+1}^{n-1}{m}_j)f_{X(r:n:(m_1,·,m_{n-1}))}\left(x\right)\hfill \\ & +& (r+\sum _{j=1}^r{m}_j)f_{X(r+1:n:(m_1,·,m_{n-1}))}\left(x\right),1\le r\le n-1\hfill \end{array}$$

(13)

Since the generalized OS includes the progressive Type II censored OS, it is clear that the case of progressive Type II censoring is subsumed in the above result. By setting $m_i=R_i$ for $i=1,2,\cdots ,m-1$, $m_i=0$ for $i=m,\cdots ,n-1,$ and $k=R_m+1$, we have

$$(m+\sum _{j=1}^m{R}_j)f_{i:m:n-1}=(m-i+\sum _{j=r+1}^m{R}_j)f_{i:m:n}+(i+\sum _{j=1}^i{R}_j)f_{i+1:m:n}$$

(14)

Using Equation (14) and the decomposition of the entropy in Equation (8) we have the following results for the entropy in the progressive censoring scheme.

RELATION 3.1

$$\begin{array}{ccc}\hfill H_{i+1\cdots m:m:n-1|i:m:n-1}& =& \frac{(n-{\sum }_{j=1}^i{R}_j-i)}{(m+{\sum }_{j=1}^m{R}_j)}{H}_{i+1\cdots m:m:n|i:m:n}\hfill \\ & +& \frac{({\sum }_{j=1}^i{R}_j+i)}{(m+{\sum }_{j=1}^m{R}_j)}{H}_{i+2\cdots m:m:n|i+1:m:n}+C_1(n,m,R_1\cdots ,R_i)\hfill \end{array}$$

(15)

where $C_1(n,m,R_1\cdots ,R_i)=$ $\frac{1}{n}\{(n-{\sum }_{j=1}^i{R}_j-i)log(n-{\sum }_{j=1}^i{R}_j-i)-log(n-{\sum }_{j=1}^i{R}_j-i)!\}$ and $n={\sum }_{j=1}^m{R}_j+m$.

PROOF. From Equation (8) we have

$$\begin{array}{ccc}\hfill H_{i+1\cdots m:m:n-1|i:m:n-1}& =& (n-\sum _{j=1}^i{R}_j-i-1){\int }_{-\infty }^{\infty }g\left(w\right)f_{i:m:n-1}\left(w\right)dw\hfill \\ & -& log(n-\sum _{j=1}^i{R}_j-i-1)!\hfill \end{array}$$

(16)

on the other hand Equation (14) yields

$$f_{i:m:n-1}\left(w\right)=\frac{(m-i+{\sum }_{j=r+1}^m{R}_j)}{(m+{\sum }_{j=1}^m{R}_j)}{f}_{i:m:n}\left(w\right)+\frac{(i+{\sum }_{j=1}^i{R}_j)}{(m+{\sum }_{j=1}^m{R}_j)}{f}_{i+1:m:n}\left(w\right)$$

(17)

combining Equations (16) and (17) and noting that $(m-i+{\sum }_{j=r+1}^m{R}_j)=(n-{\sum }_{j=1}^{j=i}{R}_j-i)$ since $n={\sum }_{j=1}^{j=m}{R}_j+m$, then the Lemma follows.

The following relation shows that the entropy of the first r of the progressive Type II censored OS of sample size $n-1$ can be obtained as a linear combination of the first r and $r+1$ of the progressive Type II censored OS of sample size n.

RELATION 3.2

$$\begin{array}{ccc}\hfill H_{1\cdots i:m:n-1}& =& \frac{(n-{\sum }_{j=1}^{j=i}{R}_j-i-1)}{(m+{\sum }_{j=1}^m{R}_j)}{H}_{1\cdots i:m:n}\hfill \\ & +& \frac{({\sum }_{j=1}^i{R}_j+i)}{(m+{\sum }_{j=1}^m{R}_j)}{H}_{1\cdots i+1:m:n}+C_2(n,m,R_1\cdots ,R_i)\hfill \end{array}$$

(18)

where

$$\begin{array}{ccc}\hfill C_2(n,m,R_1\cdots ,R_i)& =& \frac{(n-1)}{n}\{logn+\sum _{i=1}^m(n-\sum _{j=1}^i{R}_j-j-1)\}\hfill \\ & -& log(n-\sum _{j=1}^i{R}_j-i)!-(n-\sum _{j=1}^i{R}_j-i)log(n-\sum _{j=1}^i{R}_j-i)\hfill \end{array}$$

(19)

PROOF. For a sample of size $n-1$ the general decomposition of the entropy of progressive Type II censoring takes the form

$$I_{1\cdots m:m:n-1}=I_{1\cdots i:m:n-1}+I_{i+1\cdots m:m:n-1|i:m:n-1}$$

(20)

By applying RELATION 3.1 on Equation (20) we get

$$\begin{array}{ccc}\hfill H_{1\cdots m:m:n-1}& =& H_{1\cdots i:m:n-1}+\frac{(n-{\sum }_{j=1}^{j=i}{R}_j-i)}{(m+{\sum }_{j=1}^m{R}_j)}{H}_{i+1\cdots m:m:n|i:m:n}\hfill \\ & +& \frac{({\sum }_{j=1}^i{R}_j+i)}{(m+{\sum }_{j=1}^m{R}_j)}{H}_{i+2\cdots m:m:n|i+1:m:n}+C_1(n,m,R_1\cdots ,R_i)\hfill \end{array}$$

(21)

where $C_1$ defined above. Equation (21) can be written, by using Equations (5) and (6), as

$$\begin{array}{ccc}\hfill (& n& -1)H_{1:1:1}-\{logn+\sum _{i=1}^{m}log(n-\sum _{j=1}^{j=i}{R}_j-i-1)!\}=H_{1\cdots i:m:n-1}\hfill \\ & +& \frac{(n-{\sum }_{j=1}^i{R}_j-i-1)}{(m+{\sum }_{j=1}^m{R}_j)}\{n{H}_{1:1:1}-logn-\sum _{i=1}^{m}log(n-\sum _{j=1}^i{R}_j-i)!-H_{1\cdots i:m:n}\}\hfill \\ & +& \frac{({\sum }_{j=1}^i{R}_j+i)}{(m+{\sum }_{j=1}^m{R}_j)}\{n{H}_{1:1:1}-logn-\sum _{i=1}^{m}log(n-\sum _{j=1}^i{R}_j-i)!-H_{1\cdots i+1:m:n}\}\hfill \\ & +& C_1(n,R_1\cdots ,R_i)\hfill \end{array}$$

(22)

After some simplifications the result follows.

REMARK 3.1. With $R_1=R_2=\cdots =R_m=0$ all results of Section 2 and Section 3 reduce to corresponding results for the entropy in a collections usual OS.

4. Computational Method for Calculating $H_{1\cdots i:m:n}$

In this section we provide another approach to simplify the calculation of the entropy in a collection of progressively Type II censored OS. We reduce r integrals in the calculation of $H_{1\cdots r:m:n}$ to no integral where the computation of the entropy in progressively Type II censored samples simplifies to a sum; entropy of the smallest OS of varying sample size $h_{1:n}$ .

Lemma 4.1. Let $X_1,X_2\cdots X_n$ be i.i.d. random sample of size n from pdf $f\left(x\right)$ with cdf $F\left(x\right)$ and hazard function $h\left(x\right)=\frac{f\left(x\right)}{1-F\left(x\right)}$, and let $X_{1:n},X_{2:n},\cdots ,X_{n:n}$ be OS corresponding to this sample. Park [2] obtained the entropy in the smallest order statistics as

$$h_{1:n}=1-logn-{\int }_{-\infty }^{\infty }logh\left(x\right)d{F}_{1:n}\left(x\right)$$

(23)

Theorem 4.1. Let $(X_{1:m:n},X_{2:m:n}\cdots X_{m:m:n})$ be a progressively Type II censored sample with censoring scheme $(R_1,R_2,\cdots R_m)$. The entropy in the r collection of progressively Type II censored sample $(X_{1:m:n},X_{2:m:n}\cdots X_{r:m:n})$ can be written as

$$H_{1\cdots r:m:n}=r-log{c}^{\prime }\left(r\right)-\sum _{s=1}^r{c}^{\prime }\left(s\right)\sum _{i=1}^{s-1}\frac{c_{i,s-1}(R_1+1,\cdots ,R_{s-1}+1)}{R_i^{\prime }}(1-log{R}_i^{\prime }-h_{1:R_i^{\prime }})$$

(24)

where $R_i^{\prime }=(R_s^{*}+1)+{\sum }_{j=s-i}^{s-1}(R_j+1)$, $R_s^{*}=(n-s-R_1-\cdots -R_{s-1}+1)$, $c^{\prime }\left(t\right)=n(n-R_1-1)\cdots (n-R_1-\cdots -R_{t-1}-t+1)$ and

$$c_{i,s}(R_1,\cdots ,R_s)=\frac{{(-1)}^i}{\{{\prod }_{j=1}^i{\sum }_{k=s-i+1}^{s-i+j}{R}_k\}\{{\prod }_{j=1}^{s-i}{\sum }_{k=j}^{s-i}{R}_k\}}$$

(25)

in which empty products are defined as 1.

PROOF. By the Markov chain properties of progressive Type II censored samples, one can write

$$f_{1:m:n,...,r:m:n}(x_{1:m:n},\cdots ,x_{r:m:n})=f_{1:m:n}\left(x_1\right)f_{2|1:m:n}\left(x_2\right|x_1)\cdots f_{r|r-1:m:n}\left(x_r\right|x_{r-1})$$

where $f_{i+1|i:m:n}\left(x_{i+1}\right|x_i)$ is the conditional pdf of $X_{i+1:m:n}$ given $X_{i:m:n}=x_i$, which also is the density of the first order statistic of a sample of size $(n-R_1-\cdots -R_i-i)$ with the truncated density $g\left(x\right)=\frac{f\left(x\right)}{1-F\left(x_i\right)}$. Therefore, we have

$$H_{1:m:n,...,r:m:n}=H_{1:m:n}+H_{2|1:m:n}+\cdots +H_{r|r-1:m:n}$$

(26)

where $H_{i+1|i:m:n}$ is the expected entropy in $X_{i+1:m:n}$ given $X_{i:m:n}=x_i$ i.e.,

$$H_{i+1|i:m:n}=E\{-{\int }_{-\infty }^{\infty }{f}_{i+1|i:m:n}\left(x\right|x_{i:m:n})log{f}_{i+1|i:m:n}\left(x\right|x_{i:m:n})dx\}$$

(27)

By Lemma 4.1. and noting that, condition on $X_{i:m:n}=x_i$, $X_{i+1:m:n}$ has the same pdf as the first order statistic from a random sample of size $(n-R_1-\cdots -R_i-i)$ with pdf $g\left(x\right)=\frac{f\left(x\right)}{1-F\left(x_i\right)}$. Equation (27) can be written as

$$H_{i+1|i:m:n}=1-log(n-R_1-\cdots -R_i-i)-I$$

(28)

where

$$I=E\{{\int }_{x_{i:m:n}}^{\infty }{f}_{i+1|i:m:n}\left(x\right|x_{i:m:n})logh\left(x\right)dx\}$$

(29)

By changing integrals and noting that $X_{i:m:n}<X_{i+1:m:n},$ we have

$$\begin{array}{ccc}\hfill I& =& {\int }_{-\infty }^{\infty }\{{\int }_{x_i}^{\infty }{f}_{i+1|i}(x|x_i)logh\left(x\right)dx\}{f}_i(x_{i)}d{x}_i\hfill \\ & =& {\int }_{-\infty }^{\infty }logh(x)\{{\int }_{-\infty }^x{f}_{i+1|i}(x|x_i)f_i(x_{i)}d{x}_i\}dx\hfill \\ & =& {\int }_{-\infty }^{\infty }logh\left(x\right)\{{\int }_{-\infty }^x{f}_{i,i+1}(x_i,x)d{x}_i\}dx\hfill \\ & =& {\int }_{-\infty }^{\infty }\{logh\left(x\right)\}{f}_{i+1}\left(x\right)dx\hfill \end{array}$$

(30)

Therefore Equation (31), can be written as

$$H_{i+1|i:m:n}=1-log(n-R_1-\cdots -R_i-i)-{\int }_{-\infty }^{\infty }\{logh\left(x\right)\}{f}_{i+1}\left(x\right)dx$$

(31)

Thus by using Equations (26) and (31) $H_{1....r:m:n}$ can be expressed as a summation of single integral as

$$H_{1....r:m:n}=r-log{c}^{\prime }\left(r\right)-\sum _{i=1}^r{\int }_{-\infty }^{\infty }logh\left(x\right)f_{i:m:n}\left(x\right)dx$$

(32)

where $c^{\prime }\left(r\right)$ is defined above. From Theorem 1 in Balakrishnan et al. [13], we have the following relation for $f_{s:m:n}$

$$f_{s:m:n}=c^{\prime }\left(s\right)\sum _{i=1}^{s-1}{c}_{i,s-1}(R_1+1,\cdots ,R_{s-1}+1)f\left(x_s\right){(1-F\left(x_s\right))}^{R_i^{\prime }}$$

(33)

$-\infty <x_s<\infty$ where, $R_i^{\prime }$, $R_s^{*}$, $c^{\prime }\left(s\right)$ and $c_{i,s-1}(R_1+1,\cdots ,R_{s-1}+1)$ are defined above.

We reexpress Equation (33) as

$$f_{s:m:n}=c^{\prime }\left(s\right)\sum _{i=1}^{s-1}\frac{c_{i,s-1}(R_1+1,\cdots ,R_{s-1}+1)}{R_i^{\prime }}{f}_{1:R_i}\left(x_s\right)$$

(34)

where $f_{1:R_i}$ is the usual smallest order statistics in a sample of size $R_i^{\prime }$. If we use Equations (23) and (34) in Equation (32) the result follows.

We have written program in the algebraic manipulation package, MATHEMATICA [14], for computing Theorem 4.1 and Lemma 4.1 calculated above. For a pre-determined progressively Type II censoring scheme $(n,m,R_1,R_2,\cdots R_m)$ the program return the numerical values of the entropy. The electronic version of the computer program can be obtained by contacting the corresponding author.

REMARK 4.1. The entropy of the smallest usual order statistics are known for well-known distributions for example see Park [2] and Asadi et al. [15].

5. Illustrative Examples

The entropy of the smallest OS $h_{1:n}$ has the expression [2].

$$h_{1:n}=1-\frac{1}{n}-logn-{\int }_{-\infty }^{\infty }logf\left(x\right)d{F}_{1:n}$$

(35)

EXAMPLE 5.1. For the normal distribution, $f(x;\mu ,\sigma )=\frac{1}{\sqrt{2}\pi {\sigma }^2}exp-{(x-\mu )}^2/2{\sigma }^2,$ the entropy of the smallest OS $h_{1:n}$ takes the form

$$h_{1:n}=1-\frac{1}{n}-logn+\frac{log\left(2\pi \right)}{2}+log\sigma +\frac{{\mu }_{1:n}^{\left(2\right)}}{2}$$

(36)

where ${\mu }_{r:n}^{\left(2\right)}$ is the second moment of $X_{r:n}$ of the standard normal distribution, see Park [2]. We use Theorem 4.1 and Equation (36) to calculate the $H_{1\cdots r:m:n}$ given in Table 1.

Discussion

Table 1 provides the values of $H_{1\cdots r:m:n}$ for $n=5,10$ and $m=3,5$ for different schemes and $r=1\cdots ,m$. The entries were computed using Theorem 4.1, Equation (36) and MATHEMATICA [14]. For $r<m$, the table gives the values of the entropy in a collection of r of OS from a progressively Type II censored sample. For $r=m$ the table gives the values of entropy in a complete progressive Type II censored sample. The table includes the cases $r_1=r_2=\cdots =r_{m-1}=0,r_m=n-m$ which corresponds to the Type II censored sample and $r_1=r_2=\cdots =r_m=0,n=m$ which corresponds to the complete sample.

EXAMPLE 5.2. For the logistic distribution, $f\left(x\right)=\frac{exp-\left(x\right)}{{(1+exp-\left(x\right))}^2},-\infty <x<\infty$ the entropy of the smallest OS $h_{1:n}$ takes the form

$$h_{1:n}=log\beta (1,n)-(n-1)(\psi \left(n\right)-\psi (n+1)+2\psi (n+1)-\psi \left(n\right)-\psi \left(1\right)$$

(37)

where $\beta (a,b)=\frac{\Gamma \left(a\right)\Gamma \left(b\right)}{\Gamma (a+b)}$ is the beta function and $\psi \left(z\right)=\frac{dlog\Gamma \left(z\right)}{dz}$ is the digamma function, see Asadi et al. [15]. We use Theorem 4.1 and Equation (37) to calculate the $H_{1\cdots r:m:n}.$

Table 2 provides the values of $H_{1\cdots r:m:n}$ for $n=5,10$ and $m=3,5$ for different schemes and $r=1\cdots ,m$. The entries were computed using Theorem 4.1 and Equation (37) and MATHEMATICA [14]. For $r<m$, the Table gives the values of the entropy in a collection r of OS progressive Type II censored sample. For $r=m$, the Table gives the values of entropy in a complete progressive Type II censored sample.

Table 1. The entropy in a collection of order statistics from a progressive Type II censored sample from a normal distribution with unit standard deviation.

n	m	Censoring scheme	r	OS of Proressive samples	Entropy
5	3	(2,0,0)	1	$\left(X_{1:3:5}\right)$	1.0096
5	3	(2,0,0)	2	$(X_{1:3:5},X_{2:3:5})$	2.11423
5	3	(2,0,0)	3	$(X_{1:3:5},X_{2:3:5},X_{3:3:5})$	3.40163
5	3	(0,0,2)	1	$\left(X_{1:3:5}\right)$	1.0096
5	3	(0,0,2)	2	$(X_{1:3:5},X_{2:3:5})$	1.87510
5	3	(0,0,2)	3	$(X_{1:3:5},X_{2:3:5},X_{3:3:5})$	3.18129
5	3	(1,1,0)	1	$\left(X_{1:3:5}\right)$	1.0096
5	3	(1,1,0)	2	$(X_{1:3:5},X_{2:3:5})$	1.97448
5	3	(1,1,0)	3	$(X_{1:3:5},X_{2:3:5},X_{3:3:5})$	3.27328
5	5	(0,0,0,0,0)	1	$\left(X_{1:5}\right)$	1.0096
5	5	(0,0,0,0,0)	2	$(X_{1:5},X_{2:5})$	1.8751
5	5	(0,0,0,0,0)	3	$(X_{1:5},X_{2:5},X_{3:5})$	2.76331
5	5	(0,0,0,0,0)	5	$(X_{1:5},\cdots ,X_{5:5})$	5.01551
10	5	(0,0,0,0,5)	1	$\left(X_{1:5:10}\right)$	0.872403
10	5	(0,0,0,0,5)	2	$(X_{1:5:10},X_{2:5:10})$	1.52808
10	5	(0,0,0,0,5)	3	$(X_{1:5:10},X_{2:5:10},X_{3:5:10})$	2.12015
10	5	(0,0,0,0,5)	4	$(X_{1:5:10},\cdots ,X_{4:0:10})$	2.69953
10	5	(0,0,0,0,5)	5	$(X_{1:5:10},\cdots ,X_{5:5:10})$	3.29964
10	5	(5,0,0,0,0)	1	$\left(X_{1:5:10}\right)$	0.872403
10	5	(5,0,0,0,0)	2	$(X_{1:5:10},X_{2:5:10})$	1.78936
10	5	(5,0,0,0,0)	3	$(X_{1:5:10},X_{2:5:10},X_{3:5:10})$	2.69933
10	5	(5,0,0,0,0)	4	$(X_{1:5:10},\cdots ,X_{4:5:10})$	3.70963
10	5	(5,0,0,0,0)	5	$(X_{1:5:10},\cdots ,X_{5:5:10})$	4.96641
10	5	(3,2,0,0,0)	1	$\left(X_{1:5:10}\right)$	0.872403
10	5	(3,2,0,0,0)	2	$(X_{1:5:10},X_{2:5:10})$	1.65948
10	5	(3,2,0,0,0)	3	$(X_{1:5:10},X_{2:5:10},X_{3:5:10})$	2.58920
10	5	(3,2,0,0,0)	4	$(X_{1:5:10},\cdots ,X_{4:5:10})$	3.60857
10	5	(3,2,0,0,0)	5	$(X_{1:5:10},\cdots ,X_{5:5:10})$	4.86899
10	10	(0,0,0,0,0)	1	$\left(X_{1:10}\right)$	0.872403
10	10	(0,0,0,0,0)	2	$(X_{1:10},X_{2:10})$	1.52808
10	10	(0,0,0,0,0)	3	$(X_{1:5:10},X_{2:10},X_{3:10})$	2.12015
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{4:10})$	2.69953
10	10	(0,0,0,0,0)	5	$(X_{1:10},\cdots ,X_{5:10})$	3.29974
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{6:10})$	3.95362
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{7:10})$	4.57641
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{8:10})$	5.50748
10	10	(0,0,0,0,0)	10	$(X_{1:10},\cdots ,X_{10:10})$	7.62962

Table 1. The entropy in a collection of order statistics from a progressive Type II censored sample from a normal distribution with unit standard deviation.

n	m	Censoring scheme	r	OS of Proressive samples	Entropy
5	3	(2,0,0)	1	$\left(X_{1:3:5}\right)$	1.0096
5	3	(2,0,0)	2	$(X_{1:3:5},X_{2:3:5})$	2.11423
5	3	(2,0,0)	3	$(X_{1:3:5},X_{2:3:5},X_{3:3:5})$	3.40163
5	3	(0,0,2)	1	$\left(X_{1:3:5}\right)$	1.0096
5	3	(0,0,2)	2	$(X_{1:3:5},X_{2:3:5})$	1.87510
5	3	(0,0,2)	3	$(X_{1:3:5},X_{2:3:5},X_{3:3:5})$	3.18129
5	3	(1,1,0)	1	$\left(X_{1:3:5}\right)$	1.0096
5	3	(1,1,0)	2	$(X_{1:3:5},X_{2:3:5})$	1.97448
5	3	(1,1,0)	3	$(X_{1:3:5},X_{2:3:5},X_{3:3:5})$	3.27328
5	5	(0,0,0,0,0)	1	$\left(X_{1:5}\right)$	1.0096
5	5	(0,0,0,0,0)	2	$(X_{1:5},X_{2:5})$	1.8751
5	5	(0,0,0,0,0)	3	$(X_{1:5},X_{2:5},X_{3:5})$	2.76331
5	5	(0,0,0,0,0)	5	$(X_{1:5},\cdots ,X_{5:5})$	5.01551
10	5	(0,0,0,0,5)	1	$\left(X_{1:5:10}\right)$	0.872403
10	5	(0,0,0,0,5)	2	$(X_{1:5:10},X_{2:5:10})$	1.52808
10	5	(0,0,0,0,5)	3	$(X_{1:5:10},X_{2:5:10},X_{3:5:10})$	2.12015
10	5	(0,0,0,0,5)	4	$(X_{1:5:10},\cdots ,X_{4:0:10})$	2.69953
10	5	(0,0,0,0,5)	5	$(X_{1:5:10},\cdots ,X_{5:5:10})$	3.29964
10	5	(5,0,0,0,0)	1	$\left(X_{1:5:10}\right)$	0.872403
10	5	(5,0,0,0,0)	2	$(X_{1:5:10},X_{2:5:10})$	1.78936
10	5	(5,0,0,0,0)	3	$(X_{1:5:10},X_{2:5:10},X_{3:5:10})$	2.69933
10	5	(5,0,0,0,0)	4	$(X_{1:5:10},\cdots ,X_{4:5:10})$	3.70963
10	5	(5,0,0,0,0)	5	$(X_{1:5:10},\cdots ,X_{5:5:10})$	4.96641
10	5	(3,2,0,0,0)	1	$\left(X_{1:5:10}\right)$	0.872403
10	5	(3,2,0,0,0)	2	$(X_{1:5:10},X_{2:5:10})$	1.65948
10	5	(3,2,0,0,0)	3	$(X_{1:5:10},X_{2:5:10},X_{3:5:10})$	2.58920
10	5	(3,2,0,0,0)	4	$(X_{1:5:10},\cdots ,X_{4:5:10})$	3.60857
10	5	(3,2,0,0,0)	5	$(X_{1:5:10},\cdots ,X_{5:5:10})$	4.86899
10	10	(0,0,0,0,0)	1	$\left(X_{1:10}\right)$	0.872403
10	10	(0,0,0,0,0)	2	$(X_{1:10},X_{2:10})$	1.52808
10	10	(0,0,0,0,0)	3	$(X_{1:5:10},X_{2:10},X_{3:10})$	2.12015
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{4:10})$	2.69953
10	10	(0,0,0,0,0)	5	$(X_{1:10},\cdots ,X_{5:10})$	3.29974
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{6:10})$	3.95362
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{7:10})$	4.57641
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{8:10})$	5.50748
10	10	(0,0,0,0,0)	10	$(X_{1:10},\cdots ,X_{10:10})$	7.62962

Table 2. The entropy in a collection of order statistics from a progressive Type II censored sample from logistic distribution.

n	m	Censoring scheme	r	OS of Proressive samples	Entropy
5	3	(2,0,0)	1	$\left(X_{1:3:5}\right)$	1.67390
5	3	(2,0,0)	2	$(X_{1:3:5},X_{2:3:5})$	3.30409
5	3	(2,0,0)	3	$(X_{1:3:5},X_{2:3:5},X_{3:3:5})$	5.18643
5	3	(0,0,2)	1	$\left(X_{1:3:5}\right)$	1.67390
5	3	(0,0,2)	2	$(X_{1:3:5},X_{2:3:5})$	3.07589
5	3	(0,0,2)	3	$(X_{1:3:5},X_{2:3:5},X_{3:3:5})$	4.95024
5	3	(1,1,0)	1	$\left(X_{1:3:5}\right)$	1.6739
5	3	(1,1,0)	2	$(X_{1:3:5},X_{2:3:5})$	3.16708
5	3	(1,1,0)	3	$(X_{1:3:5},X_{2:3:5},X_{3:3:5})$	5.04210
5	5	(0,0,0,0,0)	1	$\left(X_{1:5}\right)$	1.67390
5	5	(0,0,0,0,0)	2	$(X_{1:5},X_{2:5})$	3.07587
5	5	(0,0,0,0,0)	3	$(X_{1:5},X_{2:5},X_{3:5})$	4.46788
5	5	(0,0,0,0,0)	5	$(X_{1:5},\cdots ,X_{5:5})$	7.9208
10	5	(0,0,0,0,5)	1	$\left(X_{1:5:10}\right)$	1.62638
10	5	(0,0,0,0,5)	2	$(X_{1:5:10},X_{2:5:10})$	2.89455
10	5	(0,0,0,0,5)	3	$(X_{1:5:10},X_{2:5:10},X_{3:5:10})$	4.03011
10	5	(0,0,0,0,5)	4	$(X_{1:5:10},\cdots ,X_{4:0:10})$	5.11611
10	5	(0,0,0,0,5)	5	$(X_{1:5:10},\cdots ,X_{5:5:10})$	6.20260
10	5	(5,0,0,0,0)	1	$\left(X_{1:5:10}\right)$	1.62638
10	5	(5,0,0,0,0)	2	$(X_{1:5:10},X_{2:5:10})$	3.10523
10	5	(5,0,0,0,0)	3	$(X_{1:5:10},X_{2:5:10},X_{3:5:10})$	4.52211
10	5	(5,0,0,0,0)	4	$(X_{1:5:10},\cdots ,X_{4:5:10})$	6.059990
10	5	(5,0,0,0,0)	5	$(X_{1:5:10},\cdots ,X_{5:5:10})$	7.96778
10	5	(3,2,0,0,0)	1	$\left(X_{1:5:10}\right)$	1.62638
10	5	(3,2,0,0,0)	2	$(X_{1:5:10},X_{2:5:10})$	2.99964
10	5	(3,2,0,0,0)	3	$(X_{1:5:10},X_{2:5:10},X_{3:5:10})$	4.43974
10	5	(3,2,0,0,0)	4	$(X_{1:5:10},\cdots ,X_{4:5:10})$	5.97958
10	5	(3,2,0,0,0)	5	$(X_{1:5:10},\cdots ,X_{5:5:10})$	7.88009
10	10	(0,0,0,0,0)	1	$\left(X_{1:10}\right)$	1.62638
10	10	(0,0,0,0,0)	2	$(X_{1:10},X_{2:10})$	2.89455
10	10	(0,0,0,0,0)	3	$(X_{1:5:10},X_{2:10},X_{3:10})$	4.03011
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{4:10})$	5.11611
10	10	(0,0,0,0,0)	5	$(X_{1:10},\cdots ,X_{5:10})$	6.20260
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{6:10})$	7.33016
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{7:10})$	8.54041
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{8:10})$	9.88642
10	10	(0,0,0,0,0)	10	$(X_{1:10},\cdots ,X_{10:10})$	13.44020

Table 2. The entropy in a collection of order statistics from a progressive Type II censored sample from logistic distribution.

n	m	Censoring scheme	r	OS of Proressive samples	Entropy
5	3	(2,0,0)	1	$\left(X_{1:3:5}\right)$	1.67390
5	3	(2,0,0)	2	$(X_{1:3:5},X_{2:3:5})$	3.30409
5	3	(2,0,0)	3	$(X_{1:3:5},X_{2:3:5},X_{3:3:5})$	5.18643
5	3	(0,0,2)	1	$\left(X_{1:3:5}\right)$	1.67390
5	3	(0,0,2)	2	$(X_{1:3:5},X_{2:3:5})$	3.07589
5	3	(0,0,2)	3	$(X_{1:3:5},X_{2:3:5},X_{3:3:5})$	4.95024
5	3	(1,1,0)	1	$\left(X_{1:3:5}\right)$	1.6739
5	3	(1,1,0)	2	$(X_{1:3:5},X_{2:3:5})$	3.16708
5	3	(1,1,0)	3	$(X_{1:3:5},X_{2:3:5},X_{3:3:5})$	5.04210
5	5	(0,0,0,0,0)	1	$\left(X_{1:5}\right)$	1.67390
5	5	(0,0,0,0,0)	2	$(X_{1:5},X_{2:5})$	3.07587
5	5	(0,0,0,0,0)	3	$(X_{1:5},X_{2:5},X_{3:5})$	4.46788
5	5	(0,0,0,0,0)	5	$(X_{1:5},\cdots ,X_{5:5})$	7.9208
10	5	(0,0,0,0,5)	1	$\left(X_{1:5:10}\right)$	1.62638
10	5	(0,0,0,0,5)	2	$(X_{1:5:10},X_{2:5:10})$	2.89455
10	5	(0,0,0,0,5)	3	$(X_{1:5:10},X_{2:5:10},X_{3:5:10})$	4.03011
10	5	(0,0,0,0,5)	4	$(X_{1:5:10},\cdots ,X_{4:0:10})$	5.11611
10	5	(0,0,0,0,5)	5	$(X_{1:5:10},\cdots ,X_{5:5:10})$	6.20260
10	5	(5,0,0,0,0)	1	$\left(X_{1:5:10}\right)$	1.62638
10	5	(5,0,0,0,0)	2	$(X_{1:5:10},X_{2:5:10})$	3.10523
10	5	(5,0,0,0,0)	3	$(X_{1:5:10},X_{2:5:10},X_{3:5:10})$	4.52211
10	5	(5,0,0,0,0)	4	$(X_{1:5:10},\cdots ,X_{4:5:10})$	6.059990
10	5	(5,0,0,0,0)	5	$(X_{1:5:10},\cdots ,X_{5:5:10})$	7.96778
10	5	(3,2,0,0,0)	1	$\left(X_{1:5:10}\right)$	1.62638
10	5	(3,2,0,0,0)	2	$(X_{1:5:10},X_{2:5:10})$	2.99964
10	5	(3,2,0,0,0)	3	$(X_{1:5:10},X_{2:5:10},X_{3:5:10})$	4.43974
10	5	(3,2,0,0,0)	4	$(X_{1:5:10},\cdots ,X_{4:5:10})$	5.97958
10	5	(3,2,0,0,0)	5	$(X_{1:5:10},\cdots ,X_{5:5:10})$	7.88009
10	10	(0,0,0,0,0)	1	$\left(X_{1:10}\right)$	1.62638
10	10	(0,0,0,0,0)	2	$(X_{1:10},X_{2:10})$	2.89455
10	10	(0,0,0,0,0)	3	$(X_{1:5:10},X_{2:10},X_{3:10})$	4.03011
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{4:10})$	5.11611
10	10	(0,0,0,0,0)	5	$(X_{1:10},\cdots ,X_{5:10})$	6.20260
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{6:10})$	7.33016
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{7:10})$	8.54041
10	10	(0,0,0,0,0)	4	$(X_{1:10},\cdots ,X_{8:10})$	9.88642
10	10	(0,0,0,0,0)	10	$(X_{1:10},\cdots ,X_{10:10})$	13.44020