Bayesian Inference for Different Entropy Measures of the Kumaraswamy Distribution Under Progressive Type-II Censoring with Binomial Removal

Abstract

The estimation of the uncertainty of random variables or the entropy of stochastic processes has attracted considerable attention in many studies. In this study, we focus on obtaining estimators for the Shannon, Rényi, Tsallis, and Havrda–Charvat entropy measures of the Kumaraswamy distribution. Maximum likelihood and Bayesian estimation methods are employed to obtain entropy estimators under progressive Type-II censoring schemes with binomial removals. The Tierney–Kadane approximation is used to obtain the Bayesian estimators. The consistency and asymptotic normality of the maximum likelihood estimators are also stated under standard regularity conditions, and approximate confidence intervals for the entropy measures are constructed accordingly. The behavior of the proposed estimators under various sample sizes and censoring schemes is investigated through an extensive Monte Carlo simulation study. Finally, the Kumaraswamy distribution is fitted to two real datasets from the fields of economics and agricultural hydrology. In both applications, the Kumaraswamy distribution is shown to provide a better fit than the three competing unit distributions. Under different progressive Type-II censoring schemes, maximum likelihood and TK-based Bayesian estimates of the four entropy measures are obtained, while Markov chain Monte Carlo is additionally used to construct Bayesian credible intervals. The resulting credible intervals are found to be similar to the corresponding maximum likelihood-based confidence intervals, supporting the practical applicability of the proposed estimation procedures for entropy measures under censored data.

1. Introduction

Entropy is a concept that quantitatively measures the uncertainty of random variables and the information shared between them. Mathematically, it represents the amount of information contained in a system based on probability distributions. With the advancement of information theory, entropy has become widely utilized not only in mathematics and statistics but also across various disciplines, including biology, genetics, chemistry, physics, quantum mechanics, reliability analysis, thermodynamics, probability theory, and electrical and communication engineering. The concept of entropy was first defined by Shannon [1] and was proposed to mathematically model information transmission in communication systems. In this context, the most commonly used entropy measure is Shannon entropy [1]. Although Shannon entropy is a fundamental measure in information theory, more flexible and generalized entropy measures are required in some cases. Therefore, Rényi entropy is introduced as a generalized alternative to Shannon entropy. However, in certain conditions and application fields, Rényi entropy is found to be insufficient, leading to the development of different types of entropy to address these limitations. Tsallis and Havrda–Charvat (HC) entropies are introduced to address different aspects of uncertainty, particularly in physical systems, social sciences, and communication. The Tsallis entropy is developed for complex systems with long-range interactions deviating from traditional thermodynamic principles and is widely used in statistical mechanics and quantum physics. The HC entropy has emerged as a significant tool in social sciences and economics, particularly for inequality measurement and income distribution analysis, by extending information theory. The Shannon, Rényi, Tsallis, and HC entropies are calculated as follows, respectively:

$$SH=-{\displaystyle \underset{-\infty }{\overset{\infty }{\int }}}log\left(f\left(x\right)\right)f\left(x\right)dx,$$

(1)

$$R_{\delta }={\displaystyle \frac{1}{1-\delta }}log\left\{{\displaystyle \underset{-\infty }{\overset{\infty }{\int }}}{\left[f\left(x\right)\right]}^{\delta }dx\right\},\delta >0,\left(\delta \ne 1\right),$$

(2)

$$T_{\delta }={\displaystyle \frac{1}{\delta -1}}\left\{1-{\displaystyle \underset{-\infty }{\overset{\infty }{\int }}}{\left[f\left(x\right)\right]}^{\delta }dx\right\},\delta >0,\left(\delta \ne 1\right),$$

(3)

$$H{C}_{\delta }={\displaystyle \frac{1}{2^{1-\delta }-1}}\left\{{\displaystyle \underset{-\infty }{\overset{\infty }{\int }}}{\left[f\left(x\right)\right]}^{\delta }dx-1\right\},\delta >0,\left(\delta \ne 1\right),$$

(4)

where $f\left(x\right)$ is a probability density function (pdf) on $\mathbb{R}$. Researchers in probability and applied statistics conduct extensive studies on entropy estimation. An overview of the primary references in this area is presented below. Kayal and Kumar [2] use maximum likelihood (ML) and Bayesian methods to estimate the Shannon entropy for the exponential distribution under the Linex loss function. They also examine the performance of these methods under different sample sizes and parameter settings. Similarly, Al-Babtain et al. [3] employed the ML method to obtain estimates for different entropy measures using the Kumaraswamy distribution. Shrahili et al. [4] estimated different types of entropy for the log-logistic distribution under progressive Type-II censored samples. Alam and Nassar [5] examined the estimation of entropy for the inverse Weibull distribution under progressive Type-II censored data by using ML and maximum product spacing (MPS) methods. Kotb and Alomari [6] employed the ML and Bayesian methods to obtain entropy estimates for the Rayleigh distribution under a progressively first-failure censored sample. Zhao et al. [7] examined the empirical entropy method under Type-II censored sampling and supported their findings through Monte Carlo simulations. Lee [8] investigated the ML and Bayesian methods for entropy estimation based on the inverse Weibull distribution under a generalized progressively hybrid censored sample. Recently, Gong and Yin [9] investigated the estimation of Shannon and Rényi entropies based on the generalized inverse exponential distribution under progressive Type-II censoring. Kumar et al. [10] examined the estimation of Shannon entropy for the Maxwell distribution using progressively first-failure censored data. Qubbaj et al. [11] considered the estimation of entropy and extropy under progressive Type-I interval censoring. In a recent study, Ren et al. [12] obtained Shannon estimators for the inverse exponential Rayleigh distribution under progressive Type-II censoring using ML and Bayesian methods.

Researchers typically perform lifetime tests on a complete sample to estimate entropy measures. However, due to time and cost limitations, it is not always feasible to observe the lifetime of all units. Therefore, estimation under censored data is a more practical and efficient approach. In the literature, various censoring schemes have been introduced for the analysis of censored data. Among these, the two most frequently used methods are the Type-I and Type-II censoring schemes. In Type-I censoring, the experiment is terminated at a predetermined time, whereas in Type-II censoring, it is terminated when a specified number of failures occur. In Type-I censoring, the experiment may conclude before any failures or events are observed. In Type-II censoring, the time until a predetermined number of failures occurs can be quite lengthy. Because of these constraints, researchers have proposed hybrid censoring as an alternative censoring scheme. Hybrid censoring is known as a combination of Type-I and Type-II censoring schemes. However, as the hybrid censoring scheme does not allow the removal of experimental units from the life test at the time of any failure, the progressive Type-II censoring scheme has been developed as an alternative approach. The progressive Type-II censoring is briefly defined as follows. Suppose that a lifetime test consists of n units, and $X_{1:m:n},X_{2:m:n},...,X_{m:m:n}$$(m<n)$ units are observed. Let $(R_1,R_2,\dots ,R_{m-1},R_m)$ denote the censoring scheme predetermined by the researcher. At the time of the first failure, $\left(R_1\right)$ units are removed from the $(n-1)$ surviving experimental units. At the time of the second failure, $\left(R_2\right)$ units are removed from the remaining $(n-R_1-2)$ surviving units in the life test. This process repeats until the $mth$ failure occurs, and the remaining $(R_m=n-m-{\displaystyle \sum _{i=1}^{m-1}}{R}_i)$ units are removed from the life test. At this point, the experiment is terminated. It is evident that the censored sample size is $n-m$. In the progressive Type-II censoring scheme, selecting $\left((R_1=0,R_2=0,\dots ,R_{m-1}=0,R_m=n-m)\right)$ results in Type-II censoring, whereas selecting $(R_1=0,R_2=0,\dots ,R_m=0)$ leads to a complete sample. However, in some studies, the censoring scheme $R_j$ may not be predetermined. Therefore, at each failure time, the distribution of the units to be removed from the experiment is assumed to follow a binomial distribution. That is, $R_j\sim \mathrm{Binom}\left(n-m-{\displaystyle \sum _{i=1}^{j-1}}{R}_i,p\right),j=1,2,\dots ,m-1$ and $R_m=n-m-{\displaystyle \sum _{i=1}^{m-1}}{R}_i$ [13]. The progressive Type-II censoring scheme is widely used in modeling reliability analysis and lifetime data. For example, Balakrishnan and Aggarwala [14] examined the parameter estimators of some distributions under the progressive Type-II censoring scheme with the ML method, and they demonstrated the advantages of this censoring scheme in reliability analysis. Dey et al. [15] investigated the effectiveness of the Bayesian method under a progressive Type-II censored sample. Then, they compared it with the ML method. Wang et al. [16] evaluated the performance of Bayes, ML and MPS estimators in a progressive Type-II censored sample. The aforementioned studies investigated the performance of estimators under the progressive Type-II censoring scheme, and they rely on traditional censoring schemes where the number of units removed at failure times is predetermined. However, the binomial removal presents an alternative approach, modeling the number of units removed from the experiment at each failure time by using a statistical distribution.

In the literature, the effects of binomial removals under the progressive Type-II censoring scheme have been examined in only a limited number of studies, with a notable absence of comprehensive analysis specifically focusing on entropy estimation. In this context, we will focus on the estimators of different types of entropy based on the Kumaraswamy distribution for progressive Type-II censored samples under binomial removals, and evaluate their performance using the ML and Bayes methods. The ML and Bayesian estimators of these entropy measures are obtained under progressive Type-II censoring with binomial removals; in addition, the consistency and asymptotic normality of the maximum likelihood estimators are stated under standard regularity conditions. The performance of the proposed estimators is evaluated through a comprehensive Monte Carlo simulation study covering different parameter settings, sample sizes, observed failure proportions, and removal probabilities. The practical usefulness of the proposed methods is demonstrated using two real datasets from the fields of economics and agricultural hydrology; asymptotic confidence intervals and credible intervals are obtained and discussed.

The remaining sections of this study are structured as follows: In Section 2, the Kumaraswamy distribution is introduced, and the entropy measures associated with this distribution are obtained in closed form. In Section 3, the ML estimators of the entropy measures are derived, and Section 4 presents the corresponding Bayes estimators. Section 5 presents a Monte Carlo simulation study. Section 6 provides two real-world data examples. Finally, Section 7 and Section 8 present the limitations and conclusion of the study.

2. Kumaraswamy Distribution

The Kumaraswamy (KW) distribution is a continuous probability distribution defined on the $(0,1)$ interval, proposed by [17]. The cumulative distribution function (cdf) and pdf of the Kumaraswamy distribution are given by

$$f_{KW}\left(x;a,b\right)=ab{x}^{a-1}{\left(1-x^a\right)}^{b-1}$$

(5)

and

$$F_{KW}\left(x;a,b\right)=1-{\left(1-x^a\right)}^b,$$

(6)

where $0<x<1$ and $a,b>0$. It is an alternative to the beta distribution with closed-form reliability, and hazard functions provide a computational advantage. The pdf is unimodal when $a>1$ and $b>1$, whereas it exhibits a U-shaped behavior when $a<1$ and $b<1$. Furthermore, the pdf increases for $a>1$ and $b\le 1$, decreases for $a\le 1$ and $b>1$, and remains constant when $a=b=1$. Due to the flexibility of its shape, the Kumaraswamy distribution has been extensively utilized in modeling various phenomena constrained with lower and upper bounds, including test scores, atmospheric temperatures, hydrological data, and unemployment rates. The parameters are identifiable: if $F_{KW}(x;a,b)=F_{KW}(x;a^{\prime },b^{\prime })$ for all $x\in (0,1)$, then letting $x\to 0$ gives $b{x}^a\sim b^{\prime }{x}^{a^{\prime }}$, where $a=a^{\prime }$ and consequently $b=b^{\prime }$. The entropy of the KW distribution has previously been examined in the literature by [3] under the complete-sample case. Accordingly, using Equation (1), and noting that $V=1-X^a$ follows a $\mathrm{Beta}(b,1)$ distribution, so that $E[logV]=-1/b$ and $E[logX]=-\{\gamma +\psi (b+1\left)\right\}/a$, the Shannon entropy of the Kumaraswamy distribution is given as

$$\begin{array}{cc}\hfill S_{SH}(a,b)& =-{\int }_0^{1}log\left(f(x;a,b)\right)f(x;a,b)dx\hfill \\ \hfill & =-log\left(ab\right)-(a-1)ab\left(-{\displaystyle \frac{\gamma }{b{a}^2}}-{\displaystyle \frac{\psi \left(b\right)}{b{a}^2}}-{\displaystyle \frac{1}{b^2{a}^2}}\right)+{\displaystyle \frac{b-1}{b}}.\hfill \end{array}$$

(7)

where $\gamma =\underset{n\to \infty }{\lim }\left(-logn+{\displaystyle \sum _{k=1}^n}{\displaystyle \frac{1}{k}}\right)$ and $\psi \left(b\right)={\displaystyle \frac{d}{db}}log\left(\mathsf{\Gamma }\left(b\right)\right)$ denote Euler’s constant and the psi function, respectively [18]. Since $\psi \left(b\right)+1/b=\psi (b+1)$, Equation (7) can be written in the compact form

$$S_{SH}(a,b)=-log\left(ab\right)+{\displaystyle \frac{a-1}{a}}\left\{\gamma +\psi (b+1)\right\}+{\displaystyle \frac{b-1}{b}},$$

(8)

which is finite for all $a,b>0$, so that no parameter restriction is needed for the Shannon entropy. The Renyi, Tsallis, and HC entropies using Equations (2)–(4) are calculated as follows. For these entropy measures, the following common integral is required, which is obtained by the substitution $t=x^a$:

$${\int }_0^1{\left[f(x;a,b)\right]}^{\delta }dx=b^{\delta }{a}^{\delta -1}B\left(\delta \left(1-{\displaystyle \frac{1}{a}}\right)+{\displaystyle \frac{1}{a}},\delta (b-1)+1\right).$$

This integral is finite when both arguments of the beta function are positive, that is,

$$\delta \left(1-{\displaystyle \frac{1}{a}}\right)+{\displaystyle \frac{1}{a}}>0,\delta (b-1)+1>0.$$

Therefore, for $0<\delta <1$, the integral is finite for all $a,b>0$, whereas, for $\delta >1$, the following conditions are required:

$$a>1-{\displaystyle \frac{1}{\delta }},b>1-{\displaystyle \frac{1}{\delta }}.$$

Throughout, the corresponding admissible parameter region is denoted by

$${\mathcal{A}}_{\delta }=\left\{\begin{array}{cc}{(0,\infty )}^2,\hfill & 0<\delta \le 1,\hfill \\ \left\{(a,b):a>1-\frac{1}{\delta },b>1-\frac{1}{\delta }\right\},\hfill & \delta >1.\hfill \end{array}\right.$$

(9)

$$\begin{array}{cc}\hfill R_{\delta }(a,b)& ={\displaystyle \frac{1}{1-\delta }}log\left\{{\int }_0^1{\left[f(x;a,b)\right]}^{\delta }dx\right\}\hfill \\ \hfill & ={\displaystyle \frac{\delta log\left(b\right)+(\delta -1)log\left(a\right)+log\left[B\left(\delta \left(1-\frac{1}{a}\right)+\frac{1}{a},\delta (b-1)+1\right)\right]}{1-\delta }},\delta \ne 1\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill T_{\delta }(a,b)& ={\displaystyle \frac{1}{\delta -1}}\left\{1-{\int }_0^1{\left[f(x;a,b)\right]}^{\delta }dx\right\}\hfill \\ \hfill & ={\displaystyle \frac{1-b^{\delta }{a}^{\delta -1}B\left(\delta \left(1-\frac{1}{a}\right)+\frac{1}{a},\delta (b-1)+1\right)}{\delta -1}},\delta \ne 1\hfill \end{array}$$

(11)

and

$$H{C}_{\delta }(a,b)={\displaystyle \frac{b^{\delta }{a}^{\delta -1}B\left(\delta \left(1-\frac{1}{a}\right)+\frac{1}{a},\delta (b-1)+1\right)-1}{2^{1-\delta }-1}},\delta \ne 1$$

(12)

where $B(.,.)$ represents the beta function.

3. Inference Procedures of Different Entropies

Procedure of Maximum Likelihood

Suppose that $X_{1:m:n},X_{2:m:n},...,X_{m:m:n}$ represent a progressive Type-II censored sample observed from the Kumaraswamy distribution. Given the censoring scheme $\mathbf{R}=(R_1,\dots ,R_m)$, the likelihood is constructed conditionally on $\mathbf{R}$; consequently, the binomial removal probabilities associated with $R_j\sim \mathrm{Binom}(·,p)$ are independent of a and b, since they depend only on p. Thus, under progressive Type-II censoring, the log-likelihood function is given by

$$\begin{array}{cc}\hfill \mathcal{l}(a,b|\mathbf{x},\mathbf{R})& =C\left(\mathbf{R}\right)+mlog\left(ab\right)+\left(a-1\right){\displaystyle \sum _{i=1}^m}log\left(x_{i:m:n}\right)+\left(b-1\right){\displaystyle \sum _{i=1}^m}log\left(1-{x^a}_{i:m:n}\right)+\hfill \\ \hfill & {\displaystyle \sum _{i=1}^m}b{R}_{i}log\left(1-{x^a}_{i:m:n}\right)\hfill \end{array}$$

(13)

where $C\left(\mathbf{R}\right)$ denotes the terms that are constant with respect to a and b. The likelihood equations related to the parameters a and b are given as follows:

$${\displaystyle \frac{\partial \mathcal{l}(a,b|\mathbf{x},\mathbf{R})}{\partial a}}={\displaystyle \frac{m}{a}}+{\displaystyle \sum _{i=1}^m}log\left(x_{i:m:n}\right)-{\displaystyle \sum _{i=1}^m}\left[b(R_i+1)-1\right]{\displaystyle \frac{x_{i:m:n}^{a}log\left(x_{i:m:n}\right)}{1-x_{i:m:n}^a}}=0.$$

(14)

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial \mathcal{l}(a,b\mid \mathbf{x},\mathbf{R})}{\partial b}}& ={\displaystyle \frac{m}{b}}+{\displaystyle \sum _{i=1}^m}(R_i+1)log\left(1-x_{i:m:n}^a\right)=0,\hfill \\ \hfill \widehat{b}\left(a\right)& =-{\displaystyle \frac{m}{{\displaystyle {\displaystyle \sum _{i=1}^m}(R_i+1)log\left(1-x_{i:m:n}^a\right)}}}.\hfill \end{array}$$

(15)

As shown above, for a given value of a, $\widehat{b}\left(a\right)$ can be obtained explicitly. However, since $\widehat{a}$ does not have a closed-form solution, in the numerical computations of this study, the maximum likelihood estimators (MLEs) of a and b are obtained jointly by maximizing the log-likelihood function using the L-BFGS-B algorithm implemented through the optim() function in R, subject to the positivity constraints $a>0$ and $b>0$ [19].

Remark 1.

Substituting $\widehat{b}\left(a\right)$ into Equation (13) yields the profile log-likelihood

$$\begin{array}{cc}\hfill {\mathcal{l}}_p\left(a\right)& =c-mlogT\left(a\right)+mloga+a{\displaystyle \sum _{i=1}^m}log{x}_{i:m:n}-{\displaystyle \sum _{i=1}^m}log\left(1-x_{i:m:n}^a\right),\hfill \\ \hfill T\left(a\right)& =-{\displaystyle \sum _{i=1}^m}(R_i+1)log\left(1-x_{i:m:n}^a\right)>0,\hfill \end{array}$$

(16)

where c does not depend on a. This representation shows that the likelihood can alternatively be reduced to a one-dimensional profile likelihood problem in a, with $\widehat{b}\left(a\right)$ subsequently obtained from Equation (15).

By virtue of the invariance property of MLE, the estimators of entropy measures are easily obtained. Accordingly, the estimators of the Shannon, Renyi, Tsallis, and HC entropy measures given in Equations (7)–(12) are derived as ${\widehat{S}}_{SH}(\widehat{a},\widehat{b})$, ${\widehat{R}}_{\delta }\left(\widehat{a},\widehat{b}\right)$, ${\widehat{T}}_{\delta }(\widehat{a},\widehat{b})$ and ${\widehat{HC}}_{\delta }\left(\widehat{a},\widehat{b}\right)$.

Under the standard regularity conditions, the MLEs $(\widehat{a},\widehat{b})$ are consistent and asymptotically normal, that is,

$$\sqrt{m}\left[(\widehat{a},\widehat{b})-(a,b)\right]\stackrel{d}{\to }{\mathcal{N}}_2\left(\mathbf{0},I^{-1}(a,b)\right),$$

where $I(a,b)$ denotes the per-observation Fisher information matrix obtained from the second derivatives of Equations (14) and (15), defined as

$$I(a,b)=-{\displaystyle \frac{1}{m}}\mathbb{E}\left[\begin{array}{cc}{\displaystyle \frac{{\partial }^2\mathcal{l}(a,b\mid \mathbf{x},\mathbf{R})}{\partial a^2}}& {\displaystyle \frac{{\partial }^2\mathcal{l}(a,b\mid \mathbf{x},\mathbf{R})}{\partial a\partial b}}\\ {\displaystyle \frac{{\partial }^2\mathcal{l}(a,b\mid \mathbf{x},\mathbf{R})}{\partial a\partial b}}& {\displaystyle \frac{{\partial }^2\mathcal{l}(a,b\mid \mathbf{x},\mathbf{R})}{\partial b^2}}\end{array}\right].$$

For computational purposes, define

$$c_i=b(R_i+1)-1.$$

The total observed information matrix is defined as

$$J_m(a,b)=-{\nabla }^2\mathcal{l}(a,b\mid \mathbf{x},\mathbf{R}).$$

The elements of the observed information matrix are given by

$${\left(J_m(a,b)\right)}_{aa}={\displaystyle \frac{m}{a^2}}+{\displaystyle \sum _{i=1}^m}{c}_i{\displaystyle \frac{x_{i:m:n}^a{\left[log\left(x_{i:m:n}\right)\right]}^2}{{\left(1-x_{i:m:n}^a\right)}^2}}.$$

$${\left(J_m(a,b)\right)}_{ab}={\left(J_m(a,b)\right)}_{ba}={\displaystyle \sum _{i=1}^m}(R_i+1){\displaystyle \frac{x_{i:m:n}^{a}log\left(x_{i:m:n}\right)}{1-x_{i:m:n}^a}}.$$

$${\left(J_m(a,b)\right)}_{bb}={\displaystyle \frac{m}{b^2}}.$$

Accordingly, the observed estimate of the per-observation information matrix is given by

$$\widehat{I}(\widehat{a},\widehat{b})={\displaystyle \frac{1}{m}}{J}_m(\widehat{a},\widehat{b}).$$

Since $c_i=b(R_i+1)-1$ may be negative when $b<1$, positive definiteness of $J_m$ is not automatic; it was verified numerically at the MLE in all cases reported in this study. Moreover, this result, together with the delta method, establishes the asymptotic normality of the entropy estimators $\left({\widehat{S}}_{\mathrm{SH}},{\widehat{R}}_{\delta },{\widehat{T}}_{\delta },{\widehat{HC}}_{\delta }\right)$. Specifically, for any entropy function $g(a,b)\in \left\{S_{\mathrm{SH}},R_{\delta },T_{\delta },H{C}_{\delta }\right\}$, which is continuously differentiable on ${\mathcal{A}}_{\delta }$ with $\nabla g(a,b)\ne \mathbf{0}$,

$$\sqrt{m}\left[g(\widehat{a},\widehat{b})-g(a,b)\right]\stackrel{d}{\to }N\left(0,\nabla g{(a,b)}^{\top }{I}^{-1}(a,b)\nabla g(a,b)\right),$$

Accordingly, an approximate $100(1-\alpha )\%$ asymptotic confidence interval (ACI) for the entropy estimator is given by

$$g(\widehat{a},\widehat{b})\pm z_{1-\alpha /2}\sqrt{{\displaystyle \frac{1}{m}}\nabla g{(\widehat{a},\widehat{b})}^T{\widehat{I}}^{-1}(\widehat{a},\widehat{b})\nabla g(\widehat{a},\widehat{b})}$$

4. Procedure of Bayes

Among estimation methods, the Bayesian estimators (BEs) provide more efficient and stable estimates by incorporating prior information into the estimation process. Let $X_{1:m:n},X_{2:m:n},...,X_{m:m:n}$ be a progressively Type-II censored sample from the Kumaraswamy distribution with parameters a and b. The BEs of the a and b parameters of the Kumaraswamy distribution require the specification of prior distributions. Thus, the prior distributions for the a and b parameters are assumed to follow the independent Gamma distributions, under the shape-rate parameterization given as

$$\pi \left(a\right)\propto a^{d_1-1}{e}^{-a{t}_1}$$

(17)

and

$$\pi \left(b\right)\propto b^{d_2-1}{e}^{-b{t}_2}$$

(18)

where $d_1,t_1$ and $d_2,t_2$ denote, respectively, the shape and rate parameters of the Gamma priors assigned to a and b. Since the Rényi, Tsallis, and HC entropies are finite only on the admissible region ${\mathcal{A}}_{\delta }$ defined in Equation (9), the joint prior is understood to be truncated to ${\mathcal{A}}_{\delta }\cap \mathcal{K}$, where $\mathcal{K}$ is a compact rectangle specified in advance, independently of the data, and chosen large enough to contain the posterior mode in all cases considered. For $\delta \le 1$, this reduces to a truncation of the Gamma priors to a compact rectangle only. On this support, all four entropy functions are bounded, which guarantees that the posterior expectations and the exponential moments used below exist. Because the Tierney–Kadane approximation is local at the posterior mode, and the mode was interior to ${\mathcal{A}}_{\delta }\cap \mathcal{K}$ in every case considered, this restriction does not affect the numerical results reported in Section 5 and Section 6. Since the Shannon entropy is finite for all $a,b>0$, no entropy-specific admissibility restriction is required for this measure. Although the Shannon entropy function itself does not depend on $\delta$, its posterior expectation may depend indirectly on $\delta$ through the truncated support ${\mathcal{A}}_{\delta }\cap \mathcal{K}$. Consequently, the TK-based Bayes estimate of the Shannon entropy is independent of $\delta$. The small differences among the corresponding MCMC estimates in Table 1 and Table 2 can be attributed to Monte Carlo variability, as a separate chain is run for each value of $\delta$. The distributions of the joint prior and posteriors for the parameters a and b are as follows:

$$\pi (a,b)=\pi \left(a\right)\pi \left(b\right)\propto a^{d_1-1}{b}^{d_2-1}{e}^{-(a{t}_1+b{t}_2)}.$$

$$\begin{array}{cc}\hfill \pi (a,b\mid \mathbf{x})& ={\displaystyle \frac{f(\mathbf{x}\mid a,b)\pi (a,b)}{f\left(\mathbf{x}\right)}}\hfill \\ \hfill & ={\displaystyle \frac{\mathsf{\Lambda }(\mathbf{x};a,b)\pi (a,b)}{{\displaystyle {\int \int }_{{\mathcal{A}}_{\delta }\cap \mathcal{K}}\mathsf{\Lambda }(\mathbf{x};a,b)\pi (a,b)dadb}}},(a,b)\in {\mathcal{A}}_{\delta }\cap \mathcal{K}.\hfill \end{array}$$

(19)

where $\mathsf{\Lambda }(\mathbf{x};a,b)={\left(ab\right)}^m{\prod }_{i=1}^m{x}_{i:m:n}^{a-1}{\left(1-x_{i:m:n}^a\right)}^{b(R_i+1)-1}.$ In this case, under the squared error loss function, the BEs of $\psi (a,b)$, which is a function of the parameters a and b, are given as follows. For notational convenience, let $\mathcal{l}(a,b\mid \mathbf{x},\mathbf{R})=log\mathsf{\Lambda }(\mathbf{x};a,b)$ and $\rho (a,b)=log\pi (a,b).$

$${\widehat{\psi }}_B=E\left[\psi (a,b)\mid \mathbf{x}\right]={\displaystyle \frac{{\displaystyle {\int \int }_{{\mathcal{A}}_{\delta }\cap \mathcal{K}}\psi (a,b)exp\left\{\mathcal{l}(a,b\mid \mathbf{x},\mathbf{R})+\rho (a,b)\right\}dadb}}{{\displaystyle {\int \int }_{{\mathcal{A}}_{\delta }\cap \mathcal{K}}exp\left\{\mathcal{l}(a,b\mid \mathbf{x},\mathbf{R})+\rho (a,b)\right\}dadb}}}.$$

(20)

where $\mathcal{l}(a,b\mid \mathbf{x},\mathbf{R})$ denotes the log-likelihood function based on the observed progressively Type-II censored sample $\mathbf{x}=(x_{1:m:n},x_{2:m:n},\dots ,x_{m:m:n})$, and $\psi (a,b)$ denotes the entropy function of interest. The entropy functions presented in Equations (7)–(12) depend on the parameters a and b. Consequently, substituting these functions into the $\psi \left(a,b|x\right)$ function yields the relevant entropy estimation. To obtain the BEs for the expression given in Equation (20) in a closed form is difficult. Hence, the Tierney–Kadane (TK) approximation can be utilized for obtaining BEs.

Table 1. Entropy estimates, SEs, and 95% ACIs and CrIs for Dataset I under different censoring scenarios.

Scenario	m	$\mathit{\delta }$	Entropy	MLE					MCMC					TK-Based BE
				Estimate	SE	95% ACI			Estimate	SE	95% CrI			Estimate
						Lower	Upper				Lower	Upper
Scenario I	31	0.5	$\mathrm{SH}$	−0.26323	0.11363	−0.48594	−0.04052		−0.25834	0.10208	−0.47326	−0.07890		−0.25850
			$R_{\delta }$	−0.17719	0.08720	−0.34809	−0.00629		−0.17638	0.07822	−0.34923	−0.04666		−0.17628
			$T_{\delta }$	−0.16957	0.07980	−0.32598	−0.01316		−0.16743	0.07104	−0.32044	−0.04612		−0.16790
			$H{C}_{\delta }$	−0.20469	0.09633	−0.39349	−0.01589		−0.20211	0.08575	−0.38680	−0.05567		−0.20267
		3.5	$\mathrm{SH}$	−0.26323	0.11363	−0.48594	−0.04052		−0.26186	0.10775	−0.48220	−0.07953		−0.25850
			$R_{\delta }$	−0.42089	0.14436	−0.70383	−0.13795		−0.41268	0.13932	−0.68196	−0.15881		−0.40821
			$T_{\delta }$	−0.74560	0.41345	−1.55594	0.06475		−0.79307	0.43237	−1.80034	−0.19495		−0.75207
			$H{C}_{\delta }$	−2.26426	1.25558	−4.72515	0.19662		−2.40843	1.31303	−5.46734	−0.59204		−2.28390
Scenario II	31	0.5	$\mathrm{SH}$	−0.29350	0.08831	−0.46658	−0.12041		−0.29112	0.08432	−0.46324	−0.13395		−0.29070
			$R_{\delta }$	−0.19988	0.06832	−0.33378	−0.06598		−0.19988	0.06508	−0.33627	−0.08355		−0.19959
			$T_{\delta }$	−0.19022	0.06182	−0.31138	−0.06906		−0.18926	0.05863	−0.30952	−0.08183		−0.18916
			$H{C}_{\delta }$	−0.22961	0.07462	−0.37587	−0.08336		−0.22846	0.07077	−0.37362	−0.09877		−0.22834
		3.5	$\mathrm{SH}$	−0.29350	0.08831	−0.46658	−0.12041		−0.28912	0.08580	−0.45930	−0.12654		−0.29070
			$R_{\delta }$	−0.46020	0.11086	−0.67748	−0.24292		−0.45091	0.10934	−0.65881	−0.23402		−0.45286
			$T_{\delta }$	−0.86392	0.35029	−1.55047	−0.17736		−0.88172	0.35486	−1.67660	−0.31804		−0.87860
			$H{C}_{\delta }$	−2.62358	1.06377	−4.70854	−0.53862		−2.67764	1.07766	−5.09158	−0.96584		−2.66818
Scenario III	31	0.5	$\mathrm{SH}$	−0.24074	0.08011	−0.39776	−0.08373		−0.23609	0.07778	−0.39328	−0.09306		−0.23774
			$R_{\delta }$	−0.16088	0.06076	−0.27997	−0.04179		−0.15921	0.05874	−0.28265	−0.05608		−0.16043
			$T_{\delta }$	−0.15458	0.05607	−0.26447	−0.04469		−0.15224	0.05401	−0.26359	−0.05530		−0.15351
			$H{C}_{\delta }$	−0.18659	0.06768	−0.31924	−0.05395		−0.18377	0.06519	−0.31818	−0.06675		−0.18530
		3.5	$\mathrm{SH}$	−0.24074	0.08011	−0.39776	−0.08373		−0.23946	0.07729	−0.39390	−0.09548		−0.23774
			$R_{\delta }$	−0.39047	0.10357	−0.59346	−0.18747		−0.38487	0.10143	−0.57766	−0.18533		−0.38240
			$T_{\delta }$	−0.66170	0.27491	−1.20051	−0.12289		−0.68096	0.27604	−1.29532	−0.23573		−0.66968
			$H{C}_{\delta }$	−2.00949	0.83485	−3.64577	−0.37321		−2.06797	0.83829	−3.93367	−0.71588		−2.03370
Complete	61	0.5	$\mathrm{SH}$	−0.21732	0.05993	−0.33477	−0.09987		−0.21628	0.05823	−0.33379	−0.10797		−0.21486
			$R_{\delta }$	−0.14267	0.04420	−0.22929	−0.05605		−0.14300	0.04289	−0.23160	−0.06585		−0.14199
			$T_{\delta }$	−0.13770	0.04115	−0.21836	−0.05705		−0.13758	0.03982	−0.21870	−0.06477		−0.13667
			$H{C}_{\delta }$	−0.16622	0.04968	−0.26358	−0.06886		−0.16607	0.04807	−0.26399	−0.07819		−0.16497
		3.5	$\mathrm{SH}$	−0.21732	0.05993	−0.33477	−0.09987		−0.21461	0.05876	−0.32803	−0.10104		−0.21486
			$R_{\delta }$	−0.36116	0.07981	−0.51758	−0.20474		−0.35485	0.07943	−0.50246	−0.19439		−0.35512
			$T_{\delta }$	−0.58670	0.19686	−0.97255	−0.20086		−0.59045	0.19673	−1.00477	−0.25030		−0.58973
			$H{C}_{\delta }$	−1.78173	0.59784	−2.95348	−0.60998		−1.79311	0.59745	−3.05132	−0.76013		−1.79090

Table 2. Entropy estimates, SEs, and 95% ACIs and CrIs for Dataset II under different censoring scenarios.

Scenario	m	$\mathit{\delta }$	Entropy	MLE					MCMC					TK-Based BE
				Estimate	SE	95% ACI			Estimate	SE	95% CrI			Estimate
						Lower	Upper				Lower	Upper
Scenario I	30	0.5	$\mathrm{SH}$	−0.77165	0.13259	−1.03152	−0.51178		−0.74995	0.12865	−0.99342	−0.47783		−0.75109
			$R_{\delta }$	−0.63434	0.13080	−0.89071	−0.37797		−0.61173	0.12568	−0.85280	−0.35327		−0.61311
			$T_{\delta }$	−0.54359	0.09525	−0.73028	−0.35689		−0.52411	0.09331	−0.69429	−0.32383		−0.52530
			$H{C}_{\delta }$	−0.65617	0.11498	−0.88152	−0.43081		−0.63265	0.11264	−0.83808	−0.39089		−0.63410
		3.5	$\mathrm{SH}$	−0.77165	0.13259	−1.03152	−0.51178		−0.74551	0.13585	−0.98866	−0.46898		−0.75109
			$R_{\delta }$	−0.96610	0.13484	−1.23040	−0.70181		−0.94482	0.13739	−1.18927	−0.65751		−0.94946
			$T_{\delta }$	−4.07708	1.50928	−7.03521	−1.11896		−4.09273	1.49668	−7.42155	−1.66985		−4.06770
			$H{C}_{\delta }$	−12.38146	4.58343	−21.36482	−3.39809		−12.42898	4.54518	−22.53807	−5.07106		−12.35297
Scenario II	30	0.5	$\mathrm{SH}$	−0.83241	0.11082	−1.04961	−0.61520		−0.81205	0.11211	−1.01673	−0.57922		−0.81655
			$R_{\delta }$	−0.69275	0.10974	−0.90783	−0.47767		−0.67059	0.11015	−0.87461	−0.44527		−0.67524
			$T_{\delta }$	−0.58551	0.07761	−0.73762	−0.43339		−0.56757	0.07939	−0.70845	−0.39919		−0.57093
			$H{C}_{\delta }$	−0.70677	0.09369	−0.89039	−0.52315		−0.68511	0.09583	−0.85518	−0.48186		−0.68918
		3.5	$\mathrm{SH}$	−0.83241	0.11082	−1.04961	−0.61520		−0.81476	0.11280	−1.01700	−0.57989		−0.81655
			$R_{\delta }$	−1.02887	0.11261	−1.24959	−0.80816		−1.01527	0.11440	−1.21983	−0.77868		−1.01669
			$T_{\delta }$	−4.83774	1.47457	−7.72784	−1.94763		−4.86878	1.47917	−8.04252	−2.40218		−4.85600
			$H{C}_{\delta }$	−14.69145	4.47804	−23.46825	−5.91465		−14.78573	4.49202	−24.42388	−7.29504		−14.74690
Scenario III	30	0.5	$\mathrm{SH}$	−0.81215	0.11188	−1.03143	−0.59286		−0.79802	0.11061	−1.00336	−0.56630		−0.79451
			$R_{\delta }$	−0.67507	0.11010	−0.89087	−0.45928		−0.65986	0.10785	−0.86438	−0.43667		−0.65643
			$T_{\delta }$	−0.57295	0.07856	−0.72693	−0.41897		−0.55995	0.07812	−0.70183	−0.39229		−0.55742
			$H{C}_{\delta }$	−0.69161	0.09483	−0.87748	−0.50575		−0.67592	0.09430	−0.84718	−0.47353		−0.67287
		3.5	$\mathrm{SH}$	−0.81215	0.11188	−1.03143	−0.59286		−0.79566	0.11371	−0.99952	−0.55366		−0.79451
			$R_{\delta }$	−1.00688	0.11452	−1.23134	−0.78242		−0.99298	0.11602	−1.19957	−0.74233		−0.99175
			$T_{\delta }$	−4.55758	1.41939	−7.33954	−1.77562		−4.58644	1.40004	−7.62557	−2.15877		−4.54312
			$H{C}_{\delta }$	−13.84065	4.31047	−22.28902	−5.39228		−13.92831	4.25169	−23.15767	−6.55585		−13.79675
Complete	40	0.5	$\mathrm{SH}$	−0.84586	0.09634	−1.03469	−0.65704		−0.82301	0.09863	−0.99839	−0.62141		−0.83244
			$R_{\delta }$	−0.70838	0.09541	−0.89539	−0.52137		−0.68386	0.09714	−0.85782	−0.48764		−0.69360
			$T_{\delta }$	−0.59652	0.06696	−0.72775	−0.46529		−0.57752	0.06960	−0.69756	−0.43275		−0.58453
			$H{C}_{\delta }$	−0.72006	0.08082	−0.87847	−0.56165		−0.69713	0.08401	−0.84203	−0.52237		−0.70559
		3.5	$\mathrm{SH}$	−0.84586	0.09634	−1.03469	−0.65704		−0.83328	0.09531	−1.00813	−0.63668		−0.83244
			$R_{\delta }$	−1.04133	0.09848	−1.23435	−0.84832		−1.03099	0.09721	−1.20875	−0.83451		−1.03006
			$T_{\delta }$	−5.00345	1.33031	−7.61082	−2.39609		−5.02052	1.30029	−7.81193	−2.82193		−4.99045
			$H{C}_{\delta }$	−15.19471	4.03995	−23.11287	−7.27654		−15.24654	3.94877	−23.72360	−8.56976		−15.15520

Tierney and Kadane Method

The TK approximation is a method introduced by [20], and it is also known as the Laplace approach. This method allows the approximate evaluation of complex integrals used in the computation of BEs. The TK approximation can be formulated as

$$\kappa (a,b)={\displaystyle \frac{1}{m}}\left\{\rho (a,b)+\mathcal{l}(a,b\mid \mathbf{x},\mathbf{R})\right\},$$

where $\mathcal{l}(a,b\mid \mathbf{x},\mathbf{R})$ denotes the log-likelihood function based on the progressively Type-II censored sample $\mathbf{x}=(x_{1:m:n},x_{2:m:n},\dots ,x_{m:m:n})$, and $\rho (a,b)=log\pi (a,b)$ denotes the logarithm of the joint prior density.

However, the entropy functions considered in this study may take negative values for some parameter combinations. Therefore, following Tierney et al.  [21], this issue is handled through the posterior moment-generating function (MGF). For an entropy function $\psi (a,b)$, define

$$M_{\psi }\left(t\right)=E\left[exp\left\{t\psi \right(a,b\left)\right\}\mid \mathbf{x}\right],t\in \mathbb{R},$$

The positivity of $exp\left\{t\psi \right(a,b\left)\right\}$ alone does not guarantee the existence of the corresponding posterior exponential moment; the TK approximation and the central-difference estimator given below additionally require $M_{\psi }\left(t\right)<\infty$ for $\left|t\right|\le \epsilon$. This requirement is not automatic: since ${\int }_0^1{\left[f(x;a,b)\right]}^{\delta }dx$ is of order $b^{(\delta -1)/a}$ as $b\to \infty$, the Tsallis and HC entropies are unbounded on the region $a<\delta -1$, on which the Gamma priors would give exponential posterior tails that are too light to compensate. As stated above, the priors are truncated to ${\mathcal{A}}_{\delta }\cap \mathcal{K}$, on which every entropy function considered is bounded. Consequently, $M_{\psi }\left(t\right)<\infty$ for all $t\in \mathbb{R}$, and the tilted objective function

$${\kappa }_t^{*}(a,b)={\displaystyle \frac{1}{m}}\left\{t\psi (a,b)+\rho (a,b)+\mathcal{l}(a,b\mid \mathbf{x},\mathbf{R})\right\}$$

attains an interior maximum for $\left|t\right|\le \epsilon$, so that the TK approximation is well defined without any additional assumption on the posterior mass outside the admissible region.

Under these conditions, the TK approximation can be applied to $M_{\psi }\left(t\right)$ as

$${\widehat{M}}_{\psi ,TK}\left(t\right)={\left({\displaystyle \frac{det{\Sigma }_t^{*}}{det\Sigma }}\right)}^{1/2}exp\left[m\left\{{\kappa }_t^{*}({\widehat{a}}_t,{\widehat{b}}_t)-\kappa (\widehat{a},\widehat{b})\right\}\right],$$

(21)

where $(\widehat{a},\widehat{b})$ maximizes $\kappa (a,b)$, $({\widehat{a}}_t,{\widehat{b}}_t)$ maximizes ${\kappa }_t^{*}(a,b)$, and $\Sigma$ and ${\Sigma }_t^{*}$ denote the corresponding inverse negative Hessian matrices evaluated at $(\widehat{a},\widehat{b})$ and $({\widehat{a}}_t,{\widehat{b}}_t)$, respectively. The numerical maximizations of $\kappa (a,b)$ and ${\kappa }_t^{*}(a,b)$ are terminated when the relative change in the corresponding objective function is less than ${10}^{-8}$. Finally, because

$$E[\psi (a,b)\mid \mathbf{x}]={\left.{\displaystyle \frac{d}{dt}}{M}_{\psi }\left(t\right)\right|}_{t=0},$$

(22)

the Bayes estimator is obtained using the central-difference approximation

$${\widehat{\psi }}_{TK}={\displaystyle \frac{{\widehat{M}}_{\psi ,TK}\left(\epsilon \right)-{\widehat{M}}_{\psi ,TK}(-\epsilon )}{2\epsilon }},\epsilon ={10}^{-4}.$$

(23)

The above procedure is applied separately by replacing $\psi (a,b)$ with $S_{\mathrm{SH}}(a,b)$, $R_{\delta }(a,b)$, $T_{\delta }(a,b)$, and $H{C}_{\delta }(a,b)$, respectively. Thus, the corresponding BEs based on the TK approximation, ${\widehat{S}}_{\mathrm{SH}}(\widehat{a},\widehat{b})$, ${\widehat{R}}_{\delta }(\widehat{a},\widehat{b})$, ${\widehat{T}}_{\delta }(\widehat{a},\widehat{b})$, and ${\widehat{HC}}_{\delta }(\widehat{a},\widehat{b})$, are obtained for the Shannon, Rényi, Tsallis, and HC entropy measures, respectively. Under the usual Laplace regularity conditions, including a unique interior posterior mode, sufficient smoothness of the tilted log-posterior, and positive definiteness of the corresponding observed information matrices, the TK approximation to $M_{\psi }\left(t\right)$ has relative error of order $O\left(m^{-2}\right)$ as $m\to \infty$ for each fixed t. Since the tilted mode $({\widehat{a}}_t,{\widehat{b}}_t)$ varies smoothly with t, this relative error is twice differentiable at $t=0$, so that the induced error in the resulting approximation to the posterior expectation is of the same order. The central-difference approximation introduces an additional truncation error of order $O\left({\epsilon }^2\right)$. The central-difference step $\epsilon ={10}^{-4}$ in Equation (23) should not be confused with the relative convergence tolerance of ${10}^{-8}$ used for the posterior-mode optimizations of $\kappa (a,b)$ and ${\kappa }_t^{*}(a,b)$; these are two distinct numerical settings. In the real-data analysis, the Markov chain Monte Carlo (MCMC)-based posterior estimates and their corresponding 95% credible intervals are obtained from the retained posterior draws, whereas the TK-based Bayes estimates are reported as Laplace-based numerical approximations for comparison. The MCMC sampling procedure is described in Appendix A.

5. Simulation Study

This section presents a comprehensive Monte Carlo simulation study to evaluate the performance of the MLEs and BEs obtained using the TK approximation for four different entropy measures based on the Kumaraswamy distribution in terms of bias and mean squared error (MSE). The simulation study is conducted under five different parameter settings, six sample sizes, three observed failure proportions, and two removal probabilities. The $R_j$’s are assumed to follow a binomial distribution. For clarity, all simulation settings are summarized in Table 3.

Table 3. Simulation settings used in the Monte Carlo study.

Simulation Setting	Values
True parameter scenarios	$S_1:(a,b,\delta )=(0.5,3,0.5)$
	$S_2:(a,b,\delta )=(3,1.5,0.5)$
	$S_3:(a,b,\delta )=(1.5,2.5,2.5)$
	$S_4:(a,b,\delta )=(2,5,3)$
	$S_5:(a,b,\delta )=(2,3,0.25)$
Prior distribution parameters	$d_1=d_2=t_1=t_2=0.01$
Sample sizes (n)	$50,100,250,500,750,1000$
Number of failures (m)	$50\%,70\%,90\%$ of n
Removal probabilities (p)	$0.25,0.75$
Censoring scheme ($R_j$)	$R_j\sim Binom\left({\displaystyle n-m-\sum _{i=1}^{j-1}{R}_i,p}\right)$
Monte Carlo replications	5000
Performance measures	Bias and MSE

In the simulation study, the prior distribution parameters were set to $d_1=d_2=t_1=t_2=0.01$. However, in order to assess the sensitivity of the TK-based Bayesian entropy estimators to this prior specification, two additional sets of prior distribution parameters were considered for scenarios $S_1$ and $S_2$ at sample sizes $n=50$, 250, and 1000. These parameter sets were specified as $(d_1,d_2,t_1,t_2)=(0.10,0.10,0.10,0.10)$ and $(d_1,d_2,t_1,t_2)=(1,1,1,1)$, respectively. The corresponding bias and MSE results are presented in Appendix B. The true parameter values $(a,b)$ and the corresponding $\delta$ values used in each scenario $S_1$–$S_5$ are chosen to satisfy the finiteness conditions for the Rényi, Tsallis, and HC entropies given in Section 2. In particular, $\delta <1$ is used in $S_1$, $S_2$, and $S_5$, for which the entropy functions are finite for all $a,b>0$, whereas $\delta >1$ is used in $S_3$ and $S_4$, and the corresponding $(a,b)$ values are selected to satisfy $a>1-1/\delta$ and $b>1-1/\delta$.

The computational steps used to obtain the MLEs and the TK-based BEs are summarized as follows:

1.

The removal numbers $R_j$ are generated according to a binomial distribution, and progressive Type-II censored samples are then obtained from the Kumaraswamy distribution based on these removal numbers.

2.

For each censored sample, although $\widehat{b}\left(a\right)$ can be obtained in closed form from Equation (15), the maximum likelihood estimates (MLEs) of a and b are obtained jointly by numerically maximizing the log-likelihood function using the L-BFGS-B algorithm implemented through the optim() function in R, subject to the positivity constraints $a>0$ and $b>0$. Subsequently, using the invariance property of the MLEs, the MLEs of the Shannon, Rényi, Tsallis, and Havrda–Charvát entropy measures are computed by substituting the estimated parameter values into the corresponding entropy functions.

3.

For Bayesian estimation, mutually independent Gamma prior distributions are assigned to the parameters a and b. The prior distribution parameters are taken as

$$d_1=d_2=t_1=t_2=0.01.$$

The posterior distribution is obtained by combining the likelihood function with the prior distributions. The posterior mode and the corresponding Hessian matrix are then obtained subject to the truncation of the prior to ${\mathcal{A}}_{\delta }\cap \mathcal{K}$ described in Section 4, using the L-BFGS-B algorithm implemented through the optim() function in R [19].

4.

The Bayesian estimates of the four entropy measures are computed using the Tierney–Kadane (TK) approximation. The posterior and tilted posterior modes, together with their corresponding Hessian matrices, are obtained numerically using the L-BFGS-B algorithm. The relative convergence tolerance for the posterior-mode optimizations is set to ${10}^{-8}$, while the central-difference step used to approximate the posterior expectations is set to $\epsilon ={10}^{-4}$.

5.

The above steps are repeated for $N=5000$ Monte Carlo replications. For each entropy measure H, the performance of the estimators is assessed using bias and mean squared error (MSE), calculated as

$$\begin{array}{cc}\hfill Bias\left(H\right)& ={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}\left({\widehat{H}}_i-H\right),\hfill \\ \hfill MSE\left(H\right)& ={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\left({\widehat{H}}_i-H\right)}^2.\hfill \end{array}$$

where $H\in \{S_{\mathrm{SH}},R_{\delta },T_{\delta },H{C}_{\delta }\}$ denotes the true value of the corresponding entropy measure, whereas ${\widehat{H}}_i$ denotes its estimate obtained in the ith Monte Carlo replication.

Table 4 summarizes the MSE values at three representative sample sizes, while the detailed results for all sample sizes, failure proportions, and removal probabilities are reported in Appendix C (Table A2, Table A3, Table A4, Table A5 and Table A6). These results provide the following findings regarding the entropy estimates obtained under the progressive Type-II censoring scheme with binomial removals. For all entropy measures and both estimation methods, the bias and MSE values decrease noticeably as the sample size (n) and the number of observed failures (m) increase, while the other model parameters are held fixed. This behavior is consistent with the expected improvement in estimation accuracy as the amount of available information increases. For $S_1$, at the small sample size ($n=50$), the TK-based BEs yield lower MSE values than the MLEs for the R’enyi, Tsallis, and HC entropies under both $p=0.25$ and $p=0.75$, whereas for the Shannon entropy the MLEs generally attain the lower MSE values. As the sample size increases ($n=750$ and $n=1000$), the MSE values obtained from the two estimation methods become almost identical for all four entropy measures. The removal probability p appears to have only a limited effect on the MSE performance of both methods, with relatively small differences between $p=0.25$ and $p=0.75$ and no consistent pattern across the entropy measures. For $S_2$, at small sample sizes, the TK-based BEs generally tend to exhibit lower MSE values than the MLEs for all four entropy measures under both $p=0.25$ and $p=0.75$, while the differences between the two removal probabilities tend to be negligible. As the sample size increases, the MSE values of the MLEs and TK-based BEs become increasingly close for both values of p, suggesting that the removal probability has only a limited effect on their MSE performance. For $S_3$, one of the more noticeable features at small sample sizes is the relatively large MSE values obtained for the Tsallis and, in particular, the HC entropy estimators compared with those for the Shannon and R’enyi entropies. In most, but not all, of the sample-size and removal-probability combinations considered, the TK-based BEs yield lower MSE values than the MLEs; a small number of cells depart from this pattern for the Tsallis entropy, so the advantage of the TK-based BE is not exceptionless in this scenario. As the sample size and the number of observed failures (m) increase, the MSE values obtained from the two estimation methods tend to become closer, whereas changes in the removal probability p do not produce a marked difference in the overall MSE behavior. In the case of $S_4$, at small sample sizes, the TK-based BEs tend to produce lower MSE values than the MLEs under both $p=0.25$ and $p=0.75$, and for the Tsallis and HC entropies this holds in all the combinations considered. Nevertheless, the absolute MSE values, particularly those corresponding to the HC entropy, are considerably higher than those observed in the other scenarios. As the sample size increases, the MSE values decrease for both removal probabilities and the differences between the MLEs and TK-based BEs become smaller, although the absolute MSE values for the HC entropy remain more pronounced than those for the other entropy measures. In the case of $S_5$, at small sample sizes, the TK-based BEs generally yield lower MSE values than the MLEs for all four entropy measures under both $p=0.25$ and $p=0.75$. As the sample size increases, the MSE values of the two estimation methods become increasingly similar for both removal probabilities. Moreover, $S_5$ tends to exhibit comparatively small MSE values among the scenarios considered. Considering the five scenarios together, the TK-based BEs tend to provide somewhat lower MSE values than the MLEs in most small-sample settings under both $p=0.25$ and $p=0.75$. The main exception is the Shannon entropy in $S_1$, for which the MLEs are generally preferable. For $S_1$–$S_5$, the differences between the MSE values of the MLEs and TK-based BEs generally become smaller as the sample size increases, and beyond $n=500$ the two estimation methods yield practically the same MSE values. Overall, the effect of the removal probability p on these patterns appears to be relatively limited across all of the scenarios considered. Table A7 (Appendix D) shows that the TK-based Bayesian estimator generally exhibits modestly higher efficiency than the MLE for parameter settings $S_2$–$S_5$. For $S_1$, however, the two estimators exhibit very similar overall efficiency; the MLE has a marginal advantage for the Shannon entropy, whereas the TK-based BE exhibits modestly higher efficiency for the R’enyi, Tsallis, and HC entropy measures. This suggests that the relative performance of the two estimators is not uniform but depends on the underlying parameter setting and entropy measure.

Table 4. Summary of MSE values for MLEs and TK-based BEs under $S_1$–$S_5$ with $m=0.7\times n$ and $p=0.25,0.75$.

Scenario	n	p	Shannon			Rényi			Tsallis			HC
			MLE	BE		MLE	BE		MLE	BE		MLE	BE
$S_1$	50	0.25	0.06874	0.06893		0.02670	0.02520		0.01112	0.01065		0.01621	0.01551
	50	0.75	0.07454	0.07489		0.02749	0.02606		0.01144	0.01100		0.01666	0.01602
	250	0.25	0.01587	0.01591		0.00669	0.00658		0.00280	0.00275		0.00408	0.00401
	250	0.75	0.01612	0.01617		0.00638	0.00629		0.00269	0.00265		0.00391	0.00386
	1000	0.25	0.00420	0.00421		0.00160	0.00159		0.00068	0.00068		0.00099	0.00099
	1000	0.75	0.00417	0.00418		0.00164	0.00163		0.00070	0.00069		0.00101	0.00101
$S_2$	50	0.25	0.00774	0.00758		0.00483	0.00468		0.00373	0.00355		0.00544	0.00518
	50	0.75	0.00878	0.00857		0.00549	0.00532		0.00422	0.00402		0.00615	0.00586
	250	0.25	0.00160	0.00159		0.00098	0.00098		0.00078	0.00077		0.00114	0.00113
	250	0.75	0.00164	0.00163		0.00100	0.00099		0.00080	0.00079		0.00116	0.00115
	1000	0.25	0.00042	0.00042		0.00026	0.00025		0.00020	0.00020		0.00030	0.00030
	1000	0.75	0.00042	0.00042		0.00025	0.00025		0.00020	0.00020		0.00030	0.00029
$S_3$	50	0.25	0.00671	0.00609		0.01034	0.00931		0.02721	0.02655		0.14651	0.14293
	50	0.75	0.00641	0.00587		0.00994	0.00910		0.02608	0.02617		0.14042	0.14089
	250	0.25	0.00108	0.00106		0.00175	0.00171		0.00384	0.00382		0.02069	0.02059
	250	0.75	0.00104	0.00102		0.00169	0.00165		0.00371	0.00370		0.01999	0.01990
	1000	0.25	0.00025	0.00025		0.00041	0.00041		0.00087	0.00087		0.00471	0.00470
	1000	0.75	0.00025	0.00024		0.00040	0.00040		0.00085	0.00085		0.00458	0.00457
$S_4$	50	0.25	0.00935	0.00841		0.01220	0.01103		0.12595	0.11816		0.89567	0.84025
	50	0.75	0.00929	0.00840		0.01214	0.01101		0.12496	0.11781		0.88861	0.83773
	250	0.25	0.00166	0.00162		0.00222	0.00217		0.01810	0.01793		0.12870	0.12752
	250	0.75	0.00165	0.00161		0.00221	0.00216		0.01789	0.01772		0.12725	0.12603
	1000	0.25	0.00040	0.00040		0.00054	0.00053		0.00412	0.00411		0.02928	0.02921
	1000	0.75	0.00040	0.00039		0.00053	0.00053		0.00409	0.00408		0.02910	0.02903
$S_5$	50	0.25	0.00700	0.00627		0.00210	0.00195		0.00175	0.00160		0.00212	0.00193
	50	0.75	0.00740	0.00666		0.00224	0.00210		0.00186	0.00171		0.00225	0.00207
	250	0.25	0.00125	0.00122		0.00033	0.00032		0.00029	0.00028		0.00035	0.00034
	250	0.75	0.00126	0.00123		0.00033	0.00032		0.00029	0.00028		0.00035	0.00034
	1000	0.25	0.00030	0.00029		0.00008	0.00007		0.00007	0.00007		0.00008	0.00008
	1000	0.75	0.00031	0.00031		0.00008	0.00008		0.00007	0.00007		0.00008	0.00008

6. Real-Data Analyses

In this section, the MLEs and BEs of Shannon, Rényi, Tsallis, and HC entropy measures based on the Kumaraswamy distribution are obtained using two real datasets from the fields of economics and agricultural hydrology under different progressive Type-II censoring schemes. Moreover, the fit of the KW distribution to these datasets is assessed through comparisons with the unit Lindley (UL), unit Rayleigh (UR), and unit Gompertz (UG) distributions. These comparisons are evaluated using several goodness-of-fit measures, namely, the Anderson–Darling (AD), Cramér–von Mises (CvM), and Kolmogorov–Smirnov (KS) test statistics, together with the corresponding p-value of the KS test.

6.1. Real Dataset I

The first dataset includes the growth rates of 61 countries. The dataset was first used to demonstrate the real-world applicability of the two-parameter unit Mirra distribution. The data were obtained from the publicly available “Determinants of Economic Growth” dataset, whose original source and access information are provided in the cited reference [22]. The first dataset consists of the values presented in Table 5.

Table 5. The first dataset.

0.140501976	0.704819918	0.324612707	0.425837249	0.324745566	0.329479247
0.330021679	0.156622976	0.157703221	0.313072860	0.337879002	0.339706242
0.352317631	0.582814276	0.358856708	0.435579330	0.517664552	0.393250912
0.442142904	0.444374621	0.450546652	0.455769300	0.468346560	0.473254889
0.484600782	0.488949597	0.509590268	0.527773321	0.160405084	0.417603940
0.534684658	0.160815045	0.221458390	0.299405932	0.543337107	0.544243515
0.550812602	0.552722335	0.560642540	0.560749650	0.567130983	0.575274825
0.626079738	0.639484167	0.646913528	0.651203632	0.681555152	0.699432909
0.605031252	0.729232311	0.742971599	0.745497823	0.779847085	0.798375845
0.814710021	0.822956383	0.830238342	0.834204197	0.979355395	0.603035331
0.613616884

The comparison results for the Kumaraswamy (KW) distribution and the competing models, together with the MLEs of the fitted models, are presented in Table 6. According to the goodness-of-fit results presented in Table 6, the KW distribution provides the best fit to the first dataset among the fitted distributions, as it yields the lowest AD, CvM, and KS statistics and the highest KS p-value. After the KW distribution is identified as the best-fitting model for Dataset I, the entropy measures are evaluated under various progressive Type-II censoring schemes. In this context, the number of observed failures in the first dataset is set to $m=31$, and three different scenarios are considered for the $R_j$ censoring scheme. The failure count $m=31$ corresponds to an observed failure proportion of approximately 50% of the real dataset, which lies within the range of the observed failure proportions (50%, 70%, and 90% of n) considered in the simulation study. The entropy parameter values $\delta =0.5$ and $\delta =3.5$ are selected to investigate the behavior of the entropy measures under both the $\delta <1$ and $\delta >1$ regimes. These choices are consistent with the finiteness conditions discussed in Section 2 and with the parameter values considered in the simulation study.

Table 6. Parameter estimates and goodness-of-fit statistics for Dataset I.

Distribution	Parameter Estimates			Goodness-of-Fit Statistics
	$\widehat{\mathit{a}}$	$\widehat{\mathit{b}}$		KS	AD	CvM	KS $\mathit{p}$-Value
KW	2.32967	2.76297		0.06900	0.41173	0.05584	0.91417
UL	0.72522	–		0.24843	5.70430	1.21840	0.00084
UR	1.29958	–		0.09634	1.01626	0.14903	0.58924
UG	1.09213	0.61619		0.10981	1.44681	0.20764	0.42349

• Scenario I:  $R_1=R_2=\cdots =R_{m-1}=0,R_m=n-m$
• Scenario II:  $R_1=n-m,R_2=R_3=\cdots =R_m=0$
• Scenario III:  $R_1=0,R_2=8,R_3=8,R_4=11,R_5=0,R_6=3,R_7=R_8=\cdots =R_m=0$

The complete data case is considered as $n=m=61$ for Dataset I. Table 1 reports the MLEs, their standard errors (SEs), and the corresponding 95% ACIs; the posterior estimates obtained via MCMC, their posterior SEs, and the corresponding 95% credible intervals (CrIs); and the TK-based BEs of the Shannon, Rényi, Tsallis, and Havrda–Charvát entropy measures for $\delta =0.5$ and $\delta =3.5$ under three progressive Type-II censoring scenarios with $m=31$, as well as for the complete-sample case.

When Table 1 is examined, it is observed that the MLE, MCMC, and TK-based Bayes estimates are close to one another in most cases. With respect to the censoring scenarios, the estimates obtained under Scenario I and Scenario III are fairly similar, whereas some discrepancies are observed under Scenario II, which yields entropy values that are slightly larger in absolute value. When evaluated in terms of SEs and interval widths, the complete-data scenario generally yields smaller standard errors and narrower confidence and credible intervals. These results indicate that the use of the complete dataset reduces the uncertainty associated with the entropy estimates. The similarity between the MLE-based confidence intervals and the Bayesian credible intervals obtained via MCMC suggests that the two estimation approaches yield comparable inferential conclusions. The effect of $\delta$ is particularly evident for the Tsallis and Havrda–Charvát entropies. When $\delta =0.5$, the standard errors and interval widths associated with these entropies are relatively small, whereas, when $\delta =3.5$, both the standard errors and interval widths increase considerably. Accordingly, the Havrda–Charvát entropy appears to exhibit greater estimation uncertainty when $\delta =3.5$. The highest uncertainty is generally observed under Scenario I when $\delta =3.5$; in this case, the ACIs for the Tsallis and HC entropies extend beyond zero, whereas the corresponding CrIs remain entirely negative. In contrast, the SEs of the Rényi entropy vary within a more limited range, whereas the Shannon entropy remains unaffected by $\delta$. Moreover, the Shannon entropy estimates are relatively close to zero, indicating a moderately dispersed fitted distribution of the country growth-rate data. The slightly more negative value under Scenario II suggests somewhat greater concentration.

6.2. Real Dataset II

The second dataset focuses on soil moisture values near the permanent wilting point of plants. The permanent wilting point defines the moisture threshold at which plants can extract water from the soil, below which water absorption becomes impossible. A total of 40 soil moisture measurements were reported using 0.12 as the threshold to evaluate reliability, resilience, and vulnerability; the original dataset and its collection context are also provided in the cited source [23]. The second dataset consists of the values presented in Table 7.

Table 7. The second dataset.

0.0816	0.2253	0.1944	0.3370	0.1208	0.0954	0.0562	0.2382	0.1949	0.3500
0.4080	0.3745	0.1647	0.2654	0.1300	0.2703	0.3837	0.3152	0.1448	0.1152
0.0717	0.2253	0.4149	0.3370	0.2500	0.1423	0.1258	0.1228	0.2948	0.4024
0.2834	0.2953	0.1647	0.1190	0.0655	0.0532	0.0296	0.2145	0.1526	0.1210

Table 8 presents the MLEs of the fitted models and the corresponding goodness-of-fit results for the Kumaraswamy (KW) distribution and the competing models. Among the fitted distributions, the KW model yields the smallest AD, CvM, and KS statistics, together with the largest KS p-value. These findings indicate that the KW distribution provides the best fit to Dataset II. Accordingly, the subsequent entropy analysis is conducted under the KW model for various progressive Type-II censoring schemes. In the second dataset, the number of failures is set to $m=30$ and three different scenarios are considered for the $R_j$ censoring scheme. This corresponds to an observed failure proportion of 75% in Dataset II, that is, a censored proportion of 25%, which is lower than the censored proportion of approximately 49% used for Dataset I. The two datasets therefore allow the proposed methodology to be evaluated under two different levels of censoring.

Table 8. Parameter estimates and goodness-of-fit statistics for Dataset II.

Distribution	Parameter Estimates			Goodness-of-Fit Statistics
	$\widehat{\mathit{a}}$	$\widehat{\mathit{b}}$		KS	AD	CvM	KS $\mathit{p}$-Value
KW	1.91906	15.27175		0.08943	0.36615	0.05937	0.90630
UL	4.11283	–		0.17830	1.58387	0.24172	0.15713
UR	0.29133	–		0.20185	1.82791	0.27756	0.07681
UG	1.22218	0.09171		0.16225	1.11249	0.14869	0.24298

• Scenario I:  $R_1=R_2=\dots =R_{m-1}=0,R_m=n-m$
• Scenario II:  $R_1=n-m,R_2=R_3=\dots =R_m=0$
• Scenario III:  $R_4=5,R_5=4,R_{13}=1,R_j=0(j\notin \{4,5,13\})$

Also, the complete data case is considered as $n=m=40$ for Dataset II. Table 2 reports the MLEs, their SEs, and the corresponding 95% CIs; the posterior estimates obtained via MCMC, their posterior SEs, and the corresponding 95% CrIs; and the TK-based Bayesian estimates of the Shannon, Rényi, Tsallis, and Havrda–Charvát entropy measures for $\delta =0.5$ and $\delta =3.5$ under three progressive Type-II censoring scenarios with $m=30$, as well as for the complete-sample case.

When Table 2 is examined, it is observed that the MLE, MCMC, and TK-based Bayes estimates are close to one another in most cases. With respect to the censoring scenarios, the estimates obtained under Scenario II and Scenario III are highly similar, whereas Scenario I yields somewhat different estimates. In terms of SEs and interval widths, the complete-data scenario produces the smallest SEs and the narrowest ACIs and CrIs for both values of $\delta$, which indicates that the use of the complete dataset reduces the uncertainty associated with the entropy estimates. The substantial overlap between the MLE-based confidence intervals and the Bayesian credible intervals obtained via MCMC indicates that the two estimation approaches provide broadly comparable inferential conclusions. The effect of $\delta$ is especially pronounced for the Tsallis and HC entropies. When $\delta =0.5$, their SEs and interval widths remain relatively small, whereas both increase considerably when $\delta =3.5$, particularly under Scenario I and Scenario II. Among the entropy measures, the Havrda–Charvát entropy exhibits the greatest estimation uncertainty for $\delta =3.5$, followed by the Tsallis entropy. In contrast, the SEs of the Rényi entropy vary within a more limited range, while the Shannon entropy remains unaffected by $\delta$. With regard to the Shannon entropy, the negative estimates indicate that the fitted soil moisture distribution is concentrated within the unit interval. This concentration pattern is consistently preserved across the different censoring schemes.

7. Limitations

The proposed analysis is subject to several limitations. First, the asymptotic results rely on identifiability, information matrix, and convergence conditions associated with the progressive censoring mechanism; therefore, finite-sample accuracy is not guaranteed. Second, although the TK-based Bayesian estimators yield numerically lower MSE values than the MLEs in several simulation settings, particularly for small sample sizes, these comparisons are based on Monte Carlo estimates obtained from a finite number of replications. No formal replication-level significance test was conducted to determine whether the observed differences exceed Monte Carlo variability. Accordingly, the reported MSE differences should be interpreted descriptively rather than as evidence of statistically significant superiority. Third, Bayesian entropy estimates may be sensitive to the specification of the Gamma prior parameters, particularly near the boundary of the entropy-existence region. The sensitivity analysis reported in Appendix B is restricted to $S_1$ and $S_2$, for which $\delta <1$ and the admissible region ${\mathcal{A}}_{\delta }$ is unrestricted. Thus, it provides evidence that the reported results in these settings are not driven by the choice of the diffuse prior $P_1$, but it does not address the $\delta >1$ scenarios, where ${\mathcal{A}}_{\delta }$ is restricted by the entropy-finiteness conditions, nor the real-data applications, which are reported under $P_1$ only. Therefore, prior sensitivity in these settings remains a limitation of the present study. Fourth, the Kumaraswamy distribution is defined on $(0,1)$; therefore, data outside this interval require a scientifically justified transformation, whose associated uncertainty is not explicitly accounted for in the present analysis. Fifth, the censoring schemes considered in the real-data applications are constructed scenarios rather than empirically recorded removal processes. Finally, the Tierney–Kadane approximation is a local Laplace approximation and may become less reliable for multimodal, highly skewed, or boundary-concentrated posterior distributions. For this reason, the MCMC-based posterior summaries and comparisons reported in the real-data analysis remain useful as complementary assessments of the TK-based results. Future studies could further strengthen the simulation-based comparison by reporting Monte Carlo standard errors or conducting replication-level statistical tests.

8. Conclusions

In this study, the estimation of four different entropy measures based on the Kumaraswamy distribution is investigated under progressive Type-II censoring with binomial removals. The MLEs and BEs of these entropy measures are derived, with the TK approximation employed to obtain the Bayesian estimators. In addition, the consistency and asymptotic normality of the ML estimators are stated under standard regularity conditions, and approximate confidence intervals are constructed based on the observed information matrix. A comprehensive Monte Carlo simulation study is conducted to evaluate the performance of the entropy estimators under different progressive Type-II censoring schemes. The simulation results indicate that, with the other parameters held fixed, the bias and MSE values of both the MLEs and BEs generally decrease as n and m increase. Across the five parameter scenarios ($S_1-S_5$), the TK-based BEs generally tend to yield lower MSE values for small sample sizes, with the Shannon entropy in $S_1$ being the main exception, whereas the two estimation methods yield increasingly comparable results as the sample size increases. Furthermore, to illustrate the practical applicability of the proposed estimation methods, two real datasets from the fields of economics and agricultural hydrology are analyzed. For both datasets, the MLE, MCMC, and TK-based Bayesian estimates are generally close to one another across the considered censoring scenarios. The approximate 95% confidence intervals based on the MLEs and the corresponding 95% Bayesian credible intervals obtained via MCMC exhibit substantial overlap in most cases, suggesting broadly comparable inferential conclusions. In general, the complete-data cases yield smaller standard errors and narrower intervals, whereas the estimation uncertainty becomes more pronounced for the Tsallis and Havrda–Charvat entropies at larger values of $\delta$. As a continuation of the present study, future research may consider the maximum product of spacings method as an alternative approach for estimating entropy measures and systematically compare its performance with the ML and Bayesian methods.