Bayesian Estimations of Shannon Entropy and Rényi Entropy of Inverse Weibull Distribution

Abstract

In this paper, under the symmetric entropy and the scale squared error loss functions, we consider the maximum likelihood (ML) estimation and Bayesian estimation of the Shannon entropy and Rényi entropy of the two-parameter inverse Weibull distribution. In the ML estimation, the dichotomy is used to solve the likelihood equation. In addition, the approximation confidence interval is given by the Delta method. Because the form of estimation results is more complex in the Bayesian estimation, the Lindley approximation method is used to achieve the numerical calculation. Finally, Monte Carlo simulations and a real dataset are used to illustrate the results derived. By comparing the mean square error between the estimated value and the real value, it can be found that the performance of ML estimation of Shannon entropy is better than that of Bayesian estimation, and there is no significant difference between the performance of ML estimation of Rényi entropy and that of Bayesian estimation.

1. Introduction

Information is an abstract concept. In the face of a large amount of data, it is easy to know how much data there are, but it is not clear how much information these data contain. Entropy is one of the important terms in physics. Shannon [1] introduced the concept of entropy into statistics, which represents the uncertainty of events. This entropy is generally called “Shannon entropy”. Generally speaking, when we hear a message within expectation, we think it contains less information. When we hear an unexpected message, we think that the amount of information it conveys to us is significant. In statistics, the probability is usually used to describe the uncertainty of an event. Therefore, Shannon believes that probability can be used to describe the amount of information contained in an event. After that, Rényi [2] generalized Shannon entropy and put forward the concept of Rényi entropy. Since then, the study of entropy has attracted a lot of attention [3,4]. For example, Chacko and Asha [5] considered the maximum likelihood (ML) estimation and Bayesian estimation of Shannon entropy for a generalized exponential distribution by the important sampling method based on record values. Liu and Gui [6] considered the ML estimation and Bayesian estimation of Shannon entropy for a two-parameter Lomax distribution by the Lindley method and the Tierney–Kadane method under a generalized progressively hybrid censoring test. Shrahili et al. [7] considered the estimation of entropy of a log-logistic distribution. The estimations of different entropy functions were obtained by the ML method, and the approximate confidence intervals were obtained by using various censoring methods and sample sizes. Mahmoud et al. [8] considered the estimation of entropy and residual entropy of a two-parameter Lomax distribution based on the generalized type-II hybrid censoring scheme. The ML estimators and Bayesian estimators of entropy and residual entropy were obtained. The simulation study of estimating performance under different sample sizes was described. Finally, the conclusion was discussed. Hassan and Mazen [9] estimated three entropy measures for the inverse Weibull distribution using progressive Type-II censored data, which were Shannon entropy, Rényi entropy, and q-entropy. The method of maximum likelihood and the maximum product of spacing were used to estimate them. Mavis et al. [10] proposed and studied a gamma-inverse Weibull distribution, and some mathematical properties were given including moments, mean deviations, Bonferroni and Lorenz curves, and entropies. Basheer [11] introduced a new generalized alpha power inverse Weibull distribution, and the Shannon entropy and Rényi entropy were obtained. Valeriia and Broderick [12] proposed the weighted inverse Weibull class of distributions and derived the expressions of Shannon entropy and Rényi entropy.

In 1982, Keller and Kamath [13] introduced the Inverse Weibull Distribution (IWD) to model the degradation of mechanical components of diesel engines. It is a useful lifetime probability distribution, and it can be used to represent various failure characteristics. Depending on the value of the shape parameter of the IWD, the risk function can be changed flexibly. The use of IWDs for data fitting is therefore more appropriate in many cases. For example, Abhijit and Anindya [14] found that the use of the IWD was superior to previous normal models when measuring concrete structures using ultrasonic pulse velocities. Chiodo et al. [15] proposed a new model generated from an appropriate mixture of IWDs for modeling extreme wind speed scenarios. Langlands et al. [16] observed that breast cancer mortality data could be analyzed using IWDs for modeling analysis. That is why the two-parameter IWD has attracted the attention of more and more researchers and has caused discussion among them in recent years [17,18]. For example, Asuman and Mahmut [19] considered the classical and Bayesian estimation of parameters and the reliability function of the IWD. In classical estimation, they derived the ML estimators and modified ML estimators. In Bayesian estimation, they utilized the Lindley method to calculate the Bayesian estimators of parameters under symmetric and asymmetric loss functions. Sultan et al. [20] discussed the estimation of parameters of the IWD based on the progressive type-II censored sample. They put forward an approximate maximum likelihood method to obtain the ML estimator and used Lindley’s approximation to obtain the Bayesian estimators. Amirzadi et al. [21] considered the Bayesian estimation of the scale parameter and the reliability in the inverse generalized Weibull distribution, in addition to general entropy, the squared log error, and the weight squared error function. They introduced a new loss function to carry out Bayesian estimation. Peng and Yan [22] studied the Bayesian estimation and prediction for shape and scale parameters of the IWD under a general progressive censoring test. Sindhu et al. [23] assumed different priors and loss functions, and discussed the Bayesian estimation of inverse Weibull mixture distributions based on doubly censored data. Mohammad and Sana [24] obtained the Bayes estimators and ML estimators for the unknown parameters of the IWD under lower record values. Faud [25] developed a linear exponential loss function, and estimated the parameter and reliability of the IWD based on lower record values under this loss function. Li and Hao [26] considered the estimation of a stress–strength model when stress and strength are two independent IWDs with different parameters. Ismail and Tamimi [27] proposed a constant-stress partially accelerated-life test model and analyzed it using type-I censored data from an IWD. Kang and Han [28] derived the approximate maximum likelihood estimators of parameters of an IWD under multiply type-II censoring and also proposed a simple graphical method for a goodness-of-fit test. Saboori et al. [29] introduced a generalized modified inverse Weibull distribution, and some statistical and probabilistic properties were derived.

This paper considers the Bayesian estimation of Shannon entropy and Rényi entropy of a two-parameter IWD based on complete samples. In Section 2, some related knowledge is introduced first, and then the specific expressions of Shannon entropy and Rényi entropy of the two-parameter IWD are derived. In Section 3, the maximum likelihood estimators of the scale parameter and shape parameter of the IWD are derived by the dichotomy method, and then the ML estimators of Shannon entropy and Rényi entropy are obtained. In Section 4, the gamma distribution is adopted as the prior distribution (PD) of the scale parameter. A non-informative PD is adopted as the PD of the shape parameter. Then, the Bayesian estimators of Shannon entropy and Rényi entropy are obtained based on the symmetric entropy loss function and scale squared error loss function. The Lindley approximation is used to achieve the numerical calculation of the Bayesian estimators of entropy, on account of the complexity of these Bayesian estimators. In Section 5, Monte Carlo simulations are utilized to simulate and compare the estimators that are mentioned above. In Section 6, a real data set is analyzed for illustrative purposes. Finally, the conclusions of the article are given in Section 7.

2. Preliminary Knowledge

The probability density function (pdf) of the two-parameter IWD is defined as Equation (1):

$$f(t;\omega ,\upsilon )=\omega \upsilon t^{-\upsilon -1}\exp (-\omega t^{-\upsilon }),\omega >0,\upsilon >0,t>0,$$

(1)

and the cumulative distribution function (cdf) of the two-parameter IWD is defined as Equation (2):

$$F(t;\omega ,\upsilon )=\exp (-\omega t^{-\upsilon }),\omega >0,\upsilon >0,t>0,$$

(2)

where the scale parameter is $\omega$ and the shape parameter is $\upsilon$.

Figure 1 shows the pdf of the IWD under different values of shape and scale parameters.

The Shannon entropy is defined in Equation (3) [1]

$$H_s(t)=-{\displaystyle {\int }_{-\infty }^{+\infty }f(t)\ln [f(t)]}dt,$$

(3)

and the Rényi entropy is defined in Equation (4) [2]

$$H_r(t)=\frac{1}{1-r}\ln {\displaystyle {\int }_{-\infty }^{+\infty }{f}^r(t)}dt,r>0,r\ne 1,$$

(4)

where $f(t)$ is the pdf of a continuous random variable $T$.

Theorem 1.

Let $T_1,T_2,\dots ,T_n$ be a random sample that follows the IWD with the pdf (1), with $t_1,t_2,\dots ,t_n$ being the sample observations of $T_1,T_2,\dots ,T_n$.

(i)

The Shannon entropy of the IWD is shown in Equation (5):

$$H_s=\frac{\upsilon +1}{\upsilon }(\ln \omega +\gamma )-\ln \omega \upsilon +1.$$

(5)

(ii)

The Rényi entropy of the IWD is shown in Equation (6):

$$H_r=\frac{1}{1-r}[-\frac{r\upsilon +r}{\upsilon }\ln r+\frac{r}{\upsilon }\ln \omega +r\ln \upsilon +\ln \mathsf{\Gamma }(r+\frac{r}{\upsilon }+1)],$$

(6)

where $\mathsf{\Gamma }(\cdot )$ is the gamma function and $\gamma$ is the Euler constant.

Proof. 

The log density of pdf (1) of the IWD is shown in Equation (7):

$$\ln [f(t)]=\ln \omega \upsilon -(\upsilon +1)\ln t-\omega t^{-\upsilon }.$$

(7)

According to the log-density function (7) and Equation (3), the Shannon entropy of the IWD can be derived as follows:

$$\begin{array}{cc}{H}_s& =-{\displaystyle {\int }_0^{+\infty }f(t)[\ln \omega \upsilon -(\upsilon +1)\ln t-\omega t^{-\upsilon }]}dt\hfill \\ & =-(\ln \omega \upsilon ){\displaystyle {\int }_0^{+\infty }f(t)}dt+(\upsilon +1){\displaystyle {\int }_0^{+\infty }f(t)\ln t}dt+\omega {\displaystyle {\int }_0^{+\infty }{t}^{-\upsilon }f(t)}dt\hfill \\ & =-\ln \omega \upsilon +(\upsilon +1)E(\ln T)+\omega E(T^{-\upsilon })\hfill \end{array}.$$

Obviously,

$$\begin{array}{cc}E(T^c)& ={\displaystyle {\int }_0^{+\infty }\omega \upsilon t^c{t}^{-\upsilon -1}{e}^{-\omega t^{-\upsilon }}}dt\hfill \\ & ={\omega }^{\frac{t}{\upsilon }}{\displaystyle {\int }_0^{+\infty }{(\omega t^{-\upsilon })}^{-\frac{c}{\upsilon }}{e}^{-\omega t^{-\upsilon }}}d(\omega t^{-\upsilon })\hfill \\ & ={\omega }^{\frac{c}{\upsilon }}\mathsf{\Gamma }(1-\frac{c}{\upsilon })\hfill \end{array}.$$

Let $c=-\upsilon$,

$$E(T^{-\upsilon })={\omega }^{-1}\mathsf{\Gamma }(2)=\frac{1}{\omega }.$$

Because

$$\begin{array}{cc}E(T^c\ln T)& =\frac{dE(T^c)}{dc}\hfill \\ & =\frac{1}{\upsilon }{\omega }^{\frac{c}{\upsilon }}\mathsf{\Gamma }(1-\frac{c}{\upsilon })\ln \omega -\frac{1}{\upsilon }{\omega }^{\frac{c}{\upsilon }}{\mathsf{\Gamma }}^{\prime }(1-\frac{c}{\upsilon })\hfill \\ & =\frac{1}{\upsilon }{\omega }^{\frac{c}{\upsilon }}[\mathsf{\Gamma }(1-\frac{c}{\upsilon })\ln \omega -{\mathsf{\Gamma }}^{\prime }(1-\frac{c}{\upsilon })].\hfill \end{array}$$

Let $c=0$,

$$E(\ln T)=\frac{1}{\upsilon }(\ln \omega +\gamma ).$$

Therefore, the Shannon entropy of the two-parameter IWD can be expressed as

$$\begin{array}{cc}{H}_s& =-\ln \omega \upsilon +(\upsilon +1)E(\ln T)+\omega E(T^{-\upsilon })\hfill \\ & =\frac{\upsilon +1}{\upsilon }(\ln \omega +\gamma )-\ln \omega \upsilon +1.\hfill \end{array}$$

Obviously,

$$\begin{array}{cc}{\displaystyle {\int }_{-\infty }^{+\infty }{f}^r(t)}dt& ={\displaystyle {\int }_0^{+\infty }{(\omega \upsilon t^{-\upsilon -1}{e}^{-\omega t^{-\upsilon }})}^r}dt\hfill \\ & ={(\omega \upsilon )}^r{\displaystyle {\int }_0^{+\infty }{t}^{-r\upsilon -r}{e}^{-r\omega t^{-\upsilon }}}dt\hfill \\ & =r^{\frac{-r\upsilon -r}{\upsilon }}{\omega }^{\frac{r}{\upsilon }}{\upsilon }^r\mathsf{\Gamma }(1+\frac{r+r\upsilon }{\upsilon })\hfill \end{array}.$$

Then, according to Equation (4), the Rényi entropy of the two-parameter IWD can be expressed as

$$H_r=\frac{1}{1-r}[\frac{-r\upsilon -r}{\upsilon }\ln r+\frac{r}{\upsilon }\ln \omega +r\ln \upsilon +\ln \mathsf{\Gamma }(1+\frac{r\upsilon +r}{\upsilon })].$$

□

3. Maximum Likelihood Estimation

Suppose that $T_1,T_2,\dots ,T_n$ is a random sample that follows the IWD with the pdf (1), and $t_1,t_2,\dots ,t_n$ are the sample observations of $T_1,T_2,\dots ,T_n$. Thus, the likelihood function (LF)can be derived as Equation (8):

$$s(t;\omega ,\upsilon )={\omega }^n{\upsilon }^n({\displaystyle \prod _{i=1}^n{t}_i^{-\upsilon -1}})\exp (-\omega {\displaystyle \sum _{i=1}^n{t}_i^{-\upsilon }}).$$

(8)

Then, the corresponding log LF of Equation (8) is shown in Equation (9):

$$S(t;\omega ,\upsilon )=\ln s(t;\omega ,\upsilon )=n\ln \omega \upsilon -(\upsilon +1){\displaystyle \sum _{i=1}^n\ln t_i}-\omega {\displaystyle \sum _{i=1}^n{t}_i^{-\upsilon }}.$$

(9)

For convenience, we denote $S(t;\omega ,\upsilon )$ as $S$. Thus, the likelihood equations can be expressed, respectively, as Equations (10) and (11):

$$\frac{\partial S}{\partial \omega }=\frac{n}{\omega }-{\displaystyle \sum _{i=1}^n{t}_i^{-\upsilon }}=0,$$

(10)

$$\frac{\partial S}{\partial \upsilon }=\frac{n}{\upsilon }-{\displaystyle \sum _{i=1}^n\ln t_i}+\omega {\displaystyle \sum _{i=1}^n{t}_i^{-\upsilon }\ln t_i}=0.$$

(11)

By simplifying Equations (10) and (11), the ML estimator $\widehat{\upsilon }$ is a unique solution of Equation (13), and the ML estimator $\widehat{\omega }$ can be obtained according to Equation (12). The uniqueness of the solution to Equation (13) is proved in Appendix A.

$$\widehat{\omega }=n{({\displaystyle \sum _{i=1}^n{t}_i^{-\widehat{\upsilon }}})}^{-1},$$

(12)

$$y(\upsilon )=\frac{n}{\upsilon }-{\displaystyle \sum _{i=1}^n\ln t_i}+n{({\displaystyle \sum _{i=1}^n{t}_i^{-\upsilon }})}^{-1}{\displaystyle \sum _{i=1}^n{t}_i^{-\upsilon }\ln t_i}.$$

(13)

According to Equation (13), it is difficult to obtain the analytical solution $\widehat{\upsilon }$. The following are the steps to obtain a numerical solution using the dichotomy method.

(i)

Give the accuracy $\epsilon$, determine the interval $[{\upsilon }_u,{\upsilon }_l]$, and verify $y({\upsilon }_u)\cdot y({\upsilon }_l)<0$.

(ii)

Find the midpoint ${\upsilon }_m$ of the interval $[{\upsilon }_u,{\upsilon }_l]$ and calculate $y({\upsilon }_m)$.

(iii)

If $y({\upsilon }_m)=0$, $\widehat{\upsilon }={\upsilon }_m$.

(iv)

If $y({\upsilon }_u)\cdot y({\upsilon }_m)<0$, ${\upsilon }_l={\upsilon }_m$; if $y({\upsilon }_l)\cdot y({\upsilon }_m)<0$, ${\upsilon }_u={\upsilon }_m$.

(v)

If $|{\upsilon }_u-{\upsilon }_l|<\epsilon$, $\widehat{\upsilon }$ is equal to ${\upsilon }_u$ or ${\upsilon }_l$. If not, return to step (ii) to step (v).

Due to the invariance of ML estimation, the ML estimators of Shannon entropy and Rényi entropy can be obtained by putting $\widehat{\omega }$ and $\widehat{\upsilon }$ into Equations (3) and (4), and their mathematical expressions are shown in Equations (14) and (15):

$${\widehat{H}}_{s1}=\frac{\widehat{\upsilon }+1}{\widehat{\upsilon }}(\ln \widehat{\omega }+\gamma )-\ln \widehat{\omega }\widehat{\upsilon }+1,$$

(14)

$${\widehat{H}}_{r1}=\frac{1}{1-r}[\frac{-r\widehat{\upsilon }-r}{\widehat{\upsilon }}\ln r+\frac{r}{\widehat{\upsilon }}\ln \widehat{\omega }+r\ln \widehat{\upsilon }+\ln \mathsf{\Gamma }(1+\frac{r\widehat{\upsilon }+r}{\widehat{\upsilon }})].$$

(15)

Next, the Delta method is used to derive the approximate confidence intervals (briefly, ACIs) of Shannon entropy and Rényi entropy.

Denote vector $D_s$ and $D_r$, respectively, as

$$D_s=(\frac{\partial H_s}{\partial \omega },\frac{\partial H_s}{\partial \upsilon })\left|{}_{\omega =\widehat{\omega },\upsilon =\widehat{\upsilon }}\right.,$$

(16)

$$D_r=(\frac{\partial H_r}{\partial \omega },\frac{\partial H_r}{\partial \upsilon })\left|{}_{\omega =\widehat{\omega },\upsilon =\widehat{\upsilon }}\right.,$$

(17)

in which $D_s$ and $D_r$ are calculated through Equations (18) and (19), respectively.

$$\frac{\partial H_s}{\partial \omega }=\frac{1}{\omega \upsilon },\frac{\partial H_s}{\partial \upsilon }=\frac{-\gamma -\ln \omega }{{\upsilon }^2}-\frac{1}{\upsilon }$$

(18)

$$\frac{\partial H_r}{\partial \omega }=\frac{r}{(1-r)\omega \upsilon }, \frac{\partial H_r}{\partial \upsilon }=\frac{r}{(1-r)\upsilon }[1+\frac{\ln r-\ln \omega }{\upsilon }-\frac{{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})}{\upsilon \mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}]$$

(19)

According to the Delta method, calculate the estimated variance of ${\widehat{H}}_{s1}$ and ${\widehat{H}}_{r1}$ as Equations (20) and (21), respectively. $I$ is the Fisher information matrix of $\omega$ and $\upsilon$, and Equation (22) gives the elements of $I$. $I^{-1}$ is the inverse matrix of $I$.

$$V_s=D_s{I}^{-1}{D}_s^{\mathrm{T}}\left|{}_{\omega =\widehat{\omega },\upsilon =\widehat{\upsilon }}\right.$$

(20)

$$V_r=D_r{I}^{-1}{D}_r^{\mathrm{T}}\left|{}_{\omega =\widehat{\omega },\upsilon =\widehat{\upsilon }}\right.$$

(21)

$$\begin{array}{l}\frac{{\partial }^{2}S}{\partial {\omega }^2}=-\frac{n}{{\omega }^2},\frac{{\partial }^{2}S}{\partial \omega \partial \upsilon }=\frac{{\partial }^{2}S}{\partial \upsilon \partial \omega }={\displaystyle \sum _{i=1}^n{t}_i^{-\upsilon }\ln t_i}\\ \frac{{\partial }^{2}S}{\partial {\upsilon }^2}=-\frac{n}{{\upsilon }^2}-\omega {\displaystyle \sum _{i=1}^n{t}_i^{-\upsilon }{(\ln t_i)}^2}\end{array}$$

(22)

Then, the $100(1-\alpha )\%$ ACI of Shannon entropy is Equation (23) and the $100(1-\alpha )\%$ ACI of Rényi entropy is Equation (24), where $z_{\raisebox{1ex}{$\alpha $}\!\left/ \!\raisebox{-1ex}{$2$}\right.}$ is the upper ($\raisebox{1ex}{$\alpha $}\!\left/ \!\raisebox{-1ex}{$2$}\right.$)th quantile of the standardized normal distribution.

$$({\widehat{H}}_{s1}-z_{\raisebox{1ex}{$\alpha $}\!\left/ \!\raisebox{-1ex}{$2$}\right.}\sqrt{V_s} , {\widehat{H}}_{s1}+z_{\raisebox{1ex}{$\alpha $}\!\left/ \!\raisebox{-1ex}{$2$}\right.}\sqrt{V_s})$$

(23)

$$({\widehat{H}}_{r1}-z_{\raisebox{1ex}{$\alpha $}\!\left/ \!\raisebox{-1ex}{$2$}\right.}\sqrt{V_r} , {\widehat{H}}_{r1}+z_{\raisebox{1ex}{$\alpha $}\!\left/ \!\raisebox{-1ex}{$2$}\right.}\sqrt{V_r})$$

(24)

4. Bayesian Estimation

Bayesian estimation is a method of introducing prior information to deal with decision problems. The advantage is that it can include the prior information in statistical inference and improve the accuracy of the taken decision. From the time when Bayesian estimation was proposed to now, many researchers have adopted this method in estimating parameters and related functions. For example, Liu and Wang [30] considered the Bayesian inference and prediction of the proportional hazards model, based on interval type-II censored data. They Assumed that the prior distribution of parameter is non-informative prior distribution and used MH algorithm to obtain the parameter estimator. Ren [31] derived the Bayesian estimator and emprical Bayesian estimator of parameter of Rayleigh distribution under symmetric entropy loss function. Furthermore, this paper also discussed the admissibility conditions for a class of inverse linear estimators. Mohammad and Mina [32] presented the Bayesian inferences of parameters of the inverse Weibull distribution based on type-I hybrid censored data and computed the Bayes estimates using the Lindley approximation. Algarni et al. [33] considered the Bayes estimation of parameters for the inverse Weibull distribution by employing a progressive type-I censored sample. The Metropolis–Hasting (MH) algorithm was used to compute the Bayesian estimates.

In addition to the areas mentioned above, there are some recent applications of the Bayesian method. Zhou and Luo [34] developed a supplier’s recursive multiperiod discounted profit model based on Bayesian information updating. Yulin et al. [35] put forward a Bayesian approach to tackle the misalignment for over-the-air computation. Taborsky et al. [36] presented a novel generic Bayesian probabilistic model to solve the problem of parameter marginalization under the constraint of forced community structure. Oliver [37] introduced the Bayesian toolkit and showed how geomorphic models might benefit from probabilistic concepts. Ran et al. [38] proposed a Bayesian approach to measure the loss of privacy in a mechanism. Luo et al. [39] used the Bayesian information criterion for model selection when revisiting the lifetime data of brake pads. Peng et al. [40] extended a general Bayesian framework to deal with the degradation analysis of sparse degradation observations and evolving observations. František et al. [41] illustrated the benefit of Bayesian estimation in parametric survival analysis. Liu et al. [42] proposed fuzzy Bayesian knowledge tracing models to address continuous score scenarios. In predictive maintenance, Zhuang et al. [43] applied the Bayes theorem to the bidirectional LSTM network to optimize prognostic performance.

In this paper, the Bayesian estimations of Shannon entropy and Rényi entropy of the IWD are investigated under symmetric entropy (SE) and scale squared error (SSE) loss functions, which are widely used in Bayesian statistical inference [44,45,46].

(i)The SE loss function is defined in Equation (25) [44]:

$$L_1(H,\widehat{H})=\frac{H}{\widehat{H}}+\frac{\widehat{H}}{H}-2,$$

(25)

where $\widehat{H}$ is the estimator of $H$.

Lemma 1.

Suppose that $T$ is the historical data information about the entropy function $H$. Then, under the SE loss function (25), the Bayesian estimator ${\widehat{H}}_1$ for any prior distribution is shown in Equation (26):

$${\widehat{H}}_1={[\frac{E(H\left|T\right.)}{E(H^{-1}\left|T\right.)}]}^{\frac{1}{2}},$$

(26)

where $E(H|T)$ is the posterior expectation of $H$ and $E(H^{-1}|T)$ is the posterior expectation of $H^{-1}$.

Proof. 

Under the SE loss function (25), the Bayesian risk of $\widehat{H}$ is

$$R(\widehat{H})=E_H(E(L_1(H,\widehat{H})\left|T\right.)).$$

To minimize $R(\widehat{H})$, we only need to minimize $E(L_1(H,\widehat{H})|T)$. For convenience, let $g(\widehat{H})=E(L_1(H,\widehat{H})|T)$.

Because

$$g(\widehat{H})={\widehat{H}}^{-1}E(H\left|T\right.)+\widehat{H}E(H^{-1}\left|T\right.)-2,$$

and the derivative is

$$g^{\prime }(\widehat{H})=-{\widehat{H}}^{-2}E(H\left|T\right.)+E(H^{-1}\left|T\right.).$$

The Bayesian estimator ${\widehat{H}}_1$ can be obtained by $g^{\prime }(\widehat{H})=0$.

(ii) The SSE loss function is defined in Equation (Equation (27)) [46]

$$L_2(H,\widehat{H})=\frac{{(H-\widehat{H})}^2}{H^k},$$

(27)

where $k$ is a nonnegative integer. □

Lemma 2.

Suppose that $T$ is the historical data information about the entropy function $H$. Then, under the SSE loss function (27), the Bayesian estimator ${\widehat{H}}_2$ for any prior distribution is

$${\widehat{H}}_2=\frac{E(H^{1-k}\left|T\right.)}{E(H^{-k}\left|T\right.)},$$

(28)

where $E(H^{1-k}|T)$ is the posterior expectation of $H^{1-k}$ and $E(H^{-k}|T)$ is the posterior expectation of $H^{-k}$.

Proof. 

Under the SSE loss function (27), the Bayesian risk of $\widehat{H}$ is

$$R(\widehat{H})=E_H(E(L_2(H,\widehat{H})\left|T\right.)).$$

To minimize $R(\widehat{H})$, we only need to minimize $E(L_2(H,\widehat{H})|T)$. Similarly, let $h(\widehat{H})=E(L_2(H,\widehat{H})|T)$.

Because

$$\begin{array}{cc}h(\widehat{H})& =E(\frac{H^2-2H\widehat{H}+{\widehat{H}}^2}{H^k}\left|T\right.)\hfill \\ & =E(H^{2-k}\left|T\right.)-2\widehat{H}E(H^{1-k}\left|T\right.)+{\widehat{H}}^{2}E(H^{-k}\left|T\right.)\hfill \end{array},$$

and the derivative of $h(\widehat{H})$ is

$$h^{\prime }(\widehat{H})=-2E(H^{1-k}\left|T\right.)+2\widehat{H}E(H^{-k}\left|T\right.).$$

The Bayes estimator ${\widehat{H}}_2$ can be obtained by $h^{\prime }(\widehat{H})=0$.

Assume that the scale parameter $\omega$ and shape parameter $\upsilon$ of the two-parameter IWD are independent random variables, where $\omega$ obeys $\mathsf{\Gamma }(a,b)$ and $\upsilon$ obeys a non-informative PD as follows, respectively:

$$P_1(\omega )=\frac{a^b}{\mathsf{\Gamma }(b)}{\omega }^{b-1}{e}^{-a\omega },a>0,b>0,$$

(29)

$$P_2(\upsilon )\propto \frac{1}{\upsilon }.$$

(30)

Thus, the joint PD of $\omega$ and $\upsilon$ is

$$P(\omega ,\upsilon )\propto \frac{a^b}{\upsilon \mathsf{\Gamma }(b)}{\omega }^{b-1}{e}^{-a\omega }.$$

(31)

Referring to the Bayesian formulation, the posterior distribution of $\omega$ and $\upsilon$ is

$$P(\omega ,\upsilon \left|T\right.)=\frac{P(\omega ,\upsilon )s(t;\omega ,\upsilon )}{{\displaystyle {\int }_0^{+\infty }{\displaystyle {\int }_0^{+\infty }P(\omega ,\upsilon )s(t;\omega ,\upsilon )}}d\omega d\upsilon }.$$

(32)

Thus, the Bayesian estimators of Shannon entropy and Rényi entropy, respectively, under SE can be expressed as

$${\widehat{H}}_{s2}={[\frac{E(H_s\left|T\right.)}{E(H_s^{-1}\left|T\right.)}]}^{\frac{1}{2}}={[\frac{{\displaystyle {\int }_0^{+\infty }{\displaystyle {\int }_0^{+\infty }{H}_{s}P(\omega ,\upsilon \left|T\right.)}}d\omega d\upsilon }{{\displaystyle {\int }_0^{+\infty }{\displaystyle {\int }_0^{+\infty }{H}_s{}^{-1}P(\omega ,\upsilon \left|T\right.)}}d\omega d\upsilon }]}^{\frac{1}{2}},$$

(33)

$${\widehat{H}}_{r2}={[\frac{E(H_r\left|T\right.)}{E(H_r^{-1}\left|T\right.)}]}^{\frac{1}{2}}={[\frac{{\displaystyle {\int }_0^{+\infty }{\displaystyle {\int }_0^{+\infty }{H}_{r}P(\omega ,\upsilon \left|T\right.)}}d\omega d\upsilon }{{\displaystyle {\int }_0^{+\infty }{\displaystyle {\int }_0^{+\infty }{H}_r^{-1}P(\omega ,\upsilon \left|T\right.)}}d\omega d\upsilon }]}^{\frac{1}{2}}.$$

(34)

The Bayesian estimators of Shannon entropy and Rényi entropy, respectively, under SSE can be expressed as

$${\widehat{H}}_{s3}=\frac{E(H_s^{1-k}\left|T\right.)}{E(H_s^{-k}\left|T\right.)}=\frac{{\displaystyle {\int }_0^{+\infty }{\displaystyle {\int }_0^{+\infty }{H}_s^{1-k}P(\omega ,\upsilon \left|T\right.)}}d\omega d\upsilon }{{\displaystyle {\int }_0^{+\infty }{\displaystyle {\int }_0^{+\infty }{H}_s^{-k}P(\omega ,\upsilon \left|T\right.)}}d\omega d\upsilon },$$

(35)

$${\widehat{H}}_{r3}=\frac{E(H_r^{1-k}\left|T\right.)}{E(H_r^{-k}\left|T\right.)}=\frac{{\displaystyle {\int }_0^{+\infty }{\displaystyle {\int }_0^{+\infty }{H}_r^{1-k}P(\omega ,\upsilon \left|T\right.)}}d\omega d\upsilon }{{\displaystyle {\int }_0^{+\infty }{\displaystyle {\int }_0^{+\infty }{H}_r^{-k}P(\omega ,\upsilon \left|T\right.)}}d\omega d\upsilon }.$$

(36)

From Equation (33) to Equation (36), it can be seen that the calculation of Bayesian estimators of Shannon and Rényi entropy are complex and difficult to calculate. Thus, the Lindley approximation will be employed to achieve the approximate calculation results of ${\widehat{H}}_{s3}$ and ${\widehat{H}}_{r3}$. □

4.1. Bayesian Estimation by Using Lindley Approximation under SE Loss Function

Referring to the Lindley approximation, $I(t)$ can be defined as

$$I(t)=E[U(\omega ,\upsilon )\left|T\right.]=\frac{{\displaystyle \int U(\omega ,\upsilon )e^{S(t;\omega ,\upsilon )+G(\omega ,\upsilon )}}d(\omega ,\upsilon )}{{\displaystyle \int e^{S(t;\omega ,\upsilon )+G(\omega ,\upsilon )}}d(\omega ,\upsilon )},$$

(37)

where $U(\omega ,\upsilon )$ is a function of independent variables $\omega$ and $\upsilon$, $S(t;\omega ,\upsilon )$ is a log LF defined in Equation (9), and $G(\omega ,\upsilon )$ is the log of the joint PD defined in Equation (31).

If the sample size is large, Equation (37) can be expressed as

$$I(t)=U(\widehat{\omega },\widehat{\upsilon })+\frac{1}{2}(A+B+C+D),$$

(38)

where $\widehat{\omega }$ and $\widehat{\upsilon }$ are the ML estimators of $\omega$ and $\upsilon$, respectively, and

$$\begin{array}{l}A=({\widehat{U}}_{\omega \omega }+2{\widehat{U}}_{\omega }{\widehat{G}}_{\omega }){\widehat{\sigma }}_{\omega \omega }+({\widehat{U}}_{\upsilon \omega }+2{\widehat{U}}_{\upsilon }{\widehat{G}}_{\omega }){\widehat{\sigma }}_{\upsilon \omega }\\ B=({\widehat{U}}_{\omega \upsilon }+2{\widehat{U}}_{\omega }{\widehat{G}}_{\upsilon }){\widehat{\sigma }}_{\omega \upsilon }+({\widehat{U}}_{\upsilon \upsilon }+2{\widehat{U}}_{\upsilon }{\widehat{G}}_{\upsilon }){\widehat{\sigma }}_{\upsilon \upsilon }\\ C=({\widehat{U}}_{\omega }{\widehat{\sigma }}_{\omega \omega }+{\widehat{U}}_{\upsilon }{\widehat{\sigma }}_{\omega \upsilon })({\widehat{S}}_{\omega \omega \omega }{\widehat{\sigma }}_{\omega \omega }+{\widehat{S}}_{\omega \upsilon \omega }{\widehat{\sigma }}_{\omega \upsilon }+{\widehat{S}}_{\upsilon \omega \omega }{\widehat{\sigma }}_{\upsilon \omega }+{\widehat{S}}_{\upsilon \upsilon \omega }{\widehat{\sigma }}_{\upsilon \upsilon })\\ D=({\widehat{U}}_{\omega }{\widehat{\sigma }}_{\upsilon \omega }+{\widehat{U}}_{\upsilon }{\widehat{\sigma }}_{\upsilon \upsilon })({\widehat{S}}_{\omega \omega \upsilon }{\widehat{\sigma }}_{\omega \omega }+{\widehat{S}}_{\omega \upsilon \upsilon }{\widehat{\sigma }}_{\omega \upsilon }+{\widehat{S}}_{\upsilon \omega \upsilon }{\widehat{\sigma }}_{\upsilon \omega }+{\widehat{S}}_{\upsilon \upsilon \upsilon }{\widehat{\sigma }}_{\upsilon \upsilon })\end{array}$$

(39)

${\sigma }_{ij}$ ($i,j=\omega ,\upsilon$) is the element of the inverse matrix of $-S_{ij}$.

The ${\widehat{U}}_{\omega \omega }$ denotes taking the second derivative of $U(\omega ,\upsilon )$ with respect to $\omega$ and putting $\widehat{\omega }$ into it. Similarly, the others can be expressed as

$$\begin{array}{l}{S}_{\omega \omega \upsilon }=S_{\omega \upsilon \omega }=S_{\upsilon \omega \omega }=0,S_{\omega \omega \omega }=\frac{3n}{{\omega }^3}\\ S_{\omega \upsilon \upsilon }=S_{\upsilon \omega \upsilon }=S_{\upsilon \upsilon \omega }=-{\displaystyle \sum _{i=1}^n{t}_i^{-\upsilon }\ln t_i},S_{\upsilon \upsilon \upsilon }=\frac{2n}{{\upsilon }^3}+\omega {\displaystyle \sum _{i=1}^n{t}_i^{-\upsilon }{(\ln t_i)}^3.}\\ G_{\omega }=\frac{b-1}{\omega }-a,G_{\upsilon }=\frac{1}{\upsilon }\end{array}$$

(40)

Under the SE loss function, the step of numerical calculation of Shannon entropy ${\widehat{H}}_{s2}$ by the Lindley approximation is shown as follows:

When $U(\omega ,\upsilon )=H_s$,

$$\begin{array}{l}{U}_{\omega }=\frac{1}{\omega \upsilon },U_{\upsilon }=\frac{-\gamma -\ln \omega }{{\upsilon }^2}-\frac{1}{\upsilon }\\ U_{\omega \omega }=-\frac{1}{{\omega }^2\upsilon },U_{\upsilon \upsilon }=\frac{2\ln \omega +2\gamma }{{\upsilon }^3}+\frac{1}{{\upsilon }^2},U_{\omega \upsilon }=U_{\upsilon \omega }=-\frac{1}{\omega {\upsilon }^2}\end{array}.$$

(41)

Putting Equations (40) and (41) into Equation (38), $E(H_s|T)$ is obtained.

Similarly, when $U(\omega ,\upsilon )=H_s{}^{-1}$,

$$\begin{array}{l}{U}_{\omega }=-H_s^{-2}\frac{1}{\omega \upsilon },U_{\upsilon }=H_s^{-2}(\frac{\gamma +\ln \omega }{{\upsilon }^2}+\frac{1}{\upsilon })\\ U_{\omega \omega }=2H_s^{-3}\frac{1}{{\omega }^2{\upsilon }^2}+H_s^{-2}\frac{1}{{\omega }^2\upsilon }\\ U_{\upsilon \upsilon }=2H_s^{-3}{(\frac{-\gamma -\ln \omega }{{\upsilon }^2}-\frac{1}{\upsilon })}^2-H_s^{-2}(\frac{2\ln \omega +2\gamma }{{\upsilon }^3}+\frac{1}{{\upsilon }^2})\\ U_{\omega \upsilon }=U_{\upsilon \omega }=2H_s^{-3}(\frac{-\gamma -\ln \omega }{\omega {\upsilon }^3}-\frac{1}{\omega {\upsilon }^2})+H_s^{-2}\frac{1}{\omega {\upsilon }^2}\end{array}.$$

(42)

Then, putting Equations (40) and (42) into Equation (38), $E(H_s^{-1}\left|T\right.)$ is obtained. Thus, the numerical calculation of Shannon entropy ${\widehat{H}}_{s2}$ is calculated by Equation (33).

Under the SE loss function, the numerical calculation of Rényi entropy ${\widehat{H}}_{r2}$ by the Lindley approximation is shown as follows:

When $U(\omega ,\upsilon )=H_r$,

$$\begin{array}{l}{U}_{\omega }=\frac{r}{(1-r)\omega \upsilon },U_{\omega \omega }=\frac{-r}{(1-r){\omega }^2\upsilon },U_{\omega \upsilon }=U_{\upsilon \omega }=\frac{-r}{(1-r)\omega {\upsilon }^2}\\ U_{\upsilon }=\frac{r}{(1-r)\upsilon }[1+\frac{\ln r-\ln \omega }{\upsilon }-\frac{{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})}{\upsilon \mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}]\\ U_{\upsilon \upsilon }=\frac{2r(\ln \omega -\ln r)}{(1-r){\upsilon }^3}-\frac{{[{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})]}^2-{\mathsf{\Gamma }}^{″}(1+r+r{\upsilon }^{-1})\mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}{(1-r)r^{-2}{\upsilon }^4{[\mathsf{\Gamma }(1+r+r{\upsilon }^{-1})]}^2}-\frac{r}{(1-r){\upsilon }^2}\end{array}$$

(43)

Putting Equations (40) and (43) into Equation (38), $E(H_r|T)$ is obtained.

When $U(\omega ,\upsilon )=H_r{}^{-1}$,

$$\begin{array}{cc}{U}_{\omega }& =\frac{-r{H}_r^{-2}}{(1-r)\omega \upsilon },U_{\upsilon }=\frac{-r{H}_r^{-2}}{(1-r)\upsilon }[1+\frac{\ln r-\ln \omega }{\upsilon }-\frac{{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})}{\upsilon \mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}]\hfill \\ U_{\omega \omega }& =\frac{2r^2{H}_r^{-3}}{{(1-r)}^2{\omega }^2{\upsilon }^2}+\frac{r{H}_r^{-2}}{(1-r){\omega }^2\upsilon },\hfill \\ U_{\omega \upsilon }& =U_{\upsilon \omega }=\frac{r^{2}2H_r^{-3}}{{(1-r)}^2\omega {\upsilon }^2}[1+\frac{\ln r-\ln \omega }{\upsilon }-\frac{1}{\upsilon }\frac{{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})}{\mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}]\hfill \\ U_{\upsilon \upsilon }& =\frac{{[{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})]}^2-{\mathsf{\Gamma }}^{″}(1+r+r{\upsilon }^{-1})\mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}{(1-r)r^{-2}{\upsilon }^4{[\mathsf{\Gamma }(1+r+r{\upsilon }^{-1})]}^2{H}_r^2}+\frac{r{H}_r^{-2}}{(1-r){\upsilon }^2}-\frac{2r(\ln \omega -\ln r)}{(1-r){\upsilon }^3{H}_r^2}\hfill \\ & +\frac{2r^2{H}_r^{-3}}{{(1-r)}^2{\upsilon }^2}[1+\frac{\ln r-\ln \omega }{\upsilon }-\frac{{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})}{\upsilon \mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}]\hfill \end{array}$$

(44)

Putting Equations (40) and (44) into Equation (38), $E(H_r^{-1}|T)$ is obtained. Thus, the numerical calculation of Shannon entropy ${\widehat{H}}_{r2}$ is calculated by Equation (34).

4.2. Bayesian Estimation by Using Lindley Approximation under SSE Loss Function

Under the SSE loss function, the step of numerical calculation of Shannon entropy ${\widehat{H}}_{s3}$ by the Lindley approximation is shown as follows:

When $U(\omega ,\upsilon )=H_s^{1-k}$,

$$\begin{array}{l}{U}_{\omega }=(1-k)H_s^{-k}\frac{1}{\omega \upsilon },U_{\upsilon }=(1-k)H_s^{-k}(\frac{-\gamma -\ln \omega }{{\upsilon }^2}-\frac{1}{\upsilon })\\ U_{\omega \omega }=-k(1-k)H_s^{-k-1}{(\frac{1}{\omega \upsilon })}^2-(1-k)H_s^{-k}\frac{1}{{\omega }^2\upsilon }\\ U_{\upsilon \upsilon }=k(1-k)H_s^{-k-1}{(\frac{\gamma +\ln \omega }{{\upsilon }^2}+\frac{1}{\upsilon })}^2+(1-k)H_s^{-k}(\frac{2\ln \omega +2\gamma }{{\upsilon }^3}+\frac{1}{{\upsilon }^2})\\ U_{\omega \upsilon }=U_{\upsilon \omega }=k(1-k)H_s^{-k-1}\frac{1}{\omega \upsilon }(\frac{\gamma +\ln \omega }{{\upsilon }^2}+\frac{1}{\upsilon })-(1-k)H_s^{-k}\frac{1}{\omega {\upsilon }^2}\end{array}.$$

(45)

Then, putting Equations (40) and (45) into Equation (38), $E(H_s^{1-k}\left|T\right.)$ is obtained.

When $U(\omega ,\upsilon )=H_s^{-k}$,

$$\begin{array}{l}{U}_{\omega }=-k{H}_s^{-k-1}\frac{1}{\omega \upsilon },U_{\upsilon }=k{H}_s^{-k-1}(\frac{\gamma +\ln \omega }{{\upsilon }^2}+\frac{1}{\upsilon })\\ U_{\omega \omega }=k(k+1)H_s^{-k-2}{(\frac{1}{\omega \upsilon })}^2+k{H}_s^{-k-1}\frac{1}{{\omega }^2\upsilon }\\ U_{\upsilon \upsilon }=k(k+1)H_s^{-k-2}{(\frac{-\gamma -\ln \omega }{{\upsilon }^2}-\frac{1}{\upsilon })}^2-k{H}_s^{-k-1}(\frac{2\ln \omega +2\gamma }{{\upsilon }^3}+\frac{1}{{\upsilon }^2})\\ U_{\omega \upsilon }=U_{\upsilon \omega }=k(k+1)H_s^{-k-2}\frac{1}{\omega \upsilon }(\frac{-\gamma -\ln \omega }{{\upsilon }^2}-\frac{1}{\upsilon })+k{H}_s^{-k-1}\frac{1}{\omega {\upsilon }^2}\end{array}.$$

(46)

Then, putting Equations (40) and (46) into Equation (38), $E(H_s^{-k}\left|T\right.)$ is obtained. Thus, the numerical calculation of Shannon entropy ${\widehat{H}}_{s3}$ is calculated by Equation (35).

Under the SSE loss function, the step of numerical calculation of Rényi entropy ${\widehat{H}}_{r3}$ by the Lindley approximation is shown as follows:

When $U(\omega ,\upsilon )=H_r^{1-k}$,

$$\begin{array}{cc}{U}_{\omega }& =\frac{(1-k)r{H}_r^{-k}}{(1-r)\omega \upsilon },U_{\upsilon }=\frac{(1-k)r{H}_r^{-k}}{(1-r)\upsilon }[1+\frac{\ln r-\ln \omega }{\upsilon }-\frac{{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})}{\upsilon \mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}]\hfill \\ U_{\omega \omega }& =(k-1)[\frac{k{r}^2{H}_r^{-k-1}}{{(1-r)}^2{\omega }^2{\upsilon }^2}+\frac{r{H}_r^{-k}}{(1-r){\omega }^2\upsilon }]\hfill \\ U_{\omega \upsilon }& =U_{\upsilon \omega }=\frac{k{r}^2{H}_r^{-k-1}(k-1)}{{(1-r)}^2\omega {\upsilon }^2}[1+\frac{\ln r-\ln \omega }{\upsilon }-\frac{{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})}{\upsilon \mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}]+\frac{r{H}_r^{-k}(k-1)}{(1-r)\omega {\upsilon }^2}\hfill \\ U_{\upsilon \upsilon }& =\frac{r^{2}k{H}_r^{-k-1}(k-1)}{{(1-r)}^2{\upsilon }^2}{[1+\frac{\ln r-\ln \omega }{\upsilon }-\frac{{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})}{\upsilon \mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}]}^2+\frac{2r(\ln \omega -\ln r)(1-k)}{(1-r){\upsilon }^3{H}_r^k}\hfill \\ & -\frac{{[{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})]}^2-{\mathsf{\Gamma }}^{″}(1+r+r{\upsilon }^{-1})\mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}{(1-r){(1-k)}^{-1}{r}^{-2}{\upsilon }^4{[\mathsf{\Gamma }(1+r+r{\upsilon }^{-1})]}^2{H}_r^k}-\frac{(1-k)H_r^{-k}r}{(1-r){\upsilon }^2}\hfill \end{array}$$

(47)

Putting Equations (40) and (47) into Equation (38), $E(H_r^{1-k}|T)$ is obtained.

When $U(\omega ,\upsilon )=H_r^{-k}$,

$$\begin{array}{cc}{U}_{\omega }& =-\frac{rk{H}_r^{-k-1}}{(1-r)\omega \upsilon },U_{\upsilon }=\frac{rk{H}_r^{-k-1}}{(1-r)\upsilon }[\frac{{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})}{\upsilon \mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}-1-\frac{\ln r-\ln \omega }{\upsilon }]\hfill \\ U_{\omega \omega }& =\frac{kr}{(1-r){\omega }^2\upsilon }[\frac{r{H}_r^{-k-2}(k-1)}{\upsilon }+H_r^{-k-1}]\hfill \\ U_{\omega \upsilon }& =U_{\upsilon \omega }=\frac{r^{2}k(k-1)H_r^{-k-2}}{{(1-r)}^2\omega {\upsilon }^2}[1+\frac{\ln r-\ln \omega }{\upsilon }-\frac{{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})}{\upsilon \mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}]+\frac{rk{H}_r^{-k-1}}{(1-r)\omega {\upsilon }^2}\hfill \\ U_{\upsilon \upsilon }& =\frac{r^{2}k(k-1)H_r^{-k-2}}{{(1-r)}^2{\upsilon }^2}{[1+\frac{\ln r-\ln \omega }{\upsilon }-\frac{{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})}{\upsilon \mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}]}^2-\frac{2rk(\ln \omega -\ln r)}{(1-r){\upsilon }^3{H}_r^{k+1}}\hfill \\ & +\frac{{[{\mathsf{\Gamma }}^{\prime }(1+r+r{\upsilon }^{-1})]}^2-{\mathsf{\Gamma }}^{″}(1+r+r{\upsilon }^{-1})\mathsf{\Gamma }(1+r+r{\upsilon }^{-1})}{(1-r)r^{-2}{\upsilon }^4{k}^{-1}{H}_r^{k+1}{[\mathsf{\Gamma }(1+r+r{\upsilon }^{-1})]}^2}+\frac{rk{H}_r^{-k-1}}{(1-r){\upsilon }^2}\hfill \end{array}$$

(48)

Putting Equation (40) and Equation (48) into Equation (38), $E(H_r^{-k}\left|T\right.)$ is obtained. Thus, the numerical calculation of Rényi entropy ${\widehat{H}}_{r3}$ is calculated by Equation (36).

5. Monte Carlo Simulation

In this chapter, a Monte Carlo simulation is used to generate random samples that obey the two-parameter IWD, and 1000 experiments are repeated with different sample sizes ($n=10,20,30,40,50,60,70,80,90,100$). The true values of the parameters in the two-parameter IWD are taken as $\omega =1$ and $\upsilon =2$, the parameters of the gamma distribution are taken as $a=5$ and $b=1$, the parameters in SSE are taken as $k=10$, and the parameters of the Rényi entropy are taken as $r=0.5$. Then, the mean squared error (briefly, MSE) is used to compare the performance of each estimator. The results of Shannon entropy are shown in

Table 1. Estimates and MSEs of Shannon entropy ($H_{s0}=1.1727$).

Sample Size ($\mathit{n}$)	Estimate			MSE
	${\widehat{\mathit{M}}}_{\mathit{s}1}$	${\widehat{\mathit{M}}}_{\mathit{s}2}$	${\widehat{\mathit{M}}}_{\mathit{s}3}$	$\mathit{M}\mathit{S}{\mathit{E}}_{\mathit{s}1}$	$\mathit{M}\mathit{S}{\mathit{E}}_{\mathit{s}2}$	$\mathit{M}\mathit{S}{\mathit{E}}_{\mathit{s}3}$
10	1.0604	0.8259	0.8914	0.1903	0.3666	0.2065
20	1.1183	0.9631	0.9683	0.0863	0.1282	0.1186
30	1.1388	1.0301	1.0076	0.0558	0.0751	0.0766
40	1.1355	1.0526	1.0292	0.0445	0.0574	0.0592
50	1.1461	1.0788	1.0379	0.0323	0.0404	0.0472
60	1.1503	1.0938	1.0506	0.0287	0.0343	0.0399
70	1.1579	1.1093	1.0646	0.0244	0.0282	0.0334
80	1.1623	1.1196	1.0694	0.0197	0.0224	0.0284
90	1.1653	1.1272	1.0803	0.0171	0.0191	0.0256
100	1.1628	1.1284	1.0777	0.0161	0.0183	0.0244

and the results of Rényi entropy are shown in

Table 2. Estimates and MSEs of Rényi entropy ($H_{r0}=1.5641$).

Sample Size ($\mathit{n}$)	Estimate			MSE
	${\widehat{\mathit{M}}}_{\mathit{r}1}$	${\widehat{\mathit{M}}}_{\mathit{r}2}$	${\widehat{\mathit{M}}}_{\mathit{r}3}$	$\mathit{M}\mathit{S}{\mathit{E}}_{\mathit{r}1}$	$\mathit{M}\mathit{S}{\mathit{E}}_{\mathit{r}2}$	$\mathit{M}\mathit{S}{\mathit{E}}_{\mathit{r}3}$
10	1.6681	1.7793	1.7682	0.0525	0.1075	0.0954
20	1.6056	1.6512	1.6587	0.0178	0.0218	0.0186
30	1.5999	1.6278	1.6229	0.0129	0.0136	0.0112
40	1.5903	1.6113	1.6082	0.0103	0.0103	0.0075
50	1.5829	1.5992	1.5972	0.0072	0.0071	0.0064
60	1.5809	1.5954	1.5896	0.0055	0.0057	0.0049
70	1.5765	1.5885	1.5878	0.0046	0.0046	0.0045
80	1.5781	1.5886	1.5857	0.0044	0.0041	0.0034
90	1.5752	1.5845	1.5779	0.0038	0.0038	0.0032
100	1.5731	1.5814	1.5775	0.0032	0.0032	0.0031

. For showing the performance of $100(1-\alpha )\%$ ACIs, the coverage probability is calculated and the results are shown in

Table 3. The coverage probability of $100(1-\alpha )\%$ ACIs with different $\alpha$.

Sample Size ($\mathit{n}$)	Shannon Entropy		Rényi Entropy
	$\mathit{\alpha }=0.1$	$\mathit{\alpha }=0.05$	$\mathit{\alpha }=0.1$	$\mathit{\alpha }=0.05$
10	0.9637	0.9752	0.9662	0.9791
20	0.9798	0.9894	0.9789	0.9884
30	0.9829	0.9916	0.9847	0.9930
40	0.9839	0.9941	0.9860	0.9953
50	0.9857	0.9946	0.9894	0.9957
60	0.9876	0.9947	0.9936	0.9954
70	0.9875	0.9947	0.9925	0.9965
80	0.9875	0.9940	0.9929	0.9972
90	0.9894	0.9955	0.9934	0.9966
100	0.9865	0.9950	0.9929	0.9971

.

For convenience, $H_{s0}$ and $H_{r0}$ represent the true values of Shannon entropy and Rényi entropy, respectively; ${\widehat{M}}_{s1}$ and ${\widehat{M}}_{r1}$ represent the mean values of 1000 ML estimates of entropy, respectively; ${\widehat{M}}_{s2}$ and ${\widehat{M}}_{r2}$ represent the mean values of 1000 Bayesian estimates of entropy, respectively, under the SE loss function; ${\widehat{M}}_{s3}$ and ${\widehat{M}}_{r3}$ represent the mean values of 1000 Bayesian estimates of entropy, respectively, under the SSE loss function; $MS{E}_{s1}$ and $MS{E}_{r1}$ represent MSEs of ML estimates of entropy, respectively; $MS{E}_{s2}$ and $MS{E}_{r2}$ represent MSEs of Bayesian estimates of entropy, respectively, under the SE loss function; $MS{E}_{s3}$ and $MS{E}_{r3}$ represent MSEs of Bayesian estimates of entropy, respectively, under the SSE loss function. The ${\widehat{M}}_{sj}$ and $MS{E}_{sj}$($j=1,2,3$) are calculated by Equations (49) and (50), where $m=1000$ and ${\widehat{H}}_{sj,i}$ represents the i-th ML estimate or Bayesian estimate of Shannon entropy. The ${\widehat{M}}_{rj}$ and $MS{E}_{rj}$ ($j=1,2,3$) are calculated by Equations (51) and (52), where $m=1000$ and ${\widehat{H}}_{rj,i}$ represents the i-th ML estimate or Bayesian estimate of Rényi entropy.

$${\widehat{M}}_{sj}=\frac{1}{m}{\displaystyle \sum _{i=1}^m{\widehat{H}}_{sj,i}},$$

(49)

$$MS{E}_{sj}=\frac{1}{m}{\displaystyle \sum _{i=1}^m{({\widehat{H}}_{sj,i}-H_{s0})}^2},$$

(50)

$${\widehat{M}}_{rj}=\frac{1}{m}{\displaystyle \sum _{i=1}^m{\widehat{H}}_{rj,i}},$$

(51)

$$MS{E}_{rj}=\frac{1}{m}{\displaystyle \sum _{i=1}^m{({\widehat{H}}_{rj,i}-H_{r0})}^2}.$$

(52)

Based on the above tables, the following findings of these study results can be drawn:

(1)

For Shannon entropy, the ML estimation performs better than the Bayesian estimation, while for Rényi entropy, the performance of ML estimation is similar to that of Bayesian estimation.

(2)

In Bayesian estimation, it is better to select the SE to estimate Shannon entropy. On the contrary, it is better to select the SSE to estimate Rényi entropy.

(3)

The sample size has a greater influence on Shannon entropy than on Rényi entropy. When the sample size increases gradually, the Bayesian estimation of Shannon entropy under SE is close to the ML estimation, but it has no obvious effect on Rényi entropy.

(4)

In Table 3, it can be noted that the coverage probability of ACIs is quite close to confidence levels.

6. Real Data Analysis

There is a real data set given by Bjerkdal [47] that represents the survival time (in days) of guinea pigs after the injection of different doses of tubercle bacilli. Kundu and Howlader [48] proved that this set of data using the IWD fitting effect is very good; therefore, this data set can be seen as a sample of the IWD. In Reference [47], the regimen number refers to the common logarithm of bacillary units contained in 0.5 mL of challenge solution. In other words, regimen 6.6 represents 4.0 × 10⁶ bacillary units per 0.5 mL. Corresponding to regimen 6.6, the 72 observed observations are listed as follows:

12, 15, 22, 24, 24, 32, 32, 33, 34, 38, 38, 43, 44, 48, 52, 53, 54, 54, 55, 56, 57, 58, 58, 59, 60, 60, 60, 60, 61, 62, 63, 65, 65, 67,68, 70, 70, 72, 73, 75, 76, 76, 81, 83, 84, 85, 87, 91, 95, 96, 98, 99, 109, 110, 121, 127, 129, 131, 143, 146, 146, 175, 175, 211,233, 258, 258, 263, 297, 341, 341, 376.

Using the proposed estimates described in the above sections, the ML estimates and Bayesian estimates of Shannon entropy and Rényi entropy are displayed in

Table 4. The estimates and ACIs of entropies based on the real data set.

	ML Estimates	Bayesian Estimates		$100(1-\mathit{\alpha })\mathit{\%}$ ACIs
		Under SE	Under SSE	$\mathit{\alpha }=0.1$	$\mathit{\alpha }=0.05$
Shannon entropy	5.6307	5.6998	4.8706	(5.1858, 6.0757)	(5.1328, 6.1287)
Rényi entropy	5.4129	4.7280	4.8706	(5.1877, 5.6381)	(5.1609, 5.6649)

. It is obvious that the ML estimates of entropies are all smaller than the Bayesian estimates under SE, and the Bayesian estimates under the SSE of entropies are all smaller than the ML estimates.

7. Conclusions

Entropy is a significant indicator for quantifying information uncertainty. In addition, the IWD is an important lifetime model in the field of reliability. The numerical description of the entropy can be used to improve the lifetime test. Therefore, this paper considers the Bayesian estimations of Shannon entropy and Rényi entropy based on the two-parameter IWD. First, the expressions of these entropies of the two-parameter IWD are derived in Theorem 1. For ML estimation, due to the invariance of ML estimation, the ML estimators of parameters are obtained by the dichotomy method at first. Then, the ML estimators of entropies can be obtained. Additionally, the approximate confidence intervals are given by the Delta method. For Bayesian estimation, the symmetric entropy loss function and scale squared error loss function are chosen. However, the forms of Bayesian estimators are complex and difficult to calculate. The Lindley approximation is used to solve this problem. Finally, the mean square errors of the above estimators are used to compare their performances. From the findings found in Section 5, it is better to use the ML estimator for Shannon entropy, and for Rényi entropy, the performances of the ML estimator and Bayesian estimator are analogous. Based on these findings, it is hoped that it will be helpful for researchers conducting lifetime tests.