Empirical Bayes Estimators for Mean Parameter of Exponential Distribution with Conjugate Inverse Gamma Prior Under Stein’s Loss

Abstract

A Bayes estimator for a mean parameter of an exponential distribution is calculated using Stein’s loss, which equally penalizes gross overestimation and underestimation. A corresponding Posterior Expected Stein’s Loss (PESL) is also determined. Additionally, a Bayes estimator for a mean parameter is obtained under a squared error loss along with its corresponding PESL. Furthermore, two methods are used to derive empirical Bayes estimators for the mean parameter of the exponential distribution with an inverse gamma prior. Numerical simulations are conducted to illustrate five aspects. Finally, theoretical studies are illustrated using Static Fatigue 90% Stress Level data.

1. Introduction

The Empirical Bayes analysis uses a conjugate prior model, estimating hyperparameters from observations and using this estimate as a conventional prior for further inference [1,2]. It should be noted that prior distribution remains unknown because of the unknown hyperparameters. However, marginal distribution could be employed to recover the prior distribution based on observed data. For further literature about empirical Bayes, researchers are referred to [1,3,4,5,6,7,8,9,10].

The motivations behind this paper could be summarized below. Firstly, to our knowledge, no one in the empirical Bayes analysis literature has considered the exponential-inverse gamma model. Additionally, as highlighted by [11], when dealing with a parameter space $\mathrm{\Theta }=\left(-\infty ,\infty \right)$ in the location case, it is appropriate that the squared error loss equally penalizes both overestimation and underestimation. However, in cases where we have a restricted positive space of parameters $\mathrm{\Theta }=\left(0,\infty \right)$ with lower bound setting at 0 as well as an asymmetric problem of estimation, selecting Stein’s loss [12,13] instead of squared error loss is more suitable since it equally penalizes both gross overestimation and underestimation—actions that tend towards 0 or ∞ will incur infinite losses. In our specific context of the exponential-inverse gamma model (1), where our interested parameter is ${\theta }_{n+1}>0$ representing a mean parameter value, we opt for Stein’s loss. Relevant literature on Stein’s loss can be found for interested readers [9,14,15,16].

The key findings of this paper are outlined below. First of all, the Bayes estimator of the mean parameter for the exponential distribution is derived under Stein’s loss, and its Posterior Expected Stein’s Loss (PESL) is calculated within an exponential-inverse gamma model. Additionally, we obtain a Bayes estimator of a mean parameter using squared error loss as well as its associated PESL. Furthermore, we employ two methods, namely the method of moments and the method of Maximum Likelihood Estimation (MLE), to estimate a mean parameter of an exponential distribution, which is assumed with a prior distribution of inverse gamma in an empirical Bayes framework. In our simulations, we investigate five aspects. The simulations suggest that MLEs are superior to moment estimators in estimating hyperparameters. At last, we illustrate our theoretical studies by applying them to analyze Static Fatigue 90% Stress Level data.

The remaining part of the paper is structured below. In Section 2, four theorems will be presented. Section 3 will provide an illustration of five aspects in simulations. In Section 4, we employ the Static Fatigue 90% Stress Level data to demonstrate the empirical Bayes estimators’ calculations for a mean parameter of an exponential distribution with an inverse gamma prior. Finally, Section 5 offers conclusions and discussions.

2. Main Results

The exponential-inverse gamma model is assumed to generate observations $X_1,X_2,\dots ,X_{n+1}$ in our study:

$$\left\{\begin{array}{c}{X}_i\left|{\theta }_i\right.\stackrel{\mathrm{independent}}{\sim }Exp\left({\theta }_i\right),\hfill \\ {\theta }_i\stackrel{\mathrm{iid}}{\sim }\widetilde{IG}(\alpha ,\beta ),i=1,2,\dots ,n+1.\hfill \end{array}\right.$$

(1)

The hyperparameters $\alpha >0$ and $\beta >0$ need to be estimated, while ${\theta }_{n+1}>0$ represents a parameter of interest that is unknown. $Exp\left({\theta }_i\right)$ denotes an exponential distribution with a mean parameter of ${\theta }_i$, and $\widetilde{IG}\left(\alpha ,\beta \right)$ denotes an inverse gamma distribution characterized by two hyperparameters. According to [1], when observing data $(x_1,x_2,\dots ,x_{n+1})={\mathit{x}}^{(n+1)}$, statisticians aim to make inferences about ${\theta }_{n+1}$. Therefore, $x_{n+1}$ directly provides information regarding the parameter ${\theta }_{n+1}$, while Supplementary Information $(x_1,x_2,\dots ,x_n)={\mathit{x}}^{\left(n\right)}$ also offered. A relationship between the primary data $x_{n+1}$ as well as Supplementary Information ${\mathit{x}}^{\left(n\right)}$ is established through their common distributions $Exp\left({\theta }_i\right)$ and $\widetilde{IG}(\alpha ,\beta )$.

There are two probability density function (pdf) forms for the exponential distribution. One form uses the mean (or scale) $\theta$ as the parameter (see [11]), and $X\sim Exp\left(\theta \right)$ with pdf $f_X\left(x|\theta \right)={\displaystyle \frac{1}{\theta }}exp\left(-{\displaystyle \frac{x}{\theta }}\right)$, for $x>0$ and $\theta >0$. Another form utilizes the rate $\lambda$ as the parameter (see [2,17]), and $X\sim \widetilde{Exp}\left(\lambda \right)$ with pdf $f_X\left(x|\lambda \right)=\lambda exp\left(-\lambda x\right)$, for $x>0$ and $\lambda >0$. The two pdfs are the same with a relationship of the parameters $\theta =1/\lambda$.

Other distributions that have two alternative forms are the gamma and inverse gamma distributions. Similar to [18], suppose that $X\sim G\left(\alpha ,\beta \right)$ and $Y=1/X\sim IG\left(\alpha ,\beta \right)$. The pdfs of X and Y are, respectively, given by

$$\begin{array}{cc}\hfill f_X\left(x|\alpha ,\beta \right)& ={\displaystyle \frac{1}{\Gamma \left(\alpha \right){\beta }^{\alpha }}}{x}^{\alpha -1}exp\left(-{\displaystyle \frac{x}{\beta }}\right),x>0,\alpha ,\beta >0,\hfill \\ \hfill f_Y\left(y|\alpha ,\beta \right)& ={\displaystyle \frac{1}{\Gamma \left(\alpha \right){\beta }^{\alpha }}}{\left({\displaystyle \frac{1}{y}}\right)}^{\alpha +1}exp\left(-{\displaystyle \frac{1}{\beta y}}\right),y>0,\alpha ,\beta >0.\hfill \end{array}$$

Moreover, suppose that $\tilde{X}\sim \tilde{G}\left(\alpha ,\beta \right)$ and $\tilde{Y}=1/\tilde{X}\sim \widetilde{IG}\left(\alpha ,\beta \right)$. The pdfs of $\tilde{X}$ and $\tilde{Y}$ are, respectively, given by

$$\begin{array}{cc}\hfill f_{\tilde{X}}\left(x|\alpha ,\beta \right)& ={\displaystyle \frac{{\beta }^{\alpha }}{\Gamma \left(\alpha \right)}}{x}^{\alpha -1}exp\left(-\beta x\right),x>0,\alpha ,\beta >0,\hfill \\ \hfill f_{\tilde{Y}}\left(y|\alpha ,\beta \right)& ={\displaystyle \frac{{\beta }^{\alpha }}{\Gamma \left(\alpha \right)}}{\left({\displaystyle \frac{1}{y}}\right)}^{\alpha +1}exp\left(-{\displaystyle \frac{\beta }{y}}\right),y>0,\alpha ,\beta >0.\hfill \end{array}$$

The exponential-gamma model is assumed to generate observations $X_1,X_2,\dots ,X_{n+1}$:

$$\left\{\begin{array}{c}{X}_i\left|{\lambda }_i\right.\stackrel{\mathrm{independent}}{\sim }\widetilde{Exp}\left({\lambda }_i\right),\hfill \\ {\lambda }_i\stackrel{\mathrm{iid}}{\sim }\tilde{G}(\alpha ,\beta ),i=1,2,\dots ,n+1.\hfill \end{array}\right.$$

(2)

Firstly, the exponential-inverse gamma model (1) and the exponential-gamma model (2) are equivalent in the sense of their marginal pdfs. For convenience, now let $Exp\left(\theta \right)$, $\widetilde{Exp}\left(\lambda \right)$, $G\left(\alpha ,\beta \right)$, $IG\left(\alpha ,\beta \right)$, $\tilde{G}\left(\alpha ,\beta \right)$, and $\widetilde{IG}\left(\alpha ,\beta \right)$ be random variables with the corresponding distributions. For example, $Exp\left(\theta \right)$ is a random variable with the $Exp\left(\theta \right)$ distribution. It is easy to see that

$$Exp\left(\theta \right)=G\left(1,\theta \right)=\tilde{G}\left(1,{\displaystyle \frac{1}{\theta }}\right)=\tilde{G}\left(1,\lambda \right)=\widetilde{Exp}\left(\lambda \right)=\widetilde{Exp}\left({\displaystyle \frac{1}{\theta }}\right).$$

Therefore, the two hierarchical models (1) and (2) are equivalent. It is straightforward to derive that the two marginal pdfs of the two hierarchical models are the same, and they are equal to

$$m\left(x|\alpha ,\beta \right)={\displaystyle \frac{\alpha {\beta }^{\alpha }}{{\left(\beta +x\right)}^{\alpha +1}}},x>0,\alpha ,\beta >0.$$

Since the two marginal pdfs are the same, the moment estimators (displayed in Theorem 2) and the MLEs (see Theorem 3) of the hyperparameters $\alpha$ and $\beta$ for the two hierarchical models (1) and (2) are the same.

Another reason to use (1) is that it motivates us to consider 16 hierarchical models of the gamma and inverse gamma distributions (see [18]). It is easy to see that $\widetilde{IG}(\alpha ,\beta )$ is a conjugate prior for the $Exp\left(\theta \right)=G\left(1,\theta \right)$ distribution. Writing in the form of likelihood-prior, it is $G\left(1,\theta \right)-\widetilde{IG}(\alpha ,\beta )$. Similarly, the $\tilde{G}(\alpha ,\beta )$ is a conjugate prior for the $\widetilde{Exp}\left(\lambda \right)=\tilde{G}\left(1,\lambda \right)$ distribution. Writing in the form of likelihood-prior, it is $\tilde{G}\left(1,\lambda \right)-\tilde{G}(\alpha ,\beta )$. The $G\left(1,\theta \right)-\widetilde{IG}(\alpha ,\beta )$ expression motivates us to consider $G\left(\nu ,\theta \right)$, $\tilde{G}\left(\nu ,\theta \right)$, $IG\left(\nu ,\theta \right)$, and $\widetilde{IG}\left(\nu ,\theta \right)$ as the likelihood, and $G\left(\alpha ,\beta \right)$, $\tilde{G}\left(\alpha ,\beta \right)$, $IG\left(\alpha ,\beta \right)$, and $\widetilde{IG}\left(\alpha ,\beta \right)$ as the prior, leading to 16 combinations of the likelihood-prior.

The introduction highlights the importance of calculating and utilizing a Bayes estimator for a mean parameter ${\theta }_{n+1}$ under Stein’s loss, considering a prior $\widetilde{IG}(\alpha ,\beta )$. This particular estimator is denoted as ${\delta }_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$. The supplement provides further justifications for why Stein’s loss outperforms squared error loss on a restricted positive space of parameters $\mathrm{\Theta }=\left(0,\infty \right)$. For clarity, Stein’s loss function is given by

$$L_s\left(\theta ,a\right)={\displaystyle \frac{a}{\theta }}-1-log{\displaystyle \frac{a}{\theta }},$$

while the squared error loss function is given by

$$L_2\left(\theta ,a\right)={\left(\theta -a\right)}^2.$$

2.1. Bayes Estimators and PESLs

In the exponential-inverse gamma model, the following theorem provides a summary of the posterior of ${\theta }_{n+1}$ and a marginal distribution of X. The proof of the theorem can be found in the supplement.

Theorem 1. 

The posterior of ${\theta }_{n+1}$ for the exponential-inverse gamma model is obtained as

$$\pi \left({\theta }_{n+1}|x_{n+1}\right)\sim \widetilde{IG}\left(\alpha *,\beta *\right),$$

(3)

in which

$$\alpha *=\alpha +1>1and\beta *=\beta +x_{n+1}>0.$$

(4)

The marginal distribution of X, furthermore, is

$$m\left(x|\alpha ,\beta \right)=\alpha {\beta }^{\alpha }{\left(\beta +x\right)}^{-\left(\alpha +1\right)},\mathit{for}x>0\mathit{and}\alpha ,\beta >0.$$

(5)

Two inequalities will be derived for any hierarchical model, assuming the existence of posterior expectations. The same as [15], under Stein’s loss, a Bayes estimator of ${\theta }_{n+1}$ is

$${\delta }_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)=\mathrm{E}{\left({\displaystyle \frac{1}{{\theta }_{n+1}}}|x_{n+1}\right)}^{-1},$$

and under squared error loss, a Bayes estimator of ${\theta }_{n+1}$ is

$${\delta }_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)=\mathrm{E}\left({\theta }_{n+1}|x_{n+1}\right).$$

The same as [15], we can show that

$${\delta }_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)\le {\delta }_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$$

(6)

by leveraging the application of Jensen’s inequality. The same as [15], the PESL at ${\delta }_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ is

$$PES{L}_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)=log\mathrm{E}\left({\displaystyle \frac{1}{{\theta }_{n+1}}}|x_{n+1}\right)+\mathrm{E}\left(log{\theta }_{n+1}|x_{n+1}\right).$$

The PESL at ${\delta }_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ is

$$\begin{array}{cc}\hfill PES{L}_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)& =\mathrm{E}\left({\theta }_{n+1}|x_{n+1}\right)\mathrm{E}\left({\displaystyle \frac{1}{{\theta }_{n+1}}}|x_{n+1}\right)-1\hfill \\ \hfill & -log\mathrm{E}\left({\theta }_{n+1}|x_{n+1}\right)+\mathrm{E}\left(log{\theta }_{n+1}|x_{n+1}\right).\hfill \end{array}$$

The following should be noted

$$PES{L}_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)\le PES{L}_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right).$$

(7)

This is a straightforward result of a method for deriving a Bayes estimator, because ${\delta }_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ is constructed to minimize the PESL.

The Bayes estimators ${\delta }_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ and ${\delta }_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$, as well as the PESLs $PES{L}_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ and $PES{L}_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$, are analytically calculated under the exponential-inverse gamma model. According to (3), we possess

$${\theta }_{n+1}|x_{n+1}\sim \widetilde{IG}\left(\alpha *,\beta *\right)\mathrm{and}{\displaystyle \frac{1}{{\theta }_{n+1}}}|x_{n+1}\sim \tilde{G}\left(\alpha *,\beta *\right).$$

The Bayes estimator of ${\theta }_{n+1}$ is elegantly expressed by the following equation under Stein’s loss (reminiscent of [15])

$${\delta }_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)={\displaystyle \frac{\beta *}{\alpha *}},$$

(8)

in which $\alpha *$ and $\beta *$ are determined by (4). Additionally, under squared error loss, we obtain

$${\delta }_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)={\displaystyle \frac{\beta *}{\alpha *-1}},\mathrm{for}\alpha *>1.$$

(9)

It can be observed that

$${\delta }_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)={\displaystyle \frac{\beta *}{\alpha *}}<{\displaystyle \frac{\beta *}{\alpha *-1}}={\delta }_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right),$$

which illustrates the theoretical investigation in (6). The analytical calculation of $PES{L}_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ and $PES{L}_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ requires the analytical calculation of $\mathrm{E}\left(\log {\theta }_{n+1}|x_{n+1}\right)$, which is given by

$$\mathrm{E}\left(\log {\theta }_{n+1}|x_{n+1}\right)=log\beta *-\psi \left(\alpha *\right),$$

in which $\psi \left(x\right)$ is a digamma function. The PESL at ${\delta }_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ and ${\delta }_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ can be obtained through algebraic operations, similar to the approach presented in [15],

$$PES{L}_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)=log\alpha *-\psi \left(\alpha *\right),$$

(10)

and

$$PES{L}_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)={\displaystyle \frac{1}{\alpha *-1}}+log(\alpha *-1)-\psi \left(\alpha *\right),\mathrm{for}\alpha *>1.$$

(11)

The demonstration of this fact is straightforward

$$\begin{array}{cc}\hfill PES{L}_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)& =log\alpha *-\psi \left(\alpha *\right)\hfill \\ \hfill & \le log(\alpha *-1)-\psi \left(\alpha *\right)+{\displaystyle \frac{1}{\alpha *-1}}=PES{L}_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right).\hfill \end{array}$$

The sentence exemplifies the theoretical investigation of (7). Notice that the PESLs solely depend on $\alpha *$, disregarding $\beta *$. Hence, the dependency of the PESLs lies exclusively on $\alpha$, without considering $\beta$ and $x_{n+1}$. In both simulations and real data sections, we will provide examples illustrating (6) and (7). Furthermore, we shall demonstrate that the dependence of the PESLs is solely on $\alpha$, excluding any influence from $\beta$ and $x_{n+1}$.

The Bayes estimators, as well as the PESLs considered in this subsection, assume prior knowledge of the hyperparameters $\alpha$ and $\beta$. In other words, the PESLs and Bayes estimators presented here are computed using the oracle method, which will be further discussed in Section 2.3.

2.2. Empirical Bayes Estimators of θ_n+1

Empirical Bayes estimators of ${\theta }_{n+1}$ can be obtained by estimating the hyperparameters from the Supplementary Information $\left(x_1,x_2,\dots ,x_n\right)={\mathit{x}}^{\left(n\right)}$. There are two commonly used methods for estimating the hyperparameters: the method of moments and the method of MLE.

The method of moments offers estimators for the hyperparameters of the exponential-inverse gamma model, and their consistencies are succinctly summarized below. The proof of the theorem can be found in the supplement.

Theorem 2. 

The hyperparameters of an exponential-inverse gamma model can be estimated using the method of moments, where

$$\begin{array}{cc}\hfill {\alpha }_1\left(n\right)& ={\displaystyle \frac{2\left(A_2-A_1^2\right)}{A_2-2A_1^2}},\hfill \\ \hfill {\beta }_1\left(n\right)& ={\displaystyle \frac{A_1{A}_2}{A_2-2A_1^2}}.\hfill \end{array}$$

Here, $A_k={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{X}_i^k$, $k=1,2$ represents X’s kth sample moment. Additionally, it should be noted that the moment estimators exhibit consistency as estimators for these hyperparameters.

The estimators of hyperparameters for the exponential-inverse gamma model, obtained through the method of MLE, are presented below, along with an overview of their consistent properties. The proof of the theorem can be found in the supplement.

Theorem 3. 

The hyperparameters’ estimators of an exponential-inverse gamma model obtained through the method of MLE, denoted as ${\alpha }_2\left(n\right)$ and ${\beta }_2\left(n\right)$, are determined by solving equations:

$$\begin{array}{cc}\hfill f_1\left(\alpha ,\beta \right)& ={\alpha }^{-1}+log\beta -{\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}log(\beta +x_i)=0,\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill f_2\left(\alpha ,\beta \right)& =\alpha {\beta }^{-1}-(\alpha +1){\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{\left(\beta +x_i\right)}^{-1}=0.\hfill \end{array}$$

(13)

Furthermore, it is worth noting that these MLEs serve as consistent estimators.

It is infeasible to analytically calculate the MLEs for $\alpha$ and $\beta$ through solving nonlinear Equations (12) and (13), necessitating the utilization of numerical solutions. Newton’s method can be employed to solve Equations (12) and (13) and find the MLEs. It should be noted that the MLEs exhibit a high degree of sensitivity to initial estimates; thus, moment estimators are commonly utilized as reliable initial estimates.

The estimators of the parameter in the exponential-inverse gamma model are enhanced through empirical Bayes methodology, obtained through both moment and MLE methods, and they are summarized below.

Theorem 4. 

Under Stein’s loss and using the method of moments, the empirical Bayes estimator of the parameter in the exponential-inverse gamma model is (8) with estimated hyperparameters according to Theorem 2. Alternatively, when employing the MLE method and under Stein’s loss, the empirical Bayes estimator of the parameter in the exponential-inverse gamma model is (8) with numerically determined hyperparameters as stated in Theorem 3.

Remark 1. 

In Theorem 4 and under Stein’s loss, the empirical Bayes estimator of the parameter in the exponential-inverse gamma model is given by (8), where the hyperparameters $(\alpha ,\beta )$ are estimated from observed data through marginal likelihood. More specifically, using the method of moments, the hyperparameters are estimated from observed data according to Theorem 2. Alternatively, when employing the MLE method, the hyperparameters are numerically determined from observed data as stated in Theorem 3.

2.3. Theoretical Analysis of PESLs and Bayes Estimators of Three Methods

PESLs and Bayes estimators for three methods (oracle, moment, and MLE) will be theoretically compared in this subsection. It is worth noting that their numerical comparisons can be found in the supplement.

Subscripts 0, 1, and 2 correspond to oracle, moment, and MLE, respectively. Following calculations as before [9,15], the PESLs of these three methods are given by ($i=0,1,2$)

$$PES{L}_i\left(a|{\mathit{x}}^{\left(n+1\right)}\right)=a{\mathrm{E}}_i\left({\displaystyle \frac{1}{{\theta }_{n+1}}}|{\mathit{x}}^{\left(n+1\right)}\right)-1-loga+{\mathrm{E}}_i\left(log{\theta }_{n+1}|{\mathit{x}}^{\left(n+1\right)}\right),$$

in which

$${\mathrm{E}}_i\left({\displaystyle \frac{1}{{\theta }_{n+1}}}|{\mathit{x}}^{\left(n+1\right)}\right)={\displaystyle \frac{{\alpha }_i^{*}\left(n\right)}{{\beta }_i^{*}\left(n\right)}}={\displaystyle \frac{{\alpha }_i\left(n\right)+1}{{\beta }_i\left(n\right)+x_{n+1}}},$$

$$\begin{array}{cc}\hfill {\mathrm{E}}_i\left(log{\theta }_{n+1}|{\mathit{x}}^{\left(n+1\right)}\right)& =log{\beta }_i^{*}\left(n\right)-\psi \left({\alpha }_i^{*}\left(n\right)\right)\hfill \\ & =log\left({\beta }_i\left(n\right)+x_{n+1}\right)-\psi \left({\alpha }_i\left(n\right)+1\right),\hfill \end{array}$$

$$\begin{array}{cc}\hfill {\alpha }_0^{*}\left(n\right)& ={\alpha }_0\left(n\right)+1=\alpha +1={\alpha }^{*},\hfill \\ \hfill {\beta }_0^{*}\left(n\right)& ={\beta }_0\left(n\right)+x_{n+1}=\beta +x_{n+1}={\beta }^{*},\hfill \\ \hfill {\alpha }_1^{*}\left(n\right)& ={\alpha }_1\left(n\right)+1,\hfill \\ \hfill {\beta }_1^{*}\left(n\right)& ={\beta }_1\left(n\right)+x_{n+1},\hfill \\ \hfill {\alpha }_2^{*}\left(n\right)& ={\alpha }_2\left(n\right)+1,\hfill \\ \hfill {\beta }_2^{*}\left(n\right)& ={\beta }_2\left(n\right)+x_{n+1}.\hfill \end{array}$$

The hyperparameters are unknown, while the moment estimators of $\alpha$ and $\beta$ in Theorem 2, and the numerically determined MLEs of $\alpha$ and $\beta$ in Theorem 3, are presented.

Under Stein’s loss, a Bayes estimators of ${\theta }_{n+1}$ minimize an associated PESL (see [1,11,15]), i.e.,

$${\delta }_{s,i}^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)=arg\underset{a\in \mathcal{A}}{min}{\mathrm{E}}_i\left[L_s\left({\theta }_{n+1},a\right)|{\mathit{x}}^{\left(n+1\right)}\right],$$

in which $\mathcal{A}=\left\{a=a\left({\mathit{x}}^{\left(n+1\right)}\right):a\left({\mathit{x}}^{\left(n+1\right)}\right)>0\right\}$ represents a space of actions. Herein a is an action (estimator), $L_s\left({\theta }_{n+1},a\right)$ denotes Stein’s loss, and ${\theta }_{n+1}>0$ refers to the unknown interested parameter. The Bayes estimators can be easily derived in a manner similar to [15]:

$${\delta }_{s,i}^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)={\displaystyle \frac{{\beta }_i\left(n\right)+x_{n+1}}{{\alpha }_i\left(n\right)+1}}.$$

The PESLs at ${\delta }_{s,i}^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$ for $i=0,1,2$ exhibit similarities to those discussed in Zhang’s work [15]:

$$PES{L}_{s,i}^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)=log\left({\alpha }_i\left(n\right)+1\right)-\psi \left({\alpha }_i\left(n\right)+1\right).$$

Under squared error loss, a Bayes estimators of ${\theta }_{n+1}$ are provided in a similar manner as described by [15]:

$${\delta }_{2,i}^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)={\displaystyle \frac{{\beta }_i\left(n\right)+x_{n+1}}{{\alpha }_i\left(n\right)}},$$

for ${\alpha }_i\left(n\right)>0,i=0,1,2$. Additionally, the PESLs at ${\delta }_{2,i}^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$ can be derived in a manner similar to that of [15] for $i=0,1,2$:

$$PES{L}_{2,i}^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)={\displaystyle \frac{1}{{\alpha }_i\left(n\right)}}+log{\alpha }_i\left(n\right)-\psi \left({\alpha }_i\left(n\right)+1\right).$$

The primary objectives of our study involve estimating the PESLs and Bayes estimators using the method of oracle. However, the hyperparameters remain unknown. The method of oracle possesses knowledge of these hyperparameters only in simulations, whereas in reality, they are undisclosed.

The positive aspect is that the PESLs and Bayes estimators can be obtained through the method of moments and method of MLE once the data ${\mathit{x}}^{\left(n\right)}=\left(x_1,\dots ,x_n\right)$ are provided. To compare the moment and MLE methods in simulations, we can assess the PESLs and Bayes estimators generated by these two methods against those obtained using the method of oracle. In simulations, the method that yields PESLs and Bayes estimators closer to those derived from the method of oracle is considered superior.

3. Simulations

We conduct simulations to examine the exponential-inverse gamma model (1) in this section. We present five key aspects of our findings. Firstly, we demonstrate an inequality (6) of Bayes estimators and an inequality (7) of PESLs using the method of oracle. Secondly, we show that both moment estimators and MLEs consistently estimate hyperparameters. Thirdly, we calculate the model’s goodness-of-fit with respect to simulated data. Fourthly, we compare numerically the PESLs and Bayes estimators obtained from three methods: oracle, moment, and MLE. Finally, the model’s marginal densities are plotted for different hyperparameters.

The simulated data are produced based on the model (1), with hyperparameters set as $\alpha =3$ and $\beta =2$. These values are selected because the conditions $\alpha >2$ and $\beta >0$ are necessary for the moment-based estimation of the hyperparameters. Alternative numerical values for the hyperparameters can also be defined.

3.1. Two Inequalities Related to Bayes Estimators and PESLs

In this subsection, we aim to demonstrate numerically the two inequalities (6) and (7) related to Bayes estimators and PESLs using the oracle approach.

First, we fix $\alpha =3$, $\beta =2$, and $x_{n+1}=1$. Figure 1 presents the histogram for ${\theta }_{n+1}|x_{n+1}$ along with the estimated density curve of $\pi \left({\theta }_{n+1}\right|x_{n+1})$. Our goal is to determine ${\delta }_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ in a way that minimizes the PESL based on $\pi \left({\theta }_{n+1}\right|x_{n+1})$. Numerical results show that

$${\delta }_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)=0.75<1={\delta }_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$$

and

$$PES{L}_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)=0.1301767<0.1758280=PES{L}_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right),$$

which exemplify the theoretical studies of (6) and (7).

Next, consider allowing one of the variables $\alpha$, $\beta$, or $x_{n+1}$ to vary while keeping the others constant. As depicted in Figure 2, both the Bayes estimators and the PESLs are represented as functions of $\alpha$, $\beta$, and $x_{n+1}$. From the left panels of the figure, it is evident that the Bayes estimators are influenced by $\alpha$, $\beta$, and $x_{n+1}$, illustrating the relationship (6). More precisely, the Bayes estimators decrease with increasing $\alpha$, increase linearly with $\beta$, and also increase linearly with $x_{n+1}$. On the other hand, the right panels reveal that the PESLs are solely dependent on $\alpha$, showing no dependence on $\beta$ or $x_{n+1}$, thus exemplifying the inequality (7). Specifically, the PESLs exhibit a decreasing trend as $\alpha$ increases. In conclusion, the findings presented in Figure 2 validate the theoretical analyses outlined in (6) and (7).

Third, given that the Bayes estimators ${\delta }_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ and ${\delta }_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$, as well as the PESLs $PES{L}_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ and $PES{L}_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$, rely on ${\alpha }^{*}$ and ${\beta }^{*}$, where ${\alpha }^{*}>1$ and ${\beta }^{*}>0$, we can visualize their surfaces over the domain $\left({\alpha }^{*},{\beta }^{*}\right)\in (1,10]\times (0,10]=D$ using the ‘persp3d()’ function from the R (1.0) package ‘rgl’ (see [16,19]). It is worth noting that while the ‘persp()’ function in the R package ‘graphics’ cannot overlay additional surfaces onto an existing one, ‘persp3d()’ has this capability. Furthermore, ‘persp3d()’ enables users to rotate the perspective plots of the surfaces according to their preferences. Figure 3 displays the surfaces of the Bayes estimators and the PESLs, along with the surfaces representing the differences between these estimators and PESLs. From the first two plots on the left, it is evident that ${\delta }_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)<{\delta }_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ for all $\left({\alpha }^{*},{\beta }^{*}\right)$ in D. Similarly, the last two plots on the right indicate that $PES{L}_s^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)<PES{L}_2^{\pi ,{\theta }_{n+1}}\left(x_{n+1}\right)$ for all $\left({\alpha }^{*},{\beta }^{*}\right)$ in D. The findings presented in Figure 3 align with the theoretical results outlined in (6) and (7).

3.2. Consistencies of Moment Estimators and MLEs

In this subsection, we aim to demonstrate through numerical examples that both the moment estimators and the MLEs serve as consistent estimators for the hyperparameters $\alpha$ and $\beta$ of the hierarchical exponential and inverse gamma model (1). It is important to note that in this subsection, only the data ${\mathit{x}}^{\left(n\right)}=\left(x_1,x_2,\dots ,x_n\right)$ are utilized.

The estimators for the hyperparameters ($\alpha$ and $\beta$) based on the method of moments and MLE are denoted as (${\alpha }_1\left(n\right)$ and ${\beta }_1\left(n\right)$) and (${\alpha }_2\left(n\right)$ and ${\beta }_2\left(n\right)$), respectively. Consistency implies that ${\alpha }_i\left(n\right)$ converges to $\alpha$ in probability as n approaches infinity, and ${\beta }_i\left(n\right)$ converges to $\beta$ in probability as n approaches infinity, for $i=1,2$. Here, $i=1$ corresponds to the moment estimator, and $i=2$ corresponds to the MLE.

Let $\theta$ represent either hyperparameter $\alpha$ or $\beta$. Then, consistency can be expressed as ${\theta }_i\left(n\right)$ converging to $\theta$ in probability as n tends to infinity, for $i=1,2$. Alternatively, consistency means that for every $\epsilon >0$ and every $\theta \in \mathrm{\Theta }$, the following holds:

$$\underset{n\to \infty }{lim}P\left(\left|{\theta }_i\left(n\right)-\theta \right|\ge \epsilon \right)=0,$$

The probabilities can be approximated by their corresponding frequencies:

$$P\left(\left|{\theta }_i\left(n\right)-\theta \right|\ge \epsilon \right)\approx {\displaystyle \frac{1}{M}}\sum _{m=1}^{M}I\left(\left|{\theta }_i^{\left(m\right)}\left(n\right)-\theta \right|\ge \epsilon \right)\triangleq F_{{\theta }_i}^{\epsilon ,M}\left(n\right),$$

for $i=1,2$, where $I\left(A\right)$ is the indicator function of A, equaling 1 if A is true and 0 otherwise, and M represents the number of simulations. Thus, the convergence of the frequencies ($F_{{\theta }_i}^{\epsilon ,M}\left(n\right)$, with $\theta$ being $\alpha$ or $\beta$, and $i=1,2$) toward 0 indicates that the estimators are consistent.

The occurrences of the moment estimators (${\alpha }_1\left(n\right)$ and ${\beta }_1\left(n\right)$) and the MLEs (${\alpha }_2\left(n\right)$ and ${\beta }_2\left(n\right)$) for the hyperparameters ($\alpha$ and $\beta$) as n changes, given $M=100$ and $\epsilon =1$, $0.5$, and $0.1$, are presented in

Table 1. The frequencies of the moment estimators and the MLEs for the hyperparameters are examined as n changes, with M set to 100 and $\epsilon$ taking values of 1, 0.5, and 0.1.

		Moment Estimators		MLEs
	n	${\mathit{F}}_{{\mathit{\alpha }}_{\mathbf{1}}}^{\mathit{\epsilon },\mathit{M}}\left(\mathit{n}\right)$	${\mathit{F}}_{{\mathit{\beta }}_{\mathbf{1}}}^{\mathit{\epsilon },\mathit{M}}\left(\mathit{n}\right)$	${\mathit{F}}_{{\mathit{\alpha }}_{\mathbf{2}}}^{\mathit{\epsilon },\mathit{M}}\left(\mathit{n}\right)$	${\mathit{F}}_{{\mathit{\beta }}_{\mathbf{2}}}^{\mathit{\epsilon },\mathit{M}}\left(\mathit{n}\right)$
ε = 1	1 × 104	0.00	0.00	0.00	0.00
	2 × 104	0.00	0.00	0.00	0.00
	4 × 104	0.00	0.00	0.00	0.00
	8 × 104	0.00	0.00	0.00	0.00
ε = 0.5	1 × 104	0.14	0.11	0.00	0.00
	2 × 104	0.02	0.01	0.00	0.00
	4 × 104	0.01	0.01	0.00	0.00
	8 × 104	0.00	0.00	0.00	0.00
ε = 0.1	1 × 104	0.77	0.74	0.40	0.31
	2 × 104	0.70	0.69	0.24	0.15
	4 × 104	0.64	0.62	0.13	0.09
	8 × 104	0.48	0.46	0.03	0.01

. By examining Table 1, we can identify the following observations.

• For $\epsilon$ values of 1, 0.5, or 0.1, the frequencies of the estimators, denoted as $F_{{\theta }_i}^{\epsilon ,M}\left(n\right)$ where $\theta =\alpha$ or $\beta$ and $i=1,2$, approach 0 as n grows toward infinity. This indicates that both the moment estimators and the MLEs are consistent estimators for the hyperparameters. When $\epsilon$ is set to 0.1, the frequencies of these estimators remain relatively high. Nevertheless, we can still observe a decreasing trend toward 0 as n increases to infinity.
• By examining the frequencies associated with $\epsilon =1$, 0.5, and 0.1, it can be seen that as $\epsilon$ decreases, the frequencies become higher. This occurs because the conditions

  $$\left|{\theta }_i^{\left(m\right)}\left(n\right)-\theta \right|\ge \epsilon ,\mathrm{for}\theta =\alpha \mathrm{or}\beta ,i=1,2,$$

  become simpler to satisfy.
• By comparing the moment estimators with the MLEs for the hyperparameters $\alpha$ and $\beta$, it can be observed that the frequencies of the MLEs are lower than those of the moment estimators. This indicates that the MLEs outperform the moment estimators in terms of consistency when estimating the hyperparameters.

3.3. Model Fit Assessment: Kolmogorov–Smirnov Test

In this subsection, we aim to evaluate how well the hierarchical exponential and inverse gamma model (1) fits the simulated data (see [9,20,21]). It is important to note that in this subsection, we only utilize ${\mathit{x}}^{\left(n\right)}=\left(x_1,x_2,\dots ,x_n\right)$. Typically, the chi-square test serves as an indicator of fit quality. Nevertheless, due to its high sensitivity—stemming from challenges in determining the number of groups and identifying appropriate cut-points—we opt for an alternative approach. Instead of relying on the chi-square test, we will employ the Kolmogorov–Smirnov test (KS test) to assess the goodness-of-fit.

The Kolmogorov–Smirnov (KS) statistic is defined as the maximum distance D between the empirical cumulative distribution function (cdf), denoted as $F_n\left(x\right)$, and the theoretical population cdf, $F_0\left(x\right)$. Mathematically, this can be expressed as

$$D=\underset{-\infty <x<\infty }{sup}\left|F_n\left(x\right)-F_0\left(x\right)\right|.$$

(14)

In the R programming environment, the KS test can be performed using the built-in function ‘ks.test()’, as detailed in references such as [22,23,24]. It is important to note that the KS test is primarily applicable for one-dimensional continuous distributions. However, adaptations of the KS test for discrete data have also been explored in recent literature, including works by [25,26,27].

In the context of our study, the null hypothesis assumes that the random variable X follows an exponential-inverse gamma distribution with parameters $\alpha$ and $\beta$, written as $H_0:X\sim Exp-\widetilde{IG}(\alpha ,\beta )$. The $Exp-\widetilde{IG}(\alpha ,\beta )$ represents the marginal distribution derived from the hierarchical exponential and inverse gamma model (1). The marginal density of the $Exp-\widetilde{IG}(\alpha ,\beta )$ distribution, as given in (5), is clearly one-dimensional and continuous. Consequently, the KS test serves as a suitable tool for assessing the goodness-of-fit in this scenario.

The outcomes of the KS test for assessing the goodness-of-fit of the model (1) to the simulated data are presented in

Table 2. The outcomes of the KS test for assessing the model (1) fit to the simulated data.

	Oracle	Moment	MLE
$\overline{D}$	0.0270	0.0230	0.0205
$\overline{p-\mathrm{value}}$	0.5102	0.6683	0.7693
(min D)%	0.16	0.15	0.69
(max p-value)%	0.16	0.15	0.69
(Accept H0)%	0.97	0.99	1.00

. It is important to note that the data were generated based on the hierarchical exponential and inverse gamma model (1) with parameters $\alpha =3$ and $\beta =2$. In this table, the hyperparameters $\alpha$ and $\beta$ are estimated using three distinct approaches. The first approach is the oracle method, where the true values of the hyperparameters $\alpha =3$ and $\beta =2$ are known. The second approach is the moment estimation method, in which $\alpha$ and $\beta$ are estimated using their respective moment estimators (as described in Theorem 2). The third approach is the MLE method, where $\alpha$ and $\beta$ are estimated using their MLEs (as outlined in Theorem 3). In this table, the sample size is set to $n=1000$, and the number of simulations conducted is $M=100$. To evaluate the performance, we employ five metrics proposed in [9]: $\overline{D}$, $\overline{p-\mathrm{value}}$, (min D)%, (max p-value)%, and $\left(\mathrm{Accept}{H}_0\right)\%$. Here, $\overline{D}$ represents the average D values (14) across M simulations, with smaller values indicating better performance. The $\overline{p-\mathrm{value}}$ denotes the average p-values from M simulations, where larger values are preferred. (min D)% refers to the proportion of M simulations in which the minimum D value is achieved among the three methods, with higher percentages being more favorable. The sum of the three (min D)% values should equal 1. (max p-value)% indicates the percentage of M simulations where the maximum p-value is attained among the three methods, with higher percentages also being desirable. Similarly, the sum of the three (max p-value)% values should equal 1. Lastly, $\left(\mathrm{Accept}{H}_0\right)\%$ reflects the proportion of M simulations in which the null hypothesis $H_0$ (defined as p-value $>0.05$) is accepted for each method, with high values being preferable. Each $\left(\mathrm{Accept}{H}_0\right)\%$ value should fall within the range of 0–100%.

By examining Table 2, the following observations can be made.

• The $\overline{D}$ values obtained using the three methods are 0.0270, 0.0230, and 0.0205, respectively. This indicates that the MLE method performs the best, followed by the moment method as the second-best, while the oracle method ranks last. One plausible explanation for this outcome is that in (14), the empirical cdf $F_n\left(x\right)$ relies on data, and the population cdfs $F_0\left(x\right)$ for both the MLE and moment methods also depend on data. In contrast, the population cdf $F_0\left(x\right)$ for the oracle method does not rely on data.
• The $\overline{p-\mathrm{value}}$ corresponding to the three methods are 0.5102, 0.6683, and 0.7693, respectively. These results align with the previous findings, ranking the MLE method first, the moment method second, and the oracle method third. The explanation for this phenomenon has been discussed in the preceding paragraph.
• The (min D)% values for the three methods are 0.16, 0.15, and 0.69, respectively. Notably, the (min D)% value of the MLE method represents more than half of the M simulations. Based on these results, the order of preference among the three methods is the MLE method, the oracle method, and the moment method.
• The (max p-value)% values for the three methods are also 0.16, 0.15, and 0.69, respectively. A smaller D value corresponds to a larger p-value, meaning the smallest D value correlates with the largest p-value. Consequently, the (min D)% and (max p-value)% values for the three methods are identical. Thus, the preferred order remains the MLE method, the oracle method, and the moment method.
• Finally, the $\left(\mathrm{Accept}{H}_0\right)\%$ values for the three methods are 0.97, 0.99, and 1.00, respectively. Once again, the ranking of the methods in terms of preference is the MLE method, the moment method, and the oracle method.

Figure 4 presents the boxplots of D values for the three methods (on the left) and the boxplots of p-values for the three methods (on the right). Based on this figure, we can identify the following observations.

• The D values obtained using the oracle method are considerably higher than those of the other two methods. Given that smaller D values are preferable, the ranking of the three methods in terms of performance is: the MLE method, followed by the moment method, and lastly the oracle method.
• Additionally, the p-values of the oracle method are notably lower than those of the other two methods. Since larger p-values are more desirable, the preference order for the three methods based on p-values remains the same: the MLE method, the moment method, and finally the oracle method.
• It is also worth noting that there is an inverse relationship between D values and p-values; smaller D values correspond to larger p-values, while larger D values result in smaller p-values.
• Overall, the MLE method outperforms the moment method in both D values and p-values.

3.4. Numerical Comparisons Between Bayes Estimators and PESLs of Three Methods

In this subsection, we will conduct a numerical comparison of the Bayes estimators and PESLs for three approaches: the oracle method, the moment method, and the MLE method. It should be highlighted that the theoretical comparisons among these three methods regarding their Bayes estimators and PESLs are presented in Section 2.3.

It is important to highlight that the data have been generated based on the hierarchical exponential and inverse gamma model (1), using $\alpha =3$ and $\beta =2$. Additionally, the oracle method is aware of the hyperparameters $\alpha =3$ and $\beta =2$ during the simulation process.

The comparisons of $\alpha$, $\beta$, ${\delta }_s$, ${\delta }_2$, $PES{L}_s$, and $PES{L}_2$ for the three methods, given a sample size of $n=10,000$ and a number of simulations $M=100$, are presented in Figure 5. By examining the figure, we can identify that the MLE method demonstrates superior performance compared to the method of moments in several aspects. Specifically:

• For the estimators of $\alpha$ and $\beta$, the MLE approach aligns significantly closer with the oracle method than the moment method.
• Regarding the Bayes estimators ${\delta }_{s,i}^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$ and ${\delta }_{2,i}^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$$\left(i=0,1,2\right)$, the MLE method exhibits a slight advantage over the moment method in terms of proximity to the oracle method. The differences among these estimators are minimal, making their respective curves nearly indistinguishable.
• In the context of the PESL functions $PES{L}_{s,i}^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$ and $PES{L}_{2,i}^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$$\left(i=0,1,2\right)$, the MLE method again shows a marked improvement over the moment method in its alignment with the oracle method.
• Overall, the graphical results consistently suggest that the MLE method outperforms the moment method, as demonstrated by the hyperparameter estimators, Bayes estimators, and PESL values obtained through the MLE method being more consistent with those derived from the oracle method than those from the moment method.

Figure 6 presents the boxplots of absolute errors obtained using the oracle method for both the moment method and the MLE method, with a sample size of $n=\mathrm{10,000}$ and $M=100$ simulations. The visualizations consistently demonstrate that the MLE method outperforms the moment method. This is because the absolute errors associated with the oracle method for the hyperparameter estimators, Bayes estimators, and PESLs are significantly smaller when using the MLE method compared to the moment method.

The summary of the averages and proportions of absolute errors for the oracle method, using both the moment method and the MLE method, is presented in

Table 3. The means and ratios of absolute errors derived from the oracle method, using both the moment method and the MLE method, for the estimators of hyperparameters, Bayes estimators, and PESLs.

	Averages		Proportions
	Moment	MLE	Moment	MLE
α	0.2458	0.1099	0.28	0.72
β	0.2354	0.0931	0.21	0.79
${\delta }_s$	0.0199	0.0106	0.22	0.78
${\delta }_2$	0.0210	0.0167	0.37	0.63
$PES{L}_s$	0.0077	0.0037	0.28	0.72
$PES{L}_2$	0.0138	0.0067	0.28	0.72

. Specifically, these averages represent the sample means of the absolute error vectors derived from the oracle method via the moment and MLE methods, where smaller values indicate superior performance. The proportions reflect the sample ratios of absolute errors achieved by each method that are equal to the minimum value between the two methods, with higher proportions being more desirable. From the table, it can be observed that the MLE method yields significantly smaller average absolute errors compared to the moment method. Additionally, the proportions of absolute errors obtained through the MLE method are notably larger than those from the moment method. Overall, the data in the table demonstrates that the MLE method outperforms the moment method in terms of both the averages and proportions of absolute errors as determined by the oracle method.

Let $\theta$ equal $\alpha$ or $\beta$. Suppose ${\widehat{\theta }}_i$ represents an estimator of $\theta$, where $i=1$ corresponds to the method of moments estimator and $i=2$ corresponds to the MLE. The Mean Square Error (MSE) for the estimator ${\widehat{\theta }}_i$ is given by (as referenced in [11]):

$$\mathrm{MSE}\left({\widehat{\theta }}_i\right)=\mathrm{E}{\left({\widehat{\theta }}_i-\theta \right)}^2\approx {\displaystyle \frac{1}{M}}\sum _{m=1}^M{\left({\widehat{\theta }}_i^{\left(m\right)}-\theta \right)}^2.$$

In a similar manner, the Mean Absolute Error (MAE) for the estimator ${\widehat{\theta }}_i$ is defined as (also from [11]):

$$\mathrm{MAE}\left({\widehat{\theta }}_i\right)=\mathrm{E}\left|{\widehat{\theta }}_i-\theta \right|\approx {\displaystyle \frac{1}{M}}\sum _{m=1}^M\left|{\widehat{\theta }}_i^{\left(m\right)}-\theta \right|.$$

It is important to note that both MSE and MAE serve as criteria for comparing two distinct estimators. These metrics function as risk functions (or expected loss functions) associated with the estimators of hyperparameters, providing a measure of the estimators’ performance. Generally, smaller values of MSE and MAE indicate superior performance of the estimator.

The MSE and MAE for the hyperparameter estimators using both the moment method and the MLE method are presented in

Table 4. The MSE and MAE of the hyperparameter estimators obtained using the moment method and the MLE method.

	MSE		MAE
	Moment	MLE	Moment	MLE
$\alpha$	0.09178	0.01877	0.24582	0.10993
$\beta$	0.08204	0.01363	0.23538	0.09313

. As shown in the table, when estimating the hyperparameters $\alpha$ and $\beta$, the MLE method significantly outperforms the moment method, since the MSE and MAE values obtained via the MLE method are considerably lower than those achieved through the moment method.

3.5. Marginal Densities for Different Hyperparameters

In this subsection, we aim to illustrate the marginal densities of the hierarchical exponential and inverse gamma model (1) using different hyperparameters $\alpha$ and $\beta$. It is important to note that the marginal density of X is defined by (5), which depends on the two hyperparameters $\alpha$ and $\beta$. Here, we will investigate the variations in the marginal densities in relation to the density where the hyperparameters are set as $\alpha =3$ and $\beta =2$. Additionally, other specific values for the hyperparameters can also be considered.

Figure 7 illustrates the marginal densities for different values of $\alpha$ ($1,3,5,7$), with $\beta$ remaining constant at 2. As observed in the figure, an increase in $\alpha$ corresponds to a higher peak value of the curve. Additionally, all marginal densities exhibit a decreasing relationship with x and display right-skewed distributions.

Figure 8 illustrates the marginal densities for different values of $\beta$ ($1,2,4,6$), with $\alpha$ kept constant at 3. As observed in the figure, an increase in $\beta$ corresponds to a reduction in the peak value of the curve. Additionally, all marginal densities exhibit a decreasing trend as functions of x and display right-skewed distributions.

4. An Illustrative Data Instance

In this section, we employ the data of Static Fatigue 90% Stress Level to exemplify our methodologies (refer to [28]). Kevlar Epoxy is a material utilized in the National Aeronautics and Space Administration (NASA) space shuttle. Strands of this epoxy were subjected to testing at 90% breaking strength. The provided data represents the time until failure in hours at the 90% stress level for a randomly selected sample of 50 epoxy strands.

The sample is $\left(x_1,x_2,\dots ,x_{n+1}\right)={\mathit{x}}^{\left(n+1\right)}$ (the data for Static Fatigue 90% Stress Level). The sample’s histogram is illustrated in Figure 9, along with its corresponding curve of density estimation. From the figure, it can be observed that the histogram exhibits a gradual decrease, indicating that the exponential-inverse gamma model would be suitable (refer to subsection “Marginal densities for various hyperparameters” in the supplement for more details).

The hyperparameters’ estimators, model’s goodness-of-fit, empirical Bayes estimators of a mean parameter for an exponential-inverse gamma model, PESLs, and mean and variance of Static Fatigue 90% Stress Level data obtained through both moment and MLE methods are reported in

Table 5. The hyperparameters’ estimators, model’s goodness-of-fit, empirical Bayes estimators of a mean parameter for an exponential-inverse gamma model, PESLs, and mean and variance of Static Fatigue 90% Stress Level data obtained through both moment and MLE methods.

		Moment	MLE
Estimators of hyperparameters	$\alpha$	17.562	23.404
	$\beta$	20.750	28.059
Model’s goodness-of-fit	D	0.1240	0.1209
	p-value	0.4389	0.4708
Empirical Bayes estimators and PESLs	${\delta }_s^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$	1.156655	1.179281
	${\delta }_2^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$	1.222516	1.229669
	$PES{L}_s^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$	0.027179	0.020628
	$PES{L}_2^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$	0.028741	0.021516
Mean and variance	$\mathrm{E}X$	1.252857	1.252418
	$\mathrm{Var}\left(X\right)$	1.771380	1.715116

. We see several findings from this table.

• The hyperparameters’ moment estimators and MLEs exhibit significant disparities. However, it does not imply that the exponential-inverse gamma model fails to fit the actual data or that both types of estimators are inconsistent in estimating the hyperparameters. The substantial divergence between these two estimation methods can be attributed to the relatively small sample size of $n=49$. Nevertheless, note that the MLEs for the hyperparameters are more dependable, as demonstrated by the simulations results in the supplement.
• To measure goodness-of-fit, use a KS test. A p-value obtained from the method of moments, $0.4389>0.05$, indicates that the $Exp-\widetilde{IG}(\alpha ,\beta )$ model with hyperparameters calculated by the method of moments well fits the sample $\left(x_1,x_2,\dots ,x_n\right)={\mathit{x}}^{\left(n\right)}$. The same goes for it, a p-value derived from the method of MLE, $0.4708>0.05$, suggests that the $Exp-\widetilde{IG}(\alpha ,\beta )$ model with hyperparameters calculated by MLEs also fits the sample well. Comparing both methods reveals that MLEs’ D values are observed to be smaller, while their corresponding p-values tend to be larger than those of moment estimators; thus indicating that estimating $\alpha$ and $\beta$ using MLEs results in a better fit for the $Exp-\widetilde{IG}(\alpha ,\beta )$ distribution to sample ${\mathit{x}}^{\left(n\right)}$.
• When estimating the hyperparameters using the MLE method, we observe that ${\delta }_s^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$ is 1.179281, which is less than ${\delta }_2^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$ at 1.229669. Additionally, $PES{L}_s^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$ is calculated as 0.020628 and it is lower than $PES{L}_2^{\pi ,{\theta }_{n+1}}\left({\mathit{x}}^{\left(n+1\right)}\right)$, which equals to 0.021516. The same trend can be observed when estimating the hyperparameters using the method of moments for both PESLs and Bayes estimators. Therefore, we have exemplified (6) and (7).
• The $\mathrm{E}X$, which represents the Static Fatigue 90% Stress Level data, is estimated as $\left(i=1,2\right)$:

  $$\mathrm{E}X={\displaystyle \frac{\beta }{\alpha -1}}\approx {\displaystyle \frac{{\beta }_i\left(n\right)}{{\alpha }_i\left(n\right)-1}}.$$
  Meanwhile, $\mathrm{Var}\left(X\right)$ can be evaluated as $\left(i=1,2\right)$:

  $$\mathrm{Var}\left(X\right)={\displaystyle \frac{\alpha {\beta }^2}{{\left(\alpha -1\right)}^2\left(\alpha -2\right)}}\approx {\displaystyle \frac{{\alpha }_i\left(n\right){\beta }_i^2\left(n\right)}{{\left({\alpha }_i\left(n\right)-1\right)}^2\left({\alpha }_i\left(n\right)-2\right)}}.$$
  It is noteworthy that despite different estimators for hyperparameters, both methods yield very similar results by $\mathrm{E}X$ and $\mathrm{Var}\left(X\right)$. Furthermore, it can be observed that

  $$\mathrm{E}X\approx 1.252418\mathrm{and}\mathrm{Var}\left(X\right)\approx 1.715116$$

  using the method of MLE. The method of moments yields similar values for $\mathrm{E}X$ and $\mathrm{Var}\left(X\right)$.

5. Conclusions and Discussions

The posterior and marginal densities for the exponential-inverse gamma model are calculated in Theorem 1. Subsequently, Bayes estimators as well as PESLs of ${\theta }_{n+1}$ are computed. Furthermore, these estimators satisfy two inequalities (6) and (7). The hyperparameters of the exponential-inverse gamma model are estimated using the method of moments as embodied in Theorem 2. Additionally, the method of MLE provides estimators for these hyperparameters, which are embodied in Theorem 3. At last, under Stein’s loss, both methods yield empirical Bayes estimators for a mean parameter of an exponential-inverse gamma model as presented in Theorem 4.

In the simulation studies, we systematically investigate five critical dimensions of the proposed framework. Firstly, we rigorously validate the inequality relationships between Bayes estimators (6) and PESLs (7) under oracle hyperparameter settings. Secondly, we establish the asymptotic consistency of both moment estimators and MLEs for hyperparameter recovery. Thirdly, the model’s fit to simulated data is quantitatively evaluated using Kolmogorov–Smirnov statistics. Fourthly, comprehensive numerical comparisons of PESLs and Bayes estimators are conducted across three approaches—oracle, moment-based, and MLE-driven methods. Finally, we characterize the model’s behavior by visualizing marginal density variations under distinct hyperparameter configurations.

The methods are illustrated using the Static Fatigue 90% Stress Level data. Table 5 summarizes hyperparameters’ estimators, model’s goodness-of-fit, empirical Bayes estimators of mean parameter for exponential-inverse gamma model, PESLs, and mean and variance of Static Fatigue 90% Stress Level data obtained through methods of moment and MLE. The exponential-inverse gamma model, when hyperparameters are calculated by the method of MLE, exhibits a superior goodness-of-fit to real data compared to that calculated by the method of moments. Furthermore, we demonstrate (6) and (7) for the real data.

The numerical results imply that MLEs outperform moment estimators in calculating hyperparameters $\alpha$ and $\beta$. Nevertheless, this improvement has a cost. In comparison to moment estimators, MLEs require more computational resources, are numerically unstable, necessitate $\beta$ greater than 0 throughout the process of iteration, and lack closed-form answers. Hyperparameters’ MLEs are highly ingenious for starting estimations, whereas estimators from the method of moments often serve as reliable initial estimates. Furthermore, if there are cases where MLEs are virtually nonexistent, the presence of a valid reason allows us to utilize estimators from the method of moments.

The inverse gamma prior was chosen for its conjugacy with the exponential likelihood, ensuring analytical tractability in deriving Bayes estimators and PESL—a critical first step in establishing theoretical properties for this underexplored model. While focusing on a single prior limits generality, it provides a foundational framework for future extensions to broader classes (e.g., non-conjugate or robust priors), which we acknowledge as valuable next steps. Our goal here was to rigorously address the uniqueness of Stein’s loss in this specific hierarchical setting, but we agree that exploring prior sensitivity would enrich the analysis and plan to investigate this in subsequent work.

The exponential model was chosen for its conjugacy with the inverse gamma prior, enabling closed-form derivations of Bayes estimators and PESL under Stein’s loss—a key focus of our theoretical analysis. This simplicity allowed us to rigorously isolate the impact of Stein’s loss in restricted parameter spaces, with direct applications to reliability (e.g., fatigue data). While the exponential case serves as a foundational step, extending this framework to other models (e.g., Weibull, gamma) would broaden its utility. Such extensions, though computationally challenging, are a promising direction for future work, and we appreciate the reviewer’s emphasis on this potential.

While our focus was on the conjugate inverse gamma prior for analytical tractability, the methodology can theoretically accommodate non-conjugate or loss-informed priors. However, this would require numerical methods (e.g., Markov Chain Monte Carlo (MCMC)) for posterior computation and alter the closed-form Bayes estimators and PESL expressions.

Direct numerical comparison with MCMC is challenging due to fundamental methodological differences: MCMC requires hyperpriors to ensure posterior integrability (full Bayesian), while our empirical Bayes approach avoids hyperpriors by estimating hyperparameters from data. However, our method leverages conjugacy to achieve closed-form estimators, eliminating MCMC’s sampling/convergence steps. This yields 10–100× faster computation (e.g., seconds vs. minutes) with comparable accuracy in conjugate settings. While full MCMC excels in generality, our framework prioritizes efficiency for exponential-inverse gamma models.

Stein’s loss was chosen due to its asymmetric penalization of extreme errors (e.g., $\theta \to 0$ or ∞), which aligns with our restricted parameter space ($\theta >0$). This ensures robustness in reliability contexts (e.g., underestimating failure rates risks catastrophic outcomes). In the supplement, we have highlighted scenarios where Stein’s loss is preferable (positive restricted parameters) versus inappropriate (unrestricted parameters) than the squared error loss function. In this manuscript, the parameter of interest is $\theta >0$, the mean parameter of the exponential distribution. Therefore, Stein’s loss function is more appropriate than the squared error loss function. Consequently, we consider the PESL as a criterion to compare Bayes estimators.

The empirical Bayes framework fundamentally differs from full hierarchical Bayesian methods by estimating hyperparameters $(\alpha ,\beta )$ via marginal likelihood maximization from observed data (Section 2.2), rather than assigning hyperpriors. This eliminates hyperprior specification, enhancing computational efficiency and reducing subjectivity—key advantages in conjugate settings like our exponential-inverse gamma model. While empirical Bayes sacrifices full probabilistic coherence, it provides tractable, data-driven estimates ideal for applications requiring rapid inference (e.g., reliability analysis). We emphasize the role of empirical Bayes in balancing computational practicality with robust hyperparameter estimation.